Web toolkit API

The Web toolkit API provides the following classes and methods.

API	Description
getInstance	Gets a singleton instance of ConversationalAIAPI.
init	Initializes the ConversationalAIAPI singleton instance.
subscribeMessage	Subscribes to the messaging channel to get real-time updates.
unsubscribe	Unsubscribes from the message channel and cleans up resources.
interrupt	Sends an interrupt message to the specified agent user.
destroy	Destroys the ConversationalAIAPI instance and cleans up resources.

Gets a singleton instance of ConversationalAIAPI.

_1

public static getInstance()

You must call init before calling this method. If not initialized, a NotFoundError exception is thrown.

Return Value: If the method call succeeds, a ConversationalAIAPI instance is returned. If not initialized, a NotFoundError exception is thrown.

Initializes the ConversationalAIAPI singleton instance.

_1

public static init(cfg: IConversationalAIAPIConfig)

This method sets the RTC and RTM engines, rendering mode, and logging options. You must call this method before calling other methods of ConversationalAIAPI.

info

• Only one instance can be initialized at a time.
• If already initialized, an error is thrown.

Parameter	Type	Description
cfg	IConversationalAIAPIConfig	Configuration object used to initialize the API. See IConversationalAIAPIConfig.

Return Value: If the method call succeeds, a ConversationalAIAPI instance is returned.

Subscribes to the messaging channel to get real-time updates.

_1

public subscribeMessage(channel: string)

This method binds the necessary RTC and RTM events and starts CovSubRenderController to process received messages.

info

• You must call init before using this method.
• If not initialized, an error is thrown.

Parameter	Type	Description
channel	string	The channel to subscribe to for messages.

Unsubscribes from the message channel and cleans up resources.

_1

public unsubscribe()

info

This method unbinds RTC and RTM events, clears channels, and cleans up CovSubRenderController. You must call subscribeMessage before calling this method, otherwise an error is thrown.

Sends an interrupt message to the specified agent user.

_1

public async interrupt(agentUserId: string)

info

If not initialized or sending fails, an error is thrown.

Parameter	Type	Description
agentUserId	string	The user ID of the agent user to be interrupted.

Destroys the ConversationalAIAPI instance and cleans up resources.

_1

public destroy()

info

• You must call unsubscribe before calling this method.
• If not initialized, an error is thrown.

Event handler interface for the Conversational AI API module.

_27

export interface IConversationalAIAPIEventHandlers {

_27

[EConversationalAIAPIEvents.AGENT_STATE_CHANGED]: (

_27

agentUserId: string,

_27

event: TStateChangeEvent

_27

) => void

_27

[EConversationalAIAPIEvents.AGENT_INTERRUPTED]: (

_27

agentUserId: string,

_27

event: {

_27

turnID: number

_27

timestamp: number

_27

}

_27

) => void

_27

[EConversationalAIAPIEvents.AGENT_METRICS]: (

_27

agentUserId: string,

_27

metrics: TAgentMetric

_27

) => void

_27

[EConversationalAIAPIEvents.AGENT_ERROR]: (

_27

agentUserId: string,

_27

error: TModuleError

_27

) => void

_27

[EConversationalAIAPIEvents.TRANSCRIPTION_UPDATED]: (

_27

transcription: ISubtitleHelperItem<

_27

Partial<IUserTranscription | IAgentTranscription>

_27

>[]

_27

) => void

_27

[EConversationalAIAPIEvents.DEBUG_LOG]: (message: string) => void

_27

}

A set of event handlers that respond to various events emitted by the conversational AI system, including agent state changes, interruptions, performance metrics, errors, and transcription updates.

Parameter	Type	Description
agentUserId	string	Unique identifier for the AI agent.
event	TStateChangeEvent | { turnID: number; timestamp: number }	Event data. The type depends on the event type. See TStateChangeEvent.
metrics	TAgentMetric	Performance indicator data of the agent. See TAgentMetric.
error	TModuleError	Error message when an error occurs in the agent. See TModuleError.
transcription	ISubtitleHelperItem<Partial<IUserTranscription | IAgentTranscription>>[]	An array of transcripts of the conversation between the user and the agent. See ISubtitleHelperItem.
message	string	Debug log message string.

Parameters used to configure the interface.

_6

export interface IConversationalAIAPIConfig {

_6

rtcEngine: IAgoraRTCClient;

_6

rtmEngine: RTMClient;

_6

renderMode?: ESubtitleHelperMode;

_6

enableLog?: boolean;

_6

}

Parameter	Type	Description
rtcEngine	IAgoraRTCClient	Agora RTC engine instance. See IAgoraRTCClient.
rtmEngine	RTMClient	Agora RTM engine instance. See RTMClient.
renderMode	ESubtitleHelperMode	Rendering mode. See ESubtitleHelperMode.
enableLog	boolean	Whether to enable logging: true enables logging, false disables logging.

Event type used to describe a change in the state of a voice agent.

_6

export type TStateChangeEvent = {

_6

state: EAgentState

_6

turnID: number

_6

timestamp: number

_6

reason: string

_6

}

info

The state change event is triggered when the state of the voice agent changes. The timestamp uses the UNIX timestamp in milliseconds.

Parameter	Type	Description
state	EAgentState	The current state of the voice agent. See EAgentState.
turnID	number	Unique identifier for the current session turn.
timestamp	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).
reason	string	Description of the reason for the status change.

Used to define the statistical data type of agent indicators.

_6

export type TAgentMetric = {

_6

type: EModuleType

_6

name: string

_6

value: number

_6

timestamp: number

_6

}

Parameter	Type	Description
type	EModuleType	Indicator module type. See EModuleType.
name	string	The name of the indicator.
value	number	The indicator value.
timestamp	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Module error type definition.

_6

export type TModuleError = {

_6

type: EModuleType

_6

code: number

_6

message: string

_6

timestamp: number

_6

}

Parameter	Type	Description
type	EModuleType	The module type where the error occurred. See EModuleType.
code	number	Module-specific error code.
message	string	Readable error description that makes it easier to understand and handle.
timestamp	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Subtitle auxiliary item interface.

_9

export interface ISubtitleHelperItem<T> {

_9

uid: string;

_9

stream_id: number;

_9

turn_id: number;

_9

_time: number;

_9

text: string;

_9

status: ETurnStatus;

_9

metadata: T | null;

_9

}

Parameter	Type	Description
uid	string	Unique user identifier.
stream_id	number	Stream identifier.
turn_id	number	The turn identifier within the session.
_time	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).
text	string	Subtitle text content.
status	ETurnStatus	The status of the subtitle item. See ETurnStatus.
metadata	T | null	Additional metadata information.

Agent state enumeration.

_7

export enum EAgentState {

_7

IDLE = "idle",

_7

LISTENING = "listening",

_7

THINKING = "thinking",

_7

SPEAKING = "speaking",

_7

SILENT = "silent"

_7

}

Value	Description
IDLE	("idle"): The agent is idle and ready for new interactions.
LISTENING	("listening"): The agent is receiving user input.
THINKING	("thinking"): The agent is processing the input it receives.
SPEAKING	("speaking"): The agent is outputting a response.
SILENT	("silent"): The agent intentionally does not respond.

Enumeration of module types for AI capabilities.

Value	Description
LLM	("llm"): Large Language Model.
MLLM	("mllm"): Multimodal Large Language Model.
TTS	("tts"): Text-to-speech module.
UNKNOWN	("unknown"): Unknown module type.

The mode type for subtitle processing.

Value	Description
TEXT	Processes subtitles in text mode.
WORD	Processes subtitles in word mode.
UNKNOWN	Unknown subtitle processing mode.

Message type enumeration.

Value	Description
USER_TRANSCRIPTION	("user.transcription"): The user's transcription of the message.
AGENT_TRANSCRIPTION	("assistant.transcription"): The assistant transcribes the message.
MSG_INTERRUPTED	("message.interrupt"): The message was interrupted.
MSG_METRICS	("message.metrics"): Message metrics information.
MSG_ERROR	("message.error"): Message error information.
MSG_STATE	("message.state"): (Deprecated) The message state.

The event type for Conversational AI API.

All events are string literals that can be used with event listeners. Events are case-sensitive.

Value	Description
AGENT_STATE_CHANGED	Agent state change event.
AGENT_INTERRUPTED	The agent was interrupted.
AGENT_METRICS	Agent indicator events.
AGENT_ERROR	Agent error event.
TRANSCRIPTION_UPDATED	Transcription update events.
DEBUG_LOG	Debug log events.

RTM event type enumeration.

Value	Description
MESSAGE	("message"): A message event is received by RTM.
PRESENCE	("presence"): A real-time presence event is received by RTM.
TOPIC	("topic"): Topic type event.
STORAGE	("storage"): Storage events.
LOCK	("lock"): Lock event.
STATUS	("status"): Status event.
LINK_STATE	("linkState"): Connection state event.
TOKEN_PRIVILEGE_WILL_EXPIRE	("tokenPrivilegeWillExpire"): Token privilege is about to expire.

Real-time communication related event types.

Value	Description
NETWORK_QUALITY	Network quality change event.
USER_PUBLISHED	The remote user publishes a media stream event.
USER_UNPUBLISHED	The remote user cancels the media stream publishing event.
STREAM_MESSAGE	The data stream message event sent by the remote end is received.
USER_JOINED	The remote user joins the channel event.
USER_LEFT	The remote user leaves the channel.
CONNECTION_STATE_CHANGE	Connection state change event.
AUDIO_METADATA	An audio metadata event was received.

Represents a custom event type.

Value	Description
MICROPHONE_CHANGED	microphone-changed: The microphone device has changed.
REMOTE_USER_CHANGED	remote-user-changed: The remote user status has changed.
REMOTE_USER_JOINED	remote-user-joined: A remote user has joined the channel.
REMOTE_USER_LEFT	remote-user-left: The remote user leaves the channel.
LOCAL_TRACKS_CHANGED	local-tracks-changed: The local audio and video tracks have changed.

TURN connection status enumeration.

Value	Description
IN_PROGRESS	(0): TURN connection in progress.
END	(1): TURN connection has ended.
INTERRUPTED	(2): TURN connection has been disconnected.