Web toolkit API

The Web toolkit API provides the following classes and methods.

API	Description
chat	Send chat messages to a conversational agent.
getInstance	Gets a singleton instance of ConversationalAIAPI.
init	Initializes the ConversationalAIAPI singleton instance.
subscribeMessage	Subscribes to the messaging channel to get real-time updates.
unsubscribe	Unsubscribes from the message channel and cleans up resources.
interrupt	Sends an interrupt message to the specified agent user.
destroy	Destroys the ConversationalAIAPI instance and cleans up resources.

Class for managing conversational AI engine interactions through Agora's RTC and RTM services.

Send chat messages to a conversational agent.

info

Since v1.7

_1

public async chat(agentUserId: string, message: IChatMessageText | IChatMessageImage)

_6

// Send an image message

_6

const imageMessage: IChatMessageImage = {

_6

messageType: EChatMessageType.IMAGE,

_6

imageData: urlImageData

_6

};

_6

await api.chat("user123", imageMessage);

Parameter	Type	Description
agentUserId	string	Unique identifier of the agent.
message	IChatMessageText | IChatMessageImage	Chat message to be sent. Should be of type IChatMessageImage, see details in IChatMessageImage.

Return Value
If the method call succeeds, a Promise is returned, indicating the result of the message sending.

Gets a singleton instance of ConversationalAIAPI. This method ensures that only one instance exists throughout the entire application lifecycle.

_1

public static getInstance()

info

• You must call init before using this method.
• If not initialized, an error is thrown.

Return Value: If the method call succeeds, a ConversationalAIAPI instance is returned. If not initialized, a NotFoundError exception is thrown.

Initializes the ConversationalAIAPI singleton instance.

_1

public static init(cfg: IConversationalAIAPIConfig)

This method sets the RTC and RTM engines, rendering mode, and logging options. You must call this method before calling other methods of ConversationalAIAPI.

info

• Only one instance can be initialized at a time.
• If already initialized, an error is thrown.

Parameter	Type	Description
cfg	IConversationalAIAPIConfig	Configuration object used to initialize the API. See IConversationalAIAPIConfig.

Return Value: If the method call succeeds, a ConversationalAIAPI instance is returned.

Subscribes to the messaging channel to get real-time updates.

_1

public subscribeMessage(channel: string)

This method binds the necessary RTC and RTM events and starts CovSubRenderController to process received messages.

info

• You must call init before using this method.
• If not initialized, an error is thrown.

Parameter	Type	Description
channel	string	The channel to subscribe to for messages.

Unsubscribes from the message channel and cleans up resources.

_1

public unsubscribe()

info

This method unbinds RTC and RTM events, clears channels, and cleans up CovSubRenderController. You must call subscribeMessage before calling this method, otherwise an error is thrown.

Sends an interrupt message to the specified agent user.

_1

public async interrupt(agentUserId: string)

info

• You must call init before using this method.
• If not initialized or sending fails, an error is thrown.

Parameter	Type	Description
agentUserId	string	The user ID of the agent user to be interrupted.

Destroys the ConversationalAIAPI instance and cleans up resources.

_1

public destroy(): void

info

• You must call unsubscribe before calling this method.
• If not initialized, an error is thrown.

Event handler interface for the Conversational AI API module.

A set of event handlers that respond to various events emitted by the conversational AI system, including agent state changes, interruptions, performance metrics, errors, and transcription updates.

Parameter	Type	Description
agentUserId	string	Unique identifier for the AI agent.
event	TStateChangeEvent | { turnID: number; timestamp: number }	Event data. The type depends on the event type. See TStateChangeEvent.
metrics	TAgentMetric	Performance indicator data of the agent. See TAgentMetric.
error	TModuleError	Error message when an error occurs in the agent. See TModuleError.
transcription	ISubtitleHelperItem<Partial<IUserTranscription | IAgentTranscription>>[]	An array of transcripts of the conversation between the user and the agent. See ISubtitleHelperItem.
message	string	Debug log message string.

Message receipt type definition.

info

Since v1.7

_6

export type TMessageReceipt = {

_6

moduleType: EModuleType

_6

messageType: EChatMessageType

_6

message: string

_6

turnId: number

_6

}

Parameter	Type	Description
moduleType	EModuleType	The type of module sending the message. See EModuleType.
messageType	EChatMessageType	The type of message. See EChatMessageType.
message	string	The content of the message.
turnId	number	Unique identifier for the conversation turn.

The IChatMessageBase interface that contains the properties of the underlying message type.

info

Since v1.7

_3

export interface IChatMessageBase {

_3

messageType: EChatMessageType

_3

}

Parameter	Type	Description
messageType	EChatMessageType	Type of message. See EChatMessageType for details.

Represents an image-based chat message, which can contain URL image data.

info

Since v1.7

_5

export interface IChatMessageImage extends IChatMessageBase {

_5

messageType: EChatMessageType.IMAGE

_5

uuid: string

_5

url?: string

_5

}

Parameter	Type	Description
messageType	EChatMessageType.IMAGE	Message type, must be IMAGE. See EChatMessageType.
uuid	string	Unique identifier for the image message.
url	string	Optional. URL pointing to the image resource.

Parameters used to configure the interface.

_6

export interface IConversationalAIAPIConfig {

_6

rtcEngine: IAgoraRTCClient;

_6

rtmEngine: RTMClient;

_6

renderMode?: ESubtitleHelperMode;

_6

enableLog?: boolean;

_6

}

Parameter	Type	Description
rtcEngine	IAgoraRTCClient	Agora RTC engine instance. See IAgoraRTCClient.
rtmEngine	RTMClient	Agora RTM engine instance. See RTMClient.
renderMode	ESubtitleHelperMode	Rendering mode. See ESubtitleHelperMode.
enableLog	boolean	Whether to enable logging: true enables logging, false disables logging.

Event type used to describe a change in the state of a voice agent.

_6

export type TStateChangeEvent = {

_6

state: EAgentState

_6

turnID: number

_6

timestamp: number

_6

reason: string

_6

}

info

The state change event is triggered when the state of the voice agent changes. The timestamp uses the UNIX timestamp in milliseconds.

Parameter	Type	Description
state	EAgentState	The current state of the voice agent. See EAgentState.
turnID	number	Unique identifier for the current session turn.
timestamp	number	Timestamp in milliseconds when the state change occurred.
reason	string	Description of the reason for the status change.

Used to store metric data during agent runtime.

_6

export type TAgentMetric = {

_6

type: EModuleType

_6

name: string

_6

value: number

_6

timestamp: number

_6

}

Parameter	Type	Description
type	EModuleType	Indicator module type. See EModuleType.
name	string	The name of the indicator.
value	number	The indicator value.
timestamp	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Used to represent error messages from different AI modules, including error type, error code, error message, and timestamp.

_6

export type TModuleError = {

_6

type: EModuleType

_6

code: number

_6

message: string

_6

timestamp: number

_6

}

Parameter	Type	Description
type	EModuleType	The module type where the error occurred. See EModuleType.
code	number	Module-specific error code.
message	string	Readable error description that makes it easier to understand and handle.
timestamp	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Transcript auxiliary item interface.

The data structure for a single transcript item in the transcription system. This includes information such as user ID, stream ID, turn ID, timestamp, text content, status, and metadata.

_9

export interface ISubtitleHelperItem<T> {

_9

uid: string;

_9

stream_id: number;

_9

turn_id: number;

_9

_time: number;

_9

text: string;

_9

status: ETurnStatus;

_9

metadata: T | null;

_9

}

Parameter	Type	Description
uid	string	Unique user identifier.
stream_id	number	Stream identifier.
turn_id	number	The turn identifier within the session.
_time	number	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).
text	string	Transcript text content.
status	ETurnStatus	The status of the transcript item. See ETurnStatus.
metadata	T | null	Additional metadata information.

info

Since v1.7

Used to set the chat message processing priority.

Value	description
INTERRUPTED	('interrupted'): Interrupts current processing and immediately handles this message.
APPEND	('append'): Adds the message to the processing queue to be handled in order.
IGNORE	('ignore'): Discards the message without processing.

Chat message types supported in conversational AI.

info

Since v1.7

Value	Description
TEXT	('text'): Text message.
IMAGE	('image'): Image message.
UNKNOWN	('unknown'): Unknown message type.

Agent state enumeration.

_7

export enum EAgentState {

_7

IDLE = "idle",

_7

LISTENING = "listening",

_7

THINKING = "thinking",

_7

SPEAKING = "speaking",

_7

SILENT = "silent"

_7

}

Value	Description
IDLE	("idle"): The agent is idle and ready for new interactions.
LISTENING	("listening"): The agent is receiving user input.
THINKING	("thinking"): The agent is processing the input it receives.
SPEAKING	("speaking"): The agent is outputting a response.
SILENT	("silent"): The agent intentionally does not respond.

Enumeration of module types for AI capabilities.

Value	Description
LLM	("llm"): Large Language Model.
MLLM	("mllm"): Multimodal Large Language Model.
TTS	("tts"): Text-to-speech module.
CONTEXT	("context"): Context management module.
UNKNOWN	("unknown"): Unknown module type.

The mode type for transcript processing.

Value	Description
TEXT	Processes transcript in text mode.
WORD	Processes transcript in word mode.
UNKNOWN	Unknown transcript processing mode.

TURN connection status enumeration.

Value	Description
IN_PROGRESS	(0): TURN connection in progress.
END	(1): TURN connection has ended.
INTERRUPTED	(2): TURN connection has been disconnected.