Android toolkit API

The Android toolkit API for Conversational AI Engine provides the following classes and methods.

API	Description
chat	Send chat messages to a conversational agent.
addHandler	Registers an event handler to receive agent session events.
removeHandler	Removes a registered event handler.
subscribeMessage	Subscribes to a channel to receive agent conversation events.
unsubscribeMessage	Unsubscribes from the channel and stops receiving events.
interrupt	Interrupts the agent’s speech.
loadAudioSettings	Sets audio parameters to optimize agent conversation performance.
destroy	Destroys the API instance and releases resources.

Send a message to the agent.

Support for sending different types of messages through the ChatMessage sealed class hierarchy:

• ImageMessage: Used to send picture messages.

_1

fun chat(agentUserId: String, message: ChatMessage, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
agentUserId	String	Agent user ID.
message	ChatMessage Message object of type ImageMessage. See ChatMessage.
completion	(error: ConversationalAIAPIError?) -> Unit	Callback function. error is null if the call succeeds, and non-null if it fails.

Registers an event handler to receive agent session events.

_1

fun addHandler(handler: IConversationalAIAPIEventHandler)

Parameter	Type	Description
handler	IConversationalAIAPIEventHandler	Event handler instance. See IConversationalAIAPIEventHandler.

Removes a registered event handler.

_1

fun removeHandler(handler: IConversationalAIAPIEventHandler)

Parameter	Type	Description
handler	IConversationalAIAPIEventHandler	Event handler instance. See IConversationalAIAPIEventHandler.

Subscribes to a channel to receive agent conversation events.

_1

fun subscribeMessage(channelName: String, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
channelName	String	Channel name.
completion	(error: ConversationalAIAPIError?) -> Unit	Callback. error is null on success, or non-null if the call fails. See ConversationalAIAPIError.

Unsubscribes from the channel and stops receiving events.

_1

fun unsubscribeMessage(channelName: String, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
channelName	String	Channel name.
completion	(error: ConversationalAIAPIError?) -> Unit	Callback for unsubscription result. error is null if successful, or non-null if failed. See ConversationalAIAPIError.

Interrupts the agent’s speech.

_1

fun interrupt(agentUserId: String, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
agentUserId	String	Agent user ID.
completion	(error: ConversationalAIAPIError?) -> Unit	Callback. error is null on success or contains error info if the call fails. See ConversationalAIAPIError.

Sets audio parameters to optimize the agent’s conversation performance.

info

You must call this method before each joinChannel call to ensure optimal audio quality.

_1

fun loadAudioSettings(scenario: Int = Constants.AUDIO_SCENARIO_AI_CLIENT)

Parameter	Type	Description
scenario	Int	Audio scene. Default is AUDIO_SCENARIO_AI_CLIENT. Optimized for agent dialogs.

Destroys the API instance and releases resources.

Call this method when the instance is no longer needed. After calling, the instance becomes unusable.

_1

fun destroy()

API	Description
onMessageError	Message processing error callback.
onMessageReceiptUpdated	Message receipt update callback.
onAgentStateChanged	Called when the agent’s status changes.
onAgentInterrupted	Called when an interrupt event occurs.
onAgentMetrics	Called when performance metrics are available.
onAgentError	Called when an agent error occurs.
onTranscriptionUpdated	Called when transcription content is updated.
onDebugLog	Called for internal debug logging.

Message processing error callback.

This callback is triggered when an error occurs during message processing. For example, if a chat message fails to be sent, this callback is triggered and the error message is returned.

_1

fun onMessageError(agentUserId: String, error: MessageError)

Parameter	Type	Description
agentUserId	String	Agent user ID.
error	MessageError	Message error information, including error type and content. See MessageError for details.

Message receipt update callback.

_1

fun onMessageReceiptUpdated(agentUserId: String, receipt: MessageReceipt)

Parameter	Type	Description
agentUserId	String	Agent user ID.
receipt	MessageReceipt	Message receipt information. See MessageReceipt for details.

Triggered when the agent’s state changes.

_1

fun onAgentStateChanged(agentUserId: String, event: StateChangeEvent)

Parameter	Type	Description
agentUserId	String	Agent user ID.
event	StateChangeEvent	State change event. See StateChangeEvent.

Triggered when an interrupt event occurs.

_1

fun onAgentInterrupted(agentUserId: String, event: InterruptEvent)

Parameter	Type	Description
agentUserId	String	Agent user ID.
event	InterruptEvent	Interrupt event. See InterruptEvent.

Triggered when performance metrics are available.

_1

fun onAgentMetrics(agentUserId: String, metric: Metric)

Parameter	Type	Description
agentUserId	String	Agent user ID.
metric	Metric	Performance metric. See Metric.

Triggered when an agent error occurs.

_1

fun onAgentError(agentUserId: String, error: ModuleError)

Parameter	Type	Description
agentUserId	String	Agent user ID.
error	ModuleError	Error details. See ModuleError.

Triggered when the transcription content is updated.

_1

fun onTranscriptionUpdated(agentUserId: String, transcription: Transcription)

info

This callback may trigger frequently. If deduplication is needed, handle it in your business logic.

Parameter	Type	Description
agentUserId	String	Agent user ID.
transcription	Transcription	Transcription data. See Transcription.

Triggered for internal debug logs.

_1

fun onDebugLog(log: String)

Parameter	Type	Description
log	String	Debug log information.

Represents an agent state change event.
Tracks session flow and updates status indications in the UI, including the event timestamp.

_5

data class StateChangeEvent(

_5

val state: AgentState,

_5

val turnId: Long,

_5

val timestamp: Long,

_5

)

Parameter	Type	Description
state	AgentState	Current agent status: silent, listening, thinking, speaking. See AgentState.
turnId	Long	Conversation turn ID.
timestamp	Long	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Indicates an interrupt event.
Triggered when the user or system interrupts the agent. Used for logging and handling.

_4

data class InterruptEvent(

_4

val turnId: Long,

_4

val timestamp: Long

_4

)

Parameter	Type	Description
turnId	Long	ID of the interrupted conversation turn.
timestamp	Long	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Used to define system performance indicator data.

_6

data class Metric(

_6

val type: ModuleType,

_6

val name: String,

_6

val value: Double,

_6

val timestamp: Long

_6

)

Parameter	Type	Description
type	ModuleType	Type of indicator. See ModuleType.
name	String	Descriptive name of the metric.
value	Double	Metric value (e.g., latency in ms).
timestamp	Long	Timestamp when metric was recorded in milliseconds since Unix epoch (January 1, 1970 UTC).

Processes and reports agent-related error information.

_6

data class ModuleError(

_6

val type: ModuleType,

_6

val code: Int,

_6

val message: String,

_6

val timestamp: Long

_6

)

Parameter	Type	Description
type	ModuleType	Error type (e.g., LLM failure, TTS exception). See ModuleType.
code	Int	Specific error code.
message	String	Description of the error.
timestamp	Long	Timestamp of the error in milliseconds since Unix epoch (January 1, 1970 UTC).

Sealed base class for all message types sent to agents.

This sealed class hierarchy provides a type-safe way to handle messages of different content types.

_1

sealed class ChatMessage

Used to send image content to the agent.

Supports specifying image files via HTTP/HTTPS links.

_4

data class ImageMessage(

_4

val uuid: String,

_4

val imageUrl: String,

_4

) : ChatMessage()

Parameter	Type	Description
uuid	String	Unique identifier for the image message.
imageUrl	String?	HTTP/HTTPS image file link, mutually exclusive with imageBase64.

MessageReceipt represents message receipt information and supports processing multiple media types through MediaInfo.

_6

data class MessageReceipt(

_6

val type: ModuleType,

_6

val chatMessageType: ChatMessageType,

_6

val turnId: Long,

_6

val message: String

_6

)

Parameter	Type	Description
type	ModuleType	Module type. See ModuleType, for example: llm, mllm, tts, context.
chatMessageType	ChatMessageType	Message error type. See ChatMessageType.
turnId	Long	Turn ID of the message.
message	String	Message content. Must be parsed according to the type field: Context type: usually a JSON string containing resource info.

Used to distinguish different types of messages in the session system.

_11

enum class ChatMessageType(val value: String) {

_11

Text("text"),

_11

Image("picture"),

_11

UNKNOWN("unknown");

_11

_11

companion object {

_11

fun fromValue(value: String): ChatMessageType {

_11

return ChatMessageType.entries.find { it.value == value } ?: UNKNOWN

_11

}

_11

}

_11

}

Parameter	Type	Description
value	String	The string value to match against.

Used to process and report message error information.

_6

data class MessageError(

_6

val chatMessageType: ChatMessageType,

_6

val code: Int,

_6

val message: String,

_6

val timestamp: Long

_6

)

Parameter	Type	Description
chatMessageType	ChatMessageType	Message error type. See ChatMessageType for details.
code	Int	Error code used to identify specific error scenarios.
message	String	Error description providing detailed explanation, usually a JSON string containing resource information.
timestamp	Long	Timestamp of when the event occurred (milliseconds since January 1, 1970 UTC).

Represents a full transcribed message used to render the UI.

_7

data class Transcription(

_7

val turnId: Long,

_7

val userId: String = "",

_7

val text: String,

_7

var status: TranscriptionStatus,

_7

var type: TranscriptionType

_7

)

Parameter	Type	Description
turnId	Long	Session turn identifier.
userId	String	User ID linked to the transcript.
text	String	Transcribed message content.
status	TranscriptionStatus	Current status of transcription. See TranscriptionStatus.
type	TranscriptionType	Transcript type (e.g., AGENT, USER). See TranscriptionType.

Holds configuration parameters for initializing the Conversational AI API.

_6

data class ConversationalAIAPIConfig(

_6

val rtcEngine: RtcEngine,

_6

val rtmClient: RtmClient,

_6

val renderMode: TranscriptionRenderMode = TranscriptionRenderMode.Word,

_6

val enableLog: Boolean = false

_6

)

Parameter	Type	Description
rtcEngine	RtcEngine	Audio/video engine instance. See RtcEngine.
rtmClient	RtmClient	Real-time messaging client. See RtmClient.
renderMode	TranscriptionRenderMode	Transcription rendering style. Default: word-by-word. See TranscriptionRenderMode.
enableLog	Boolean	Enables logging if true. Default is false.

Represents an error type in the Conversational AI API.

_19

sealed class ConversationalAIAPIError : Exception() {

_19

data class RtmError(val code: Int, val msg: String) : ConversationalAIAPIError()

_19

data class RtcError(val code: Int, val msg: String) : ConversationalAIAPIError()

_19

data class UnknownError(val msg: String) : ConversationalAIAPIError()

_19

_19

val errorCode: Int

_19

get() = when (this) {

_19

is RtmError -> this.code

_19

is RtcError -> this.code

_19

is UnknownError -> -100

_19

}

_19

_19

val errorMessage: String

_19

get() = when (this) {

_19

is RtmError -> this.msg

_19

is RtcError -> this.msg

_19

is UnknownError -> this.msg

_19

}

_19

}

Property	Type	Description
errorCode	Int	RtmError/RtcError: specific code; UnknownError: returns -100.
errorMessage	String	Human-readable description of the error.

Controls the priority with which the agent handles incoming messages during an interaction.

Value	Description
INTERRUPT	High priority: Immediately interrupt the current interaction and process the message. Suitable for urgent or time-sensitive content.
APPEND	Medium priority: The message is queued for processing after the current interaction is completed and is suitable for subsequent questions.
IGNORE	Low priority: This message is only processed when the agent is idle, and will be discarded during ongoing interactions. Suitable for optional content.

Represents the current state of the agent.

Value	Description
SILENT	Agent is silent.
LISTENING	Agent is listening.
THINKING	Agent is processing.
SPEAKING	Agent is speaking.
UNKNOWN	Unknown state.

Performance module type enumeration.

Value	Description
LLM	LLM inference latency measurement.
MLLM	MLLM inference latency measurement.
TTS	Text-to-speech synthesis latency measurement.
UNKNOWN	Unknown type.

Used to distinguish different types of messages in the system.

Value	Description
ASSISTANT	AI assistant message.
USER	User message.
ERROR	Error message.
METRICS	Performance metrics message.
INTERRUPT	Interrupt message.
UNKNOWN	Unknown message type.

Transcript rendering mode.

Value	Description
Word	Word-by-word transcription and rendering.
Text	Full text transcription and rendering.

Transcription source type.

Value	Description
AGENT	Agent transcription.
USER	User transcription.

Indicates the current status of the transcription.

Value	Description
IN_PROGRESS	The transcription is still being generated or the speech is still in progress.
END	The transcription completed normally.
INTERRUPTED	Transcription was interrupted before completion.
UNKNOWN	Unknown status.