Android toolkit API | Agora Docs

The Android toolkit API for Conversational AI Engine provides the following classes and methods.

IConversationalAIAPI class

API	Description
`chat`	Send chat messages to a conversational agent.
`addHandler`	Registers an event handler to receive agent session events.
`removeHandler`	Removes a registered event handler.
`subscribeMessage`	Subscribes to a channel to receive agent conversation events.
`unsubscribeMessage`	Unsubscribes from the channel and stops receiving events.
`interrupt`	Interrupts the agent’s speech.
`loadAudioSettings`	Sets audio parameters to optimize agent conversation performance.
`destroy`	Destroys the API instance and releases resources.

`chat`

Send a message to the agent.

Support for sending different types of messages through the ChatMessage sealed class hierarchy:

ImageMessage: Used to send picture messages.

fun chat(agentUserId: String, message: ChatMessage, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`message`	`ChatMessage`	Message object of type ImageMessage. See ChatMessage for details.
`completion`	`(error: ConversationalAIAPIError?) -> Unit`	Callback function. `error` is null if the call succeeds, and non-null if it fails.

`addHandler`

Registers an event handler to receive agent session events.

fun addHandler(handler: IConversationalAIAPIEventHandler)

Parameter	Type	Description
`handler`	`IConversationalAIAPIEventHandler`	Event handler instance. See `IConversationalAIAPIEventHandler`.

`removeHandler`

Removes a registered event handler.

fun removeHandler(handler: IConversationalAIAPIEventHandler)

Parameter	Type	Description
`handler`	`IConversationalAIAPIEventHandler`	Event handler instance. See `IConversationalAIAPIEventHandler`.

`subscribeMessage`

Subscribes to a channel to receive agent conversation events.

fun subscribeMessage(channelName: String, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
`channelName`	`String`	Channel name.
`completion`	`(error: ConversationalAIAPIError?) -> Unit`	Callback. `error` is `null` on success, or non-null if the call fails. See `ConversationalAIAPIError`.

`unsubscribeMessage`

Unsubscribes from the channel and stops receiving events.

fun unsubscribeMessage(channelName: String, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
`channelName`	`String`	Channel name.
`completion`	`(error: ConversationalAIAPIError?) -> Unit`	Callback for unsubscription result. `error` is `null` if successful, or non-null if failed. See `ConversationalAIAPIError`.

`interrupt`

Interrupts the agent’s speech.

fun interrupt(agentUserId: String, completion: (error: ConversationalAIAPIError?) -> Unit)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`completion`	`(error: ConversationalAIAPIError?) -> Unit`	Callback. `error` is `null` on success or contains error info if the call fails. See `ConversationalAIAPIError`.

`loadAudioSettings`

Sets audio parameters to optimize the agent’s conversation performance.

You must call this method before each joinChannel call to ensure optimal audio quality.

fun loadAudioSettings(scenario: Int = Constants.AUDIO_SCENARIO_AI_CLIENT)

Parameter	Type	Description
`scenario`	`Int`	Audio scene. `AUDIO_SCENARIO_AI_CLIENT` is the default. Use it for Conversational AI agent dialogue scenarios to optimize audio quality for speech recognition and semantic understanding. If you enable the AI Avatar feature, set the scenario to `AUDIO_SCENARIO_DEFAULT` for better mixing results.

`destroy`

Destroys the API instance and releases resources.

Call this method when the instance is no longer needed. After calling, the instance becomes unusable.

fun destroy()

IConversationalAIAPIEventHandler class

API	Description
`onMessageError`	Message processing error callback.
`onMessageReceiptUpdated`	Message receipt update callback.
`onAgentStateChanged`	Called when the agent’s status changes.
`onAgentListeningChanged`	Called when the agent's listening state changes.
`onAgentThinkingChanged`	Called when the agent's thinking state changes.
`onAgentSpeakingChanged`	Called when the agent's speaking state changes.
`onAgentInterrupted`	Called when an interrupt event occurs.
`onAgentMetrics`	Called when performance metrics are available.
`onAgentError`	Called when an agent error occurs.
`onTranscriptUpdated`	Called when transcription content is updated.
`onDebugLog`	Called for internal debug logging.

`onMessageError`

Message processing error callback.

This callback is triggered when an error occurs during message processing. For example, if a chat message fails to be sent, this callback is triggered and the error message is returned.

fun onMessageError(agentUserId: String, error: MessageError)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`error`	`MessageError`	Message error information, including error type and content. See `MessageError` for details.

`onMessageReceiptUpdated`

Message receipt update callback.

fun onMessageReceiptUpdated(agentUserId: String, receipt: MessageReceipt)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`receipt`	`MessageReceipt`	Message receipt information. See `MessageReceipt` for details.

`onAgentStateChanged`

Triggered when the agent’s state changes.

fun onAgentStateChanged(agentUserId: String, event: StateChangeEvent)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`event`	`StateChangeEvent`	State change event. See `StateChangeEvent`.

`onAgentListeningChanged`

Triggered when the agent's listening state changes.

This callback is a fine-grained complement to onAgentStateChanged, used to monitor whether the agent is actively listening to user input.

fun onAgentListeningChanged(agentUserId: String, isListening: Boolean)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`isListening`	`Boolean`	Whether the agent is currently in a listening state.

`onAgentThinkingChanged`

Triggered when the agent's thinking state changes.

This callback is a fine-grained complement to onAgentStateChanged, used to monitor whether the agent is actively processing a request.

fun onAgentThinkingChanged(agentUserId: String, isThinking: Boolean)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`isThinking`	`Boolean`	Whether the agent is currently in a thinking state.

`onAgentSpeakingChanged`

Triggered when the agent's speaking state changes.

This callback is a fine-grained complement to onAgentStateChanged, used to monitor whether the agent is actively playing back speech.

fun onAgentSpeakingChanged(agentUserId: String, isSpeaking: Boolean)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`isSpeaking`	`Boolean`	Whether the agent is currently in a speaking state.

`onAgentInterrupted`

Triggered when an interrupt event occurs.

fun onAgentInterrupted(agentUserId: String, event: InterruptEvent)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`event`	`InterruptEvent`	Interrupt event. See `InterruptEvent`.

`onAgentMetrics`

Triggered when performance metrics become available.

fun onAgentMetrics(agentUserId: String, metric: Metric)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`metric`	`Metric`	Performance metric. See `Metric`.

`onAgentError`

Triggered when an agent error occurs.

fun onAgentError(agentUserId: String, error: ModuleError)

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`error`	`ModuleError`	Error details. See `ModuleError`.

`onTranscriptUpdated`

Triggered when the transcription content is updated.

fun onTranscriptUpdated(agentUserId: String, transcript: Transcript)

This callback may trigger frequently. If deduplication is needed, handle it in your business logic.

Parameter	Type	Description
`agentUserId`	`String`	Agent user ID.
`transcript`	`Transcript`	Transcript data. See `Transcript`.

`onDebugLog`

Triggered for internal debug logs.

fun onDebugLog(log: String)

Parameter	Type	Description
`log`	`String`	Debug log information.

Structures

`StateChangeEvent`

Represents an agent state change event.
Tracks session flow and updates status indications in the UI, including the event timestamp.

data class StateChangeEvent(
    val state: AgentState,
    val turnId: Long,
    val timestamp: Long,
)

Parameter	Type	Description
`state`	`AgentState`	Current agent status: `silent`, `listening`, `thinking`, `speaking`. See `AgentState`.
`turnId`	`Long`	Conversation turn ID.
`timestamp`	`Long`	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

`InterruptEvent`

Indicates an interrupt event.
Triggered when the user or system interrupts the agent or when the system detects a high-priority message. Used for logging and handling.

data class InterruptEvent(
    val turnId: Long,
    val timestamp: Long
)

Parameter	Type	Description
`turnId`	`Long`	ID of the interrupted conversation turn.
`timestamp`	`Long`	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

`Metric`

Used to record and transmit system performance data. This data can be used for performance monitoring, system optimization, and user experience improvement.

data class Metric(
    val type: ModuleType,
    val name: String,
    val value: Double,
    val timestamp: Long
)

Parameter	Type	Description
`type`	`ModuleType`	Type of indicator. See `ModuleType`.
`name`	`String`	Descriptive name of the metric.
`value`	`Double`	Metric value (e.g., latency in ms).
`timestamp`	`Long`	Timestamp when metric was recorded in milliseconds since Unix epoch (January 1, 1970 UTC).

`ModuleError`

Processes and reports agent-related error information.

data class ModuleError(
    val type: ModuleType,
    val code: Int,
    val message: String,
    val timestamp: Long,
    val turnId: Long? = null
)

Parameter	Type	Description
`type`	`ModuleType`	Error type (e.g., LLM failure, TTS exception). See `ModuleType`.
`code`	`Int`	Specific error code.
`message`	`String`	Description of the error.
`timestamp`	`Long`	Timestamp of the error in milliseconds since Unix epoch (January 1, 1970 UTC).
`turnId`	`Long?`	(Optional) `turnId` corresponding to the image upload error.

`ChatMessage`

Sealed base class for all message types sent to agents.

This sealed class hierarchy provides a type-safe way to handle messages of different content types.

sealed class ChatMessage

`ImageMessage`

Used to send image content to the agent.

Supports specifying image files via HTTP/HTTPS links.

data class ImageMessage(
    val uuid: String,
    val imageUrl: String,
) : ChatMessage()

Parameter	Type	Description
`uuid`	String	Unique identifier for the image message.
`imageUrl`	String?	HTTP/HTTPS image file link.

MessageReceipt

MessageReceipt represents message receipt information and supports processing multiple media types through MediaInfo.

data class MessageReceipt(
    val type: ModuleType,
    val chatMessageType: ChatMessageType,
    val turnId: Long,
    val message: String
)

Parameter	Type	Description
`type`	ModuleType	Module type. See ModuleType, for example: llm, mllm, tts, context.
`chatMessageType`	ChatMessageType	Message error type. See ChatMessageType.
`turnId`	Long	Turn ID of the message.
`message`	String	Message content. Must be parsed according to the `type` field. For `context`, this is usually a JSON string containing resource information.

ChatMessageType

Used to distinguish different types of messages in the session system.

enum class ChatMessageType(val value: String) {
    Text("text"),
    Image("picture"),
    UNKNOWN("unknown");

    companion object {
        fun fromValue(value: String): ChatMessageType {
            return ChatMessageType.entries.find { it.value == value } ?: UNKNOWN
        }
    }
}

Parameter	Type	Description
`value`	`String`	The string value to match against.

MessageError

Used to process and report message error information.

data class MessageError(
    val chatMessageType: ChatMessageType,
    val code: Int,
    val message: String,
    val timestamp: Long
)

Parameter	Type	Description
`chatMessageType`	`ChatMessageType`	Message error type. See `ChatMessageType` for details.
`code`	`Int`	Error code used to identify specific error scenarios.
`message`	`String`	Error description providing detailed explanation, usually a JSON string containing resource information.
`timestamp`	`Long`	Timestamp of when the event occurred (milliseconds since January 1, 1970 UTC).

`Transcript`

Represents a full transcribed message used to render the UI.

data class Transcript(
    val turnId: Long,
    val userId: String = "",
    val text: String,
    var status: TranscriptStatus,
    var type: TranscriptType
)

Parameter	Type	Description
`turnId`	`Long`	Session turn identifier.
`userId`	`String`	User ID linked to the transcript.
`text`	`String`	Transcribed message content.
`status`	`TranscriptStatus`	Current status of transcription. See `TranscriptStatus`.
`type`	`TranscriptType`	Transcript type (e.g., `AGENT`, `USER`). See `TranscriptType`.

`ConversationalAIAPIConfig`

Holds configuration parameters for initializing the Conversational AI API.

data class ConversationalAIAPIConfig(
    val rtcEngine: RtcEngine,
    val rtmClient: RtmClient,
    val renderMode: TranscriptRenderMode = TranscriptRenderMode.Word,
    val enableLog: Boolean = false
)

Parameter	Type	Description
`rtcEngine`	`RtcEngine`	Audio/video engine instance. See `RtcEngine`.
`rtmClient`	`RtmClient`	Real-time messaging client. See the Signaling Android API reference.
`renderMode`	`TranscriptRenderMode`	Transcript rendering style. Default: word-by-word. See `TranscriptRenderMode`.
`enableLog`	`Boolean`	Enables logging if `true`. Default is `false`.

`ConversationalAIAPIError`

Represents an error type in the Conversational AI API.

sealed class ConversationalAIAPIError : Exception() {
    data class RtmError(val code: Int, val msg: String) : ConversationalAIAPIError()
    data class RtcError(val code: Int, val msg: String) : ConversationalAIAPIError()
    data class UnknownError(val msg: String) : ConversationalAIAPIError()

    val errorCode: Int
        get() = when (this) {
            is RtmError -> this.code
            is RtcError -> this.code
            is UnknownError -> -100
        }

    val errorMessage: String
        get() = when (this) {
            is RtmError -> this.msg
            is RtcError -> this.msg
            is UnknownError -> this.msg
        }
}

Property	Type	Description
`errorCode`	`Int`	`RtmError`/`RtcError`: specific code; `UnknownError`: returns `-100`.
`errorMessage`	`String`	Human-readable description of the error.

Enum classes

Priority

Controls the priority with which the agent handles incoming messages during an interaction.

Value	Description
`INTERRUPT`	High priority: Immediately interrupt the current interaction and process the message. Suitable for urgent or time-sensitive content.
`APPEND`	Medium priority: The message is queued for processing after the current interaction is completed and is suitable for subsequent questions.
`IGNORE`	Low priority: This message is only processed when the agent is idle, and will be discarded during ongoing interactions. Suitable for optional content.

AgentState

Represents the current state of the agent.

Value	Description
`SILENT`	Agent is silent.
`LISTENING`	Agent is listening.
`THINKING`	Agent is processing or thinking.
`SPEAKING`	Agent is speaking.
`UNKNOWN`	Unknown state.

ModuleType

Performance module type enumeration.

Value	Description
`LLM`	LLM inference latency measurement.
`MLLM`	MLLM inference latency measurement.
`TTS`	Text-to-speech synthesis latency measurement.
`UNKNOWN`	Unknown type.

MessageType

Used to distinguish different types of messages in the system.

Value	Description
`ASSISTANT`	AI assistant transcript message.
`USER`	User transcript message.
`ERROR`	Error message.
`METRICS`	Performance metrics message.
`INTERRUPT`	Interrupt message.
`UNKNOWN`	Unknown message type.
`MESSAGE_RECEIPT`	Message receipt.

TranscriptRenderMode

Transcript rendering mode.

Value	Description
`Word`	Word-by-word transcription and rendering.
`Text`	Full text transcription and rendering.

TranscriptType

Transcript source type.

Value	Description
`AGENT`	Agent transcript.
`USER`	User transcript.

TranscriptStatus

Indicates the current status of the transcription.

Value	Description
`IN_PROGRESS`	The transcript is still being generated or the speech is still in progress.
`END`	The transcription completed normally.
`INTERRUPTED`	Transcription was interrupted before completion.
`UNKNOWN`	Unknown status.

Contact Us

Contact Us