# Android toolkit API (/en/api-reference/api-ref/conversational-ai/client-toolkit/android)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

The [Android toolkit API](https://github.com/AgoraIO-Community/Conversational-AI-Demo/tree/main/Android/scenes/convoai/src/main/java/io/agora/scene/convoai/convoaiApi) for Conversational AI Engine provides the following classes and methods.

## IConversationalAIAPI class [#iconversationalaiapi-class]

| API                                         | Description                                                       |
| ------------------------------------------- | ----------------------------------------------------------------- |
| [`chat`](#chat)                             | Send chat messages to a conversational agent.                     |
| [`addHandler`](#addhandler)                 | Registers an event handler to receive agent session events.       |
| [`removeHandler`](#removehandler)           | Removes a registered event handler.                               |
| [`subscribeMessage`](#subscribemessage)     | Subscribes to a channel to receive agent conversation events.     |
| [`unsubscribeMessage`](#unsubscribemessage) | Unsubscribes from the channel and stops receiving events.         |
| [`interrupt`](#interrupt)                   | Interrupts the agent’s speech.                                    |
| [`loadAudioSettings`](#loadaudiosettings)   | Sets audio parameters to optimize agent conversation performance. |
| [`destroy`](#destroy)                       | Destroys the API instance and releases resources.                 |

### `chat` [#chat]

Send a message to the agent.

Support for sending different types of messages through the `ChatMessage` sealed class hierarchy:

* `ImageMessage`: Used to send picture messages.

```kotlin
fun chat(agentUserId: String, message: ChatMessage, completion: (error: ConversationalAIAPIError?) -> Unit)
```

| Parameter     | Type                                         | Description                                                                                        |
| ------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `agentUserId` | `String`                                     | Agent user ID.                                                                                     |
| `message`     | `ChatMessage`                                | Message object of type [ImageMessage](#imagemessage). See [ChatMessage](#chatmessage) for details. |
| `completion`  | `(error: ConversationalAIAPIError?) -> Unit` | Callback function. `error` is null if the call succeeds, and non-null if it fails.                 |

### `addHandler` [#addhandler]

Registers an event handler to receive agent session events.

```kotlin
fun addHandler(handler: IConversationalAIAPIEventHandler)
```

| Parameter | Type                               | Description                                                                                                |
| --------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `handler` | `IConversationalAIAPIEventHandler` | Event handler instance. See [`IConversationalAIAPIEventHandler`](#iconversationalaiapieventhandler-class). |

### `removeHandler` [#removehandler]

Removes a registered event handler.

```kotlin
fun removeHandler(handler: IConversationalAIAPIEventHandler)
```

| Parameter | Type                               | Description                                                                                                |
| --------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `handler` | `IConversationalAIAPIEventHandler` | Event handler instance. See [`IConversationalAIAPIEventHandler`](#iconversationalaiapieventhandler-class). |

### `subscribeMessage` [#subscribemessage]

Subscribes to a channel to receive agent conversation events.

```kotlin
fun subscribeMessage(channelName: String, completion: (error: ConversationalAIAPIError?) -> Unit)
```

| Parameter     | Type                                         | Description                                                                                                                         |
| ------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `channelName` | `String`                                     | Channel name.                                                                                                                       |
| `completion`  | `(error: ConversationalAIAPIError?) -> Unit` | Callback. `error` is `null` on success, or non-null if the call fails. See [`ConversationalAIAPIError`](#conversationalaiapierror). |

### `unsubscribeMessage` [#unsubscribemessage]

Unsubscribes from the channel and stops receiving events.

```kotlin
fun unsubscribeMessage(channelName: String, completion: (error: ConversationalAIAPIError?) -> Unit)
```

| Parameter     | Type                                         | Description                                                                                                                                              |
| ------------- | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `channelName` | `String`                                     | Channel name.                                                                                                                                            |
| `completion`  | `(error: ConversationalAIAPIError?) -> Unit` | Callback for unsubscription result. `error` is `null` if successful, or non-null if failed. See [`ConversationalAIAPIError`](#conversationalaiapierror). |

### `interrupt` [#interrupt]

Interrupts the agent’s speech.

```kotlin
fun interrupt(agentUserId: String, completion: (error: ConversationalAIAPIError?) -> Unit)
```

| Parameter     | Type                                         | Description                                                                                                                                   |
| ------------- | -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `agentUserId` | `String`                                     | Agent user ID.                                                                                                                                |
| `completion`  | `(error: ConversationalAIAPIError?) -> Unit` | Callback. `error` is `null` on success or contains error info if the call fails. See [`ConversationalAIAPIError`](#conversationalaiapierror). |

### `loadAudioSettings` [#loadaudiosettings]

Sets audio parameters to optimize the agent’s conversation performance.

<CalloutContainer type="info">
  <CalloutDescription>
    You must call this method **before each** `joinChannel` call to ensure optimal audio quality.
  </CalloutDescription>
</CalloutContainer>

```kotlin
fun loadAudioSettings(scenario: Int = Constants.AUDIO_SCENARIO_AI_CLIENT)
```

| Parameter  | Type  | Description                                                                                                                                                                                                                                                                                             |
| ---------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `scenario` | `Int` | Audio scene. `AUDIO_SCENARIO_AI_CLIENT` is the default. Use it for Conversational AI agent dialogue scenarios to optimize audio quality for speech recognition and semantic understanding. If you enable the AI Avatar feature, set the scenario to `AUDIO_SCENARIO_DEFAULT` for better mixing results. |

### `destroy` [#destroy]

Destroys the API instance and releases resources.

Call this method when the instance is no longer needed. After calling, the instance becomes unusable.

```kotlin
fun destroy()
```

## IConversationalAIAPIEventHandler class [#iconversationalaiapieventhandler-class]

| API                                                   | Description                                      |
| ----------------------------------------------------- | ------------------------------------------------ |
| [`onMessageError`](#onmessageerror)                   | Message processing error callback.               |
| [`onMessageReceiptUpdated`](#onmessagereceiptupdated) | Message receipt update callback.                 |
| [`onAgentStateChanged`](#onagentstatechanged)         | Called when the agent’s status changes.          |
| [`onAgentListeningChanged`](#onagentlisteningchanged) | Called when the agent's listening state changes. |
| [`onAgentThinkingChanged`](#onagentthinkingchanged)   | Called when the agent's thinking state changes.  |
| [`onAgentSpeakingChanged`](#onagentspeakingchanged)   | Called when the agent's speaking state changes.  |
| [`onAgentInterrupted`](#onagentinterrupted)           | Called when an interrupt event occurs.           |
| [`onAgentMetrics`](#onagentmetrics)                   | Called when performance metrics are available.   |
| [`onAgentError`](#onagenterror)                       | Called when an agent error occurs.               |
| [`onTranscriptUpdated`](#ontranscriptupdated)         | Called when transcription content is updated.    |
| [`onDebugLog`](#ondebuglog)                           | Called for internal debug logging.               |

### `onMessageError` [#onmessageerror]

Message processing error callback.

This callback is triggered when an error occurs during message processing. For example, if a chat message fails to be sent, this callback is triggered and the error message is returned.

```kotlin
fun onMessageError(agentUserId: String, error: MessageError)
```

| Parameter     | Type           | Description                                                                                  |
| ------------- | -------------- | -------------------------------------------------------------------------------------------- |
| `agentUserId` | `String`       | Agent user ID.                                                                               |
| `error`       | `MessageError` | Message error information, including error type and content. See `MessageError` for details. |

### `onMessageReceiptUpdated` [#onmessagereceiptupdated]

Message receipt update callback.

```kotlin
fun onMessageReceiptUpdated(agentUserId: String, receipt: MessageReceipt)
```

| Parameter     | Type             | Description                                                    |
| ------------- | ---------------- | -------------------------------------------------------------- |
| `agentUserId` | `String`         | Agent user ID.                                                 |
| `receipt`     | `MessageReceipt` | Message receipt information. See `MessageReceipt` for details. |

### `onAgentStateChanged` [#onagentstatechanged]

Triggered when the agent’s state changes.

```kotlin
fun onAgentStateChanged(agentUserId: String, event: StateChangeEvent)
```

| Parameter     | Type               | Description                                                      |
| ------------- | ------------------ | ---------------------------------------------------------------- |
| `agentUserId` | `String`           | Agent user ID.                                                   |
| `event`       | `StateChangeEvent` | State change event. See [`StateChangeEvent`](#statechangeevent). |

### `onAgentListeningChanged` [#onagentlisteningchanged]

Triggered when the agent's listening state changes.

This callback is a fine-grained complement to `onAgentStateChanged`, used to monitor whether the agent is actively listening to user input.

```kotlin
fun onAgentListeningChanged(agentUserId: String, isListening: Boolean)
```

| Parameter     | Type      | Description                                          |
| ------------- | --------- | ---------------------------------------------------- |
| `agentUserId` | `String`  | Agent user ID.                                       |
| `isListening` | `Boolean` | Whether the agent is currently in a listening state. |

### `onAgentThinkingChanged` [#onagentthinkingchanged]

Triggered when the agent's thinking state changes.

This callback is a fine-grained complement to `onAgentStateChanged`, used to monitor whether the agent is actively processing a request.

```kotlin
fun onAgentThinkingChanged(agentUserId: String, isThinking: Boolean)
```

| Parameter     | Type      | Description                                         |
| ------------- | --------- | --------------------------------------------------- |
| `agentUserId` | `String`  | Agent user ID.                                      |
| `isThinking`  | `Boolean` | Whether the agent is currently in a thinking state. |

### `onAgentSpeakingChanged` [#onagentspeakingchanged]

Triggered when the agent's speaking state changes.

This callback is a fine-grained complement to `onAgentStateChanged`, used to monitor whether the agent is actively playing back speech.

```kotlin
fun onAgentSpeakingChanged(agentUserId: String, isSpeaking: Boolean)
```

| Parameter     | Type      | Description                                         |
| ------------- | --------- | --------------------------------------------------- |
| `agentUserId` | `String`  | Agent user ID.                                      |
| `isSpeaking`  | `Boolean` | Whether the agent is currently in a speaking state. |

### `onAgentInterrupted` [#onagentinterrupted]

Triggered when an interrupt event occurs.

```kotlin
fun onAgentInterrupted(agentUserId: String, event: InterruptEvent)
```

| Parameter     | Type             | Description                                               |
| ------------- | ---------------- | --------------------------------------------------------- |
| `agentUserId` | `String`         | Agent user ID.                                            |
| `event`       | `InterruptEvent` | Interrupt event. See [`InterruptEvent`](#interruptevent). |

### `onAgentMetrics` [#onagentmetrics]

Triggered when performance metrics become available.

```kotlin
fun onAgentMetrics(agentUserId: String, metric: Metric)
```

| Parameter     | Type     | Description                                  |
| ------------- | -------- | -------------------------------------------- |
| `agentUserId` | `String` | Agent user ID.                               |
| `metric`      | `Metric` | Performance metric. See [`Metric`](#metric). |

### `onAgentError` [#onagenterror]

Triggered when an agent error occurs.

```kotlin
fun onAgentError(agentUserId: String, error: ModuleError)
```

| Parameter     | Type          | Description                                       |
| ------------- | ------------- | ------------------------------------------------- |
| `agentUserId` | `String`      | Agent user ID.                                    |
| `error`       | `ModuleError` | Error details. See [`ModuleError`](#moduleerror). |

### `onTranscriptUpdated` [#ontranscriptupdated]

Triggered when the transcription content is updated.

```kotlin
fun onTranscriptUpdated(agentUserId: String, transcript: Transcript)
```

<CalloutContainer type="info">
  <CalloutDescription>
    This callback may trigger frequently. If deduplication is needed, handle it in your business logic.
  </CalloutDescription>
</CalloutContainer>

| Parameter     | Type         | Description                                       |
| ------------- | ------------ | ------------------------------------------------- |
| `agentUserId` | `String`     | Agent user ID.                                    |
| `transcript`  | `Transcript` | Transcript data. See [`Transcript`](#transcript). |

### `onDebugLog` [#ondebuglog]

Triggered for internal debug logs.

```kotlin
fun onDebugLog(log: String)
```

| Parameter | Type     | Description            |
| --------- | -------- | ---------------------- |
| `log`     | `String` | Debug log information. |

## Structures [#structures]

### `StateChangeEvent` [#statechangeevent]

Represents an agent state change event.\
Tracks session flow and updates status indications in the UI, including the event timestamp.

```kotlin
data class StateChangeEvent(
    val state: AgentState,
    val turnId: Long,
    val timestamp: Long,
)
```

| Parameter   | Type                        | Description                                                                                           |
| ----------- | --------------------------- | ----------------------------------------------------------------------------------------------------- |
| `state`     | [`AgentState`](#agentstate) | Current agent status: `silent`, `listening`, `thinking`, `speaking`. See [`AgentState`](#agentstate). |
| `turnId`    | `Long`                      | Conversation turn ID.                                                                                 |
| `timestamp` | `Long`                      | Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).                                     |

### `InterruptEvent` [#interruptevent]

Indicates an interrupt event.\
Triggered when the user or system interrupts the agent or when the system detects a high-priority message. Used for logging and handling.

```kotlin
data class InterruptEvent(
    val turnId: Long,
    val timestamp: Long
)
```

| Parameter   | Type   | Description                                                       |
| ----------- | ------ | ----------------------------------------------------------------- |
| `turnId`    | `Long` | ID of the interrupted conversation turn.                          |
| `timestamp` | `Long` | Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC). |

### `Metric` [#metric]

Used to record and transmit system performance data. This data can be used for performance monitoring, system optimization, and user experience improvement.

```kotlin
data class Metric(
    val type: ModuleType,
    val name: String,
    val value: Double,
    val timestamp: Long
)
```

| Parameter   | Type                        | Description                                                                                |
| ----------- | --------------------------- | ------------------------------------------------------------------------------------------ |
| `type`      | [`ModuleType`](#moduletype) | Type of indicator. See [`ModuleType`](#moduletype).                                        |
| `name`      | `String`                    | Descriptive name of the metric.                                                            |
| `value`     | `Double`                    | Metric value (e.g., latency in ms).                                                        |
| `timestamp` | `Long`                      | Timestamp when metric was recorded in milliseconds since Unix epoch (January 1, 1970 UTC). |

### `ModuleError` [#moduleerror]

Processes and reports agent-related error information.

```kotlin
data class ModuleError(
    val type: ModuleType,
    val code: Int,
    val message: String,
    val timestamp: Long,
    val turnId: Long? = null
)
```

| Parameter   | Type                        | Description                                                                     |
| ----------- | --------------------------- | ------------------------------------------------------------------------------- |
| `type`      | [`ModuleType`](#moduletype) | Error type (e.g., LLM failure, TTS exception). See [`ModuleType`](#moduletype). |
| `code`      | `Int`                       | Specific error code.                                                            |
| `message`   | `String`                    | Description of the error.                                                       |
| `timestamp` | `Long`                      | Timestamp of the error in milliseconds since Unix epoch (January 1, 1970 UTC).  |
| `turnId`    | `Long?`                     | (Optional) `turnId` corresponding to the image upload error.                    |

### `ChatMessage` [#chatmessage]

Sealed base class for all message types sent to agents.

This sealed class hierarchy provides a type-safe way to handle messages of different content types.

```kotlin
sealed class ChatMessage
```

### `ImageMessage` [#imagemessage]

Used to send image content to the agent.

Supports specifying image files via HTTP/HTTPS links.

```kotlin
data class ImageMessage(
    val uuid: String,
    val imageUrl: String,
) : ChatMessage()
```

| Parameter  | Type    | Description                              |
| ---------- | ------- | ---------------------------------------- |
| `uuid`     | String  | Unique identifier for the image message. |
| `imageUrl` | String? | HTTP/HTTPS image file link.              |

### MessageReceipt [#messagereceipt]

`MessageReceipt` represents message receipt information and supports processing multiple media types through `MediaInfo`.

```kotlin
data class MessageReceipt(
    val type: ModuleType,
    val chatMessageType: ChatMessageType,
    val turnId: Long,
    val message: String
)
```

| Parameter         | Type            | Description                                                                                                                                  |
| ----------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `type`            | ModuleType      | Module type. See ModuleType, for example: llm, mllm, tts, context.                                                                           |
| `chatMessageType` | ChatMessageType | Message error type. See [ChatMessageType](#chatmessagetype).                                                                                 |
| `turnId`          | Long            | Turn ID of the message.                                                                                                                      |
| `message`         | String          | Message content. Must be parsed according to the `type` field. For `context`, this is usually a JSON string containing resource information. |

### ChatMessageType [#chatmessagetype]

Used to distinguish different types of messages in the session system.

```kotlin
enum class ChatMessageType(val value: String) {
    Text("text"),
    Image("picture"),
    UNKNOWN("unknown");

    companion object {
        fun fromValue(value: String): ChatMessageType {
            return ChatMessageType.entries.find { it.value == value } ?: UNKNOWN
        }
    }
}
```

| Parameter | Type     | Description                        |
| --------- | -------- | ---------------------------------- |
| `value`   | `String` | The string value to match against. |

### MessageError [#messageerror]

Used to process and report message error information.

```kotlin
data class MessageError(
    val chatMessageType: ChatMessageType,
    val code: Int,
    val message: String,
    val timestamp: Long
)
```

| Parameter         | Type              | Description                                                                                              |
| ----------------- | ----------------- | -------------------------------------------------------------------------------------------------------- |
| `chatMessageType` | `ChatMessageType` | Message error type. See `ChatMessageType` for details.                                                   |
| `code`            | `Int`             | Error code used to identify specific error scenarios.                                                    |
| `message`         | `String`          | Error description providing detailed explanation, usually a JSON string containing resource information. |
| `timestamp`       | `Long`            | Timestamp of when the event occurred (milliseconds since January 1, 1970 UTC).                           |

### `Transcript` [#transcript]

Represents a full transcribed message used to render the UI.

```kotlin
data class Transcript(
    val turnId: Long,
    val userId: String = "",
    val text: String,
    var status: TranscriptStatus,
    var type: TranscriptType
)
```

| Parameter | Type               | Description                                                                       |
| --------- | ------------------ | --------------------------------------------------------------------------------- |
| `turnId`  | `Long`             | Session turn identifier.                                                          |
| `userId`  | `String`           | User ID linked to the transcript.                                                 |
| `text`    | `String`           | Transcribed message content.                                                      |
| `status`  | `TranscriptStatus` | Current status of transcription. See [`TranscriptStatus`](#transcriptstatus).     |
| `type`    | `TranscriptType`   | Transcript type (e.g., `AGENT`, `USER`). See [`TranscriptType`](#transcripttype). |

### `ConversationalAIAPIConfig` [#conversationalaiapiconfig]

Holds configuration parameters for initializing the Conversational AI API.

```kotlin
data class ConversationalAIAPIConfig(
    val rtcEngine: RtcEngine,
    val rtmClient: RtmClient,
    val renderMode: TranscriptRenderMode = TranscriptRenderMode.Word,
    val enableLog: Boolean = false
)
```

| Parameter    | Type                   | Description                                                                                                                                   |
| ------------ | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `rtcEngine`  | `RtcEngine`            | Audio/video engine instance. See [`RtcEngine`](https://api-ref.agora.io/en/video-sdk/android/4.x/API/class_irtcengine.html#class_irtcengine). |
| `rtmClient`  | `RtmClient`            | Real-time messaging client. See the [Signaling Android API reference](https://api-ref.agora.io/en/signaling-sdk/android/2.x/index.html).      |
| `renderMode` | `TranscriptRenderMode` | Transcript rendering style. Default: word-by-word. See [`TranscriptRenderMode`](#transcriptrendermode).                                       |
| `enableLog`  | `Boolean`              | Enables logging if `true`. Default is `false`.                                                                                                |

### `ConversationalAIAPIError` [#conversationalaiapierror]

Represents an error type in the Conversational AI API.

```kotlin
sealed class ConversationalAIAPIError : Exception() {
    data class RtmError(val code: Int, val msg: String) : ConversationalAIAPIError()
    data class RtcError(val code: Int, val msg: String) : ConversationalAIAPIError()
    data class UnknownError(val msg: String) : ConversationalAIAPIError()

    val errorCode: Int
        get() = when (this) {
            is RtmError -> this.code
            is RtcError -> this.code
            is UnknownError -> -100
        }

    val errorMessage: String
        get() = when (this) {
            is RtmError -> this.msg
            is RtcError -> this.msg
            is UnknownError -> this.msg
        }
}
```

| Property       | Type     | Description                                                           |
| -------------- | -------- | --------------------------------------------------------------------- |
| `errorCode`    | `Int`    | `RtmError`/`RtcError`: specific code; `UnknownError`: returns `-100`. |
| `errorMessage` | `String` | Human-readable description of the error.                              |

## Enum classes [#enum-classes]

### Priority [#priority]

Controls the priority with which the agent handles incoming messages during an interaction.

| Value       | Description                                                                                                                                            |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `INTERRUPT` | High priority: Immediately interrupt the current interaction and process the message. Suitable for urgent or time-sensitive content.                   |
| `APPEND`    | Medium priority: The message is queued for processing after the current interaction is completed and is suitable for subsequent questions.             |
| `IGNORE`    | Low priority: This message is only processed when the agent is idle, and will be discarded during ongoing interactions. Suitable for optional content. |

### AgentState [#agentstate]

Represents the current state of the agent.

| Value       | Description                      |
| ----------- | -------------------------------- |
| `SILENT`    | Agent is silent.                 |
| `LISTENING` | Agent is listening.              |
| `THINKING`  | Agent is processing or thinking. |
| `SPEAKING`  | Agent is speaking.               |
| `UNKNOWN`   | Unknown state.                   |

### ModuleType [#moduletype]

Performance module type enumeration.

| Value     | Description                                   |
| --------- | --------------------------------------------- |
| `LLM`     | LLM inference latency measurement.            |
| `MLLM`    | MLLM inference latency measurement.           |
| `TTS`     | Text-to-speech synthesis latency measurement. |
| `UNKNOWN` | Unknown type.                                 |

### MessageType [#messagetype]

Used to distinguish different types of messages in the system.

| Value             | Description                      |
| ----------------- | -------------------------------- |
| `ASSISTANT`       | AI assistant transcript message. |
| `USER`            | User transcript message.         |
| `ERROR`           | Error message.                   |
| `METRICS`         | Performance metrics message.     |
| `INTERRUPT`       | Interrupt message.               |
| `UNKNOWN`         | Unknown message type.            |
| `MESSAGE_RECEIPT` | Message receipt.                 |

### TranscriptRenderMode [#transcriptrendermode]

Transcript rendering mode.

| Value  | Description                               |
| ------ | ----------------------------------------- |
| `Word` | Word-by-word transcription and rendering. |
| `Text` | Full text transcription and rendering.    |

### TranscriptType [#transcripttype]

Transcript source type.

| Value   | Description       |
| ------- | ----------------- |
| `AGENT` | Agent transcript. |
| `USER`  | User transcript.  |

### TranscriptStatus [#transcriptstatus]

Indicates the current status of the transcription.

| Value         | Description                                                                 |
| ------------- | --------------------------------------------------------------------------- |
| `IN_PROGRESS` | The transcript is still being generated or the speech is still in progress. |
| `END`         | The transcription completed normally.                                       |
| `INTERRUPTED` | Transcription was interrupted before completion.                            |
| `UNKNOWN`     | Unknown status.                                                             |