# iOS toolkit API (/en/api-reference/api-ref/conversational-ai/client-toolkit/ios)
> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).
The [iOS toolkit API](https://github.com/AgoraIO-Community/Conversational-AI-Demo/tree/main/iOS/Scenes/ConvoAI/ConvoAI/ConvoAI/Classes/ConversationalAIAPI) provides the following classes and methods.
## ConversationalAIAPI class [#conversationalaiapi-class]
| API | Description |
| --------------------------------------------------- | ----------------------------------------------------------- |
| [`chat`](#chat) | Send a message to the agent. |
| [`addHandler`](#addhandler) | Add an event handler to receive the callback. |
| [`removeHandler`](#removehandler) | Removes an event handler. |
| [`subscribeMessage`](#subscribemessage) | Subscribe to channel messages. |
| [`unsubscribeMessage`](#unsubscribemessage) | Unsubscribe from channel messages. |
| [`interrupt`](#interrupt) | Interrupt the agent's current speech or task processing. |
| [`loadAudioSettings`\[1/2\]](#loadaudiosettings-12) | Set audio best practice parameters for optimal performance. |
| [`loadAudioSettings`\[2/2\]](#loadaudiosettings-22) | Set audio best practice parameters for specific scenarios. |
| [`destroy`](#destroy) | Destroys the API instance and releases all resources. |
### `chat` [#chat]
Sends a message to the agent for processing.
You can use this method to send text and image messages to the agent, and the completion callback indicates the success or failure of the operation.
```objc
@objc func chat(agentUserId: String, message: ChatMessage, completion: @escaping (ConversationalAIAPIError?) -> Void)
```
| Parameter | Type | Description |
| ------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. Must be globally unique. |
| `message` | `ChatMessage` | Message object containing the image URL. See [`ChatMessage`](#chatmessage) for details. |
| `completion` | `@escaping (ConversationalAIAPIError?) -> Void` | Callback function invoked when the operation completes. Returns `nil` on success, or error information on failure. See [`ConversationalAIAPIError`](#conversationalaiapierror) for details. |
### `addHandler` [#addhandler]
Add an event handler to receive the callback.
You can register a delegate to receive session events, state changes, and other notifications.
```objc
@objc func addHandler(handler: ConversationalAIAPIEventHandler)
```
| Parameter | Type | Description |
| --------- | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `handler` | `ConversationalAIAPIEventHandler` | Event handler to implement the `ConversationalAIAPIEventHandler` protocol. See [`ConversationalAIAPIEventHandler`](#conversationalaiapieventhandler-class). |
### `removeHandler` [#removehandler]
Removes an event handler.
```objc
@objc func removeHandler(handler: ConversationalAIAPIEventHandler)
```
| Parameter | Type | Description |
| --------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `handler` | `ConversationalAIAPIEventHandler` | The event handler to remove. See [`ConversationalAIAPIEventHandler`](#conversationalaiapieventhandler-class). |
### `subscribeMessage` [#subscribemessage]
Subscribe to channel messages.
Set channel parameters and register message subscription callbacks. This method is called when the channel changes, usually when the agent starts.
```objc
@objc func subscribeMessage(channelName: String, completion: @escaping (ConversationalAIAPIError?) -> Void)
```
| Parameter | Type | Description |
| ------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `channelName` | `String` | The name of the channel to subscribe to. |
| `completion` | `(ConversationalAIAPIError?) -> Void` | A callback that returns error information when subscription fails. See [`ConversationalAIAPIError`](#conversationalaiapierror). |
### `unsubscribeMessage` [#unsubscribemessage]
Unsubscribe from channel messages.
Calling this method can stop receiving messages from the specified channel, which is suitable for scenarios where the connection with the agent is disconnected.
```objc
@objc func unsubscribeMessage(channelName: String, completion: @escaping (ConversationalAIAPIError?) -> Void)
```
| Parameter | Type | Description |
| ------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `channelName` | `String` | The name of the channel to unsubscribe from. |
| `completion` | `(ConversationalAIAPIError?) -> Void` | The callback after the unsubscription operation is completed. You can get error information through this callback. See [`ConversationalAIAPIError`](#conversationalaiapierror). |
### `interrupt` [#interrupt]
Interrupt the agent's current ongoing speech or task processing.
This method can be used to interrupt an agent that is currently speaking or processing a task.
```objc
@objc func interrupt(agentUserId: String, completion: @escaping (ConversationalAIAPIError?) -> Void)
```
If `error` has a value, it means the message sending failed. If `error` is nil, it means the message was sent successfully, but it does not guarantee that the agent was successfully interrupted.
| Parameter | Type | Description |
| ------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `agentUserId` | `String` | The RTM user ID of the agent, must be globally unique. |
| `completion` | `(ConversationalAIAPIError?) -> Void` | The callback function when the operation is completed. You can get the result or error information of the operation through the parameters of the callback. See [`ConversationalAIAPIError`](#conversationalaiapierror). |
### `loadAudioSettings[1/2]` [#loadaudiosettings12]
Set audio best practice parameters for optimal performance.
Sets the audio parameters needed for optimal performance in agent conversations. By default, `.aiClient` Audio Scene is used.
```objc
@objc func loadAudioSettings()
```
To enable audio best practices, you must call this method before each `joinChannel` call.
**Sample code:**
```swift
// Set audio best practice parameters before joining the channel
api.loadAudioSettings() // Use default scenario
// Then join the channel
rtcEngine.joinChannel(byToken: token, channelId: channelName, info: nil, uid: userId)
```
### `loadAudioSettings[2/2]` [#loadaudiosettings22]
Set audio best practice parameters for specific scenarios.
This method allows you to configure the audio parameters required for optimal performance in your agent conversations.
```objc
@objc func loadAudioSettings(scenario: AgoraAudioScenario)
```
If you need to enable audio best practices, you must call this method before each `joinChannel` call.
| Parameter | Type | Description |
| ---------- | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `scenario` | `AgoraAudioScenario` | The audio scenario. See [AgoraAudioScenario](https://api-ref.agora.io/en/voice-sdk/ios/4.x/documentation/agorartckit/agoraaudioscenario). If you enable the AI Avatar feature, set the scenario to `default` for better mixing results. |
### `destroy` [#destroy]
Destroys the API instance and releases all resources.
Calling this method destroys the current API instance and releases all resources. After calling this method, the instance cannot be used again. Please call this method when you no longer need to use the API.
```objc
@objc func destroy()
```
## ConversationalAIAPIEventHandler class [#conversationalaiapieventhandler-class]
| API | Description |
| ----------------------------------------------------- | -------------------------------------------------------------------- |
| [`onMessageError`](#onmessageerror) | A callback triggered when an error occurs during message processing. |
| [`onMessageReceiptUpdated`](#onmessagereceiptupdated) | Image message information update callback. |
| [`onAgentStateChanged`](#onagentstatechanged) | Callback when the agent status changes. |
| [`onAgentListeningChanged`](#onagentlisteningchanged) | Called when the agent's listening state changes. |
| [`onAgentThinkingChanged`](#onagentthinkingchanged) | Called when the agent's thinking state changes. |
| [`onAgentSpeakingChanged`](#onagentspeakingchanged) | Called when the agent's speaking state changes. |
| [`onAgentInterrupted`](#onagentinterrupted) | The callback triggered when an interrupt event occurs. |
| [`onAgentMetrics`](#onagentmetrics) | A callback that is triggered when performance metrics are available. |
| [`onAgentError`](#onagenterror) | Callback when an error occurs in the AI module. |
| [`onTranscriptUpdated`](#ontranscriptupdated) | Transcription content update callback. |
### `onMessageError` [#onmessageerror]
This callback is triggered when an error occurs during message processing. For example, if sending a chat message fails, an error message is returned.
```objc
@objc func onMessageError(agentUserId: String, error: MessageError)
```
| Parameter | Type | Description |
| ------------- | -------------- | ----------------------------------------------------------------------------- |
| `agentUserId` | `String` | RTM user ID of the agent. |
| `error` | `MessageError` | [`MessageError`](#messageerror) object containing the error type and details. |
### `onMessageReceiptUpdated` [#onmessagereceiptupdated]
Image message information update callback.
This callback is triggered when an image is processed in a session and provides image metadata.
```objc
@objc func onMessageReceiptUpdated(agentUserId: String, messageReceipt: MessageReceipt)
```
| Parameter | Type | Description |
| ---------------- | ---------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `agentUserId` | `String` | RTM user ID of the agent. |
| `messageReceipt` | `MessageReceipt` | Message receipt containing the type, module, and image information. See [`MessageReceipt`](#messagereceipt) for more details. |
### `onAgentStateChanged` [#onagentstatechanged]
Callback when the agent status changes.
This callback is triggered when the agent state changes, such as switching from idle to silent, listening, thinking, or speaking. You can use this callback to update the user interface or track the flow of the conversation.
```objc
@objc func onAgentStateChanged(agentUserId: String, event: StateChangeEvent)
```
| Parameter | Type | Description |
| ------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `event` | `StateChangeEvent` | Agent status change event, including status, round ID, timestamp and reason. See [`StateChangeEvent`](#statechangeevent). |
### `onAgentListeningChanged` [#onagentlisteningchanged]
Triggered when the agent's listening state changes.
This callback is a fine-grained complement to `onAgentStateChanged`, used to monitor whether the agent has started or stopped listening to user input.
```objc
@objc optional func onAgentListeningChanged(agentUserId: String, isListening: Bool)
```
| Parameter | Type | Description |
| ------------- | -------- | ---------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `isListening` | `Bool` | Whether the agent is currently in a listening state. |
### `onAgentThinkingChanged` [#onagentthinkingchanged]
Triggered when the agent's thinking state changes.
This callback is a fine-grained complement to `onAgentStateChanged`, used to monitor whether the agent has started or stopped processing a request.
```objc
@objc optional func onAgentThinkingChanged(agentUserId: String, isThinking: Bool)
```
| Parameter | Type | Description |
| ------------- | -------- | --------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `isThinking` | `Bool` | Whether the agent is currently in a thinking state. |
### `onAgentSpeakingChanged` [#onagentspeakingchanged]
Triggered when the agent's speaking state changes.
This callback is a fine-grained complement to `onAgentStateChanged`, used to monitor whether the agent has started or stopped playing back speech.
```objc
@objc optional func onAgentSpeakingChanged(agentUserId: String, isSpeaking: Bool)
```
| Parameter | Type | Description |
| ------------- | -------- | --------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `isSpeaking` | `Bool` | Whether the agent is currently in a speaking state. |
### `onAgentInterrupted` [#onagentinterrupted]
The callback triggered when an interrupt event occurs.
```objc
@objc func onAgentInterrupted(agentUserId: String, event: InterruptEvent)
```
This callback may not be synchronized with the agent's state. It is not recommended to process business logic in this callback.
| Parameter | Type | Description |
| ------------- | ---------------- | ------------------------------------------------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `event` | `InterruptEvent` | Interrupt event, including round ID and timestamp. See [`InterruptEvent`](#interruptevent). |
### `onAgentMetrics` [#onagentmetrics]
A callback that is triggered when performance metrics are available.
This callback provides performance data, such as LLM inference latency and TTS speech synthesis latency, for monitoring system performance.
```objc
@objc func onAgentMetrics(agentUserId: String, metrics: Metric)
```
This performance indicator callback is not necessarily synchronized with the agent's state, so it is not recommended to process business logic in this callback.
| Parameter | Type | Description |
| ------------- | -------- | ------------------------------------------------------------------------------------ |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `metrics` | `Metric` | Performance indicator, including type, value and timestamp. See [`Metric`](#metric). |
### `onAgentError` [#onagenterror]
Callback when an error occurs in the AI module.
This callback is called when an error occurs in a module component (such as LLM, TTS, etc.). It can be used for error monitoring, logging, and implementing service degradation strategies.
```objc
@objc func onAgentError(agentUserId: String, error: ModuleError)
```
This callback is not necessarily synchronized with the state of the agent, so it is not recommended to process business logic in this callback.
| Parameter | Type | Description |
| ------------- | ------------- | --------------------------------------------------------------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `error` | `ModuleError` | Module error, including type, error code, error message and timestamp. See [`ModuleError`](#moduleerror). |
### `onTranscriptUpdated` [#ontranscriptupdated]
Transcript content update callback.
This callback is triggered when the speech transcript content in the session is updated.
```objc
@objc func onTranscriptUpdated(agentUserId: String, transcript: Transcript)
```
| Parameter | Type | Description |
| ------------- | ------------ | ---------------------------------------------------------------------------------------------- |
| `agentUserId` | `String` | The RTM user ID of the agent. |
| `transcript` | `Transcript` | Transcript data, including text content, status and metadata. See [`Transcript`](#transcript). |
## Structures [#structures]
### `ChatMessage` [#chatmessage]
Defines a common interface for different types of chat messages.
```objc
@objc public protocol ChatMessage {
var messageType: ChatMessageType { get }
}
```
| Parameter | Type | Description |
| ------------- | ----------------- | -------------------------------------------------------------------- |
| `messageType` | `ChatMessageType` | Message type. See [`ChatMessageType`](#chatmessagetype) for details. |
### `ImageMessage` [#imagemessage]
Used to send image content to an agent.
Images are displayed as URLs, which are HTTP/HTTPS links pointing to the image file (recommended for large images). Example usage: `ImageMessage(uuid = "img_123", url = "https://example.com/image.jpg")`
```objc
@objc public class ImageMessage: NSObject, ChatMessage {
@objc public let messageType: ChatMessageType = .image
@objc public let uuid: String
@objc public let url: String?
}
```
| Parameter | Type | Description |
| ------------- | ----------------- | --------------------------------------------------------------------------------------------- |
| `messageType` | `ChatMessageType` | Type of the message. See `ChatMessageType` for more details. |
| `uuid` | `String` | Unique identifier for the image message. The agent uses this UUID to identify the image. |
| `url` | `String?` | HTTP/HTTPS link to the image file which the proxy will use to download and process the image. |
### `MessageReceipt` [#messagereceipt]
The `MessageReceipt` model is used to track message processing status and metadata.
```objc
@objc public class MessageReceipt: NSObject {
@objc public let moduleType: ModuleType
@objc public let messageType: ChatMessageType
@objc public let message: String
@objc public let turnId: Int
}
```
| Parameter | Type | Description |
| ------------- | ----------------- | ---------------------------------------------------------------------- |
| `moduleType` | `ModuleType` | Module type. See `ModuleType` for more details. |
| `messageType` | `ChatMessageType` | Type of the message. See `ChatMessageType` for more details. |
| `message` | `String` | Image information. |
| `turnId` | `Int` | Conversation turn ID, used to identify a specific turn in the session. |
### `MessageError` [#messageerror]
A data class for processing and reporting message errors.
Contains the error type, error code, error description, and timestamp.
```objc
@objc public class MessageError: NSObject {
@objc public let type: ChatMessageType
@objc public let code: Int
@objc public let message: String
@objc public let timestamp: TimeInterval
@objc public init(type: ChatMessageType, code: Int, message: String, timestamp: TimeInterval)
}
```
| Parameter | Type | Description |
| ----------- | ----------------- | --------------------------------------------------------------------------------------------------------------- |
| `type` | `ChatMessageType` | Message error type. See ChatMessageType. |
| `code` | `Int` | Error code to identify specific error conditions. |
| `message` | `String` | Error description, providing detailed error explanation, usually a JSON string containing resource information. |
| `timestamp` | `TimeInterval` | Timestamp of the error (milliseconds since January 1, 1970, UTC). |
### `TextMessage` [#textmessage]
Used to send natural language text content to an agent.
Supports priority control and interruptibility settings, enabling fine-grained control over how the AI processes and responds to text input.
```objc
@objc public class TextMessage: NSObject, ChatMessage {
@objc public let messageType: ChatMessageType = .text
@objc public let priority: Priority
@objc public let responseInterruptable: Bool
@objc public let text: String?
}
```
| Parameter | Type | Description |
| ----------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `messageType` | `ChatMessageType` | Message type. See [`ChatMessageType`](#chatmessagetype). |
| `priority` | `Priority` | Message handling priority. See `Priority`. |
| `responseInterruptable` | `Bool` | Whether the response to this message can be interrupted by a higher-priority message. `YES` (default) means the response can be interrupted. `NO` means the response cannot be interrupted. |
| `text` | `String?` | Text content of the message. |
### `StateChangeEvent` [#statechangeevent]
Indicates an agent state change event, including complete state information and timestamp.
Used to track session flow and update UI status indicators.
```objc
@objc public class StateChangeEvent: NSObject {
@objc public let state: AgentState
@objc public let turnId: Int
@objc public let timestamp: TimeInterval
@objc public let reason: String
@objc public init(state: AgentState, turnId: Int, timestamp: TimeInterval, reason: String)
public override var description: String {
return "StateChangeEvent(state: \(state), turnId: \(turnId), timestamp: \(timestamp), reason: \(reason))"
}
}
```
| Parameter | Type | Description |
| ----------- | --------------------------- | ----------------------------------------------------------------- |
| `state` | [`AgentState`](#agentstate) | Current agent state. See [`AgentState`](#agentstate). |
| `turnId` | `Int` | Session round ID, used to identify a specific session round. |
| `timestamp` | `TimeInterval` | Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC). |
| `reason` | `String` | The reason for the status change. |
### `InterruptEvent` [#interruptevent]
Indicates a session interruption event.
It is usually triggered when the user actively interrupts the AI speech or the system detects a high-priority message. It is used to record the interruption behavior and perform corresponding processing.
```objc
@objc public class InterruptEvent: NSObject {
@objc public let turnId: Int
@objc public let timestamp: TimeInterval
@objc public init(turnId: Int, timestamp: TimeInterval) {
self.turnId = turnId
self.timestamp = timestamp
}
public override var description: String {
return "InterruptEvent(turnId: \(turnId), timestamp: \(timestamp))"
}
}
```
| Parameter | Type | Description |
| ----------- | -------------- | ----------------------------------------------------------------- |
| `turnId` | `Int` | The ID of the interrupted session round. |
| `timestamp` | `TimeInterval` | Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC). |
### `Metric` [#metric]
Used to record and transmit system performance data.
For example, LLM inference delay, TTS synthesis delay, etc. This data can be used for performance monitoring, system optimization, and user experience improvement.
```objc
@objc public class Metric: NSObject {
@objc public let type: ModuleType
@objc public let name: String
@objc public let value: Double
@objc public let timestamp: TimeInterval
@objc public init(type: ModuleType, name: String, value: Double, timestamp: TimeInterval)
public override var description: String {
return "Metric(type: \(type.stringValue), name: \(name), value: \(value), timestamp: \(timestamp))"
}
}
```
| Parameter | Type | Description |
| ----------- | --------------------------- | --------------------------------------------------------------------------- |
| `type` | [`ModuleType`](#moduletype) | Indicator type. See [`ModuleType`](#moduletype). |
| `name` | `String` | The indicator name of a specific performance item. |
| `value` | `Double` | Metric value, usually latency (milliseconds) or other quantitative metrics. |
| `timestamp` | `TimeInterval` | Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC). |
### `ModuleError` [#moduleerror]
Used to process and report AI module related error information.
Contains error type, error code, error description, and timestamp to facilitate error monitoring, logging, and troubleshooting.
```objc
@objc public class ModuleError: NSObject {
@objc public let type: ModuleType
@objc public let code: Int
@objc public let message: String
@objc public let timestamp: TimeInterval
@objc public init(type: ModuleType, code: Int, message: String, timestamp: TimeInterval) {
self.type = type
self.code = code
self.message = message
self.timestamp = timestamp
}
public override var description: String {
return "ModuleError(type: \(type.stringValue), code: \(code), message: \(message), timestamp: \(timestamp))"
}
}
```
| Parameter | Type | Description |
| ----------- | --------------------------- | ----------------------------------------------------------------- |
| `type` | [`ModuleType`](#moduletype) | Error type. See [`ModuleType`](#moduletype). |
| `code` | `Int` | Error codes that identify specific error conditions. |
| `message` | `String` | Provides an error description that details the error. |
| `timestamp` | `TimeInterval` | Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC). |
### `Transcript` [#transcript]
Used to represent a user-visible transcript message.
Complete data class for rendering transcripts at the UI level.
```objc
@objc public class Transcript: NSObject {
@objc public let turnId: Int
@objc public let userId: String
@objc public let text: String
@objc public var status: TranscriptStatus
@objc public var type: TranscriptType
}
```
| Parameter | Type | Description |
| --------- | ------------------ | ---------------------------------------------------------------------------------------- |
| `turnId` | `Int` | Unique identifier for the session turn. |
| `userId` | `String` | The user identifier associated with this transcript. |
| `text` | `String` | The actual transcript text content. |
| `status` | `TranscriptStatus` | The current status of the transcription. See [`TranscriptStatus`](#transcriptstatus). |
| `type` | `TranscriptType` | The current type of transcript (agent or user). See [`TranscriptType`](#transcripttype). |
### `ConversationalAIAPIConfig` [#conversationalaiapiconfig]
Conversational AI API initialization configuration class.
Contains the configuration parameters required for Conversational AI API initialization, including `rtcEngine` for audio and video communication, `rtmEngine` for message communication, and transcript rendering mode settings.
```objc
@objc public class ConversationalAIAPIConfig: NSObject {
@objc public weak var rtcEngine: AgoraRtcEngineKit?
@objc public weak var rtmEngine: AgoraRtmClientKit?
@objc public var renderMode: TranscriptRenderMode
@objc public var enableLog: Bool
}
```
| Parameter | Type | Description |
| ------------ | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `rtcEngine` | `AgoraRtcEngineKit?` | Engine instance used for audio and video communication. See [`AgoraRtcEngineKit`](https://api-ref.agora.io/en/video-sdk/ios/4.x/API/class_agorartcenginekit.html). |
| `rtmEngine` | `AgoraRtmClientKit?` | Client instance for real-time messaging. See the [Signaling iOS API reference](https://agoraio.github.io/AgoraRtm_Apple/documentation/agorartm). |
| `renderMode` | `TranscriptRenderMode` | Transcript rendering mode. See [`TranscriptRenderMode`](#transcriptrendermode). |
| `enableLog` | `Bool` | Whether to enable verbose logging. `YES` enables verbose logging. `NO` (default) disables logging. |
### `ConversationalAIAPIError` [#conversationalaiapierror]
Class for logging and communicating error information.
```objc
@objc public class ConversationalAIAPIError: NSObject {
@objc public let type: ConversationalAIAPIErrorType
@objc public let code: Int
@objc public let message: String
@objc public init(type: ConversationalAIAPIErrorType, code: Int, message: String) {
self.type = type
self.code = code
self.message = message
}
public override var description: String {
return "ConversationalAIAPIError(type: \(type), code: \(code), message: \(message))"
}
}
```
| Property | Type | Description |
| --------- | ------------------------------ | -------------------------------------------------------------------------------- |
| `type` | `ConversationalAIAPIErrorType` | Error type. See [`ConversationalAIAPIErrorType`](#conversationalaiapierrortype). |
| `code` | `Int` | Error codes that identify specific error conditions. |
| `message` | `String` | Provides an error description that details the error. |
## Enum classes [#enum-classes]
### `ChatMessageType` [#chatmessagetype]
Message type enumeration.
Used to distinguish different message types within a session.
| Value | Description |
| --------- | -------------------------- |
| `text` | (0): Text message type. |
| `image` | (1): Image message type. |
| `unknown` | (2): Unknown message type. |
### `Priority` [#priority]
Controls the priority with which the agent processes messages.
| Value | Description |
| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `interrupt` | (0): High priority. The agent will immediately interrupt the current interaction and process the message. Suitable for urgent or time-sensitive messages. |
| `append` | (1): Medium priority. The agent will queue the message after the current interaction is completed, suitable for subsequent questions. |
| `ignore` | (2): Low priority. If the agent is currently interacting, the message will be discarded and only processed when idle, suitable for optional content. |
### `AgentState` [#agentstate]
Represents the different states of the agent during the dialogue.
| Value | Description |
| ----------- | --------------------------------------------------------------------------------- |
| `idle` | (0): Idle state, the agent is not actively processing. |
| `silent` | (1): Silent state, the agent remains silent but is ready to listen. |
| `listening` | (2): Listening state, the agent is actively listening to user input. |
| `thinking` | (3): Thinking state, the agent is processing user input and generating responses. |
| `speaking` | (4): Speaking state, the agent is speaking or outputting audio content. |
| `unknown` | (5): Unknown state, used for fallback processing of unrecognized states. |
### `ModuleType` [#moduletype]
Represents different types of AI modules for performance monitoring.
| Value | Description |
| --------- | ----------------------------------------------- |
| `llm` | (0): Reasoning with large language models. |
| `mllm` | (1): Multimodal large language model reasoning. |
| `tts` | (2): Text-to-speech synthesis. |
| `context` | (3): Context module type. |
| `unknown` | (4): Unknown module type. |
### `MessageType` [#messagetype]
Used to distinguish different types of messages in the session system.
| Value | Description |
| ----------- | --------------------------------- |
| `metrics` | The indicator message type. |
| `error` | The error message type. |
| `assistant` | AI agent transcript message type. |
| `user` | User transcript message type. |
| `interrupt` | The interrupt message type. |
| `state` | Status message type. |
| `imageInfo` | Image information message type. |
| `unknown` | Unknown message type. |
### `TranscriptRenderMode` [#transcriptrendermode]
Transcript rendering mode.
| Value | Description |
| ------- | ------------------------------------------------------------------------------------------------- |
| `words` | (0): Word-by-word transcript rendering mode, updated every time a word is processed. |
| `text` | (1): Sentence-by-sentence transcript rendering mode, updated when the complete sentence is ready. |
### `TranscriptType` [#transcripttype]
Distinguish the source type of the transcribed text.
By identifying whether the transcribed text comes from an agent or a user, it helps manage the conversation flow and interface presentation.
| Value | Description |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `agent` | Transcript generated by the agent. Contains the responses and statements of the agent assistant, and is used to render the agent's voice in the conversational interface. |
| `user` | Transcribed text from the user. Contains the text converted from the user's voice input, which is used to display the user's voice content in the conversation process. |
### `TranscriptStatus` [#transcriptstatus]
Indicates the current state of the transcription in the conversation flow.
Used to track and manage the lifecycle status of transcripts.
| Value | Description |
| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `inprogress` | (0): Transcription in progress. This state is set while the transcript is being generated or played, indicating that the content is still being processed or streamed. |
| `end` | (1): Transcription completed. This status is set when text generation ends normally, indicating the natural end of the transcript segment. |
| `interrupted` | (2): Transcription interrupted. This state is set when text generation is stopped prematurely, which is applicable when it is interrupted by a higher priority message. |
### `ConversationalAIAPIErrorType` [#conversationalaiapierrortype]
Used to distinguish different error types in conversational agent systems.
| Value | Description |
| ---------- | -------------------------------------------------- |
| `unknown` | (0): Unknown error type. |
| `rtcError` | (2): RTC (real-time communication) related errors. |
| `rtmError` | (3): RTM (Real Time Messaging) related errors. |