iOS toolkit API

The iOS toolkit API provides the following classes and methods.

API	Description
chat	Send a message to the agent.
addHandler	Add an event handler to receive the callback.
removeHandler	Removes an event handler.
subscribeMessage	Subscribe to channel messages.
unsubscribeMessage	Unsubscribe from channel messages.
interrupt	Interrupt the agent's current speech or task processing.
loadAudioSettings[1/2]	Set audio best practice parameters for optimal performance.
loadAudioSettings[2/2]	Set audio best practice parameters for specific scenarios.
destroy	Destroys the API instance and releases all resources.

Sends a message to the agent for processing.

You can use this method to send text and image messages to the agent, and the completion callback indicates the success or failure of the operation.

_1

@objc func chat(agentUserId: String, message: ChatMessage, completion: @escaping (ConversationalAIAPIError?) -> Void)

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent. Must be globally unique.
message	ChatMessage	Message object containing the image URL. See ChatMessage for details.
completion	@escaping (ConversationalAIAPIError?) -> Void	Callback function invoked when the operation completes. Returns nil on success. Returns error information on failure, see ConversationalAIAPIError for details.

Add an event handler to receive the callback.

You can register a delegate to receive session events, state changes, and other notifications.

_1

@objc func addHandler(handler: ConversationalAIAPIEventHandler)

Parameter	Type	Description
handler	ConversationalAIAPIEventHandler	Event handler to implement the ConversationalAIAPIEventHandler protocol. See ConversationalAIAPIEventHandler.

Removes an event handler.

_1

@objc func removeHandler(handler: ConversationalAIAPIEventHandler)

Parameter	Type	Description
handler	ConversationalAIAPIEventHandler	The event handler to remove. See ConversationalAIAPIEventHandler.

Subscribe to channel messages.

Set channel parameters and register message subscription callbacks. This method is called when the channel changes, usually when the agent starts.

_1

@objc func subscribeMessage(channelName: String, completion: @escaping (ConversationalAIAPIError?) -> Void)

Parameter	Type	Description
channelName	String	The name of the channel to subscribe to.
completion	(ConversationalAIAPIError?) -> Void	A callback that returns error information when subscription fails. See ConversationalAIAPIError.

Unsubscribe from channel messages.

Calling this method can stop receiving messages from the specified channel, which is suitable for scenarios where the connection with the agent is disconnected.

_1

@objc func unsubscribeMessage(channelName: String, completion: @escaping (ConversationalAIAPIError?) -> Void)

Parameter	Type	Description
channelName	String	The name of the channel to unsubscribe from.
completion	(ConversationalAIAPIError?) -> Void	The callback after the unsubscription operation is completed. You can get error information through this callback. See ConversationalAIAPIError.

Interrupt the agent's current ongoing speech or task processing.

This method can be used to interrupt an agent that is currently speaking or processing a task.

_1

@objc func interrupt(agentUserId: String, completion: @escaping (ConversationalAIAPIError?) -> Void)

info

If error has a value, it means the message sending failed. If error is nil, it means the message was sent successfully, but it does not guarantee that the agent was successfully interrupted.

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent, must be globally unique.
completion	(ConversationalAIAPIError?) -> Void	The callback function when the operation is completed. You can get the result or error information of the operation through the parameters of the callback. See ConversationalAIAPIError.

Set audio best practice parameters for optimal performance.

Sets the audio parameters needed for optimal performance in agent conversations. By default, .aiClient Audio Scene is used.

_1

@objc func loadAudioSettings()

info

To enable audio best practices, you must call this method before each joinChannel call.

Sample code:

_5

// Set audio best practice parameters before joining the channel

_5

api.loadAudioSettings() // Use default scenario

_5

_5

// Then join the channel

_5

rtcEngine.joinChannel(byToken: token, channelId: channelName, info: nil, uid: userId)

Set audio best practice parameters for specific scenarios.

This method allows you to configure the audio parameters required for optimal performance in your agent conversations.

_1

@objc func loadAudioSettings(scenario: AgoraAudioScenario)

info

If you need to enable audio best practices, you must call this method before each joinChannel call.

Parameter	Type	Description
scenario	AgoraAudioScenario	The audio scenario. See AgoraAudioScenario. If you enable the AI Avatar feature, set the scenario to default for better mixing results.

Destroys the API instance and releases all resources.

Calling this method destroys the current API instance and releases all resources. After calling this method, the instance cannot be used again. Please call this method when you no longer need to use the API.

_1

@objc func destroy()

API	Description
onMessageError	A callback triggered when an error occurs during message processing.
onMessageReceiptUpdated	Image message information update callback.
onAgentStateChanged	Callback when the agent status changes.
onAgentInterrupted	The callback triggered when an interrupt event occurs.
onAgentMetrics	A callback that is triggered when performance metrics are available.
onAgentError	Callback when an error occurs in the AI module.
onTranscriptUpdated	Transcription content update callback.

This callback is triggered when an error occurs during message processing. For example, if sending a chat message fails, an error message is returned.

_1

@objc func onMessageError(agentUserId: String, error: MessageError)

Parameter	Type	Description
agentUserId	String	RTM user ID of the agent.
error	MessageError	MessageError object containing the error type and details.

Image message information update callback.

This callback is triggered when an image is processed in a session and provides image metadata.

_1

@objc func onMessageReceiptUpdated(agentUserId: String, messageReceipt: MessageReceipt)

Parameter	Type	Description
agentUserId	String	RTM user ID of the agent.
messageReceipt	MessageReceipt	Message receipt containing the type, module, and image information. See MessageReceipt for more details.

Callback when the agent status changes.

This callback is triggered when the agent state changes, such as switching from idle to silent, listening, thinking, or speaking. You can use this callback to update the user interface or track the flow of the conversation.

_1

@objc func onAgentStateChanged(agentUserId: String, event: StateChangeEvent)

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent.
event	StateChangeEvent	Agent status change event, including status, round ID, timestamp and reason. See StateChangeEvent.

The callback triggered when an interrupt event occurs.

_1

@objc func onAgentInterrupted(agentUserId: String, event: InterruptEvent)

info

This callback may not be synchronized with the agent's state. It is not recommended to process business logic in this callback.

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent.
event	InterruptEvent	Interrupt event, including round ID and timestamp. See InterruptEvent.

A callback that is triggered when performance metrics are available.

This callback provides performance data, such as LLM inference latency and TTS speech synthesis latency, for monitoring system performance.

_1

@objc func onAgentMetrics(agentUserId: String, metrics: Metric)

info

This performance indicator callback is not necessarily synchronized with the agent's state, so it is not recommended to process business logic in this callback.

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent.
metrics	Metric	Performance indicator, including type, value and timestamp. See Metric.

Callback when an error occurs in the AI module.

This callback is called when an error occurs in a module component (such as LLM, TTS, etc.). It can be used for error monitoring, logging, and implementing service degradation strategies.

_1

@objc func onAgentError(agentUserId: String, error: ModuleError)

info

This callback is not necessarily synchronized with the state of the agent, so it is not recommended to process business logic in this callback.

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent.
error	ModuleError	Module error, including type, error code, error message and timestamp. See ModuleError.

Transcript content update callback.

This callback is triggered when the speech transcript content in the session is updated.

_1

@objc func onTranscriptUpdated(agentUserId: String, transcript: Transcript)

Parameter	Type	Description
agentUserId	String	The RTM user ID of the agent.
transcript	Transcript	Transcript data, including text content, status and metadata. See Transcript.

Defines a common interface for different types of chat messages.

_3

@objc public protocol ChatMessage {

_3

var messageType: ChatMessageType { get }

_3

}

Parameter	Type	Description
messageType	ChatMessageType	Message type. See ChatMessageType for details.

Used to send image content to an agent.

Images are displayed as URLs, which are HTTP/HTTPS links pointing to the image file (recommended for large images). Example usage: ImageMessage(uuid = "img_123", url = "https://example.com/image.jpg")

_5

@objc public class ImageMessage: NSObject, ChatMessage {

_5

@objc public let messageType: ChatMessageType = .image

_5

@objc public let uuid: String

_5

@objc public let url: String?

_5

}

Parameter	Type	Description
messageType	ChatMessageType	Type of the message. See ChatMessageType for more details.
uuid	String	Unique identifier for the image message. The agent uses this UUID to identify the image.
url	String?	HTTP/HTTPS link to the image file which the proxy will use to download and process the image.

The MessageReceipt model is used to track message processing status and metadata.

_6

@objc public class MessageReceipt: NSObject {

_6

@objc public let moduleType: ModuleType

_6

@objc public let messageType: ChatMessageType

_6

@objc public let message: String

_6

@objc public let turnId: Int

_6

}

Parameter	Type	Description
moduleType	ModuleType	Module type. See ModuleType for more details.
messageType	ChatMessageType	Type of the message. See ChatMessageType for more details.
message	String	Image information.
turnId	Int	Conversation turn ID, used to identify a specific turn in the session.

A data class for processing and reporting message errors.

Contains the error type, error code, error description, and timestamp.

_7

@objc public class MessageError: NSObject {

_7

@objc public let type: ChatMessageType

_7

@objc public let code: Int

_7

@objc public let message: String

_7

@objc public let timestamp: TimeInterval

_7

@objc public init(type: ChatMessageType, code: Int, message: String, timestamp: TimeInterval)

_7

}

Parameter	Type	Description
type	ChatMessageType	Message error type. See ChatMessageType.
code	Int	Error code to identify specific error conditions.
message	String	Error description, providing detailed error explanation, usually a JSON string containing resource information.
timestamp	TimeInterval	Timestamp of the error (milliseconds since January 1, 1970, UTC).

Used to send natural language text content to an agent.

Supports priority control and interruptibility settings, enabling fine-grained control over how the AI processes and responds to text input.

_6

@objc public class TextMessage: NSObject, ChatMessage {

_6

@objc public let messageType: ChatMessageType = .text

_6

@objc public let priority: Priority

_6

@objc public let responseInterruptable: Bool

_6

@objc public let text: String?

_6

}

Parameter	Type	Description
messageType	ChatMessageType	Message type. See ChatMessageType.
priority	Priority	Message handling priority. See Priority.
responseInterruptable	Bool	Whether the response to this message can be interrupted by a higher-priority message: YES (default): Response can be interrupted. NO: Response cannot be interrupted.
text	String?	Text content of the message.

Indicates an agent state change event, including complete state information and timestamp.

Used to track session flow and update UI status indicators.

_12

@objc public class StateChangeEvent: NSObject {

_12

@objc public let state: AgentState

_12

@objc public let turnId: Int

_12

@objc public let timestamp: TimeInterval

_12

@objc public let reason: String

_12

_12

@objc public init(state: AgentState, turnId: Int, timestamp: TimeInterval, reason: String)

_12

_12

public override var description: String {

_12

return "StateChangeEvent(state: \(state), turnId: \(turnId), timestamp: \(timestamp), reason: \(reason))"

_12

}

_12

}

Parameter	Type	Description
state	AgentState	Current agent state. See AgentState.
turnId	Int	Session round ID, used to identify a specific session round.
timestamp	TimeInterval	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).
reason	String	The reason for the status change.

Indicates a session interruption event.

It is usually triggered when the user actively interrupts the AI speech or the system detects a high-priority message. It is used to record the interruption behavior and perform corresponding processing.

_13

@objc public class InterruptEvent: NSObject {

_13

@objc public let turnId: Int

_13

@objc public let timestamp: TimeInterval

_13

_13

@objc public init(turnId: Int, timestamp: TimeInterval) {

_13

self.turnId = turnId

_13

self.timestamp = timestamp

_13

}

_13

_13

public override var description: String {

_13

return "InterruptEvent(turnId: \(turnId), timestamp: \(timestamp))"

_13

}

_13

}

Parameter	Type	Description
turnId	Int	The ID of the interrupted session round.
timestamp	TimeInterval	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Used to record and transmit system performance data.

For example, LLM inference delay, TTS synthesis delay, etc. This data can be used for performance monitoring, system optimization, and user experience improvement.

_12

@objc public class Metric: NSObject {

_12

@objc public let type: ModuleType

_12

@objc public let name: String

_12

@objc public let value: Double

_12

@objc public let timestamp: TimeInterval

_12

_12

@objc public init(type: ModuleType, name: String, value: Double, timestamp: TimeInterval)

_12

_12

public override var description: String {

_12

return "Metric(type: \(type.stringValue), name: \(name), value: \(value), timestamp: \(timestamp))"

_12

}

_12

}

Parameter	Type	Description
type	ModuleType	Indicator type. See ModuleType.
name	String	The indicator name of a specific performance item.
value	Double	Metric value, usually latency (milliseconds) or other quantitative metrics.
timestamp	TimeInterval	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Used to process and report AI module related error information.

Contains error type, error code, error description, and timestamp to facilitate error monitoring, logging, and troubleshooting.

_17

@objc public class ModuleError: NSObject {

_17

@objc public let type: ModuleType

_17

@objc public let code: Int

_17

@objc public let message: String

_17

@objc public let timestamp: TimeInterval

_17

_17

@objc public init(type: ModuleType, code: Int, message: String, timestamp: TimeInterval) {

_17

self.type = type

_17

self.code = code

_17

self.message = message

_17

self.timestamp = timestamp

_17

}

_17

_17

public override var description: String {

_17

return "ModuleError(type: \(type.stringValue), code: \(code), message: \(message), timestamp: \(timestamp))"

_17

}

_17

}

Parameter	Type	Description
type	ModuleType	Error type. See ModuleType.
code	Int	Error codes that identify specific error conditions.
message	String	Provides an error description that details the error.
timestamp	TimeInterval	Timestamp in milliseconds since Unix epoch (January 1, 1970 UTC).

Used to represent a user-visible transcript message.

Complete data class for rendering transcripts at the UI level.

_7

@objc public class Transcript: NSObject {

_7

@objc public let turnId: Int

_7

@objc public let userId: String

_7

@objc public let text: String

_7

@objc public var status: TranscriptStatus

_7

@objc public var type: TranscriptType

_7

}

Parameter	Type	Description
turnId	Int	Unique identifier for the session turn.
userId	String	The user identifier associated with this transcript.
text	String	The actual transcript text content.
status	TranscriptStatus	The current status of the transcription. See TranscriptStatus.
type	TranscriptType	The current type of transcript (agent or user). See TranscriptType.

Conversational AI API initialization configuration class.

Contains the configuration parameters required for Conversational AI API initialization, including rtcEngine for audio and video communication, rtmEngine for message communication, and transcript rendering mode settings.

_6

@objc public class ConversationalAIAPIConfig: NSObject {

_6

@objc public weak var rtcEngine: AgoraRtcEngineKit?

_6

@objc public weak var rtmEngine: AgoraRtmClientKit?

_6

@objc public var renderMode: TranscriptRenderMode

_6

@objc public var enableLog: Bool

_6

}

Parameter	Type	Description
rtcEngine	AgoraRtcEngineKit?	Engine instance used for audio and video communication. See AgoraRtcEngineKit.
rtmEngine	AgoraRtmClientKit?	Client instance for real-time messaging. See AgoraRtmClientKit.
renderMode	TranscriptRenderMode	Transcript rendering mode. See TranscriptRenderMode.
enableLog	Bool	Whether to enable verbose logging: YES: Enable verbose logging. NO: (Default) Disable logging.

Class for logging and communicating error information.

_15

@objc public class ConversationalAIAPIError: NSObject {

_15

@objc public let type: ConversationalAIAPIErrorType

_15

@objc public let code: Int

_15

@objc public let message: String

_15

_15

@objc public init(type: ConversationalAIAPIErrorType, code: Int, message: String) {

_15

self.type = type

_15

self.code = code

_15

self.message = message

_15

}

_15

_15

public override var description: String {

_15

return "ConversationalAIAPIError(type: \(type), code: \(code), message: \(message))"

_15

}

_15

}

Property	Type	Description
type	ConversationalAIAPIErrorType	Error type. See ConversationalAIAPIErrorType.
code	Int	Error codes that identify specific error conditions.
message	String	Provides an error description that details the error.

Message type enumeration.

Used to distinguish different message types within a session.

Value	Description
text	(0): Text message type.
image	(1): Image message type.
unknown	(2): Unknown message type.

Controls the priority with which the agent processes messages.

Value	Description
interrupt	(0): High priority. The agent will immediately interrupt the current interaction and process the message. Suitable for urgent or time-sensitive messages.
append	(1): Medium priority. The agent will queue the message after the current interaction is completed, suitable for subsequent questions.
ignore	(2): Low priority. If the agent is currently interacting, the message will be discarded and only processed when idle, suitable for optional content.

Represents the different states of the agent during the dialogue.

Value	Description
idle	(0): Idle state, the agent is not actively processing.
silent	(1): Silent state, the agent remains silent but is ready to listen.
listening	(2): Listening state, the agent is actively listening to user input.
thinking	(3): Thinking state, the agent is processing user input and generating responses.
speaking	(4): Speaking state, the agent is speaking or outputting audio content.
unknown	(5): Unknown state, used for fallback processing of unrecognized states.

Represents different types of AI modules for performance monitoring.

Value	Description
llm	(0): Reasoning with large language models.
mllm	(1): Multimodal large language model reasoning.
tts	(2): Text-to-speech synthesis.
context	(3): Context module type.
unknown	(4): Unknown module type.

Used to distinguish different types of messages in the session system.

Value	Description
metrics	The indicator message type.
error	The error message type.
assistant	AI agent transcript message type.
user	User transcript message type.
interrupt	The interrupt message type.
state	Status message type.
imageInfo	Image information message type.
unknown	Unknown message type.

Transcript rendering mode.

Value	Description
words	(0): Word-by-word transcript rendering mode, updated every time a word is processed.
text	(1): Sentence-by-sentence transcript rendering mode, updated when the complete sentence is ready.

Distinguish the source type of the transcribed text.

By identifying whether the transcribed text comes from an agent or a user, it helps manage the conversation flow and interface presentation.

Value	Description
agent	Transcript generated by the agent. Contains the responses and statements of the agent assistant, and is used to render the agent's voice in the conversational interface.
user	Transcribed text from the user. Contains the text converted from the user's voice input, which is used to display the user's voice content in the conversation process.

Indicates the current state of the transcription in the conversation flow.

Used to track and manage the lifecycle status of transcripts.

Value	Description
inprogress	(0): Transcription in progress. This state is set while the transcript is being generated or played, indicating that the content is still being processed or streamed.
end	(1): Transcription completed. This status is set when text generation ends normally, indicating the natural end of the transcript segment.
interrupted	(2): Transcription interrupted. This state is set when text generation is stopped prematurely, which is applicable when it is interrupted by a higher priority message.

Used to distinguish different error types in conversational agent systems.

Value	Description
unknown	(0): Unknown error type.
rtcError	(2): RTC (real-time communication) related errors.
rtmError	(3): RTM (Real Time Messaging) related errors.