# Python (/en/api-reference/api-ref/server-sdk/python)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

Full API reference for the Agora Conversational AI Python SDK.

## Agora / AsyncAgora Client [#agora--asyncagora-client]

`Agora` (sync) and `AsyncAgora` (async) extend the Fern-generated base client with regional domain pool support and three authentication modes. Use `Agora` for synchronous applications and `AsyncAgora` for asyncio-based applications.

```python
from agora_agent import Agora, Area
client = Agora(
    area=Area.US,
    app_id='your-app-id',
    app_certificate='your-app-certificate',
)
```

See [Sync vs. Async](#sync-vs-async) to choose the right client for your application.

### Constructor [#constructor]

```python
Agora(
    area: Area,
    app_id: str = None,
    app_certificate: str = None,
    username: str = None,
    password: str = None,
    auth_token: str = None,
    headers: Dict[str, str] = None,
    timeout: float = 60,
    follow_redirects: bool = True,
    httpx_client: httpx.Client = None,
)
```

Provide either `app_id` + `app_certificate` for app-credentials mode, or `username` + `password` for Basic Auth.

| Parameter          | Type             | Required | Default | Description                                                                                    |
| ------------------ | ---------------- | -------- | ------- | ---------------------------------------------------------------------------------------------- |
| `area`             | [`Area`](#area)  | Yes      | —       | Region for API routing                                                                         |
| `app_id`           | `str`            | Yes\*    | —       | Agora App ID                                                                                   |
| `app_certificate`  | `str`            | Yes\*    | —       | Agora App Certificate (app-credentials mode). Keep this secret and never expose it client-side |
| `username`         | `str`            | Yes\*    | —       | Customer ID (Basic Auth mode)                                                                  |
| `password`         | `str`            | Yes\*    | —       | Customer Secret (Basic Auth mode)                                                              |
| `auth_token`       | `str`            | No       | —       | Pre-built `agora token=<value>` string                                                         |
| `headers`          | `Dict[str, str]` | No       | `None`  | Additional headers sent with every request                                                     |
| `timeout`          | `float`          | No       | `60`    | Request timeout in seconds                                                                     |
| `follow_redirects` | `bool`           | No       | `True`  | Whether to follow HTTP redirects                                                               |
| `httpx_client`     | `httpx.Client`   | No       | `None`  | Custom httpx client instance                                                                   |

\* Provide either `app_id` + `app_certificate`, or `username` + `password`.

Authentication mode is resolved from the parameters you provide:

| Parameters provided          | Resolved mode       |
| ---------------------------- | ------------------- |
| `app_id` + `app_certificate` | `"app-credentials"` |
| `auth_token`                 | `"token"`           |
| `username` + `password`      | `"basic"`           |

`AsyncAgora` has the same constructor signature, except `httpx_client` accepts `httpx.AsyncClient` instead of `httpx.Client`.

```python
from agora_agent import AsyncAgora, Area
client = AsyncAgora(
    area=Area.US,
    app_id='your-app-id',
    app_certificate='your-app-certificate',
)
```

See [Authentication](/en/api-reference/api-ref/conversational-ai/authentication) for details on each mode.

### Properties [#properties]

#### `pool` [#pool]

Access the underlying `Pool` object for advanced domain management.

```python
pool = client.pool
pool.get_area()  # Area.US
```

* **Returns:** `Pool`

### Methods [#methods]

The following methods are available in addition to the Fern-generated sub-client methods.

#### `next_region()` [#next_region]

Cycles to the next region prefix in the domain pool. Call this after a request failure to try a different regional endpoint. Synchronous on both `Agora` and `AsyncAgora`.

```python
client.next_region()
```

#### `select_best_domain()` [#select_best_domain]

Triggers DNS-based domain selection to find the fastest-responding domain suffix. Results are cached for 30 seconds.

```python
# Sync (Agora)
client.select_best_domain()

# Async (AsyncAgora) — requires await
await client.select_best_domain()
```

#### `get_current_url()` [#get_current_url]

Returns the full API URL currently in use as a `str`. Synchronous on both `Agora` and `AsyncAgora`.

```python
url = client.get_current_url()
# Example: 'https://api-us-west-1.agora.io/api/conversational-ai-agent'
```

### Sub-clients [#sub-clients]

Both `Agora` and `AsyncAgora` expose Fern-generated sub-clients for direct REST API access. You typically do not need these when using the [agentkit layer](#agent).

| Property               | Sync type            | Async type                | Description                      |
| ---------------------- | -------------------- | ------------------------- | -------------------------------- |
| `client.agents`        | `AgentsClient`       | `AsyncAgentsClient`       | Start, stop, list, update agents |
| `client.telephony`     | `TelephonyClient`    | `AsyncTelephonyClient`    | Telephony operations             |
| `client.phone_numbers` | `PhoneNumbersClient` | `AsyncPhoneNumbersClient` | Phone number management          |

Sub-clients are lazily initialized on first access. For most use cases, prefer the [`AgentSession`](#agentsession-asyncagentsession) API over calling `client.agents` directly.

For full method signatures and request parameters, see the [REST API reference](/en/api-reference/api-ref/conversational-ai/join).

## Agent [#agent]

`Agent` is an immutable configuration object. Each builder method returns a new `Agent` instance — the original is never modified. Define one `Agent` at startup and call `create_session()` on it for each user conversation.

```python
from agora_agent import Agent
```

### Constructor [#constructor-1]

```python
Agent(
    name: Optional[str] = None,
    instructions: Optional[str] = None,
    turn_detection: Optional[TurnDetectionConfig] = None,
    interruption: Optional[InterruptionConfig] = None,
    sal: Optional[SalConfig] = None,
    advanced_features: Optional[Dict[str, Any]] = None,
    parameters: Optional[SessionParams] = None,
    greeting: Optional[str] = None,
    failure_message: Optional[str] = None,
    max_history: Optional[int] = None,
    geofence: Optional[GeofenceConfig] = None,
    labels: Optional[Dict[str, str]] = None,
    rtc: Optional[RtcConfig] = None,
    filler_words: Optional[FillerWordsConfig] = None,
)
```

All parameters are optional. Use the [builder methods](#builder-methods-1) to set vendor configuration after construction.

| Parameter           | Type                            | Default | Description                                                |
| ------------------- | ------------------------------- | ------- | ---------------------------------------------------------- |
| `name`              | `Optional[str]`                 | `None`  | Agent name, used as the default session name               |
| `instructions`      | `Optional[str]`                 | `None`  | LLM system prompt                                          |
| `turn_detection`    | `Optional[TurnDetectionConfig]` | `None`  | Voice activity detection settings                          |
| `interruption`      | `Optional[InterruptionConfig]`  | `None`  | Unified interruption control configuration                 |
| `sal`               | `Optional[SalConfig]`           | `None`  | Selective Attention Locking configuration                  |
| `advanced_features` | `Optional[Dict[str, Any]]`      | `None`  | Advanced features, for example `{'enable_rtm': True}`      |
| `parameters`        | `Optional[SessionParams]`       | `None`  | Additional session parameters                              |
| `greeting`          | `Optional[str]`                 | `None`  | Auto-spoken greeting when agent joins                      |
| `failure_message`   | `Optional[str]`                 | `None`  | Message spoken when an LLM call fails                      |
| `max_history`       | `Optional[int]`                 | `None`  | Maximum conversation turns kept in LLM context             |
| `geofence`          | `Optional[GeofenceConfig]`      | `None`  | Regional access restriction                                |
| `labels`            | `Optional[Dict[str, str]]`      | `None`  | Custom key-value labels returned in notification callbacks |
| `rtc`               | `Optional[RtcConfig]`           | `None`  | RTC media encryption                                       |
| `filler_words`      | `Optional[FillerWordsConfig]`   | `None`  | Filler words played while waiting for the LLM response     |

### Builder methods [#builder-methods]

All builder methods return a **new** `Agent` instance. The original is never modified.

#### `with_llm(vendor)` [#with_llmvendor]

Sets the LLM vendor for the cascading flow. Pass an instance of [`OpenAI`](#openai-1), [`AzureOpenAI`](#azureopenai-1), [`Anthropic`](#anthropic-1), or [`Gemini`](#gemini-1).

```python
with_llm(vendor: BaseLLM) -> Agent
```

#### `with_tts(vendor)` [#with_ttsvendor]

Sets the TTS vendor. Records the vendor's `sample_rate` for [avatar](#avatar-vendors-1) validation.

```python
with_tts(vendor: BaseTTS) -> Agent
```

#### `with_stt(vendor)` [#with_sttvendor]

Sets the STT vendor. Pass an instance of any [STT vendor class](#stt-vendors-1).

```python
with_stt(vendor: BaseSTT) -> Agent
```

#### `with_mllm(vendor)` [#with_mllmvendor]

Sets the MLLM vendor for multimodal flow. Calling `with_mllm()` automatically sets `mllm.enable = True`. MLLM sessions do not require TTS, STT, or LLM vendors.

```python
with_mllm(vendor: BaseMLLM) -> Agent
```

#### `with_avatar(vendor)` [#with_avatarvendor]

Sets the avatar vendor. Raises `ValueError` if the TTS sample rate does not match the avatar's required rate.

```python
with_avatar(vendor: BaseAvatar) -> Agent
```

**Raises:** `ValueError` — if TTS sample rate does not match the avatar's `required_sample_rate`.

#### `with_turn_detection(config)` [#with_turn_detectionconfig]

Configures cascading-flow SOS/EOS voice activity detection. Use `config.start_of_speech` and `config.end_of_speech` for SOS/EOS detection. Use `with_interruption()` for interruption behavior and MLLM vendor `turn_detection` for MLLM turn detection.

```python
with_turn_detection(config: TurnDetectionConfig) -> Agent
```

#### `with_interruption(config)` [#with_interruptionconfig]

Configures unified interruption behavior using the top-level `interruption` object. Use this for `start_of_speech` and `keywords` interruption modes.

```python
with_interruption(config: InterruptionConfig) -> Agent
```

#### `with_instructions(instructions)` [#with_instructionsinstructions]

Overrides the LLM system prompt on a new `Agent` instance.

```python
with_instructions(instructions: str) -> Agent
```

#### `with_greeting(greeting)` [#with_greetinggreeting]

Overrides the greeting message on a new `Agent` instance.

```python
with_greeting(greeting: str) -> Agent
```

#### `with_name(name)` [#with_namename]

Overrides the agent name on a new `Agent` instance.

```python
with_name(name: str) -> Agent
```

#### Other builder methods [#other-builder-methods]

The following methods follow the same pattern — each returns a new `Agent` instance with the updated configuration.

| Method                             | Parameter type      | Description                                   |
| ---------------------------------- | ------------------- | --------------------------------------------- |
| `with_sal(config)`                 | `SalConfig`         | Set Selective Attention Locking configuration |
| `with_advanced_features(features)` | `Dict[str, Any]`    | Set advanced features                         |
| `with_tools(enabled)`              | `bool`              | Enable or disable MCP tool invocation         |
| `with_parameters(parameters)`      | `SessionParams`     | Set session parameters                        |
| `with_failure_message(message)`    | `str`               | Set the message spoken when the LLM fails     |
| `with_max_history(n)`              | `int`               | Set the maximum conversation history length   |
| `with_geofence(geofence)`          | `GeofenceConfig`    | Set geofence configuration                    |
| `with_labels(labels)`              | `Dict[str, str]`    | Set custom labels                             |
| `with_rtc(rtc)`                    | `RtcConfig`         | Set RTC configuration                         |
| `with_filler_words(filler_words)`  | `FillerWordsConfig` | Set filler words configuration                |

### `create_session()` [#create_session]

Creates an [`AgentSession`](#agentsession-asyncagentsession) bound to a specific client and channel. Does not start the agent — call [`session.start()`](#start-1) to join the channel.

```python
create_session(
    client: Agora | AsyncAgora,
    channel: str,
    agent_uid: str,
    remote_uids: List[str],
    name: Optional[str] = None,
    token: Optional[str] = None,
    idle_timeout: Optional[int] = None,
    enable_string_uid: Optional[bool] = None,
    expires_in: Optional[int] = None,
) -> AgentSession
```

`create_session()` fields:

| Parameter           | Type                    | Required | Description                                                                                                                                                                  |
| ------------------- | ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `client`            | `Agora` or `AsyncAgora` | Yes      | Authenticated client                                                                                                                                                         |
| `channel`           | `str`                   | Yes      | Channel name to join                                                                                                                                                         |
| `agent_uid`         | `str`                   | Yes      | The agent's RTC UID                                                                                                                                                          |
| `remote_uids`       | `List[str]`             | Yes      | Remote user UIDs the agent listens and responds to                                                                                                                           |
| `name`              | `Optional[str]`         | No       | Session name. Defaults to agent name                                                                                                                                         |
| `token`             | `Optional[str]`         | No       | Pre-built RTC+RTM token. Omit to auto-generate from app credentials                                                                                                          |
| `expires_in`        | `Optional[int]`         | No       | Token lifetime in seconds. Only applies when the token is auto-generated. Valid range: 1–86400. Use [`expires_in_hours()`](#expires_in_hours-expires_in_minutes) for clarity |
| `idle_timeout`      | `Optional[int]`         | No       | Seconds before the agent auto-exits when no audio is detected                                                                                                                |
| `enable_string_uid` | `Optional[bool]`        | No       | Use string UIDs instead of numeric UIDs                                                                                                                                      |

### `to_properties()` [#to_properties]

Converts the agent configuration into a `StartAgentsRequestProperties` object for the Agora API. Called internally by `AgentSession.start()`.

```python
to_properties(
    channel: str,
    agent_uid: str,
    remote_uids: List[str],
    idle_timeout: Optional[int] = None,
    enable_string_uid: Optional[bool] = None,
    token: Optional[str] = None,
    app_id: Optional[str] = None,
    app_certificate: Optional[str] = None,
    expires_in: Optional[int] = None,
) -> StartAgentsRequestProperties
```

**Raises:** `ValueError` if neither `token` nor `app_id`+`app_certificate` is provided, or if required vendors (LLM, TTS) are missing in cascading mode.

### Properties [#properties-1]

Read-only properties available on any `Agent` instance.

| Property            | Type                            | Description                                |
| ------------------- | ------------------------------- | ------------------------------------------ |
| `name`              | `Optional[str]`                 | Agent name                                 |
| `instructions`      | `Optional[str]`                 | LLM system prompt                          |
| `greeting`          | `Optional[str]`                 | Greeting message                           |
| `failure_message`   | `Optional[str]`                 | Message spoken when LLM fails              |
| `max_history`       | `Optional[int]`                 | Maximum conversation history length        |
| `llm`               | `Optional[Dict[str, Any]]`      | LLM config dict (from `to_config()`)       |
| `tts`               | `Optional[Dict[str, Any]]`      | TTS config dict                            |
| `stt`               | `Optional[Dict[str, Any]]`      | STT config dict                            |
| `mllm`              | `Optional[Dict[str, Any]]`      | MLLM config dict                           |
| `avatar`            | `Optional[Dict[str, Any]]`      | Avatar config dict                         |
| `turn_detection`    | `Optional[TurnDetectionConfig]` | Turn detection configuration               |
| `interruption`      | `Optional[InterruptionConfig]`  | Unified interruption control configuration |
| `sal`               | `Optional[SalConfig]`           | SAL configuration                          |
| `advanced_features` | `Optional[Dict[str, Any]]`      | Advanced features                          |
| `parameters`        | `Optional[SessionParams]`       | Session parameters                         |
| `geofence`          | `Optional[GeofenceConfig]`      | Geofence configuration                     |
| `labels`            | `Optional[Dict[str, str]]`      | Custom labels                              |
| `rtc`               | `Optional[RtcConfig]`           | RTC configuration                          |
| `filler_words`      | `Optional[FillerWordsConfig]`   | Filler words configuration                 |
| `config`            | `Dict[str, Any]`                | Full configuration snapshot                |

### Type aliases [#type-aliases]

Public aliases over Fern-generated types: `LlmConfig`, `SttConfig`, `AsrConfig` (= `SttConfig`), `MllmConfig`, `AvatarConfig`, session/conversation types, and think types (`ThinkOnListeningAction`, etc.).

Think value constants: `ThinkOnListeningActionInject`, `ThinkOnListeningActionInterrupt`, `ThinkOnListeningActionIgnore`, `ThinkOnThinkingActionInterrupt`, `ThinkOnThinkingActionIgnore`, `ThinkOnSpeakingActionInterrupt`, `ThinkOnSpeakingActionIgnore`.

## AgentSession / AsyncAgentSession [#agentsession--asyncagentsession]

`AgentSession` (sync) and `AsyncAgentSession` (async) manage the full lifecycle of a running agent. Obtain a session by calling [`agent.create_session()`](#create_session) — direct construction is available for advanced use cases.

```python
from agora_agent import AgentSession
from agora_agent import AsyncAgentSession
# or from top-level:
from agora_agent import AgentSession, AsyncAgentSession
```

### Constructor [#constructor-2]

Sessions are normally created via `Agent.create_session()`. Direct construction is available for advanced use:

```python
AgentSession(
    client: Any,
    agent: Agent,
    app_id: str,
    name: str,
    channel: str,
    agent_uid: str,
    remote_uids: List[str],
    app_certificate: Optional[str] = None,
    token: Optional[str] = None,
    idle_timeout: Optional[int] = None,
    enable_string_uid: Optional[bool] = None,
)
```

`AsyncAgentSession` has the same constructor signature.

| Parameter           | Type                    | Required | Description                                 |
| ------------------- | ----------------------- | -------- | ------------------------------------------- |
| `client`            | `Agora` or `AsyncAgora` | Yes      | Authenticated client                        |
| `agent`             | `Agent`                 | Yes      | Agent configuration                         |
| `app_id`            | `str`                   | Yes      | Agora App ID                                |
| `name`              | `str`                   | Yes      | Session name                                |
| `channel`           | `str`                   | Yes      | Channel name                                |
| `agent_uid`         | `str`                   | Yes      | UID for the agent                           |
| `remote_uids`       | `List[str]`             | Yes      | UIDs of remote participants                 |
| `app_certificate`   | `Optional[str]`         | No       | App Certificate (for auto token generation) |
| `token`             | `Optional[str]`         | No       | Pre-built RTC token                         |
| `idle_timeout`      | `Optional[int]`         | No       | Idle timeout in seconds                     |
| `enable_string_uid` | `Optional[bool]`        | No       | Enable string UIDs                          |

### Methods [#methods-1]

The following methods are available on both `AgentSession` and `AsyncAgentSession`. Methods that make API calls require `await` on `AsyncAgentSession`.

#### `start()` [#start]

Starts the agent session. Generates an RTC token if not provided, validates avatar/TTS config for cascading sessions, and calls the Agora API. MLLM sessions do not require TTS; an enabled avatar is rejected when MLLM is configured (a disabled avatar is allowed).

|               | Sync (`AgentSession`)                                                                  | Async (`AsyncAgentSession`) |
| ------------- | -------------------------------------------------------------------------------------- | --------------------------- |
| **Signature** | `start() -> str`                                                                       | `async start() -> str`      |
| **Returns**   | Agent ID                                                                               | Agent ID                    |
| **Raises**    | `RuntimeError` if not in `idle`, `stopped`, or `error` state                           | Same                        |
| **Raises**    | `ValueError` if avatar/TTS sample rate mismatch or an enabled avatar is used with MLLM | Same                        |

```python
# Sync
agent_id = session.start()

# Async
agent_id = await session.start()
```

#### `stop()` [#stop]

Stops the agent session and removes the agent from the channel. If the agent has already stopped (404 from API), transitions to `stopped` without raising.

|               | Sync                                     | Async                  |
| ------------- | ---------------------------------------- | ---------------------- |
| **Signature** | `stop() -> None`                         | `async stop() -> None` |
| **Raises**    | `RuntimeError` if not in `running` state | Same                   |

```python
# Sync
session.stop()

# Async
await session.stop()
```

#### `say(text, priority=None, interruptable=None)` [#saytext-prioritynone-interruptablenone]

Instructs the agent to speak the given text.

|               | Sync                                                                                           | Async             |
| ------------- | ---------------------------------------------------------------------------------------------- | ----------------- |
| **Signature** | `say(text: str, priority: Optional[str] = None, interruptable: Optional[bool] = None) -> None` | Same with `async` |
| **Raises**    | `RuntimeError` if not in `running` state                                                       | Same              |

| Parameter       | Type   | Required | Description                            |
| --------------- | ------ | -------- | -------------------------------------- |
| `text`          | `str`  | Yes      | Text to speak                          |
| `priority`      | `str`  | No       | `INTERRUPT`, `APPEND`, or `IGNORE`     |
| `interruptable` | `bool` | No       | Whether the message can be interrupted |

```python
# Sync
session.say('One moment while I look that up.', priority='INTERRUPT', interruptable=False)

# Async
await session.say('One moment while I look that up.', priority='INTERRUPT', interruptable=False)
```

#### `interrupt()` [#interrupt]

Interrupts the agent while speaking or thinking.

|               | Sync                                     | Async                       |
| ------------- | ---------------------------------------- | --------------------------- |
| **Signature** | `interrupt() -> None`                    | `async interrupt() -> None` |
| **Raises**    | `RuntimeError` if not in `running` state | Same                        |

```python
# Sync
session.interrupt()

# Async
await session.interrupt()
```

#### `update(properties)` [#updateproperties]

Updates the agent configuration mid-session without restarting. Accepts a partial properties object in REST API format.

|               | Sync                                     | Async                                   |
| ------------- | ---------------------------------------- | --------------------------------------- |
| **Signature** | `update(properties: Any) -> None`        | `async update(properties: Any) -> None` |
| **Raises**    | `RuntimeError` if not in `running` state | Same                                    |

```python
from agora_agent.agents.types import UpdateAgentsRequestProperties

# Sync
session.update(properties)

# Async
await session.update(properties)
```

### `think(text, ...)` [#thinktext-]

Injects a custom text instruction into the running agent.

In API v2.7, omitting `on_listening_action` uses the server default `interrupt`. Pass `on_listening_action='inject'` explicitly to preserve the pre-v2.7 behavior.

```python
session.think('Summarize the last answer', on_listening_action='inject')
```

#### `get_history()` [#get_history]

Fetches the conversation history for this session. Requires a valid agent ID — `start()` must have been called successfully.

|               | Sync                          | Async                        |
| ------------- | ----------------------------- | ---------------------------- |
| **Signature** | `get_history() -> Any`        | `async get_history() -> Any` |
| **Raises**    | `RuntimeError` if no agent ID | Same                         |

```python
# Sync
history = session.get_history()

# Async
history = await session.get_history()
```

### `get_turns(page_index=None, page_size=None)` [#get_turnspage_indexnone-page_sizenone]

Retrieves paginated turn analytics for a completed or running session. In v2.7, the API defaults to page 1 and up to 50 turns per page. Responses include `agent_id`, `name`, `channel`, `total_turn_count`, `pagination`, and `turns`.

```python
page = session.get_turns(page_index=1, page_size=50)
```

### `get_all_turns(page_size=None)` [#get_all_turnspage_sizenone]

Fetches all turn pages and returns a single `GetTurnsAgentsResponse` with the combined `turns` list.

```python
all_turns = session.get_all_turns(page_size=50)
```

#### `get_info()` [#get_info]

Fetches current agent metadata from the API. Requires a valid agent ID.

|               | Sync                          | Async                     |
| ------------- | ----------------------------- | ------------------------- |
| **Signature** | `get_info() -> Any`           | `async get_info() -> Any` |
| **Raises**    | `RuntimeError` if no agent ID | Same                      |

```python
# Sync
info = session.get_info()

# Async
info = await session.get_info()
```

#### `on(event, handler)` [#onevent-handler]

Registers an event handler. Synchronous on both `AgentSession` and `AsyncAgentSession`. Register handlers **before** calling `start()` to avoid missing the `started` event.

```python
session.on('started', lambda data: print(f'Started: {data}'))
```

| Parameter | Type                  | Description                                  |
| --------- | --------------------- | -------------------------------------------- |
| `event`   | `str`                 | Event type: `started`, `stopped`, or `error` |
| `handler` | `Callable[..., None]` | Callback function                            |

#### `off(event, handler)` [#offevent-handler]

Removes a previously registered event handler. Synchronous on both `AgentSession` and `AsyncAgentSession`.

```python
session.off('started', my_handler)
```

### Properties [#properties-2]

The following read-only properties are available on both `AgentSession` and `AsyncAgentSession` instances.

| Property | Type                                  | Description                                                                                      |
| -------- | ------------------------------------- | ------------------------------------------------------------------------------------------------ |
| `id`     | `Optional[str]`                       | Agent ID, populated after `start()` resolves                                                     |
| `status` | `str`                                 | Current session state: `'idle'`, `'starting'`, `'running'`, `'stopping'`, `'stopped'`, `'error'` |
| `agent`  | `Agent`                               | The agent configuration this session was created from                                            |
| `app_id` | `str`                                 | The Agora App ID for this session                                                                |
| `raw`    | `AgentsClient` or `AsyncAgentsClient` | Direct access to the Fern-generated agents client for advanced operations                        |

### State transitions [#state-transitions]

| Current state | Allowed actions                                                             |
| ------------- | --------------------------------------------------------------------------- |
| `idle`        | `start()`                                                                   |
| `starting`    | (waiting for API)                                                           |
| `running`     | `stop()`, `say()`, `interrupt()`, `update()`, `get_history()`, `get_info()` |
| `stopping`    | (waiting for API)                                                           |
| `stopped`     | `start()` (restart)                                                         |
| `error`       | `start()` (retry)                                                           |

## Vendors [#vendors]

All vendor classes are imported from `agora_agent.agentkit.vendors`.

```python
from agora_agent import OpenAI, ElevenLabsTTS, DeepgramTTS, DeepgramSTT, OpenAIRealtime, XaiGrok, GenericAvatar
```

### LLM vendors [#llm-vendors]

Use with [`with_llm()`](#with_llmvendor).

<CalloutContainer type="info">
  <CalloutDescription>
    `greeting_configs` accepts either a dict or `LlmGreetingConfigs`. In v2.7, `greeting_configs.interruptable=False` makes the greeting uninterruptible; `True` follows the global `interruption` settings.
  </CalloutDescription>
</CalloutContainer>

#### OpenAI [#openai]

```python
from agora_agent import OpenAI

llm = OpenAI(api_key='your-key', model='gpt-4o-mini', temperature=0.7)
```

| Parameter            | Type             | Required | Default | Description                                                                |
| -------------------- | ---------------- | -------- | ------- | -------------------------------------------------------------------------- |
| `api_key`            | `str`            | No       | `None`  | OpenAI API key. Omit to use Agora-managed credentials for supported models |
| `model`              | `str`            | Yes      | —       | Model name                                                                 |
| `base_url`           | `str`            | No       | `None`  | Custom base URL. Only valid when `api_key` is set                          |
| `temperature`        | `float`          | No       | `None`  | Sampling temperature (0.0–2.0)                                             |
| `top_p`              | `float`          | No       | `None`  | Nucleus sampling (0.0–1.0)                                                 |
| `max_tokens`         | `int`            | No       | `None`  | Maximum tokens to generate                                                 |
| `system_messages`    | `List[Dict]`     | No       | `None`  | Additional system messages                                                 |
| `greeting_message`   | `str`            | No       | `None`  | Agent greeting message                                                     |
| `failure_message`    | `str`            | No       | `None`  | Message spoken when the LLM call fails                                     |
| `max_history`        | `int`            | No       | `None`  | Maximum number of conversation history messages to cache                   |
| `input_modalities`   | `List[str]`      | No       | `None`  | Input modalities                                                           |
| `output_modalities`  | `List[str]`      | No       | `None`  | Output modalities                                                          |
| `greeting_configs`   | `Dict[str, Any]` | No       | `None`  | Greeting configuration                                                     |
| `template_variables` | `Dict[str, str]` | No       | `None`  | Template variables for system prompt interpolation                         |
| `headers`            | `Dict[str, str]` | No       | `None`  | Custom HTTP headers forwarded to the LLM provider                          |
| `params`             | `Dict[str, Any]` | No       | `None`  | Additional model parameters                                                |
| `vendor`             | `str`            | No       | `None`  | Vendor override                                                            |
| `mcp_servers`        | `List[Dict]`     | No       | `None`  | MCP server connections                                                     |

#### AzureOpenAI [#azureopenai]

```python
from agora_agent import AzureOpenAI

llm = AzureOpenAI(
    api_key='your-azure-key',
    model='gpt-4o-mini',
    endpoint='https://your-resource.openai.azure.com',
    deployment_name='gpt-4o-mini',
)
```

| Parameter            | Type             | Required | Default                | Description                                              |
| -------------------- | ---------------- | -------- | ---------------------- | -------------------------------------------------------- |
| `api_key`            | `str`            | Yes      | —                      | Azure OpenAI API key                                     |
| `model`              | `str`            | Yes      | —                      | Azure deployment model name                              |
| `endpoint`           | `str`            | Yes      | —                      | Azure endpoint URL                                       |
| `deployment_name`    | `str`            | Yes      | —                      | Azure deployment name                                    |
| `api_version`        | `str`            | No       | `'2024-08-01-preview'` | Azure API version                                        |
| `temperature`        | `float`          | No       | `None`                 | Sampling temperature (0.0–2.0)                           |
| `top_p`              | `float`          | No       | `None`                 | Nucleus sampling (0.0–1.0)                               |
| `max_tokens`         | `int`            | No       | `None`                 | Maximum tokens to generate                               |
| `system_messages`    | `List[Dict]`     | No       | `None`                 | Additional system messages                               |
| `greeting_message`   | `str`            | No       | `None`                 | Agent greeting message                                   |
| `failure_message`    | `str`            | No       | `None`                 | Message spoken when the LLM call fails                   |
| `max_history`        | `int`            | No       | `None`                 | Maximum number of conversation history messages to cache |
| `input_modalities`   | `List[str]`      | No       | `None`                 | Input modalities                                         |
| `output_modalities`  | `List[str]`      | No       | `None`                 | Output modalities                                        |
| `params`             | `Dict[str, Any]` | No       | `None`                 | Additional model parameters                              |
| `headers`            | `Dict[str, str]` | No       | `None`                 | Custom HTTP headers forwarded to the LLM provider        |
| `greeting_configs`   | `Dict[str, Any]` | No       | `None`                 | Greeting configuration                                   |
| `template_variables` | `Dict[str, str]` | No       | `None`                 | Template variables for system prompt interpolation       |
| `vendor`             | `str`            | No       | `None`                 | Vendor override                                          |
| `mcp_servers`        | `List[Dict]`     | No       | `None`                 | MCP server connections                                   |

#### Anthropic [#anthropic]

```python
from agora_agent import Anthropic

llm = Anthropic(
    api_key='your-anthropic-key',
    url='https://api.anthropic.com/v1/messages',
    model='claude-opus-4-8',
    max_tokens=1024,
    headers={'anthropic-version': '2023-06-01'},
)
```

| Parameter            | Type             | Required | Default | Description                                              |
| -------------------- | ---------------- | -------- | ------- | -------------------------------------------------------- |
| `api_key`            | `str`            | Yes      | —       | Anthropic API key                                        |
| `model`              | `str`            | Yes      | —       | Model name                                               |
| `url`                | `str`            | Yes      | —       | Anthropic messages endpoint URL                          |
| `max_tokens`         | `int`            | Yes      | —       | Maximum tokens to generate                               |
| `headers`            | `Dict[str, str]` | Yes      | —       | Request headers, including `anthropic-version`           |
| `temperature`        | `float`          | No       | `None`  | Sampling temperature (0.0–1.0)                           |
| `top_p`              | `float`          | No       | `None`  | Nucleus sampling (0.0–1.0)                               |
| `system_messages`    | `List[Dict]`     | No       | `None`  | Additional system messages                               |
| `greeting_message`   | `str`            | No       | `None`  | Agent greeting message                                   |
| `failure_message`    | `str`            | No       | `None`  | Message spoken when the LLM call fails                   |
| `max_history`        | `int`            | No       | `None`  | Maximum number of conversation history messages to cache |
| `input_modalities`   | `List[str]`      | No       | `None`  | Input modalities                                         |
| `output_modalities`  | `List[str]`      | No       | `None`  | Output modalities                                        |
| `params`             | `Dict[str, Any]` | No       | `None`  | Additional model parameters                              |
| `greeting_configs`   | `Dict[str, Any]` | No       | `None`  | Greeting configuration                                   |
| `template_variables` | `Dict[str, str]` | No       | `None`  | Template variables for system prompt interpolation       |
| `vendor`             | `str`            | No       | `None`  | Vendor override                                          |
| `mcp_servers`        | `List[Dict]`     | No       | `None`  | MCP server connections                                   |

#### Gemini [#gemini]

```python
from agora_agent import Gemini

llm = Gemini(api_key='your-google-key', model='gemini-2.0-flash-exp')
```

| Parameter            | Type             | Required | Default | Description                                              |
| -------------------- | ---------------- | -------- | ------- | -------------------------------------------------------- |
| `api_key`            | `str`            | Yes      | —       | Google AI API key                                        |
| `model`              | `str`            | Yes      | —       | Model name                                               |
| `url`                | `str`            | No       | `None`  | Custom API endpoint URL                                  |
| `temperature`        | `float`          | No       | `None`  | Sampling temperature (0.0–2.0)                           |
| `top_p`              | `float`          | No       | `None`  | Nucleus sampling (0.0–1.0)                               |
| `top_k`              | `int`            | No       | `None`  | Top-k sampling                                           |
| `max_output_tokens`  | `int`            | No       | `None`  | Maximum output tokens                                    |
| `system_messages`    | `List[Dict]`     | No       | `None`  | Additional system messages                               |
| `greeting_message`   | `str`            | No       | `None`  | Agent greeting message                                   |
| `failure_message`    | `str`            | No       | `None`  | Message spoken when the LLM call fails                   |
| `max_history`        | `int`            | No       | `None`  | Maximum number of conversation history messages to cache |
| `input_modalities`   | `List[str]`      | No       | `None`  | Input modalities                                         |
| `output_modalities`  | `List[str]`      | No       | `None`  | Output modalities                                        |
| `params`             | `Dict[str, Any]` | No       | `None`  | Additional model parameters                              |
| `headers`            | `Dict[str, str]` | No       | `None`  | Custom HTTP headers forwarded to the LLM provider        |
| `greeting_configs`   | `Dict[str, Any]` | No       | `None`  | Greeting configuration                                   |
| `template_variables` | `Dict[str, str]` | No       | `None`  | Template variables for system prompt interpolation       |
| `vendor`             | `str`            | No       | `None`  | Vendor override                                          |
| `mcp_servers`        | `List[Dict]`     | No       | `None`  | MCP server connections                                   |

#### Other LLM vendors [#other-llm-vendors]

The SDK also includes named helpers for the remaining Agora-supported LLM providers. These helpers choose the correct request format internally.

| Class           | Provider              | Key parameters                                         |
| --------------- | --------------------- | ------------------------------------------------------ |
| `Groq`          | Groq                  | `api_key`, `model`, `base_url`                         |
| `VertexAILLM`   | Google Vertex AI      | `api_key`, `model`, `project_id`, `location`, `url?`   |
| `AmazonBedrock` | Amazon Bedrock        | `access_key`, `secret_key`, `region`, `model`, `url?`  |
| `Dify`          | Dify                  | `api_key`, `url`, `model`, `user?`, `conversation_id?` |
| `CustomLLM`     | OpenAI-compatible LLM | `api_key`, `base_url`, `model`                         |

These helpers also accept the common LLM fields (`system_messages`, `greeting_message`, `failure_message`, `max_history`, `params`, `headers`, etc.). `Groq` and `CustomLLM` extend `OpenAI`; `VertexAILLM` extends `Gemini`.

### TTS vendors [#tts-vendors]

Use with [`with_tts()`](#with_ttsvendor). The `sample_rate` option determines avatar compatibility — see [`with_avatar()`](#with_avatarvendor).

#### ElevenLabsTTS [#elevenlabstts]

```python
from agora_agent.agentkit.vendors import ElevenLabsTTS
```

| Parameter                    | Type        | Required | Default | Description                                      |
| ---------------------------- | ----------- | -------- | ------- | ------------------------------------------------ |
| `key`                        | `str`       | Yes      | —       | ElevenLabs API key                               |
| `model_id`                   | `str`       | Yes      | —       | Model ID, for example `'eleven_flash_v2_5'`      |
| `voice_id`                   | `str`       | Yes      | —       | Voice ID                                         |
| `base_url`                   | `str`       | Yes      | —       | WebSocket base URL                               |
| `sample_rate`                | `int`       | No       | `None`  | Sample rate in Hz: 16000, 22050, 24000, or 44100 |
| `skip_patterns`              | `List[int]` | No       | `None`  | Skip patterns for bracketed content              |
| `optimize_streaming_latency` | `int`       | No       | `None`  | Latency optimization level (0–4)                 |
| `stability`                  | `float`     | No       | `None`  | Voice stability (0.0–1.0)                        |
| `similarity_boost`           | `float`     | No       | `None`  | Similarity boost (0.0–1.0)                       |
| `style`                      | `float`     | No       | `None`  | Style exaggeration (0.0–1.0)                     |
| `use_speaker_boost`          | `bool`      | No       | `None`  | Enable speaker boost                             |

#### MicrosoftTTS [#microsofttts]

```python
from agora_agent.agentkit.vendors import MicrosoftTTS
```

| Parameter           | Type             | Required | Default | Description                                     |
| ------------------- | ---------------- | -------- | ------- | ----------------------------------------------- |
| `key`               | `str`            | Yes      | —       | Azure subscription key                          |
| `region`            | `str`            | Yes      | —       | Azure region, for example `'eastus'`            |
| `voice_name`        | `str`            | Yes      | —       | Voice name, for example `'en-US-JennyNeural'`   |
| `sample_rate`       | `int`            | No       | `None`  | Sample rate in Hz: 8000, 16000, 24000, or 48000 |
| `speed`             | `float`          | No       | `None`  | Speaking rate multiplier                        |
| `volume`            | `float`          | No       | `None`  | Audio volume                                    |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional Microsoft TTS parameters             |
| `skip_patterns`     | `List[int]`      | No       | `None`  | Skip patterns for bracketed content             |

#### OpenAITTS [#openaitts]

Fixed sample rate: 24000 Hz.

```python
from agora_agent.agentkit.vendors import OpenAITTS
```

| Parameter       | Type        | Required | Default | Description                                                                                 |
| --------------- | ----------- | -------- | ------- | ------------------------------------------------------------------------------------------- |
| `api_key`       | `str`       | No       | `None`  | OpenAI API key. Omit to use Agora-managed credentials for [supported models](/en/ai/models) |
| `voice`         | `str`       | Yes      | —       | Voice name: `'alloy'`, `'echo'`, `'fable'`, `'onyx'`, `'nova'`, or `'shimmer'`              |
| `model`         | `str`       | No       | `None`  | Model name. Required (with `api_key` and `base_url`) for BYOK                               |
| `base_url`      | `str`       | No       | `None`  | Endpoint URL. Required (with `api_key` and `model`) for BYOK                                |
| `instructions`  | `str`       | No       | `None`  | Custom voice instructions                                                                   |
| `speed`         | `float`     | No       | `None`  | Speech speed multiplier                                                                     |
| `skip_patterns` | `List[int]` | No       | `None`  | Skip patterns for bracketed content                                                         |

#### CartesiaTTS [#cartesiatts]

```python
from agora_agent.agentkit.vendors import CartesiaTTS
```

| Parameter       | Type        | Required | Default | Description                                              |
| --------------- | ----------- | -------- | ------- | -------------------------------------------------------- |
| `api_key`       | `str`       | Yes      | —       | Cartesia API key                                         |
| `voice_id`      | `str`       | Yes      | —       | Voice ID (serialized to the `voice` `{mode, id}` object) |
| `model_id`      | `str`       | Yes      | —       | Model ID                                                 |
| `base_url`      | `str`       | No       | `None`  | WebSocket URL                                            |
| `language`      | `str`       | No       | `None`  | Target language                                          |
| `sample_rate`   | `int`       | No       | `None`  | Sample rate in Hz: 8000–48000                            |
| `skip_patterns` | `List[int]` | No       | `None`  | Skip patterns for bracketed content                      |

#### GoogleTTS [#googletts]

```python
from agora_agent.agentkit.vendors import GoogleTTS
```

| Parameter           | Type        | Required | Default | Description                                                                        |
| ------------------- | ----------- | -------- | ------- | ---------------------------------------------------------------------------------- |
| `key`               | `str`       | Yes      | —       | Google Cloud service account credentials JSON string (serialized to `credentials`) |
| `voice_name`        | `str`       | Yes      | —       | Voice name                                                                         |
| `language_code`     | `str`       | No       | `None`  | Language code, for example `'en-US'`                                               |
| `sample_rate_hertz` | `int`       | No       | `None`  | Sample rate in Hz                                                                  |
| `skip_patterns`     | `List[int]` | No       | `None`  | Skip patterns for bracketed content                                                |

#### AmazonTTS [#amazontts]

```python
from agora_agent.agentkit.vendors import AmazonTTS
```

| Parameter       | Type        | Required | Default | Description                                                      |
| --------------- | ----------- | -------- | ------- | ---------------------------------------------------------------- |
| `access_key`    | `str`       | Yes      | —       | AWS access key                                                   |
| `secret_key`    | `str`       | Yes      | —       | AWS secret key                                                   |
| `region`        | `str`       | Yes      | —       | AWS region, for example `'us-east-1'`                            |
| `voice_id`      | `str`       | Yes      | —       | Amazon Polly voice ID                                            |
| `engine`        | `str`       | Yes      | —       | Polly engine: `standard`, `neural`, `long-form`, or `generative` |
| `skip_patterns` | `List[int]` | No       | `None`  | Skip patterns for bracketed content                              |

#### HumeAITTS [#humeaitts]

```python
from agora_agent.agentkit.vendors import HumeAITTS
```

| Parameter          | Type        | Required | Default | Description                                      |
| ------------------ | ----------- | -------- | ------- | ------------------------------------------------ |
| `key`              | `str`       | Yes      | —       | Hume AI API key                                  |
| `voice_id`         | `str`       | Yes      | —       | Hume AI voice ID                                 |
| `provider`         | `str`       | Yes      | —       | Voice provider type: `HUME_AI` or `CUSTOM_VOICE` |
| `config_id`        | `str`       | No       | `None`  | Configuration ID                                 |
| `base_url`         | `str`       | No       | `None`  | Base URL                                         |
| `speed`            | `float`     | No       | `None`  | Playback speed                                   |
| `trailing_silence` | `float`     | No       | `None`  | Trailing silence in seconds                      |
| `skip_patterns`    | `List[int]` | No       | `None`  | Skip patterns for bracketed content              |

#### RimeTTS [#rimetts]

```python
from agora_agent.agentkit.vendors import RimeTTS
```

| Parameter       | Type        | Required | Default | Description                         |
| --------------- | ----------- | -------- | ------- | ----------------------------------- |
| `key`           | `str`       | Yes      | —       | Rime API key                        |
| `speaker`       | `str`       | Yes      | —       | Speaker ID                          |
| `model_id`      | `str`       | Yes      | —       | Model ID                            |
| `base_url`      | `str`       | No       | `None`  | WebSocket URL                       |
| `skip_patterns` | `List[int]` | No       | `None`  | Skip patterns for bracketed content |

#### FishAudioTTS [#fishaudiotts]

```python
from agora_agent.agentkit.vendors import FishAudioTTS
```

| Parameter       | Type        | Required | Default | Description                                       |
| --------------- | ----------- | -------- | ------- | ------------------------------------------------- |
| `key`           | `str`       | Yes      | —       | Fish Audio API key                                |
| `reference_id`  | `str`       | Yes      | —       | Reference ID                                      |
| `backend`       | `str`       | Yes      | —       | Backend model version, for example `'speech-1.5'` |
| `skip_patterns` | `List[int]` | No       | `None`  | Skip patterns for bracketed content               |

#### MiniMaxTTS [#minimaxtts]

```python
from agora_agent.agentkit.vendors import MiniMaxTTS
```

| Parameter           | Type             | Required | Default | Description                                                       |
| ------------------- | ---------------- | -------- | ------- | ----------------------------------------------------------------- |
| `model`             | `str`            | Yes      | —       | Model name, for example `'speech-02-turbo'`                       |
| `key`               | `str`            | No       | `None`  | MiniMax API key. Required for BYOK; omit for preset-backed models |
| `group_id`          | `str`            | No       | `None`  | MiniMax group ID. Required for BYOK                               |
| `voice_id`          | `str`            | No       | `None`  | Voice style identifier (provide `voice_id` or `timber_weights`)   |
| `url`               | `str`            | No       | `None`  | WebSocket endpoint (BYOK)                                         |
| `speed`             | `float`          | No       | `None`  | Speaking speed                                                    |
| `vol`               | `float`          | No       | `None`  | Volume gain                                                       |
| `pitch`             | `float`          | No       | `None`  | Pitch adjustment                                                  |
| `emotion`           | `str`            | No       | `None`  | Emotion style                                                     |
| `sample_rate`       | `int`            | No       | `None`  | Output sample rate in Hz                                          |
| `language_boost`    | `str`            | No       | `None`  | Language boost strategy                                           |
| `timber_weights`    | `List[Dict]`     | No       | `None`  | Alternative timbre mix config                                     |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional MiniMax TTS parameters                                 |
| `skip_patterns`     | `List[int]`      | No       | `None`  | Skip patterns for bracketed content                               |

#### DeepgramTTS [#deepgramtts]

```python
from agora_agent.agentkit.vendors import DeepgramTTS
```

| Parameter           | Type             | Required | Default | Description                                                                   |
| ------------------- | ---------------- | -------- | ------- | ----------------------------------------------------------------------------- |
| `api_key`           | `str`            | Yes      | —       | Deepgram API key                                                              |
| `model`             | `str`            | Yes      | —       | Model name, for example `'aura-2-thalia-en'`                                  |
| `base_url`          | `str`            | No       | `None`  | WebSocket endpoint. Defaults server-side to `wss://api.deepgram.com/v1/speak` |
| `sample_rate`       | `int`            | No       | `None`  | Sample rate in Hz                                                             |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional Deepgram TTS parameters                                            |
| `skip_patterns`     | `List[int]`      | No       | `None`  | Skip patterns for bracketed content                                           |

#### MurfTTS [#murftts]

```python
from agora_agent.agentkit.vendors import MurfTTS
```

| Parameter       | Type        | Required | Default | Description                                     |
| --------------- | ----------- | -------- | ------- | ----------------------------------------------- |
| `key`           | `str`       | Yes      | —       | Murf API key                                    |
| `voice_id`      | `str`       | No       | `None`  | Voice ID, for example `'Ariana'` or `'Natalie'` |
| `base_url`      | `str`       | No       | `None`  | WebSocket endpoint                              |
| `locale`        | `str`       | No       | `None`  | Voice locale                                    |
| `rate`          | `float`     | No       | `None`  | Speech rate                                     |
| `pitch`         | `float`     | No       | `None`  | Pitch adjustment                                |
| `model`         | `str`       | No       | `None`  | TTS model                                       |
| `sample_rate`   | `int`       | No       | `None`  | Audio sample rate                               |
| `skip_patterns` | `List[int]` | No       | `None`  | Skip patterns for bracketed content             |

#### SarvamTTS [#sarvamtts]

```python
from agora_agent.agentkit.vendors import SarvamTTS
```

| Parameter              | Type        | Required | Default | Description                         |
| ---------------------- | ----------- | -------- | ------- | ----------------------------------- |
| `key`                  | `str`       | Yes      | —       | Sarvam API key                      |
| `speaker`              | `str`       | Yes      | —       | Speaker name                        |
| `target_language_code` | `str`       | Yes      | —       | Target language code                |
| `pitch`                | `float`     | No       | `None`  | Pitch adjustment                    |
| `pace`                 | `float`     | No       | `None`  | Speed of speech                     |
| `loudness`             | `float`     | No       | `None`  | Volume level                        |
| `sample_rate`          | `int`       | No       | `None`  | Audio sample rate in Hz             |
| `skip_patterns`        | `List[int]` | No       | `None`  | Skip patterns for bracketed content |

### STT vendors [#stt-vendors]

Use with [`with_stt()`](#with_sttvendor).

#### DeepgramSTT [#deepgramstt]

All parameters are optional.

```python
from agora_agent.agentkit.vendors import DeepgramSTT
```

| Parameter           | Type             | Required | Default | Description                                                                                   |
| ------------------- | ---------------- | -------- | ------- | --------------------------------------------------------------------------------------------- |
| `api_key`           | `str`            | No       | `None`  | Deepgram API key. Omit to use Agora-managed credentials for [supported models](/en/ai/models) |
| `model`             | `str`            | No       | `None`  | Model name, for example `'nova-2'`                                                            |
| `language`          | `str`            | No       | `None`  | Language code, for example `'en-US'`                                                          |
| `keyterm`           | `str`            | No       | `None`  | Boost specialized terms and brands                                                            |
| `smart_format`      | `bool`           | No       | `None`  | Enable smart formatting                                                                       |
| `punctuation`       | `bool`           | No       | `None`  | Enable punctuation                                                                            |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional vendor parameters                                                                  |

#### SpeechmaticsSTT [#speechmaticsstt]

```python
from agora_agent.agentkit.vendors import SpeechmaticsSTT
```

| Parameter           | Type             | Required | Default | Description                          |
| ------------------- | ---------------- | -------- | ------- | ------------------------------------ |
| `api_key`           | `str`            | Yes      | —       | Speechmatics API key                 |
| `language`          | `str`            | Yes      | —       | Language code, for example `'en'`    |
| `model`             | `str`            | No       | `None`  | Model name                           |
| `uri`               | `str`            | No       | `None`  | Speechmatics streaming WebSocket URL |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional parameters                |

#### MicrosoftSTT [#microsoftstt]

```python
from agora_agent.agentkit.vendors import MicrosoftSTT
```

| Parameter           | Type             | Required | Default | Description                          |
| ------------------- | ---------------- | -------- | ------- | ------------------------------------ |
| `key`               | `str`            | Yes      | —       | Azure subscription key               |
| `region`            | `str`            | Yes      | —       | Azure region, for example `'eastus'` |
| `language`          | `str`            | Yes      | —       | Language code, for example `'en-US'` |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional parameters                |

#### OpenAISTT [#openaistt]

```python
from agora_agent.agentkit.vendors import OpenAISTT
```

| Parameter                   | Type             | Required | Default | Description                                                   |
| --------------------------- | ---------------- | -------- | ------- | ------------------------------------------------------------- |
| `api_key`                   | `str`            | Yes      | —       | OpenAI API key                                                |
| `model`                     | `str`            | No       | `None`  | Transcription model. Default: `'gpt-4o-mini-transcribe'`      |
| `language`                  | `str`            | No       | `None`  | Language code                                                 |
| `prompt`                    | `str`            | No       | `None`  | Prompt that guides transcription                              |
| `input_audio_transcription` | `Dict[str, Any]` | No       | `None`  | OpenAI transcription settings (`model`, `prompt`, `language`) |
| `additional_params`         | `Dict[str, Any]` | No       | `None`  | Additional parameters                                         |

The serialized configuration requires a transcription `prompt` and `language` — provide them through the `prompt` and `language` parameters or within `input_audio_transcription`. `model` defaults to `gpt-4o-mini-transcribe`.

#### GoogleSTT [#googlestt]

```python
from agora_agent.agentkit.vendors import GoogleSTT
```

| Parameter                | Type             | Required | Default | Description                                    |
| ------------------------ | ---------------- | -------- | ------- | ---------------------------------------------- |
| `project_id`             | `str`            | Yes      | —       | Google Cloud project ID                        |
| `location`               | `str`            | Yes      | —       | Google Cloud region                            |
| `adc_credentials_string` | `str`            | Yes      | —       | Google service account credentials JSON string |
| `language`               | `str`            | Yes      | —       | Language code, for example `'en-US'`           |
| `model`                  | `str`            | No       | `None`  | Recognition model                              |
| `additional_params`      | `Dict[str, Any]` | No       | `None`  | Additional parameters                          |

#### AmazonSTT [#amazonstt]

```python
from agora_agent.agentkit.vendors import AmazonSTT
```

| Parameter           | Type             | Required | Default | Description                           |
| ------------------- | ---------------- | -------- | ------- | ------------------------------------- |
| `access_key`        | `str`            | Yes      | —       | AWS access key ID                     |
| `secret_key`        | `str`            | Yes      | —       | AWS secret access key                 |
| `region`            | `str`            | Yes      | —       | AWS region, for example `'us-east-1'` |
| `language`          | `str`            | Yes      | —       | Language code                         |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional parameters                 |

#### AssemblyAISTT [#assemblyaistt]

```python
from agora_agent.agentkit.vendors import AssemblyAISTT
```

| Parameter           | Type             | Required | Default | Description                        |
| ------------------- | ---------------- | -------- | ------- | ---------------------------------- |
| `api_key`           | `str`            | Yes      | —       | AssemblyAI API key                 |
| `language`          | `str`            | Yes      | —       | Language code                      |
| `uri`               | `str`            | No       | `None`  | AssemblyAI streaming WebSocket URL |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional parameters              |

#### AresSTT [#aresstt]

```python
from agora_agent.agentkit.vendors import AresSTT
```

| Parameter           | Type             | Required | Default | Description           |
| ------------------- | ---------------- | -------- | ------- | --------------------- |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional parameters |

#### SarvamSTT [#sarvamstt]

```python
from agora_agent.agentkit.vendors import SarvamSTT
```

| Parameter           | Type             | Required | Default | Description                                 |
| ------------------- | ---------------- | -------- | ------- | ------------------------------------------- |
| `api_key`           | `str`            | Yes      | —       | Sarvam API key                              |
| `language`          | `str`            | Yes      | —       | Language code, for example `'en'` or `'hi'` |
| `model`             | `str`            | No       | `None`  | Model name                                  |
| `additional_params` | `Dict[str, Any]` | No       | `None`  | Additional parameters                       |

### MLLM vendors [#mllm-vendors]

Use with [`with_mllm()`](#with_mllmvendor) for multimodal end-to-end audio processing without separate STT or TTS steps. Calling `with_mllm()` automatically enables the MLLM module (sets `mllm.enable=true`); the older `advanced_features.enable_mllm` flag is deprecated.

#### OpenAIRealtime [#openairealtime]

```python
from agora_agent.agentkit.vendors import OpenAIRealtime
```

| Parameter                   | Type                      | Required | Default | Description                                                             |
| --------------------------- | ------------------------- | -------- | ------- | ----------------------------------------------------------------------- |
| `api_key`                   | `str`                     | Yes      | —       | OpenAI API key                                                          |
| `model`                     | `str`                     | No       | `None`  | Model name, for example `'gpt-4o-realtime-preview'`                     |
| `voice`                     | `str`                     | No       | `None`  | Voice identifier                                                        |
| `instructions`              | `str`                     | No       | `None`  | System instructions                                                     |
| `input_audio_transcription` | `Dict[str, Any]`          | No       | `None`  | Audio transcription settings                                            |
| `url`                       | `str`                     | No       | `None`  | Custom WebSocket URL                                                    |
| `greeting_message`          | `str`                     | No       | `None`  | Agent greeting message                                                  |
| `failure_message`           | `str`                     | No       | `None`  | Message played when the model call fails                                |
| `input_modalities`          | `List[str]`               | No       | `None`  | Input modalities, for example `['audio']`                               |
| `output_modalities`         | `List[str]`               | No       | `None`  | Output modalities, for example `['text', 'audio']`                      |
| `messages`                  | `List[Dict]`              | No       | `None`  | Conversation messages for short-term memory                             |
| `params`                    | `Dict[str, Any]`          | No       | `None`  | Additional parameters                                                   |
| `turn_detection`            | `MllmTurnDetectionConfig` | No       | `None`  | MLLM turn detection configuration; overrides top-level `turn_detection` |

#### GeminiLive [#geminilive]

```python
from agora_agent.agentkit.vendors import GeminiLive
```

| Parameter           | Type                      | Required | Default | Description                                                             |
| ------------------- | ------------------------- | -------- | ------- | ----------------------------------------------------------------------- |
| `api_key`           | `str`                     | Yes      | —       | Google Gemini API key                                                   |
| `model`             | `str`                     | Yes      | —       | Gemini Live model name                                                  |
| `url`               | `str`                     | No       | `None`  | Custom WebSocket URL                                                    |
| `instructions`      | `str`                     | No       | `None`  | System instructions                                                     |
| `voice`             | `str`                     | No       | `None`  | Voice name                                                              |
| `affective_dialog`  | `bool`                    | No       | `None`  | Enable affective dialog                                                 |
| `proactive_audio`   | `bool`                    | No       | `None`  | Enable proactive audio                                                  |
| `transcribe_agent`  | `bool`                    | No       | `None`  | Transcribe agent speech                                                 |
| `transcribe_user`   | `bool`                    | No       | `None`  | Transcribe user speech                                                  |
| `http_options`      | `Dict[str, Any]`          | No       | `None`  | HTTP options                                                            |
| `greeting_message`  | `str`                     | No       | `None`  | Agent greeting message                                                  |
| `failure_message`   | `str`                     | No       | `None`  | Message played when the model call fails                                |
| `input_modalities`  | `List[str]`               | No       | `None`  | Input modalities                                                        |
| `output_modalities` | `List[str]`               | No       | `None`  | Output modalities                                                       |
| `messages`          | `List[Dict]`              | No       | `None`  | Conversation messages for short-term memory                             |
| `additional_params` | `Dict[str, Any]`          | No       | `None`  | Additional parameters                                                   |
| `turn_detection`    | `MllmTurnDetectionConfig` | No       | `None`  | MLLM turn detection configuration; overrides top-level `turn_detection` |

#### VertexAI [#vertexai]

```python
from agora_agent.agentkit.vendors import VertexAI
```

| Parameter                | Type                      | Required | Default | Description                                                             |
| ------------------------ | ------------------------- | -------- | ------- | ----------------------------------------------------------------------- |
| `model`                  | `str`                     | Yes      | —       | Model name, for example `'gemini-2.0-flash-exp'`                        |
| `project_id`             | `str`                     | Yes      | —       | Google Cloud project ID                                                 |
| `location`               | `str`                     | Yes      | —       | Google Cloud location, for example `'us-central1'`                      |
| `adc_credentials_string` | `str`                     | Yes      | —       | Application Default Credentials JSON string                             |
| `url`                    | `str`                     | No       | `None`  | Custom WebSocket URL                                                    |
| `instructions`           | `str`                     | No       | `None`  | System instructions for the model                                       |
| `voice`                  | `str`                     | No       | `None`  | Voice name, for example `'Aoede'` or `'Charon'`                         |
| `affective_dialog`       | `bool`                    | No       | `None`  | Enable affective dialog                                                 |
| `proactive_audio`        | `bool`                    | No       | `None`  | Enable proactive audio                                                  |
| `transcribe_agent`       | `bool`                    | No       | `None`  | Transcribe agent speech                                                 |
| `transcribe_user`        | `bool`                    | No       | `None`  | Transcribe user speech                                                  |
| `http_options`           | `Dict[str, Any]`          | No       | `None`  | HTTP options                                                            |
| `greeting_message`       | `str`                     | No       | `None`  | Agent greeting message                                                  |
| `failure_message`        | `str`                     | No       | `None`  | Message played when the model call fails                                |
| `input_modalities`       | `List[str]`               | No       | `None`  | Input modalities                                                        |
| `output_modalities`      | `List[str]`               | No       | `None`  | Output modalities                                                       |
| `messages`               | `List[Dict]`              | No       | `None`  | Conversation messages for short-term memory                             |
| `additional_params`      | `Dict[str, Any]`          | No       | `None`  | Additional parameters                                                   |
| `turn_detection`         | `MllmTurnDetectionConfig` | No       | `None`  | MLLM turn detection configuration; overrides top-level `turn_detection` |

#### XaiGrok [#xaigrok]

[xAI Grok](/en/ai/models/mllm/xai) MLLM vendor (`mllm.vendor`: `"xai"`).

| Parameter           | Type                      | Required | Default                      | Description                                   |
| ------------------- | ------------------------- | -------- | ---------------------------- | --------------------------------------------- |
| `api_key`           | `str`                     | Yes      | —                            | xAI API key                                   |
| `url`               | `str`                     | No       | `wss://api.x.ai/v1/realtime` | xAI Realtime WebSocket URL                    |
| `voice`             | `str`                     | No       | `None`                       | Voice identifier, for example `eve` or `rex`  |
| `language`          | `str`                     | No       | `None`                       | Language code, for example `en`               |
| `sample_rate`       | `int`                     | No       | `None`                       | Audio sample rate in Hz                       |
| `greeting_message`  | `str`                     | No       | `None`                       | Greeting message                              |
| `failure_message`   | `str`                     | No       | `None`                       | Message played when the model call fails      |
| `input_modalities`  | `List[str]`               | No       | `None`                       | Input modalities                              |
| `output_modalities` | `List[str]`               | No       | `None`                       | Output modalities                             |
| `messages`          | `List[Dict]`              | No       | `None`                       | Conversation messages                         |
| `params`            | `Dict[str, Any]`          | No       | `None`                       | Additional xAI parameters                     |
| `turn_detection`    | `MllmTurnDetectionConfig` | No       | `None`                       | Supports `agora_vad` and `server_vad` for xAI |

### Avatar vendors [#avatar-vendors]

Use with [`with_avatar()`](#with_avatarvendor). Each avatar vendor requires a specific TTS sample rate enforced at runtime.

#### HeyGenAvatar [#heygenavatar]

**Deprecated — renamed to [`LiveAvatarAvatar`](#liveavataravatar).** `HeyGenAvatar` still works (serializes `vendor: "heygen"`) but emits a deprecation warning. Use `LiveAvatarAvatar` for new code.

Requires TTS at **24,000 Hz**.

```python
from agora_agent.agentkit.vendors import HeyGenAvatar
```

| Parameter               | Type   | Required | Default | Description                                                                                                              |
| ----------------------- | ------ | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------ |
| `api_key`               | `str`  | Yes      | —       | HeyGen API key                                                                                                           |
| `quality`               | `str`  | Yes      | —       | Video quality: `'low'`, `'medium'`, or `'high'`                                                                          |
| `agora_uid`             | `str`  | Yes      | —       | Agora UID for the avatar video stream                                                                                    |
| `agora_token`           | `str`  | No       | `None`  | Avatar token. When omitted, `AgentSession.start()` generates one for `agora_uid` using the same token path as the agent. |
| `avatar_id`             | `str`  | No       | `None`  | HeyGen avatar ID                                                                                                         |
| `enable`                | `bool` | No       | `True`  | Enable or disable the avatar                                                                                             |
| `disable_idle_timeout`  | `bool` | No       | `None`  | Disable the idle timeout                                                                                                 |
| `activity_idle_timeout` | `int`  | No       | `None`  | Idle timeout in seconds. Default: 120                                                                                    |

#### AkoolAvatar [#akoolavatar]

Requires TTS at **16,000 Hz**.

```python
from agora_agent.agentkit.vendors import AkoolAvatar
```

| Parameter   | Type   | Required | Default | Description                  |
| ----------- | ------ | -------- | ------- | ---------------------------- |
| `api_key`   | `str`  | Yes      | —       | Akool API key                |
| `avatar_id` | `str`  | No       | `None`  | Avatar ID                    |
| `enable`    | `bool` | No       | `True`  | Enable or disable the avatar |

#### LiveAvatarAvatar [#liveavataravatar]

Required TTS sample rate: **24000 Hz**

Same options as [`HeyGenAvatar`](#heygenavatar), but serializes `vendor: "liveavatar"`. `agora_token` is optional and generated by `AgentSession.start()` when omitted.

#### AnamAvatar [#anamavatar]

```python
from agora_agent.agentkit.vendors import AnamAvatar
```

| Parameter    | Type   | Required | Default | Description                  |
| ------------ | ------ | -------- | ------- | ---------------------------- |
| `api_key`    | `str`  | Yes      | —       | Anam API key                 |
| `persona_id` | `str`  | No       | `None`  | Persona ID                   |
| `enable`     | `bool` | No       | `True`  | Enable or disable the avatar |

#### GenericAvatar [#genericavatar]

```python
from agora_agent.agentkit.vendors import GenericAvatar
```

| Parameter       | Type   | Required | Default | Description                                                              |
| --------------- | ------ | -------- | ------- | ------------------------------------------------------------------------ |
| `api_key`       | `str`  | Yes      | —       | Generic avatar provider API key                                          |
| `agora_uid`     | `str`  | Yes      | —       | Avatar RTC UID. Must differ from the agent UID.                          |
| `api_base_url`  | `str`  | Yes      | —       | Avatar provider API base URL                                             |
| `avatar_id`     | `str`  | Yes      | —       | Avatar ID                                                                |
| `agora_token`   | `str`  | No       | `None`  | Optional avatar token. Generated by `AgentSession.start()` when omitted. |
| `agora_appid`   | `str`  | No       | `None`  | Optional; filled from the session App ID when omitted.                   |
| `agora_channel` | `str`  | No       | `None`  | Optional; filled from the session channel when omitted.                  |
| `enable`        | `bool` | No       | `True`  | Enable or disable the avatar                                             |

Avatar tokens are separate from the agent join token but generated with the same `generate_convo_ai_token` path, using the avatar's `agora_uid` as `uid`.

## Token utilities [#token-utilities]

Helper functions for generating and managing tokens. Use these when you need control over token lifetime or when generating tokens outside of a session.

```python
from agora_agent.agentkit.token import generate_convo_ai_token, generate_rtc_token
from agora_agent.agentkit import expires_in_hours, expires_in_minutes
```

### `generate_convo_ai_token()` [#generate_convo_ai_token]

Generates a Conversational AI token combining RTC and RTM privileges. This is the same token the SDK generates automatically in app-credentials mode. Use this when passing a pre-built token to [`create_session()`](#create_session).

```python
generate_convo_ai_token(
    app_id: str,
    app_certificate: str,
    channel_name: str,
    account: str,
    token_expire: int = 86400,
    privilege_expire: int = 0,
) -> str
```

| Parameter          | Type  | Required | Default | Description                                                       |
| ------------------ | ----- | -------- | ------- | ----------------------------------------------------------------- |
| `app_id`           | `str` | Yes      | —       | Agora App ID                                                      |
| `app_certificate`  | `str` | Yes      | —       | Agora App Certificate                                             |
| `channel_name`     | `str` | Yes      | —       | The channel the token grants access to                            |
| `account`          | `str` | Yes      | —       | The UID this token is issued for, as a string                     |
| `token_expire`     | `int` | No       | `86400` | Token lifetime in seconds. Valid range: 1–86400                   |
| `privilege_expire` | `int` | No       | `0`     | Seconds until privileges expire. `0` means same as `token_expire` |

**Returns:** `str` — the generated token.

```python
from agora_agent.agentkit.token import generate_convo_ai_token
from agora_agent.agentkit import expires_in_hours

token = generate_convo_ai_token(
    app_id='your-app-id',
    app_certificate='your-app-certificate',
    channel_name='support-room-123',
    account='1',
    token_expire=expires_in_hours(12),
)
```

### `generate_rtc_token()` [#generate_rtc_token]

Generates an RTC-only token for channel join. Use `generate_convo_ai_token()` instead for most Conversational AI use cases.

```python
generate_rtc_token(
    app_id: str,
    app_certificate: str,
    channel: str,
    uid: int,
    role: int = 1,
    expiry_seconds: int = 86400,
) -> str
```

| Parameter         | Type  | Required | Default | Description                                     |
| ----------------- | ----- | -------- | ------- | ----------------------------------------------- |
| `app_id`          | `str` | Yes      | —       | Agora App ID                                    |
| `app_certificate` | `str` | Yes      | —       | Agora App Certificate                           |
| `channel`         | `str` | Yes      | —       | Channel name                                    |
| `uid`             | `int` | Yes      | —       | User ID. Use `0` for any user                   |
| `role`            | `int` | No       | `1`     | RTC role: `1` for publisher, `2` for subscriber |
| `expiry_seconds`  | `int` | No       | `86400` | Token lifetime in seconds. Valid range: 1–86400 |

**Returns:** `str` — the generated token.

### `expires_in_hours()` / `expires_in_minutes()` [#expires_in_hours--expires_in_minutes]

Helper functions for specifying token lifetimes. Use with [`create_session()`](#create_session) or token generation functions. Values are validated and capped at the Agora maximum of 86400 seconds (24 hours).

```python
expires_in_hours(hours: int) -> int
expires_in_minutes(minutes: int) -> int
```

| Function                | Returns         | Behavior                                                                       |
| ----------------------- | --------------- | ------------------------------------------------------------------------------ |
| `expires_in_hours(n)`   | `int` — seconds | Raises `ValueError` if `n` ≤ 0. Warns and caps at 86400 if result exceeds 24 h |
| `expires_in_minutes(n)` | `int` — seconds | Raises `ValueError` if `n` ≤ 0. Warns and caps at 86400 if result exceeds 24 h |

```python
from agora_agent.agentkit import expires_in_hours, expires_in_minutes

session = agent.create_session(
    client,
    channel='support-room-123',
    agent_uid='1',
    remote_uids=['100'],
    expires_in=expires_in_hours(12),
)
```

## Types and enums [#types-and-enums]

Shared types and enums used across `Agora`, `AsyncAgora`, `Agent`, `AgentSession`, and vendor classes.

### `Area` [#area]

Region used for API routing. Pass to `Agora` or `AsyncAgora` via the `area` parameter.

```python
from agora_agent import Area
```

| Value     | Region         |
| --------- | -------------- |
| `Area.US` | United States  |
| `Area.EU` | Europe         |
| `Area.AP` | Asia-Pacific   |
| `Area.CN` | China mainland |

### `AgentSessionEvent` [#agentsessionevent]

Valid event names for `session.on()` and `session.off()`.

```python
'started' | 'stopped' | 'error'
```

| Value       | Payload                     | Description                           |
| ----------- | --------------------------- | ------------------------------------- |
| `'started'` | `dict` with `agent_id: str` | Agent successfully joined the channel |
| `'stopped'` | `dict` with `agent_id: str` | Agent left the channel                |
| `'error'`   | `Exception`                 | An unrecoverable error occurred       |

### `SpeakPriority` [#speakpriority]

Controls how the agent handles a [`say()`](#say-text-priority-none-interruptable-none) call relative to its current activity. Pass as a string to `session.say()`.

| Value         | Description                                                     |
| ------------- | --------------------------------------------------------------- |
| `'INTERRUPT'` | Agent immediately stops current speech and delivers the message |
| `'APPEND'`    | Message is queued and delivered after current speech ends       |
| `'IGNORE'`    | Message is discarded if the agent is currently speaking         |

### `ApiError` [#apierror]

Raised when the API returns a 4xx or 5xx response. Catch this to inspect the status code and response body.

```python
from agora_agent.core.api_error import ApiError

try:
    agent_id = session.start()
except ApiError as e:
    print(e.status_code)
    print(e.body)
```

| Property      | Type  | Description                          |
| ------------- | ----- | ------------------------------------ |
| `status_code` | `int` | HTTP status code returned by the API |
| `body`        | `Any` | Raw response body from the API       |

## Sync vs. Async [#sync-vs-async]

The Python SDK provides two parallel client and session hierarchies — synchronous and asynchronous. Choose based on your application's runtime model.

|              | Sync           | Async               |
| ------------ | -------------- | ------------------- |
| Client       | `Agora`        | `AsyncAgora`        |
| Session      | `AgentSession` | `AsyncAgentSession` |
| HTTP backend | `httpx.Client` | `httpx.AsyncClient` |

**Use `Agora` (sync) when:**

* You are writing scripts, CLI tools, or batch jobs
* Your web framework is synchronous, for example Flask or Django without async views
* You want the simplest possible code

**Use `AsyncAgora` (async) when:**

* Your application uses `asyncio`, for example FastAPI, Starlette, or aiohttp
* You need to manage multiple concurrent agent sessions efficiently
* You want non-blocking I/O

The `Agent` builder class is the same for both — it does not make HTTP calls, so it has no async variant. Pass an `AsyncAgora` client to `agent.create_session()` to receive an `AsyncAgentSession`.