Python

Full API reference for the Agora Conversational AI Python SDK.

Install the SDK from PyPI. The import package is agora_agent.

pip install agora-agents

Agora / AsyncAgora Client

Agora (sync) and AsyncAgora (async) extend the Fern-generated base client with regional domain pool support and three authentication modes. Use Agora for synchronous applications and AsyncAgora for asyncio-based applications.

from agora_agent import Agora, Area
client = Agora(
    area=Area.US,
    app_id='your-app-id',
    app_certificate='your-app-certificate',
)

See Sync vs. Async to choose the right client for your application.

Constructor

Agora(
    *,
    area: Area,
    app_id: str,
    app_certificate: str,
    customer_id: Optional[str] = None,
    customer_secret: Optional[str] = None,
    auth_token: Optional[str] = None,
    headers: Optional[Dict[str, str]] = None,
    timeout: Optional[float] = None,
    follow_redirects: Optional[bool] = True,
    httpx_client: Optional[httpx.Client] = None,
    debug: bool = False,
)

app_id and app_certificate are always required. Add customer_id and customer_secret for Basic Auth, or auth_token for token mode.

Authentication mode is resolved from the parameters you provide:

Passing customer_id without customer_secret, or combining auth_token with customer_id, raises ValueError.

AsyncAgora has the same constructor signature, except httpx_client accepts httpx.AsyncClient instead of httpx.Client.

from agora_agent import AsyncAgora, Area
client = AsyncAgora(
    area=Area.US,
    app_id='your-app-id',
    app_certificate='your-app-certificate',
)

See Authentication for details on each mode.

Properties

pool

Access the underlying Pool object for advanced domain management.

pool = client.pool
pool.get_area()  # Area.US

• Returns: Pool

area and area_scope

area returns the configured Area. area_scope returns "global" or "cn".

Methods

The following methods are available in addition to the Fern-generated sub-client methods.

next_region()

Cycles to the next region prefix in the domain pool. Call this after a request failure to try a different regional endpoint. Synchronous on both Agora and AsyncAgora.

client.next_region()

select_best_domain()

Triggers DNS-based domain selection to find the fastest-responding domain suffix. Results are cached for 30 seconds.

# Sync (Agora)
client.select_best_domain()

# Async (AsyncAgora) — requires await
await client.select_best_domain()

stop_agent(agent_id)

Stops an agent by ID without an AgentSession reference, for example, from an end-call handler. The method treats a 404 response as success. Synchronous on Agora, async on AsyncAgora.

# Sync (Agora)
client.stop_agent(agent_id)

# Async (AsyncAgora)
await client.stop_agent(agent_id)

get_current_url()

Returns the full API URL currently in use as a str. Synchronous on both Agora and AsyncAgora.

url = client.get_current_url()
# Example: 'https://api-us-west-1.agora.io/api/conversational-ai-agent'

Sub-clients

Both Agora and AsyncAgora expose Fern-generated sub-clients for direct REST API access. You typically do not need these when using the agentkit layer.

Sub-clients are lazily initialized on first access. For most use cases, prefer the AgentSession API over calling client.agents directly.

For full method signatures and request parameters, see the REST API reference.

Agent

Agent is an immutable configuration object. Each builder method returns a new Agent instance — the original is never modified. Define one Agent at startup and call create_session() on it for each user conversation.

from agora_agent import Agent

Constructor

Agent(
    client: Agora | AsyncAgora,
    instructions: Optional[str] = None,
    turn_detection: Optional[TurnDetectionConfig | dict] = None,
    interruption: Optional[InterruptionConfig] = None,
    sal: Optional[SalConfig] = None,
    advanced_features: Optional[Dict[str, Any]] = None,
    parameters: Optional[SessionParams | dict] = None,
    greeting: Optional[str] = None,
    failure_message: Optional[str] = None,
    max_history: Optional[int] = None,
    geofence: Optional[GeofenceConfig] = None,
    labels: Optional[Dict[str, str]] = None,
    rtc: Optional[RtcConfig] = None,
    filler_words: Optional[FillerWordsConfig] = None,
    greeting_configs: Optional[Dict[str, Any]] = None,
    pipeline_id: Optional[str] = None,
)

client is required. Omitting it raises TypeError. All other parameters are optional. Use the builder methods to set vendor configuration after construction.

Builder methods

All builder methods return a new Agent instance. The original is never modified.

with_llm(vendor)

Sets the LLM vendor for the cascading flow. Pass an instance of OpenAI, AzureOpenAI, Anthropic, or Gemini.

with_llm(vendor: BaseLLM) -> Agent

with_tts(vendor)

Sets the TTS vendor. Records the vendor's sample_rate for avatar validation. Raises ValueError if an avatar is already set and its required sample rate differs.

with_tts(vendor: BaseTTS) -> Agent

with_stt(vendor)

Sets the STT vendor. Pass an instance of any STT vendor class.

with_stt(vendor: BaseSTT) -> Agent

with_mllm(vendor)

Sets the MLLM vendor for multimodal flow. Pass an instance of any MLLM vendor class, including OpenAIGPTLive. Calling with_mllm() automatically sets mllm.enable = True. MLLM sessions do not require TTS, STT, or LLM vendors.

with_mllm(vendor: BaseMLLM) -> Agent

with_avatar(vendor)

Sets the avatar vendor. Raises ValueError if the TTS sample rate does not match the avatar's required rate.

with_avatar(vendor: BaseAvatar) -> Agent

Raises: ValueError — if TTS sample rate does not match the avatar's required_sample_rate.

with_turn_detection(config)

Configures cascading-flow SOS/EOS voice activity detection. Use config.start_of_speech and config.end_of_speech for SOS/EOS detection. Use with_interruption() for interruption behavior and MLLM vendor turn_detection for MLLM turn detection.

with_turn_detection(config: TurnDetectionConfig) -> Agent

with_interruption(config)

Configures unified interruption behavior using the top-level interruption object. Use this for start_of_speech and keywords interruption modes.

with_interruption(config: InterruptionConfig) -> Agent

with_instructions(instructions)

Overrides the LLM system prompt on a new Agent instance. Deprecated. Set system_messages on the LLM vendor instead.

with_instructions(instructions: str) -> Agent

with_greeting(greeting)

Overrides the greeting message on a new Agent instance. Deprecated. Set greeting_message on the LLM or MLLM vendor instead.

with_greeting(greeting: str) -> Agent

Other builder methods

The following methods follow the same pattern — each returns a new Agent instance with the updated configuration.

create_session()

Creates a sync AgentSession using the client bound to this Agent. Doesn't start the agent. Call session.start() to join the channel.

create_session(
    channel: str,
    agent_uid: str,
    remote_uids: List[str],
    name: Optional[str] = None,
    token: Optional[str] = None,
    idle_timeout: Optional[int] = None,
    enable_string_uid: Optional[bool] = None,
    preset: Optional[str | Sequence[str]] = None,
    pipeline_id: Optional[str] = None,
    expires_in: Optional[int] = None,
    debug: Optional[bool] = None,
    warn: Optional[Callable[[str], None]] = None,
) -> AgentSession

create_session() fields:

create_async_session()

Takes the same parameters as create_session() and returns an AsyncAgentSession. Use it with an Agent bound to an AsyncAgora client.

create_async_session(...) -> AsyncAgentSession

to_properties()

Converts the agent configuration into a StartAgentsRequestProperties object for the Agora API. Called internally by AgentSession.start().

to_properties(
    channel: str,
    agent_uid: str,
    remote_uids: List[str],
    idle_timeout: Optional[int] = None,
    enable_string_uid: Optional[bool] = None,
    token: Optional[str] = None,
    app_id: Optional[str] = None,
    app_certificate: Optional[str] = None,
    expires_in: Optional[int] = None,
    skip_vendor_validation: bool = False,
    skip_vendor_validation_categories: Optional[AbstractSet[str]] = None,
    allow_missing_vendor_categories: Optional[AbstractSet[str]] = None,
) -> StartAgentsRequestProperties

skip_vendor_validation is deprecated. Use skip_vendor_validation_categories and allow_missing_vendor_categories instead.

Raises: ValueError if neither token nor app_id+app_certificate is provided, if required vendors (LLM, TTS) are missing in cascading mode, if an enabled avatar is combined with MLLM, or if agent_uid isn't numeric when the SDK generates a token.

Properties

Read-only properties available on any Agent instance.

Type aliases

Public aliases over Fern-generated types: LlmConfig, SttConfig, AsrConfig (= SttConfig), MllmConfig, AvatarConfig, session/conversation types, and think types (ThinkOnListeningAction, etc.).

Think value constants: ThinkOnListeningActionInject, ThinkOnListeningActionInterrupt, ThinkOnListeningActionIgnore, ThinkOnThinkingActionInterrupt, ThinkOnThinkingActionIgnore, ThinkOnSpeakingActionInterrupt, ThinkOnSpeakingActionIgnore. The think action parameters also accept the string append, which has no convenience constant.

FillerWordsConfig supports static and generated filler words. In generated mode, generated_config, prompt, and the static fallback phrases are optional. If you omit generated_config, the service uses its default generation settings. Agora hosts the generation service, so you can't configure its model endpoint, API key, or model parameters. The only supported fallback strategy is static. For the field structure, see Configure generated filler words.

Public aliases also include FillerWordsContentGeneratedConfig and the custom tool types LlmToolConfig, LlmToolFunctionConfig, LlmToolFunctionParametersConfig, LlmToolExecutionConfig, and LlmToolServerConfig.

FillerWordsContentGeneratedConfig sets the conversation context for generated filler words. Use context_message_limit (1 to 6, defaults to 1) to set how many of the most recent conversation messages to use, counting the current turn's message, and history_character_limit (0 to 10000, defaults to 1000) to cap the combined characters of the earlier messages. For details, see Talking while waiting.

AgentSession / AsyncAgentSession

AgentSession (sync) and AsyncAgentSession (async) manage the full lifecycle of a running agent. Obtain a session with agent.create_session() (sync) or agent.create_async_session() (async). Direct construction is available for advanced use cases.

from agora_agent.agentkit import AgentSession, AsyncAgentSession

Constructor

Sessions are normally created via Agent.create_session(). Direct construction is available for advanced use:

AgentSession(
    client: Any,
    agent: Agent,
    app_id: str,
    name: str,
    channel: str,
    agent_uid: str,
    remote_uids: List[str],
    app_certificate: Optional[str] = None,
    token: Optional[str] = None,
    idle_timeout: Optional[int] = None,
    enable_string_uid: Optional[bool] = None,
    preset: Optional[str | Sequence[str]] = None,
    pipeline_id: Optional[str] = None,
    expires_in: Optional[int] = None,
    debug: Optional[bool] = None,
    warn: Optional[Callable[[str], None]] = None,
)

AsyncAgentSession has the same constructor signature.

Methods

The following methods are available on both AgentSession and AsyncAgentSession. Methods that make API calls require await on AsyncAgentSession.

start()

Starts the agent session. Generates an RTC token if not provided, validates avatar/TTS config for cascading sessions, and calls the Agora API. MLLM sessions do not require TTS; an enabled avatar is rejected when MLLM is configured (a disabled avatar is allowed).

# Sync
agent_id = session.start()

# Async
agent_id = await session.start()

stop()

Stops the agent session and removes the agent from the channel. If the agent has already stopped (404 from API), transitions to stopped without raising.

# Sync
session.stop()

# Async
await session.stop()

say(text, priority=None, interruptable=None)

Instructs the agent to speak the given text.

# Sync
session.say('One moment while I look that up.', priority='INTERRUPT', interruptable=False)

# Async
await session.say('One moment while I look that up.', priority='INTERRUPT', interruptable=False)

interrupt()

Interrupts the agent while speaking or thinking.

# Sync
session.interrupt()

# Async
await session.interrupt()

update(properties)

Updates the agent configuration mid-session without restarting. Accepts a partial properties object in REST API format.

from agora_agent.agentkit import AgentConfigUpdate

# Sync
session.update(properties)

# Async
await session.update(properties)

think(text, ...)

Injects a custom text instruction into the running agent.

In API v2.7, omitting on_listening_action uses the server default interrupt. Pass on_listening_action='inject' explicitly to preserve the pre-v2.7 behavior.

on_listening_action, on_thinking_action, and on_speaking_action all accept append. With append, the instruction doesn't interrupt the current interaction. The agent waits for the current user turn, LLM inference, or TTS playback to finish, then appends the instruction to the context as a separate user message and starts a new turn.

session.think(
    'Summarize the last answer',
    on_listening_action='append',
    on_thinking_action='append',
    on_speaking_action='append',
)

get_history()

Fetches the conversation history for this session. Requires a valid agent ID — start() must have been called successfully.

# Sync
history = session.get_history()

# Async
history = await session.get_history()

get_turns(*, page_index=None, page_size=None)

Retrieves paginated turn analytics for a completed or running session. Parameters are keyword-only. Raises RuntimeError if there's no agent ID. Requires await on AsyncAgentSession. In v2.7, the API defaults to page 1 and up to 50 turns per page. Responses include agent_id, name, channel, total_turn_count, pagination, and turns.

page = session.get_turns(page_index=1, page_size=50)

get_all_turns(*, page_size=None)

Fetches all turn pages and returns a single GetTurnsAgentsResponse with the combined turns list.

all_turns = session.get_all_turns(page_size=50)

get_info()

Fetches current agent metadata from the API. Requires a valid agent ID.

# Sync
info = session.get_info()

# Async
info = await session.get_info()

on(event, handler)

Registers an event handler. Synchronous on both AgentSession and AsyncAgentSession. Register handlers before calling start() to avoid missing the started event.

session.on('started', lambda data: print(f'Started: {data}'))

off(event, handler)

Removes a previously registered event handler. Synchronous on both AgentSession and AsyncAgentSession.

session.off('started', my_handler)

Properties

The following read-only properties are available on both AgentSession and AsyncAgentSession instances.

State transitions

get_history(), get_info(), get_turns(), and get_all_turns() work in any state after start() has returned an agent ID.

Vendors

Vendor classes are exported from both agora_agent and agora_agent.agentkit.vendors.

from agora_agent import OpenAI, ElevenLabsTTS, DeepgramTTS, DeepgramSTT, OpenAIRealtime, XaiGrok, GenericAvatar

LLM vendors

Use with with_llm().

OpenAI

from agora_agent import OpenAI

llm = OpenAI(
    api_key='your-key',
    base_url='https://api.openai.com/v1/chat/completions',
    model='gpt-4o-mini',
    temperature=0.7,
)

AzureOpenAI

from agora_agent import AzureOpenAI

llm = AzureOpenAI(
    api_key='your-azure-key',
    model='gpt-4o-mini',
    endpoint='https://your-resource.openai.azure.com',
    deployment_name='gpt-4o-mini',
)

Anthropic

from agora_agent import Anthropic

llm = Anthropic(
    api_key='your-anthropic-key',
    url='https://api.anthropic.com/v1/messages',
    model='claude-opus-4-8',
    max_tokens=1024,
    headers={'anthropic-version': '2023-06-01'},
)

Gemini

from agora_agent import Gemini

llm = Gemini(api_key='your-google-key', model='gemini-2.0-flash-exp')

Other LLM vendors

The SDK also includes named helpers for the remaining Agora-supported LLM providers. These helpers choose the correct request format internally.

These helpers also accept common LLM fields, such as system_messages, greeting_message, failure_message, max_history, params, and headers. Groq and CustomLLM use the OpenAI request style, and VertexAILLM uses the Gemini request style.

The LLM vendor template_variables parameter declares fixed string values known when you create the agent. Custom tools can reference these values in server.url, server.headers, or server.body with {{template_variables.<name>}}. The model and engine supply {{args.<name>}} and {{tool_call_id}} at runtime, so you don't need to declare them in template_variables.

The LLM vendor tools parameter declares synchronous custom tools. Each tool includes a function definition and the server request configuration. The SDK enables tool calling for tools and mcp_servers only after you call with_tools() or with_tools(True). For more information, see Call custom tools.

TTS vendors

Use with with_tts(). The sample_rate option determines avatar compatibility — see with_avatar().

ElevenLabsTTS

from agora_agent.agentkit.vendors import ElevenLabsTTS

MicrosoftTTS

from agora_agent.agentkit.vendors import MicrosoftTTS

OpenAITTS

Fixed sample rate: 24000 Hz.

from agora_agent.agentkit.vendors import OpenAITTS

CartesiaTTS

from agora_agent.agentkit.vendors import CartesiaTTS

GoogleTTS

from agora_agent.agentkit.vendors import GoogleTTS

AmazonTTS

from agora_agent.agentkit.vendors import AmazonTTS

HumeAITTS

from agora_agent.agentkit.vendors import HumeAITTS

RimeTTS

from agora_agent.agentkit.vendors import RimeTTS

In byok mode (default), key, speaker, and model_id are required. In managed mode, base_url and model_id are required.

FishAudioTTS

from agora_agent.agentkit.vendors import FishAudioTTS

MiniMaxTTS

from agora_agent.agentkit.vendors import MiniMaxTTS

DeepgramTTS

from agora_agent.agentkit.vendors import DeepgramTTS

MurfTTS

from agora_agent.agentkit.vendors import MurfTTS

SarvamTTS

from agora_agent.agentkit.vendors import SarvamTTS

TypecastTTS

from agora_agent.agentkit.vendors import TypecastTTS

GradiumTTS

from agora_agent.agentkit.vendors import GradiumTTS

MistralTTS

from agora_agent.agentkit.vendors import MistralTTS

GenericTTS

Custom OpenAI-compatible HTTP TTS. url is required and must be an HTTP or HTTPS endpoint that includes a host. An invalid URL format, or a non-HTTP(S) scheme such as ws or wss, raises a ValueError. A valid URL is serialized as tts.vendor = "generic_http".

from agora_agent import GenericTTS

XaiTTS

from agora_agent.agentkit.vendors import XaiTTS

SmallestAITTS

from agora_agent.agentkit.vendors import SmallestAITTS

Voice identifiers are model-specific, so voice_id must belong to the model set in model. For details, see Smallest AI.

STT vendors

Use with with_stt().

DeepgramSTT

api_key is required unless model is an Agora-managed model (nova-2 or nova-3).

from agora_agent.agentkit.vendors import DeepgramSTT

SpeechmaticsSTT

from agora_agent.agentkit.vendors import SpeechmaticsSTT

MicrosoftSTT

from agora_agent.agentkit.vendors import MicrosoftSTT

OpenAISTT

from agora_agent.agentkit.vendors import OpenAISTT

The serialized configuration requires a transcription prompt and language — provide them through the prompt and language parameters or within input_audio_transcription. model defaults to gpt-4o-mini-transcribe.

GoogleSTT

from agora_agent.agentkit.vendors import GoogleSTT

AmazonSTT

from agora_agent.agentkit.vendors import AmazonSTT

AssemblyAISTT

from agora_agent.agentkit.vendors import AssemblyAISTT

AresSTT

from agora_agent.agentkit.vendors import AresSTT

GeminiSTT

from agora_agent.agentkit.vendors import GeminiSTT

XaiSTT

from agora_agent.agentkit.vendors import XaiSTT

SarvamSTT

from agora_agent.agentkit.vendors import SarvamSTT

SmallestAISTT

from agora_agent.agentkit.vendors import SmallestAISTT

Boolean options are serialized as the strings "true" or "false" for the Smallest AI wire protocol. For details, see Smallest AI.

MLLM vendors

Use with with_mllm() for multimodal end-to-end audio processing without separate STT or TTS steps. Calling with_mllm() automatically enables the MLLM module (sets mllm.enable=true); the older advanced_features.enable_mllm flag is deprecated.

OpenAIRealtime

from agora_agent.agentkit.vendors import OpenAIRealtime

AzureOpenAIRealtime

from agora_agent.agentkit.vendors import AzureOpenAIRealtime

GeminiLive

from agora_agent.agentkit.vendors import GeminiLive

VertexAI

from agora_agent.agentkit.vendors import VertexAI

XaiGrok

xAI Grok MLLM vendor (mllm.vendor: "xai").

OpenAIGPTLive

OpenAI GPT-Live MLLM vendor (mllm.vendor: "openai_gpt_live"). with_mllm() raises ValueError if url isn't a full ws:// or wss:// endpoint, headers isn't a JSON object string, delegation isn't client or responses, or session_params overrides a protected field.

from agora_agent import OpenAIGPTLive

mllm = OpenAIGPTLive(
    api_key='your-openai-key',
    greeting="Hello! I'm GPT-live. How can I help you today?",
    model='gpt-live-1',
    voice='marin',
    prompt='your-system-prompt',
)