TypeScript

Full API reference for the Agora Conversational AI TypeScript SDK.

AgoraClient

AgoraClient extends the Fern-generated base client with domain pool support for regional URL cycling and three authentication modes. Pass appId and appCertificate only for the recommended app-credentials mode. The SDK mints fresh REST tokens per request and generates RTC join tokens at session start.

import { AgoraClient, Area } from 'agora-agents';

Constructor

const client = new AgoraClient(options: AgoraClient.Options);

The authentication mode is resolved automatically from the options you provide.

Authentication mode is resolved from the options you provide:

Passing customerId without customerSecret throws an error.

See Authentication for details on each mode.

Properties

The following read-only properties are available on any AgoraClient instance.

Methods

The following methods are available in addition to the Fern-generated sub-client methods.

nextRegion()

Cycles to the next region prefix in the domain pool. Call this after a request failure to try a different regional endpoint.

client.nextRegion();

selectBestDomain(signal?)

Runs a DNS check to select the best domain suffix. Calls made within 30 seconds of the last successful selection are no-ops.

await client.selectBestDomain();

stopAgent(agentId)

Stops an agent by ID without an AgentSession reference, for example, from an end-call handler. The method treats a 404 response as success.

stopAgent(agentId: string): Promise<void>

getCurrentURL()

Returns the full API URL currently in use as a string.

const url = client.getCurrentURL();
// Example: 'https://api-us-west-1.agora.io/api/conversational-ai-agent'

Sub-clients

AgoraClient exposes Fern-generated sub-clients for direct REST API access. You typically do not need these when using the agentkit layer.

For full method signatures and request parameters, see the REST API reference.

Agent

Agent is an immutable configuration object. Each builder method returns a new Agent instance — the original is never modified. Define one Agent at startup and call createSession() on it for each user conversation.

import { Agent } from 'agora-agents';

Constructor

new Agent(options: AgentOptions)

client is required. All other options are optional. Use the builder methods to set vendor configuration after construction.

Builder methods

All builder methods return a new Agent instance. The original is never modified.

withLlm(vendor)

Sets the LLM vendor. Pass an instance of OpenAI, AzureOpenAI, Anthropic, Gemini, or any other LLM vendor.

withLlm(vendor: LlmVendor): Agent<TTSSampleRate, TArea>

withTts(vendor)

Sets the TTS vendor. The sample rate type is captured and tracked for avatar compatibility.

withTts<SR extends number>(vendor: TtsVendor<SR>): Agent<SR, TArea>

withStt(vendor)

Sets the STT vendor. Pass an instance of any STT vendor class.

withStt(vendor: SttVendor): Agent<TTSSampleRate, TArea>

withMllm(vendor)

Sets the MLLM vendor for multimodal mode. Pass OpenAIRealtime, AzureOpenAIRealtime, GeminiLive, VertexAI, XaiGrok, or OpenAIGPTLive. Calling withMllm() automatically sets mllm.enable = true. MLLM mode does not require withTts() / withLlm() / withStt().

withMllm(vendor: GlobalMllmVendor | CNMllmVendor): Agent<TTSSampleRate, TArea>

withAvatar(vendor)

Sets the avatar vendor. The this constraint enforces at compile time that the agent's TTS sample rate matches the avatar's required rate.

withAvatar<RequiredSR extends number>(
  this: Agent<RequiredSR>,
  vendor: AvatarVendor<RequiredSR>
): Agent<RequiredSR, TArea>

withTurnDetection(config)

Configures cascading-flow turn detection. Pass { config: { start_of_speech, end_of_speech } } for SOS/EOS detection. Use withInterruption() for interruption behavior and MLLM vendor turnDetection for MLLM turn detection.

withTurnDetection(config: TurnDetectionConfig): Agent<TTSSampleRate, TArea>

withInterruption(config)

Configures unified interruption behavior using the top-level interruption object. Use this for start_of_speech and keywords interruption modes.

withInterruption(config: InterruptionConfig): Agent<TTSSampleRate, TArea>

withInstructions(text)

Overrides the LLM system prompt on a new Agent instance. Deprecated. Set systemMessages on the LLM vendor instead.

withInstructions(instructions: string): Agent<TTSSampleRate, TArea>

withGreeting(text)

Overrides the greeting message on a new Agent instance. Deprecated. Set greetingMessage on the LLM or MLLM vendor instead.

withGreeting(greeting: string): Agent<TTSSampleRate, TArea>

Other builder methods

The following methods follow the same pattern — each returns a new Agent instance with the updated configuration.

createSession(options)

Creates an AgentSession for a channel using the agent's client. Doesn't start the agent. Call session.start() to join the channel.

createSession(options: SessionOptions): AgentSession

SessionOptions fields:

preset is session-scoped because the underlying Agora start/join API applies presets per session, not per reusable Agent definition.

When you omit credentials for supported reseller-backed vendor models, AgentKit infers the matching session preset automatically:

• Deepgram STT: nova-2, nova-3
• OpenAI LLM: gpt-4o-mini, gpt-4.1-mini, gpt-5-nano, gpt-5-mini
• OpenAI TTS: tts-1
• MiniMax TTS: speech-2.6-turbo, speech-2.8-turbo

If you provide your own vendor API key for those same models, AgentKit keeps the request in BYOK mode and does not infer a preset.

Properties

Read-only properties available on any Agent instance.

toProperties(opts)

Low-level method to convert the agent configuration to the Fern request format. Used internally by AgentSession.start(). You typically do not need to call this directly unless building custom request bodies.

toProperties(opts): StartAgentsRequest.Properties

In cascading mode, throws if TTS or LLM isn't set. If you don't call withStt(), ASR defaults to ARES (fengming for Area.CN). turnDetection.language defaults to en-US and is also sent as asr.language.

Type aliases

Public aliases over Fern-generated types include LlmConfig, SttConfig, AsrConfig (= SttConfig), MllmConfig, AvatarConfig, session/conversation types, and think types (ThinkOnListeningAction, etc.).

Think value constants: ThinkOnListeningActionInject, ThinkOnListeningActionInterrupt, ThinkOnListeningActionIgnore, ThinkOnThinkingActionInterrupt, ThinkOnThinkingActionIgnore, ThinkOnSpeakingActionInterrupt, ThinkOnSpeakingActionIgnore.

The think action options also accept the string append, which has no convenience constant. With append, the instruction doesn't interrupt the current interaction. The agent waits for the current user turn, LLM inference, or TTS playback to finish, then appends the instruction to the context as a separate user message and starts a new turn.

FillerWordsConfig supports static and generated filler words. Generated mode requires static fallback phrases, while generated_config and prompt are optional. Agora hosts the generation service, so you can't configure its model endpoint, API key, or model parameters. For the field structure, see Configure generated filler words.

FillerWordsContentGeneratedConfig sets the conversation context for generated filler words. Use context_message_limit (1 to 6, defaults to 1) to set how many of the most recent conversation messages to use, counting the current turn's message, and history_character_limit (0 to 10000, defaults to 1000) to cap the combined characters of the earlier messages. For details, see Talking while waiting.

Public aliases also include FillerWordsContentGeneratedConfig and the custom tool types LlmTool, LlmToolFunction, LlmToolServer, and LlmToolExecution. llm.tools is independent of MCP server configuration, but both require withTools(true) to enable tool calling.

AgentSession

AgentSession manages the full lifecycle of a running agent. Create sessions using agent.createSession(); do not call the constructor directly.

import { AgentSession } from 'agora-agents';

State machine

A session progresses through the following states:

idle ──► starting ──► running ──► stopping ──► stopped
            │                         │
            ▼                         ▼
          error                     error

start() can also be called from stopped or error state to restart the session. say(), interrupt(), update(), and think() reject on failure but don't change the session state.

Methods

The following methods are available on an AgentSession instance.

start()

Starts the agent session. Generates tokens if not provided, sends the start request, and returns the agent ID. Resolves explicit preset values and also infers reseller presets from supported vendor configs when credentials are omitted.

start(): Promise<string>

• Transitions: idle / stopped / error → starting → running
• Throws if called in starting, running, or stopping state
• Throws if avatar config is invalid (wrong TTS sample rate)
• Throws if no token is provided and the client has no appCertificate
• Throws if agentUid (or avatar agoraUid) isn't numeric when tokens are auto-generated
• Throws if appId or appCertificate isn't exactly 32 characters when tokens are auto-generated
• Throws if MLLM is enabled together with an enabled avatar — avatars are only supported with the cascading ASR + LLM + TTS pipeline
• Applies explicit preset values when provided and sends Agora-managed configuration when supported vendor credentials are omitted
• Fills generic avatar agora_appid and agora_channel from the session when omitted
• Generates avatar agora_token for HeyGenAvatar, LiveAvatarAvatar, GenericAvatar, Tavus, Protoface, and LemonSlice when agoraToken is omitted and the client has an appCertificate. Other vendors (AkoolAvatar, AnamAvatar) never receive an auto-generated token.

stop()

Stops the agent session and removes the agent from the channel. If the agent has already stopped — for example due to idle timeout — resolves silently rather than throwing a 404 error.

stop(): Promise<void>

• Transitions: running → stopping → stopped
• Throws if called outside running state

say(text, options?)

Instructs the agent to speak the given text.

say(text: string, options?: SayOptions): Promise<void>

• Only valid in running state

interrupt()

Interrupts the agent's current speech.

interrupt(): Promise<void>

• Only valid in running state

think(text, options?)

Injects a custom text instruction into the running agent.

think(text: string, options?: ThinkOptions): Promise<ThinkResponse>

• Only valid in running state

update(config)

Updates the agent configuration mid-session without restarting. Accepts a partial configuration object in REST API format.

update(config: AgentConfigUpdate): Promise<void>

• Only valid in running state

getHistory()

Fetches the conversation history for this session. Requires a valid agent ID — start() must have been called successfully.

getHistory(): Promise<ConversationHistory>

getTurns(options?)

Fetches turn-by-turn analytics for this session, including start/end events and latency metrics. Requires a valid agent ID and start() must have been called successfully.

getTurns(options?: GetTurnsOptions): Promise<ConversationTurns>

• options.page_index: page number, starting from 1
• options.page_size: number of turns per page

getAllTurns(options?)

Fetches all turn analytics pages and merges the turns array. Requires a valid agent ID and start() must have been called successfully.

getAllTurns(options?: Omit<GetTurnsOptions, "page_index">): Promise<ConversationTurns>

• Requires a valid agentId
• For very long sessions, prefer processing pages with getTurns() to avoid holding all turns in memory

getInfo()

Fetches current agent metadata from the API. Requires a valid agent ID.

getInfo(): Promise<SessionInfo>

on(event, handler)

Subscribes to a session event. Register handlers before calling start() to avoid missing the started event.

on<T>(event: AgentSessionEvent, handler: AgentSessionEventHandler<T>): void

off(event, handler)

Unsubscribes a previously registered event handler.

off<T>(event: AgentSessionEvent, handler: AgentSessionEventHandler<T>): void

Events

The session emits the following events. See AgentSessionEvent and AgentSessionEventHandler for type details.

Properties

The following read-only properties are available on any AgentSession instance.

Using session.raw

Access the generated REST client to call endpoints not yet wrapped:

await session.raw.getTurns({
  appid: session.appId,
  agentId: session.id!,
});

You must pass appid and agentId manually when using raw methods.

Presets and BYOK

Prefer configuring vendors on the Agent builder. When you omit credentials for supported Agora-managed models, AgentKit sends the matching Agora-managed configuration at session start.

preset is an advanced session option for project-specific settings, not for selecting Agora-managed models. Most applications should use the builder instead.

• Omit vendor credentials on the builder for supported Agora-managed models.
• Provide vendor API keys when you want BYOK.
• Pass preset on agent.createSession(...) only when you need to access specific project-specific settings.

Supported Agora-managed models:

• Deepgram STT: nova-2, nova-3
• OpenAI LLM: gpt-4o-mini, gpt-4.1-mini, gpt-5-nano, gpt-5-mini
• OpenAI TTS: tts-1
• MiniMax TTS: speech-2.6-turbo, speech-2.8-turbo

Vendors

All vendor classes are imported from agora-agents. Pass vendor instances to the Agent builder methods.

LLM vendors

Use with withLlm().

OpenAI

new OpenAI(options: OpenAIOptions)

apiKey is optional for the following reseller preset models: gpt-4o-mini, gpt-4.1-mini, gpt-5-nano, gpt-5-mini. If apiKey is omitted for one of those models, AgentKit infers the matching session preset. This no-key branch is only available with the default OpenAI endpoint and without a custom vendor hint. If apiKey is provided, AgentKit uses standard BYOK behavior instead.

AzureOpenAI

new AzureOpenAI(options: AzureOpenAIOptions)

Anthropic

new Anthropic(options: AnthropicOptions)

Gemini

new Gemini(options: GeminiOptions)

Other LLM vendors

Groq and CustomLLM share OpenAI's option shape; VertexAILLM mirrors Gemini. All accept common LLM fields, such as systemMessages, greetingMessage, failureMessage, maxHistory, params, headers, and tools.

tools declares custom tools the LLM can choose to call. Each tool includes a model-visible function definition and the server configuration for a synchronous GET or POST request. withTools(true) applies to both tools and MCP server configuration. For more information, see Call custom tools.

TTS vendors

Use with withTts(). The sampleRate option determines avatar compatibility — see withAvatar().

ElevenLabsTTS

new ElevenLabsTTS<SR extends ElevenLabsSampleRate>(options: ElevenLabsTTSOptions<SR>)

MicrosoftTTS

new MicrosoftTTS<SR extends MicrosoftSampleRate>(options: MicrosoftTTSOptions<SR>)

OpenAITTS

Fixed at 24,000 Hz — no configurable sample rate.

new OpenAITTS(options: OpenAITTSOptions)

apiKey is optional only for the reseller-backed tts-1 preset path. If omitted with model: 'tts-1' or no explicit model, AgentKit infers openai_tts_1. If provided, the request stays in BYOK mode.

CartesiaTTS

new CartesiaTTS<SR extends CartesiaSampleRate>(options: CartesiaTTSOptions<SR>)

Other TTS vendors

SmallestAITTS also accepts numberPronunciationLanguage?, mathNotation?, pronunciationDicts?, sessionId?, and requestId?. Voice identifiers are model-specific, so voiceId must belong to the model set in model. For details, see Smallest AI.

For MiniMaxTTS, key is optional only for reseller-backed models: speech-2.6-turbo, speech-2.8-turbo. If key is omitted for one of those models, AgentKit infers the matching session preset. In that preset-backed path, groupId, voiceId, and url are optional overrides rather than required fields. If key is provided, AgentKit uses BYOK, and groupId, voiceId, and url are required.

GenericTTS

Custom OpenAI-compatible HTTP TTS. url is required and must be an absolute HTTP or HTTPS address — the constructor throws if url is missing, badly formatted, or uses a non-HTTP(S) scheme such as ws: or wss:. A valid URL is serialized as tts.vendor = "generic_http".

new GenericTTS(options: GenericTTSOptions)

STT vendors

Use with withStt().

DeepgramSTT

new DeepgramSTT(options?: DeepgramSTTOptions)

Provide apiKey, or omit it and set model to nova-2 or nova-3 to use Agora-managed credentials. The constructor throws otherwise.

For nova-2 and nova-3, omit apiKey to use Agora-managed credentials. For all other Deepgram models, apiKey is required.

Other STT vendors

SmallestAISTT also accepts the boolean options wordTimestamps?, sentenceTimestamps?, diarize?, vadEvents?, endpointing?, format?, finalizeOnWords?, punctuate?, capitalize?, itnNormalize?, fullTranscript?, redactPii?, and redactPci?, plus eouTimeoutMs? and maxWords?. Boolean options are serialized as the strings "true" or "false" for the Smallest AI wire protocol. For details, see Smallest AI.

For OpenAISTT, the serialized configuration requires a transcription prompt and language — provide them through the prompt and language options or within inputAudioTranscription. model defaults to gpt-4o-mini-transcribe.

MLLM vendors

Use with withMllm() for multimodal end-to-end audio processing without separate STT or TTS steps. Calling withMllm() automatically enables the MLLM module (sets mllm.enable = true); the older advancedFeatures.enable_mllm flag is deprecated.

OpenAIRealtime

new OpenAIRealtime(options: OpenAIRealtimeOptions)

AzureOpenAIRealtime

new AzureOpenAIRealtime(options: AzureOpenAIRealtimeOptions)

GeminiLive

new GeminiLive(options: GeminiLiveOptions)

VertexAI

new VertexAI(options: VertexAIOptions)

XaiGrok

new XaiGrok(options: XaiGrokOptions)

OpenAIGPTLive

OpenAI GPT-Live MLLM vendor (mllm.vendor: "openai_gpt_live"). The constructor throws if apiKey is empty. withMllm() throws if url isn't a full ws:// or wss:// endpoint, headers isn't a JSON object string, delegation isn't client or responses, or sessionParams overrides a protected field.

new OpenAIGPTLive(options: OpenAIGPTLiveOptions)

inputAudioTranscription and turnDetection are deprecated and accepted only for compatibility. GPT-Live doesn't support them: setting inputAudioTranscription throws, and the SDK ignores turnDetection and logs a warning.

Avatar vendors

Use with withAvatar(). Some avatar vendors require a specific TTS sample rate, enforced at compile time and runtime.

HeyGenAvatar

Deprecated. Use LiveAvatarAvatar for new integrations. HeyGenAvatar still works and serializes vendor: "heygen". Requires TTS at 24,000 Hz.

new HeyGenAvatar(options: HeyGenAvatarOptions)

AkoolAvatar

Requires TTS at 16,000 Hz.

new AkoolAvatar(options: AkoolAvatarOptions)

LiveAvatarAvatar

Requires TTS at 24,000 Hz.

new LiveAvatarAvatar(options: LiveAvatarAvatarOptions)

AnamAvatar

new AnamAvatar(options: AnamAvatarOptions)

GenericAvatar

Generic avatars can omit agoraAppId, agoraChannel, and agoraToken. AgentKit fills them from the session at start().

new GenericAvatar(options: GenericAvatarOptions)

Tavus

Tavus avatars can omit agoraAppId, agoraChannel, and agoraToken. AgentKit fills them from the session at start(). Serializes vendor: "generic". For the endpoint and a full example, see Tavus.

new Tavus(options: TavusOptions)

Protoface

Protoface avatars can omit agoraAppId, agoraChannel, and agoraToken. AgentKit fills them from the session at start(). Serializes vendor: "generic". For the endpoint and a full example, see Protoface.

new Protoface(options: ProtofaceOptions)

LemonSlice

LemonSlice avatars can omit agoraAppId, agoraChannel, and agoraToken. AgentKit fills them from the session at start(). Serializes vendor: "generic". For the endpoint and a full example, see LemonSlice.

new LemonSlice(options: LemonSliceOptions)

Token utilities

Helper functions and classes for generating and managing tokens. Use these when you need control over token lifetime, or when generating tokens outside of a session.

import { generateConvoAIToken, ExpiresIn } from 'agora-agents';

generateConvoAIToken(options)

Generates a Conversational AI token combining RTC and RTM privileges. This is the same token the SDK generates automatically in app-credentials mode. Use this when you need a token outside of a session, or when passing a pre-built token to SessionOptions.token.

generateConvoAIToken(options: GenerateConvoAITokenOptions): string

Returns: string — the generated token.

Throws: if appId or appCertificate isn't exactly 32 characters.

const token = generateConvoAIToken({
  appId: 'your-app-id',
  appCertificate: 'your-app-certificate',
  channelName: 'support-room-123',
  uid: 1,
  tokenExpire: ExpiresIn.hours(12),
});

ExpiresIn

Helper for specifying token lifetimes. Use with SessionOptions.expiresIn or generateConvoAIToken. Values are validated and capped at the Agora maximum of 86400 seconds (24 hours).

Types and enums

Shared types and enums used across AgoraClient, Agent, AgentSession, and vendor classes.

Area

Region used for API routing. Pass to AgoraClient via the area option.

import { Area } from 'agora-agents';

AgoraAuthMode

The resolved authentication mode on an AgoraClient instance. Read via client.authMode.

type AgoraAuthMode = "app-credentials" | "token" | "basic";

AgentSessionEvent

Union type of all valid event names for session.on() and session.off().

type AgentSessionEvent = "started" | "stopped" | "error";

AgentSessionEventHandler

Generic handler type for session event callbacks.

type AgentSessionEventHandler<T = unknown> = (data: T) => void;

SpeakPriority

Controls how the agent handles a say() call relative to its current activity.

type SpeakPriority = "INTERRUPT" | "APPEND" | "IGNORE";

The SpeakPriorityInterrupt, SpeakPriorityAppend, and SpeakPriorityIgnore constants are also exported.

AgoraError

Thrown when the API returns a 4xx or 5xx response. Catch this to inspect the status code and response body.

import { AgoraError } from 'agora-agents';

try {
  const agentId = await session.start();
} catch (err) {
  if (err instanceof AgoraError) {
    console.error('Status:', err.statusCode);
    console.error('Message:', err.message);
    console.error('Body:', err.body);
  }
}