ARES

Updated

Integrate ARES ASR into Conversational AI Engine.

Adaptive Recognition Engine for Speech (ARES) provides built-in real-time speech-to-text, offering seamless integration with low latency and reliable performance for conversational AI applications.

Sample configuration

The following example shows how to configure ARES ASR when starting a conversational AI agent.

from agora_agent import Agent, AresSTT

# client is your configured Agora client
agent = (
    Agent(client)
    .with_stt(AresSTT(
        keywords=['Agora', 'Conversational AI', 'RTC'],
    ))
    .with_llm(...)  # configure your LLM vendor
    .with_tts(...)  # configure your TTS vendor
)

AresSTT has no language parameter. ARES uses the recognition language set through the agent's turn detection configuration, using with_turn_detection(), which defaults to en-US.

import { Agent, AresSTT } from 'agora-agents';

// client is your configured Agora client
const agent = new Agent({ client })
  .withStt(new AresSTT({
    keywords: ['Agora', 'Conversational AI', 'RTC'],
  }))
  .withLlm(/* configure your LLM vendor */)
  .withTts(/* configure your TTS vendor */);

AresSTT has no language option. ARES uses the recognition language set through the agent's turn detection configuration, using withTurnDetection(), which defaults to en-US.

import (
    "github.com/AgoraIO/agora-agents-go/v2/agentkit"
    "github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"
)

// client is your configured Agora client
agent := agentkit.NewAgent(client).WithStt(
    vendors.NewAresSTT(vendors.AresSTTOptions{
        Keywords: []string{"Agora", "Conversational AI", "RTC"},
    }),
).WithLlm(/* configure your LLM vendor */).
  WithTts(/* configure your TTS vendor */)

AresSTTOptions has no Language field. ARES uses the recognition language set through the agent's turn detection configuration, using WithTurnDetection(), which defaults to en-US.

Use the following asr configuration in your request:

"asr": {
  "vendor": "ares",
  "language": "en-US",
  "keywords": ["Agora", "Conversational AI", "RTC"]
}

Key parameters

asr required
ParameterDescription
vendorstring
required

ASR provider. Set to ares to use Adaptive Recognition Engine for Speech.

languagestring
required

One or more BCP-47 language tags identifying the languages used for agent interaction. To recognize multiple languages, pass the tags as a single comma-separated string, for example, "en-US,hi-IN,es-MX". See Supported languages for the complete list.

Semantic end-of-speech detection (turn_detection.config.end_of_speech.mode set to semantic) works only when language is en-US, en-IN, zh-CN, zh-HK, or zh-TW. If language includes any other language, or more than one language, the agent uses VAD for end-of-speech detection instead. Agents SDK v2.15.0 and later return an error for these configurations.

keywordsarray[string]
optional

Keywords used to improve ASR recognition accuracy for specific terms, such as brand names, product names, or industry jargon. Supports up to 128 keywords. See Configure ASR keywords for details.

The table extends beyond the current width. Scroll horizontally to view more columns.