Improve ASR accuracy with keywords

In a conversational AI scenario, users may say non-dictionary words such as brand names, product names, personal names, place names, or industry-specific terms. To improve recognition accuracy for these terms, configure ASR keywords in the agent's asr block.

PrerequisitesCopy Anchor Link

Before you begin, make sure that you have created an agent and started a session as described in Create an agent and start a session.

Configure ASR keywordsCopy Anchor Link

When you create or update an agent, pass the keywords in its ARES configuration. The following example adds three product terms as ARES keywords:

from agora_agent import Agent, AresSTT

# client is your configured Agora client
agent = (
    Agent(client=client)
    .with_stt(AresSTT(keywords=["Agora", "Conversational AI", "RTC"]))
    .with_llm(...)  # configure your LLM vendor
    .with_tts(...)  # configure your TTS vendor
)

import { Agent, AresSTT } from 'agora-agents';

// client is your configured Agora client
const agent = new Agent({ client })
  .withStt(new AresSTT({ keywords: ['Agora', 'Conversational AI', 'RTC'] }))
  .withLlm(/* configure your LLM vendor */)
  .withTts(/* configure your TTS vendor */);

// client is your configured Agora client
agent := agentkit.NewAgent(client).
	WithStt(vendors.NewAresSTT(vendors.AresSTTOptions{
		Keywords: []string{"Agora", "Conversational AI", "RTC"},
	})).
	WithLlm( /* configure your LLM vendor */ ).
	WithTts( /* configure your TTS vendor */ )

"pipeline": {
  "asr": {
    "credential": { "mode": "managed" },
    "vendor": "ares",
    "language": "en-US",
    "params": {
      "keywords": [
        "Agora",
        "Conversational AI",
        "RTC"
      ]
    }
  }
}

You can configure up to 128 keywords.

ASR keywords versus keyword interruptionCopy Anchor Link

ASR keywords and keyword interruption are separate features, and each uses its own list of keywords:

Best practicesCopy Anchor Link

Keep the following tips in mind when choosing keywords:

• Prioritize brand names, product names, personal names, place names, model numbers, and abbreviations that are prone to misrecognition.
• Avoid configuring too many common words, short words, or generic terms, such as "okay" or "start".
• For mixed-language scenarios, you can configure terms in multiple languages together, such as a product's name in its local language, its English name, and its abbreviation.
• Before going live, validate recognition results with real production audio, and check both the target keywords and similar-sounding words.

ReferenceCopy Anchor Link

For more information, see the following pages:

• ARES
• Interrupt the agent mid-response

API referenceCopy Anchor Link

• Create an agent
• Update an agent