Looking for the previous docs site? Visit the legacy docs homepage. (opens in a new tab)
Agora Docs

Talking while waiting

Updated

Keep conversations natural by playing short phrases, such as "Let me check", while the agent waits for the LLM to respond.

Preview documentation

You're reading the Conversational AI Platform (Preview) docs. Switch to Main.

In conversational AI, delays in LLM responses may cause users to wonder if the agent is still processing, repeat their question, or disengage entirely. Filler words address this by playing short phrases while the agent waits for the LLM to generate a response.

Common use cases for filler words include:

  • Tool calls: When the agent calls tools, response times can increase significantly. Filler words bridge this gap while the agent waits for tool results.
  • Complex queries: Queries that require more LLM processing time benefit from a brief acknowledgment to signal that the agent is working on a response.
  • Customer service scenarios: In support interactions, filler words such as "Let me look into that for you" reassure users that their request is being handled.

Understand the tech

You configure filler words on the agent. When the LLM hasn't responded within a wait threshold, the agent plays a filler word. The filler word comes from one of the following sources:

  • Static: The agent picks a phrase from a list you provide.
  • Generated: Agora generates a short phrase that fits the conversation, based on instructions you provide. If a generated phrase isn't ready in time, the agent falls back to a static phrase. Generated mode requires the agent's LLM to use your own provider key with OpenAI, Groq, or xAI.

Playing a filler word doesn't cancel or restart the LLM request. The agent continues with its reply once the LLM responds. The agent plays at most one filler word per turn, and if the LLM responds before the wait threshold, the agent doesn't play a filler word.

Prerequisites

Before you begin, make sure that you have created an agent and started a session as described in Create an agent and start a session.

Configure static filler words

Configure filler words when you create or update the agent. In the following example, the agent plays a phrase from the list when the LLM hasn't responded within 1.5 seconds. The SDK examples start from the agent builder in Create an agent.

Set the wait time in fixed_time_config, and the phrases and selection rule in static_config:

filler_agent = agent.with_filler_words(
    {
        "enable": True,
        "trigger": {
            "mode": "fixed_time",
            "fixed_time_config": {"response_wait_ms": 1500},
        },
        "content": {
            "mode": "static",
            "static_config": {
                "phrases": [
                    "Please wait.",
                    "Okay.",
                    "Uh-huh.",
                    "Let me look into that.",
                    "One moment.",
                ],
                "selection_rule": "shuffle",
            },
        },
    }
)

Set the wait time in fixed_time_config, and the phrases and selection rule in static_config:

const fillerAgent = agent.withFillerWords({
  enable: true,
  trigger: {
    mode: 'fixed_time',
    fixed_time_config: { response_wait_ms: 1500 },
  },
  content: {
    mode: 'static',
    static_config: {
      phrases: [
        'Please wait.',
        'Okay.',
        'Uh-huh.',
        'Let me look into that.',
        'One moment.',
      ],
      selection_rule: 'shuffle',
    },
  },
});

Set the wait time in FixedTimeConfig, and the phrases and selection rule in StaticConfig. The Go examples on this page also import the SDK's root package as Agora "github.com/AgoraIO/agora-agents-go/v2".

fillerAgent := agent.WithFillerWords(&agentkit.FillerWordsConfig{
	Enable: Agora.Bool(true),
	Trigger: &agentkit.FillerWordsTrigger{
		Mode: Agora.String("fixed_time"),
		FixedTimeConfig: &agentkit.FillerWordsTriggerFixedTimeConfig{
			ResponseWaitMs: Agora.Int(1500),
		},
	},
	Content: &agentkit.FillerWordsContent{
		Mode: agentkit.FillerWordsContentModeStatic.Ptr(),
		StaticConfig: &agentkit.FillerWordsContentStaticConfig{
			Phrases: []string{
				"Please wait.",
				"Okay.",
				"Uh-huh.",
				"Let me look into that.",
				"One moment.",
			},
			SelectionRule: agentkit.FillerWordsSelectionRuleShuffle.Ptr(),
		},
	},
})

Add a filler_words object to the agent configuration, at the same level as instructions and pipeline:

"filler_words": {
  "enable": true,
  "trigger": {
    "mode": "fixed_time",
    "response_wait_ms": 1500
  },
  "content": {
    "mode": "static",
    "phrases": [
      "Please wait.",
      "Okay.",
      "Uh-huh.",
      "Let me look into that.",
      "One moment."
    ],
    "selection_rule": "shuffle"
  }
}
FieldDescription
enableWhether to play filler words.
trigger.modeWhen to play a filler word. Set to fixed_time.
trigger.response_wait_msHow long in milliseconds to wait for the LLM before playing a filler word, from 100 to 10000. Required.
content.modeSet to static.
content.phrasesThe filler phrases, 1 to 100 non-empty strings.
content.selection_ruleHow the agent picks the next phrase: shuffle (random order, the default) or round_robin (list order, starting over after the last phrase).

Configure generated filler words

To have Agora generate a phrase that fits the conversation, use the same settings with the content mode set to generated. Generated mode requires the agent's LLM to use your own key with OpenAI, Groq, or xAI. With a managed LLM, the request is rejected. For details, see Use your own provider keys. In the instructions, describe the kind of phrase to generate. Set the fallback strategy to static to use a static phrase when a generated one isn't ready in time.

Put the instructions in generated_config.prompt. This example assumes that agent uses your own OpenAI key, as in Use the key in an agent.

generated_filler_agent = agent.with_filler_words(
    {
        "enable": True,
        "trigger": {
            "mode": "fixed_time",
            "fixed_time_config": {"response_wait_ms": 1500},
        },
        "content": {
            "mode": "generated",
            "generated_config": {
                "prompt": "Generate a short, natural filler phrase that shows you're working on the request. Don't answer the question.",
                "fallback_strategy": "static",
            },
        },
    }
)

Put the instructions in generated_config.prompt. This example assumes that agent uses your own OpenAI key, as in Use the key in an agent.

const generatedFillerAgent = agent.withFillerWords({
  enable: true,
  trigger: {
    mode: 'fixed_time',
    fixed_time_config: { response_wait_ms: 1500 },
  },
  content: {
    mode: 'generated',
    generated_config: {
      prompt: "Generate a short, natural filler phrase that shows you're working on the request. Don't answer the question.",
      fallback_strategy: 'static',
    },
  },
});

Put the instructions in GeneratedConfig.Prompt. This example assumes that agent uses your own OpenAI key, as in Use the key in an agent.

generatedFillerAgent := agent.WithFillerWords(&agentkit.FillerWordsConfig{
	Enable: Agora.Bool(true),
	Trigger: &agentkit.FillerWordsTrigger{
		Mode: Agora.String("fixed_time"),
		FixedTimeConfig: &agentkit.FillerWordsTriggerFixedTimeConfig{
			ResponseWaitMs: Agora.Int(1500),
		},
	},
	Content: &agentkit.FillerWordsContent{
		Mode: agentkit.FillerWordsContentModeGenerated.Ptr(),
		GeneratedConfig: &agentkit.FillerWordsContentGeneratedConfig{
			Prompt:           Agora.String("Generate a short, natural filler phrase that shows you're working on the request. Don't answer the question."),
			FallbackStrategy: agentkit.FillerWordsFallbackStrategyStatic.Ptr(),
		},
	},
})
"filler_words": {
  "enable": true,
  "trigger": {
    "mode": "fixed_time",
    "response_wait_ms": 1500
  },
  "content": {
    "mode": "generated",
    "instructions": "Generate a short, natural filler phrase that shows you're working on the request. Don't answer the question.",
    "fallback_strategy": "static"
  }
}
FieldDescription
content.modeSet to generated.
content.instructionsRequired. The instructions used to generate filler phrases. You can use the agent's variables in them, for example {{customer_name}}. For details, see Personalize conversations.
content.fallback_strategyWhat to use when a generated phrase isn't available. Set to static.

The enable and trigger fields work the same way as for static filler words.

Best practices

Keep the following tips in mind when configuring filler words:

  • Keep phrases short and natural: Use brief, conversational phrases that sound like something a person would say.
  • Match phrasing to your use case: For customer support agents, use reassuring phrases like "Let me look into that for you." For casual assistants, use informal phrases like "Hmm, one sec."
  • Tune the trigger threshold: Start with a response_wait_ms of 1500 ms, and adjust it based on your LLM's typical response time. If your agent often calls tools, consider a lower threshold to cover the longer waits.
  • Provide enough variety: Include at least 4 to 6 phrases to avoid sounding repetitive. Use shuffle to vary the phrases across turns.

Reference

For more information, see the following pages:

API reference

Contact Us

+1 (408) 879-5885

2804 Mission College Blvd.Santa Clara, CA, USA 95054

AgoraCopyright © 2026 AgoraAll rights reserved