OpenAI
Updated
Integrate OpenAI TTS into Conversational AI Engine.
OpenAI provides natural-sounding text-to-speech with customizable voice instructions and multiple voice options.
Info
This integration is fully supported for use with Conversational AI Engine. While it has completed functional validation, it is newer to the platform, and additional provider-specific edge cases may be identified as usage scales across a broader range of applications and workloads.
Sample configuration
The following examples show how to configure OpenAI TTS when starting a conversational AI agent.
from agora_agent import Agent
from agora_agent.agentkit.vendors import OpenAITTS
# client is your configured Agora client
agent = (
Agent(client)
.with_stt(...) # configure your STT vendor
.with_llm(...) # configure your LLM vendor
.with_tts(OpenAITTS(
api_key='your-openai-key', # omit api_key and base_url to use Agora-managed mode
base_url='https://api.openai.com/v1',
model='gpt-4o-mini-tts',
voice='coral',
instructions='Please use standard American English, natural tone, moderate pace, and steady intonation',
speed=1,
))
)import { Agent, OpenAITTS } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withStt(/* configure your STT vendor */)
.withLlm(/* configure your LLM vendor */)
.withTts(new OpenAITTS({
apiKey: 'your-openai-key', // omit apiKey and baseUrl to use Agora-managed mode
baseUrl: 'https://api.openai.com/v1',
model: 'gpt-4o-mini-tts',
voice: 'coral',
instructions: 'Please use standard American English, natural tone, moderate pace, and steady intonation',
speed: 1,
}));import "github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"
// client is your configured Agora client
agent := agentkit.NewAgent(client).WithStt(/* configure your STT vendor */).
WithLlm(/* configure your LLM vendor */).
WithTts(
vendors.NewOpenAITTS(vendors.OpenAITTSOptions{
APIKey: "your-openai-key", // omit APIKey and BaseURL to use Agora-managed mode
BaseURL: "https://api.openai.com/v1",
Model: "gpt-4o-mini-tts",
Voice: "coral",
Instructions: "Please use standard American English, natural tone, moderate pace, and steady intonation",
Speed: agora.Float64(1),
}),
)-
Managed mode
To use OpenAI TTS with Agora-managed credentials, set
credential_modeto"managed"in thettsblock. When using managed mode,params.api_keyis not required. You can still use thettsfield to configure additional settings such asvoiceandinstructions. For more information, see Use managed mode."tts": { "credential_mode": "managed", "vendor": "openai", "params": { "base_url": "https://api.openai.com/v1", "model": "tts-1", "voice": "coral", "instructions": "Please use standard American English, natural tone, moderate pace, and steady intonation", "speed": 1 } } -
BYOK
"tts": { "vendor": "openai", "params": { "base_url": "https://api.openai.com/v1", "api_key": "<your_openai_key>", "model": "gpt-4o-mini-tts", "voice": "coral", "instructions": "Please use standard American English, natural tone, moderate pace, and steady intonation", "speed": 1 } }
Caution
The parameters listed on this page are validated for use with Conversational AI Engine. Required parameters must be provided as documented. Any additional parameters are passed through directly to the underlying vendor without validation. For advanced configuration options, available voices, and detailed parameter descriptions, see the OpenAI TTS documentation.
Key parameters
The following parameters are required when using your own API key (BYOK). When using managed mode, api_key is not required. For more information, see Use managed mode.
base_urlstringThe endpoint URL for the OpenAI TTS service. Required when using your own API key (BYOK). See How to use data residency.
api_keystringThe API key used for authentication. Required when using your own API key (BYOK). Get your API key from the OpenAI Console.
modelstringIdentifier of the model to be used. Required when using your own API key (BYOK).
voicestringThe voice identifier for speech synthesis.
instructionsstringCustom instructions for voice style, accent, pace, and tone. Helps fine-tune the speech characteristics.
speednumber- Default value
- 1.0
Speaking rate multiplier. Values between 0.25 and 4.0, where 1.0 is normal speed.
Info
OpenAI TTS models do not support changing the sample rate. The audio output is fixed at 24,000 Hz.
