Smallest AI
Updated
Use Smallest AI for text-to-speech (TTS) in a Conversational AI Platform agent.
Preview documentation
Smallest AI provides low-latency streaming text-to-speech (TTS) through its Lightning model family, with voices across a broad set of languages.
Sample configuration
The following example shows how to configure Smallest AI as the agent's TTS when you create or update an agent. For a complete example, see Create an agent and start a session.
Smallest AI supports BYOK only. It isn't available in managed mode. Store your Smallest AI API key in a secret set, and refer to it from the agent. For details, see Use your own provider keys.
from agora_agent import Agent, SmallestAITTS, secret_ref
# client is your configured Agora client
agent = (
Agent(client=client)
.with_stt(...) # configure your ASR vendor
.with_llm(...) # configure your LLM vendor
.with_tts(
SmallestAITTS(
api_key=secret_ref("support_prod", "smallestai"),
url="https://api.smallest.ai/waves/v1/tts/live",
model="lightning_v3.1_pro",
voice_id="hazel",
sample_rate=16000,
speed=1.0,
language="zh",
number_pronunciation_language="zh",
math_notation=True,
pronunciation_dicts=[],
session_id="session_001",
request_id="request_001",
)
)
)import { Agent, SmallestAITTS, secretRef } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withStt(/* configure your ASR vendor */)
.withLlm(/* configure your LLM vendor */)
.withTts(
new SmallestAITTS({
apiKey: secretRef('support_prod', 'smallestai'),
url: 'https://api.smallest.ai/waves/v1/tts/live',
model: 'lightning_v3.1_pro',
voiceId: 'hazel',
sampleRate: 16000,
speed: 1.0,
language: 'zh',
numberPronunciationLanguage: 'zh',
mathNotation: true,
pronunciationDicts: [],
sessionId: 'session_001',
requestId: 'request_001',
}),
);sampleRate, speed, mathNotation := 16000, 1.0, true
// client is your configured Agora client
agent := agentkit.NewAgent(client).
WithStt( /* configure your ASR vendor */ ).
WithLlm( /* configure your LLM vendor */ ).
WithTts(vendors.NewSmallestAITTS(vendors.SmallestAITTSOptions{
APIKey: agentkit.SecretRef("support_prod", "smallestai"),
URL: "https://api.smallest.ai/waves/v1/tts/live",
Model: "lightning_v3.1_pro",
VoiceID: "hazel",
SampleRate: &sampleRate,
Speed: &speed,
Language: "zh",
NumberPronunciationLanguage: "zh",
MathNotation: &mathNotation,
PronunciationDicts: []string{},
SessionID: "session_001",
RequestID: "request_001",
}))Set credential.mode to byok and credential.fields.api_key to a reference to your Smallest AI API key in a secret set. In the following example, the key is stored as smallestai in the support_prod set:
"tts": {
"credential": {
"mode": "byok",
"fields": {
"api_key": "$secrets.support_prod.smallestai"
}
},
"vendor": "smallestai",
"url": "https://api.smallest.ai/waves/v1/tts/live",
"params": {
"model": "lightning_v3.1_pro",
"voice_id": "hazel",
"sample_rate": 16000,
"speed": 1.0,
"language": "zh",
"number_pronunciation_language": "zh",
"math_notation": true,
"pronunciation_dicts": [],
"session_id": "session_001",
"request_id": "request_001"
}
}Key parameters
The tts block takes the following fields. In the SDKs, the Smallest AI TTS class sets them from its options.
vendorstringSet to smallestai.
credentialobjectSet mode to byok and fields.api_key to a $secrets.<set>.<key> reference to your Smallest AI API key.
urlstringThe Smallest AI streaming TTS endpoint, for example https://api.smallest.ai/waves/v1/tts/live.
paramsobjectmodelstringThe Smallest AI TTS model to use, such as lightning_v3.1_pro or lightning_v3.1. Voice identifiers are model-specific. Make sure voice_id belongs to the model you select.
voice_idstringThe Smallest AI voice identifier to use, for example hazel.
sample_rateintegerOutput audio sample rate in Hz, for example 16000.
speednumberSpeech rate multiplier, where 1.0 is the model's default pace.
languagestringLanguage code used for speech synthesis, for example zh.
number_pronunciation_languagestringLanguage code used when reading numbers aloud. Set this when numbers should be spoken in a different language from the surrounding text.
math_notationbooleanWhether to read mathematical notation as spoken mathematics rather than as literal symbols.
pronunciation_dictsarray[string]Identifiers of Smallest AI pronunciation dictionaries to apply, used to override how specific terms are spoken.
session_idstringCaller-supplied session identifier, passed through to Smallest AI for tracing.
request_idstringCaller-supplied request identifier, passed through to Smallest AI for tracing.
Caution
The parameters listed on this page are validated for use with the Conversational AI Platform. Required parameters must be provided as documented. Any additional parameters in params are passed through directly to Smallest AI without validation. For a full list of supported options, refer to the Smallest AI documentation.
This integration is fully supported for use with the Conversational AI Platform. While it has completed functional validation, it is newer to the platform, and additional provider-specific edge cases may be identified as usage scales across a broader range of applications and workloads.