Smallest AI
Updated
Use Smallest AI for speech recognition (ASR) in a Conversational AI Platform agent.
Preview documentation
Smallest AI provides real-time streaming transcription through its Pulse model. Use it as the ASR model in a cascading pipeline with any supported LLM and TTS vendor.
Sample configuration
The following example shows how to configure Smallest AI as the agent's ASR when you create or update an agent. For a complete example, see Create an agent and start a session.
Smallest AI supports BYOK only. It isn't available in managed mode. Store your Smallest AI API key in a secret set, and refer to it from the agent. For details, see Use your own provider keys.
The SDK converts the boolean options to the "true" and "false" strings that Smallest AI expects.
from agora_agent import Agent, SmallestAISTT, secret_ref
# client is your configured Agora client
agent = (
Agent(client=client)
.with_stt(
SmallestAISTT(
api_key=secret_ref("support_prod", "smallestai"),
language="zh",
url="wss://api.us.smallest.ai/waves/v1/stt/live",
sample_rate=16000,
encoding="linear16",
word_timestamps=True,
sentence_timestamps=True,
diarize=True,
vad_events=True,
endpointing=True,
eou_timeout_ms=480,
format=True,
finalize_on_words=True,
max_words="100",
punctuate=True,
capitalize=True,
itn_normalize=True,
full_transcript=True,
keywords="Codex:2,Smallest AI:2",
redact_pii=False,
redact_pci=False,
)
)
.with_llm(...) # configure your LLM vendor
.with_tts(...) # configure your TTS vendor
)The SDK converts the boolean options to the "true" and "false" strings that Smallest AI expects.
import { Agent, SmallestAISTT, secretRef } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withStt(
new SmallestAISTT({
apiKey: secretRef('support_prod', 'smallestai'),
language: 'zh',
url: 'wss://api.us.smallest.ai/waves/v1/stt/live',
sampleRate: 16000,
encoding: 'linear16',
wordTimestamps: true,
sentenceTimestamps: true,
diarize: true,
vadEvents: true,
endpointing: true,
eouTimeoutMs: 480,
format: true,
finalizeOnWords: true,
maxWords: '100',
punctuate: true,
capitalize: true,
itnNormalize: true,
fullTranscript: true,
keywords: 'Codex:2,Smallest AI:2',
redactPii: false,
redactPci: false,
}),
)
.withLlm(/* configure your LLM vendor */)
.withTts(/* configure your TTS vendor */);The SDK converts the boolean options to the "true" and "false" strings that Smallest AI expects.
sampleRate, eouTimeoutMs := 16000, 480
// client is your configured Agora client
agent := agentkit.NewAgent(client).
WithStt(vendors.NewSmallestAISTT(vendors.SmallestAISTTOptions{
APIKey: agentkit.SecretRef("support_prod", "smallestai"),
Language: "zh",
URL: "wss://api.us.smallest.ai/waves/v1/stt/live",
SampleRate: &sampleRate,
Encoding: "linear16",
WordTimestamps: true,
SentenceTimestamps: true,
Diarize: true,
VADEvents: true,
Endpointing: true,
EOUTimeoutMs: &eouTimeoutMs,
Format: true,
FinalizeOnWords: true,
MaxWords: "100",
Punctuate: true,
Capitalize: true,
ITNNormalize: true,
FullTranscript: true,
Keywords: "Codex:2,Smallest AI:2",
})).
WithLlm( /* configure your LLM vendor */ ).
WithTts( /* configure your TTS vendor */ )Set credential.mode to byok and credential.fields.api_key to a reference to your Smallest AI API key in a secret set. In the following example, the key is stored as smallestai in the support_prod set. Only credential and vendor are required:
"asr": {
"credential": {
"mode": "byok",
"fields": {
"api_key": "$secrets.support_prod.smallestai"
}
},
"vendor": "smallestai",
"language": "zh",
"url": "wss://api.us.smallest.ai/waves/v1/stt/live",
"params": {
"sample_rate": 16000,
"encoding": "linear16",
"word_timestamps": "true",
"sentence_timestamps": "true",
"diarize": "true",
"vad_events": "true",
"endpointing": "true",
"eou_timeout_ms": 480,
"format": "true",
"finalize_on_words": "true",
"max_words": "100",
"punctuate": "true",
"capitalize": "true",
"itn_normalize": "true",
"full_transcript": "true",
"keywords": "Codex:2,Smallest AI:2",
"redact_pii": "false",
"redact_pci": "false"
}
}Info
Smallest AI expects its boolean options as the strings "true" and "false", as shown in the example.
Key parameters
The asr block takes the following fields. In the SDKs, SmallestAISTT sets them from its options, and passes any other Smallest AI parameters from its additional parameters option to params.
| Parameter | Description |
|---|---|
vendorstring | Set to |
credentialobject | How the agent authenticates with Smallest AI. Set |
languagestring | The language code for speech recognition, for example, |
urlstring | The Smallest AI streaming ASR WebSocket endpoint, for example, |
paramsobject |
|
Caution
The parameters listed on this page are validated for use with the Conversational AI Platform. Required parameters must be provided as documented. Any additional parameters in params are passed through directly to Smallest AI without validation. For a full list of supported options, refer to the Smallest AI documentation.
This integration is fully supported for use with the Conversational AI Platform. While it has completed functional validation, it is newer to the platform, and additional provider-specific edge cases may be identified as usage scales across a broader range of applications and workloads.
