Microsoft Azure
Updated
Use Microsoft Azure for speech recognition (ASR) in a Conversational AI Platform agent.
Preview documentation
Microsoft Azure provides enterprise-grade automatic speech recognition with support for multiple languages and robust noise handling capabilities, optimized for real-time conversational applications.
Sample configuration
The following example shows how to configure Microsoft Azure as the agent's ASR when you create or update an agent. For a complete example, see Create an agent and start a session.
Microsoft Azure supports BYOK only. It isn't available in managed mode. Store your Azure Speech key in a secret set, and refer to it from the agent. Your Azure region isn't secret. For details, see Use your own provider keys.
from agora_agent import Agent, MicrosoftSTT, secret_ref
# client is your configured Agora client
agent = (
Agent(client=client)
.with_stt(
MicrosoftSTT(
key=secret_ref("support_prod", "microsoft"),
region="eastus",
language="en-US",
additional_params={"phrase_list": ["agora", "conversational", "ai", "engine"]},
)
)
.with_llm(...) # configure your LLM vendor
.with_tts(...) # configure your TTS vendor
)import { Agent, MicrosoftSTT, secretRef } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withStt(
new MicrosoftSTT({
key: secretRef('support_prod', 'microsoft'),
region: 'eastus',
language: 'en-US',
additionalParams: { phrase_list: ['agora', 'conversational', 'ai', 'engine'] },
}),
)
.withLlm(/* configure your LLM vendor */)
.withTts(/* configure your TTS vendor */);// client is your configured Agora client
agent := agentkit.NewAgent(client).
WithStt(vendors.NewMicrosoftSTT(vendors.MicrosoftSTTOptions{
Key: agentkit.SecretRef("support_prod", "microsoft"),
Region: "eastus",
Language: "en-US",
AdditionalParams: map[string]interface{}{
"phrase_list": []string{"agora", "conversational", "ai", "engine"},
},
})).
WithLlm( /* configure your LLM vendor */ ).
WithTts( /* configure your TTS vendor */ )Set credential.mode to byok and credential.fields.key to a reference to your Azure Speech key in a secret set. Pass the region in params. In the following example, the key is stored as microsoft in the support_prod set:
"asr": {
"credential": {
"mode": "byok",
"fields": {
"key": "$secrets.support_prod.microsoft"
}
},
"vendor": "microsoft",
"language": "en-US",
"params": {
"language": "en-US",
"region": "eastus",
"phrase_list": ["agora", "conversational", "ai", "engine"]
}
}Key parameters
The asr block takes the following fields. In the SDKs, MicrosoftSTT sets them from its options, and passes any other Microsoft Azure parameters from its additional parameters option to params.
vendorstringSet to microsoft.
credentialobjectHow the agent authenticates with Microsoft Azure. Set mode to byok and fields.key to a $secrets.<set>.<key> reference to your Azure Speech key. Get your key from the Azure Portal.
languagestringThe language code for speech recognition, for example, en-US, es-ES, or fr-FR. See supported languages for language codes.
paramsobjectregionstringThe Azure region where the speech service is hosted, for example, eastus or westus2.
phrase_listarray[string]A list of words or phrases provided in advance to improve recognition accuracy.
Caution
The parameters listed on this page are validated for use with the Conversational AI Platform. Required parameters must be provided as documented. Any additional parameters in params are passed through directly to Microsoft Azure without validation. For a full list of supported options, refer to the Microsoft Azure Speech-to-Text documentation.