Amazon Transcribe
Updated
Use Amazon Transcribe for speech recognition (ASR) in a Conversational AI Platform agent.
Preview documentation
Amazon provides advanced automatic speech recognition with high accuracy and support for multiple languages, designed for real-time conversational AI applications.
Sample configuration
The following example shows how to configure Amazon Transcribe as the agent's ASR when you create or update an agent. For a complete example, see Create an agent and start a session.
Amazon Transcribe supports BYOK only. It isn't available in managed mode. Store your AWS credentials in a secret set, and refer to them from the agent. For details, see Use your own provider keys.
from agora_agent import Agent, AmazonSTT, secret_ref
# client is your configured Agora client
agent = (
Agent(client=client)
.with_stt(
AmazonSTT(
access_key=secret_ref("support_prod", "amazon_access_key_id"),
secret_key=secret_ref("support_prod", "amazon"),
region="<AWS_ASR_REGION>",
language="en-US",
additional_params={"media_sample_rate_hz": 16000, "media_encoding": "pcm"},
)
)
.with_llm(...) # configure your LLM vendor
.with_tts(...) # configure your TTS vendor
)import { Agent, AmazonSTT, secretRef } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withStt(
new AmazonSTT({
accessKey: secretRef('support_prod', 'amazon_access_key_id'),
secretKey: secretRef('support_prod', 'amazon'),
region: '<AWS_ASR_REGION>',
language: 'en-US',
additionalParams: { media_sample_rate_hz: 16000, media_encoding: 'pcm' },
}),
)
.withLlm(/* configure your LLM vendor */)
.withTts(/* configure your TTS vendor */);// client is your configured Agora client
agent := agentkit.NewAgent(client).
WithStt(vendors.NewAmazonSTT(vendors.AmazonSTTOptions{
AccessKey: agentkit.SecretRef("support_prod", "amazon_access_key_id"),
SecretKey: agentkit.SecretRef("support_prod", "amazon"),
Region: "<AWS_ASR_REGION>",
Language: "en-US",
AdditionalParams: map[string]interface{}{
"media_sample_rate_hz": 16000,
"media_encoding": "pcm",
},
})).
WithLlm( /* configure your LLM vendor */ ).
WithTts( /* configure your TTS vendor */ )Set credential.mode to byok, and set credential.fields.access_key_id and credential.fields.secret_access_key to references to your AWS access key ID and secret access key in a secret set. In the following example, they're stored as amazon_access_key_id and amazon in the support_prod set:
"asr": {
"credential": {
"mode": "byok",
"fields": {
"access_key_id": "$secrets.support_prod.amazon_access_key_id",
"secret_access_key": "$secrets.support_prod.amazon"
}
},
"vendor": "amazon",
"language": "en-US",
"params": {
"language_code": "en-US",
"region": "<AWS_ASR_REGION>",
"media_sample_rate_hz": 16000,
"media_encoding": "pcm"
}
}Key parameters
The asr block takes the following fields. In the SDKs, AmazonSTT sets them from its options, and passes any other Amazon Transcribe parameters from its additional parameters option to params.
vendorstringSet to amazon.
credentialobjectHow the agent authenticates with Amazon Transcribe. Set mode to byok, and set fields.access_key_id and fields.secret_access_key to $secrets.<set>.<key> references to your AWS access key ID and secret access key. Get both from the AWS IAM Console.
languagestringThe language code for speech recognition, for example, en-US, es-US, or fr-FR. See supported languages for available language codes.
paramsobjectregionstringThe AWS region where the Transcribe service is hosted, for example, us-east-1, us-west-2, or eu-west-1. See AWS regions for available regions.
language_codestringThe language code for Amazon Transcribe. Use the same value as language.
media_sample_rate_hzintegerThe sample rate in Hertz for the audio input, for example, 16000 or 8000.
media_encodingstringThe encoding format of the audio input, for example, pcm, opus, or flac.
Caution
The parameters listed on this page are validated for use with the Conversational AI Platform. Required parameters must be provided as documented. Any additional parameters in params are passed through directly to Amazon Transcribe without validation. For a full list of supported options, refer to the Amazon official documentation.
This integration is fully supported for use with the Conversational AI Platform. While it has completed functional validation, it is newer to the platform, and additional provider-specific edge cases may be identified as usage scales across a broader range of applications and workloads.