Skip to main content

Real-time translation (Beta)

In addition to transcribing the host's audio in real time, Real-Time STT also supports translation during transcription. For example, in scenarios like international conferences, you can transcribe the host's speech, translate the transcribed content, and then push both the original text and the translation back to the channel as subtitles.

Understand the tech

Following are the key features of real-time translation:

  • Instant translation
    Live speech-to-text translation to keep conversations flowing seamlessly in real-time communication or live streaming.

  • Multi-language support
    Manage multilingual interactions with speech translation from up to two source languages into five target languages, and support for 30+ languages.

  • High accuracy
    Advanced Speech Recognition (ASR) captures spoken language and converts it accurately to text using sophisticated recognition technologies.

  • Translated captions
    Live captions are continually updated during speech, providing readable, translated text. Video Text Track (VTT) files can be stored in the cloud for future reference, AI analysis, or compliance.

  • Ultra-low Latency translation
    Seamless translation with an end-to-start latency of under 1 second and average end-to-end latency of under 3 seconds.

  • LLM integration
    Process translated text using custom large language models (LLMs) or integrate additional AI services to enhance capabilities and streamline workflows.

This page shows you how to set up translation of the transcribed content when starting a transcription task.

Prerequisites

To follow this guide, first implement basic speech-to-text transcription by following the Rest quickstart.

Implementation

When calling start to start a transcription task, set translateConfig to translate the transcribed text content.

Sample request

The following example shows you how to set up translation when transcribing. The example includes the URL, header, and body. To record and encrypt at the same time in the transcription task, refer to Record captions and Encrypt captions.


_30
curl --location --request POST 'https://api.agora.io/v2/projects/{{appId}}/rtsc/speech-to-text/tasks?builderToken={{tokenName}}' \
_30
--header 'Content-Type: application/json' \
_30
--header 'Accept: application/json' \
_30
--header 'Authorization: <http_basic_auth>' \
_30
--data '{
_30
"languages": [
_30
"zh-CN"
_30
],
_30
"maxIdleTime": 50,
_30
"rtcConfig": {
_30
"channelName": "<YourChannelName>",
_30
"subBotUid": "<YourSubscribeUid>",
_30
"pubBotUid": "<YourPublishUid>",
_30
"subscribeAudioUids": [uint]
_30
},
_30
"translateConfig": {
_30
"languages": [
_30
{
_30
"source": "zh-CN",
_30
"target": [
_30
"en-US",
_30
"ar-SA",
_30
"id-ID",
_30
"fr-FR",
_30
"ja-JP"
_30
]
_30
}
_30
]
_30
}
_30
}'

ParameterTypeDescription
sourcestring arrayThe source language for the translation.
targetarrayThe target languages for translation. You can configure up to 5 target languages. See Supported languages.

Sample response


_5
{
_5
"taskId": "4e1cc28f21486930fcb77a805af20752",
_5
"createTs": 1730974708,
_5
"status": "STARTED"
_5
}

ParameterTypeDescription
taskIdstringThe task ID.
createTsnumberThe timestamp when the task was created.
statusenumTask Status:
  • IDLE: The task is not initialized.
  • PREPARING: The task has received an initialization request.
  • PREPARED: Task initialization is complete.
  • STARTING: The task is starting.
  • CREATED: Part of the task startup is completed.
  • STARTED: All tasks are started and completed.
  • IN_PROGRESS: Transcription in progress.
  • STOPPING: The task is paused.
  • STOPPED: The ask has terminated.
  • FAILURE_STOP: Task termination has failed.
  • To query, update, or stop the transcription task, refer to the Rest quickstart.

    Supported languages

    The translation feature of Real-Time STT supports the following languages:

    LanguageParameter
    Arabic (Egypt)ar-EG
    Arabic (Jordan)ar-JO
    Arabic (Saudi Arabia)ar-SA
    Arabic (UAE)ar-AE
    Cantonese (Traditional)zh-HK
    Chinese (Simplified)zh-CN
    Chinese (Traditional)zh-TW
    Englishen-US
    Frenchfr-FR
    Germande-DE
    Hindihi-IN
    Indonesianid-ID
    Italianit-IT
    Japaneseja-JP
    Koreanko-KR
    Malayms-MY
    Persian (Iran)fa-IR
    Portuguesept-PT
    Russianru-RU
    Spanishes-ES
    Thaith-TH
    Turkishtr-TR
    Vietnamesevi-VN
    vundefined