Looking for the previous docs site? Visit the legacy docs homepage. (opens in a new tab)
Agora Docs

Communicate with RTC SDK

Updated

Stream audio and video to and from RTC SDK.

If your business requires not only high-quality audio and video transmission, but also easy integration, low memory usage, and low power consumption, for applications such as smart cameras and smart doorbells, Agora recommends that you use the RTC SDK with the IoT SDK.

The following table shows the data formats supported when the IoT SDK communicates with the RTC SDK (v4.x):

RTC SDK TypeAudio formatsVideo formats
Native/third-party frameworks (Android, iOS/macOS, Windows, Electron, Unity, Flutter, React Native) and Server SDKG722, G711, Opus, AACH.264, JPEG
Web (v4.x)G722, G711, OpusH.264
Embedded systemsG711, OpusH.264, JPEG

The sending and receiving ends can communicate with each other only if the set data format is consistent and complies with the above table.

The following sections describe how to set the audio and video data formats in the SDKs.

Set up IoT SDK

Audio

IoT SDK only supports mono audio data, and the data sending interval is 20 ms. If your device captures stereo audio, convert it to mono before encoding.

To send and receive AAC audio data, set audio_frame_info_t.data_type to AUDIO_DATA_TYPE_AACLC or AUDIO_DATA_TYPE_HEAAC and specify the corresponding data size when calling the agora_rtc_send_audio_data method. The size of AAC audio data needs to be resolved by the user.

To send and receive Opus, G722, or G711 audio data:

  1. When calling agora_rtc_join_channel, set rtc_channel_options_t.audio_codec_opt.audio_codec_type to the codec type and sampling rate you want to use.
  2. When calling agora_rtc_send_audio_data, set audio_frame_info_t.data_type to AUDIO_DATA_TYPE_PCM.

Use the following recommended sampling rates and data sizes:

CodecSampling rateData size
Opus16 kHz640 bytes
Opus48 kHz1920 bytes
G72216 kHz640 bytes
G7118 kHz320 bytes

Video

In IoT SDK, H.264 video data is sent and received by default.

To send and receive JPEG video data, when calling the agora_rtc_send_video_data method, set video_frame_info_t.data_type to VIDEO_DATA_TYPE_GENERIC_JPEG.

Set up RTC SDK for Native/third-party frameworks and RTC Server SDK

Audio

In Native/third-party framework SDKs (v4.x) and RTC Server SDKs, Opus or AAC audio data is sent and received by default.

To send and receive G722 or G711 audio data, use the following methods:

RTC SDK typeConfiguration
AndroidagoraEngine.setAudioProfile(6) or agoraEngine.setParameters("{\"che.audio. custom_payload_type\":9}")
iOS/macOSagoraKit.setAudioProfile(6) or agoraKit.setParameters("{\"che.audio. custom_payload_type\":9}")
WindowsagoraEngine->setAudioProfile(6) or agora::base::AParameter apm(agoraEngine); apm->setParameters("{\"che.audio. custom_payload_type\":9}")
Electronthis.agoraEngine?.setAudioProfile(6) or this.agoraEngine?.setParameters("{\"che.audio. custom_payload_type\":9}")
Unityawait agoraEngine.setAudioProfile(profile: AudioProfileType.audioProfileIot) or await agoraEngine.setParameters("{\"che.audio. custom_payload_type\":9}")
Flutterawait agoraEngine.setAudioProfile(profile: AudioProfileType.audioProfileIot) or await agoraEngine.setParameters("{\"che.audio. custom_payload_type\":9}")
React Nativeawait agoraEngine.current?.setAudioProfile(6) or await agoraEngine.current?.setParameters("{\"che.audio. custom_payload_type\":9}")
Server C++connection->getAgoraParameter()->setParameters("{\"che.audio.custom_payload_type\":9}")
Server JavaagoraParameter.setParameters("{\"che.audio.custom_payload_type\":9}")
Server Pythonconnection.get_agora_parameter.set_parameters("{\"che.audio.custom_payload_type\":9}")
Server Goconnection.GetAgoraParameter.SetParameters("{\"che.audio.custom_payload_type\":9}")
RTC SDK typeConfiguration
AndroidagoraEngine.setParameters("{\"che.audio. custom_payload_type\":8}")
iOS/macOSagoraKit.setParameters("{\"che.audio. custom_payload_type\":8}")
Windowsagora::base::AParameter apm(agoraEngine);apm->agoraEngine->setParameters("{\"che.audio. custom_payload_type\":8}")
Electronthis.agoraEngine?.setParameters("{\"rtc.audio. custom_payload_type\":8}")
Unityawait agoraEngine.setParameters("{\"che.audio. custom_payload_type\":8}")
Flutterawait agoraEngine.setParameters("{\"che.audio. custom_payload_type\":8}")
React Nativeawait agoraEngine.current?.setParameters("{\"che.audio. custom_payload_type\":8}")
Server C++connection->getAgoraParameter()->setParameters("{\"che.audio.custom_payload_type\":8}")
Server JavaagoraParameter.setParameters("{\"che.audio.custom_payload_type\":8}")
Server Pythonconnection.get_agora_parameter.set_parameters("{\"che.audio.custom_payload_type\":8}")
Server Goconnection.GetAgoraParameter.SetParameters("{\"che.audio.custom_payload_type\":8}")
RTC SDK typeConfiguration
AndroidagoraEngine.setParameters("{\"che.audio. custom_payload_type\":0}")
iOS/macOSagoraKit.setParameters("{\"che.audio. custom_payload_type\":0}")
Windowsagora::base::AParameter apm(agoraEngine);apm->agoraEngine->setParameters("{\"che.audio. custom_payload_type\":0}")
Electronthis.agoraEngine?.setParameters("{\"rtc.audio. custom_payload_type\":0}")
Unityawait agoraEngine.setParameters("{\"che.audio. custom_payload_type\":0}")
Flutterawait agoraEngine.setParameters("{\"che.audio. custom_payload_type\":0}")
React Nativeawait agoraEngine.current?.setParameters("{\"che.audio. custom_payload_type\":0}")
Server C++connection->getAgoraParameter()->setParameters("{\"che.audio.custom_payload_type\":0}")
Server JavaagoraParameter.setParameters("{\"che.audio.custom_payload_type\":0}")
Server Pythonconnection.get_agora_parameter.set_parameters("{\"che.audio.custom_payload_type\":0}")
Server Goconnection.GetAgoraParameter.SetParameters("{\"che.audio.custom_payload_type\":0}")

Video

In Native/third-party framework SDKs (v4.x) and RTC Server SDKs, H.264 video data is sent and received by default.

To send and receive JPEG video data, use the following methods:

RTC SDK typeMethod
AndroidagoraEngine.setParameters("{\"engine.video.codec_type\": \"20\"}")
iOS/macOSagoraKit.setParameters("{\"engine.video.codec_type\": \"20\"}")
Windowsagora::base::AParameter apm(agoraEngine);
apm->setParameters("{\"engine.video.codec_type\": \"20\"}")
Electronthis.agoraEngine.setParameters("{\"engine.video.codec_type\": \"20\"}");
UnityagoraEngine.SetParameters("{\"engine.video.codec_type\": \"20\"}");
Flutterawait agoraEngine.setParameters("{\"engine.video.codec_type\": \"20\"}");
React Nativethis.engine.setParameters("{\"engine.video.codec_type\": \"20\"}");
Server C++connection->getAgoraParameter()->setParameters("{\"engine.video.codec_type\": \"20\"}");
Server JavaagoraParameter.setParameters("{\"engine.video.codec_type\": \"20\"}");
Server Pythonconnection.get_agora_parameter.set_parameters("{\"engine.video.codec_type\": \"20\"}");
Server Goconnection.GetAgoraParameter.SetParameters("{\"engine.video.codec_type\": \"20\"}");

Set up RTC SDK for Web

Audio

In RTC SDK for Web (v4.x), Opus audio data is sent and received by default.

To send and receive G722, G711 audio data, call the following method:

  • G711 (PCMA): AgoraRTC.createClient({mode: "live", codec: "h264", audioCodec: "pcma"})
  • G711 (PCMU): AgoraRTC.createClient({mode: "live", codec: "h264", audioCodec: "pcmu"})
  • G722: AgoraRTC.createClient({mode: "live", codec: "h264", audioCodec: "g722"})

Only the Web SDK version 4.9.0 or higher supports setting the audio encoding format.

Video

In Web SDK (v4.x), VP8 video data is sent and received by default. You call createClient to set the video data format to "h264".

Set up RTC SDK for embedded systems

In RTC SDK for embedded systems, Opus audio data and H.264 video data are sent and received by default. If you want to receive JPEG video data or send and receive G711 audio data, contact support@agora.io for activation.

RTC SDK for embedded systems does not support sending JPEG video data.

Contact Us

+1 (408) 879-5885

2804 Mission College Blvd.Santa Clara, CA, USA 95054

AgoraCopyright © 2026 AgoraAll rights reserved