Python

Updated

Agora Server Gateway SDK API reference for Python.

Server Gateway SDK API reference for Python.

Overview

AgoraService

The AgoraService class serves as the entry point for the SDK, allowing you to create media tracks and connections.

initialize

Initializes the AgoraService object.

def initialize(self, config: AgoraServiceConfig)

Parameters

ParameterDescription
configInitialization configuration. See AgoraServiceConfig.

Returns

  • 0: Success.
  • < 0: Failure.

release

Releases the AgoraService object.

def release(self)

create_rtc_connection

Creates an RTC connection object.

def create_rtc_connection(self, con_config: RTCConnConfig)

Caution

To create multiple connections, Agora recommends using multiple processes to avoid memory leaks and other issues.

Parameters

ParameterDescription
con_configRTC connection configuration. See RTCConnConfig.

Returns

  • RTCConnection object, if the method call succeeds.
  • None, if the method call fails.

create_custom_audio_track_pcm

Creates a local audio track object with an audio source that sends PCM audio data. This track object can be used to send audio data in PCM format.

def create_custom_audio_track_pcm(self, audio_pcm_data_sender: AudioPcmDataSender)

Parameters

ParameterDescription
audio_pcm_data_senderPCM format audio data source. See AudioPcmDataSender.

Returns

  • LocalAudioTrack object, if the method call succeeds.
  • None, if the method call fails.

create_custom_audio_track_encoded

Creates a local audio track object with an audio source that sends encoded audio data. This track object can be used to send encoded audio data, such as Opus format encoded data.

def create_custom_audio_track_encoded(self, audio_encoded_frame_sender: AudioEncodedFrameSender, mix_mode: int)

Parameters

ParameterDescription
audio_encoded_frame_senderAudio source that sends encoded audio data. See AudioEncodedFrameSender.
mix_modeMix mode for the encoded audio in the channel. - 0: Mix all audio tracks in the channel. - 1: Do not mix audio.

Returns

  • LocalAudioTrack object, if the method call succeeds.
  • None, if the method call fails.

create_media_node_factory

Creates a media node factory object.

def create_media_node_factory(self)

Returns

  • MediaNodeFactory object, if the method call succeeds.
  • None, if the method call fails.

create_custom_video_track_frame

Creates a local video track object with a video source that sends YUV format video data. This track object can be used to send video data in YUV format.

def create_custom_video_track_frame(self, video_frame_sender: VideoFrameSender)

Parameters

ParameterDescription
video_frame_senderYUV format video source object. See VideoFrameSender.

Returns

  • LocalVideoTrack object, if the method call succeeds.
  • None, if the method call fails.

create_custom_video_track_encoded

Creates a local video track object with a video source that sends encoded video data. This track object can be used to send encoded video data, such as H.264 or VP8 format encoded data.

def create_custom_video_track_encoded(self, video_encoded_frame_sender: VideoEncodedImageSender, options: SenderOptions)

Parameters

ParameterDescription
video_encoded_frame_senderVideo source that sends encoded video data. See VideoEncodedImageSender.
optionsVideo track settings. See SenderOptions.

Returns

  • LocalVideoTrack object, if the method call succeeds.
  • None, if the method call fails.

set_log_file

Sets the path and size of the SDK log file.

def set_log_file(self, log_path: str, log_size: int = 512 * 1024)

The SDK generates 5 SDK log files and 5 API call log files by default, following these rules:

  • The SDK log files names: agorasdk.log, agorasdk.1.log, agorasdk.2.log, agorasdk.3.log, agorasdk.4.log.
  • The API call log files names: agoraapi.log, agoraapi.1.log, agoraapi.2.log, agoraapi.3.log, agoraapi.4.log.
  • Each SDK log file has the default size of 1,024 KB; each API call log file has the default size of 2,048 KB. All log files are UTF-8 encoded.
  • The latest logs are always written to agorasdk.log and agoraapi.log.

When agorasdk.log is full, the SDK handles the log files as follows:

  • Deletes the agorasdk.4.log file, if it exists.
  • Renames agorasdk.3.log to agorasdk.4.log.
  • Renames agorasdk.2.log to agorasdk.3.log.
  • Renames agorasdk.1.log to agorasdk.2.log.
  • Creates a new agorasdk.log file.

The agoraapi.log file follows the same rotation rules as agorasdk.log.

Parameters

ParameterDescription
log_pathThe full path of the log file. The log file is UTF-8 encoded.
log_sizeThe size of a single agorasdk.log log file in KB, with a range of [128, 20480]. The default is 1,024 KB. If you set file_size to less than 128 KB, the SDK automatically adjusts it to 128 KB; if you set file_size to more than 20,480 KB, the SDK automatically adjusts it to 20,480 KB.

Returns

  • 0: Success.
  • < 0: Failure.

RTCConnection

The RtcConnection class is used to manage the connection between the SDK and the RTC channel. After the connection is complete, you can send and receive media streams in the RTC channel through the LocalUser object. Connecting to a channel is an asynchronous operation, and you can use the IRtcConnectionObserver object to listen for connection status changes and events. The RtcConnection object also monitors whether remote users join or leave the channel.

connect

Connects to an RTC channel.

def connect(self, token: str, chan_id: str, user_id: str) -> int

When the method is successful, the connection status changes from to 2.

Depending on whether the connection is successful, the connection status changes to 3 or 5. The SDK triggers the on_connected or on_disconnected callback to notify you of the connection status change.

Parameters

ParameterDescription
tokenRTC token. See Use tokens.
chan_idChannel name. Must be a string and no more than 64 bytes. The supported character types include: - All lowercase letters: a to z; - All uppercase letters: A to Z; - All Arabic numerals: 0 to 9; - Space; - Punctuation and other special symbols: !, #, $, %, &, (, ), +, -, :, ;, <, =, ., >, ?, @, [, ], ^, _, {, }, |, ~, ,
user_idThe local user ID. No more than 64 bytes. If you do not set the user ID or set it to None, the SDK automatically returns a user ID in the on_connected callback. You must record and manage this ID yourself. Depending on the user ID type you selected in the initialize method, the supported character set is different: The supported character set for integer user IDs includes 10 digits 0-9. The supported character set for string user IDs includes: - 26 lowercase English letters: a-z; - 26 uppercase English letters: A-Z; - 10 digits: 0-9; - Space; - !, #, $, %, &, (, ), +, -, :, ;, <, =, ., >, ?, @, [, ], ^, _, {, }, |, ~, ,

Returns

  • 0: Success.
  • < 0: Failure.
    • -2: The parameters are invalid.
    • -8: The current connection status is not 1.

disconnect

Disconnects from the RTC channel. When the SDK disconnects from the channel, the connection status changes to 1. You can also learn about the disconnection through the on_disconnected callback.

def disconnect(self) -> int

Returns

  • 0: Success.
  • < 0: Failure.

renew_token

Renews the token. The token expires after a certain period of time. When you receive the on_token_privilege_will_expire callback, you must generate a new Token from your server and use this method to update the token. Otherwise, the SDK disconnects from the channel.

def renew_token(self, token: str) -> int

Parameters

ParameterDescription
tokenThe new token.

Returns

  • 0: Success.
  • < 0: Failure.

register_observer

Registers the RTC connection observer.

def register_observer(self, conn_observer: IRTCConnectionObserver) -> int

Parameters

ParameterDescription
conn_observerIRTCConnectionObserver object.

Returns

  • 0: Success.
  • < 0: Failure.

unregister_observer

Destroys the RTC connection observer.

def unregister_observer(self) -> int

Returns

  • 0: Success.
  • < 0: Failure.

create_data_stream

Creates a data stream.

def create_data_stream(self, reliable: bool, ordered: bool) -> int

Each user can create up to 5 data streams in each channel.

Note

  • Agora does not support setting reliable to 1 and ordered to 0.
  • If you need the data packet to arrive immediately at the receiver, do not set the ordered parameter to 1.

Parameters

ParameterDescription
reliableWhether the data stream is reliable: - 1: The receiver receives the data sent by the sender within 5 seconds. - 0: The receiver does not guarantee to receive the data. If the data is lost, it does not report an error.
orderedWhether the data stream is ordered: - 1: The receiver receives the data packets in the order sent by the sender. - 0: The receiver does not guarantee to receive the data packets in the order sent by the sender.

Returns

  • stream_id: Returns the data stream ID, if the method call succeeds.
  • None: Failure.

send_stream_message

Sends a data stream message.

def send_stream_message(self, stream_id: int, data: str) -> int

The method sends a data stream message to all users in the channel. The SDK implements the following restrictions for this method:

  • Up to 30 packets per second, and each packet is no more than 1 KB.
  • Up to 6 KB of data per second per client.
  • Each client can have up to 5 data channels at the same time.

After successfully calling this method, the remote end triggers the on_stream_message callback, and remote users can obtain the received stream messages in this callback.

Note

  • Ensure that you have called create_data_stream before calling this method.
  • This method is only applicable to the host in live streaming use-cases.

Parameters

ParameterDescription
stream_idData stream ID. You can obtain it through create_data_stream.
dataData to send.

Returns

  • 0: Success.
  • < 0: Failure.

get_agora_parameter

Gets the AgoraParameter object.

def get_agora_parameter(self)

Returns

  • AgoraParameter object, if the method call succeeds.
  • None, if the method call fails.

get_local_user

Gets the LocalUser object. Each connection corresponds to one local user.

def get_local_user(self)

Returns

  • LocalUser object, if the method call succeeds.
  • None, if the method call fails.

release

Releases the RTC connection.

def release(self)

enable_encryption

def enable_encryption(self, enabled: int, config: EncryptionConfig)

Enables encryption.

Since

v2.2.2.

This method needs to be called before connect. You need to set the encryption mode and encryption configuration in this method.

Parameters

enabled

Whether to enable encryption functionality:

  • 1: Enable
  • 0: Disable

config

The encryption configuration: EncryptionConfig.

IRTCConnectionObserver

The IRTCConnectionObserver class, used to listen to the connection status of the SDK.

on_connected

Occurs when the connection state of the SDK changes to 3.

def on_connected(self, agora_rtc_conn, conn_info, reason)

Parameters

agora_rtc_conn

The RTC connection object. See RtcConnection.

conn_info

Connection information. See RTCConnInfo.

reason

Connection state change reason:

  • 0: Establishing a network connection.

  • 1: Successfully joined the channel.

  • 2: Network connection interrupted.

  • 3: Network connection forbidden by the server.

  • 4: Failed to join the channel.

  • 5: Left the channel.

  • 6: Invalid App ID. Please change to a valid App ID and try again.

  • 7: Invalid channel name. Please change to a valid channel name and try again.

  • 8: The generated token is invalid. Generally, the following reasons:

    • App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token.
    • The uid specified in connect when joining the channel does not match the uid used when generating the Token.
  • 9: The generated Token is invalid.

  • 10: This user is banned by the server.

  • 11: SDK attempts to reconnect due to setting a proxy server.

  • 12: The network connection state changes due to updating the Token.

  • 13: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator.

  • 14: SDK connection to the server times out, enters the automatic reconnection state.

  • 15: Rejoined the channel successfully.

  • 16: SDK and the server lose connection.

  • 17: The connection state changes due to the echo test.

  • 18: The user changed the local IP address.

on_disconnected

Occurs when the connection state of the SDK changes to 1.

def on_disconnected(self, agora_rtc_conn, conn_info, reason)

Parameters

agora_rtc_conn

The RTC connection object. See RtcConnection.

conn_info

The connection information. See RTCConnInfo.

reason

Connection state change reason:

  • 0: Establishing a network connection.

  • 1: Successfully joined the channel.

  • 2: Network connection interrupted.

  • 3: Network connection forbidden by the server.

  • 4: Failed to join the channel.

  • 5: Left the channel.

  • 6: Invalid App ID. Please change to a valid App ID and try again.

  • 7: Invalid channel name. Please change to a valid channel name and try again.

  • 8: The generated token is invalid. Generally, the following reasons:

    • App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token.
    • The uid specified in connect when joining the channel does not match the uid used when generating the Token.
  • 9: The generated Token is invalid.

  • 10: This user is banned by the server.

  • 11: SDK attempts to reconnect due to setting a proxy server.

  • 12: The network connection state changes due to updating the Token.

  • 13: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator.

  • 14: SDK connection to the server times out, enters the automatic reconnection state.

  • 15: Rejoined the channel successfully.

  • 16: SDK and the server lose connection.

  • 17: The connection state changes due to the echo test.

  • 18: The user changed the local IP address.

on_token_privilege_will_expire

Occurs when the token will expire in 30 seconds.

def on_token_privilege_will_expire(self, agora_rtc_conn, token)

In the call process, if the Token is about to expire, the SDK triggers this callback 30 seconds before the Token expires to remind the app to update the Token.

When you receive this callback, you need to generate a new Token on the server side and call renew_token to pass the newly generated Token to the SDK.

Parameters

ParameterDescription
agora_rtc_connRTC connection object. See RtcConnection.
tokenThe token that is about to expire.

on_token_privilege_did_expire

Occurs when the token expires.

def on_token_privilege_did_expire(self, agora_rtc_conn)

In the call process, if the token expires, the SDK triggers this callback to remind the app to update the Token.

When you receive this callback, you need to generate a new Token on the server side and call connect to reconnect the channel.

Parameters

ParameterDescription
agora_rtc_connRTC connection object. See RtcConnection.

on_user_joined

Occurs when a remote user joins the channel. You can get the remote user's ID through this callback.

def on_user_joined(self, agora_rtc_conn, user_id)

Parameters

ParameterDescription
agora_rtc_connRTC connection object. See RtcConnection.
user_idThe remote user's ID.

on_user_left

Occurs when a remote user leaves the channel. You can get the reason for the remote user leaving the channel through the reason parameter.

def on_user_left(self, agora_rtc_conn, user_id, reason)

Parameters

agora_rtc_conn

The RTC connection object. See RtcConnection.

user_id

The remote user's ID.

reason

Connection state change reason:

  • 0: Establishing a network connection.

  • 1: Successfully joined the channel.

  • 2: Network connection interrupted.

  • 3: Network connection forbidden by the server.

  • 4: Failed to join the channel.

  • 5: Left the channel.

  • 6: Invalid App ID. Please change to a valid App ID and try again.

  • 7: Invalid channel name. Please change to a valid channel name and try again.

  • 8: The generated token is invalid. Generally, the following reasons:

    • App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token.
    • The uid specified in connect when joining the channel does not match the uid used when generating the Token.
  • 9: The generated Token is invalid.

  • 10: This user is banned by the server.

  • 11: SDK attempts to reconnect due to setting a proxy server.

  • 12: The network connection state changes due to updating the Token.

  • 13: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator.

  • 14: SDK connection to the server times out, enters the automatic reconnection state.

  • 15: Rejoined the channel successfully.

  • 16: SDK and the server lose connection.

  • 17: The connection state changes due to the echo test.

  • 18: The user changed the local IP address.

on_stream_message_error

Occurs when a data stream message error occurs. This callback indicates that the local user has not received the stream message sent by the remote user using the send_stream_message method.

def on_stream_message_error(self, agora_rtc_conn, user_id_str, stream_id, code, missed, cached)

Parameters

ParameterDescription
agora_rtc_connThe RTC connection object. See RtcConnection.
user_id_strThe user ID that sends the message.
stream_idThe stream ID of the received message.
codeThe code of the error that has occurred.
missedThe number of lost messages.
cachedThe number of messages cached when the data stream is interrupted.

on_error

Occurs when there is an error. This callback indicates that an error related to the network or media occurred during the SDK runtime. Generally, the error reported by the SDK means that the SDK cannot automatically recover and requires intervention from the App or user prompt.

def on_error(self, agora_rtc_conn, error_code, error_msg)

Parameters

ParameterDescription
agora_rtc_connThe RTC connection object. See RtcConnection.
error_codeThe error code.
error_msgThe error description.

on_encryption_error

def on_encryption_error(self, agora_rtc_conn, error_type)

Occurs when encryption errors occur.

Since

v2.2.2.

Parameters

ParameterDescription
agora_rtc_connRtcConnection object.
error_typeError code.

LocalUser

The LocalUser class. Defines the behavior and state of the local user. Each RTC connection has its own user. You can get the local user through get_local_user. Each user has two roles: the host (can send and receive audio and video streams) and the audience (can only receive audio and video streams).

set_user_role

Sets the user role. You can call this method before or after the user connects to the channel.

def set_user_role(self, role)

Note

If the role set in the token parameter of connect is different from the role set here, the connection will fail.

Parameters

ParameterDescription'
roleUser role. See ClientRoleType.

Returns

  • 0: Success.
  • < 0: Failure.

set_audio_encoder_configuration

Sets the audio encoding. The SDK applies the configuration to all audio tracks sent.

def set_audio_encoder_configuration(self, config)

Parameters

ParameterDescription
configAudio encoding configuration. See AudioEncoderConfiguration.

Returns

  • 0: Success.
  • < 0: Failure.

publish_audio

Publishes the local audio track to the channel. By default, all published audio tracks are mixed.

def publish_audio(self, agora_local_audio_track:LocalAudioTrack)

Parameters

ParameterDescription
agora_local_audio_trackThe local audio track to publish. See LocalAudioTrack.

Returns

  • 0: Success.
  • < 0: Failure.
    • -5: If the local user's role is not the host, this error is returned.

unpublish_audio

Stops publishing the local audio track to the channel.

def unpublish_audio(self, agora_local_audio_track:LocalAudioTrack)

Parameters

ParameterDescription
agora_local_audio_trackThe local audio track to stop publishing. See LocalAudioTrack.

Returns

  • 0: Success.
  • < 0: Failure.

publish_video

Publishes the local video track to the channel.

def publish_video(self, agora_local_video_track:LocalVideoTrack)

Parameters

ParameterDescription
agora_local_video_trackThe local video track to publish. See LocalVideoTrack.

Returns

  • 0: Success.
  • < 0: Failure.

unpublish_video

Stops publishing the local video track to the channel.

def unpublish_video(self, agora_local_video_track:LocalVideoTrack)

Parameters

ParameterDescription
agora_local_video_trackThe local video track to stop publishing. See LocalVideoTrack.

Returns

  • 0: Success.
  • < 0: Failure.

subscribe_audio

Subscribes to the audio stream sent by a specific remote user in the channel.

def subscribe_audio(self, user_id)

Parameters

ParameterDescription
user_idThe remote user ID.

Returns

  • 0: Success.
  • < 0: Failure.
    • -2: If the user does not exist or user_id is invalid, this error is returned.

subscribe_all_audio

Subscribes to all the audio streams sent by remote users in the channel. This method automatically subscribes to the audio streams sent by users who join the channel later.

def subscribe_all_audio(self)

Returns

  • 0: Success.
  • < 0: Failure.

unsubscribe_audio

Stops subscribing to the audio stream sent by a specific remote user in the channel.

def unsubscribe_audio(self, user_id)

Parameters

ParameterDescription
user_idThe remote user ID.

Returns

  • 0: Success.
  • < 0: Failure.
    • -2: If the user does not exist or user_id is invalid, this error is returned.

unsubscribe_all_audio

Stops subscribing to all the audio streams sent by remote users in the channel. This method automatically cancels the subscription of the audio streams sent by users who join the channel later. You must call subscribe_audio or subscribe_all_audio to restore the subscription of audio.

def unsubscribe_all_audio(self)

Returns

  • 0: Success.
  • < 0: Failure.

set_playback_audio_frame_before_mixing_parameters

Sets the audio data format reported by on_playback_audio_frame_before_mixing.

def set_playback_audio_frame_before_mixing_parameters(self, channels, sample_rate_hz)

Parameters

ParameterDescription
channelsThe number of audio channels, which can be set to 1 (mono) or 2 (stereo).
sample_rate_hzThe sample rate of the audio data, which can be set to 8000, 16000, 32000, 44100, or 48000.

Returns

  • 0: Success.
  • < 0: Failure.

register_audio_frame_observer

Registers the audio frame observer. You must implement the IAudioFrameObserver class and register the callback based on your needs.

def register_audio_frame_observer(self, observer: IAudioFrameObserver, enable_vad: int, vad_configure)

Parameters

observer

IAudioFrameObserver class.

enable_vad

Whether to enable the built-in voice activity detection (VAD):

vad_configure

The VAD configuration, an AudioVadConfigV2 object. The SDK uses this parameter only when enable_vad is 1.

Returns

  • 0: Success.
  • < 0: Failure.

unregister_audio_frame_observer

Destroys the IAudioFrameObserver object.

def unregister_audio_frame_observer(self)

Returns

  • 0: Success.
  • < 0: Failure.

register_video_encoded_frame_observer

Registers the encoded video frame observer.

def register_video_encoded_frame_observer(self, agora_video_encoded_frame_observer:IVideoEncodedFrameObserver)

Parameters

ParameterDescription
agora_video_encoded_frame_observerThe encoded video frame observer. See IVideoEncodedFrameObserver.

Returns

  • 0: Success.
  • < 0: Failure.

unregister_video_encoded_frame_observer

Destroys the encoded video frame observer.

def unregister_video_encoded_frame_observer(self)

Returns

  • 0: Success.
  • < 0: Failure.

subscribe_video

Subscribes to the video stream sent by a specific remote user in the channel.

def subscribe_video(self, user_id, options:VideoSubscriptionOptions)

Parameters

ParameterDescription
user_idThe remote user ID.
optionsThe subscription options. See VideoSubscriptionOptions.

Returns

  • 0: Success.
  • < 0: Failure.
    • -2: If the user does not exist or user_id is invalid, this error is returned.

subscribe_all_video

Subscribes to all the video streams sent by remote users in the channel. This method automatically subscribes to the video streams sent by users who join the channel later.

def subscribe_all_video(self, options:VideoSubscriptionOptions)

Parameters

ParameterDescription
optionsThe subscription options. See VideoSubscriptionOptions.

Returns

  • 0: Success.
  • < 0: Failure.

unsubscribe_video

Stops subscribing to the video stream sent by a specific remote user in the channel.

def unsubscribe_video(self, user_id)

Parameters

ParameterDescription
user_idThe remote user ID.

Returns

  • 0: Success.
  • < 0: Failure.
    • -2: If the user does not exist or user_id is invalid, this error is returned.

unsubscribe_all_video

Stops subscribing to all the video streams sent by remote users in the channel. This method automatically cancels the subscription of the video streams sent by users who join the channel later. You must call subscribe_video or subscribe_all_video to restore the subscription.

def unsubscribe_all_video(self)

Returns

  • 0: Success.
  • < 0: Failure.

register_local_user_observer

Registers the local user observer.

def register_local_user_observer(self, observer:ILocalUserObserver)

Parameters

ParameterDescription
observerLocalUserObserver class.

Returns

  • 0: Success.
  • < 0: Failure.

unregister_local_user_observer

Destroys the local user observer.

def unregister_local_user_observer(self)

Returns

  • 0: Success.
  • < 0: Failure.

register_video_frame_observer

Registers the IVideoFrameObserver object.

def register_video_frame_observer(self, agora_video_frame_observer2:IVideoFrameObserver)

Parameters

ParameterDescription
agora_video_frame_observer2IVideoFrameObserver class.

Returns

  • 0: Success.
  • < 0: Failure.

unregister_video_frame_observer

Destroys the IVideoFrameObserver object.

def unregister_video_frame_observer(self)

Returns

  • 0: Success.
  • < 0: Failure.

get_rtc_connection

Gets the RtcConnection object.

def get_rtc_connection(self)

Returns

  • RtcConnection object: Success.
  • None: Failure.

release

Releases all resources of the LocalUser object.

def release(self)

send_audio_meta_data

def send_audio_meta_data(self, data)

Sends the audio metadata.

Since

v2.2.0.

A successful call of this method triggers the on_audio_metadata_received callback on the remote end. The remote user can get the audio metadata in this callback.

Parameters

ParameterDescription
dataThe audio metadata.

Returns

  • 0: Success.
  • < 0: Failure.

LocalUserObserver

The LocalUserObserver class.

on_local_audio_track_state_changed

Triggered when the local audio track state changes.

def on_local_audio_track_state_changed(self, agora_local_user, agora_local_audio_track, state, error)

Parameters

ParameterDescription
agora_local_userLocalUserObserver class.
agora_local_audio_trackLocal audio track. See LocalAudioTrack.
stateThe current state of the audio track.
errorError information.

on_user_audio_track_state_changed

Occurs when the remote audio track state changes.

def on_user_audio_track_state_changed(self, agora_local_user, user_id, agora_remote_audio_track, state, reason, elapsed)

Parameters

ParameterDescription
agora_local_userThe LocalUserObserver class.
user_idThe remote user ID.
agora_remote_audio_trackRemote audio track.
stateThe current state of the audio track.
reasonThe reason for the state change.
elapsedThe time elapsed after the remote user connects to the channel (ms).

on_stream_message

Callback for receiving data stream messages. This callback indicates that the local user has received the stream message sent by the remote user through the send_stream_message method.

def on_stream_message(self, agora_local_user, user_id, stream_id, data, length)

Parameters

ParameterDescription
agora_local_userLocalUserObserver class.
user_idThe user ID that sends the message.
stream_idThe Stream ID of the received message.
dataThe received data.
lengthThe data length (bytes).

on_user_info_updated

Occurs when the user media information is updated.

def on_user_info_updated(self, agora_local_user, user_id, msg, val)

Parameters

ParameterDescription
agora_local_userThe LocalUserObserver class.
user_idThe user ID that sends the message.
msgUser media information: - 0: The user has stopped publishing audio streams. - 1: The user has stopped publishing video streams. - 4: The user has enabled video functionality.
valWhether the user stops sending streams. - 0: Do not stop sending streams. - 1: Stop sending streams.

on_audio_meta_data_received

def on_audio_meta_data_received(self, agora_local_user, user_id, data):

Occurs when the remote audio metadata is received.

This callback indicates that the local user has received the audio metadata sent by the remote user using the send_audio_meta_data method.

Parameters

ParameterDescription
agora_local_userThe LocalUserObserver class
user_idThe ID of the user who sends the audio metadata.
dataThe audio metadata.

LocalAudioTrack

The LocalAudioTrack class is the base class for the local audio track.

set_enabled

Enables or disables the local audio track. After the local audio track is enabled, the SDK can capture, process, and encode local audio.

def set_enabled(self, enable)

Parameters

ParameterDescription
enableWhether to enable the audio track: - 1: Enable the audio track. - 0: Disable the audio track.

adjust_publish_volume

Adjusts the audio publishing volume.

def adjust_publish_volume(self, volume)

Parameters

ParameterDescription
volumeThe audio publishing volume. The value range is from [0,100]. 0 means mute, and 100 means the original volume.

Returns

  • 0: Success.
  • < 0: Failure.

set_max_buffer_audio_frame_number

Sets the maximum capacity of the audio sending buffer.

def set_max_buffer_audio_frame_number(self, num)

In AI interaction applications, the frequency of sending audio frames by the sender may be uneven, for example, sending 1 frame every 20 ms or 3 frames every 20 ms. If the SDK sends data according to the actual sending frequency to the Agora server, the receiver may experience stuttering or frame loss. To optimize the audio transmission performance, the SDK sets an audio sending buffer. Regardless of the actual sending frequency of audio frames by the sender, the SDK stores data in the audio sending buffer. After the data in the audio sending buffer reaches the minimum allowed sending time, the SDK starts sending data to the receiver at a frequency of 10 ms per frame.

The default maximum capacity of the audio sending buffer is 6000, which means it can store up to 6000 frames of data, or 60 seconds of data. If you need a larger buffer capacity, you can call this method to adjust it. If you also need to set the minimum allowed sending time for the audio sending buffer, call the set_send_delay_ms method.

Note

This method must be called before the publish_audio method.

Parameters

ParameterDescription
numThe maximum number of audio frames that the buffer can store. For example, if you enter 6000, it means 6000 frames of audio data.

clear_sender_buffer

Clears the current audio sending buffer.

def clear_sender_buffer(self)

If you no longer need the data in the current audio sending buffer, call this method to clear the buffer. For example, a user asks AI a question, and AI is answering the question through the SDK. At this time, the audio sending buffer of the SDK is constantly receiving and sending data. When the user interrupts AI's reply and asks another question, you can call this method to clear the buffered data that has not been sent out, so that the SDK can quickly buffer the AI's response to the new topic.

Returns

  • 0: Success.
  • < 0: Failure.

set_send_delay_ms

Sets the minimum data duration that the audio sending buffer allows to send.

def set_send_delay_ms(self, delay_ms:int)

The default minimum data duration that the SDK allows to send is 160 ms, which means the SDK waits until the audio sending buffer is filled with 160 ms of data before sending the data to the receiver. This will have the following effects:

  • If the sender sends data that is greater than or equal to 160 ms, for example 800 ms, the SDK sends the data at a frequency of 10 ms per frame.
  • If the sender sends data that is less than 160 ms, for example 100 ms, the SDK waits until the buffer data reaches 160 ms before sending the data at a frequency of 10 ms per frame. This will cause a delay of up to 60 ms.

You can adjust the minimum data duration according to your actual business needs by calling this method.

Parameters

ParameterDescription
delay_msThe minimum data duration that the buffer allows to send (ms). This parameter must be set to a multiple of 10, and its value must be greater than 0 and less than 10 times the value of the number parameter in the set_max_buffer_audio_frame_number method.

release

Releases the local audio track resources.

def release(self)

LocalVideoTrack

The LocalVideoTrack class. Provides basic methods for the local video track. You can publish one or multiple video tracks through the publish_video method.

set_enabled

Enables or disables the local video track. After the local video track is enabled, the SDK can capture, process, and encode the local video.

def set_enabled(self, enable)

Parameters

ParameterDescription
enableEnable or disable the local video track: - 0: Disable the local video track. - 1: Enable the local video track.

set_video_encoder_configuration

Sets the video encoding properties.

def set_video_encoder_configuration(self, config)

Parameters

ParameterDescription
configVideo encoding parameter configuration. See VideoEncoderConfig.

Returns

  • 0: Success.
  • < 0: Failure.

MediaNodeFactory

The MediaNodeFactory class.

create_audio_pcm_data_sender

Creates a PCM data sender module. This method creates an AudioPcmDataSender object, which can be used in the create_custom_audio_track_pcm method.

def create_audio_pcm_data_sender(self)

Returns

  • AudioPcmDataSender object, if the method call succeeds.
  • None, if the method call fails.

create_audio_encoded_frame_sender

Creates an encoded audio data sender module. This method creates an AudioEncodedFrameSender object, which can be used in the create_custom_audio_track_encoded method.

def create_audio_encoded_frame_sender(self)

Returns

  • AudioEncodedFrameSender object, if the method call succeeds.
  • None, if the method call fails.

create_video_frame_sender

Creates a YUV video frame sender module. This method creates a VideoFrameSender object, which can be used in the create_custom_video_track_frame method.

def create_video_frame_sender(self)

Returns

  • VideoFrameSender object, if the method call succeeds.
  • None, if the method call fails.

create_video_encoded_image_sender

Creates an encoded video sender module. This method creates a VideoEncodedImageSender object, which can be used in the create_custom_video_track_encoded method.

def create_video_encoded_image_sender(self)

Returns

  • VideoEncodedImageSender object, if the method call succeeds.
  • None, if the method call fails.

release

Releases the MediaNodeFactory object.

def release(self)

AudioPcmDataSender

The AudioPcmDataSender class.

send_audio_pcm_data

Sends PCM audio data to the local audio track.

def send_audio_pcm_data(self, frame: PcmAudioFrame)

Parameters

ParameterDescription
frameThe PcmAudioFrame object, containing the PCM audio data to send.

Returns

  • 0: Success.
  • < 0: Failure.

AudioEncodedFrameSender

The AudioEncodedFrameSender Class.

send_encoded_audio_frame

Sends an encoded audio frame to the audio track.

def send_encoded_audio_frame(self, buffer_ptr: int, buffer_size: int, frame_info: EncodedAudioFrameInfo)

Parameters

ParameterDescription
buffer_ptrThe encoded audio data to send.
buffer_sizeThe size of the data (bytes).
frame_infoThe audio frame information. See EncodedAudioFrameInfo.

Returns

  • 0: Success.
  • < 0: Failure.

IAudioFrameObserver

The IAudioFrameObserver class.

on_playback_audio_frame_before_mixing

Gets the data before the playback audio is mixed.

def on_playback_audio_frame_before_mixing(self, agora_local_user, channelId, uid, frame: AudioFrame, vad_result_state: int, vad_result_bytearray: bytearray)

Parameters

agora_local_user

The LocalUserObserver class.

channelId

The channel name.

uid

The remote user ID.

frame

The audio raw data. See AudioFrame.

vad_result_state

The VAD state. Valid only when you enable VAD in register_audio_frame_observer:

  • < 0: VAD is not enabled.
  • 0: Not speaking.
  • 1: Started speaking.
  • 2: Speaking.
  • 3: Stopped speaking.

vad_result_bytearray

The audio data processed by VAD. When VAD is enabled, use this data instead of frame for downstream processing, such as speech recognition.

Returns

  • 0:The audio frame data is invalid, and the data is discarded.
  • 1:The audio frame data is valid, and the data is sent.

VideoFrameSender

The VideoFrameSender class.

send_video_frame

Sends a YUV format video frame to the video track.

def send_video_frame(self, frame:ExternalVideoFrame)

Parameters

ParameterDescription
frameThe video frame to send, see ExternalVideoFrame.

Returns

  • 0: Success.
  • < 0: Failure.

IVideoFrameObserver

The IVideoFrameObserver class.

on_frame

def on_frame(self, channel_id, remote_uid, frame:VideoFrame)

Occurs when a video frame is received.

Parameters

ParameterDescription
channel_idThe channel name.
remote_uidThe remote user ID.
frameThe video data. See VideoFrame

VideoEncodedImageSender

The VideoEncodedImageSender class.

send_encoded_video_image

Sends an encoded video frame to the video track.

def send_encoded_video_image(self, buffer_ptr:int, buffer_size:int, frame_info:EncodedVideoFrameInfo)

Parameters

ParameterDescription
buffer_ptrThe video buffer.
buffer_sizeThe size of the video data (bytes).
frame_infoThe encoded video frame information: see EncodedVideoFrameInfo.

Returns

  • 0: Success.
  • < 0: Failure.

IVideoEncodedFrameObserver

The IVideoEncodedFrameObserver class.

on_encoded_video_frame

Occurs when an encoded video frame is received.

def on_encoded_video_frame(self, uid, image_buffer, length, video_encoded_frame_info:EncodedVideoFrameInfo)

Parameters

ParameterDescription
uidThe remote user ID.
image_bufferThe video frame buffer.
lengthThe length of the video frame data.
video_encoded_frame_infoThe encoded video frame information. See EncodedVideoFrameInfo

Returns

Whether to accept the encoded video frame:

  • 0: Do not accept.
  • 1: Accept.

Data type

Classes

AgoraServiceConfig

Global configuration of the AgoraServiceConfig class. Set when calling the initialize method.

@dataclass(kw_only=True)
class AgoraServiceConfig:
    log_path: str = ""
    log_size: int = 0
    enable_audio_processor: int = 1
    enable_audio_device: int = 0
    enable_video: int = 0
    context: object = None
    appid: str = ""
    area_code: int = AreaCode.AREA_CODE_GLOB.value
    channel_profile: ChannelProfileType = ChannelProfileType.CHANNEL_PROFILE_LIVE_BROADCASTING
    audio_scenario: AudioScenarioType = AudioScenarioType.AUDIO_SCENARIO_CHORUS
    use_string_uid: int = 0
    domain_limit: int = 0
    should_callbck_when_muted: int = 0

log_path

The full path to the log file. If this parameter is set to None, the log is located in the current application's pwd directory

log_size

The size of a single log file in KB. The default value is 1024 KB. If you set this parameter to 1024 KB, the SDK outputs a maximum of 5 MB of log files. If you set this parameter to less than 1024 KB, the setting does not take effect, and the maximum size of a single log file is still 1024 KB.

enable_audio_processor

Whether to enable the audio processing module. - 0: Disable the audio processing module. You will not be able to create an audio track. - 1: (Default) Enable the audio processing module.

enable_audio_device

Whether to enable the Audio Device Module (ADM). ADM is used to manage audio devices such as recording and playing audio. - 0: Disable the audio device module. You cannot record or play audio. - 1: (Default) Enable the audio device module. You can record and play audio. Note: If you set enable_audio_device to 0 and enable_audio_processor to 1, you will not be able to use audio devices, but you can push PCM audio data.

enable_video

Whether to enable video. - 0: (Default) Disable video. - 1: Enable video.

context

The context.

appid

Your Agora project's App ID. If your Agora project has enabled Token authentication, you can also assign this parameter to an RTC Token. See Agora account management for details.

area_code

The access region of the server. This feature is an advanced setting suitable for scenarios with access security restrictions. The area code supports bitwise operations. After specifying the access region, the SDK connects to Agora servers within the specified region. See AreaCode for details.

channel_profile

Channel profile. See ChannelProfileType for details.

audio_scenario

Audio encoding properties. See AudioScenarioType for details.

use_string_uid

Whether to use String user IDs.

  • 0: Do not use.
  • 1: Use.

domain_limit

Whether to enable domain limit:

  • 1: Yes. Only connects to domains that are parsed by DNS.
  • 0: (Default) No. There is no limit when connecting to domains.

should_callbck_when_muted

Whether the SDK still triggers the on_playback_audio_frame_before_mixing callback after a remote user mutes their audio:

  • 0: (Default) No. The SDK stops triggering the callback when the remote user mutes their audio.
  • Greater than 0: Yes. The SDK keeps triggering the callback when the remote user mutes their audio.

AudioEncoderConfiguration

Audio encoder configuration.

@dataclass(kw_only=True)
class AudioEncoderConfiguration:
    audioProfile: AudioProfileType = AudioProfileType.AUDIO_PROFILE_DEFAULT
ParameterDescription
audioProfileAudio profile. For details, see AudioProfileType.

AudioFrame

The AudioFrame class.

@dataclass(frozen=True, kw_only=True)
class AudioFrame:
    type: int
    samples_per_channel: int
    bytes_per_sample: int
    channels: int
    samples_per_sec: int
    buffer: bytearray
    render_time_ms: int
    avsync_type: int
    far_field_flag: int
    rms: int
    voice_prob: int
    music_prob: int
    pitch: int
ParameterDescription
typeType of audio frame. Only 0 is supported, indicating 16-bit PCM data.
samples_per_channelNumber of samples per channel.
bytes_per_sampleNumber of bytes per sample. For PCM, typically 16 bits, which is two bytes.
channelsNumber of channels (if stereo, data is interleaved).
samples_per_secNumber of samples per second per channel.
bufferAudio data buffer (if stereo, data is interleaved). The buffer size is calculated as buffer = samples × channels × bytes_per_sample.
render_time_msRender timestamp of the external audio frame. You can use this timestamp to restore the order of audio frames; in scenarios with video (including those using external video sources), this parameter can be used to achieve audio-video synchronization.
avsync_typeReserved parameter.
far_field_flagThe far-field flag of the audio frame. Only used in audio frame observer callbacks.
rmsThe root mean square (RMS) of the audio frame. Only used in audio frame observer callbacks.
voice_probThe probability that the audio frame contains voice. Only used in audio frame observer callbacks.
music_probThe probability that the audio frame contains music. Only used in audio frame observer callbacks.
pitchThe pitch of the audio frame. Only used in audio frame observer callbacks.

AudioSubscriptionOptions

The audio subscription options.

@dataclass(kw_only=True)
class AudioSubscriptionOptions:
    packet_only: int = 0
    pcm_data_only: int = 0
    bytes_per_sample: int = 0
    number_of_channels: int = 0
    sample_rate_hz: int = 0
ParameterDescription
packet_onlyWhether to subscribe to audio packets only: - 0: (Default) The SDK automatically decodes the audio. - 1: Subscribe to audio packets only. The SDK will not decode the audio. Note: If this parameter is set to 1, other parameters in the structure are invalid.
pcm_data_onlyWhether to subscribe to PCM data only: - 0: (Default) The SDK automatically encodes the remote audio stream. - 1: Subscribe to PCM data only.
bytes_per_sampleNumber of bytes per audio sample.
number_of_channelsNumber of audio channels.
sample_rate_hzAudio sampling rate (Hz).

EncodedAudioFrameInfo

The encoded audio data.

@dataclass(kw_only=True)
class EncodedAudioFrameInfo:
    capture_timems: int = 0
    codec: AudioCodecType = AudioCodecType.AUDIO_CODEC_AACLC
    number_of_channels: int = 1
    sample_rate: int = 16000
    samples_per_channel: int = 1024
    send_even_if_empty: int = 1
    speech: int = 1
ParameterDescription
capture_timemsUnix timestamp of the audio frame (milliseconds).
codecAudio codec type. See AudioCodecType for details.
number_of_channelsNumber of channels in the audio frame.
sample_rateSampling rate of the audio frame (Hz).
samples_per_channelNumber of samples per channel. Defaults to 1024 for AAC codec format and 960 for Opus codec format.
send_even_if_emptyWhether to send empty audio frames: - 0: Do not send empty audio frames. - 1: (Default) Send empty audio frames.
speechWhether the audio source is speech: - 0: The audio source is not speech. - 1: (Default) The audio source is speech.

EncodedVideoFrameInfo

The encoded video data.

@dataclass(kw_only=True)
class EncodedVideoFrameInfo:
    codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE
    width: int = 0
    height: int = 0
    frames_per_second: int = 0
    frame_type: int = 0
    rotation: int = 0
    track_id: int = 0
    capture_time_ms: int = 0
    decode_time_ms: int = 0
    uid: int = 0
    stream_type: int = 0
ParameterDescription
codec_typeVideo codec type. See VideoCodecType for details.
widthWidth of the video frame (pixels).
heightHeight of the video frame (pixels).
frames_per_secondNumber of video frames per second. Used to calculate the timestamp of the encoded video frame. If set to 0, the SDK uses the real timestamp.
frame_typeType of the encoded video frame: - 0: Blank frame - 3: Keyframe. For H.264 codec format, the SDK classifies I-frames with SPS and PPS information as keyframes. - 4: Delta frame. For H.264 codec format, the SDK classifies P-frames and I-frames without SPS and PPS information as delta frames. - 5: B-frame - 6: Dropped frame - 7: Unknown
rotationRotation information of the encoded video frame: - 0: 0 degrees clockwise. - 90: 90 degrees clockwise. - 180: 180 degrees clockwise. - 270: 270 degrees clockwise.
track_idTrack ID of the video frame.
capture_time_msUnix timestamp for capturing the external encoded video frame (milliseconds).
decode_time_msTimestamp for decoding the external encoded video frame (milliseconds).
uidUser ID.
stream_typeStream type of the video frame: - 0: High stream. Higher resolution and bitrate. - 1: Low stream. Lower resolution and bitrate.

EncryptionConfig

@dataclass(kw_only=True)
class EncryptionConfig:
    encryption_mode: int
    encryption_key: str
    encryption_kdf_salt: bytearray = None

Encryption configuration.

encryption_mode

Encryption mode:

  • 1: 128-bit AES encryption, XTS mode.
  • 2: 128-bit AES encryption, ECB mode.
  • 3: 256-bit AES encryption, XTS mode.
  • 4: 128-bit SM4 encryption, ECB mode.
  • 5: 128-bit AES encryption, GCM mode.
  • 6: 256-bit AES encryption, GCM mode.
  • 7: (Default) 128-bit AES encryption, GCM mode. This encryption mode requires setting a salt (encryption_kdf_salt).
  • 8: 256-bit AES encryption, GCM mode. This encryption mode requires setting a salt (encryption_kdf_salt).

encryption_key

Encryption key, string type, with no length limit. It is recommended to use a 32-byte key.

encryption_kdf_salt

Encryption salt, 32 bytes in length. It is recommended to generate the salt using OpenSSL on the server side.

ExternalVideoFrame

The YUV format video data.

@dataclass(kw_only=True)
class ExternalVideoFrame:
    type: int = 1
    format: int = 0
    buffer: bytearray = None
    stride: int = 0
    height: int = 0
    crop_left: int = 0
    crop_top: int = 0
    crop_right: int = 0
    crop_bottom: int = 0
    rotation: int = 0
    timestamp: int = 0
    egl_context: bytearray = None
    egl_type: int = 0
    texture_id: int = 0
    matrix: list = field(default_factory=list)
    metadata: bytearray = None
    alpha_buffer: bytearray = None
    fill_alpha_buffer: int = 0
    alpha_mode: int = 0
    color_space: ColorSpaceType = field(default_factory=ColorSpaceType)    

type

Video data type: - 1: Type is raw data. - 2: Type is raw data. - 3: Type is Texture.

format

Pixel format: - 0: Raw video pixel format. - 1: I420 format. - 4: RGBA format. - 16: I422 format. - 17: ID3D11TEXTURE2D format. Currently supported types are DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_B8G8R8A8_TYPELESS, and DXGI_FORMAT_NV12. Note: The SDK currently does not support the Alpha channel. The incoming Alpha value will be discarded.

buffer

Video buffer.

stride

The stride of the incoming video frame, in pixels rather than bytes. For Texture, this value refers to the width of the Texture.

height

Height of the incoming video frame.

crop_left

Field related to raw data. Specifies the number of pixels cropped from the left. Default is 0. This parameter is only applicable to raw video data.

crop_top

Field related to raw data. Specifies the number of pixels cropped from the top. Default is 0. This parameter is only applicable to raw video data.

crop_right

Field related to raw data. Specifies the number of pixels cropped from the right. Default is 0. This parameter is only applicable to raw video data.

crop_bottom

Field related to raw data. Specifies the number of pixels cropped from the bottom. Default is 0. This parameter is only applicable to raw video data.

rotation

Field related to raw data. Specifies whether the incoming video frame should be rotated clockwise. Possible values are 0, 90, 180, 270. Default is 0.

timestamp

Timestamp of the incoming video frame, in milliseconds. Incorrect timestamps can cause frame drops or audio-video desynchronization.

egl_context

This parameter is only applicable to Texture format video data. - When using the OpenGL interface defined by Khronos (javax.microedition.khronos.egl.), set the eglContext to this field. - When using the OpenGL interface defined by Android (android.opengl.), set the eglContext to this field.

egl_type

This parameter is only applicable to Texture format video data. Refers to the EGL context type.

texture_id

This parameter is only applicable to Texture format video data. Refers to the Texture ID of the video frame.

matrix

This parameter is only applicable to Texture format video data. Refers to an input 4x4 transformation matrix, typically an identity matrix.

metadata

The metadata buffer of the video frame. The default value is None.

alpha_buffer

Alpha channel data output by the portrait segmentation algorithm. This data matches the size of the video frame, with each pixel value ranging from [0,255], where 0 represents the background and 255 represents the foreground (portrait). You can set this parameter to render the video background with various effects, such as transparency, solid color, images, or videos.

fill_alpha_buffer

[For BGRA or RGBA only] Whether to extract alphaBuffer from BGRA or RGBA data:

  • 1: Yes. Set it to true if you do not explicitly specify the alphaBuffer.
  • 0: (Default) No.

alpha_mode

The relative position between alphaBuffer and the frame:

  • 0: (Default) Normal frame.
  • 1: alphaBuffer is above the frame.
  • 2: alphaBuffer is below the frame.
  • 3: alphaBuffer is on the left of the frame.
  • 4: alphaBuffer is on the right of the frame.

color_space

The color space of the video frame. See ColorSpaceType.

PcmAudioFrame

PCM audio data.

@dataclass(kw_only=True)
class PcmAudioFrame:
    data: bytearray = None
    samples_per_channel: int = 0
    bytes_per_sample: int = 0
    number_of_channels: int = 0
    sample_rate: int = 0
    timestamp: int = 0
ParameterDescription
dataPCM audio data to be sent.
timestampTimestamp of the audio data collection (milliseconds).
samples_per_channelNumber of audio samples collected per channel in 10 milliseconds.
bytes_per_sampleNumber of bytes per sample.
number_of_channelsNumber of channels per sample.
sample_rateSampling rate (Hz). The minimum value is 8000.

RTCConnConfig

RTC connection configuration.

@dataclass(kw_only=True)
class RTCConnConfig:
    auto_subscribe_audio: int = 0
    auto_subscribe_video: int = 0
    enable_audio_recording_or_playout: int = 0
    max_send_bitrate: int = 0
    min_port: int = 0
    max_port: int = 0
    audio_subs_options: 'AudioSubscriptionOptions' = field(default_factory=AudioSubscriptionOptions)
    client_role_type: ClientRoleType = ClientRoleType.CLIENT_ROLE_BROADCASTER
    channel_profile: ChannelProfileType = ChannelProfileType.CHANNEL_PROFILE_LIVE_BROADCASTING
    audio_recv_media_packet: int = 0
    audio_recv_encoded_frame: int = 0
    video_recv_media_packet: int = 0
ParameterDescription
auto_subscribe_audioWhether to automatically subscribe to all audio tracks. - 0: Do not automatically subscribe to all audio tracks. - 1: (Default) Automatically subscribe to all audio tracks.
auto_subscribe_videoWhether to automatically subscribe to all video tracks. - 0: Do not automatically subscribe to all video tracks. - 1: (Default) Automatically subscribe to all video tracks.
enable_audio_recording_or_playoutWhether to enable audio recording or playout. - 0: Disable audio recording or playout. - 1: Enable audio recording or playout.
max_send_bitrateThe maximum send bitrate.
min_portThe minimum port.
max_portThe maximum port.
audio_subs_optionsAudio subscription options. See AudioSubscriptionOptions for details.
client_role_typeUser role. See ClientRoleType for details.
channel_profileChannel profile. See ChannelProfileType for details.
audio_recv_media_packetWhether to receive audio media packets. - 0: Do not receive. - 1: Receive.
video_recv_media_packetWhether to receive video media packets. - 0: Do not receive. - 1: Receive.

RTCConnInfo

The RTC connection information.

@dataclass(frozen=True, kw_only=True)
class RTCConnInfo():
    id: int
    channel_id: str
    state: int
    local_user_id: str
    internal_uid: int
ParameterDescription
idThe ID of the RTC connection.
channel_idRTC channel ID. If you have not called the connect method, this parameter returns None.
stateConnection state: - 1: Network connection disconnected. - 2: Establishing network connection. - 3: Network connected. - 4: Re-establishing network connection. - 5: Network connection failed.
local_user_idLocal user ID.
internal_uidFor internal use only.

SenderOptions

Configuration of the local video track object that includes the video source for sending encoded video data.

@dataclass(kw_only=True)
class SenderOptions:
    target_bitrate: int
    cc_mode: TCcMode = TCcMode.CC_ENABLED
    codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE
ParameterDescription
target_bitrateThe target encoding bitrate (Kbps) of the current encoder, which is an estimated value by the SDK based on the current network conditions.
cc_modeCongestion control mode. See TCcMode for details.
codec_typeVideo codec type. See VideoCodecType for details.

VideoDimensions

Resolution of the video encoding.

@dataclass(kw_only=True)
class VideoDimensions:
    width: int
    height: int
ParameterDescription
widthWidth of the video frame (pixels).
heightHeight of the video frame (pixels).

VideoEncoderConfiguration

Video encoding properties.

@dataclass(kw_only=True)
class VideoEncoderConfiguration:
    dimensions: VideoDimensions
    codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE
    frame_rate: int = 15
    bitrate: int = 0
    min_bitrate: int = 0
    orientation_mode: int = 0
    degradation_preference: int = 0
    mirror_mode: int = 0
    encode_alpha: int = 0
ParameterDescription
codec_typeVideo codec type. See VideoCodecType for details.
dimensionsResolution of the video encoding (px). This parameter measures the encoding quality, represented as width × height. The default value is 640 × 360. Users can set the resolution as needed. See VideoDimensions for details.
frame_rateFrame rate of the video encoding (fps), default is 15.
bitrateVideo encoding bitrate, in Kbps. - 0: (Recommended) Standard bitrate mode. In this mode, the video bitrate is twice the base bitrate. - -1: Adapted bitrate mode. In this mode, the video bitrate is the same as the base bitrate. When this mode is selected for live streaming, the video frame rate may be lower than the set value.
min_bitrateMinimum encoding bitrate, in Kbps. The SDK will automatically adjust the video encoding bitrate based on network conditions. Setting this parameter higher than the default value can force the video encoder to output higher quality images, but may cause packet loss and affect video playback fluency under poor network conditions, leading to stuttering. Therefore, unless there are special requirements for image quality, it is recommended not to modify this parameter. Note: This parameter is only applicable in live streaming scenarios.
orientation_modeOrientation mode of the video encoding: - 0: (Default) In this mode, the SDK outputs the video orientation consistent with the captured video. The receiver will rotate the video based on the received video rotation information. This mode is suitable for scenarios where the receiver can adjust the video orientation. If the captured video is in landscape mode, the output video will also be in landscape mode. If the captured video is in portrait mode, the output video will also be in portrait mode. - 1: In this mode, the SDK outputs the video in landscape mode. If the captured video is in portrait mode, the video encoder will crop it. This mode is suitable for scenarios where the receiver cannot adjust the video orientation, such as direct push to CDN. - 2: In this mode, the SDK outputs the video in portrait mode. If the captured video is in landscape mode, the video encoder will crop it. This mode is suitable for scenarios where the receiver cannot adjust the video orientation, such as direct push to CDN.
degradation_preferenceVideo encoding degradation preference when bandwidth is limited: - 0: (Default) When bandwidth is limited, the video encoding prioritizes reducing the frame rate while maintaining video quality. This degradation preference is suitable for scenarios where image quality is prioritized. - 1: When bandwidth is limited, the video encoding prioritizes reducing video quality while maintaining the frame rate. This degradation preference is suitable for scenarios where smoothness is prioritized and quality reduction is acceptable. - 2: When bandwidth is limited, the video encoding reduces both frame rate and video quality. The reduction in 2 is less than in 0 and 1, suitable for scenarios with limited smoothness and quality. - 3: When bandwidth is limited, the video encoding prioritizes reducing the frame rate.
mirror_modeWhether to enable mirror mode when sending encoded video, only affecting the video seen by the remote user. - 0: (Default) The mirror mode is determined by the SDK. - 1: Enable mirror mode. - 2: Disable mirror mode.
encode_alphaWhether to encode and send the alpha data to the remote user when the video frame contains alpha data. - 0: (Default) Do not encode or send the alpha data. - 1: Encode and send the alpha data.

VideoFrame

Properties of the video frame. The video data format is YUV420. The buffer is given as a pointer to a pointer. This interface cannot modify the pointer of the buffer, only the content of the buffer.

@dataclass(kw_only=True)
class VideoFrame():
    type: int = 0
    width: int = 0
    height: int = 0
    y_stride: int = 0
    u_stride: int = 0
    v_stride: int = 0
    y_buffer: bytearray = None
    u_buffer: bytearray = None
    v_buffer: bytearray = None
    rotation: int = 0
    render_time_ms: int = 0
    avsync_type: int = 0
    metadata: bytearray = None
    shared_context: str = None
    texture_id: int = 0
    matrix: list = None
    alpha_buffer: bytearray = None
    alpha_mode: int = 0
ParameterDescription
typeType of video data: - 1: Raw data. - 2: Raw data. - 3: Texture.
widthWidth of the video frame (pixels).
heightHeight of the video frame (pixels).
y_strideStride of the Y buffer in the YUV data.
u_strideStride of the U buffer in the YUV data.
v_strideStride of the V buffer in the YUV data.
y_bufferPointer to the pointer of the Y buffer in the YUV data.
u_bufferPointer to the pointer of the U buffer in the YUV data.
v_bufferPointer to the pointer of the V buffer in the YUV data.
rotationClockwise rotation angle of the frame before rendering the video. Currently supports 0 degrees, 90 degrees, 180 degrees, and 270 degrees.
render_time_msTimestamp of the external audio frame. This parameter is mandatory. You can use this timestamp to restore the order of audio frames; in scenarios with video (including those using external video sources), this parameter can help achieve audio-video synchronization.
avsync_typeReserved parameter.
metadataThe metadata buffer of the video frame.
shared_contextThe EGL context. Only applicable to Texture format video data.
texture_idThe Texture ID of the video frame. Only applicable to Texture format video data.
matrixThe input 4x4 transformation matrix. Only applicable to Texture format video data.
alpha_bufferThe alpha channel data of the video frame. The data matches the size of the video frame.
alpha_modeThe relative position between alpha_buffer and the frame. - 0: (Default) Normal frame. - 1: alpha_buffer is above the frame. - 2: alpha_buffer is below the frame. - 3: alpha_buffer is on the left of the frame. - 4: alpha_buffer is on the right of the frame.

VideoSubscriptionOptions

Video subscription options.

@dataclass(kw_only=True)
class VideoSubscriptionOptions:
    type: VideoStreamType = VideoStreamType.VIDEO_STREAM_HIGH
    encodedFrameOnly: bool = False
ParameterDescription
typeType of video to subscribe to. See VideoStreamType for details.
encodedFrameOnlyWhether to subscribe only to encoded video data: - 0: (Default) Subscribe only to decoded video data. - 1: Subscribe only to encoded video data.

Enums

AreaCode

Server access regions. This feature is an advanced setting and is used in scenarios with access security restrictions. The region code supports bitwise operations. After specifying the access region, the SDK will connect to the Agora servers within the specified region.

EnumerationDescription
AREA_CODE_CN0x00000001: Mainland China.
AREA_CODE_NA0x00000002: North America.
AREA_CODE_EU0x00000004: Europe.
AREA_CODE_AS0x00000008: Asia (excluding Mainland China).
AREA_CODE_JP0x00000010: Japan.
AREA_CODE_IN0x00000020: India.
AREA_CODE_GLOB0xFFFFFFFF: (Default) Global.

AudioCodecType

Audio codec types.

EnumerationDescription
AUDIO_CODEC_OPUS1: OPUS
AUDIO_CODEC_PCMA3: PCMA
AUDIO_CODEC_PCMU4: PCMU
AUDIO_CODEC_G7225: G722
AUDIO_CODEC_AACLC8: AAC LC
AUDIO_CODEC_HEAAC9: HE AAC
AUDIO_CODEC_HEAAC211: HEAAC2

AudioProfileType

Audio profiles.

AUDIO_PROFILE_DEFAULT

0: Default audio profile.

  • In communication scenarios: 16 kHz sampling rate, voice codec, mono, maximum encoding bitrate of 16 Kbps.
  • In live broadcast scenarios: 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 64 Kbps.

AUDIO_PROFILE_SPEECH_STANDARD

1: Specifies a 32 kHz sampling rate, voice codec, mono, maximum encoding bitrate of 18 Kbps.

AUDIO_PROFILE_MUSIC_STANDARD

2: Specifies a 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 64 Kbps.

AUDIO_PROFILE_MUSIC_STANDARD_STEREO

3: Specifies a 48 kHz sampling rate, music codec, stereo, maximum encoding bitrate of 80 Kbps.

AUDIO_PROFILE_MUSIC_HIGH_QUALITY

4: Specifies a 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 96 Kbps.

AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO

5: Specifies a 48 kHz sampling rate, music codec, stereo, maximum encoding bitrate of 128 Kbps.

AUDIO_PROFILE_IOT

6: Specifies a 16 kHz sampling rate, voice codec, mono, with echo cancellation algorithm AES.

AUDIO_PROFILE_NUM

7: Enumeration boundary.

AudioScenarioType

Audio scenario types.

EnumerationDescription
AUDIO_SCENARIO_DEFAULT0: (Default) Automatic scenario, automatically matches the appropriate audio quality based on user role and audio routing.
AUDIO_SCENARIO_GAME_STREAMING3: High-quality scenario, suitable for music-centric scenarios.
AUDIO_SCENARIO_CHATROOM5: Chatroom scenario, suitable for scenarios where users frequently switch between speaking and listening. In this scenario, audience members will receive a prompt to request microphone permission.
AUDIO_SCENARIO_CHORUS7: Chorus scenario, suitable for real-time chorus with extremely low latency under good network conditions.
AUDIO_SCENARIO_MEETING8: Meeting scenario, suitable for multi-person meetings focused on voice.
AUDIO_SCENARIO_NUM9: Enumeration boundary.

ChannelProfileType

Channel usage scenarios.

EnumerationDescription
CHANNEL_PROFILE_COMMUNICATION0: (Default) Communication scenario. Recommended for channels with only two users.
CHANNEL_PROFILE_LIVE_BROADCASTING1: Live broadcasting scenario. This scenario has two user roles: host and audience. Hosts can publish and receive audio and video streams, while the audience can only receive streams. Suitable for voice chat rooms, video live broadcasts, interactive large classes, etc.

ClientRoleType

User roles.

EnumerationDescription
CLIENT_ROLE_BROADCASTER1: Broadcaster. Broadcasters can both send and receive media streams.
CLIENT_ROLE_AUDIENCE2: Audience. Audience can only receive media streams.

TCcMode

Congestion control modes.

EnumerationDescription
CC_ENABLED0: Enable congestion control mode.
CC_DISABLED1: Disable congestion control mode.

VideoCodecType

Video codec types.

EnumerationDescription
VIDEO_CODEC_NONE0: None
VIDEO_CODEC_VP81: VP8
VIDEO_CODEC_H2642: H.264
VIDEO_CODEC_H2653: H.265
VIDEO_CODEC_GENERIC6: Generic
VIDEO_CODEC_GENERIC_H2647: Generic H264
VIDEO_CODEC_AV112: AV1
VIDEO_CODEC_VP913: VP9
VIDEO_CODEC_GENERIC_JPEG20: JPEG

VideoStreamType

Video stream types.

EnumerationDescription
VIDEO_STREAM_HIGH0: (Default) High stream. Higher resolution and bitrate.
VIDEO_STREAM_LOW1: Low stream. Lower resolution and bitrate.