Python
Updated
Agora Server Gateway SDK API reference for Python.
Server Gateway SDK API reference for Python.
Overview
-
AgoraServiceMethod Description initializeInitializes the AgoraServiceobject.releaseReleases the AgoraServiceobject.create_rtc_connectionCreates an RTC connection object. create_custom_audio_track_pcmCreates a local audio track object with a source that sends PCM audio data. create_custom_audio_track_encodedCreates a local audio track object with a source that sends encoded audio data. create_media_node_factoryCreates a media node factory object. create_custom_video_track_frameCreates a local video track object with a source that sends YUV format video data. create_custom_video_track_encodedCreates a local video track object with a source that sends encoded video data. set_log_fileSets the path and size of the SDK log file. -
RTCConnectionMethod Description connectConnects to the RTC channel. disconnectDisconnects from the RTC channel. renew_tokenRenews the token. register_observerRegisters an RTC connection observer. unregister_observerDestroys the RTC connection observer. create_data_streamCreates a data stream. send_stream_messageSends a data stream message. get_agora_parameterGets the AgoraParameterobject.get_local_userGets the LocalUserobject.releaseReleases the RTC connection. enable_encryptionEnables encryption. -
IRTCConnectionObserverCallback Description on_connectedOccurs when the SDK connection state changes to 3.on_disconnectedOccurs when the SDK connection state changes to 1.on_token_privilege_will_expireOccurs when the token will expire in 30 seconds. on_token_privilege_did_expireOccurs when the token has expired. on_user_joinedOccurs when a remote user joins the channel. on_user_leftOccurs when a remote user leaves the channel. on_stream_message_errorOccurs when there is an error receiving a data stream message. on_errorOccurs when an error occurs. on_encryption_errorOccurs when encryption errors occur. -
LocalUserMethod Description set_user_roleSets the user role. set_audio_encoder_configurationSets the audio encoder configuration. publish_audioSends the local audio track to the channel. unpublish_audioStops sending the local audio track to the channel. publish_videoSends the local video track to the channel. unpublish_videoStops sending the local video track to the channel. subscribe_audioSubscribes to the audio stream sent by a specific remote user in the channel. subscribe_all_audioSubscribes to all audio streams sent by remote users in the channel. unsubscribe_audioStops subscribing to the audio stream sent by a specific remote user in the channel. unsubscribe_all_audioStops subscribing to all audio streams sent by remote users in the channel. set_playback_audio_frame_before_mixing_parametersSets the audio data format for the on_playback_audio_frame_before_mixingcallback.register_audio_frame_observerRegisters an audio frame observer. unregister_audio_frame_observerDestroys the IAudioFrameObserverobject.register_video_encoded_frame_observerRegisters an encoded video frame observer. unregister_video_encoded_frame_observerDestroys the encoded video frame observer. subscribe_videoSubscribes to the video stream sent by a specific remote user in the channel. subscribe_all_videoSubscribes to all video streams sent by remote users in the channel. unsubscribe_videoStops subscribing to the video stream sent by a specific remote user in the channel. unsubscribe_all_videoStops subscribing to all video streams sent by remote users in the channel. register_local_user_observerRegisters a local user observer. unregister_local_user_observerDestroys the local user observer. register_video_frame_observerRegisters an IVideoFrameObserverobject.unregister_video_frame_observerDestroys the IVideoFrameObserverobject.get_rtc_connectionGets the RtcConnectionobject.releaseReleases all resources of the LocalUserobject.send_audio_meta_dataSends the audio metadata. -
LocalUserObserverCallback Description on_local_audio_track_state_changedOccurs when the state of a local audio track changes. on_user_audio_track_state_changedOccurs when the state of a remote audio track changes. on_stream_messageOccurs when a data stream message is received. on_user_info_updatedOccurs when user media information is updated. on_audio_metadata_receivedOccurs when the local user receives the audio metadata from the remote user. -
LocalAudioTrackMethod Description set_enabledEnables or disables the local audio track. adjust_publish_volumeAdjusts the audio publish volume. set_max_buffer_audio_frame_numberSets the maximum capacity of the audio send buffer. clear_sender_bufferClears the current audio send buffer. set_send_delay_msSets the minimum data duration allowed to be sent by the audio send buffer. releaseReleases the local audio track resources. -
LocalVideoTrackMethod Description set_enabledEnables or disables the local video track. set_video_encoder_configurationSets the video encoder properties. -
MediaNodeFactoryMethod Description create_audio_pcm_data_senderCreates a PCM data sending module. create_audio_encoded_frame_senderCreates an encoded audio data sending module. create_video_frame_senderCreates a YUV video frame sending module. create_video_encoded_image_senderCreates an encoded video sending module. releaseReleases the MediaNodeFactoryobject. -
AudioPcmDataSenderMethod Description send_audio_pcm_dataSends PCM format audio data to the local audio track. -
AudioEncodedFrameSenderMethod Description send_encoded_audio_frameSends an encoded audio frame to the local audio track. -
IAudioFrameObserverCallback Description on_playback_audio_frame_before_mixingOccurs when the audio data before mixing is received. -
VideoFrameSenderMethod Description send_video_frameSends a YUV format video frame to the video track. -
IVideoFrameObserverCallback Description on_frameOccurs when a video frame is received. -
VideoEncodedImageSenderMethod Description send_encoded_video_imageSends an encoded video frame to the video track. -
IVideoEncodedFrameObserverCallback Description on_encoded_video_frameOccurs when the SDK receives an encoded video frame.
AgoraService
The AgoraService class serves as the entry point for the SDK, allowing you to create media tracks and connections.
initialize
Initializes the AgoraService object.
def initialize(self, config: AgoraServiceConfig)Parameters
| Parameter | Description |
|---|---|
config | Initialization configuration. See AgoraServiceConfig. |
Returns
0: Success.- <
0: Failure.
release
Releases the AgoraService object.
def release(self)create_rtc_connection
Creates an RTC connection object.
def create_rtc_connection(self, con_config: RTCConnConfig)Caution
To create multiple connections, Agora recommends using multiple processes to avoid memory leaks and other issues.
Parameters
| Parameter | Description |
|---|---|
con_config | RTC connection configuration. See RTCConnConfig. |
Returns
RTCConnectionobject, if the method call succeeds.None, if the method call fails.
create_custom_audio_track_pcm
Creates a local audio track object with an audio source that sends PCM audio data. This track object can be used to send audio data in PCM format.
def create_custom_audio_track_pcm(self, audio_pcm_data_sender: AudioPcmDataSender)Parameters
| Parameter | Description |
|---|---|
audio_pcm_data_sender | PCM format audio data source. See AudioPcmDataSender. |
Returns
LocalAudioTrackobject, if the method call succeeds.None, if the method call fails.
create_custom_audio_track_encoded
Creates a local audio track object with an audio source that sends encoded audio data. This track object can be used to send encoded audio data, such as Opus format encoded data.
def create_custom_audio_track_encoded(self, audio_encoded_frame_sender: AudioEncodedFrameSender, mix_mode: int)Parameters
| Parameter | Description |
|---|---|
audio_encoded_frame_sender | Audio source that sends encoded audio data. See AudioEncodedFrameSender. |
mix_mode | Mix mode for the encoded audio in the channel. - 0: Mix all audio tracks in the channel. - 1: Do not mix audio. |
Returns
LocalAudioTrackobject, if the method call succeeds.None, if the method call fails.
create_media_node_factory
Creates a media node factory object.
def create_media_node_factory(self)Returns
MediaNodeFactoryobject, if the method call succeeds.None, if the method call fails.
create_custom_video_track_frame
Creates a local video track object with a video source that sends YUV format video data. This track object can be used to send video data in YUV format.
def create_custom_video_track_frame(self, video_frame_sender: VideoFrameSender)Parameters
| Parameter | Description |
|---|---|
video_frame_sender | YUV format video source object. See VideoFrameSender. |
Returns
LocalVideoTrackobject, if the method call succeeds.None, if the method call fails.
create_custom_video_track_encoded
Creates a local video track object with a video source that sends encoded video data. This track object can be used to send encoded video data, such as H.264 or VP8 format encoded data.
def create_custom_video_track_encoded(self, video_encoded_frame_sender: VideoEncodedImageSender, options: SenderOptions)Parameters
| Parameter | Description |
|---|---|
video_encoded_frame_sender | Video source that sends encoded video data. See VideoEncodedImageSender. |
options | Video track settings. See SenderOptions. |
Returns
LocalVideoTrackobject, if the method call succeeds.None, if the method call fails.
set_log_file
Sets the path and size of the SDK log file.
def set_log_file(self, log_path: str, log_size: int = 512 * 1024)The SDK generates 5 SDK log files and 5 API call log files by default, following these rules:
- The SDK log files names:
agorasdk.log,agorasdk.1.log,agorasdk.2.log,agorasdk.3.log,agorasdk.4.log. - The API call log files names:
agoraapi.log,agoraapi.1.log,agoraapi.2.log,agoraapi.3.log,agoraapi.4.log. - Each SDK log file has the default size of 1,024 KB; each API call log file has the default size of 2,048 KB. All log files are UTF-8 encoded.
- The latest logs are always written to
agorasdk.logandagoraapi.log.
When agorasdk.log is full, the SDK handles the log files as follows:
- Deletes the
agorasdk.4.logfile, if it exists. - Renames
agorasdk.3.logtoagorasdk.4.log. - Renames
agorasdk.2.logtoagorasdk.3.log. - Renames
agorasdk.1.logtoagorasdk.2.log. - Creates a new
agorasdk.logfile.
The agoraapi.log file follows the same rotation rules as agorasdk.log.
Parameters
| Parameter | Description |
|---|---|
log_path | The full path of the log file. The log file is UTF-8 encoded. |
log_size | The size of a single agorasdk.log log file in KB, with a range of [128, 20480]. The default is 1,024 KB. If you set file_size to less than 128 KB, the SDK automatically adjusts it to 128 KB; if you set file_size to more than 20,480 KB, the SDK automatically adjusts it to 20,480 KB. |
Returns
0: Success.- <
0: Failure.
RTCConnection
The RtcConnection class is used to manage the connection between the SDK and the RTC channel. After the connection is complete, you can send and receive media streams in the RTC channel through the LocalUser object. Connecting to a channel is an asynchronous operation, and you can use the IRtcConnectionObserver object to listen for connection status changes and events. The RtcConnection object also monitors whether remote users join or leave the channel.
connect
Connects to an RTC channel.
def connect(self, token: str, chan_id: str, user_id: str) -> intWhen the method is successful, the connection status changes from to 2.
Depending on whether the connection is successful, the connection status changes to 3 or 5. The SDK triggers the on_connected or on_disconnected callback to notify you of the connection status change.
Parameters
| Parameter | Description |
|---|---|
token | RTC token. See Use tokens. |
chan_id | Channel name. Must be a string and no more than 64 bytes. The supported character types include: - All lowercase letters: a to z; - All uppercase letters: A to Z; - All Arabic numerals: 0 to 9; - Space; - Punctuation and other special symbols: !, #, $, %, &, (, ), +, -, :, ;, <, =, ., >, ?, @, [, ], ^, _, {, }, |, ~, , |
user_id | The local user ID. No more than 64 bytes. If you do not set the user ID or set it to None, the SDK automatically returns a user ID in the on_connected callback. You must record and manage this ID yourself. Depending on the user ID type you selected in the initialize method, the supported character set is different: The supported character set for integer user IDs includes 10 digits 0-9. The supported character set for string user IDs includes: - 26 lowercase English letters: a-z; - 26 uppercase English letters: A-Z; - 10 digits: 0-9; - Space; - !, #, $, %, &, (, ), +, -, :, ;, <, =, ., >, ?, @, [, ], ^, _, {, }, |, ~, , |
Returns
0: Success.- <
0: Failure.-2: The parameters are invalid.-8: The current connection status is not1.
disconnect
Disconnects from the RTC channel. When the SDK disconnects from the channel, the connection status changes to 1. You can also learn about the disconnection through the on_disconnected callback.
def disconnect(self) -> intReturns
0: Success.- <
0: Failure.
renew_token
Renews the token. The token expires after a certain period of time. When you receive the on_token_privilege_will_expire callback, you must generate a new Token from your server and use this method to update the token. Otherwise, the SDK disconnects from the channel.
def renew_token(self, token: str) -> intParameters
| Parameter | Description |
|---|---|
token | The new token. |
Returns
0: Success.- <
0: Failure.
register_observer
Registers the RTC connection observer.
def register_observer(self, conn_observer: IRTCConnectionObserver) -> intParameters
| Parameter | Description |
|---|---|
conn_observer | IRTCConnectionObserver object. |
Returns
0: Success.- <
0: Failure.
unregister_observer
Destroys the RTC connection observer.
def unregister_observer(self) -> intReturns
0: Success.- <
0: Failure.
create_data_stream
Creates a data stream.
def create_data_stream(self, reliable: bool, ordered: bool) -> intEach user can create up to 5 data streams in each channel.
Note
- Agora does not support setting
reliableto1andorderedto0. - If you need the data packet to arrive immediately at the receiver, do not set the
orderedparameter to1.
Parameters
| Parameter | Description |
|---|---|
reliable | Whether the data stream is reliable: - 1: The receiver receives the data sent by the sender within 5 seconds. - 0: The receiver does not guarantee to receive the data. If the data is lost, it does not report an error. |
ordered | Whether the data stream is ordered: - 1: The receiver receives the data packets in the order sent by the sender. - 0: The receiver does not guarantee to receive the data packets in the order sent by the sender. |
Returns
stream_id: Returns the data stream ID, if the method call succeeds.None: Failure.
send_stream_message
Sends a data stream message.
def send_stream_message(self, stream_id: int, data: str) -> intThe method sends a data stream message to all users in the channel. The SDK implements the following restrictions for this method:
- Up to 30 packets per second, and each packet is no more than 1 KB.
- Up to 6 KB of data per second per client.
- Each client can have up to 5 data channels at the same time.
After successfully calling this method, the remote end triggers the on_stream_message callback, and remote users can obtain the received stream messages in this callback.
Note
- Ensure that you have called
create_data_streambefore calling this method. - This method is only applicable to the host in live streaming use-cases.
Parameters
| Parameter | Description |
|---|---|
stream_id | Data stream ID. You can obtain it through create_data_stream. |
data | Data to send. |
Returns
0: Success.- <
0: Failure.
get_agora_parameter
Gets the AgoraParameter object.
def get_agora_parameter(self)Returns
AgoraParameterobject, if the method call succeeds.None, if the method call fails.
get_local_user
Gets the LocalUser object. Each connection corresponds to one local user.
def get_local_user(self)Returns
LocalUserobject, if the method call succeeds.None, if the method call fails.
release
Releases the RTC connection.
def release(self)enable_encryption
def enable_encryption(self, enabled: int, config: EncryptionConfig)Enables encryption.
Since
v2.2.2.
This method needs to be called before connect. You need to set the encryption mode and encryption configuration in this method.
Parameters
enabled
Whether to enable encryption functionality:
1: Enable0: Disable
config
The encryption configuration: EncryptionConfig.
IRTCConnectionObserver
The IRTCConnectionObserver class, used to listen to the connection status of the SDK.
on_connected
Occurs when the connection state of the SDK changes to 3.
def on_connected(self, agora_rtc_conn, conn_info, reason)Parameters
agora_rtc_conn
The RTC connection object. See RtcConnection.
conn_info
Connection information. See RTCConnInfo.
reason
Connection state change reason:
-
0: Establishing a network connection. -
1: Successfully joined the channel. -
2: Network connection interrupted. -
3: Network connection forbidden by the server. -
4: Failed to join the channel. -
5: Left the channel. -
6: Invalid App ID. Please change to a valid App ID and try again. -
7: Invalid channel name. Please change to a valid channel name and try again. -
8: The generated token is invalid. Generally, the following reasons:- App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token.
- The uid specified in
connectwhen joining the channel does not match the uid used when generating the Token.
-
9: The generated Token is invalid. -
10: This user is banned by the server. -
11: SDK attempts to reconnect due to setting a proxy server. -
12: The network connection state changes due to updating the Token. -
13: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator. -
14: SDK connection to the server times out, enters the automatic reconnection state. -
15: Rejoined the channel successfully. -
16: SDK and the server lose connection. -
17: The connection state changes due to the echo test. -
18: The user changed the local IP address.
on_disconnected
Occurs when the connection state of the SDK changes to 1.
def on_disconnected(self, agora_rtc_conn, conn_info, reason)Parameters
agora_rtc_conn
The RTC connection object. See RtcConnection.
conn_info
The connection information. See RTCConnInfo.
reason
Connection state change reason:
-
0: Establishing a network connection. -
1: Successfully joined the channel. -
2: Network connection interrupted. -
3: Network connection forbidden by the server. -
4: Failed to join the channel. -
5: Left the channel. -
6: Invalid App ID. Please change to a valid App ID and try again. -
7: Invalid channel name. Please change to a valid channel name and try again. -
8: The generated token is invalid. Generally, the following reasons:- App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token.
- The uid specified in
connectwhen joining the channel does not match the uid used when generating the Token.
-
9: The generated Token is invalid. -
10: This user is banned by the server. -
11: SDK attempts to reconnect due to setting a proxy server. -
12: The network connection state changes due to updating the Token. -
13: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator. -
14: SDK connection to the server times out, enters the automatic reconnection state. -
15: Rejoined the channel successfully. -
16: SDK and the server lose connection. -
17: The connection state changes due to the echo test. -
18: The user changed the local IP address.
on_token_privilege_will_expire
Occurs when the token will expire in 30 seconds.
def on_token_privilege_will_expire(self, agora_rtc_conn, token)In the call process, if the Token is about to expire, the SDK triggers this callback 30 seconds before the Token expires to remind the app to update the Token.
When you receive this callback, you need to generate a new Token on the server side and call renew_token to pass the newly generated Token to the SDK.
Parameters
| Parameter | Description |
|---|---|
agora_rtc_conn | RTC connection object. See RtcConnection. |
token | The token that is about to expire. |
on_token_privilege_did_expire
Occurs when the token expires.
def on_token_privilege_did_expire(self, agora_rtc_conn)In the call process, if the token expires, the SDK triggers this callback to remind the app to update the Token.
When you receive this callback, you need to generate a new Token on the server side and call connect to reconnect the channel.
Parameters
| Parameter | Description |
|---|---|
agora_rtc_conn | RTC connection object. See RtcConnection. |
on_user_joined
Occurs when a remote user joins the channel. You can get the remote user's ID through this callback.
def on_user_joined(self, agora_rtc_conn, user_id)Parameters
| Parameter | Description |
|---|---|
agora_rtc_conn | RTC connection object. See RtcConnection. |
user_id | The remote user's ID. |
on_user_left
Occurs when a remote user leaves the channel. You can get the reason for the remote user leaving the channel through the reason parameter.
def on_user_left(self, agora_rtc_conn, user_id, reason)Parameters
agora_rtc_conn
The RTC connection object. See RtcConnection.
user_id
The remote user's ID.
reason
Connection state change reason:
-
0: Establishing a network connection. -
1: Successfully joined the channel. -
2: Network connection interrupted. -
3: Network connection forbidden by the server. -
4: Failed to join the channel. -
5: Left the channel. -
6: Invalid App ID. Please change to a valid App ID and try again. -
7: Invalid channel name. Please change to a valid channel name and try again. -
8: The generated token is invalid. Generally, the following reasons:- App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token.
- The uid specified in
connectwhen joining the channel does not match the uid used when generating the Token.
-
9: The generated Token is invalid. -
10: This user is banned by the server. -
11: SDK attempts to reconnect due to setting a proxy server. -
12: The network connection state changes due to updating the Token. -
13: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator. -
14: SDK connection to the server times out, enters the automatic reconnection state. -
15: Rejoined the channel successfully. -
16: SDK and the server lose connection. -
17: The connection state changes due to the echo test. -
18: The user changed the local IP address.
on_stream_message_error
Occurs when a data stream message error occurs. This callback indicates that the local user has not received the stream message sent by the remote user using the send_stream_message method.
def on_stream_message_error(self, agora_rtc_conn, user_id_str, stream_id, code, missed, cached)Parameters
| Parameter | Description |
|---|---|
agora_rtc_conn | The RTC connection object. See RtcConnection. |
user_id_str | The user ID that sends the message. |
stream_id | The stream ID of the received message. |
code | The code of the error that has occurred. |
missed | The number of lost messages. |
cached | The number of messages cached when the data stream is interrupted. |
on_error
Occurs when there is an error. This callback indicates that an error related to the network or media occurred during the SDK runtime. Generally, the error reported by the SDK means that the SDK cannot automatically recover and requires intervention from the App or user prompt.
def on_error(self, agora_rtc_conn, error_code, error_msg)Parameters
| Parameter | Description |
|---|---|
agora_rtc_conn | The RTC connection object. See RtcConnection. |
error_code | The error code. |
error_msg | The error description. |
on_encryption_error
def on_encryption_error(self, agora_rtc_conn, error_type)Occurs when encryption errors occur.
Since
v2.2.2.
Parameters
| Parameter | Description |
|---|---|
agora_rtc_conn | RtcConnection object. |
error_type | Error code. |
LocalUser
The LocalUser class. Defines the behavior and state of the local user. Each RTC connection has its own user. You can get the local user through get_local_user. Each user has two roles: the host (can send and receive audio and video streams) and the audience (can only receive audio and video streams).
set_user_role
Sets the user role. You can call this method before or after the user connects to the channel.
def set_user_role(self, role)Note
If the role set in the token parameter of connect is different from the role set here, the connection will fail.
Parameters
| Parameter | Description' |
|---|---|
role | User role. See ClientRoleType. |
Returns
0: Success.- <
0: Failure.
set_audio_encoder_configuration
Sets the audio encoding. The SDK applies the configuration to all audio tracks sent.
def set_audio_encoder_configuration(self, config)Parameters
| Parameter | Description |
|---|---|
config | Audio encoding configuration. See AudioEncoderConfiguration. |
Returns
0: Success.- <
0: Failure.
publish_audio
Publishes the local audio track to the channel. By default, all published audio tracks are mixed.
def publish_audio(self, agora_local_audio_track:LocalAudioTrack)Parameters
| Parameter | Description |
|---|---|
agora_local_audio_track | The local audio track to publish. See LocalAudioTrack. |
Returns
0: Success.- <
0: Failure.-5: If the local user's role is not the host, this error is returned.
unpublish_audio
Stops publishing the local audio track to the channel.
def unpublish_audio(self, agora_local_audio_track:LocalAudioTrack)Parameters
| Parameter | Description |
|---|---|
agora_local_audio_track | The local audio track to stop publishing. See LocalAudioTrack. |
Returns
0: Success.- <
0: Failure.
publish_video
Publishes the local video track to the channel.
def publish_video(self, agora_local_video_track:LocalVideoTrack)Parameters
| Parameter | Description |
|---|---|
agora_local_video_track | The local video track to publish. See LocalVideoTrack. |
Returns
0: Success.- <
0: Failure.
unpublish_video
Stops publishing the local video track to the channel.
def unpublish_video(self, agora_local_video_track:LocalVideoTrack)Parameters
| Parameter | Description |
|---|---|
agora_local_video_track | The local video track to stop publishing. See LocalVideoTrack. |
Returns
0: Success.- <
0: Failure.
subscribe_audio
Subscribes to the audio stream sent by a specific remote user in the channel.
def subscribe_audio(self, user_id)Parameters
| Parameter | Description |
|---|---|
user_id | The remote user ID. |
Returns
0: Success.- <
0: Failure.-2: If the user does not exist oruser_idis invalid, this error is returned.
subscribe_all_audio
Subscribes to all the audio streams sent by remote users in the channel. This method automatically subscribes to the audio streams sent by users who join the channel later.
def subscribe_all_audio(self)Returns
0: Success.- <
0: Failure.
unsubscribe_audio
Stops subscribing to the audio stream sent by a specific remote user in the channel.
def unsubscribe_audio(self, user_id)Parameters
| Parameter | Description |
|---|---|
user_id | The remote user ID. |
Returns
0: Success.- <
0: Failure.-2: If the user does not exist oruser_idis invalid, this error is returned.
unsubscribe_all_audio
Stops subscribing to all the audio streams sent by remote users in the channel. This method automatically cancels the subscription of the audio streams sent by users who join the channel later. You must call subscribe_audio or subscribe_all_audio to restore the subscription of audio.
def unsubscribe_all_audio(self)Returns
0: Success.- <
0: Failure.
set_playback_audio_frame_before_mixing_parameters
Sets the audio data format reported by on_playback_audio_frame_before_mixing.
def set_playback_audio_frame_before_mixing_parameters(self, channels, sample_rate_hz)Parameters
| Parameter | Description |
|---|---|
channels | The number of audio channels, which can be set to 1 (mono) or 2 (stereo). |
sample_rate_hz | The sample rate of the audio data, which can be set to 8000, 16000, 32000, 44100, or 48000. |
Returns
0: Success.- <
0: Failure.
register_audio_frame_observer
Registers the audio frame observer. You must implement the IAudioFrameObserver class and register the callback based on your needs.
def register_audio_frame_observer(self, observer: IAudioFrameObserver, enable_vad: int, vad_configure)Parameters
observer
IAudioFrameObserver class.
enable_vad
Whether to enable the built-in voice activity detection (VAD):
0: Disable VAD.1: Enable VAD. The SDK returns the VAD results in theon_playback_audio_frame_before_mixingcallback.
vad_configure
The VAD configuration, an AudioVadConfigV2 object. The SDK uses this parameter only when enable_vad is 1.
Returns
0: Success.- <
0: Failure.
unregister_audio_frame_observer
Destroys the IAudioFrameObserver object.
def unregister_audio_frame_observer(self)Returns
0: Success.- <
0: Failure.
register_video_encoded_frame_observer
Registers the encoded video frame observer.
def register_video_encoded_frame_observer(self, agora_video_encoded_frame_observer:IVideoEncodedFrameObserver)Parameters
| Parameter | Description |
|---|---|
agora_video_encoded_frame_observer | The encoded video frame observer. See IVideoEncodedFrameObserver. |
Returns
0: Success.- <
0: Failure.
unregister_video_encoded_frame_observer
Destroys the encoded video frame observer.
def unregister_video_encoded_frame_observer(self)Returns
0: Success.- <
0: Failure.
subscribe_video
Subscribes to the video stream sent by a specific remote user in the channel.
def subscribe_video(self, user_id, options:VideoSubscriptionOptions)Parameters
| Parameter | Description |
|---|---|
user_id | The remote user ID. |
options | The subscription options. See VideoSubscriptionOptions. |
Returns
0: Success.- <
0: Failure.-2: If the user does not exist oruser_idis invalid, this error is returned.
subscribe_all_video
Subscribes to all the video streams sent by remote users in the channel. This method automatically subscribes to the video streams sent by users who join the channel later.
def subscribe_all_video(self, options:VideoSubscriptionOptions)Parameters
| Parameter | Description |
|---|---|
options | The subscription options. See VideoSubscriptionOptions. |
Returns
0: Success.- <
0: Failure.
unsubscribe_video
Stops subscribing to the video stream sent by a specific remote user in the channel.
def unsubscribe_video(self, user_id)Parameters
| Parameter | Description |
|---|---|
user_id | The remote user ID. |
Returns
0: Success.- <
0: Failure.-2: If the user does not exist oruser_idis invalid, this error is returned.
unsubscribe_all_video
Stops subscribing to all the video streams sent by remote users in the channel. This method automatically cancels the subscription of the video streams sent by users who join the channel later. You must call subscribe_video or subscribe_all_video to restore the subscription.
def unsubscribe_all_video(self)Returns
0: Success.- <
0: Failure.
register_local_user_observer
Registers the local user observer.
def register_local_user_observer(self, observer:ILocalUserObserver)Parameters
| Parameter | Description |
|---|---|
observer | LocalUserObserver class. |
Returns
0: Success.- <
0: Failure.
unregister_local_user_observer
Destroys the local user observer.
def unregister_local_user_observer(self)Returns
0: Success.- <
0: Failure.
register_video_frame_observer
Registers the IVideoFrameObserver object.
def register_video_frame_observer(self, agora_video_frame_observer2:IVideoFrameObserver)Parameters
| Parameter | Description |
|---|---|
agora_video_frame_observer2 | IVideoFrameObserver class. |
Returns
0: Success.- <
0: Failure.
unregister_video_frame_observer
Destroys the IVideoFrameObserver object.
def unregister_video_frame_observer(self)Returns
0: Success.- <
0: Failure.
get_rtc_connection
Gets the RtcConnection object.
def get_rtc_connection(self)Returns
RtcConnectionobject: Success.None: Failure.
release
Releases all resources of the LocalUser object.
def release(self)send_audio_meta_data
def send_audio_meta_data(self, data)Sends the audio metadata.
Since
v2.2.0.
A successful call of this method triggers the on_audio_metadata_received callback on the remote end. The remote user can get the audio metadata in this callback.
Parameters
| Parameter | Description |
|---|---|
data | The audio metadata. |
Returns
0: Success.- <
0: Failure.
LocalUserObserver
The LocalUserObserver class.
on_local_audio_track_state_changed
Triggered when the local audio track state changes.
def on_local_audio_track_state_changed(self, agora_local_user, agora_local_audio_track, state, error)Parameters
| Parameter | Description |
|---|---|
agora_local_user | LocalUserObserver class. |
agora_local_audio_track | Local audio track. See LocalAudioTrack. |
state | The current state of the audio track. |
error | Error information. |
on_user_audio_track_state_changed
Occurs when the remote audio track state changes.
def on_user_audio_track_state_changed(self, agora_local_user, user_id, agora_remote_audio_track, state, reason, elapsed)Parameters
| Parameter | Description |
|---|---|
agora_local_user | The LocalUserObserver class. |
user_id | The remote user ID. |
agora_remote_audio_track | Remote audio track. |
state | The current state of the audio track. |
reason | The reason for the state change. |
elapsed | The time elapsed after the remote user connects to the channel (ms). |
on_stream_message
Callback for receiving data stream messages. This callback indicates that the local user has received the stream message sent by the remote user through the send_stream_message method.
def on_stream_message(self, agora_local_user, user_id, stream_id, data, length)Parameters
| Parameter | Description |
|---|---|
agora_local_user | LocalUserObserver class. |
user_id | The user ID that sends the message. |
stream_id | The Stream ID of the received message. |
data | The received data. |
length | The data length (bytes). |
on_user_info_updated
Occurs when the user media information is updated.
def on_user_info_updated(self, agora_local_user, user_id, msg, val)Parameters
| Parameter | Description |
|---|---|
agora_local_user | The LocalUserObserver class. |
user_id | The user ID that sends the message. |
msg | User media information: - 0: The user has stopped publishing audio streams. - 1: The user has stopped publishing video streams. - 4: The user has enabled video functionality. |
val | Whether the user stops sending streams. - 0: Do not stop sending streams. - 1: Stop sending streams. |
on_audio_meta_data_received
def on_audio_meta_data_received(self, agora_local_user, user_id, data):Occurs when the remote audio metadata is received.
This callback indicates that the local user has received the audio metadata sent by the remote user using the send_audio_meta_data method.
Parameters
| Parameter | Description |
|---|---|
agora_local_user | The LocalUserObserver class |
user_id | The ID of the user who sends the audio metadata. |
data | The audio metadata. |
LocalAudioTrack
The LocalAudioTrack class is the base class for the local audio track.
set_enabled
Enables or disables the local audio track. After the local audio track is enabled, the SDK can capture, process, and encode local audio.
def set_enabled(self, enable)Parameters
| Parameter | Description |
|---|---|
enable | Whether to enable the audio track: - 1: Enable the audio track. - 0: Disable the audio track. |
adjust_publish_volume
Adjusts the audio publishing volume.
def adjust_publish_volume(self, volume)Parameters
| Parameter | Description |
|---|---|
volume | The audio publishing volume. The value range is from [0,100]. 0 means mute, and 100 means the original volume. |
Returns
- 0: Success.
- < 0: Failure.
set_max_buffer_audio_frame_number
Sets the maximum capacity of the audio sending buffer.
def set_max_buffer_audio_frame_number(self, num)In AI interaction applications, the frequency of sending audio frames by the sender may be uneven, for example, sending 1 frame every 20 ms or 3 frames every 20 ms. If the SDK sends data according to the actual sending frequency to the Agora server, the receiver may experience stuttering or frame loss. To optimize the audio transmission performance, the SDK sets an audio sending buffer. Regardless of the actual sending frequency of audio frames by the sender, the SDK stores data in the audio sending buffer. After the data in the audio sending buffer reaches the minimum allowed sending time, the SDK starts sending data to the receiver at a frequency of 10 ms per frame.
The default maximum capacity of the audio sending buffer is 6000, which means it can store up to 6000 frames of data, or 60 seconds of data. If you need a larger buffer capacity, you can call this method to adjust it. If you also need to set the minimum allowed sending time for the audio sending buffer, call the set_send_delay_ms method.
Note
This method must be called before the publish_audio method.
Parameters
| Parameter | Description |
|---|---|
num | The maximum number of audio frames that the buffer can store. For example, if you enter 6000, it means 6000 frames of audio data. |
clear_sender_buffer
Clears the current audio sending buffer.
def clear_sender_buffer(self)If you no longer need the data in the current audio sending buffer, call this method to clear the buffer. For example, a user asks AI a question, and AI is answering the question through the SDK. At this time, the audio sending buffer of the SDK is constantly receiving and sending data. When the user interrupts AI's reply and asks another question, you can call this method to clear the buffered data that has not been sent out, so that the SDK can quickly buffer the AI's response to the new topic.
Returns
- 0: Success.
- < 0: Failure.
set_send_delay_ms
Sets the minimum data duration that the audio sending buffer allows to send.
def set_send_delay_ms(self, delay_ms:int)The default minimum data duration that the SDK allows to send is 160 ms, which means the SDK waits until the audio sending buffer is filled with 160 ms of data before sending the data to the receiver. This will have the following effects:
- If the sender sends data that is greater than or equal to 160 ms, for example 800 ms, the SDK sends the data at a frequency of 10 ms per frame.
- If the sender sends data that is less than 160 ms, for example 100 ms, the SDK waits until the buffer data reaches 160 ms before sending the data at a frequency of 10 ms per frame. This will cause a delay of up to 60 ms.
You can adjust the minimum data duration according to your actual business needs by calling this method.
Parameters
| Parameter | Description |
|---|---|
delay_ms | The minimum data duration that the buffer allows to send (ms). This parameter must be set to a multiple of 10, and its value must be greater than 0 and less than 10 times the value of the number parameter in the set_max_buffer_audio_frame_number method. |
release
Releases the local audio track resources.
def release(self)LocalVideoTrack
The LocalVideoTrack class. Provides basic methods for the local video track. You can publish one or multiple video tracks through the publish_video method.
set_enabled
Enables or disables the local video track. After the local video track is enabled, the SDK can capture, process, and encode the local video.
def set_enabled(self, enable)Parameters
| Parameter | Description |
|---|---|
enable | Enable or disable the local video track: - 0: Disable the local video track. - 1: Enable the local video track. |
set_video_encoder_configuration
Sets the video encoding properties.
def set_video_encoder_configuration(self, config)Parameters
| Parameter | Description |
|---|---|
config | Video encoding parameter configuration. See VideoEncoderConfig. |
Returns
- 0: Success.
- < 0: Failure.
MediaNodeFactory
The MediaNodeFactory class.
create_audio_pcm_data_sender
Creates a PCM data sender module. This method creates an AudioPcmDataSender object, which can be used in the create_custom_audio_track_pcm method.
def create_audio_pcm_data_sender(self)Returns
AudioPcmDataSenderobject, if the method call succeeds.None, if the method call fails.
create_audio_encoded_frame_sender
Creates an encoded audio data sender module. This method creates an AudioEncodedFrameSender object, which can be used in the create_custom_audio_track_encoded method.
def create_audio_encoded_frame_sender(self)Returns
AudioEncodedFrameSenderobject, if the method call succeeds.None, if the method call fails.
create_video_frame_sender
Creates a YUV video frame sender module. This method creates a VideoFrameSender object, which can be used in the create_custom_video_track_frame method.
def create_video_frame_sender(self)Returns
VideoFrameSenderobject, if the method call succeeds.None, if the method call fails.
create_video_encoded_image_sender
Creates an encoded video sender module. This method creates a VideoEncodedImageSender object, which can be used in the create_custom_video_track_encoded method.
def create_video_encoded_image_sender(self)Returns
VideoEncodedImageSenderobject, if the method call succeeds.None, if the method call fails.
release
Releases the MediaNodeFactory object.
def release(self)AudioPcmDataSender
The AudioPcmDataSender class.
send_audio_pcm_data
Sends PCM audio data to the local audio track.
def send_audio_pcm_data(self, frame: PcmAudioFrame)Parameters
| Parameter | Description |
|---|---|
frame | The PcmAudioFrame object, containing the PCM audio data to send. |
Returns
- 0: Success.
- < 0: Failure.
AudioEncodedFrameSender
The AudioEncodedFrameSender Class.
send_encoded_audio_frame
Sends an encoded audio frame to the audio track.
def send_encoded_audio_frame(self, buffer_ptr: int, buffer_size: int, frame_info: EncodedAudioFrameInfo)Parameters
| Parameter | Description |
|---|---|
buffer_ptr | The encoded audio data to send. |
buffer_size | The size of the data (bytes). |
frame_info | The audio frame information. See EncodedAudioFrameInfo. |
Returns
- 0: Success.
- < 0: Failure.
IAudioFrameObserver
The IAudioFrameObserver class.
on_playback_audio_frame_before_mixing
Gets the data before the playback audio is mixed.
def on_playback_audio_frame_before_mixing(self, agora_local_user, channelId, uid, frame: AudioFrame, vad_result_state: int, vad_result_bytearray: bytearray)Parameters
agora_local_user
The LocalUserObserver class.
channelId
The channel name.
uid
The remote user ID.
frame
The audio raw data. See AudioFrame.
vad_result_state
The VAD state. Valid only when you enable VAD in register_audio_frame_observer:
- <
0: VAD is not enabled. 0: Not speaking.1: Started speaking.2: Speaking.3: Stopped speaking.
vad_result_bytearray
The audio data processed by VAD. When VAD is enabled, use this data instead of frame for downstream processing, such as speech recognition.
Returns
0:The audio frame data is invalid, and the data is discarded.1:The audio frame data is valid, and the data is sent.
VideoFrameSender
The VideoFrameSender class.
send_video_frame
Sends a YUV format video frame to the video track.
def send_video_frame(self, frame:ExternalVideoFrame)Parameters
| Parameter | Description |
|---|---|
frame | The video frame to send, see ExternalVideoFrame. |
Returns
- 0: Success.
- < 0: Failure.
IVideoFrameObserver
The IVideoFrameObserver class.
on_frame
def on_frame(self, channel_id, remote_uid, frame:VideoFrame)Occurs when a video frame is received.
Parameters
| Parameter | Description |
|---|---|
channel_id | The channel name. |
remote_uid | The remote user ID. |
frame | The video data. See VideoFrame |
VideoEncodedImageSender
The VideoEncodedImageSender class.
send_encoded_video_image
Sends an encoded video frame to the video track.
def send_encoded_video_image(self, buffer_ptr:int, buffer_size:int, frame_info:EncodedVideoFrameInfo)Parameters
| Parameter | Description |
|---|---|
buffer_ptr | The video buffer. |
buffer_size | The size of the video data (bytes). |
frame_info | The encoded video frame information: see EncodedVideoFrameInfo. |
Returns
- 0: Success.
- < 0: Failure.
IVideoEncodedFrameObserver
The IVideoEncodedFrameObserver class.
on_encoded_video_frame
Occurs when an encoded video frame is received.
def on_encoded_video_frame(self, uid, image_buffer, length, video_encoded_frame_info:EncodedVideoFrameInfo)Parameters
| Parameter | Description |
|---|---|
uid | The remote user ID. |
image_buffer | The video frame buffer. |
length | The length of the video frame data. |
video_encoded_frame_info | The encoded video frame information. See EncodedVideoFrameInfo |
Returns
Whether to accept the encoded video frame:
0: Do not accept.1: Accept.
Data type
Classes
AgoraServiceConfig
Global configuration of the AgoraServiceConfig class. Set when calling the initialize method.
@dataclass(kw_only=True)
class AgoraServiceConfig:
log_path: str = ""
log_size: int = 0
enable_audio_processor: int = 1
enable_audio_device: int = 0
enable_video: int = 0
context: object = None
appid: str = ""
area_code: int = AreaCode.AREA_CODE_GLOB.value
channel_profile: ChannelProfileType = ChannelProfileType.CHANNEL_PROFILE_LIVE_BROADCASTING
audio_scenario: AudioScenarioType = AudioScenarioType.AUDIO_SCENARIO_CHORUS
use_string_uid: int = 0
domain_limit: int = 0
should_callbck_when_muted: int = 0log_path
The full path to the log file. If this parameter is set to None, the log is located in the current application's pwd directory
log_size
The size of a single log file in KB. The default value is 1024 KB. If you set this parameter to 1024 KB, the SDK outputs a maximum of 5 MB of log files. If you set this parameter to less than 1024 KB, the setting does not take effect, and the maximum size of a single log file is still 1024 KB.
enable_audio_processor
Whether to enable the audio processing module. - 0: Disable the audio processing module. You will not be able to create an audio track. - 1: (Default) Enable the audio processing module.
enable_audio_device
Whether to enable the Audio Device Module (ADM). ADM is used to manage audio devices such as recording and playing audio. - 0: Disable the audio device module. You cannot record or play audio. - 1: (Default) Enable the audio device module. You can record and play audio. Note: If you set enable_audio_device to 0 and enable_audio_processor to 1, you will not be able to use audio devices, but you can push PCM audio data.
enable_video
Whether to enable video. - 0: (Default) Disable video. - 1: Enable video.
context
The context.
appid
Your Agora project's App ID. If your Agora project has enabled Token authentication, you can also assign this parameter to an RTC Token. See Agora account management for details.
area_code
The access region of the server. This feature is an advanced setting suitable for scenarios with access security restrictions. The area code supports bitwise operations. After specifying the access region, the SDK connects to Agora servers within the specified region. See AreaCode for details.
channel_profile
Channel profile. See ChannelProfileType for details.
audio_scenario
Audio encoding properties. See AudioScenarioType for details.
use_string_uid
Whether to use String user IDs.
0: Do not use.1: Use.
domain_limit
Whether to enable domain limit:
1: Yes. Only connects to domains that are parsed by DNS.0: (Default) No. There is no limit when connecting to domains.
should_callbck_when_muted
Whether the SDK still triggers the on_playback_audio_frame_before_mixing callback after a remote user mutes their audio:
0: (Default) No. The SDK stops triggering the callback when the remote user mutes their audio.- Greater than
0: Yes. The SDK keeps triggering the callback when the remote user mutes their audio.
AudioEncoderConfiguration
Audio encoder configuration.
@dataclass(kw_only=True)
class AudioEncoderConfiguration:
audioProfile: AudioProfileType = AudioProfileType.AUDIO_PROFILE_DEFAULT| Parameter | Description |
|---|---|
audioProfile | Audio profile. For details, see AudioProfileType. |
AudioFrame
The AudioFrame class.
@dataclass(frozen=True, kw_only=True)
class AudioFrame:
type: int
samples_per_channel: int
bytes_per_sample: int
channels: int
samples_per_sec: int
buffer: bytearray
render_time_ms: int
avsync_type: int
far_field_flag: int
rms: int
voice_prob: int
music_prob: int
pitch: int| Parameter | Description |
|---|---|
type | Type of audio frame. Only 0 is supported, indicating 16-bit PCM data. |
samples_per_channel | Number of samples per channel. |
bytes_per_sample | Number of bytes per sample. For PCM, typically 16 bits, which is two bytes. |
channels | Number of channels (if stereo, data is interleaved). |
samples_per_sec | Number of samples per second per channel. |
buffer | Audio data buffer (if stereo, data is interleaved). The buffer size is calculated as buffer = samples × channels × bytes_per_sample. |
render_time_ms | Render timestamp of the external audio frame. You can use this timestamp to restore the order of audio frames; in scenarios with video (including those using external video sources), this parameter can be used to achieve audio-video synchronization. |
avsync_type | Reserved parameter. |
far_field_flag | The far-field flag of the audio frame. Only used in audio frame observer callbacks. |
rms | The root mean square (RMS) of the audio frame. Only used in audio frame observer callbacks. |
voice_prob | The probability that the audio frame contains voice. Only used in audio frame observer callbacks. |
music_prob | The probability that the audio frame contains music. Only used in audio frame observer callbacks. |
pitch | The pitch of the audio frame. Only used in audio frame observer callbacks. |
AudioSubscriptionOptions
The audio subscription options.
@dataclass(kw_only=True)
class AudioSubscriptionOptions:
packet_only: int = 0
pcm_data_only: int = 0
bytes_per_sample: int = 0
number_of_channels: int = 0
sample_rate_hz: int = 0| Parameter | Description |
|---|---|
packet_only | Whether to subscribe to audio packets only: - 0: (Default) The SDK automatically decodes the audio. - 1: Subscribe to audio packets only. The SDK will not decode the audio. Note: If this parameter is set to 1, other parameters in the structure are invalid. |
pcm_data_only | Whether to subscribe to PCM data only: - 0: (Default) The SDK automatically encodes the remote audio stream. - 1: Subscribe to PCM data only. |
bytes_per_sample | Number of bytes per audio sample. |
number_of_channels | Number of audio channels. |
sample_rate_hz | Audio sampling rate (Hz). |
EncodedAudioFrameInfo
The encoded audio data.
@dataclass(kw_only=True)
class EncodedAudioFrameInfo:
capture_timems: int = 0
codec: AudioCodecType = AudioCodecType.AUDIO_CODEC_AACLC
number_of_channels: int = 1
sample_rate: int = 16000
samples_per_channel: int = 1024
send_even_if_empty: int = 1
speech: int = 1| Parameter | Description |
|---|---|
capture_timems | Unix timestamp of the audio frame (milliseconds). |
codec | Audio codec type. See AudioCodecType for details. |
number_of_channels | Number of channels in the audio frame. |
sample_rate | Sampling rate of the audio frame (Hz). |
samples_per_channel | Number of samples per channel. Defaults to 1024 for AAC codec format and 960 for Opus codec format. |
send_even_if_empty | Whether to send empty audio frames: - 0: Do not send empty audio frames. - 1: (Default) Send empty audio frames. |
speech | Whether the audio source is speech: - 0: The audio source is not speech. - 1: (Default) The audio source is speech. |
EncodedVideoFrameInfo
The encoded video data.
@dataclass(kw_only=True)
class EncodedVideoFrameInfo:
codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE
width: int = 0
height: int = 0
frames_per_second: int = 0
frame_type: int = 0
rotation: int = 0
track_id: int = 0
capture_time_ms: int = 0
decode_time_ms: int = 0
uid: int = 0
stream_type: int = 0| Parameter | Description |
|---|---|
codec_type | Video codec type. See VideoCodecType for details. |
width | Width of the video frame (pixels). |
height | Height of the video frame (pixels). |
frames_per_second | Number of video frames per second. Used to calculate the timestamp of the encoded video frame. If set to 0, the SDK uses the real timestamp. |
frame_type | Type of the encoded video frame: - 0: Blank frame - 3: Keyframe. For H.264 codec format, the SDK classifies I-frames with SPS and PPS information as keyframes. - 4: Delta frame. For H.264 codec format, the SDK classifies P-frames and I-frames without SPS and PPS information as delta frames. - 5: B-frame - 6: Dropped frame - 7: Unknown |
rotation | Rotation information of the encoded video frame: - 0: 0 degrees clockwise. - 90: 90 degrees clockwise. - 180: 180 degrees clockwise. - 270: 270 degrees clockwise. |
track_id | Track ID of the video frame. |
capture_time_ms | Unix timestamp for capturing the external encoded video frame (milliseconds). |
decode_time_ms | Timestamp for decoding the external encoded video frame (milliseconds). |
uid | User ID. |
stream_type | Stream type of the video frame: - 0: High stream. Higher resolution and bitrate. - 1: Low stream. Lower resolution and bitrate. |
EncryptionConfig
@dataclass(kw_only=True)
class EncryptionConfig:
encryption_mode: int
encryption_key: str
encryption_kdf_salt: bytearray = NoneEncryption configuration.
encryption_mode
Encryption mode:
- 1: 128-bit AES encryption, XTS mode.
- 2: 128-bit AES encryption, ECB mode.
- 3: 256-bit AES encryption, XTS mode.
- 4: 128-bit SM4 encryption, ECB mode.
- 5: 128-bit AES encryption, GCM mode.
- 6: 256-bit AES encryption, GCM mode.
- 7: (Default) 128-bit AES encryption, GCM mode. This encryption mode requires setting a salt (
encryption_kdf_salt). - 8: 256-bit AES encryption, GCM mode. This encryption mode requires setting a salt (
encryption_kdf_salt).
encryption_key
Encryption key, string type, with no length limit. It is recommended to use a 32-byte key.
encryption_kdf_salt
Encryption salt, 32 bytes in length. It is recommended to generate the salt using OpenSSL on the server side.
ExternalVideoFrame
The YUV format video data.
@dataclass(kw_only=True)
class ExternalVideoFrame:
type: int = 1
format: int = 0
buffer: bytearray = None
stride: int = 0
height: int = 0
crop_left: int = 0
crop_top: int = 0
crop_right: int = 0
crop_bottom: int = 0
rotation: int = 0
timestamp: int = 0
egl_context: bytearray = None
egl_type: int = 0
texture_id: int = 0
matrix: list = field(default_factory=list)
metadata: bytearray = None
alpha_buffer: bytearray = None
fill_alpha_buffer: int = 0
alpha_mode: int = 0
color_space: ColorSpaceType = field(default_factory=ColorSpaceType) type
Video data type: - 1: Type is raw data. - 2: Type is raw data. - 3: Type is Texture.
format
Pixel format: - 0: Raw video pixel format. - 1: I420 format. - 4: RGBA format. - 16: I422 format. - 17: ID3D11TEXTURE2D format. Currently supported types are DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_B8G8R8A8_TYPELESS, and DXGI_FORMAT_NV12. Note: The SDK currently does not support the Alpha channel. The incoming Alpha value will be discarded.
buffer
Video buffer.
stride
The stride of the incoming video frame, in pixels rather than bytes. For Texture, this value refers to the width of the Texture.
height
Height of the incoming video frame.
crop_left
Field related to raw data. Specifies the number of pixels cropped from the left. Default is 0. This parameter is only applicable to raw video data.
crop_top
Field related to raw data. Specifies the number of pixels cropped from the top. Default is 0. This parameter is only applicable to raw video data.
crop_right
Field related to raw data. Specifies the number of pixels cropped from the right. Default is 0. This parameter is only applicable to raw video data.
crop_bottom
Field related to raw data. Specifies the number of pixels cropped from the bottom. Default is 0. This parameter is only applicable to raw video data.
rotation
Field related to raw data. Specifies whether the incoming video frame should be rotated clockwise. Possible values are 0, 90, 180, 270. Default is 0.
timestamp
Timestamp of the incoming video frame, in milliseconds. Incorrect timestamps can cause frame drops or audio-video desynchronization.
egl_context
This parameter is only applicable to Texture format video data. - When using the OpenGL interface defined by Khronos (javax.microedition.khronos.egl.), set the eglContext to this field. - When using the OpenGL interface defined by Android (android.opengl.), set the eglContext to this field.
egl_type
This parameter is only applicable to Texture format video data. Refers to the EGL context type.
texture_id
This parameter is only applicable to Texture format video data. Refers to the Texture ID of the video frame.
matrix
This parameter is only applicable to Texture format video data. Refers to an input 4x4 transformation matrix, typically an identity matrix.
metadata
The metadata buffer of the video frame. The default value is None.
alpha_buffer
Alpha channel data output by the portrait segmentation algorithm. This data matches the size of the video frame, with each pixel value ranging from [0,255], where 0 represents the background and 255 represents the foreground (portrait). You can set this parameter to render the video background with various effects, such as transparency, solid color, images, or videos.
fill_alpha_buffer
[For BGRA or RGBA only] Whether to extract alphaBuffer from BGRA or RGBA data:
1: Yes. Set it to true if you do not explicitly specify thealphaBuffer.0: (Default) No.
alpha_mode
The relative position between alphaBuffer and the frame:
0: (Default) Normal frame.1:alphaBufferis above the frame.2:alphaBufferis below the frame.3:alphaBufferis on the left of the frame.4:alphaBufferis on the right of the frame.
color_space
The color space of the video frame. See ColorSpaceType.
PcmAudioFrame
PCM audio data.
@dataclass(kw_only=True)
class PcmAudioFrame:
data: bytearray = None
samples_per_channel: int = 0
bytes_per_sample: int = 0
number_of_channels: int = 0
sample_rate: int = 0
timestamp: int = 0| Parameter | Description |
|---|---|
data | PCM audio data to be sent. |
timestamp | Timestamp of the audio data collection (milliseconds). |
samples_per_channel | Number of audio samples collected per channel in 10 milliseconds. |
bytes_per_sample | Number of bytes per sample. |
number_of_channels | Number of channels per sample. |
sample_rate | Sampling rate (Hz). The minimum value is 8000. |
RTCConnConfig
RTC connection configuration.
@dataclass(kw_only=True)
class RTCConnConfig:
auto_subscribe_audio: int = 0
auto_subscribe_video: int = 0
enable_audio_recording_or_playout: int = 0
max_send_bitrate: int = 0
min_port: int = 0
max_port: int = 0
audio_subs_options: 'AudioSubscriptionOptions' = field(default_factory=AudioSubscriptionOptions)
client_role_type: ClientRoleType = ClientRoleType.CLIENT_ROLE_BROADCASTER
channel_profile: ChannelProfileType = ChannelProfileType.CHANNEL_PROFILE_LIVE_BROADCASTING
audio_recv_media_packet: int = 0
audio_recv_encoded_frame: int = 0
video_recv_media_packet: int = 0| Parameter | Description |
|---|---|
auto_subscribe_audio | Whether to automatically subscribe to all audio tracks. - 0: Do not automatically subscribe to all audio tracks. - 1: (Default) Automatically subscribe to all audio tracks. |
auto_subscribe_video | Whether to automatically subscribe to all video tracks. - 0: Do not automatically subscribe to all video tracks. - 1: (Default) Automatically subscribe to all video tracks. |
enable_audio_recording_or_playout | Whether to enable audio recording or playout. - 0: Disable audio recording or playout. - 1: Enable audio recording or playout. |
max_send_bitrate | The maximum send bitrate. |
min_port | The minimum port. |
max_port | The maximum port. |
audio_subs_options | Audio subscription options. See AudioSubscriptionOptions for details. |
client_role_type | User role. See ClientRoleType for details. |
channel_profile | Channel profile. See ChannelProfileType for details. |
audio_recv_media_packet | Whether to receive audio media packets. - 0: Do not receive. - 1: Receive. |
video_recv_media_packet | Whether to receive video media packets. - 0: Do not receive. - 1: Receive. |
RTCConnInfo
The RTC connection information.
@dataclass(frozen=True, kw_only=True)
class RTCConnInfo():
id: int
channel_id: str
state: int
local_user_id: str
internal_uid: int| Parameter | Description |
|---|---|
id | The ID of the RTC connection. |
channel_id | RTC channel ID. If you have not called the connect method, this parameter returns None. |
state | Connection state: - 1: Network connection disconnected. - 2: Establishing network connection. - 3: Network connected. - 4: Re-establishing network connection. - 5: Network connection failed. |
local_user_id | Local user ID. |
internal_uid | For internal use only. |
SenderOptions
Configuration of the local video track object that includes the video source for sending encoded video data.
@dataclass(kw_only=True)
class SenderOptions:
target_bitrate: int
cc_mode: TCcMode = TCcMode.CC_ENABLED
codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE| Parameter | Description |
|---|---|
target_bitrate | The target encoding bitrate (Kbps) of the current encoder, which is an estimated value by the SDK based on the current network conditions. |
cc_mode | Congestion control mode. See TCcMode for details. |
codec_type | Video codec type. See VideoCodecType for details. |
VideoDimensions
Resolution of the video encoding.
@dataclass(kw_only=True)
class VideoDimensions:
width: int
height: int| Parameter | Description |
|---|---|
width | Width of the video frame (pixels). |
height | Height of the video frame (pixels). |
VideoEncoderConfiguration
Video encoding properties.
@dataclass(kw_only=True)
class VideoEncoderConfiguration:
dimensions: VideoDimensions
codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE
frame_rate: int = 15
bitrate: int = 0
min_bitrate: int = 0
orientation_mode: int = 0
degradation_preference: int = 0
mirror_mode: int = 0
encode_alpha: int = 0| Parameter | Description |
|---|---|
codec_type | Video codec type. See VideoCodecType for details. |
dimensions | Resolution of the video encoding (px). This parameter measures the encoding quality, represented as width × height. The default value is 640 × 360. Users can set the resolution as needed. See VideoDimensions for details. |
frame_rate | Frame rate of the video encoding (fps), default is 15. |
bitrate | Video encoding bitrate, in Kbps. - 0: (Recommended) Standard bitrate mode. In this mode, the video bitrate is twice the base bitrate. - -1: Adapted bitrate mode. In this mode, the video bitrate is the same as the base bitrate. When this mode is selected for live streaming, the video frame rate may be lower than the set value. |
min_bitrate | Minimum encoding bitrate, in Kbps. The SDK will automatically adjust the video encoding bitrate based on network conditions. Setting this parameter higher than the default value can force the video encoder to output higher quality images, but may cause packet loss and affect video playback fluency under poor network conditions, leading to stuttering. Therefore, unless there are special requirements for image quality, it is recommended not to modify this parameter. Note: This parameter is only applicable in live streaming scenarios. |
orientation_mode | Orientation mode of the video encoding: - 0: (Default) In this mode, the SDK outputs the video orientation consistent with the captured video. The receiver will rotate the video based on the received video rotation information. This mode is suitable for scenarios where the receiver can adjust the video orientation. If the captured video is in landscape mode, the output video will also be in landscape mode. If the captured video is in portrait mode, the output video will also be in portrait mode. - 1: In this mode, the SDK outputs the video in landscape mode. If the captured video is in portrait mode, the video encoder will crop it. This mode is suitable for scenarios where the receiver cannot adjust the video orientation, such as direct push to CDN. - 2: In this mode, the SDK outputs the video in portrait mode. If the captured video is in landscape mode, the video encoder will crop it. This mode is suitable for scenarios where the receiver cannot adjust the video orientation, such as direct push to CDN. |
degradation_preference | Video encoding degradation preference when bandwidth is limited: - 0: (Default) When bandwidth is limited, the video encoding prioritizes reducing the frame rate while maintaining video quality. This degradation preference is suitable for scenarios where image quality is prioritized. - 1: When bandwidth is limited, the video encoding prioritizes reducing video quality while maintaining the frame rate. This degradation preference is suitable for scenarios where smoothness is prioritized and quality reduction is acceptable. - 2: When bandwidth is limited, the video encoding reduces both frame rate and video quality. The reduction in 2 is less than in 0 and 1, suitable for scenarios with limited smoothness and quality. - 3: When bandwidth is limited, the video encoding prioritizes reducing the frame rate. |
mirror_mode | Whether to enable mirror mode when sending encoded video, only affecting the video seen by the remote user. - 0: (Default) The mirror mode is determined by the SDK. - 1: Enable mirror mode. - 2: Disable mirror mode. |
encode_alpha | Whether to encode and send the alpha data to the remote user when the video frame contains alpha data. - 0: (Default) Do not encode or send the alpha data. - 1: Encode and send the alpha data. |
VideoFrame
Properties of the video frame. The video data format is YUV420. The buffer is given as a pointer to a pointer. This interface cannot modify the pointer of the buffer, only the content of the buffer.
@dataclass(kw_only=True)
class VideoFrame():
type: int = 0
width: int = 0
height: int = 0
y_stride: int = 0
u_stride: int = 0
v_stride: int = 0
y_buffer: bytearray = None
u_buffer: bytearray = None
v_buffer: bytearray = None
rotation: int = 0
render_time_ms: int = 0
avsync_type: int = 0
metadata: bytearray = None
shared_context: str = None
texture_id: int = 0
matrix: list = None
alpha_buffer: bytearray = None
alpha_mode: int = 0| Parameter | Description |
|---|---|
type | Type of video data: - 1: Raw data. - 2: Raw data. - 3: Texture. |
width | Width of the video frame (pixels). |
height | Height of the video frame (pixels). |
y_stride | Stride of the Y buffer in the YUV data. |
u_stride | Stride of the U buffer in the YUV data. |
v_stride | Stride of the V buffer in the YUV data. |
y_buffer | Pointer to the pointer of the Y buffer in the YUV data. |
u_buffer | Pointer to the pointer of the U buffer in the YUV data. |
v_buffer | Pointer to the pointer of the V buffer in the YUV data. |
rotation | Clockwise rotation angle of the frame before rendering the video. Currently supports 0 degrees, 90 degrees, 180 degrees, and 270 degrees. |
render_time_ms | Timestamp of the external audio frame. This parameter is mandatory. You can use this timestamp to restore the order of audio frames; in scenarios with video (including those using external video sources), this parameter can help achieve audio-video synchronization. |
avsync_type | Reserved parameter. |
metadata | The metadata buffer of the video frame. |
shared_context | The EGL context. Only applicable to Texture format video data. |
texture_id | The Texture ID of the video frame. Only applicable to Texture format video data. |
matrix | The input 4x4 transformation matrix. Only applicable to Texture format video data. |
alpha_buffer | The alpha channel data of the video frame. The data matches the size of the video frame. |
alpha_mode | The relative position between alpha_buffer and the frame. - 0: (Default) Normal frame. - 1: alpha_buffer is above the frame. - 2: alpha_buffer is below the frame. - 3: alpha_buffer is on the left of the frame. - 4: alpha_buffer is on the right of the frame. |
VideoSubscriptionOptions
Video subscription options.
@dataclass(kw_only=True)
class VideoSubscriptionOptions:
type: VideoStreamType = VideoStreamType.VIDEO_STREAM_HIGH
encodedFrameOnly: bool = False| Parameter | Description |
|---|---|
type | Type of video to subscribe to. See VideoStreamType for details. |
encodedFrameOnly | Whether to subscribe only to encoded video data: - 0: (Default) Subscribe only to decoded video data. - 1: Subscribe only to encoded video data. |
Enums
AreaCode
Server access regions. This feature is an advanced setting and is used in scenarios with access security restrictions. The region code supports bitwise operations. After specifying the access region, the SDK will connect to the Agora servers within the specified region.
| Enumeration | Description |
|---|---|
AREA_CODE_CN | 0x00000001: Mainland China. |
AREA_CODE_NA | 0x00000002: North America. |
AREA_CODE_EU | 0x00000004: Europe. |
AREA_CODE_AS | 0x00000008: Asia (excluding Mainland China). |
AREA_CODE_JP | 0x00000010: Japan. |
AREA_CODE_IN | 0x00000020: India. |
AREA_CODE_GLOB | 0xFFFFFFFF: (Default) Global. |
AudioCodecType
Audio codec types.
| Enumeration | Description |
|---|---|
AUDIO_CODEC_OPUS | 1: OPUS |
AUDIO_CODEC_PCMA | 3: PCMA |
AUDIO_CODEC_PCMU | 4: PCMU |
AUDIO_CODEC_G722 | 5: G722 |
AUDIO_CODEC_AACLC | 8: AAC LC |
AUDIO_CODEC_HEAAC | 9: HE AAC |
AUDIO_CODEC_HEAAC2 | 11: HEAAC2 |
AudioProfileType
Audio profiles.
AUDIO_PROFILE_DEFAULT
0: Default audio profile.
- In communication scenarios: 16 kHz sampling rate, voice codec, mono, maximum encoding bitrate of 16 Kbps.
- In live broadcast scenarios: 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 64 Kbps.
AUDIO_PROFILE_SPEECH_STANDARD
1: Specifies a 32 kHz sampling rate, voice codec, mono, maximum encoding bitrate of 18 Kbps.
AUDIO_PROFILE_MUSIC_STANDARD
2: Specifies a 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 64 Kbps.
AUDIO_PROFILE_MUSIC_STANDARD_STEREO
3: Specifies a 48 kHz sampling rate, music codec, stereo, maximum encoding bitrate of 80 Kbps.
AUDIO_PROFILE_MUSIC_HIGH_QUALITY
4: Specifies a 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 96 Kbps.
AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO
5: Specifies a 48 kHz sampling rate, music codec, stereo, maximum encoding bitrate of 128 Kbps.
AUDIO_PROFILE_IOT
6: Specifies a 16 kHz sampling rate, voice codec, mono, with echo cancellation algorithm AES.
AUDIO_PROFILE_NUM
7: Enumeration boundary.
AudioScenarioType
Audio scenario types.
| Enumeration | Description |
|---|---|
AUDIO_SCENARIO_DEFAULT | 0: (Default) Automatic scenario, automatically matches the appropriate audio quality based on user role and audio routing. |
AUDIO_SCENARIO_GAME_STREAMING | 3: High-quality scenario, suitable for music-centric scenarios. |
AUDIO_SCENARIO_CHATROOM | 5: Chatroom scenario, suitable for scenarios where users frequently switch between speaking and listening. In this scenario, audience members will receive a prompt to request microphone permission. |
AUDIO_SCENARIO_CHORUS | 7: Chorus scenario, suitable for real-time chorus with extremely low latency under good network conditions. |
AUDIO_SCENARIO_MEETING | 8: Meeting scenario, suitable for multi-person meetings focused on voice. |
AUDIO_SCENARIO_NUM | 9: Enumeration boundary. |
ChannelProfileType
Channel usage scenarios.
| Enumeration | Description |
|---|---|
CHANNEL_PROFILE_COMMUNICATION | 0: (Default) Communication scenario. Recommended for channels with only two users. |
CHANNEL_PROFILE_LIVE_BROADCASTING | 1: Live broadcasting scenario. This scenario has two user roles: host and audience. Hosts can publish and receive audio and video streams, while the audience can only receive streams. Suitable for voice chat rooms, video live broadcasts, interactive large classes, etc. |
ClientRoleType
User roles.
| Enumeration | Description |
|---|---|
CLIENT_ROLE_BROADCASTER | 1: Broadcaster. Broadcasters can both send and receive media streams. |
CLIENT_ROLE_AUDIENCE | 2: Audience. Audience can only receive media streams. |
TCcMode
Congestion control modes.
| Enumeration | Description |
|---|---|
CC_ENABLED | 0: Enable congestion control mode. |
CC_DISABLED | 1: Disable congestion control mode. |
VideoCodecType
Video codec types.
| Enumeration | Description |
|---|---|
VIDEO_CODEC_NONE | 0: None |
VIDEO_CODEC_VP8 | 1: VP8 |
VIDEO_CODEC_H264 | 2: H.264 |
VIDEO_CODEC_H265 | 3: H.265 |
VIDEO_CODEC_GENERIC | 6: Generic |
VIDEO_CODEC_GENERIC_H264 | 7: Generic H264 |
VIDEO_CODEC_AV1 | 12: AV1 |
VIDEO_CODEC_VP9 | 13: VP9 |
VIDEO_CODEC_GENERIC_JPEG | 20: JPEG |
VideoStreamType
Video stream types.
| Enumeration | Description |
|---|---|
VIDEO_STREAM_HIGH | 0: (Default) High stream. Higher resolution and bitrate. |
VIDEO_STREAM_LOW | 1: Low stream. Lower resolution and bitrate. |
