# Python (/en/api-reference/api-ref/server-gateway/python) > For AI agents: see the complete documentation index at [llms.txt](/llms.txt). Server Gateway SDK API reference for Python. ## Overview - `AgoraService` | Method | Description | | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | | [`initialize`](#initialize) | Initializes the `AgoraService` object. | | [`release`](#release) | Releases the `AgoraService` object. | | [`create_rtc_connection`](#create_rtc_connection) | Creates an RTC connection object. | | [`create_custom_audio_track_pcm`](#create_custom_audio_track_pcm) | Creates a local audio track object with a source that sends PCM audio data. | | [`create_custom_audio_track_encoded`](#create_custom_audio_track_encoded) | Creates a local audio track object with a source that sends encoded audio data. | | [`create_media_node_factory`](#create_media_node_factory) | Creates a media node factory object. | | [`create_custom_video_track_frame`](#create_custom_video_track_frame) | Creates a local video track object with a source that sends YUV format video data. | | [`create_custom_video_track_encoded`](#create_custom_video_track_encoded) | Creates a local video track object with a source that sends encoded video data. | | [`set_log_file`](#set_log_file) | Sets the path and size of the SDK log file. | - `RTCConnection` | Method | Description | | --------------------------------------------- | ------------------------------------- | | [`connect`](#connect) | Connects to the RTC channel. | | [`disconnect`](#disconnect) | Disconnects from the RTC channel. | | [`renew_token`](#renew_token) | Renews the token. | | [`register_observer`](#register_observer) | Registers an RTC connection observer. | | [`unregister_observer`](#unregister_observer) | Destroys the RTC connection observer. | | [`create_data_stream`](#create_data_stream) | Creates a data stream. | | [`send_stream_message`](#send_stream_message) | Sends a data stream message. | | [`get_agora_parameter`](#get_agora_parameter) | Gets the `AgoraParameter` object. | | [`get_local_user`](#get_local_user) | Gets the `LocalUser` object. | | [`release`](#rtc-connection-release) | Releases the RTC connection. | | [`enable_encryption`](#enable_encryption) | Enables encryption. | - `IRTCConnectionObserver` | Callback | Description | | ------------------------------------------------------------------- | -------------------------------------------------------------- | | [`on_connected`](#on_connected) | Occurs when the SDK connection state changes to `3`. | | [`on_disconnected`](#on_disconnected) | Occurs when the SDK connection state changes to `1`. | | [`on_token_privilege_will_expire`](#on_token_privilege_will_expire) | Occurs when the token will expire in 30 seconds. | | [`on_token_privilege_did_expire`](#on_token_privilege_did_expire) | Occurs when the token has expired. | | [`on_user_joined`](#on_user_joined) | Occurs when a remote user joins the channel. | | [`on_user_left`](#on_user_left) | Occurs when a remote user leaves the channel. | | [`on_stream_message_error`](#on_stream_message_error) | Occurs when there is an error receiving a data stream message. | | [`on_error`](#on_error) | Occurs when an error occurs. | | [`on_encryption_error`](#on_encryption_error) | Occurs when encryption errors occur. | - `LocalUser` | Method | Description | | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | [`set_user_role`](#set_user_role) | Sets the user role. | | [`set_audio_encoder_configuration`](#set_audio_encoder_configuration) | Sets the audio encoder configuration. | | [`publish_audio`](#publish_audio) | Sends the local audio track to the channel. | | [`unpublish_audio`](#unpublish_audio) | Stops sending the local audio track to the channel. | | [`publish_video`](#publish_video) | Sends the local video track to the channel. | | [`unpublish_video`](#unpublish_video) | Stops sending the local video track to the channel. | | [`subscribe_audio`](#subscribe_audio) | Subscribes to the audio stream sent by a specific remote user in the channel. | | [`subscribe_all_audio`](#subscribe_all_audio) | Subscribes to all audio streams sent by remote users in the channel. | | [`unsubscribe_audio`](#unsubscribe_audio) | Stops subscribing to the audio stream sent by a specific remote user in the channel. | | [`unsubscribe_all_audio`](#unsubscribe_all_audio) | Stops subscribing to all audio streams sent by remote users in the channel. | | [`set_playback_audio_frame_before_mixing_parameters`](#set_playback_audio_frame_before_mixing_parameters) | Sets the audio data format for the [`on_playback_audio_frame_before_mixing`](#on_playback_audio_frame_before_mixing) callback. | | [`register_audio_frame_observer`](#register_audio_frame_observer) | Registers an audio frame observer. | | [`unregister_audio_frame_observer`](#unregister_audio_frame_observer) | Destroys the [`IAudioFrameObserver`](#iaudioframeobserver) object. | | [`register_video_encoded_frame_observer`](#register_video_encoded_frame_observer) | Registers an encoded video frame observer. | | [`unregister_video_encoded_frame_observer`](#unregister_video_encoded_frame_observer) | Destroys the encoded video frame observer. | | [`subscribe_video`](#subscribe_video) | Subscribes to the video stream sent by a specific remote user in the channel. | | [`subscribe_all_video`](#subscribe_all_video) | Subscribes to all video streams sent by remote users in the channel. | | [`unsubscribe_video`](#unsubscribe_video) | Stops subscribing to the video stream sent by a specific remote user in the channel. | | [`unsubscribe_all_video`](#unsubscribe_all_video) | Stops subscribing to all video streams sent by remote users in the channel. | | [`register_local_user_observer`](#register_local_user_observer) | Registers a local user observer. | | [`unregister_local_user_observer`](#unregister_local_user_observer) | Destroys the local user observer. | | [`register_video_frame_observer`](#register_video_frame_observer) | Registers an [`IVideoFrameObserver`](#ivideoframeobserver) object. | | [`unregister_video_frame_observer`](#unregister_video_frame_observer) | Destroys the [`IVideoFrameObserver`](#ivideoframeobserver) object. | | [`get_rtc_connection`](#get_rtc_connection) | Gets the `RtcConnection` object. | | [`release`](#local-user-release) | Releases all resources of the `LocalUser` object. | | [`send_audio_meta_data`](#send_audio_meta_data) | Sends the audio metadata. | - `LocalUserObserver` | Callback | Description | | --------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | [`on_local_audio_track_state_changed`](#on_local_audio_track_state_changed) | Occurs when the state of a local audio track changes. | | [`on_user_audio_track_state_changed`](#on_user_audio_track_state_changed) | Occurs when the state of a remote audio track changes. | | [`on_stream_message`](#on_stream_message) | Occurs when a data stream message is received. | | [`on_user_info_updated`](#on_user_info_updated) | Occurs when user media information is updated. | | [`on_audio_metadata_received`](#on_audio_meta_data_received) | Occurs when the local user receives the audio metadata from the remote user. | - `LocalAudioTrack` | Method | Description | | ------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | [`set_enabled`](#set_enabled) | Enables or disables the local audio track. | | [`adjust_publish_volume`](#adjust_publish_volume) | Adjusts the audio publish volume. | | [`set_max_buffer_audio_frame_number`](#set_max_buffer_audio_frame_number) | Sets the maximum capacity of the audio send buffer. | | [`clear_sender_buffer`](#clear_sender_buffer) | Clears the current audio send buffer. | | [`set_send_delay_ms`](#set_send_delay_ms) | Sets the minimum data duration allowed to be sent by the audio send buffer. | | [`release`](#local-audio-track-release) | Releases the local audio track resources. | - `LocalVideoTrack` | Method | Description | | --------------------------------------------------------------------- | ------------------------------------------ | | [`set_enabled`](#local-video-track-set-enabled) | Enables or disables the local video track. | | [`set_video_encoder_configuration`](#set_video_encoder_configuration) | Sets the video encoder properties. | - `MediaNodeFactory` | Method | Description | | ------------------------------------------------------------------------- | --------------------------------------------- | | [`create_audio_pcm_data_sender`](#create_audio_pcm_data_sender) | Creates a PCM data sending module. | | [`create_audio_encoded_frame_sender`](#create_audio_encoded_frame_sender) | Creates an encoded audio data sending module. | | [`create_video_frame_sender`](#create_video_frame_sender) | Creates a YUV video frame sending module. | | [`create_video_encoded_image_sender`](#create_video_encoded_image_sender) | Creates an encoded video sending module. | | [`release`](#media-node-factory-release) | Releases the `MediaNodeFactory` object. | - `AudioPcmDataSender` | Method | Description | | --------------------------------------------- | ----------------------------------------------------- | | [`send_audio_pcm_data`](#send_audio_pcm_data) | Sends PCM format audio data to the local audio track. | - `AudioEncodedFrameSender` | Method | Description | | ------------------------------------------------------- | ------------------------------------------------------ | | [`send_encoded_audio_frame`](#send_encoded_audio_frame) | Sends an encoded audio frame to the local audio track. | - `IAudioFrameObserver` | Callback | Description | | --------------------------------------------------------------------------------- | ----------------------------------------------------- | | [`on_playback_audio_frame_before_mixing`](#on_playback_audio_frame_before_mixing) | Occurs when the audio data before mixing is received. | - `VideoFrameSender` | Method | Description | | --------------------------------------- | -------------------------------------------------- | | [`send_video_frame`](#send_video_frame) | Sends a YUV format video frame to the video track. | - `IVideoFrameObserver` | Callback | Description | | ----------------------- | -------------------------------------- | | [`on_frame`](#on_frame) | Occurs when a video frame is received. | - `VideoEncodedImageSender` | Method | Description | | ------------------------------------------------------- | ------------------------------------------------ | | [`send_encoded_video_image`](#send_encoded_video_image) | Sends an encoded video frame to the video track. | - `IVideoEncodedFrameObserver` | Callback | Description | | --------------------------------------------------- | ---------------------------------------------------- | | [`on_encoded_video_frame`](#on_encoded_video_frame) | Occurs when the SDK receives an encoded video frame. | ## AgoraService The `AgoraService` class serves as the entry point for the SDK, allowing you to create media tracks and connections. ### `initialize` Initializes the `AgoraService` object. ```python def initialize(self, config: AgoraServiceConfig) ``` #### Parameters | Parameter | Description | | --------- | ------------------------------------------------------------------------------ | | `config` | Initialization configuration. See [`AgoraServiceConfig`](#agoraserviceconfig). | #### Returns - `0`: Success. - < `0`: Failure. ### `release` Releases the `AgoraService` object. ```python def release(self) ``` ### `create_rtc_connection` Creates an RTC connection object. ```python def create_rtc_connection(self, con_config: RTCConnConfig) ``` > **Caution** > > To create multiple connections, Agora recommends using multiple processes to avoid memory leaks and other issues. #### Parameters | Parameter | Description | | ------------ | -------------------------------------------------------------------- | | `con_config` | RTC connection configuration. See [`RTCConnConfig`](#rtcconnconfig). | #### Returns - `RTCConnection` object, if the method call succeeds. - `None`, if the method call fails. ### `create_custom_audio_track_pcm` Creates a local audio track object with an audio source that sends PCM audio data. This track object can be used to send audio data in PCM format. ```python def create_custom_audio_track_pcm(self, audio_pcm_data_sender: AudioPcmDataSender) ``` #### Parameters | Parameter | Description | | ----------------------- | ------------------------------------------------------------------------------ | | `audio_pcm_data_sender` | PCM format audio data source. See [`AudioPcmDataSender`](#audiopcmdatasender). | #### Returns - `LocalAudioTrack` object, if the method call succeeds. - `None`, if the method call fails. ### `create_custom_audio_track_encoded` Creates a local audio track object with an audio source that sends encoded audio data. This track object can be used to send encoded audio data, such as Opus format encoded data. ```python def create_custom_audio_track_encoded(self, audio_encoded_frame_sender: AudioEncodedFrameSender, mix_mode: int) ``` #### Parameters | Parameter | Description | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `audio_encoded_frame_sender` | Audio source that sends encoded audio data. See [`AudioEncodedFrameSender`](#audioencodedframesender). | | `mix_mode` | Mix mode for the encoded audio in the channel. - `0`: Mix all audio tracks in the channel. - `1`: Do not mix audio. | #### Returns - `LocalAudioTrack` object, if the method call succeeds. - `None`, if the method call fails. ### `create_media_node_factory` Creates a media node factory object. ```python def create_media_node_factory(self) ``` #### Returns - `MediaNodeFactory` object, if the method call succeeds. - `None`, if the method call fails. ### `create_custom_video_track_frame` Creates a local video track object with a video source that sends YUV format video data. This track object can be used to send video data in YUV format. ```python def create_custom_video_track_frame(self, video_frame_sender: VideoFrameSender) ``` #### Parameters | Parameter | Description | | -------------------- | ---------------------------------------------------------------------------- | | `video_frame_sender` | YUV format video source object. See [`VideoFrameSender`](#videoframesender). | #### Returns - `LocalVideoTrack` object, if the method call succeeds. - `None`, if the method call fails. ### `create_custom_video_track_encoded` Creates a local video track object with a video source that sends encoded video data. This track object can be used to send encoded video data, such as H.264 or VP8 format encoded data. ```python def create_custom_video_track_encoded(self, video_encoded_frame_sender: VideoEncodedImageSender, options: SenderOptions) ``` #### Parameters | Parameter | Description | | ---------------------------- | ------------------------------------------------------------------------------------------------------ | | `video_encoded_frame_sender` | Video source that sends encoded video data. See [`VideoEncodedImageSender`](#videoencodedimagesender). | | `options` | Video track settings. See [`SenderOptions`](#senderoptions). | #### Returns - `LocalVideoTrack` object, if the method call succeeds. - `None`, if the method call fails. ### `set_log_file` Sets the path and size of the SDK log file. ```python def set_log_file(self, log_path: str, log_size: int = 512 * 1024) ``` The SDK generates 5 SDK log files and 5 API call log files by default, following these rules: - The SDK log files names: `agorasdk.log`, `agorasdk.1.log`, `agorasdk.2.log`, `agorasdk.3.log`, `agorasdk.4.log`. - The API call log files names: `agoraapi.log`, `agoraapi.1.log`, `agoraapi.2.log`, `agoraapi.3.log`, `agoraapi.4.log`. - Each SDK log file has the default size of 1,024 KB; each API call log file has the default size of 2,048 KB. All log files are UTF-8 encoded. - The latest logs are always written to `agorasdk.log` and `agoraapi.log`. When `agorasdk.log` is full, the SDK handles the log files as follows: - Deletes the `agorasdk.4.log` file, if it exists. - Renames `agorasdk.3.log` to `agorasdk.4.log`. - Renames `agorasdk.2.log` to `agorasdk.3.log`. - Renames `agorasdk.1.log` to `agorasdk.2.log`. - Creates a new `agorasdk.log` file. The `agoraapi.log` file follows the same rotation rules as `agorasdk.log`. #### Parameters | Parameter | Description | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `log_path` | The full path of the log file. The log file is UTF-8 encoded. | | `log_size` | The size of a single `agorasdk.log` log file in KB, with a range of \[128, 20480]. The default is 1,024 KB. If you set `file_size` to less than 128 KB, the SDK automatically adjusts it to 128 KB; if you set `file_size` to more than 20,480 KB, the SDK automatically adjusts it to 20,480 KB. | #### Returns - `0`: Success. - < `0`: Failure. ## RTCConnection The `RtcConnection` class is used to manage the connection between the SDK and the RTC channel. After the connection is complete, you can send and receive media streams in the RTC channel through the `LocalUser` object. Connecting to a channel is an asynchronous operation, and you can use the [`IRtcConnectionObserver`](#irtcconnectionobserver) object to listen for connection status changes and events. The `RtcConnection` object also monitors whether remote users join or leave the channel. ### `connect` Connects to an RTC channel. ```python def connect(self, token: str, chan_id: str, user_id: str) -> int ``` When the method is successful, the connection status changes from to `2`. Depending on whether the connection is successful, the connection status changes to `3` or `5`. The SDK triggers the [`on_connected`](#on_connected) or [`on_disconnected`](#on_disconnected) callback to notify you of the connection status change. #### Parameters | Parameter | Description | | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `token` | RTC token. See [Use tokens](/en/realtime-media/rtc/build/authenticate-users/authentication-workflow). | | `chan_id` | Channel name. Must be a string and no more than 64 bytes. The supported character types include: - All lowercase letters: a to z; - All uppercase letters: A to Z; - All Arabic numerals: 0 to 9; - Space; - Punctuation and other special symbols: `!`, `#`, `$`, `%`, `&`, `(`, `)`, `+`, `-`, `:`, `;`, `<`, `=`, `.`, `>`, `?`, `@`, `[`, `]`, `^`, `_`, `{`, `}`, `\|`, `~`, `,` | | `user_id` | The local user ID. No more than 64 bytes. If you do not set the user ID or set it to `None`, the SDK automatically returns a user ID in the `on_connected` callback. You must record and manage this ID yourself. Depending on the user ID type you selected in the [`initialize`](#initialize) method, the supported character set is different: The supported character set for integer user IDs includes 10 digits 0-9. The supported character set for string user IDs includes: - 26 lowercase English letters: a-z; - 26 uppercase English letters: A-Z; - 10 digits: 0-9; - Space; - `!`, `#`, `$`, `%`, `&`, `(`, `)`, `+`, `-`, `:`, `;`, `<`, `=`, `.`, `>`, `?`, `@`, `[`, `]`, `^`, `_`, `{`, `}`, `\|`, `~`, `,` | #### Returns - `0`: Success. - < `0`: Failure. - `-2`: The parameters are invalid. - `-8`: The current connection status is not `1`. ### `disconnect` Disconnects from the RTC channel. When the SDK disconnects from the channel, the connection status changes to `1`. You can also learn about the disconnection through the [`on_disconnected`](#on_disconnected) callback. ```python def disconnect(self) -> int ``` #### Returns - `0`: Success. - < `0`: Failure. ### `renew_token` Renews the token. The token expires after a certain period of time. When you receive the [`on_token_privilege_will_expire`](#on_token_privilege_will_expire) callback, you must generate a new Token from your server and use this method to update the token. Otherwise, the SDK disconnects from the channel. ```python def renew_token(self, token: str) -> int ``` #### Parameters | Parameter | Description | | --------- | -------------- | | `token` | The new token. | #### Returns - `0`: Success. - < `0`: Failure. ### `register_observer` Registers the RTC connection observer. ```python def register_observer(self, conn_observer: IRTCConnectionObserver) -> int ``` #### Parameters | Parameter | Description | | --------------- | ----------------------------------------------------------- | | `conn_observer` | [`IRTCConnectionObserver`](#irtcconnectionobserver) object. | #### Returns - `0`: Success. - < `0`: Failure. ### `unregister_observer` Destroys the RTC connection observer. ```python def unregister_observer(self) -> int ``` #### Returns - `0`: Success. - < `0`: Failure. ### `create_data_stream` Creates a data stream. ```python def create_data_stream(self, reliable: bool, ordered: bool) -> int ``` Each user can create up to 5 data streams in each channel. > **Note** > > - Agora does not support setting `reliable` to `1` and `ordered` to `0`. > - If you need the data packet to arrive immediately at the receiver, do not set the `ordered` parameter to `1`. #### Parameters | Parameter | Description | | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `reliable` | Whether the data stream is reliable: - `1`: The receiver receives the data sent by the sender within 5 seconds. - `0`: The receiver does not guarantee to receive the data. If the data is lost, it does not report an error. | | `ordered` | Whether the data stream is ordered: - `1`: The receiver receives the data packets in the order sent by the sender. - `0`: The receiver does not guarantee to receive the data packets in the order sent by the sender. | #### Returns - `stream_id`: Returns the data stream ID, if the method call succeeds. - `None`: Failure. ### `send_stream_message` Sends a data stream message. ```python def send_stream_message(self, stream_id: int, data: str) -> int ``` The method sends a data stream message to all users in the channel. The SDK implements the following restrictions for this method: - Up to 30 packets per second, and each packet is no more than 1 KB. - Up to 6 KB of data per second per client. - Each client can have up to 5 data channels at the same time. After successfully calling this method, the remote end triggers the [`on_stream_message`](#on_stream_message) callback, and remote users can obtain the received stream messages in this callback. > **Note** > > - Ensure that you have called [`create_data_stream`](#create_data_stream) before calling this method. > - This method is only applicable to the host in live streaming use-cases. #### Parameters | Parameter | Description | | ----------- | -------------------------------------------------------------------------------------- | | `stream_id` | Data stream ID. You can obtain it through [`create_data_stream`](#create_data_stream). | | `data` | Data to send. | #### Returns - `0`: Success. - < `0`: Failure. ### get\_agora\_parameter Gets the `AgoraParameter` object. ```python def get_agora_parameter(self) ``` #### Returns - `AgoraParameter` object, if the method call succeeds. - `None`, if the method call fails. ### `get_local_user` Gets the `LocalUser` object. Each connection corresponds to one local user. ```python def get_local_user(self) ``` #### Returns - `LocalUser` object, if the method call succeeds. - `None`, if the method call fails. ### `release` Releases the RTC connection. ```python def release(self) ``` ### `enable_encryption` ```python def enable_encryption(self, enabled: int, config: EncryptionConfig) ``` Enables encryption. > **Info: Since** > > v2.2.2. This method needs to be called before [`connect`](#connect). You need to set the encryption mode and encryption configuration in this method. #### Parameters **`enabled`** Whether to enable encryption functionality: - `1`: Enable - `0`: Disable **`config`** The encryption configuration: [`EncryptionConfig`](#encryptionconfig). ## IRTCConnectionObserver The `IRTCConnectionObserver` class, used to listen to the connection status of the SDK. ### `on_connected` Occurs when the connection state of the SDK changes to `3`. ```python def on_connected(self, agora_rtc_conn, conn_info, reason) ``` #### Parameters **`agora_rtc_conn`** The RTC connection object. See [`RtcConnection`](#rtcconnection). **`conn_info`** Connection information. See [`RTCConnInfo`](#rtcconninfo). **`reason`** Connection state change reason: - `0`: Establishing a network connection. - `1`: Successfully joined the channel. - `2`: Network connection interrupted. - `3`: Network connection forbidden by the server. - `4`: Failed to join the channel. - `5`: Left the channel. - `6`: Invalid App ID. Please change to a valid App ID and try again. - `7`: Invalid channel name. Please change to a valid channel name and try again. - `8`: The generated token is invalid. Generally, the following reasons: - App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token. - The uid specified in [`connect`](#connect) when joining the channel does not match the uid used when generating the Token. - `9`: The generated Token is invalid. - `10`: This user is banned by the server. - `11`: SDK attempts to reconnect due to setting a proxy server. - `12`: The network connection state changes due to updating the Token. - `13`: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator. - `14`: SDK connection to the server times out, enters the automatic reconnection state. - `15`: Rejoined the channel successfully. - `16`: SDK and the server lose connection. - `17`: The connection state changes due to the echo test. - `18`: The user changed the local IP address. ### `on_disconnected` Occurs when the connection state of the SDK changes to `1`. ```python def on_disconnected(self, agora_rtc_conn, conn_info, reason) ``` #### Parameters **`agora_rtc_conn`** The RTC connection object. See [`RtcConnection`](#rtcconnection). **`conn_info`** The connection information. See [`RTCConnInfo`](#rtcconninfo). **`reason`** Connection state change reason: - `0`: Establishing a network connection. - `1`: Successfully joined the channel. - `2`: Network connection interrupted. - `3`: Network connection forbidden by the server. - `4`: Failed to join the channel. - `5`: Left the channel. - `6`: Invalid App ID. Please change to a valid App ID and try again. - `7`: Invalid channel name. Please change to a valid channel name and try again. - `8`: The generated token is invalid. Generally, the following reasons: - App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token. - The uid specified in [`connect`](#connect) when joining the channel does not match the uid used when generating the Token. - `9`: The generated Token is invalid. - `10`: This user is banned by the server. - `11`: SDK attempts to reconnect due to setting a proxy server. - `12`: The network connection state changes due to updating the Token. - `13`: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator. - `14`: SDK connection to the server times out, enters the automatic reconnection state. - `15`: Rejoined the channel successfully. - `16`: SDK and the server lose connection. - `17`: The connection state changes due to the echo test. - `18`: The user changed the local IP address. ### `on_token_privilege_will_expire` Occurs when the token will expire in 30 seconds. ```python def on_token_privilege_will_expire(self, agora_rtc_conn, token) ``` In the call process, if the Token is about to expire, the SDK triggers this callback 30 seconds before the Token expires to remind the app to update the Token. When you receive this callback, you need to generate a new Token on the server side and call [`renew_token`](#renew_token) to pass the newly generated Token to the SDK. #### Parameters | Parameter | Description | | ---------------- | ------------------------------------------------------------- | | `agora_rtc_conn` | RTC connection object. See [`RtcConnection`](#rtcconnection). | | `token` | The token that is about to expire. | ### `on_token_privilege_did_expire` Occurs when the token expires. ```python def on_token_privilege_did_expire(self, agora_rtc_conn) ``` In the call process, if the token expires, the SDK triggers this callback to remind the app to update the Token. When you receive this callback, you need to generate a new Token on the server side and call [`connect`](#connect) to reconnect the channel. #### Parameters | Parameter | Description | | ---------------- | ------------------------------------------------------------- | | `agora_rtc_conn` | RTC connection object. See [`RtcConnection`](#rtcconnection). | ### `on_user_joined` Occurs when a remote user joins the channel. You can get the remote user's ID through this callback. ```python def on_user_joined(self, agora_rtc_conn, user_id) ``` #### Parameters | Parameter | Description | | ---------------- | ------------------------------------------------------------- | | `agora_rtc_conn` | RTC connection object. See [`RtcConnection`](#rtcconnection). | | `user_id` | The remote user's ID. | ### `on_user_left` Occurs when a remote user leaves the channel. You can get the reason for the remote user leaving the channel through the `reason` parameter. ```python def on_user_left(self, agora_rtc_conn, user_id, reason) ``` #### Parameters **`agora_rtc_conn`** The RTC connection object. See [`RtcConnection`](#rtcconnection). **`user_id`** The remote user's ID. **`reason`** Connection state change reason: - `0`: Establishing a network connection. - `1`: Successfully joined the channel. - `2`: Network connection interrupted. - `3`: Network connection forbidden by the server. - `4`: Failed to join the channel. - `5`: Left the channel. - `6`: Invalid App ID. Please change to a valid App ID and try again. - `7`: Invalid channel name. Please change to a valid channel name and try again. - `8`: The generated token is invalid. Generally, the following reasons: - App Certificate is enabled on the console, but the Token is not used when joining the channel. When App Certificate is enabled, you must use a Token. - The uid specified in [`connect`](#connect) when joining the channel does not match the uid used when generating the Token. - `9`: The generated Token is invalid. - `10`: This user is banned by the server. - `11`: SDK attempts to reconnect due to setting a proxy server. - `12`: The network connection state changes due to updating the Token. - `13`: The client IP address changes, possibly due to changes in network type or the IP or port of the network operator. - `14`: SDK connection to the server times out, enters the automatic reconnection state. - `15`: Rejoined the channel successfully. - `16`: SDK and the server lose connection. - `17`: The connection state changes due to the echo test. - `18`: The user changed the local IP address. ### `on_stream_message_error` Occurs when a data stream message error occurs. This callback indicates that the local user has not received the stream message sent by the remote user using the [`send_stream_message`](#send_stream_message) method. ```python def on_stream_message_error(self, agora_rtc_conn, user_id_str, stream_id, code, missed, cached) ``` #### Parameters | Parameter | Description | | ---------------- | ------------------------------------------------------------------ | | `agora_rtc_conn` | The RTC connection object. See [`RtcConnection`](#rtcconnection). | | `user_id_str` | The user ID that sends the message. | | `stream_id` | The stream ID of the received message. | | `code` | The code of the error that has occurred. | | `missed` | The number of lost messages. | | `cached` | The number of messages cached when the data stream is interrupted. | ### `on_error` Occurs when there is an error. This callback indicates that an error related to the network or media occurred during the SDK runtime. Generally, the error reported by the SDK means that the SDK cannot automatically recover and requires intervention from the App or user prompt. ```python def on_error(self, agora_rtc_conn, error_code, error_msg) ``` #### Parameters | Parameter | Description | | ---------------- | ----------------------------------------------------------------- | | `agora_rtc_conn` | The RTC connection object. See [`RtcConnection`](#rtcconnection). | | `error_code` | The error code. | | `error_msg` | The error description. | ### `on_encryption_error` ```python def on_encryption_error(self, agora_rtc_conn, error_type) ``` Occurs when encryption errors occur. > **Info: Since** > > v2.2.2. #### Parameters | Parameter | Description | | ---------------- | ----------------------------------------- | | `agora_rtc_conn` | [`RtcConnection`](#rtcconnection) object. | | `error_type` | Error code. | ## LocalUser The `LocalUser` class. Defines the behavior and state of the local user. Each RTC connection has its own user. You can get the local user through [`get_local_user`](#get_local_user). Each user has two roles: the host (can send and receive audio and video streams) and the audience (can only receive audio and video streams). ### `set_user_role` Sets the user role. You can call this method before or after the user connects to the channel. ```python def set_user_role(self, role) ``` > **Note** > > If the `role` set in the `token` parameter of [`connect`](#connect) is different from the role set here, the connection will fail. #### Parameters | Parameter | Description' | | --------- | --------------------------------------------------- | | `role` | User role. See [`ClientRoleType`](#clientroletype). | #### Returns - `0`: Success. - < `0`: Failure. ### `set_audio_encoder_configuration` Sets the audio encoding. The SDK applies the configuration to all audio tracks sent. ```python def set_audio_encoder_configuration(self, config) ``` #### Parameters | Parameter | Description | | --------- | -------------------------------------------------------------------------------------------- | | `config` | Audio encoding configuration. See [`AudioEncoderConfiguration`](#audioencoderconfiguration). | #### Returns - `0`: Success. - < `0`: Failure. ### `publish_audio` Publishes the local audio track to the channel. By default, all published audio tracks are mixed. ```python def publish_audio(self, agora_local_audio_track:LocalAudioTrack) ``` #### Parameters | Parameter | Description | | ------------------------- | ---------------------------------------------------------------------------- | | `agora_local_audio_track` | The local audio track to publish. See [`LocalAudioTrack`](#localaudiotrack). | #### Returns - `0`: Success. - < `0`: Failure. - `-5`: If the local user's role is not the host, this error is returned. ### `unpublish_audio` Stops publishing the local audio track to the channel. ```python def unpublish_audio(self, agora_local_audio_track:LocalAudioTrack) ``` #### Parameters | Parameter | Description | | ------------------------- | ------------------------------------------------------------------------------------ | | `agora_local_audio_track` | The local audio track to stop publishing. See [`LocalAudioTrack`](#localaudiotrack). | #### Returns - `0`: Success. - < `0`: Failure. ### `publish_video` Publishes the local video track to the channel. ```python def publish_video(self, agora_local_video_track:LocalVideoTrack) ``` #### Parameters | Parameter | Description | | ------------------------- | ---------------------------------------------------------------------------- | | `agora_local_video_track` | The local video track to publish. See [`LocalVideoTrack`](#localvideotrack). | #### Returns - `0`: Success. - < `0`: Failure. ### `unpublish_video` Stops publishing the local video track to the channel. ```python def unpublish_video(self, agora_local_video_track:LocalVideoTrack) ``` #### Parameters | Parameter | Description | | ------------------------- | ------------------------------------------------------------------------------------ | | `agora_local_video_track` | The local video track to stop publishing. See [`LocalVideoTrack`](#localvideotrack). | #### Returns - `0`: Success. - < `0`: Failure. ### `subscribe_audio` Subscribes to the audio stream sent by a specific remote user in the channel. ```python def subscribe_audio(self, user_id) ``` #### Parameters | Parameter | Description | | --------- | ------------------- | | `user_id` | The remote user ID. | #### Returns - `0`: Success. - < `0`: Failure. - `-2`: If the user does not exist or `user_id` is invalid, this error is returned. ### `subscribe_all_audio` Subscribes to all the audio streams sent by remote users in the channel. This method automatically subscribes to the audio streams sent by users who join the channel later. ```python def subscribe_all_audio(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### `unsubscribe_audio` Stops subscribing to the audio stream sent by a specific remote user in the channel. ```python def unsubscribe_audio(self, user_id) ``` #### Parameters | Parameter | Description | | --------- | ------------------- | | `user_id` | The remote user ID. | #### Returns - `0`: Success. - < `0`: Failure. - `-2`: If the user does not exist or `user_id` is invalid, this error is returned. ### `unsubscribe_all_audio` Stops subscribing to all the audio streams sent by remote users in the channel. This method automatically cancels the subscription of the audio streams sent by users who join the channel later. You must call [`subscribe_audio`](#subscribe_audio) or [`subscribe_all_audio`](#subscribe_all_audio) to restore the subscription of audio. ```python def unsubscribe_all_audio(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### `set_playback_audio_frame_before_mixing_parameters` Sets the audio data format reported by [`on_playback_audio_frame_before_mixing`](#on_playback_audio_frame_before_mixing). ```python def set_playback_audio_frame_before_mixing_parameters(self, channels, sample_rate_hz) ``` #### Parameters | Parameter | Description | | ---------------- | ----------------------------------------------------------------------------------------------------- | | `channels` | The number of audio channels, which can be set to `1` (mono) or `2` (stereo). | | `sample_rate_hz` | The sample rate of the audio data, which can be set to `8000`, `16000`, `32000`, `44100`, or `48000`. | #### Returns - `0`: Success. - < `0`: Failure. ### `register_audio_frame_observer` Registers the audio frame observer. You must implement the [`IAudioFrameObserver`](#iaudioframeobserver) class and register the callback based on your needs. ```python def register_audio_frame_observer(self, observer: IAudioFrameObserver, enable_vad: int, vad_configure) ``` #### Parameters **`observer`** [`IAudioFrameObserver`](#iaudioframeobserver) class. **`enable_vad`** Whether to enable the built-in voice activity detection (VAD): - `0`: Disable VAD. - `1`: Enable VAD. The SDK returns the VAD results in the [`on_playback_audio_frame_before_mixing`](#on_playback_audio_frame_before_mixing) callback. **`vad_configure`** The VAD configuration, an `AudioVadConfigV2` object. The SDK uses this parameter only when `enable_vad` is `1`. #### Returns - `0`: Success. - < `0`: Failure. ### `unregister_audio_frame_observer` Destroys the [`IAudioFrameObserver`](#iaudioframeobserver) object. ```python def unregister_audio_frame_observer(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### `register_video_encoded_frame_observer` Registers the encoded video frame observer. ```python def register_video_encoded_frame_observer(self, agora_video_encoded_frame_observer:IVideoEncodedFrameObserver) ``` #### Parameters | Parameter | Description | | ------------------------------------ | -------------------------------------------------------------------------------------------------- | | `agora_video_encoded_frame_observer` | The encoded video frame observer. See [`IVideoEncodedFrameObserver`](#ivideoencodedframeobserver). | #### Returns - `0`: Success. - < `0`: Failure. ### `unregister_video_encoded_frame_observer` Destroys the encoded video frame observer. ```python def unregister_video_encoded_frame_observer(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### `subscribe_video` Subscribes to the video stream sent by a specific remote user in the channel. ```python def subscribe_video(self, user_id, options:VideoSubscriptionOptions) ``` #### Parameters | Parameter | Description | | --------- | -------------------------------------------------------------------------------------- | | `user_id` | The remote user ID. | | `options` | The subscription options. See [`VideoSubscriptionOptions`](#videosubscriptionoptions). | #### Returns - `0`: Success. - < `0`: Failure. - `-2`: If the user does not exist or `user_id` is invalid, this error is returned. ### `subscribe_all_video` Subscribes to all the video streams sent by remote users in the channel. This method automatically subscribes to the video streams sent by users who join the channel later. ```python def subscribe_all_video(self, options:VideoSubscriptionOptions) ``` #### Parameters | Parameter | Description | | --------- | -------------------------------------------------------------------------------------- | | `options` | The subscription options. See [`VideoSubscriptionOptions`](#videosubscriptionoptions). | #### Returns - `0`: Success. - < `0`: Failure. ### `unsubscribe_video` Stops subscribing to the video stream sent by a specific remote user in the channel. ```python def unsubscribe_video(self, user_id) ``` #### Parameters | Parameter | Description | | --------- | ------------------- | | `user_id` | The remote user ID. | #### Returns - `0`: Success. - < `0`: Failure. - `-2`: If the user does not exist or `user_id` is invalid, this error is returned. ### `unsubscribe_all_video` Stops subscribing to all the video streams sent by remote users in the channel. This method automatically cancels the subscription of the video streams sent by users who join the channel later. You must call [`subscribe_video`](#subscribe_video) or [`subscribe_all_video`](#subscribe_all_video) to restore the subscription. ```python def unsubscribe_all_video(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### `register_local_user_observer` Registers the local user observer. ```python def register_local_user_observer(self, observer:ILocalUserObserver) ``` #### Parameters | Parameter | Description | | ---------- | ------------------------------------------------ | | `observer` | [`LocalUserObserver`](#localuserobserver) class. | #### Returns - `0`: Success. - < `0`: Failure. ### `unregister_local_user_observer` Destroys the local user observer. ```python def unregister_local_user_observer(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### register\_video\_frame\_observer Registers the [`IVideoFrameObserver`](#ivideoframeobserver) object. ```python def register_video_frame_observer(self, agora_video_frame_observer2:IVideoFrameObserver) ``` #### Parameters | Parameter | Description | | ----------------------------- | ---------------------------------------------------- | | `agora_video_frame_observer2` | [`IVideoFrameObserver`](#ivideoframeobserver) class. | #### Returns - `0`: Success. - < `0`: Failure. ### `unregister_video_frame_observer` Destroys the [`IVideoFrameObserver`](#ivideoframeobserver) object. ```python def unregister_video_frame_observer(self) ``` #### Returns - `0`: Success. - < `0`: Failure. ### `get_rtc_connection` Gets the `RtcConnection` object. ```python def get_rtc_connection(self) ``` #### Returns - `RtcConnection` object: Success. - `None`: Failure. ### `release` Releases all resources of the `LocalUser` object. ```python def release(self) ``` ### `send_audio_meta_data` ```python def send_audio_meta_data(self, data) ``` Sends the audio metadata. > **Info: Since** > > v2.2.0. A successful call of this method triggers the [`on_audio_metadata_received`](#on_audio_meta_data_received) callback on the remote end. The remote user can get the audio metadata in this callback. #### Parameters | Parameter | Description | | --------- | ------------------- | | `data` | The audio metadata. | #### Returns - `0`: Success. - < `0`: Failure. ## LocalUserObserver The `LocalUserObserver` class. ### `on_local_audio_track_state_changed` Triggered when the local audio track state changes. ```python def on_local_audio_track_state_changed(self, agora_local_user, agora_local_audio_track, state, error) ``` #### Parameters | Parameter | Description | | ------------------------- | ------------------------------------------------------------- | | `agora_local_user` | `LocalUserObserver` class. | | `agora_local_audio_track` | Local audio track. See [`LocalAudioTrack`](#localaudiotrack). | | `state` | The current state of the audio track. | | `error` | Error information. | ### `on_user_audio_track_state_changed` Occurs when the remote audio track state changes. ```python def on_user_audio_track_state_changed(self, agora_local_user, user_id, agora_remote_audio_track, state, reason, elapsed) ``` #### Parameters | Parameter | Description | | -------------------------- | -------------------------------------------------------------------- | | `agora_local_user` | The `LocalUserObserver` class. | | `user_id` | The remote user ID. | | `agora_remote_audio_track` | Remote audio track. | | `state` | The current state of the audio track. | | `reason` | The reason for the state change. | | `elapsed` | The time elapsed after the remote user connects to the channel (ms). | ### `on_stream_message` Callback for receiving data stream messages. This callback indicates that the local user has received the stream message sent by the remote user through the [`send_stream_message`](#send_stream_message) method. ```python def on_stream_message(self, agora_local_user, user_id, stream_id, data, length) ``` #### Parameters | Parameter | Description | | ------------------ | -------------------------------------- | | `agora_local_user` | `LocalUserObserver` class. | | `user_id` | The user ID that sends the message. | | `stream_id` | The Stream ID of the received message. | | `data` | The received data. | | `length` | The data length (bytes). | ### `on_user_info_updated` Occurs when the user media information is updated. ```python def on_user_info_updated(self, agora_local_user, user_id, msg, val) ``` #### Parameters | Parameter | Description | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `agora_local_user` | The `LocalUserObserver` class. | | `user_id` | The user ID that sends the message. | | `msg` | User media information: - `0`: The user has stopped publishing audio streams. - `1`: The user has stopped publishing video streams. - `4`: The user has enabled video functionality. | | `val` | Whether the user stops sending streams. - `0`: Do not stop sending streams. - `1`: Stop sending streams. | ### `on_audio_meta_data_received` ```python def on_audio_meta_data_received(self, agora_local_user, user_id, data): ``` Occurs when the remote audio metadata is received. This callback indicates that the local user has received the audio metadata sent by the remote user using the [`send_audio_meta_data`](#send_audio_meta_data) method. #### Parameters | Parameter | Description | | ------------------ | ------------------------------------------------ | | `agora_local_user` | The `LocalUserObserver` class | | `user_id` | The ID of the user who sends the audio metadata. | | `data` | The audio metadata. | ## LocalAudioTrack The `LocalAudioTrack` class is the base class for the local audio track. ### `set_enabled` Enables or disables the local audio track. After the local audio track is enabled, the SDK can capture, process, and encode local audio. ```python def set_enabled(self, enable) ``` #### Parameters | Parameter | Description | | --------- | ------------------------------------------------------------------------------------------------- | | `enable` | Whether to enable the audio track: - `1`: Enable the audio track. - `0`: Disable the audio track. | ### `adjust_publish_volume` Adjusts the audio publishing volume. ```python def adjust_publish_volume(self, volume) ``` #### Parameters | Parameter | Description | | --------- | ------------------------------------------------------------------------------------------------------------------- | | `volume` | The audio publishing volume. The value range is from \[0,100]. `0` means mute, and `100` means the original volume. | #### Returns - 0: Success. - < 0: Failure. ### `set_max_buffer_audio_frame_number` Sets the maximum capacity of the audio sending buffer. ```python def set_max_buffer_audio_frame_number(self, num) ``` In AI interaction applications, the frequency of sending audio frames by the sender may be uneven, for example, sending 1 frame every 20 ms or 3 frames every 20 ms. If the SDK sends data according to the actual sending frequency to the Agora server, the receiver may experience stuttering or frame loss. To optimize the audio transmission performance, the SDK sets an audio sending buffer. Regardless of the actual sending frequency of audio frames by the sender, the SDK stores data in the audio sending buffer. After the data in the audio sending buffer reaches the minimum allowed sending time, the SDK starts sending data to the receiver at a frequency of 10 ms per frame. The default maximum capacity of the audio sending buffer is 6000, which means it can store up to 6000 frames of data, or 60 seconds of data. If you need a larger buffer capacity, you can call this method to adjust it. If you also need to set the minimum allowed sending time for the audio sending buffer, call the [`set_send_delay_ms`](#set_send_delay_ms) method. > **Note** > > This method must be called before the [`publish_audio`](#publish_audio) method. #### Parameters | Parameter | Description | | --------- | --------------------------------------------------------------------------------------------------------------------------------- | | `num` | The maximum number of audio frames that the buffer can store. For example, if you enter 6000, it means 6000 frames of audio data. | ### `clear_sender_buffer` Clears the current audio sending buffer. ```python def clear_sender_buffer(self) ``` If you no longer need the data in the current audio sending buffer, call this method to clear the buffer. For example, a user asks AI a question, and AI is answering the question through the SDK. At this time, the audio sending buffer of the SDK is constantly receiving and sending data. When the user interrupts AI's reply and asks another question, you can call this method to clear the buffered data that has not been sent out, so that the SDK can quickly buffer the AI's response to the new topic. #### Returns - 0: Success. - < 0: Failure. ### `set_send_delay_ms` Sets the minimum data duration that the audio sending buffer allows to send. ```python def set_send_delay_ms(self, delay_ms:int) ``` The default minimum data duration that the SDK allows to send is 160 ms, which means the SDK waits until the audio sending buffer is filled with 160 ms of data before sending the data to the receiver. This will have the following effects: - If the sender sends data that is greater than or equal to 160 ms, for example 800 ms, the SDK sends the data at a frequency of 10 ms per frame. - If the sender sends data that is less than 160 ms, for example 100 ms, the SDK waits until the buffer data reaches 160 ms before sending the data at a frequency of 10 ms per frame. This will cause a delay of up to 60 ms. You can adjust the minimum data duration according to your actual business needs by calling this method. #### Parameters | Parameter | Description | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `delay_ms` | The minimum data duration that the buffer allows to send (ms). This parameter must be set to a multiple of 10, and its value must be greater than 0 and less than 10 times the value of the `number` parameter in the [`set_max_buffer_audio_frame_number`](#set_max_buffer_audio_frame_number) method. | ### `release` Releases the local audio track resources. ```python def release(self) ``` ## LocalVideoTrack The `LocalVideoTrack` class. Provides basic methods for the local video track. You can publish one or multiple video tracks through the [`publish_video`](#publish_video) method. ### `set_enabled` Enables or disables the local video track. After the local video track is enabled, the SDK can capture, process, and encode the local video. ```python def set_enabled(self, enable) ``` #### Parameters | Parameter | Description | | --------- | ------------------------------------------------------------------------------------------------------------------- | | `enable` | Enable or disable the local video track: - `0`: Disable the local video track. - `1`: Enable the local video track. | ### `set_video_encoder_configuration` Sets the video encoding properties. ```python def set_video_encoder_configuration(self, config) ``` #### Parameters | Parameter | Description | | --------- | ----------------------------------------------------------------------------------------------- | | `config` | Video encoding parameter configuration. See [`VideoEncoderConfig`](#videoencoderconfiguration). | #### Returns - 0: Success. - < 0: Failure. ## MediaNodeFactory The `MediaNodeFactory` class. ### `create_audio_pcm_data_sender` Creates a PCM data sender module. This method creates an [`AudioPcmDataSender`](#audiopcmdatasender) object, which can be used in the [`create_custom_audio_track_pcm`](#create_custom_audio_track_pcm) method. ```python def create_audio_pcm_data_sender(self) ``` #### Returns - `AudioPcmDataSender` object, if the method call succeeds. - `None`, if the method call fails. ### `create_audio_encoded_frame_sender` Creates an encoded audio data sender module. This method creates an [`AudioEncodedFrameSender`](#audioencodedframesender) object, which can be used in the [`create_custom_audio_track_encoded`](#create_custom_audio_track_encoded) method. ```python def create_audio_encoded_frame_sender(self) ``` #### Returns - `AudioEncodedFrameSender` object, if the method call succeeds. - `None`, if the method call fails. ### `create_video_frame_sender` Creates a YUV video frame sender module. This method creates a [`VideoFrameSender`](#videoframesender) object, which can be used in the [`create_custom_video_track_frame`](#create_custom_video_track_frame) method. ```python def create_video_frame_sender(self) ``` #### Returns - `VideoFrameSender` object, if the method call succeeds. - `None`, if the method call fails. ### `create_video_encoded_image_sender` Creates an encoded video sender module. This method creates a [`VideoEncodedImageSender`](#videoencodedimagesender) object, which can be used in the [`create_custom_video_track_encoded`](#create_custom_video_track_encoded) method. ```python def create_video_encoded_image_sender(self) ``` #### Returns - `VideoEncodedImageSender` object, if the method call succeeds. - `None`, if the method call fails. ### `release` Releases the `MediaNodeFactory` object. ```python def release(self) ``` ## AudioPcmDataSender The `AudioPcmDataSender` class. ### `send_audio_pcm_data` Sends PCM audio data to the local audio track. ```python def send_audio_pcm_data(self, frame: PcmAudioFrame) ``` #### Parameters | Parameter | Description | | --------- | ------------------------------------------------------------------------------------ | | `frame` | The [`PcmAudioFrame`](#pcmaudioframe) object, containing the PCM audio data to send. | #### Returns - 0: Success. - < 0: Failure. ## AudioEncodedFrameSender The `AudioEncodedFrameSender` Class. ### `send_encoded_audio_frame` Sends an encoded audio frame to the audio track. ```python def send_encoded_audio_frame(self, buffer_ptr: int, buffer_size: int, frame_info: EncodedAudioFrameInfo) ``` #### Parameters | Parameter | Description | | ------------- | ----------------------------------------------------------------------------------- | | `buffer_ptr` | The encoded audio data to send. | | `buffer_size` | The size of the data (bytes). | | `frame_info` | The audio frame information. See [`EncodedAudioFrameInfo`](#encodedaudioframeinfo). | #### Returns - 0: Success. - < 0: Failure. ## IAudioFrameObserver The `IAudioFrameObserver` class. ### `on_playback_audio_frame_before_mixing` Gets the data before the playback audio is mixed. ```python def on_playback_audio_frame_before_mixing(self, agora_local_user, channelId, uid, frame: AudioFrame, vad_result_state: int, vad_result_bytearray: bytearray) ``` #### Parameters **`agora_local_user`** The `LocalUserObserver` class. **`channelId`** The channel name. **`uid`** The remote user ID. **`frame`** The audio raw data. See [`AudioFrame`](#audioframe). **`vad_result_state`** The VAD state. Valid only when you enable VAD in [`register_audio_frame_observer`](#register_audio_frame_observer): - < `0`: VAD is not enabled. - `0`: Not speaking. - `1`: Started speaking. - `2`: Speaking. - `3`: Stopped speaking. **`vad_result_bytearray`** The audio data processed by VAD. When VAD is enabled, use this data instead of `frame` for downstream processing, such as speech recognition. #### Returns - `0`：The audio frame data is invalid, and the data is discarded. - `1`：The audio frame data is valid, and the data is sent. ## VideoFrameSender The `VideoFrameSender` class. ### `send_video_frame` Sends a YUV format video frame to the video track. ```python def send_video_frame(self, frame:ExternalVideoFrame) ``` #### Parameters | Parameter | Description | | --------- | ------------------------------------------------------------------------- | | `frame` | The video frame to send, see [`ExternalVideoFrame`](#externalvideoframe). | #### Returns - 0: Success. - < 0: Failure. ## IVideoFrameObserver The `IVideoFrameObserver` class. ### `on_frame` ```python def on_frame(self, channel_id, remote_uid, frame:VideoFrame) ``` Occurs when a video frame is received. #### Parameters | Parameter | Description | | ------------ | ----------------------------------------------- | | `channel_id` | The channel name. | | `remote_uid` | The remote user ID. | | `frame` | The video data. See [`VideoFrame`](#videoframe) | ## VideoEncodedImageSender The `VideoEncodedImageSender` class. ### `send_encoded_video_image` Sends an encoded video frame to the video track. ```python def send_encoded_video_image(self, buffer_ptr:int, buffer_size:int, frame_info:EncodedVideoFrameInfo) ``` #### Parameters | Parameter | Description | | ------------- | ------------------------------------------------------------------------------------------- | | `buffer_ptr` | The video buffer. | | `buffer_size` | The size of the video data (bytes). | | `frame_info` | The encoded video frame information: see [`EncodedVideoFrameInfo`](#encodedvideoframeinfo). | #### Returns - 0: Success. - < 0: Failure. ## IVideoEncodedFrameObserver The `IVideoEncodedFrameObserver` class. ### `on_encoded_video_frame` Occurs when an encoded video frame is received. ```python def on_encoded_video_frame(self, uid, image_buffer, length, video_encoded_frame_info:EncodedVideoFrameInfo) ``` #### Parameters | Parameter | Description | | -------------------------- | ------------------------------------------------------------------------------------------ | | `uid` | The remote user ID. | | `image_buffer` | The video frame buffer. | | `length` | The length of the video frame data. | | `video_encoded_frame_info` | The encoded video frame information. See [`EncodedVideoFrameInfo`](#encodedvideoframeinfo) | #### Returns Whether to accept the encoded video frame: - `0`: Do not accept. - `1`: Accept. ## Data type ### Classes #### `AgoraServiceConfig` Global configuration of the `AgoraServiceConfig` class. Set when calling the [`initialize`](#initialize) method. ```python @dataclass(kw_only=True) class AgoraServiceConfig: log_path: str = "" log_size: int = 0 enable_audio_processor: int = 1 enable_audio_device: int = 0 enable_video: int = 0 context: object = None appid: str = "" area_code: int = AreaCode.AREA_CODE_GLOB.value channel_profile: ChannelProfileType = ChannelProfileType.CHANNEL_PROFILE_LIVE_BROADCASTING audio_scenario: AudioScenarioType = AudioScenarioType.AUDIO_SCENARIO_CHORUS use_string_uid: int = 0 domain_limit: int = 0 should_callbck_when_muted: int = 0 ``` **`log_path`** The full path to the log file. If this parameter is set to `None`, the log is located in the current application's pwd directory **`log_size`** The size of a single log file in KB. The default value is 1024 KB. If you set this parameter to 1024 KB, the SDK outputs a maximum of 5 MB of log files. If you set this parameter to less than 1024 KB, the setting does not take effect, and the maximum size of a single log file is still 1024 KB. **`enable_audio_processor`** Whether to enable the audio processing module. - `0`: Disable the audio processing module. You will not be able to create an audio track. - `1`: (Default) Enable the audio processing module. **`enable_audio_device`** Whether to enable the Audio Device Module (ADM). ADM is used to manage audio devices such as recording and playing audio. - `0`: Disable the audio device module. You cannot record or play audio. - `1`: (Default) Enable the audio device module. You can record and play audio. Note: If you set `enable_audio_device` to `0` and `enable_audio_processor` to `1`, you will not be able to use audio devices, but you can push PCM audio data. **`enable_video`** Whether to enable video. - `0`: (Default) Disable video. - `1`: Enable video. **`context`** The context. **`appid`** Your Agora project's App ID. If your Agora project has enabled Token authentication, you can also assign this parameter to an RTC Token. See [Agora account management](/en/introduction/account) for details. **`area_code`** The access region of the server. This feature is an advanced setting suitable for scenarios with access security restrictions. The area code supports bitwise operations. After specifying the access region, the SDK connects to Agora servers within the specified region. See [`AreaCode`](#areacode) for details. **`channel_profile`** Channel profile. See [`ChannelProfileType`](#channelprofiletype) for details. **`audio_scenario`** Audio encoding properties. See [`AudioScenarioType`](#audioscenariotype) for details. **`use_string_uid`** Whether to use String user IDs. - `0`: Do not use. - `1`: Use. **`domain_limit`** Whether to enable domain limit: - `1`: Yes. Only connects to domains that are parsed by DNS. - `0`: (Default) No. There is no limit when connecting to domains. **`should_callbck_when_muted`** Whether the SDK still triggers the [`on_playback_audio_frame_before_mixing`](#on_playback_audio_frame_before_mixing) callback after a remote user mutes their audio: - `0`: (Default) No. The SDK stops triggering the callback when the remote user mutes their audio. - Greater than `0`: Yes. The SDK keeps triggering the callback when the remote user mutes their audio. #### `AudioEncoderConfiguration` Audio encoder configuration. ```python @dataclass(kw_only=True) class AudioEncoderConfiguration: audioProfile: AudioProfileType = AudioProfileType.AUDIO_PROFILE_DEFAULT ``` | Parameter | Description | | -------------- | ------------------------------------------------------------------------ | | `audioProfile` | Audio profile. For details, see [`AudioProfileType`](#audioprofiletype). | #### `AudioFrame` The `AudioFrame` class. ```python @dataclass(frozen=True, kw_only=True) class AudioFrame: type: int samples_per_channel: int bytes_per_sample: int channels: int samples_per_sec: int buffer: bytearray render_time_ms: int avsync_type: int far_field_flag: int rms: int voice_prob: int music_prob: int pitch: int ``` | Parameter | Description | | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | Type of audio frame. Only `0` is supported, indicating 16-bit PCM data. | | `samples_per_channel` | Number of samples per channel. | | `bytes_per_sample` | Number of bytes per sample. For PCM, typically 16 bits, which is two bytes. | | `channels` | Number of channels (if stereo, data is interleaved). | | `samples_per_sec` | Number of samples per second per channel. | | `buffer` | Audio data buffer (if stereo, data is interleaved). The buffer size is calculated as `buffer = samples × channels × bytes_per_sample`. | | `render_time_ms` | Render timestamp of the external audio frame. You can use this timestamp to restore the order of audio frames; in scenarios with video (including those using external video sources), this parameter can be used to achieve audio-video synchronization. | | `avsync_type` | Reserved parameter. | | `far_field_flag` | The far-field flag of the audio frame. Only used in audio frame observer callbacks. | | `rms` | The root mean square (RMS) of the audio frame. Only used in audio frame observer callbacks. | | `voice_prob` | The probability that the audio frame contains voice. Only used in audio frame observer callbacks. | | `music_prob` | The probability that the audio frame contains music. Only used in audio frame observer callbacks. | | `pitch` | The pitch of the audio frame. Only used in audio frame observer callbacks. | #### `AudioSubscriptionOptions` The audio subscription options. ```python @dataclass(kw_only=True) class AudioSubscriptionOptions: packet_only: int = 0 pcm_data_only: int = 0 bytes_per_sample: int = 0 number_of_channels: int = 0 sample_rate_hz: int = 0 ``` | Parameter | Description | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `packet_only` | Whether to subscribe to audio packets only: - `0`: (Default) The SDK automatically decodes the audio. - `1`: Subscribe to audio packets only. The SDK will not decode the audio. **Note**: If this parameter is set to `1`, other parameters in the structure are invalid. | | `pcm_data_only` | Whether to subscribe to PCM data only: - `0`: (Default) The SDK automatically encodes the remote audio stream. - `1`: Subscribe to PCM data only. | | `bytes_per_sample` | Number of bytes per audio sample. | | `number_of_channels` | Number of audio channels. | | `sample_rate_hz` | Audio sampling rate (Hz). | #### `EncodedAudioFrameInfo` The encoded audio data. ```python @dataclass(kw_only=True) class EncodedAudioFrameInfo: capture_timems: int = 0 codec: AudioCodecType = AudioCodecType.AUDIO_CODEC_AACLC number_of_channels: int = 1 sample_rate: int = 16000 samples_per_channel: int = 1024 send_even_if_empty: int = 1 speech: int = 1 ``` | Parameter | Description | | --------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `capture_timems` | Unix timestamp of the audio frame (milliseconds). | | `codec` | Audio codec type. See [`AudioCodecType`](#audiocodectype) for details. | | `number_of_channels` | Number of channels in the audio frame. | | `sample_rate` | Sampling rate of the audio frame (Hz). | | `samples_per_channel` | Number of samples per channel. Defaults to 1024 for AAC codec format and 960 for Opus codec format. | | `send_even_if_empty` | Whether to send empty audio frames: - `0`: Do not send empty audio frames. - `1`: (Default) Send empty audio frames. | | `speech` | Whether the audio source is speech: - `0`: The audio source is not speech. - `1`: (Default) The audio source is speech. | #### `EncodedVideoFrameInfo` The encoded video data. ```python @dataclass(kw_only=True) class EncodedVideoFrameInfo: codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE width: int = 0 height: int = 0 frames_per_second: int = 0 frame_type: int = 0 rotation: int = 0 track_id: int = 0 capture_time_ms: int = 0 decode_time_ms: int = 0 uid: int = 0 stream_type: int = 0 ``` | Parameter | Description | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `codec_type` | Video codec type. See [`VideoCodecType`](#videocodectype) for details. | | `width` | Width of the video frame (pixels). | | `height` | Height of the video frame (pixels). | | `frames_per_second` | Number of video frames per second. Used to calculate the timestamp of the encoded video frame. If set to 0, the SDK uses the real timestamp. | | `frame_type` | Type of the encoded video frame: - `0`: Blank frame - `3`: Keyframe. For H.264 codec format, the SDK classifies I-frames with SPS and PPS information as keyframes. - `4`: Delta frame. For H.264 codec format, the SDK classifies P-frames and I-frames without SPS and PPS information as delta frames. - `5`: B-frame - `6`: Dropped frame - `7`: Unknown | | `rotation` | Rotation information of the encoded video frame: - `0`: 0 degrees clockwise. - `90`: 90 degrees clockwise. - `180`: 180 degrees clockwise. - `270`: 270 degrees clockwise. | | `track_id` | Track ID of the video frame. | | `capture_time_ms` | Unix timestamp for capturing the external encoded video frame (milliseconds). | | `decode_time_ms` | Timestamp for decoding the external encoded video frame (milliseconds). | | `uid` | User ID. | | `stream_type` | Stream type of the video frame: - `0`: High stream. Higher resolution and bitrate. - `1`: Low stream. Lower resolution and bitrate. | #### `EncryptionConfig` ```python @dataclass(kw_only=True) class EncryptionConfig: encryption_mode: int encryption_key: str encryption_kdf_salt: bytearray = None ``` Encryption configuration. **`encryption_mode`** Encryption mode: - 1: 128-bit AES encryption, XTS mode. - 2: 128-bit AES encryption, ECB mode. - 3: 256-bit AES encryption, XTS mode. - 4: 128-bit SM4 encryption, ECB mode. - 5: 128-bit AES encryption, GCM mode. - 6: 256-bit AES encryption, GCM mode. - 7: (Default) 128-bit AES encryption, GCM mode. This encryption mode requires setting a salt (`encryption_kdf_salt`). - 8: 256-bit AES encryption, GCM mode. This encryption mode requires setting a salt (`encryption_kdf_salt`). **`encryption_key`** Encryption key, string type, with no length limit. It is recommended to use a 32-byte key. **`encryption_kdf_salt`** Encryption salt, 32 bytes in length. It is recommended to generate the salt using OpenSSL on the server side. #### `ExternalVideoFrame` The YUV format video data. ```python @dataclass(kw_only=True) class ExternalVideoFrame: type: int = 1 format: int = 0 buffer: bytearray = None stride: int = 0 height: int = 0 crop_left: int = 0 crop_top: int = 0 crop_right: int = 0 crop_bottom: int = 0 rotation: int = 0 timestamp: int = 0 egl_context: bytearray = None egl_type: int = 0 texture_id: int = 0 matrix: list = field(default_factory=list) metadata: bytearray = None alpha_buffer: bytearray = None fill_alpha_buffer: int = 0 alpha_mode: int = 0 color_space: ColorSpaceType = field(default_factory=ColorSpaceType) ``` **`type`** Video data type: - `1`: Type is raw data. - `2`: Type is raw data. - `3`: Type is Texture. **`format`** Pixel format: - `0`: Raw video pixel format. - `1`: I420 format. - `4`: RGBA format. - `16`: I422 format. - `17`: `ID3D11TEXTURE2D` format. Currently supported types are `DXGI_FORMAT_B8G8R8A8_UNORM`, `DXGI_FORMAT_B8G8R8A8_TYPELESS`, and `DXGI_FORMAT_NV12`. **Note**: The SDK currently does not support the Alpha channel. The incoming Alpha value will be discarded. **`buffer`** Video buffer. **`stride`** The stride of the incoming video frame, in pixels rather than bytes. For Texture, this value refers to the width of the Texture. **`height`** Height of the incoming video frame. **`crop_left`** Field related to raw data. Specifies the number of pixels cropped from the left. Default is 0. This parameter is only applicable to raw video data. **`crop_top`** Field related to raw data. Specifies the number of pixels cropped from the top. Default is 0. This parameter is only applicable to raw video data. **`crop_right`** Field related to raw data. Specifies the number of pixels cropped from the right. Default is 0. This parameter is only applicable to raw video data. **`crop_bottom`** Field related to raw data. Specifies the number of pixels cropped from the bottom. Default is 0. This parameter is only applicable to raw video data. **`rotation`** Field related to raw data. Specifies whether the incoming video frame should be rotated clockwise. Possible values are 0, 90, 180, 270. Default is 0. **`timestamp`** Timestamp of the incoming video frame, in milliseconds. Incorrect timestamps can cause frame drops or audio-video desynchronization. **`egl_context`** This parameter is only applicable to Texture format video data. - When using the OpenGL interface defined by Khronos (javax.microedition.khronos.egl.*), set the eglContext to this field. - When using the OpenGL interface defined by Android (android.opengl.*), set the eglContext to this field. **`egl_type`** This parameter is only applicable to Texture format video data. Refers to the EGL context type. **`texture_id`** This parameter is only applicable to Texture format video data. Refers to the Texture ID of the video frame. **`matrix`** This parameter is only applicable to Texture format video data. Refers to an input 4x4 transformation matrix, typically an identity matrix. **`metadata`** The metadata buffer of the video frame. The default value is `None`. **`alpha_buffer`** Alpha channel data output by the portrait segmentation algorithm. This data matches the size of the video frame, with each pixel value ranging from \[0,255], where 0 represents the background and 255 represents the foreground (portrait). You can set this parameter to render the video background with various effects, such as transparency, solid color, images, or videos. **`fill_alpha_buffer`** \[For BGRA or RGBA only] Whether to extract `alphaBuffer` from BGRA or RGBA data: - `1`: Yes. Set it to true if you do not explicitly specify the `alphaBuffer`. - `0`: (Default) No. **`alpha_mode`** The relative position between `alphaBuffer` and the frame: - `0`: (Default) Normal frame. - `1`: `alphaBuffer` is above the frame. - `2`: `alphaBuffer` is below the frame. - `3`: `alphaBuffer` is on the left of the frame. - `4`: `alphaBuffer` is on the right of the frame. **`color_space`** The color space of the video frame. See [`ColorSpaceType`](https://developer.mozilla.org/en-US/docs/Web/API/VideoColorSpace). #### `PcmAudioFrame` PCM audio data. ```python @dataclass(kw_only=True) class PcmAudioFrame: data: bytearray = None samples_per_channel: int = 0 bytes_per_sample: int = 0 number_of_channels: int = 0 sample_rate: int = 0 timestamp: int = 0 ``` | Parameter | Description | | --------------------- | ----------------------------------------------------------------- | | `data` | PCM audio data to be sent. | | `timestamp` | Timestamp of the audio data collection (milliseconds). | | `samples_per_channel` | Number of audio samples collected per channel in 10 milliseconds. | | `bytes_per_sample` | Number of bytes per sample. | | `number_of_channels` | Number of channels per sample. | | `sample_rate` | Sampling rate (Hz). The minimum value is 8000. | #### `RTCConnConfig` RTC connection configuration. ```python @dataclass(kw_only=True) class RTCConnConfig: auto_subscribe_audio: int = 0 auto_subscribe_video: int = 0 enable_audio_recording_or_playout: int = 0 max_send_bitrate: int = 0 min_port: int = 0 max_port: int = 0 audio_subs_options: 'AudioSubscriptionOptions' = field(default_factory=AudioSubscriptionOptions) client_role_type: ClientRoleType = ClientRoleType.CLIENT_ROLE_BROADCASTER channel_profile: ChannelProfileType = ChannelProfileType.CHANNEL_PROFILE_LIVE_BROADCASTING audio_recv_media_packet: int = 0 audio_recv_encoded_frame: int = 0 video_recv_media_packet: int = 0 ``` | Parameter | Description | | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `auto_subscribe_audio` | Whether to automatically subscribe to all audio tracks. - `0`: Do not automatically subscribe to all audio tracks. - `1`: (Default) Automatically subscribe to all audio tracks. | | `auto_subscribe_video` | Whether to automatically subscribe to all video tracks. - `0`: Do not automatically subscribe to all video tracks. - `1`: (Default) Automatically subscribe to all video tracks. | | `enable_audio_recording_or_playout` | Whether to enable audio recording or playout. - `0`: Disable audio recording or playout. - `1`: Enable audio recording or playout. | | `max_send_bitrate` | The maximum send bitrate. | | `min_port` | The minimum port. | | `max_port` | The maximum port. | | `audio_subs_options` | Audio subscription options. See [`AudioSubscriptionOptions`](#audiosubscriptionoptions) for details. | | `client_role_type` | User role. See [`ClientRoleType`](#clientroletype) for details. | | `channel_profile` | Channel profile. See [`ChannelProfileType`](#channelprofiletype) for details. | | `audio_recv_media_packet` | Whether to receive audio media packets. - `0`: Do not receive. - `1`: Receive. | | `video_recv_media_packet` | Whether to receive video media packets. - `0`: Do not receive. - `1`: Receive. | #### `RTCConnInfo` The RTC connection information. ```python @dataclass(frozen=True, kw_only=True) class RTCConnInfo(): id: int channel_id: str state: int local_user_id: str internal_uid: int ``` | Parameter | Description | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | The ID of the RTC connection. | | `channel_id` | RTC channel ID. If you have not called the [`connect`](#connect) method, this parameter returns `None`. | | `state` | Connection state: - `1`: Network connection disconnected. - `2`: Establishing network connection. - `3`: Network connected. - `4`: Re-establishing network connection. - `5`: Network connection failed. | | `local_user_id` | Local user ID. | | `internal_uid` | For internal use only. | #### `SenderOptions` Configuration of the local video track object that includes the video source for sending encoded video data. ```python @dataclass(kw_only=True) class SenderOptions: target_bitrate: int cc_mode: TCcMode = TCcMode.CC_ENABLED codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE ``` | Parameter | Description | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `target_bitrate` | The target encoding bitrate (Kbps) of the current encoder, which is an estimated value by the SDK based on the current network conditions. | | `cc_mode` | Congestion control mode. See [`TCcMode`](#tccmode) for details. | | `codec_type` | Video codec type. See [`VideoCodecType`](#videocodectype) for details. | #### `VideoDimensions` Resolution of the video encoding. ```python @dataclass(kw_only=True) class VideoDimensions: width: int height: int ``` | Parameter | Description | | --------- | ----------------------------------- | | `width` | Width of the video frame (pixels). | | `height` | Height of the video frame (pixels). | #### `VideoEncoderConfiguration` Video encoding properties. ```python @dataclass(kw_only=True) class VideoEncoderConfiguration: dimensions: VideoDimensions codec_type: VideoCodecType = VideoCodecType.VIDEO_CODEC_NONE frame_rate: int = 15 bitrate: int = 0 min_bitrate: int = 0 orientation_mode: int = 0 degradation_preference: int = 0 mirror_mode: int = 0 encode_alpha: int = 0 ``` | Parameter | Description | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `codec_type` | Video codec type. See [`VideoCodecType`](#videocodectype) for details. | | `dimensions` | Resolution of the video encoding (px). This parameter measures the encoding quality, represented as width × height. The default value is 640 × 360. Users can set the resolution as needed. See [`VideoDimensions`](#videodimensions) for details. | | `frame_rate` | Frame rate of the video encoding (fps), default is 15. | | `bitrate` | Video encoding bitrate, in Kbps. - `0`: (Recommended) Standard bitrate mode. In this mode, the video bitrate is twice the base bitrate. - `-1`: Adapted bitrate mode. In this mode, the video bitrate is the same as the base bitrate. When this mode is selected for live streaming, the video frame rate may be lower than the set value. | | `min_bitrate` | Minimum encoding bitrate, in Kbps. The SDK will automatically adjust the video encoding bitrate based on network conditions. Setting this parameter higher than the default value can force the video encoder to output higher quality images, but may cause packet loss and affect video playback fluency under poor network conditions, leading to stuttering. Therefore, unless there are special requirements for image quality, it is recommended not to modify this parameter. **Note**: This parameter is only applicable in live streaming scenarios. | | `orientation_mode` | Orientation mode of the video encoding: - `0`: (Default) In this mode, the SDK outputs the video orientation consistent with the captured video. The receiver will rotate the video based on the received video rotation information. This mode is suitable for scenarios where the receiver can adjust the video orientation. If the captured video is in landscape mode, the output video will also be in landscape mode. If the captured video is in portrait mode, the output video will also be in portrait mode. - `1`: In this mode, the SDK outputs the video in landscape mode. If the captured video is in portrait mode, the video encoder will crop it. This mode is suitable for scenarios where the receiver cannot adjust the video orientation, such as direct push to CDN. - `2`: In this mode, the SDK outputs the video in portrait mode. If the captured video is in landscape mode, the video encoder will crop it. This mode is suitable for scenarios where the receiver cannot adjust the video orientation, such as direct push to CDN. | | `degradation_preference` | Video encoding degradation preference when bandwidth is limited: - `0`: (Default) When bandwidth is limited, the video encoding prioritizes reducing the frame rate while maintaining video quality. This degradation preference is suitable for scenarios where image quality is prioritized. - `1`: When bandwidth is limited, the video encoding prioritizes reducing video quality while maintaining the frame rate. This degradation preference is suitable for scenarios where smoothness is prioritized and quality reduction is acceptable. - `2`: When bandwidth is limited, the video encoding reduces both frame rate and video quality. The reduction in `2` is less than in `0` and `1`, suitable for scenarios with limited smoothness and quality. - `3`: When bandwidth is limited, the video encoding prioritizes reducing the frame rate. | | `mirror_mode` | Whether to enable mirror mode when sending encoded video, only affecting the video seen by the remote user. - `0`: (Default) The mirror mode is determined by the SDK. - `1`: Enable mirror mode. - `2`: Disable mirror mode. | | `encode_alpha` | Whether to encode and send the alpha data to the remote user when the video frame contains alpha data. - `0`: (Default) Do not encode or send the alpha data. - `1`: Encode and send the alpha data. | #### `VideoFrame` Properties of the video frame. The video data format is YUV420. The buffer is given as a pointer to a pointer. This interface cannot modify the pointer of the buffer, only the content of the buffer. ```python @dataclass(kw_only=True) class VideoFrame(): type: int = 0 width: int = 0 height: int = 0 y_stride: int = 0 u_stride: int = 0 v_stride: int = 0 y_buffer: bytearray = None u_buffer: bytearray = None v_buffer: bytearray = None rotation: int = 0 render_time_ms: int = 0 avsync_type: int = 0 metadata: bytearray = None shared_context: str = None texture_id: int = 0 matrix: list = None alpha_buffer: bytearray = None alpha_mode: int = 0 ``` | Parameter | Description | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | Type of video data: - `1`: Raw data. - `2`: Raw data. - `3`: Texture. | | `width` | Width of the video frame (pixels). | | `height` | Height of the video frame (pixels). | | `y_stride` | Stride of the Y buffer in the YUV data. | | `u_stride` | Stride of the U buffer in the YUV data. | | `v_stride` | Stride of the V buffer in the YUV data. | | `y_buffer` | Pointer to the pointer of the Y buffer in the YUV data. | | `u_buffer` | Pointer to the pointer of the U buffer in the YUV data. | | `v_buffer` | Pointer to the pointer of the V buffer in the YUV data. | | `rotation` | Clockwise rotation angle of the frame before rendering the video. Currently supports 0 degrees, 90 degrees, 180 degrees, and 270 degrees. | | `render_time_ms` | Timestamp of the external audio frame. This parameter is mandatory. You can use this timestamp to restore the order of audio frames; in scenarios with video (including those using external video sources), this parameter can help achieve audio-video synchronization. | | `avsync_type` | Reserved parameter. | | `metadata` | The metadata buffer of the video frame. | | `shared_context` | The EGL context. Only applicable to Texture format video data. | | `texture_id` | The Texture ID of the video frame. Only applicable to Texture format video data. | | `matrix` | The input 4x4 transformation matrix. Only applicable to Texture format video data. | | `alpha_buffer` | The alpha channel data of the video frame. The data matches the size of the video frame. | | `alpha_mode` | The relative position between `alpha_buffer` and the frame. - `0`: (Default) Normal frame. - `1`: `alpha_buffer` is above the frame. - `2`: `alpha_buffer` is below the frame. - `3`: `alpha_buffer` is on the left of the frame. - `4`: `alpha_buffer` is on the right of the frame. | #### `VideoSubscriptionOptions` Video subscription options. ```python @dataclass(kw_only=True) class VideoSubscriptionOptions: type: VideoStreamType = VideoStreamType.VIDEO_STREAM_HIGH encodedFrameOnly: bool = False ``` | Parameter | Description | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | Type of video to subscribe to. See [`VideoStreamType`](#videostreamtype) for details. | | `encodedFrameOnly` | Whether to subscribe only to encoded video data: - `0`: (Default) Subscribe only to decoded video data. - `1`: Subscribe only to encoded video data. | ### Enums #### `AreaCode` Server access regions. This feature is an advanced setting and is used in scenarios with access security restrictions. The region code supports bitwise operations. After specifying the access region, the SDK will connect to the Agora servers within the specified region. | Enumeration | Description | | ---------------- | ---------------------------------------------- | | `AREA_CODE_CN` | `0x00000001`: Mainland China. | | `AREA_CODE_NA` | `0x00000002`: North America. | | `AREA_CODE_EU` | `0x00000004`: Europe. | | `AREA_CODE_AS` | `0x00000008`: Asia (excluding Mainland China). | | `AREA_CODE_JP` | `0x00000010`: Japan. | | `AREA_CODE_IN` | `0x00000020`: India. | | `AREA_CODE_GLOB` | `0xFFFFFFFF`: (Default) Global. | #### `AudioCodecType` Audio codec types. | Enumeration | Description | | -------------------- | ------------ | | `AUDIO_CODEC_OPUS` | `1`: OPUS | | `AUDIO_CODEC_PCMA` | `3`: PCMA | | `AUDIO_CODEC_PCMU` | `4`: PCMU | | `AUDIO_CODEC_G722` | `5`: G722 | | `AUDIO_CODEC_AACLC` | `8`: AAC LC | | `AUDIO_CODEC_HEAAC` | `9`: HE AAC | | `AUDIO_CODEC_HEAAC2` | `11`: HEAAC2 | #### `AudioProfileType` Audio profiles. **`AUDIO_PROFILE_DEFAULT`** `0`: Default audio profile. - In communication scenarios: 16 kHz sampling rate, voice codec, mono, maximum encoding bitrate of 16 Kbps. - In live broadcast scenarios: 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 64 Kbps. **`AUDIO_PROFILE_SPEECH_STANDARD`** `1`: Specifies a 32 kHz sampling rate, voice codec, mono, maximum encoding bitrate of 18 Kbps. **`AUDIO_PROFILE_MUSIC_STANDARD`** `2`: Specifies a 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 64 Kbps. **`AUDIO_PROFILE_MUSIC_STANDARD_STEREO`** `3`: Specifies a 48 kHz sampling rate, music codec, stereo, maximum encoding bitrate of 80 Kbps. **`AUDIO_PROFILE_MUSIC_HIGH_QUALITY`** `4`: Specifies a 48 kHz sampling rate, music codec, mono, maximum encoding bitrate of 96 Kbps. **`AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO`** `5`: Specifies a 48 kHz sampling rate, music codec, stereo, maximum encoding bitrate of 128 Kbps. **`AUDIO_PROFILE_IOT`** `6`: Specifies a 16 kHz sampling rate, voice codec, mono, with echo cancellation algorithm AES. **`AUDIO_PROFILE_NUM`** `7`: Enumeration boundary. #### `AudioScenarioType` Audio scenario types. | Enumeration | Description | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AUDIO_SCENARIO_DEFAULT` | `0`: (Default) Automatic scenario, automatically matches the appropriate audio quality based on user role and audio routing. | | `AUDIO_SCENARIO_GAME_STREAMING` | `3`: High-quality scenario, suitable for music-centric scenarios. | | `AUDIO_SCENARIO_CHATROOM` | `5`: Chatroom scenario, suitable for scenarios where users frequently switch between speaking and listening. In this scenario, audience members will receive a prompt to request microphone permission. | | `AUDIO_SCENARIO_CHORUS` | `7`: Chorus scenario, suitable for real-time chorus with extremely low latency under good network conditions. | | `AUDIO_SCENARIO_MEETING` | `8`: Meeting scenario, suitable for multi-person meetings focused on voice. | | `AUDIO_SCENARIO_NUM` | `9`: Enumeration boundary. | #### `ChannelProfileType` Channel usage scenarios. | Enumeration | Description | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CHANNEL_PROFILE_COMMUNICATION` | `0`: (Default) Communication scenario. Recommended for channels with only two users. | | `CHANNEL_PROFILE_LIVE_BROADCASTING` | `1`: Live broadcasting scenario. This scenario has two user roles: host and audience. Hosts can publish and receive audio and video streams, while the audience can only receive streams. Suitable for voice chat rooms, video live broadcasts, interactive large classes, etc. | #### `ClientRoleType` User roles. | Enumeration | Description | | ------------------------- | ----------------------------------------------------------------------- | | `CLIENT_ROLE_BROADCASTER` | `1`: Broadcaster. Broadcasters can both send and receive media streams. | | `CLIENT_ROLE_AUDIENCE` | `2`: Audience. Audience can only receive media streams. | #### `TCcMode` Congestion control modes. | Enumeration | Description | | ------------- | ------------------------------------- | | `CC_ENABLED` | `0`: Enable congestion control mode. | | `CC_DISABLED` | `1`: Disable congestion control mode. | #### VideoCodecType Video codec types. | Enumeration | Description | | -------------------------- | ----------------- | | `VIDEO_CODEC_NONE` | `0`: None | | `VIDEO_CODEC_VP8` | `1`: VP8 | | `VIDEO_CODEC_H264` | `2`: H.264 | | `VIDEO_CODEC_H265` | `3`: H.265 | | `VIDEO_CODEC_GENERIC` | `6`: Generic | | `VIDEO_CODEC_GENERIC_H264` | `7`: Generic H264 | | `VIDEO_CODEC_AV1` | `12`: AV1 | | `VIDEO_CODEC_VP9` | `13`: VP9 | | `VIDEO_CODEC_GENERIC_JPEG` | `20`: JPEG | #### `VideoStreamType` Video stream types. | Enumeration | Description | | ------------------- | ---------------------------------------------------------- | | `VIDEO_STREAM_HIGH` | `0`: (Default) High stream. Higher resolution and bitrate. | | `VIDEO_STREAM_LOW` | `1`: Low stream. Lower resolution and bitrate. |