Interactive Broadcast API

There is a new release of the Developer Center! If you'd like to check it out, please click  here

Interactive Broadcast API

Java Interface Class Description
Basic Methods (RtcEngine) The RtcEngine class provides the main methods that can be invoked by your application.
Audio Effect Methods (IAudioEffectManager) The IAudioEffectManager class provides the methods to manage the audio effects.
Push-Stream Methods (PublisherConfiguration) The PublisherConfiguration class provides the method to configure pushing streams for CDN (content delivery network) Live scenarios.
Callback Methods (IRtcEngineEventHandler) The IRtcEngineEventHandler class enables callback event notifications to your application.

Basic Methods (RtcEngine)

Developer suite: io.agora.rtc

The RtcEngine interface class is the main interface class of the Agora Native SDK. Call the methods of this class to use all the functionalities of the SDK. Agora recommends calling the RtcEngine API methods in the same thread instead of in multiple threads. In previous versions, this class was named AgoraAudio, and is renamed to RtcEngine from version 1.0.

Create an RtcEngine Object (create)

public static synchronized RtcEngine create(Context context,
                                             String appId,
                                             IRtcEngineEventHandler handler);

This method creates an RtcEngine object.

The Agora Native SDK only supports one RtcEngine instance at a time, therefore the application should create one RtcEngine object only. Unless otherwise specified, all called methods provided by the RtcEngine class are executed asynchronously. Agora recommends calling the interface methods in the same thread. Unless otherwise specified, the following rule applies to all APIs whose return values are integer types: A return value of 0 indicates that the call was successful, and a return value less than 0 indicates that the call failed.

Name Description
context The context of Android Activity.
appId The App ID issued to the application developers by Agora. Apply for a new one from Agora if the key is missing in your kit.
handler IRtcEngineEventHandler is an abstract class that provides default implementations. The SDK uses this class to report to the application on SDK runtime events.
Return Value An RtcEngine object.

Implementing a Live Voice Broadcast

The following figure shows how to implement the basic live voice broadcast function. Developers can add more functions according to the modules described in this document. Ensure that an RtcEngine object has been created for the application according to Create an RtcEngine Object (create).

../_images/android_audio_api_live.png

Set the Channel Profile (setChannelProfile)

public abstract int setChannelProfile(int profile);

This method configures the channel profile. The Agora RtcEngine needs to know what scenario the application is in to apply different methods for optimization.

The Agora Native SDK supports the following profiles:

Profile Description
Communication Default setting. This is used in one-on-one calls, where all users in the channel can talk freely.
Live Broadcast Live Broadcast. Host and audience roles that can be set by calling setClientRole. The host sends and receives voice, while the audience receives voice only with the sending function disabled.
Gaming Gaming Mode. Any user in the channel can talk freely. This mode uses the codec with low-power consumption and low bitrate by default.

Note

  • Only one profile can be used at the same time in the same channel. If you want to switch to another profile, use destroy to destroy the current Engine and create a new one using Create an RtcEngine Object (create) before calling this method to set the new channel profile.
  • Make sure that different App IDs are used for different channel profiles.
  • This method must be called and configured before a user joining a channel, because the channel profile cannot be configured when the channel is in use.
Name Description
profile

The channel profile. Choose one of the following:

  • CHANNEL_PROFILE_COMMUNICATION = 0: Communication (default)
  • CHANNEL_PROFILE_LIVE_BROADCASTING = 1: Live Broadcast
  • CHANNEL_PROFILE_GAME = 2: Gaming
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the User Role (setClientRole)

public abstract int setClientRole(int role);

This method allows you to set the user role as a host or an audience (default) before joining a channel. This method also allows you to switch the user role after joining a channel.

Name Description
role

The user role in a live broadcast:

  • CLIENT_ROLE_BROADCASTER = 1; Host
  • CLIENT_ROLE_AUDIENCE = 2; Audience(default)
Return Value
  • 0: The method is called successfully
  • <0: Failed to call the method

Enable the Audio Mode (enableAudio)

public abstract int enableAudio();

This method enables the audio mode. This function is enabled by default.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Note

This method sets to enable the internal engine, and still works after leaveChannel is called.

Disable the Audio Mode (disableAudio)

public abstract int disableAudio();

This method disables the audio mode.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Note

This method sets to disable the internal engine, and still works after leaveChannel is called.

Set the Audio Profile (setAudioProfile)

public abstract int setAudioProfile(int profile, int scenario);

This method sets the audio parameters and application scenarios.

Note

This API is only valid when you call it before joining a channel.

Name Description
profile

Sets the sampling rate, bitrate, encode mode, and the number of channels:

  • AUDIO_PROFILE_DEFAULT = 0: default. In the communication mode, the default value is 1; in the live-broadcast mode, the default value is 2
  • AUDIO_PROFILE_SPEECH_STANDARD = 1: Specifies the sampling rate as 32 kHz, audio encoding, single channel, and bitrate up to 18 kbit/s
  • AUDIO_PROFILE_MUSIC_STANDARD = 2: Specifies the sampling rate as 48 kHz, music encoding, single channel, and bitrate up to 48 kbit/s
  • AUDIO_PROFILE_MUSIC_STANDARD_STEREO = 3: Specifies the sampling rate as 48 kHz, music encoding, dual-channel, and bitrate up to 56 kbit/s
  • AUDIO_PROFILE_MUSIC_HIGH_QUALITY = 4: Specifies the sampling rate as 48 kHz, music encoding, single channel, and bitrate up to 128 kbit/s
  • AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO = 5: Specifies the sampling rate as 48 kHz, music encoding, dual-channel, and bitrate up to 192 kbit/s
scenario

Sets the audio application scenarios:

  • AUDIO_SCENARIO_DEFAULT = 0: default
  • AUDIO_SCENARIO_CHARROOM_ENTERTAINMENT = 1: Applicable to the entertainment scenario that supports voice during gameplay
  • AUDIO_SCENARIO_EDUCATION = 2: Applicable to the education scenario that prioritizes fluency and stability
  • AUDIO_SCENARIO_GAME_STREAMING = 3: Applicable to the live gaming scenario that needs to enable the gaming audio effects in the speaker mode in a live broadcast scenario
  • AUDIO_SCENARIO_SHOWROOM = 4: Applicable to the showroom scenario that optimizes the audio quality with professional external equipment
  • AUDIO_SCENARIO_CHATROOM_GAMING = 5: Applicable to the gaming scenario
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Note

  • Use this method before joinChannel, else it does not work.
  • In the communication mode, setting profile works, but not scenario.
  • In the communication and live-broadcast mode, the bitrate may be different from your settings due to network self-adaptation.
  • In scenarios with music teaching, Agora recommends setting profile as MUSIC_HIGH_QUALITY(4) and scenario as GAME_STREAMING(3).
  • CHATROOM_ENTERTAINMENT(1) and CHATROOM_GAMING(5) are recommended for audio-only scenarios.

Set High-quality Audio Preferences (setHighQualityAudioParameters)

public abstract int setHighQualityAudioParameters(boolean fullband, boolean stereo, boolean fullBitrate);

This method sets high-quality audio preferences. Call this method and set all the three modes before joining a channel. Do NOT call this method again after joining a channel.

Name Description
fullband

Full-band codec (48 kHz sampling rate), not compatible with versions before v1.7.4.

  • true: Enable full-band codec.
  • false: Disable full-band codec.
stereo

Stereo codec, not compatible with versions before v1.7.4.

  • true: Enable stereo codec.
  • false: Disable stereo codec.
fullBitrate

High bitrate. Recommended in voice-only mode.

  • true: Enable high bitrate mode.
  • false: Disable high bitrate mode.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Note

Agora does not recommend using this method. If you want to set the audio profile, see Set the Audio Profile (setAudioProfile) .

Join a Channel (joinChannel)

public abstract int joinChannel(String token,
                                String channelName,
                                String optionalInfo,
                                int optionalUid);

This method allows a user to join a channel. Users in the same channel can talk to each other; and multiple users in the same channel can start a group chat. Users using different App IDs cannot call each other. Once in a call, the user must call the leaveChannel method to exit the current call before entering another channel.

Note

A channel does not accept duplicate UIDs, such as two users with the same UID. If your app supports logging in a user from different devices at the same time, ensure that you use different UIDs. For example, if you already used the same UID, make the UIDs different by adding the respective device ID to the UID. This is not applicable if your app does not support a user logging in from different devices at the same time. In this case, when you log in a new device, you will be logged out from the other device.

Name Description
token

A Token generated by the application.

This parameter is optional if the user uses a static App ID. In this case, pass NULL as the parameter value.

If the user uses a Token, Agora issues an additional App Certificate to the application developers. The developers can then generate a user key using the algorithm and App Certificate provided by Agora for user authentication on the server.

In most circumstances, the static App ID will suffice. For added security, use a Token.

channelName

A string providing the unique channel name for the AgoraRTC session. The length must be within 64 bytes.

The following is the supported scope: a-z, A-Z, 0-9, space, !

#$%&, ()+,

-, :;<=., >?

@[], ^_, {|}, ~

optionalInfo (Optional) Additional information about the channel. Other users in the channel will not receive this information.
optionalUid

(Optional) User ID: A 32-bit unsigned integer ranging from 1 to (2^32-1). It must be unique. If not assigned (or set to 0), the SDK will assign one and return it in the onJoinChannelSuccess callback. The app needs to record and maintain the returned value as the SDK does not maintain it.

The UID is represented as a 32-bit unsigned integer in the SDK. Since unsigned integers are not supported by Java, the UID is handled as a 32-bit signed integer and larger numbers are interpreted as negative numbers in Java. If necessary, the UID can be converted to a 64-bit integer through “uid&0xffffffffL”.

Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.
  • ERR_INVALID_ARGUMENT (-2): The passed argument is invalid.
  • ERR_NOT_READY (-3): Initialization failed.

Leave a Channel (leaveChannel)

public abstract int leaveChannel();

This method allows a user to leave a channel, such as hanging up or exiting a call.

After joining a channel, the user must call the leaveChannel method to end the call before joining another one. The leaveChannel method releases all resources related to the call. The leaveChannel method is called asynchronously, and the user has not actually left the channel when the call returns. Once the user leaves the channel, the SDK triggers the onLeaveChannel callback.

Note

If you call destroy() immediately after leaveChannel, the leavechannel process will be interrupted, and the SDK will not trigger the onLeaveChannel callback.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Audio Effects

Set the Local Voice Pitch (setLocalVoicePitch)

public abstract int setLocalVoicePitch(double pitch);

This method changes the voice pitch of the local speaker.

Name Description
pitch Voice frequency, in the range of [0.5, 2.0]. The default value is 1.0.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Local Voice Equalization (setLocalVoiceEqualization)

public abstract int setLocalVoiceEqualization(int bandFrequency, int bandGain);

This method sets the local voice equalization effect.

Name Description
bandFrequency The band frequency ranging from 0 to 9; representing the respective 10-band center frequencies of voice effects, including 31, 62, 125, 500, 1k, 2k, 4k, 8k, and 16k Hz.
bandGain Gain of each band in dB; ranging from -15 to 15.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Local Voice Reverberation (setLocalVoiceReverb)

public abstract int setLocalVoiceReverb(int reverbKey, int value);

This method sets local voice reverberation.

Name Description
reverbKey The reverberation key. This method contains five reverberation keys. For details, see the description of each value:
value
  • AUDIO_REVERB_DRY_LEVEL = 0, (dB, [-20,10]), level of the dry signal
  • AUDIO_REVERB_WET_LEVEL = 1, (dB, [-20,10]), level of the early reflection signal (wet signal)
  • AUDIO_REVERB_ROOM_SIZE = 2, ([0, 100]), room size of the reflection
  • AUDIO_REVERB_WET_DELAY = 3, (ms, [0, 200]), length of the initial latency of the wet signal in ms
  • AUDIO_REVERB_STRENGTH = 4, ([0, 100]), length of the late reverberation
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Implementing a Live Video Broadcast

The following figure shows how to implement the basic live video broadcast function. Developers can add more functions according to the modules described in this document. Ensure that an RtcEngine object has been created for the application according to Create an RtcEngine Object (create).

For which APIs are the same as the ones used in a live voice broadcast, see Implementing a Live Voice Broadcast.

../_images/android_video_api_live.png

Create a Video Renderer View (createRendererView)

public static SurfaceView createRendererView (Context context);

This method creates a video renderer view. It returns the SurfaceView type.

The operation and layout of the view are managed by the application, and the Agora SDK renders the view provided by the application.

The view to display videos must be created using this method instead of directly calling SurfaceView.

Name Description
context The context of Android Activity.

Enable the Video Mode (enableVideo)

public abstract int enableVideo();

This method enables the video mode.

The application can call this method either before entering a channel or during a call. If it is called before entering a channel, the service starts in the video mode; if it is called during a call, it switches from the audio to video mode. To disable the video mode, call the disablevideo method.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Note

This method sets to enable the internal engine, and still works after leaveChannel is called.

Disable the Video Mode (disableVideo)

public abstract int disableVideo();

This method disables the video mode. The application can call this method either before entering a channel or during a call. If it is called before entering a channel, the service starts in the audio mode; if it is called during a call, it switches from the video to audio mode. To enable the video mode, call the enablevideo method.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Note

This method sets to disable the internal engine, and still works after leaveChannel is called.

Enable the Local Video (enableLocalVideo)

public abstract int enableLocalVideo(boolean enabled);

This method enables/disables the local video, which is only applicable to the scenario when the user only wants to watch the remote video without sending any video stream to the other user.

  • Call this method after enableVideo, otherwise this method may not work properly.
  • After enableVideo is called, the local video will be enabled by default. This method is useds to enable and disable the local video while the remote video remains unaffected.
Name Description
enabled

Sets to enable the local video:

  • true: Enable the local video (default)
  • false: Disable the local video. Once the local video is disabled, the remote users can no longer receive the video stream of this user, while this user can still receive the video streams of other remote users. When set to false, this method does not require a local camera.
Return value
  • 0: Method call succeeded
  • < 0: Method call failed

Note

This method sets to enable/disable the internal engine, and still works after leaveChannel is called.

Set the Video Encoder Configuration (setVideoEncoderConfiguration)

public abstract int setVideoEncoderConfiguration(VideoEncoderConfiguration config);

This method sets the video encoder configuration. Each configuration profile corresponds to a set of video parameters, including the resolution, frame rate, bitrate, and video orientation. The parameters specified in this method are the maximum values under ideal network conditions. If the video engine cannot render the videos using the specified parameters due to poor network conditions, the parameters further down the list are considered until success.

If the user does not need to set the video encoding profile after joining the channel, Agora recommends calling this method before enableVideo to minimize the time delay in receiving the first video frame.

Namwe Description
config The video configuration. See the below for the description of VideoEncoderConfiguration.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Definition of VideoEncoderConfiguration

public VideoEncoderConfiguration(
        VideoDimensions dimensions, FRAME_RATE frameRate, int bitrate, ORIENTATION_MODE orientationMode);

Where VideoDimendion is defined as follows:

static public class VideoDimensions {
    public int width;
    public int height;

    public VideoDimensions(int width, int height) {
        this.width = width;
        this.height = height;
    }

    public VideoDimensions() {
        this.width = 0;
        this.height = 0;
    }
}
Name Description
dimensions

You can either set the video resolution manually or choose one of the following enumerations:

  • VD_120x120
  • VD_160x120
  • VD_180x180
  • VD_240x180
  • VD_320x180
  • VD_240x240
  • VD_320x240
  • VD_424x240
  • VD_360x360
  • VD_480x360
  • VD_640x360
  • VD_480x480
  • VD_640x480
  • VD_840x480
  • VD_960x720 [1]
  • VD_1280x720 [1]
frameRate

Frame rate of the video. Choose one of the following frame rates:

  • FRAME_RATE_FPS_1(1): 1 fps
  • FRAME_RATE_FPS_7(7): 7 fps
  • FRAME_RATE_FPS_10(10): 10 fps
  • FRAME_RATE_FPS_15(15): 15 fps
  • FRAME_RATE_FPS_24(24): 24 fps
  • FRAME_RATE_FPS_30(30): 30 fps
bitrate

Bitrate of the video. See Base Bitrate Table to set your desired bitrate. If the desired bitrate goes beyond the limits, the SDK will choose a value from within the limits. You can also choose from the following options:

  • STANDARD_BITRATE(0): (Recommended) The standard mode. In this mode, the bitrate under the live broadcast and communication profile differs:
    • In the communication profile, the video bitrate is the same as the base bitrate.
    • In the live broadcast profile, the video bitrate is twice the base bitrate.
  • COMPATIBLE_BITRATE(-1): The compatible mode. In this mode, the bitrate stays the same regardless of the profile. In a live broadcast channel, if you choose this mode, the video frame rate may be lower than the value listed in Base Bitrate Table.

Agora uses different video codec for different channel profiles for optimized user experience. For example, a communication channel prioritizes smoothness while a live broadcast one emphasizes video quality. This means higher bitrate requirements for a live broadcast channel. Therefore, Agora recommends setting this parameter as STANDARD_BITRATE.

orientationMode

Video orientation mode of the video:

  • ORIENTATION_MODE_ADAPTIVE(0): (Default) The output video always follows the orientation of the captured video, because the receiver takes the rotational information passed on from the video encoder. Mainly used between Agora’s Native SDKs.
    • If the captured video is in landscape mode, the output video is in landscape mode.
    • If the captured video is in portrait mode, the output video is in portrait mode.
  • ORIENTATION_MODE_FIXED_LANDSCAPE(1): The output video is always in landscape mode. If the captured video is in portrait mode, the video encoder crops it to fit the output. Applies to situations where the receiving end cannot process the rotational information. For example, CDN streaming.
  • ORIENTATION_MODE_FIXED_PORTRAIT(2): The output video is always in portrait mode. If the captured video is in landscape mode, the video encoder crops it to fit the output. Applies to situations where the receiving end cannot process the rotational information. For example, CDN streaming.

For scenarios with video rotation, Agora provides Basic: Video Rotation Guide to guide users how to set this parameter to get the video orientation they want.

Footnotes:

[1](1, 2) Whether 720p can be supported depends on the device. If the device cannot support 720p, the frame rate will be lower than the one listed in the table. Agora optimizes the video performances on the lower-end devices. Contact support@agora.io if you have additional requirements.
Base Bitrate Table

Note

The base bitrate in this table applies to a communication channel.

A live broadcast channel generally requires higher bitrate for better video quality. Agora recommends setting the bitrate mode as STANDARD_BITRATE. You can also set the bitrate as the base bitrate value x 2.

Video Profile Resolution (Width x Height) Frame Rate (fps) Base Bitrate (Kbps)
120P 160 x 120 15 65
120P_3 120 x 120 15 50
180P 320 x 180 15 140
180P_3 180 x 180 15 100
180P_4 240 x 180 15 120
240P 320 x 240 15 200
240P_3 240 x 240 15 140
240P_4 424 x 240 15 220
360P 640 x 360 15 400
360P_3 360 x 360 15 260
360P_4 640 x 360 30 600
360P_6 360 x 360 30 400
360P_7 480 x 360 15 320
360P_8 480 x 360 30 490
480P 640 x 480 15 500
480P_3 480 x 480 15 400
480P_4 640 x 480 30 750
480P_6 480 x 480 30 600
480P_8 848 x 480 15 610
480P_9 848 x 480 30 930
480P_10 640 x 480 10 400
720P 1280 x 720 15 1130
720P_3 1280 x 720 30 1710
720P_5 960 x 720 15 910
720P_6 960 x 720 30 1380

The following bitrate values apply to the live broadcast profile. Setting the bitrate mode as STANDARD_BITRATE or COMPATIBLE_BITRATE does not change the bitrate value.

Video Profile Resolution (Width x Height) Frame Rate (fps) Bitrate (Kbps)
360P_9 640x360 15 800
360P_10 640x360 24 800
360P_11 640x360 24 1000

Set the Video Profile (setVideoProfile)

Note

This method has been deprecated. If you hope to set the video encoder profile, Agora recommends using Set the Video Encoder Configuration (setVideoEncoderConfiguration).

public abstract int setVideoProfile(int profile, boolean swapWidthAndHeight);

This method sets the video encoding profile. If the user does not need to set the video encoding profile after joining the channel, Agora recommends calling this method before enableVideo to minimize the time delay in receiving the first video frame.

Name Description
profile The video profile. See the Video Resolution, Frame Rate and Bitrate Table.
swapWidthAndHeight

The width and height of the output video is consistent with that of the video profile you set. This parameter sets whether to swap the width and height of the stream:

  • true: Swap the width and height. After that the video will be in the portrait mode, that is, width < height.
  • false: (Default) Do not swap the width and height, and the video remains in the landscape mode, that is, width > height.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.
Manually Set the Video Profile (SetVideoProfile)

Note

This method has been deprecated. If you hope to set the video encoder profile, Agora recommends using Set the Video Encoder Configuration (setVideoEncoderConfiguration).

public abstract int setVideoProfile(int width, int height, int frameRate, int bitrate);

If the video profiles listed in the table do not meet your needs, use this method to manually set the width, height, frame rate, and bitrate of the video. If the user does not need to set the video encoding profile after joining the channel, Agora recommends calling this method before enableVideo to minimize the time delay in receiving the first video frame.

Name Description
width Width of the video that you want to set. The highest value is 1280.
height Height of the video that you want to set. The highest value is 720.
frameRate Frame rate of the video that you want to set. The highest value is 30. You can set it to 5, 10, 15, 24, 30, and so on.
bitrate

Bitrate of the video that you want to set. You need to manually work out the bitrate according to the width, height, and frame rate. With the same width and height, the bitrate varies with the change of the frame rate [2]:

  • If the frame rate is 5 fps, divide the recommended bitrate in the table by 2.
  • If the frame rate is 15 fps, use the recommended bitrate in the table.
  • If the frame rate is 30 fps, multiply the recommended bitrate in the table by 1.5.
  • Calculate your bitrate with the ratio if you choose other frame rates.

If the bitrate you set is beyond the proper range, the SDK will automatically adjust it to a value within the range.

Set the Local Video View (setupLocalVideo)

public abstract int setupLocalVideo(VideoCanvas local);

This method configures the video display settings on the local machine.

The application calls this method to bind with the video window (view) of local video streams and configure the video display settings. Call this method after initialization to configure the local video display settings before entering a channel. After leaving the channel, the bind is still valid, which means the window still displays. To unbind the window, set the view value to NULL when calling setupLocalVideo.

Name Description
local

Video display settings. Class VideoCanvas:

  • view: Video display window (view).
  • renderMode: Video display mode.
    • RENDER_MODE_HIDDEN (1): Uniformly scale the video until it fills the visible boundaries. One dimension of the video may have clipped contents.
    • RENDER_MODE_FIT (2): Uniformly scale the video until one of its dimension fits the boundary. Areas that are not filled due to the disparity in the aspect ratio will be filled with black.
  • Return value: The local user ID as set in the joinchannel method.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Remote Video View (setupRemoteVideo)

public abstract int setupRemoteVideo(VideoCanvas remote);

This method binds the remote user to the video display window, that is, sets the view for the user of the specified uid. The application should specify the uid of the remote video in the method call before the user enters a channel.

If the remote uid is unknown to the application, set it later when the application receives the onUserJoined event. If the Video Recording function is enabled, the Video Recording Service joins the channel as a dumb client, which means other clients also receive this onUserJoined event. Your application should not bind it with the view, because it does not send any video stream. If your application cannot recognize the dumb client, bind it with the view when the onFirstRemoteVideoDecoded event is triggered. To unbind the user with the view, set the view to null. Once the user has left the channel, the SDK unbinds the remote user.

Name Description
remote

Video display settings. Class VideoCanvas:

  • view: Video display window (view).

  • renderMode: Video display mode.

    • RENDER_MODE_HIDDEN (1): Uniformly scale the video until it fills the visible boundaries. One dimension of the video may have clipped contents.
    • RENDER_MODE_FIT (2): Uniformly scale the video until one of its dimension fits the boundary. Areas that are not filled due to the disparity in the aspect ratio will be filled with black.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Enable Dual-stream Mode (enableDualStreamMode)

public abstract int enableDualStreamMode(boolean enabled);

This method sets the stream mode to single- (default) or dual-stream mode. After it is enabled, the receiver can choose to receive the high stream, that is, high-resolution high-bitrate video stream, or low stream, that is, low-resolution low-bitrate video stream.

Name Description
enabled

The mode is in single stream or dual stream:

  • true: Dual stream
  • false: Single stream
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Remote Video-stream Type (setRemoteVideoStreamType)

public abstract int setRemoteVideoStreamType(int uid, int streamType);

This method specifies the video-stream type of the remote user to be received by the local user when the remote user sends dual streams.

  • If dual-stream mode is enabled by calling enableDualStreamMode, you will receive the high-video stream by default. This method allows the application to adjust the corresponding video-stream type according to the size of the video windows to save the bandwidth and calculation resources.
  • If dual-stream mode is not enabled, you will receive the high-video stream by default.

The result after calling this method will be returned in onApiCallExecuted. The Agora SDK receives the high-video stream by default to save the bandwidth. If needed, users can switch to the low-video stream using this method.

Name Description
uid User ID
streamType

Set the video-stream size.

The following lists the video-stream types:

  • VIDEO_STREAM_HIGH(0): High-video stream
  • VIDEO_STREAM_LOW(1): Low-video stream
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

The resolution of the high-video stream is 1.1, 4.3, or 16.9. By default, the aspect ratio of the low-video stream is the same as that of the high-video stream. Once the resolution of the high-video stream is set, the system automatically sets the aspect ratio for the low-video stream.

Resolution Frame Rate Keyframe Interval Bitrate (kbit/s)
160 x 160 5 5 45
160 x 120 5 5 32
160 x 90 5 5 28

Set the Default Remote Video Stream Type (setRemoteDefaultVideoStreamType)

public abstract int setRemoteDefaultVideoStreamType(int streamType);

This method sets the default remote video stream type to high or low.

Name Description
streamType

The default remote video stream:

  • VIDEO_STREAM_HIGH(0): High-video stream
  • VIDEO_STREAM_LOW(1): Low-video stream
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set High-quality Video Preferences (setVideoQualityParameters)

public abstract int setVideoQualityParameters(boolean preferFrameRateOverImageQuality);

This method allows users to set video preferences.

Name Description
preferFrameRateOverImageQuality

The video preference to be set:

  • true: Frame rate over image quality
  • false: Image quality over frame rate (default)
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Start a Video Preview (startPreview)

public abstract int startPreview();

This method starts the local video preview before joining the channel. Before using this method, you need to:

  • Call setupLocalVideo to set up the local preview window and configure the attributes
  • Call enableVideo to enable video

Note

Once the local video preview is enabled, the preview will not be shut down if you call leaveChannel to leave the channel. To shut down the preview, call stopPreview.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Stop a Video Preview (stopPreview)

public abstract int stopPreview();

This method stops the local video preview and closes the video.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Picture-in-picture Layout (setVideoCompositingLayout)

Note

This method has been deprecated and Agora recommends you not to use it. If you want to set the compositing layout in the CDN broadcast, call the setLiveTranscoding method.

public abstract int setVideoCompositingLayout(const VideoCompositingLayout layout);

This method sets the picture-in-picture layouts for live broadcast. This method is only applicable when you want to push streams at the Agora server. When you push stream at the server:

  1. Define a canvas, its width and height (video resolution), the background color, and the number of videos you want to display in total.
  2. Define the position and size of each video on the canvas, and whether it is cropped or zoomed to fit.

The push stream application will format the information of the customized layouts as JSON and package it to the Supplemental Enhancement Information (SEI) of each key frame when generating the H.264 video stream and pushing it to the CDN vendors through the RTMP protocol.

Parameter details are explained in the following table, and you can also refer to Advanced: Adjusting the Picture-in-picture Layout.

Note

  • Call this method after joining a channel.
  • The application should only allow one user to call this method in the same channel, if more than one user calls this method, the other users must call clearVideoCompositingLayout to remove the settings.
Name Description
canvasWidth Width of the entire canvas (display window or screen).
canvasHeight Height of the entire canvas (display window or screen).
backgroundColor Enter any of the 6-digit symbols defined in RGB.
regions

Array of AgoraRtcVideoCompositingRegion. Each host in the channel can have a region to display the video on the screen with the following parameters to be set:

  • uid: User ID of the user with the video to be displayed in the region.
    • x[0.0,1.0]: Horizontal position of the region on the screen.
    • y[0.0,1.0]: Vertical position of the region on the screen.
    • width[0.0, 1.0]: Actual width of the region.
    • height[0.0, 1.0]: Actual height of the region.
  • zOrder[0, 100]: 0 means the region is at the bottom, and 100 means the region is at the top.
  • alpha[0.0, 1.0]: 0 means the region is transparent, and 1 means the region is opaque. The default value is 1.0.
  • renderMode:
    • RENDER_MODE_HIDDEN(1): Cropped
    • RENDER_MODE_FIT(2): Zoomed to fit
appData Application defined data.

Here is an example used to describe the position and size of the region:

../_images/sei_overview.png

Remove the Picture-in-Picture Layout Settings (clearVideoCompositingLayout)

Note

This method has been deprecated and Agora recommends you not to use it.

public abstract int clearVideoCompositingLayout();

This method removes the settings made after calling setVideoCompositingLayout.

Set the Local Video Display Mode (setLocalRenderMode)

public abstract int setLocalRenderMode(int mode);

This method configures the local video display mode.

The application can call this method multiple times during a call to change the display mode.

Name Description
mode

Configures the video display mode:

  • RENDER_MODE_HIDDEN (1): Uniformly scale the video until it fills the visible boundaries. One dimension of the video may have clipped contents.
  • RENDER_MODE_FIT (2): Uniformly scale the video until one of its dimension fits the boundary. Areas that are not filled due to the disparity in the aspect ratio will be filled with black.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Remote Video Display Mode (setRemoteRenderMode)

public abstract int setRemoteRenderMode(int uid, int mode);

This method configures the remote video display mode. The application can call this method multiple times during a call to change the display mode.

Name Description
uid The user ID of the user whose video streams are received.
mode

Configures the video display mode:

  • RENDER_MODE_HIDDEN (1): Uniformly scale the video until it fills the visible boundaries. One dimension of the video may have clipped contents.
  • RENDER_MODE_FIT (2): Uniformly scale the video until one of its dimension fits the boundary. Areas that are not filled due to the disparity in the aspect ratio will be filled with black.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Local Publish Fallback Option (setLocalPublishFallbackOption)

public abstract int setLocalPublishFallbackOption(int option);

This method disables/enables the local publish stream fallback option according to the network conditions.

If you set option as STREAM_FALLBACK_OPTION_AUDIO_ONLY, the SDK will:

  • Disable the upstream video when the network cannot support both video and audio.
  • Reenable the video when the network condition improves.

When the local publish stream falls back to audio-only or when the audio stream switches back to the video, the Local Publish Fallback to Audio Only Callback (onLocalPublishFallbackToAudioOnly) callback will be triggered.

Note

Agora does not recommend using this method for CDN streaming, because the remote CDN user will notice a lag when the local publish stream falls back to audio-only.

Name Description
option

The fallback options of the local publish stream:

  • STREAM_FALLBACK_OPTION_DISABLED = 0: (Default) No fallback operation to the local publish stream when the uplink network condition is poor, though the quality of the stream cannot be guaranteed.
  • STREAM_FALLBACK_OPTION_AUDIO_ONLY = 2: The local publish stream falls back to audio only when the uplink network condition is poor.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Remote Subscribe Fallback Option (setRemoteSubscribeFallbackOption)

public abstract int setRemoteSubscribeFallbackOption(int option);

This method disables/enables the remote subscribe stream fallback option according to the network conditions.

If you use this API and set option as STREAM_FALLBACK_OPTION_AUDIO_ONLY, the SDK will automatically switch the video from a high-stream to a low-stream, or turn off the video when the downlink network condition cannot support both audio and video to guarantee the quality of the audio. In the meantime, the SDK keeps track of the network quality and restores the video stream when the network conditions improve. Once the local publish stream falls back to audio only, or the audio stream switches back to the video stream, the Remote Subscribe Fallback To Audio Only Callback (onRemoteSubscribeFallbackToAudioOnly) callback will be triggered.

Name Description
option

Fallback options for the remote subscribe stream:

  • STREAM_FALLBACK_OPTION_DISABLED = 0: No fallback operation will be applied to the remote subscribe stream when the downlink condition deteriorates. The quality of the stream cannot be guaranteed.
  • STREAM_FALLBACK_OPTION_VIDEO_STREAM_LOW = 1: (Default) The remote subscribe stream falls back to low stream when the downlink condition deteriorates. This option works only for setRemoteSubscribeFallbackOption and not for setLocalPublishFallbackOption.
  • STREAM_FALLBACK_OPTION_AUDIO_ONLY = 2: The remote subscribe stream falls back to low video streams under poor downlink network conditions and further falls back to audio-only when the downlink condition deteriorates.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Add a Watermark to the Local Video Stream (addVideoWatermark)

public abstract int addVideoWatermark(AgoraImage watermark);

This method adds a watermark in the PNG file format onto the local video stream so that the recording device and the audience in the channel and CDN can see or capture it. To add the PNG file onto a CDN publishing stream only, see the method described in Set Live Transcoding (setLiveTranscoding).

The Definition of AgoraImage
public class AgoraImage {
    public String url;
    public int x;
    public int y;
    public int width;
    public int height;

    public AgoraImage() {
    }
}
Name Description
url URL address of the image on the broadcasting video.
x Position of the image on the upper left of the broadcasting video on the horizontal axis.
y Position of the image on the upper left of the broadcasting video on the vertical axis.
width Width of the image on the broadcasting video.
height Height of the image on the broadcasting video.

Note

  • The url descriptions are different for local video streaming and CDN streaming. In a local broadcast, url refers to the definite path of the image file to be added onto the local video; while in a CDN broadcast, url refers to the Url address of the image to be added onto the CDN broadcast.
  • The source file of the image must be in the PNG file format. If the width and height of the PNG file differ from your settings in this method, the PNG file will be cropped to conform to your settings.
  • The Agora SDK supports adding only one watermark onto the live stream. The newly added watermark will replace the previous one.
  • If you set orientationMode as Adaptive in the Set the Video Encoder Configuration (setVideoEncoderConfiguration) method, the watermark image rotates with the video frame and rotates around the upper left corner of the watermark image.

Clear the Watermark on the Video Stream (clearVideoWatermarks)

public abstract int clearVideoWatermarks();

This method clears the watermark image on the video stream.

Set the Local Video Mirror Mode (setLocalVideoMirrorMode)

public abstract int setLocalVideoMirrorMode(int mode);

This method sets the local video mirror mode. Use this method before startPreview, or it does not take effect until you re-enable startPreview.

Name Description
mode

Sets the local video mirror mode

  • 0: The default mirror mode, that is, the mode set by the SDK
  • 1: Enable the mirror mode
  • 2: Disable the mirror mode
Return value
  • 0: Method call succeeded
  • < 0: Method call failed

Manage the Camera

Switch between Front and Back Cameras (switchCamera)

public abstract int switchCamera();

This method switches between front and back cameras.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Check whether Camera Zoom is Supported (isCameraZoomSupported)

public abstract boolean isCameraZoomSupported();

It returns:

  • true: The device supports the camera zoom function
  • false: The device does not support the camera zoom function

Check whether Camera Flash is Supported (isCameraTorchSupported)

public abstract boolean isCameraTorchSupported();

It returns:

  • true: The device supports the camera flash function
  • false: The device does not support the camera flash function

Note

The App generally enables the front camera by default. If your front camera does not support front camera torch, this method will return false. If you want to check if the rear camera torch is supported, call switchCamera before using this method.

Check whether Manual Focus is Supported (isCameraFocusSupported)

public abstract boolean isCameraFocusSupported();

It returns:

  • true: The device supports the manual focus function
  • false: The device does not support the manual focus function

Check whether Autofocus is Supported (isCameraAutoFocusFaceModeSupported)

public abstract boolean isCameraAutoFocusFaceModeSupported();

It returns:

  • true: The device supports the autofocus function;
  • false: The device does not support the autofocus function;

Set the Camera Zoom Ratio (setCameraZoomFactor)

public abstract int setCameraZoomFactor(float factor);
Name Description
factor Camera zoom factor ranging from 1.0 to the max zoom

Get the Supported Max Zoom Ratio (getCameraMaxZoomFactor)

public abstract float getCameraMaxZoomFactor();

This method gets the max zoom ratio supported by the camera.

Set the Manual Focus Position (setCameraFocusPositionInPreview)

public abstract int setCameraFocusPositionInPreview(float positionX, float positionY);
Name Description
positionX Horizontal coordinate of the touch point in the view
positionY Vertical coordinate of the touch point in the view

Enable the Camera Flash (setCameraTorchOn)

public abstract int setCameraTorchOn(boolean isOn);
Name Description
isOn Whether to enable the camera flash function: true/false

Enable the Camera Autofocus (setCameraAutoFocusFaceModeEnabled)

public abstract int setCameraAutoFocusFaceModeEnabled(boolean enabled);
Name Description
enabled Whether to enable the camera autofocus function: true/false

Enable Web SDK Interoperability (enableWebSdkInteroperability)

public int enableWebSdkInteroperability(bool enabled);

This method enables interoperability with the Agora Web SDK.

Name Description
enabled

Whether to enable the interoperability with the Agora Web SDK is enabled:

  • True: Enable the interoperability
  • False: Disable the interoperability
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Audio Route

Set the Default Audio Route (setDefaultAudioRouteToSpeakerphone)

public abstract int setDefaultAudioRoutetoSpeakerphone(boolean defaultToSpeaker);

This method modifies the default audio route if necessary.

Note

  • Call this method only if you want to change the default settings.
  • This method only works in audio mode.
  • Call this method before joinChannel.

The default audio routes are listed in the following table:

Channel Mode Default Audio Route
Communication

Voice Communication: Earpiece

Video Communication: Speakerphone

If the user in a communication channel has called disableVideo or if the user has called muteLocalVideoStream and muteAllRemoteVideoStreams, the audio route is switched back to the earpiece automatically.

Live Broadcast Speakerphone
Game Voice Speakerphone

Modify the default audio route if necessary according to the following table:

Name Description
defaultToSpeaker
  • true: Speakerphone.
  • false: Earpiece.

Whether the audio is routed to the speakerphone or earpiece, once a headset is plugged in or Bluetooth is connected,

the audio route will be changed. The audio route will be switched to default once removing the headset or disconnecting Bluetooth.

Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Enable the Speakerphone (setEnableSpeakerphone)

public abstract int setEnableSpeakerphone(boolean enabled);

This method enables the audio routing to the speakerphone.

After calling this method, the SDK will return the onAudioRouteChanged callback to indicate the changes.

Note

Read the default audio route explanation according to Set the Default Audio Route (setDefaultAudioRouteToSpeakerphone) and check whether it is necessary to call this method.

Name Description
enabled
  • true:
    • If this API is called after joining a channel, whether the audio was routed to the headset, Bluetooth device, or earpiece, it will be routed to the speaker.
    • If this API is called before joining a channel, when joining a channel, the audio will be routed to the speaker whether the user uses a headset or Bluetooth device.
  • false: The audio will follow the default audio route mentioned in Set the Default Audio Route (setDefaultAudioRouteToSpeakerphone).
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Check Whether the Speakerphone is Enabled (isSpeakerphoneEnabled)

public abstract boolean isSpeakerphoneEnabled();

This method checks whether the speakerphone is enabled or disabled.

Name Description
Return value
  • true: The speakerphone is enabled.
  • false: The speakerphone is not enabled.

Enable In-ear Monitoring (enableInEarMonitoring)

public abstract int enableInEarMonitoring(boolean enabled);

This method enables or disables the in-ear monitoring function.

Name Description
enabled
  • true: Enable in-ear monitoring.
  • false: Disable in-ear monitoring.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Audio Volume

Set the Speakerphone Volume (setSpeakerphoneVolume)

public abstract int setSpeakerphoneVolume(int volume);

This method sets the speakerphone volume.

Name Description
volume Sets the speakerphone volume between 0 (lowest volume) to 255 (highest volume).
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Enable the Audio Volume Indication (enableAudioVolumeIndication)

public abstract int enableAudioVolumeIndication(int interval, int smooth);

This method allows the SDK to regularly report to the application on which user is speaking and the volume of the speaker. Once the method is enabled, the SDK returns the volume indication at the set time interval in the Audio Volume Indication Callback (onAudioVolumeIndication) callback, regardless of whether anyone is speaking in the channel.

Name Description
interval

Time interval between two consecutive volume indications:

  • <= 0: Disables the volume indication.
  • > 0: The time interval between two consecutive volume indications in milliseconds. Agora recommends setting it to more than 200 ms. Do not set it lower than 10 ms, or the onAudioVolumeIndication callback will not be triggered.
smooth Smoothing factor. The default value is 3.
Return values
  • 0: Method call succeeded.
  • < 0: Method call failed.

Set the Volume of the In-ear Monitor (setInEarMonitoringVolume)

public abstract int setInEarMonitoringVolume(int volume);

This method sets the volume of the in-ear monitor.

Name Description
volume Volume of the in-ear monitor, ranging from 0 to 100. The default value is 100.
Return values
  • 0: Method call succeeded.
  • < 0: Method call failed.

Custom the Video Source and Video Renderer

Set the Video Source (setVideoSource)

public abstract int setVideoSource(IVideoSource source);

This method sets the video source. In real-time communication, the Agora SDK generally uses the default video input source, that is, the built-in camera to publish streams. To use external video source, call The IVideoSource Interface to set the custom video source and then use this method to add the external video source into the SDK.

Name Description
source The custom video source. See descriptions in The IVideoSource Interface for details.

Set the Local Video Renderer (setLocalVideoRenderer)

public abstract int setLocalVideoRenderer(IVideoSink render);

This method sets the local video renderer. In real-time communication, the Agora SDK generally uses the default video renderer to render videos. To use the external video renderer, call The IVideoSink Interface to set the custom local video renderer and then use this method to add the external renderer into the SDK.

Name Description
render The custom local video renderer. See descriptions in The IVideoSink Interface for details.

Set the Remote Video Renderer (setRemoteVideoRenderer)

public abstract int setRemoteVideoRenderer(int uid, IVideoSink render);

This method sets the remote local renderer. In real-time communication, the Agora SDK generally uses the default video renderer to render videos. To use the external video renderer, call The IVideoSink Interface to set the custom remote video renderer and then use this method to add the external renderer into the SDK.

Name Description
render The custom remote video renderer. See descriptions in The IVideoSink Interface for details.

Custom Media Source and Sink API

The Custom Media Source and Sink API enables Apps to create the custom media source and sink, and pass them to the underlying media engine of the SDK to replace the default media source and sink. This API will replace the Configure the External Data API and Raw Data API in the future.

The Custom Media Source and Sink API consists of the following two sets of interfaces:

The IVideoSource Interface Defines a set of protocols to implement the custom video source and pass it to the underlying media engine to replace the default video source.
The IVideoSink Interface Defines a set of protocols to implement the custom video sink and pass it to the underlying media engine to replace the default video sink.

Note

  • Currently, the Custom Media Source & Sink API supports customizing the video device only. Agora will add APIs to enable the audio device customization in the future. For now, Agora recommends using the Configure the External Data API to customize the audio device.
  • All methods in the IVideoSource and IVideoSink Interfaces are callback methods. The underlying media engine of the SDK maintains a finite state machine and uses these functions at the right time. So do not use these methods directly in the App.

The IVideoSource Interface

public interface IVideoSource {
  boolean onInitialize(IVideoFrameConsumer consumer);
  boolean onStart();
  void onStop()
  void onDispose();
  int getBufferType();
}

By default, when enabling real-time communications, the Agora SDK enables the default video input device (built-in camera) to start video streaming. The IVideoSource Interface defines a set of protocols to create customized video source objects and pass them to the media engine to replace the default camera source so that you can take ownership of the video source and manipulate it.

Once you have implemented this interface, the Agora Media Engine will automatically release its ownership of the current video input device and pass it on to you, so that you can use the same video input device to capture the video stream.

Initialize the Video Source (onInitialize)
boolean onInitialize(IVideoFrameConsumer consumer);

This callback function initializes the video source. Developers can enable the camera or initialize the video source and then pass one of the following return values to inform the media engine whether the video source is ready:

Name Description
consumer The IVideoFrameConsumer object which the Media Engine passes to the developer. The developer needs to reserve this object, and then pass the video frame to the Media Engine through this object once the video source is initialized. See the following contents for the definition of IVideoFrameConsumer.
Return Value

Developers need to pass the return value to the Media Engine to inform it if the video source is ready:

  • true: The external video source is initialized
  • false: The external video source is not ready or fails to be initialized; the media engine will stop and report the error.

Note

When initializing the video source, the developers need to specify a Buffer type in the Get the Buffer Type (getBufferType) method, and pass the video source in that specified type to the Media Engine.

Definition of IVideoFrameConsumer

public interface IVideoFrameConsumer {
  void consumeByteBufferFrame(ByteBuffer buffer, int format, int width, int height, int rotation, long timestamp);
  void consumeByteArrayFrame(byte[] data, int format, int width, int height, int rotation, long timestamp);
  void consumeTextureFrame(int textureId, int format, int width, int height, int rotation, long timestamp, float[] matrix);
 }

IVideoFrameConsumer supports video data in three buffer types: ByteBuffer, ByteArray, and Texture. Call the Get the Buffer Type (getBufferType) method to specify a buffer type. The video data can only be transmitted in the corresponding type.

For the definition of each video buffer, see the following description:

The ByteBuffer Type

Name Description
buffer Buffer type (ByteBuffer)
format

Pixel format of the video frame:

  • I420(1): YUV420p
  • NV21(3): YUV420sp
  • RGBA(4): RGBA8888
  • TEXTURE_2D(10): GL_TEXTURE_2D
  • TEXTURE_OES(11): GL_TEXTURE_OES
width Width of the video frame.
height Height of the video frame.
rotation Clockwise rotating angle (0, 90, 180, and 270 degrees) of the video frame.
timestamp Timestamp (ms) of the video frame.

The ByteArray Type

Name Description
data Byte type (data)
format

Pixel format of the video frame:

  • I420(1): YUV420p
  • NV21(3): YUV420sp
  • RGBA(4): RGBA8888
  • TEXTURE_2D(10): GL_TEXTURE_2D
  • TEXTURE_OES(11): GL_TEXTURE_OES
width Width of the video frame.
height Height of the video frame.
rotation Clockwise rotating angle (0, 90, 180, and 270 degrees) of the video frame.
timestamp Timestamp (ms) of the video frame.

The Texture Type

Name Description
textureId ID of the texture
format

Pixel format of the video frame:

  • I420(1): YUV420p
  • NV21(3): YUV420sp
  • RGBA(4): RGBA8888
  • TEXTURE_2D(10): GL_TEXTURE_2D
  • TEXTURE_OES(11): GL_TEXTURE_OES
width Width of the video frame
height Height of the video frame
rotation Clockwise rotating angle of the video frame. You can rotate the frame by 0, 90, 180 and 270 degree
timestamp The timestamp of the video frame in millisecond
matrix Matrix of the texture. The float is between 0 and 1, such as 0.1, 0.2…

Note

When passing the texture frame to the media engine, you need to use the consumeTextureFrame() method in the GL environment in which the textureId is created.

For the definition of the YUV image format, see the following links for more information:

http://www.fourcc.org/yuv.php

https://msdn.microsoft.com/en-us/library/windows/desktop/dd206750(v=vs.85).aspx

Enable the Video Source (onStart)
boolean OnStart();

This callback function is called when the underlying media engine is ready to start video streaming. You should start the video source to capture the video frame. Once the frame is ready, use the IVideoFrameConsumer to consume the video frame.

Name Description
Return Value

Developers need to pass the return value to the Media Engine to inform it if the video source is enabled:

  • true: The external video source is enabled and IVideoFrameConsumer is called to receive video frames.
  • false: The external video source is not ready or fails to be enabled; the media engine will stop and report the error.
Stop the Video Source (onStop)
void OnStop();

This callback function is called when the media engine stops streaming. You should then stop capturing the video frame and consuming it. After this method, the video frames will be discarded by engine.

Release the Video Source (onDispose)
void OnDispose();

This callback function is called when the IVideoFrameConsumer is released by the media engine. You can now release the video source as well as the IVideoFrameconsumer .

Get the Buffer Type (getBufferType)
int getBufferType();

This callback function is called then the Media Engine wants to know the buffer type of the video source when it is being initialized. Developers need to specify one buffer type and then pass the type to the Media Engine:

Name Description
Return Value

Developers need to pass the return value to the Media Engine to inform it of the video buffer type:

  • BYTE_BUFFER(1)
  • BYTE_ARRAY(2)
  • TEXTURE(3)

The IVideoSink Interface

public interface IVideoSink extends IVideoFrameConsumer {
    boolean onInitialize();
    boolean onStart();
    void onStop();
    void onDispose();
    public long getEGLContextHandle();
    int getBufferType();
    int getPixelFormat();
}

By default, when you try to enable real-time communication, Agora SDK enables the default video sink start video rendering. Now, buy calling Set the Local Video Renderer (setLocalVideoRenderer) and Set the Remote Video Renderer (setRemoteVideoRenderer), however, you can change the default video sink. The IVideoSink interface defines a set of protocols to create customized video sink and pass them to the media engine to replace the default video renderer.

Once you have implemented this interface, you will receive callbacks from the media engine to indicate the state of the custom video sink and the underlying media engine and enable their synchronization. Follow each callback to handle resource allocation and release and receive the video frame from the media engine.

Initialize the Video Sink (onInitialize)
boolean onInitialize();

This callback function initializes the video sink. Pass one of the following return values to the media engine:

  • true: If the video sink is initialized
  • false: If the video sink is not ready or fails to be initialized; the media engine stops and reports the error.

You can also initialize the video sink before this method is called and return true in this method.

Enable the Video Sink (onStart)
boolean onStart();

This callback function is called when the media engine starts streaming.

  • true: If the video sink is ready.
  • false: If the video sink is not ready.
Disable the Video Sink (onStop)
void onStop();

This callback function is called when the media engine stops video streaming. You should then stop the video sink.

Release the Video Sink (onDispose)
void onDispose();

This callback function is called when the media engine wants to release the video sink.

Get the EGLContextHandle (getEGLContextHandle)
public long getEGLContextHandle();

This callback returns the native handle of the EGLContext if you are using OpenGL and have your own EGLContxt. You can share your EGLContext using the Texture ID returned by the SDK in your own EGLContext.

If you do not have an EGLContext, this callback returns 0; if the buffer type is set to Texture, the media engine will create an EGLContext internally and return the Texture ID.

Get the Buffer Type (getBufferType)
void getBufferType();

This callback function is called then the Media Engine wants to know the buffer type of the video frame when it is being initialized. Developers need to specify one buffer type and then pass the type to the Media Engine:

Name Description
Return Value

Developers need to pass the return value to the Media Engine to inform it of the video buffer type:

  • BYTE_BUFFER(1)
  • BYTE_ARRAY(2)
  • TEXTURE(3)
Get the Pixel Format (getPixelFormat)
void getPixelFormat();

This callback function gets the pixel format of the video frame when it is being initialized. You need to specify one pixel format and then pass it to the Media Engine:

format

The pixel format:

  • I420(1): YUV420p
  • NV21(3): YUV420sp
  • RGBA(4): RGBA8888
  • TEXTURE_2D(10): GL_TEXTURE_2D
  • TEXTURE_OES(11): GL_TEXTURE_OES

Configure the External Data API

Configure the External Audio Data

Note

If you use the external audio data API to capture audio, you may encounter echo problems. Developers need to take care of such problems themselves.

Set the External Audio Source (setExternalAudioSource)
public abstract int setExternalAudioSource(boolean enabled, int sampleRate, int channels);

This method sets the external audio source.

Name Description
enabled Enable the function of the external audio source: true/false.
sampleRate Sampling rate of the external audio source
channelsPerFrame Number of external audio source channels (two channels maximum)
Return value
  • 0: Method call succeeded.
  • < 0: Method call failed.
Push the External Audio Frame (pushExternalAudioFrameRawData)
public abstract int pushExternalAudioFrame(byte[] data, long timestamp);

This method pushes the external audio frame to the Agora SDK for encoding.

Name Description
data External audio data
timestamp Timestamp of the external audio frame to be synchronized with the external video source
Return value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Configure the External Video Data

This section describes how to configure an external video source to capture video data. The external video source function in libvideoprp is merged with the texture encoding interfaces. The YUV/RGBA/textured images are sent to the Agora SDK for encoding.

Note

It is recommended to use this method for apps under development.

Check Whether Texture Encoding Is Supported (isTextureEncodeSupported)
public abstract boolean isTextureEncodeSupported();

This method checks whether texture encoding is supported.

Name Description
Return Value

true: Texture encoding is supported.

false: Texture encoding is not supported.

Configure the External Video Source (setExternalVideoSource)
public abstract void setExternalVideoSource(boolean enable, boolean useTexture, boolean pushMode);

This method sets whether to use an external video source:

  • If yes, whether to use the texture as an input;
  • If yes, whether to push the video frame to the Agora SDK for encoding.
Name Description
enable [2]

Whether to use an external video source:

  • true: Use external video source.
  • false: Do not use external video source.
useTexture

Whether to use the texture as an input:

  • true: Use the texture as an input.
  • false: Do not use the texture as an input.
pushMode

Whether the external video source needs to call PushExternalVideoFrame to send the video frame to the Agora SDK:

  • true: Use the push mode.
  • false: Use the pull mode (not supported yet).
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Footnotes

[2](1, 2) Currently, the Agora SDK does not support switching video sources dynamically in the channel. If an external video source is enabled and you are in a channel, if you want to switch to an internal video source, you must exit the channel. Then call this method to set enable as false, and join the channel again.
Push the External Video Frame (pushExternalVideoFrame)
public abstract boolean pushExternalVideoFrame(AgoraVideoFrame frame);

This method pushes the video frame using the AgoraVideoFrame class and passes it to the Agora SDK.

Call setExternalVideoSource and set the pushMode parameter as true before calling this method. Otherwise, it will return a failure after calling this method.

Note

During a video call, this method does not support pushing textured stream.

Name Description
frame Video frame that contains the video data to be encoded by the Agora SDK.
Return Value
  • true: Frame is pushed successfully
  • false: Failed to push the frame

The definition of the AgoraVideoFrame class:

public AgoraVideoFrame() {

  //mandatory
    public int format;
    public long timeStamp;
    public int stride;
    public int height;

    // Texture related
    public javax.microedition.khronos.egl.EGLContext eglContext11;
    public android.opengl.EGLContext eglContext14;
    public int textureID;
    public boolean syncMode;
    public float[] transform;

    // Raw data related
    public byte[] buf;
    public int cropLeft;
    public int cropTop;
    public int cropRight;
    public int cropBottom;
    public int rotation;
}

An explanation of the fields are listed in the following table:

format Mandatory Field

Format of the incoming video frame, which must be specified as one of the following:

  • AgoraVideoFrame.FORMAT_TEXTURE_2D
  • AgoraVideoFrame.FORMAT_TEXTURE_OES
  • AgoraVideoFrame.FORMAT_I420
  • AgoraVideoFrame.FORMAT_NV21
  • AgoraVideoFrame.FORMAT_RGBA
timestamp Mandatory Field Timestamp of the incoming video frame (ms). An incorrect timestamp will result in frame loss or unsynchronized audio and video.
stride Mandatory Field Line spacing of the incoming video frame, which must be in pixels instead of bytes. For textures, it is the width of the texture.
height Mandatory Field Height of the incoming video frame.
eglContext11 Texture Related Field When using the OpenGL interface (javax.microedition.khronos.egl.*) defined by Khronos, you need to set the EGLContext to this field.
eglContext14 Texture Related Field When using the OpenGL interface (android.opengl.*) defined by Android, you need to set the EGLContext to this field.
textureID Texture Related Field Texture ID used by the video frame.
transform Texture Related Field Incoming 4 x 4 transformational matrix. The typical value is a unit matrix.
syncMode Texture Related Field

(Optional) Sets whether to wait for the encoding completion of the previous one video frame:

  • true: Wait.
  • false: Do not wait.
buf Raw Data Field Content data of the incoming video frame.
cropLeft, cropTop, cropRight, cropBottom Raw Data Field (Optional) Specifies the number of pixels trimmed in each direction, which is set as 0 by default.
rotation Raw Data Field (Optional) Specifies whether to rotate the incoming video group. Optional values: 0, 90, 180, or 270 clockwise. It is set as 0 by default.

Sample Code:

When the format of the incoming video frame is TEXTURE_2D and is EGLContext defined by Khronos:

import io.agora.rtc.video.AgoraVideoFrame;

private final static float[] UNIQUE_MAT = {
    1.0f, 0.0f, 0.0f, 0.0f,
    0.0f, 1.0f, 0.0f, 0.0f,
    0.0f, 0.0f, 1.0f, 0.0f,
    0.0f, 0.0f, 0.0f, 1.0f
};

AgoraVideoFrame vf = new AgoraVideoFrame();
vf.format = AgoraVideoFrame.FORMAT_TEXTURE_2D;
vf.timeStamp = System.currentTimeMillis();
vf.stride = 352;
vf.height = 640;
vf.textureID = textureId;
vf.eglContext11 = eglContext;
vf.transform = UNIQUE_MAT;
mRtcEngine.pushExternalVideoFrame(vf);

When the format of the incoming video frame is NV21 YUV:

AgoraVideoFrame vf = new AgoraVideoFrame();
vf.format = AgoraVideoFrame.FORMAT_NV21;
vf.timeStamp = System.currentTimeMillis();
vf.stride = 640;
vf.height = 480;
vf.rotation = rotation;
vf.buf = data;
mRtcEngine.pushExternalVideoFrame(vf);

Raw Data API

Set the Recording Audio Format (setRecordingAudioFrameParameters)

public abstract int setRecordingAudioFrameParameters(int sampleRate,
                                                     int channel,
                                                     int mode,
                                                     int samplesPerCall);

This method sets the format of the callback data in onRecordAudioFrame.

Name Description
sampleRate It specifies the sampling rate in the callback data returned by onRecordAudioFrame, which can set be as 8000, 16000, 32000, 44100 or 48000.
channel

It specifies the number of channels in the callback data returned by onRecordAudioFrame, which can be set as 1 or 2:

  • 1: Mono
  • 2: Dual-Track
mode

It specifies the use mode of onRecordAudioFrame callback:

  • AgoraAudioRawFrameOperationModeReadOnly = 0: Read-only mode, users only read the AudioFrame data without modifying anything. For example,

    The users acquire data with Agora SDK, then push RTMP streams by themselves, then they can use this mode.

  • AgoraAudioRawFrameOperationModeWriteOnly = 1: Write-only mode, users replace the AudioFrame data with their own data and pass to the SDK for encoding. For example,

    The users acquire data by themselves, then they can use this mode.

  • AgoraAudioRawFrameOperationModeReadWrite = 2: Read and write mode, users read the data from AudioFrame, modify it and then play it. For example,

    The users have their own sound-effect processing module, and they want to do some voice pre-processing based on actual needs, for example, voice change, then they can use this mode.

samplesPerCall It specifies the sampling points in the called data returned in onRecordAudioFrame, for example, it is usually set as 1024 for stream pushing.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Playback Audio Format (setPlaybackAudioFrameParameters)

public abstract int setPlaybackAudioFrameParameters(int sampleRate,
                                                    int channel,
                                                    int mode,
                                                    int samplesPerCall);

This method sets the format of the callback data in onPlaybackAudioFrame.

Name Description
sampleRate Specifies the sampling rate in the callback data returned by onPlaybackAudioFrame, which can set be as 8000, 16000, 32000, 44100, or 48000.
channel

Specifies the number of channels in the callback data returned by onPlaybackAudioFrame, which can be set as 1 or 2:

  • 1: Mono
  • 2: Dual-Track
mode

Specifies the use mode of the onPlaybackAudioFrame callback:

  • AgoraAudioRawFrameOperationModeReadOnly = 0: Read-only mode; users only read the AudioFrame data without modifying anything. For example,

    Users can set this mode when they acquire data with the Agora SDK, then push RTMP streams by themselves.

  • AgoraAudioRawFrameOperationModeWriteOnly = 1: Write-only mode; users replace the AudioFrame data with their own data. For example,

    Users can use this mode when they acquire data by themselves.

  • AgoraAudioRawFrameOperationModeReadWrite = 2: Read and write mode; users read the data from AudioFrame, modify it, and then play it. For example,

    Users can use this mode when they have their own sound-effect processing module, and want to do some voice post-processing, such as a voice change.

samplesPerCall Specifies the sampling points in the called data returned in onPlaybackAudioFrame. For example, it is usually set as 1024 for stream pushing.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Mixed Recording and Playback Data Format (setMixedAudioFrameParameters)

public abstract int setMixedAudioFrameParameters(int sampleRate, int samplesPerCall);

This method sets the format of the callback data in onMixedAudioFrame.

Name Description
sampleRate Specifies the sampling rate in the callback data returned by onMixedAudioFrame, which can set be as 8000, 16000, 32000, 44100, or 48000.
samplesPerCall Specifies the sampling points in the called data returned in onMixedAudioFrame, for example, it is usually set as 1024 for stream pushing.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Register the Audio Observer Object (registerAudioFrameObserver)

int IMediaEngine::registerAudioFrameObserver(IAudioFrameObserver* observer);

This method registers the audio observer object. When you need the engine to return the callback of onRecordAudioFrame, onPlaybackAudioFrame, or onPlaybackAudioFrameObserver, call this method to register the callback.

Name Description
observer

Interface object instance.

Set the value to NULL to cancel registering if necessary.

Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Register the Video Observer Object (registerVideoFrameObserver)

int registerVideoFrameObserver(agora::media::IVideoFrameObserver *observer);

This method registers the video observer object. When you need the engine to return the callback of onCaptureVideoFrame or onRenderVideoFrame, call this method to register the callback.

Name Description
observer

Interface object instance.

Set the value to NULL to cancel registering if necessary.

Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

On Android platforms, registerVideoFrameObserver is defined in libHDACEngine.so, which need to be loaded manually.

Mute the Audio and Video Stream

Mute the Local Audio Stream (muteLocalAudioStream)

public abstract int muteLocalAudioStream(boolean muted);

This method enables/disables sending local audio streams to the network.

Note

When set to true, this method does not disable the microphone, and thus does not affect any ongoing recording.

Name Description
On
  • true: Stops sending local audio streams to the network.
  • false: Allows sending local audio streams to the network.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Mute all Remote Audio Streams (muteAllRemoteAudioStreams)

public abstract int muteAllRemoteAudioStreams(boolean muted);

This method enables/disables playing all remote users’ audio streams.

Note

When set to true, this method stops playing audio streams without affecting the audio stream receiving process.

Name Description
On
  • true: Stops playing all the received audio streams
  • false: Allows playing all the received audio streams
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Mute a Specified Remote Audio Stream (muteRemoteAudioStream)

public abstract int muteRemoteAudioStream(int uid, boolean muted);

This method mutes/unmutes a specified remote user’s audio stream.

Note

When set to true, this method stops playing audio streams without affecting the audio stream receiving process.

Name Description
uid User ID whose audio streams the user intends to mute.
muted
  • true: Stops playing a specified user’s audio streams.
  • false: Allows playing a specified user’s audio streams.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Mute the Local Video Stream (muteLocalVideoStream)

public abstract int muteLocalVideoStream(boolean muted);

This method enables/disables sending local video streams to the network.

Note

When set to true, this method does not disable the camera and thus does not affect the retrieval of local video streams. This method responds faster than enableLocalVideo (false).

Name Description
mute
  • true: Stops sending local video streams to the network.
  • false: Allows sending local video streams to the network.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Mute all Remote Video Streams (muteAllRemoteVideoStreams)

public abstract int muteAllRemoteVideoStreams(boolean muted);

This method enables/disables playing all remote callers’ video streams.

Note

When set to true, this method stops playing video streams without affecting the video stream receiving process.

Name Description
mute
  • true: Stops playing all the received video streams.
  • false: Allows playing all the received video streams.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Mute a Specified Remote Video Stream (muteRemoteVideoStream)

public abstract int muteRemoteVideoStream(int uid, boolean muted);

This method pauses/resumes receiving a specified user’s video stream.

Note

When set to true, this method stops playing video streams without affecting the video stream receiving process.

Name Description
uid Specified user’s user ID.
mute
  • true: Stops playing a specified user’s video stream.
  • false: Allows playing a specified user’s video stream.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Audio Mixing

Start Audio Mixing (startAudioMixing)

public abstract int startAudioMixing(String filePath,
                                     boolean loopback,
                                     boolean replace,
                                     int cycle);

This method mixes the specified local audio file with the audio stream from the microphone; or, it replaces the microphone’s audio stream with the specified local audio file. You can choose whether the other user can hear the local audio playback and specify the number of loop playbacks. This API also supports online music playback.

Note

  • To use this API, ensure that the Android device version is 4.2 or later, and the API version is 16 or later.
  • Call this API when you are in the channel, otherwise, it may cause issues.
  • If you call this API on an emulator, only the MP3 file format is supported.
Name Description
filePath

Name and path of the local audio file to be mixed:

  • If the path begins with /assets/, find the audio file in the /assets/ directory.
  • Otherwise, find the audio file in the absolute path.

Supported audio formats: mp3, aac, m4a, 3gp, wav, flac

loopback
  • true: Only the local user can hear the remix or the replaced audio stream.
  • false: Both users can hear the remix or the replaced audio stream.
replace
  • true: The content of the local audio file replaces the audio stream from the microphone.
  • false: Local audio file mixed with the audio stream from the microphone.
cycle

Number of loop playbacks:

  • Positive integer: Number of loop playbacks
  • -1: Infinite loop
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Stop Audio Mixing (stopAudioMixing)

public abstract int stopAudioMixing();

This method stops audio mixing. Call this API when you are in a channel.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Pause Audio Mixing (pauseAudioMixing)

public abstract int pauseAudioMixing();

This method pauses audio mixing. Call this API when you are in a channel.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Resume Audio Mixing (resumeAudioMixing)

public abstract int resumeAudioMixing();

This method resumes audio mixing. Call this API when you are in a channel.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Adjust the Audio Mixing Volume (adjustAudioMixingVolume)

public abstract int adjustAudioMixingVolume(int volume);

This method adjusts the volume during audio mixing. Call this API when you are in a channel.

Name Description
volume Volume ranging from 0 to 100. By default, 100 is the original volume.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Get the Audio Mixing Duration (getAudioMixingDuration)

public abstract int getAudioMixingDuration();

This method gets the duration (ms) of audio mixing. Call this API when you are in a channel.

Get the Current Audio Position (getAudioMixingCurrentPosition)

public abstract int getAudioMixingCurrentPosition();

This method gets the playback position (ms) of the audio. Call this API when you are in a channel.

Drag the Audio Progress Bar (setAudioMixingPosition)

public abstract int setAudioMixingPosition(int pos);

This method drags the playback progress bar of the audio mixing file to where you want to play instead of playing it from the beginning.

Name Description
pos Integer. The position of the audio mixing file (ms).

Recording

Start an Audio Recording (startAudioRecording)

public abstract int startAudioRecording(String filePath, int quality);

This method starts an audio recording. The SDK allows recording during a call, which supports either one of the following two formats:

  • .wav: Large file size with high sound fidelity OR
  • .aac: Small file size with low sound fidelity

Ensure that the directory to save the recording file exists and is writable. This method is usually called after the joinChannel() method. The recording automatically stops when the leaveChannel() method is called.

Name Description
filePath Full file path of the recording file. The string of the file name is in UTF-8 code.
quality

Audio recording quality:

  • AUDIO_RECORDING_QUALITY_LOW = 0: Low quality, file size is around 1.2 MB after 10 minutes of recording.
  • AUDIO_RECORDING_QUALITY_MEDIUM = 1: Medium quality, file size is around 2 MB after 10 minutes of recording.
  • AUDIO_RECORDING_QUALITY_HIGH = 2: High quality, file size is around 3.75 MB after 10 minutes of recording.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Stop an Audio Recording (stopAudioRecording)

public abstract int stopAudioRecording();

This method stops recording on the client.

Note

Call this method before calling leaveChannel, otherwise, the recording automatically stops when the leaveChannel method is called.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Adjust the Recording Volume (adjustRecordingSignalVolume)

public abstract int adjustRecordingSignalVolume(int volume);

This method adjusts the recording volume.

Name Description
volume

Recording volume, ranges from 0 to 400:

  • 0: Mute
  • 100: Original volume
  • 400: (Maximum) Four times the original volume with signal clipping protection
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Adjust the Playback Volume (adjustPlaybackSignalVolume)

public abstract int adjustPlaybackSignalVolume(int volume);

This method adjusts the playback volume.

Name Description
volume

Playback volume, ranges from 0 to 400:

  • 0: Mute
  • 100: Original volume
  • 400: (Maximum) Four times the original volume with signal clipping protection
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Encryption

Register a Packet Observer (registerAgoraPacketObserver)

JNIEXPORT jint registerAgoraPacketObserver(void* engine,agora::IPacketObserver* observer);

This method registers a packet observer. When sending or receiving audio or video packets, the Agora SDK calls back the interface defined by IPacketObserver, and the application can use this interface to process data, performing, for example, data encryption and decryption. When the application initializes the RtcEngine instance, use dlopen/dlsym to find registerAgoraPacketObserver, the derived function of libagora-rtc-sdk-jni.so.

Note

The maximum size of the processed network packet is 1200 bytes. Any packet larger than the maximum size may not be sent successfully.

Name Description
engine It can be retrieved by calling the API RtcEngine.getNativeHandle().
observer Callback interface of the send/receive packet.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Enable Built-in Encryption and Set the Encryption Secret (setEncryptionSecret)

public abstract int setEncryptionSecret(String secret);

Use setEncryptionSecret to specify an encryption password to enable built-in encryption before joining a channel. All users in a channel must set the same encryption password. The encryption password is automatically cleared once a user has left the channel. If the encryption password is not specified or set to empty, the encryption function will be disabled.

Note

Do not use this method together with CDN.

Name Description
secret Encryption Password
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Set the Built-in Encryption Mode (setEncryptionMode)

public abstract int setEncryptionMode(String encryptionMode);

The Agora Native SDK supports built-in encryption, which is in AES-128-XTS mode by default. If you want to use other modes, call this API to set the encryption mode.

All users in the same channel must use the same encryption mode and password. Refer to information related to the AES encryption algorithm on the differences between encryption modes.

Call setEncryptionSecret to enable the built-in encryption function before calling this API.

Note

Do not use this function together with CDN.

Name Description
encryptionMode

Encryption mode. The following modes are supported:

  • “aes-128-xts”:128-bit AES encryption, XTS mode
  • “aes-128-ecb”:128-bit AES encryption, ECB mode
  • “aes-256-xts”: 256-bit AES encryption, XTS mode
  • “”: When it is set to NUL string, the encryption is in “aes-128-xts” by default.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Establish a Data Channel

Create a Data Stream (createDataStream)

public abstract int createDataStream(boolean reliable, boolean ordered);

This method creates a data stream. Each user can only have up to five data channels at the same time.

Note

Set reliable and ordered both as true or both as false. Do not set one as true and the other as false.

Name Description
reliable
  • true: The recipients will receive data from the sender within 5 seconds. If the recipient does not receive the sent data within 5 seconds, the data channel will report an error to the application.
  • false: The recipients may not receive any data, and the SDK will not report any error upon data missing.
ordered
  • true: The recipients will receive data in the order of the sender.
  • false: The recipients will not receive data in the order of the sender.
Return Value
  • <0: Returns an error code when it fails to create the data stream. [3]
  • >0: Returns the Stream ID when the data stream is created.

Footnotes

[3]The error code is related to the positive integer displayed in Error Codes and Warning Codes, for example, if it returns -2, then it indicates 2: ERR_INVALID_ARGUMENT in Error Codes and Warning Codes.

Send a Data Stream (sendStreamMessage)

public abstract int sendStreamMessage(int streamId, byte[] message);

This method sends data stream messages to all users in a channel. Up to 30 packets can be sent per second in a channel with each packet having a maximum size of 1 kB. The API controls the data channel transfer rate. Each client can send up to 6 kB of data per second. Each user can have up to five data channels simultaneously.

Name Description
streamId Stream ID returned by createDataStream.
message Data to be sent
Return Value

When it fails to send the message, the following error code will be returned:

ERR_SIZE_TOO_LARGE/ERR_TOO_OFTEN/ERR_BITRATE_LIMIT

Test and Detection

Start an Audio Call Test (startEchoTest)

public abstract int startEchoTest();

This method launches an audio call test to determine whether the audio devices (for example, headset and speaker) and the network connection are working properly. In the test, the user first speaks, and the recording is played back in 10 seconds. If the user can hear the recording in 10 seconds, it indicates that the audio devices and network connection work properly.

Note

  • After calling the startEchoTest method, call stopEchoTest to end the test; otherwise, the application cannot run the next echo test, nor can it call the joinChannel method to start a new call.
  • This method is applicable only when the user role is broadcaster. If you switch the channel mode from communication to live broadcast, ensure that setClientRole is called to change the user role from audience(default in the live broadcast mode) to broadcaster before using this method.
Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.
  • ERR_REFUSED (-5): Failed to launch the echo test, for example, initialization failed.

Stop an Audio Call Test (stopEchoTest)

public abstract int stopEchoTest();

This method stops an audio call test.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.
  • ERR_REFUSED(-5): Failed to stop the echo test. It could be that the echo test is not running.

Enable the Network Test (enableLastmileTest)

public abstract int enableLastmileTest();

This method tests the quality of the user’s network connection and is disabled by default.

This method is mainly used in two scenarios:

  • Before users join a channel, this method can be called to check the uplink network quality.
  • Before the audience in a channel switch to a host role, this method can be called to check the uplink network quality.

In both scenarios, calling this method consumes extra network traffic, which affects the communication quality.

Call disableLastmileTest to disable it immediately once the users have received the onLastmileQuality callback before they join the channel or switch the user role.

Note

For current hosts, do not call this method.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Disable the Network Test (disableLastmileTest)

public abstract int disableLastmileTest();

This method disables the network connection quality test.

Name Description
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Feedback

Retrieve the Current Call ID (getCallId)

public abstract String getCallId();

This method retrieves the current call ID.

When a user joins a channel on a client, a CallId is generated to identify the call from the client. Some methods such as rate and complain need to be called after the call ends in order to submit feedback to the SDK. These methods require assigned values of the CallId parameters. To use these feedback methods, call the getCallId method to retrieve the CallId during the call, and then pass the value as an argument in the feedback methods after the call ends.

Name Description
Return Value Current call ID.

Rate the Call (rate)

public abstract int rate(String callId,
                         int rating,
                         String description);

This method lets the user rate the call. It is usually called after the call ends.

Name Description
callId Call ID retrieved from the getCallId method.
rating Rating for the call between 1 (lowest score) to 10 (highest score).
description

A given description for the call with a length less than 800 bytes.

This parameter is optional.

Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.
  • ERR_INVALID_ARGUMENT (-2): The passed argument is invalid, for example, callId invalid.
  • ERR_NOT_READY (-3): The SDK status is incorrect, for example, initialization failed.

Complain about the Call Quality (complain)

public abstract int complain(String callId,
                             String description);

This method allows the user to complain about the call quality. It is usually called after the call ends.

Name Description
callId Call ID retrieved from the getCallId method.
description

A given description of the call with a length less than 800 bytes.

This parameter is optional.

Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.
  • ERR_INVALID_ARGUMENT (-2): The passed argument is invalid, for example, callId invalid.
  • ERR_NOT_READY (-3): The SDK status is incorrect, for example, initialization failed.

Other Functions

Renew Token (renewtoken)

public abstract int renewToken(String token);

This method updates the Token.

The key expires after a certain period of time once the Token schema is enabled when:

  • The onError callback reports the ERR_TOKEN_EXPIRED(109) error, or
  • The onRequestToken callback reports the ERR_TOKEN_EXPIRED(109) error, or
  • The user receives the onTokenPrivilegeWillExpire callback,

The application should retrieve a new key and then call this method to renew it. Failure to do so will result in the SDK disconnecting with the server.

Name Description
token Specifies the Token to be renewed.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Specify a Log File (setLogFile)

public abstract int setLogFile(String filePath);

This method specifies an SDK output log file. The log file records all the log data of the SDK’s operation. Ensure that the directory to save the log file exists and is writable.

Name Description
filePath File path of the log file. The string of the log file is in UTF-8 code.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Note

The default log file location is at: sdcard/<appname>/agorasdk.log, where appname is the name of the application.

Set the Log Filter (setLogFilter)

public abstract int setLogFilter(int filter);

This method sets the output log level of the SDK. You can use either one or a combination of the filters.

The log level follows the sequence of OFF, CRITICAL, ERROR, WARNING, INFO, and DEBUG. Choose a level, and you can see logs that precede that level.

For example, if you set the log level as WARNING, then you can see logs in levels CRITICAL, ERROR and WARNING.

Name Description
filter

Set the levels of the filters:

  • LOG_FILTER_OFF = 0: Output no log.
  • LOG_FILTER_DEBUG = 0x80f: Output all the API logs.
  • LOG_FILTER_INFO = 0x0f: Output logs of the CRITICAL, ERROR, WARNING and INFO level.
  • LOG_FILTER_WARNING = 0x0e: Output logs of the CRITICAL, ERROR and WARNING level.
  • LOG_FILTER_ERROR = 0x0c: Output logs of the CRITICAL and ERROR level.
  • LOG_FILTER_CRITICAL = 0x08: Output logs of the CRITICAL level.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Destroy the Engine Instance (destroy)

public static synchronized void destroy();

This method releases all the resources used by the Agora SDK. This is useful for applications that occasionally make voice or video calls, to free up resources for other operations when not making calls.

Once the application has called destroy() to destroy the created RtcEngine instance, no other methods in the SDK can be used and no callbacks occur.

To start communications again, a new Initialize(sharedEngineWithappId) must be performed to establish a new AgoraRtcEngineKit instance.

Note

  • Use this method in the sub thread.
  • This method is called synchronously. The result returns after the IRtcEngine object resources are released. The app should not call this interface in the callback generated by the SDK, otherwise the SDK must wait for the callback to return before it can reclaim the related object resources, causing a deadlock.

Get the SDK Version (getSdkVersion)

public static String getSdkVersion();

This method returns the string of the version number in char format.

Audio Effect Methods (IAudioEffectManager)

Get the Audio Effect Volume (getEffectsVolume)

public double getEffectsVolume();

This method gets the volume of the audio effects from 0.0 to 100.0.

Set the Audio Effect Volume (setEffectsVolume)

public int setEffectsVolume(double volume);

This method sets the volume of the audio effects.

Name Description
volume Value ranges from 0.0 to 100.0. 100.0 is the default value.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Adjust the Audio Effect Volume in Real Time (setVolumeOfEffect)

public int setVolumeOfEffect(int soundId, double volume);

This method adjusts the volume of the specified audio effect in real time.

Name Description
soundId ID of the audio effect. Each audio effect has a unique ID.
volume Value ranges from 0.0 to 100.0. 100.0 is the default value.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Play the Audio Effect (playEffect)

public int playEffect(int soundId, String filePath, int loopCount, double pitch, double pan, double gain, boolean publish);

This method plays the specified audio effect.

Name Description
soundId ID of the specified audio effect. Each audio effect has a unique ID [4].
filePath The absolute file path of the audio effect file.
loopCount

Set the number of times looping the audio effect:

  • 0: Play the audio effect once
  • 1: Play the audio effect twice
  • -1: Play the audio effect in a loop indefinitely, until stopEffect or stopAllEffects is called
pitch Set whether to change the pitch of the audio effect. The range is [0.5, 2]. The default value is 1, which means no change to the pitch. The smaller the value, the lower the pitch.
pan

Spatial position of the local audio effect. The range is [-1.0, 1.0]

  • 0.0: The audio effect shows ahead.
  • 1.0: The audio effect shows on the right.
  • -1.0: The audio effect shows on the left.
gain Volume of the audio effect. The range is [0.0, 100,0]. The default value is 100.0. The smaller the value, the lower the volume of the audio effect.
publish

Set whether to publish the specified audio effect to the remote stream:

  • true: The audio effect, played locally, is published to the Agora Cloud and the remote users can hear it.
  • false: The audio effect, played locally, is not published to the Agora Cloud and the remote users cannot hear it.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Footnotes

[4]If you preloaded the audio effect into the memory through preloadEffect, ensure that the soundID value is set to the same value as in preloadEffect.

Note

In version v2.1.x, the following method, which is not recommended by Agora, is used.

public int playEffect(int soundId, String filePath, boolean loop, double pitch, double pan, double gain);

Stop Playing an Audio Effect (stopEffect)

public int stopEffect(int soundId);

This method stops playing a specific audio effect.

Name Description
soundId ID of the audio effect. Each audio effect has a unique ID.
Return Value
  • 0: Method call succeeded.
  • <0: Method call failed.

Stop Playing all Audio Effects (stopAllEffects)

public int stopAllEffects();

This method stops playing all the audio effects.

Preload an Audio Effect (preloadEffect)

public int preloadEffect(int soundId, String filePath);

This method preloads a specific audio effect file (compressed audio file) to the memory.

Note

To ensure smooth communication, pay attention to the size of the audio effect file. Agora recommends using this method to preload the audio effect before calling joinChannel.

Name Description
soundId ID of the audio effect. Each audio effect has a unique ID.
filePath Absolute path of the audio effect file.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Release an Audio Effect (unloadEffect)

public int unloadEffect(int soundId);

This method releases a specific preloaded audio effect from the memory.

Name Description
soundId ID of the audio effect. Each audio effect has a unique ID.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Pause an Audio Effect (pauseEffect)

public int pauseEffect(int soundId);

This method pauses a specific audio effect.

Name Description
soundId The ID of the audio effect. Each audio effect has a unique ID.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Pause all Audio Effects (pauseAllEffects)

public int pauseAllEffects();

This method pauses all the audio effects.

Name Description
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Resume an Audio Effect (resumeEffect)

public int resumeEffect(int soundId);

This method resumes playing a specific audio effect.

Name Description
soundId ID of the audio effect. Each audio effect has a unique ID.
Return Value
  • 0: Method call succeeded
  • <0: Method call failed

Resume all Audio Effects (resumeAllEffects)

public int resumeAllEffects();

This method resumes all the audio effects.

Push-stream Methods (PublisherConfiguration)

Before calling the methods listed in this section:

Set Live Transcoding (setLiveTranscoding)

Note

This feature is expected to be available in Q3 2018.

public abstract int setLiveTranscoding(LiveTranscoding transcoding);

This method is used for CDN Live to set the video layout and audio for CDN Live.

Parameter Description
transcoding Configuration of LiveTranscoding.

Definition of LiveTranscoding

public int width;
public int height;
public int videoBitrate;
public int videoFramerate;
public boolean lowLatency;
public int videoGop;
public AgoraImage watermark;
public AgoraImage backgroundImage;
public AudioSampleRateType audioSampleRate;
public int audioBitrate;
public int audioChannels;
public VideoCodecProfileType videoCodecProfile;
public String userConfigExtraInfo;
public String metadata;
public int backgroundColor;
public static class TranscodingUser {
    public int uid;
    public int x;
    public int y;
    public int width;
    public int height;
    public int zOrder;
    public float alpha;
    public int audioChannel;

    public TranscodingUser() {
        alpha = 1;
    }
}

public LiveTranscoding() {
    width = 360;
    height = 640;
    videoBitrate = 400;
    videoCodecProfile = VideoCodecProfileType.HIGH;
    videoGop = 30;
    videoFramerate = 15;
    watermark = new AgoraImage();
    backgroundImage = new AgoraImage();

    lowLatency = false;
    audioSampleRate = AudioSampleRateType.TYPE_44100;
    audioBitrate = 48;
    audioChannels = 1;
    backgroundColor = 0xFF000000;
    userConfigExtraInfo = null;
    metadata = null;
}
Parameter Description
width Width of the canvas. The default value is 360.
height Height of the canvas. The default value is 640.
videoBitrate Bitrate of the output video stream set for CDN live broadcast. Default value is 400 kbit/s.
videoCodecProfile

Self-defined codec profiles:

  • VIDEO_CODEC_PROFILE_TYPE_BASELINE = 66, lowest video codec rate
  • VIDEO_CODEC_PROFILE_TYPE_MAIN = 77
  • VIDEO_CODEC_PROFILE_TYPE_HIGH = 100, highest video codec rate (default)
videoGop Interval between the I frames.
videoFramerate Frame rate of the output video stream set for CDN live broadcast. The default value is 15 fps.
watermark The Url address of the watermark image added to the CDN publishing stream. The audience of the CDN publishing stream can see the watermark. See The Definition of AgoraImage for the definition of the watermark.
backgroungImage The Url address of the background image added to the CDN publishing stream. The audience of the CDN publishing stream can see the background image. See The Definition of AgoraImage for the definition of the background image.
lowLatency

Low latency mode:

  • true: Low latency with unassured quality.
  • false: (Default)High latency with assured quality.
audioSampleRate

Audio samping rate:

  • AUDIO_SAMPLE_RATE_TYPE_32000
  • AUDIO_SAMPLE_RATE_TYPE_44100 (Default)
  • AUDIO_SAMPLE_RATE_TYPE_48000
audioBitrate Bitrate of the audio output stream in kbit/s. The default value is 48 and the highest value is 128.
audioChannel

Agora’s self-defined audio channel types. Agora recommends choosing 1 or 2. Options 3, 4, and 5 require special players:

  • 1: Mono (Default)
  • 2: Dual-sound channels
  • 3: Three-sound channels
  • 4: Four-sound channels
  • 5: Five-sound channels
backgroundColor Background color of the output video stream for CDN live broadcast. It is defined as int color = (A & 0xff) << 24 | (R & 0xff) << 16 | (G & 0xff) << 8 | (B & 0xff). Users can set the backgroundColor with ARGB ColorInt.
userConfigExtraInfo Information embedded in the SEI in the video output from the push output.

Add a Stream URL (addPublishStreamUrl)

public abstract int addPublishStreamUrl(String url, boolean transcodingEnabled)

This method is used in CDN live. It adds the URL to which the host publishes the stream.

Parameter Description
url URL to which the host publishes the stream.
transcodingEnabled

Whether to enable transcoding or not.

  • true: Enable transcoding
  • false: Disable transcoding

Note

  • Ensure that this API is called AFTER you have joined the channel.
  • This method only adds one URL each time it is called.
  • Do not add special characters such as Chinese to the url.

Remove a Stream URL (removePublishStreamUrl)

public abstract int removePublishStreamUrl(String url);

This method is used in CDN live. It removes the URL to which the host publishes the stream.

Parameter Description
url URL to which the host publishes the stream.

Note

  • This method only removes one URL each time it is called.
  • Do not add special characters such as Chinese to the url.

Add a User (addUser)

public int addUser(TranscodingUser user);

This method adds a user to the existing users.

Parameter Description
user Transcoding user object. See Definition of transcodingUser for the detailed definition of users.

Definition of transcodingUser

public static class TranscodingUser {
    public int uid;
    public int x;
    public int y;
    public int width;
    public int height;
    public int zOrder;
    public float alpha;
    public int audioChannel;

    public TranscodingUser() {
        alpha = 1;
    }
}
Parameter Description
uid User ID of the CDN live.
x Horizontal position of the top left corner of the video frame
y Vertical position of the top left corner of the video frame
width Width of the video frame
height Height of the video frame
zOrder

Layer of the video frame. Between 1 and 100:

  • 1: Default, lowest
  • 100: Highest
alpha

Transparency of the video frame. Between 0 and 1.0:

  • 0: Completely transparent
  • 1.0: Opaque
audioChannel

The audio channel of the sound. The default value is 0:

  • 0: (default) Supports dual channels at most, depending on the upstream of the broadcaster
  • 1: The audio stream of the broadcaster is in the FL audio channel. If the upstream of the broadcaster uses dual-sound channel, only the left sound channel will be used for streaming
  • 2: The audio stream of the broadcaster is in the FC audio channel. If the upstream of the broadcaster uses dual-sound channel, only the left sound channel will be used for streaming
  • 3: The audio stream of the broadcaster is in the FR audio channel. If the upstream of the broadcaster uses dual-sound channel, only the left sound channel will be used for streaming
  • 4: The audio stream of the broadcaster is in the BL audio channel. If the upstream of the broadcaster uses dual-sound channel, only the left sound channel will be used for streaming
  • 5: The audio stream of the broadcaster is in the BR audio channel. If the upstream of the broadcaster uses dual-sound channel, only the left sound channel will be used for streaming

Get the Users (getUsers)

public final ArrayList<TranscodingUser> getUsers();

This method gets all the users involved in transcoding. The UID list returned in this method is read-only, and developers should not modify the data.

Parameter Description
users All users in the CDN streaming channel. See Definition of transcodingUser for the detailed definition of users.

Set the Users in Batches (setUsers)

public void setUsers(ArrayList<TranscodingUser> users);

This method sets all the users involved in transcoding. This method replaces the original data with the new user data.

Parameter Description
users All the users involved in transcoding. See Definition of transcodingUser for the detailed definition of users.

Set the Users in Batches (setUsers)

public void setUsers(Map<TranscodingUser> users);

This method sets all the users involved in transcoding. This method replaces the original data with the new user data.

Parameter Description
users All the users involved in transcoding. See Definition of transcodingUser for the detailed definition of users.

Remove a User (removeUser)

public int removeUser(int uid);

This method removes a specified user from the channel.

Parameter Description
uid UID of the user to be removed.

Get the User Count (getUserCount)

public int getUserCount();

This method gets the user count in the CDN streaming channel.

Get the Background Color (getBackgroundColor)

public int getBackgroundColor;

This method gets the backgroung color.

Set the Background Color (setBackgroundColor)

public void setBackgroundColor(int color)

This method sets the background color to either red, green, or blue.

Parameter Description
color Desired background color

Set the Background RGB (setBackgroundColor)

public void setBackgroundColor(int red, int green, int blue)

This method sets the background color to different variations of red, green, or blue.

Parameter Description
red Red color
green Green color
blue Blue color

Get the Red Component (getRed)

Note

This method has been deprecated. Agora recommends using Get the Background Color (getBackgroundColor) to get the background color.

public int getRed();

This method gets the red component from the background color.

Get the Green Component (getGreen)

Note

This method has been deprecated. Agora recommends using Get the Background Color (getBackgroundColor) to get the background color.

public int getGreen()

This method gets the green component from the background color.

Get the Blue Component (getBlue)

Note

This method has been deprecated. Agora recommends using Get the Background Color (getBackgroundColor) to get the background color.

public int getBlue()

This method gets the blue component from the background color.

Set the Red Component (setRed)

Note

This method has been deprecated. Agora recommends using Set the Background RGB (setBackgroundColor) to set the background color.

public void setRed(int red)

This method sets the red component of the background color while keeping the other two components unchanged.

Set the Green Component (setGreen)

Note

This method has been deprecated. Agora recommends using Set the Background RGB (setBackgroundColor) to set the background color.

public void setGreen(int green)

This method sets the green component of the background color while keeping the other two components unchanged.

Set the Blue Component (setBlue)

Note

This method has been deprecated. Agora recommends using Set the Background RGB (setBackgroundColor) to set the background color.

public void setBlue(int blue)

This method sets the blue component of the background color while keeping the other two components unchanged.

Configure Push-stream (configPublisher)

Note

This method is deprecated. Although you can still use it to publish to CDN, Agora recommends using the following methods instead:

  • addPublishStreamUrl()
  • removePublishStreamUrl()
  • setLiveTranscoding()

Agora will continue improving the CDN Live feature on top of the abovementioned APIs.

public abstract int configPublisher(PublisherConfiguration config);

This method configures the push-stream settings for the engine before joining a channel for CDN Live. Agora provides a Builder class to configure the push-stream for CDN Live, for example,

PublisherConfiguration config = New PublisherConfiguration.Builder()
  .owner(true)
  .size(360, 640)
  .framerate(15)
  .bitrate(500)
  .defaultLayout(1)
  .lifecycle(STREAM_LIFE_CYCLE_BIND2CHANNEL)
  .rawStreamUrl(rtmpUrl)
  .publisherUrl(rtmpUrl)
  .injectStream(Url, width, height)
  .build();

mRtcEngine.configPublisher(config);

Note

  • Check the The Builder Class on how to call each method included for the push-stream settings configuration.
  • Set the resolution, frame rate, and bitrate the same as those of the uplink host to avoid video quality degradation.
  • Currently, this method is in beta for test and use. Contact sales@agora.io to use this method.

The Builder Class

Note

This Class has been deprecated. If you want to configure the push-stream, Agora recommends using the Set Live Transcoding (setLiveTranscoding) method.

Set the RTMP Stream Owner (owner)

public Builder owner(boolean isRoomOwner)

In the Builder Class, this method sets whether the current host is the RTMP stream owner.

Name Description
isRoomOwner
  • true: Yes (default), push-stream configuration.
  • false: No push-stream configuration.

Set the Stream Resolution (size)

public Builder size(int width, int height)

In the Builder Class, this method sets the resolution of the output data stream set for CDN Live.

Name Description
width Width of the output data stream set for CDN Live. 360 is the default value.
height Height of the output data stream set for CDN Live. 640 is the default value.

Set the Stream Frame Rate (frameRate)

public Builder frameRate(int framerate)

In the Builder class, this method sets the frame rate of the output data stream set for CDN Live.

Name Description
framerate Frame rate of the output data stream set for CDN Live. 15 fps is the default value.

Set the Stream Bitrate (bitRate)

public Builder bitRate(int bitrate)

In the Builder class, this method sets the bitrate of the output data stream set for CDN Live.

Name Description
bitrate Bitrate of the output data stream set for CDN Live. 500 kbit/s is the default value.

Set the Default Layout (defaultLayout)

public Builder defaultLayout(int layoutStyle)

In the Builder class, this method sets the Default Layout if you do not use Flexible Adjustment.

Name Description
layoutStyle
  • 0: Tile horizontally
  • 1: Layered windows
  • 2: Tile vertically

Set the Publishing URL (publishUrl)

public Builder publishUrl(String url)

In the Builder class, this method configures the push-stream address for the picture-in-picture layouts.

Name Description
url Configures the push-stream address for the picture-in-picture layouts. The default value is NULL.

Set the Raw Stream URL (rawStreamUrl)

public Builder rawStreamUrl(String url)

In the Builder class, this method sets the push-stream address of the original stream which does not require picture-blending.

Name Description
url Push-stream address of the original stream. The default value is NULL.

Set the Inject Stream (injectStream)

public Builder injectStream(String url, int width, int height)

In the Builder class, this method injects a stream to the current channel.

Name Description
url Address of the stream to be injected to the channel.
width Width of the stream. N/A, set as 0 for now.
height Height of the stream. N/A, set as 0 for now.

Add Extra Information (extraInfo)

public Builder extraInfo(String optionalInfo)
Name Description
optionalInfo Reserved Field. The default value is NULL.

Add an Inject Video Stream

Add an Inject Stream URL (addInjectStreamUrl)

public abstract int addInjectStreamUrl(String url, LiveInjectStreamConfig config)

This method adds a voice or video stream into an ongoing broadcast.

If successful, you can find the stream in the channel and the uid of the stream is 666.

Name Description
url URL address to be added into the ongoing live broadcast. You can use the RTMP, HLS, and FLV protocols.
config Definition of the added voice or video stream

Definition of LiveInjectStreamConfig

public int width;
public int height;
public int videoGop;
public int videoFramerate;
public int videoBitrate;
public AudioSampleRateType audioSampleRate;
public int audioBitrate;
public int audioChannels;

public LiveInjectStreamConfig() {
    width = 0;
    height = 0;
    videoGop = 30;
    videoFramerate = 15;
    videoBitrate = 400;
    audioSampleRate = AudioSampleRateType.TYPE_44100;
    audioBitrate = 48;
    audioChannels = 1;
}
Name Description
width Width of the stream to be added into the broadcast. The default value is 0; same width as the original stream.
height Height of the stream to be added into the broadcast. The default value is 0; same height as the original stream.
videoGop Video GOP of the stream to be added into the broadcast. The default value is 30.
videoFramerate Video frame rate of the stream to be added into the broadcast. The default value is 15 fps.
videoBitrate Video bitrate of the stream to be added into the broadcast. The default value is 400 kbit/s.
audioSampleRate Audio sampling rate of the stream to be added into the broadcast. The default value is 44100.
audioBitrate Audio bitrate of the stream to be added into the broadcast. The default value is 48.
audioChannels Audio channels to be added into the broadcast. The default value is 1.

Remove an Inject Stream URL (removeInjectStreamUrl)

public abstract int removeInjectStreamUrl(String url)

This method removes an injected stream URL.

Name Description
url URL address of the added stream to be removed.

Callback Methods (IRtcEngineEventHandler)

Developer Suite: io.agora.rtc

The SDK uses the IRtcEngineEventHandler interface class to send callback event notifications to the application, and the application inherits the methods of this interface class to retrieve these event notifications.

All methods in this interface class have their (empty) default implementations, and the application can inherit only some of the required events instead of all of them. In the callback methods, the application should avoid time-consuming tasks or call blocking APIs (such as SendMessage), otherwise, the SDK may not work properly.

Join Channel Callback (onJoinChannelSuccess)

public void onJoinChannelSuccess(String channel,
                                 int uid,
                                 int elapsed);

This callback indicates that the user has successfully joined the specified channel with the channel ID and user ID assigned. The channel ID is assigned based on the channel name specified in join() API. If the User ID is not specified when join() is called, the server assigns one automatically.

Name Description
channel The channel name.
uid

User ID.

If the uid is specified in the joinChannel method, it returns the specified ID; if not, it returns an ID that is automatically assigned by the Agora server.

elapsed Time elapsed (ms) from calling joinChannel until this event occurs.

Rejoin Channel Callback (onRejoinChannelSuccess)

public void onRejoinChannelSuccess(String channel,
                                   int uid,
                                   int elapsed);

When the client loses connection with the server because of network problems, the SDK automatically attempts to reconnect, and then triggers this callback method upon reconnection.

Name Description
channel Channel name.
uid User ID.
elapsed Time elapsed (ms) from calling joinChannel until this event occurs.

Warning Occurred Callback (onWarning)

public void onWarning(int warn);

This callback indicates that some warning occurred during SDK runtime of the SDK. In most cases, the application can ignore the warnings reported by the SDK because the SDK can usually fix the issue and resume running. For instance, the SDK may report an ERR_OPEN_CHANNEL_TIMEOUT warning upon disconnection with the server and attempts to reconnect.

Name Description
warn Warning Code. For more information, see Error Codes and Warning Codes

Error Occurred Callback (onError)

public void onError(int err);

This callback indicates that a network or media error occurred during SDK runtime.

In most cases, reporting an error means that the SDK cannot fix the issue and resume running, and therefore requires actions from the application or simply informs the user on the issue. For instance, the SDK reports an ERR_START_CALL error when failing to initialize a call. In this case, the application informs the user that the call initialization failed and calls the leaveChannel method to exit the channel.

Name Description
err

Error codes:

  • ERR_INVALID_VENDOR_KEY(101): Invalid App ID.
  • ERR_INVALID_CHANNEL_NAME(102): Invalid channel name.
  • ERR_LOOKUP_CHANNEL_REJECTED(105): Failed to look up the channel, because the server rejected the request.
  • ERR_OPEN_CHANNEL_REJECTED(107): Failed to join the channel, because the media server rejected the request.
  • ERR_LOAD_MEDIA_ENGINE(1001): Failed to load the media engine.
  • ERR_START_CALL(1002): Failed to turn on the local audio or video devices and thus failed to start a call.
  • ERR_START_CAMERA(1003): Failed to turn on the local camera.

API Call Executed Callback (onApiCallExecuted)

public void onApiCallExecuted(int error, String api, String result)

This callback is triggered when the API has been executed.

Name Description
error The error code that the SDK returns when the method call fails. If the SDK returns o, then the method has been called successfully.
api The API that the SDK executes
result The result of calling the API

Leave a Channel Callback (onLeaveChannel)

public void onLeaveChannel(RtcStats stats);

When the application calls the leaveChannel() method, the SDK uses this callback to notify the application that the user has successfully left the channel. With this callback function, the application retrieves information such as the call duration and the statistics of data received/transmitted by the SDK.Audio Quality Callback (onAudioQuality)

Definition of RtcStats

public static class RtcStats {
    public int totalDuration; // in seconds
    public int txBytes;
    public int rxBytes;
    public int txKBitRate;
    public int rxKBitRate;
    public int txAudioKBitRate;
    public int rxAudioKBitRate;
    public int txVideoKBitRate;
    public int rxVideoKBitRate;
    public int lastmileDelay;
    public int users;
    public double cpuTotalUsage;
    public double cpuAppUsage;
}
Name Description
stats

Statistics of the call:

  • totalDuration: Call duration in seconds, represented by an aggregate value.
  • txBytes: Total number of bytes transmitted, represented by an aggregate value.
  • rxBytes: Total number of bytes received, represented by an aggregate value.
  • txKBitRate: Transmission bitrate in kbit/s, represented by an instantaneous value.
  • rxKBitRate: Receive bitrate in kbit/s, represented by an instantaneous value.
  • txAudioKBitrate: Audio transmission bitrate in kbit/s, represented by an instantaneous value.
  • rxAudioKBitRate: Audio receive bitrate in kbit/s, represented by an instantaneous value.
  • txVideoKBitRate: Video transmission bitrate in kbit/s, represented by an instantaneous value.
  • rxVideoKBitRate: Video receive bitrate in kbit/s, represented by an instantaneous value.
  • lastmileDelay: The time delay in milliseconds from the Client to the VO Server
  • users: The instant number of users in the channel when the user leaves the channel.
  • puTotalUsage: System CPU usage (%).
  • cpuAppUsage: Application CPU usage (%).

Audio Route Changed Callback (onAudioRouteChanged)

public void onAudioRouteChanged(int routing);

The SDK notifies the application that the audio route is changed after calling setEnableSpeakerphone successfully.

It notifies the current audio route is switched to an earpiece, a speakerphone, headset or Bluetooth device. The definition of routing is listed as follows:

public static final int AUDIO_ROUTE_DEFAULT = -1;
public static final int AUDIO_ROUTE_HEADSET = 0;
public static final int AUDIO_ROUTE_EARPIECE = 1;
public static final int AUDIO_ROUTE_HEADSETNOMIC = 2;
public static final int AUDIO_ROUTE_SPEAKERPHONE = 3;
public static final int AUDIO_ROUTE_LOUDSPEAKER = 4;
public static final int AUDIO_ROUTE_HEADSETBLUETOOTH = 5;

Remote Audio Transport Statistics Callback (onRemoteAudioTransportStats)

public void onRemoteAudioTransportStats(int uid,
                                        int delay,
                                        int lost,
                                        int rxKBitRate);

This callback is triggered every two seconds after the user receives the audio data packet sent from a remote user.

Name Description
uid User ID of the remote user whose audio data packet has been received.
delay Time delay (ms) from the remote user to the local client.
lost Packet loss rate (%).
rxKBitRate Received audio bitrate (kbit/s) of the packet from the remote user.

Remote Video Transport Statistics Callback (onRemoteVideoTransportStats)

public void onRemoteVideoTransportStats(int uid,
                                        int delay,
                                        int lost,
                                        int rxKBitRate);

This callback is triggered every two seconds after the user receives the video data packet sent from a remote user.

Name Description
uid User ID of the remote user whose video packet has been received.
delay Time delay (ms) from the remote user to the local client.
lost Packet loss rate (%).
rxKBitRate Received video bitrate (kbit/s) of the packet from the remote user.

Audio Quality Callback (onAudioQuality)

public void onAudioQuality(int uid,
                           int quality,
                           short delay,
                           short lost);

During a call, this callback is triggered once every two seconds to report on the audio quality of the current call. By default it is enabled.

Name Description
uid User ID of the speaker
quality

Rating of the audio quality:

  • QUALITY_UNKNOWN(0): The network quality is unknown.
  • QUALITY_EXCELLENT(1): The network quality is excellent.
  • QUALITY_GOOD(2): The network quality is quite good, but the bitrate may be slightly lower than excellent.
  • QUALITY_POOR(3): Users can feel the communication slightly impaired.
  • QUALITY_BAD(4): Users can communicate only not very smoothly.
  • QUALITY_VBAD(5): The network is so bad that users can hardly communicate.
  • QUALITY_DOWN(6): Users cannot communicate at all.
delay Time delay in milliseconds.
lost The packet loss rate(%).

Audio Volume Indication Callback (onAudioVolumeIndication)

public void onAudioVolumeIndication(AudioVolumeInfo[] speakers, int totalVolume);

This callback indicates who is speaking and the speaker’s volume.

By default, the indication is disabled. If needed, use the enableAudioVolumeIndication() method to configure it.

Name Description
speakers

The speakers (array). Each speaker ():

  • uid: User ID of the speaker. The uid of the local user is 0.
  • volume: The volume of the speaker that ranges from 0 (lowest volume) to 255 (highest volume).
totalVolume Total volume after audio mixing that ranges from 0 (lowest volume) to 255 (highest volume).

In the returned speakers array:

  • If the uid is 0, that is, the local user is the speaker, the returned volume is the same as totalVolumn.
  • If the uid is not 0 and the volume is 0, it indicates that the user specified by the uid did not speak.
  • If a uid is contained in the previous speakers array but not in the present one, it indicates that the user specified by the uid did not speak.

Other User Joined the Channel Callback (onUserJoined)

public void onUserJoined(int uid, int elapsed);

This callback method notifies the application the broadcaster has joined the channel. If the broadcaster is already in the channel when the application joins the channel, the SDK also reports to the application on the broadcaster that is already in the channel.

Note

In the live broadcast scenario:

  • The broadcaster can receive the callback when another broadcaster joins the channel.
  • All the audience in the channel can receive the callback when the new broadcaster joins the channel.
  • When a web application joins the channel, this callback is triggered as long as the web application publishes streams.
Name Description
uid User ID.
elapsed Time delay (ms) from calling joinChannel until this callback is triggered.

Other User is Offline Callback (onUserOffline)

public void onUserOffline(int uid, int reason);

This callback notifies the application that the broadcaster has left the channel or gone offline.

The SDK reads the timeout data to determine if a user has left the channel (or has gone offline). If no data package is received from the user in 15 seconds, the SDK takes assumes the user is offline. A poor network connection may lead to false detections; therefore, use signaling for reliable offline detection.

Name Description
uid User ID
reason

Reasons for the user going offline:

  • USER_OFFLINE_QUIT(0): User has quit the call.
  • USER_OFFLINE_DROPPED(1): The SDK timed out and the user dropped offline because it has not received any data package for a period of time.
  • USER_OFFLINE_BECOME_AUDIENCE: Triggered when the client role has changed from the broadcaster to the audience.

Get the Recorded Audio Frame (onRecordAudioFrame)

virtual  bool  onRecordAudioFrame(AudioFrame&audioFrame)  override {
return  true;
}

This method gets the recorded audio frame.

Name Description
AudioFrame
  • samples: Number of samples in the frame.
  • bytesPerSample: Number of bytes per sample: 2 for PCM 16.
  • channels: Number of channels (stereo data is interleaved).
  • samplesPerSec: Sampling rate.
  • buffer: Data buffer.
  • renderTimeMs: Timestamp to render the audio stream. This timestamp is used to synchronize the audio stream renderer while rendering the audio streams. [5]

Footnotes

[5]This timestamp is for rendering the audio stream, and is not the timestamp of capturing the audio stream.
struct  AudioFrame  {
AUDIO_FRAME_TYPE  type;
int  samples;
int  bytesPerSample;
int  channels;
int  samplesPerSec;
void*  buffer;
int64_t renderTimeMs;
}

Get the Playback Audio Frame (onPlaybackAudioFrame)

virtual  bool  onPlaybackAudioFrame(AudioFrame&  audioFrame)  override {
return  true;
}

This method gets the playback audio frame. The parameter description is the same as onRecordAudioFrame.

Get the Playback Audio Frame of a Specific User (onPlaybackAudioFrameBeforeMixing)

virtual  bool  onPlaybackAudioFrameBeforeMixing(unsigned  int  uid,  AudioFrame&  audioFrame)  override
{
return  true;
}

This method gets the playback audio frame of a specific user. The parameter description is the same as onRecordAudioFrame.

Name Description
uid UID of a specified user.

Get the Mixed Recording and Playback Audio Data (onMixedAudioFrame)

virtual bool onMixedAudioFrame(AudioFrame& audioFrame) override
{
return true;
}

This method gets the mixed recording and playback audio data. It only returns the single-channel data.

Name Description
AudioFrame
  • samples: Number of samples in the frame.
  • bytesPerSample: Number of bytes per sample: 2 for PCM 16.
  • channels: Number of channels (stereo data is interleaved).
  • samplesPerSec: Sampling rate
  • buffer: Data buffer
  • renderTimeMs: Timestamp to render the audio stream. Use this timestamp to synchronize the audio stream renderer while rendering the audio streams. [6]

Get the Captured Video Frame (onCaptureVideoFrame)

virtual bool onCaptureVideoFrame(VideoFrame&videoFrame);

This method gets the camera’s captured image.

Name Description
VideoFrame
  • yBuffer: Pointer to the Y buffer pointer in the YUV data.
  • uBuffer: Pointer to the U buffer pointer in the YUV data.
  • vBuffer: Pointer to the V buffer pointer in the YUV data.
  • width: Video pixel width.
  • height: Video pixel height.
  • yStride: Line span of the Y buffer in the YUV data.
  • uStride: Line span of the U buffer in the YUV data.
  • vStride: Line span of the V buffer in the YUV data.
  • rotation: Set the rotation of this frame before rendering the video; it supports 0, 90, 180, and 270 degrees clockwise.
  • renderTimeMs: Timestamp to render the video stream. Use this timestamp to synchronize the video stream renderer while rendering the video streams. [6]
Return Value None

Footnotes

[6](1, 2) This timestamp is for rendering the video stream, not for capturing the video stream.

The video data format is YUV420. The buffer provides a pointer to a pointer. However, the pointer of the buffer cannot be modified. Only the contents of the buffer can be modified.

struct  VideoFrame  {
VIDEO_FRAME_TYPE  type;
int  width;
int  height;
int  yStride;
int  uStride;
int  vStride;
void*  yBuffer;
void*  uBuffer;
void*  vBuffer;
int rotation; // rotation of this frame (0, 90, 180, 270)
int64_t renderTimeMs;
};

Get the Video Frame of Other User (onRenderVideoFrame)

virtual bool onRenderVideoFrame(unsigned int uid, VideoFrame& videoFrame)

This method processes the received image of the other user (post-processing).

Name Description
uid UID of a specified user.
VideoFrame
  • yBuffer: Pointer to the Y buffer pointer in the YUV data.
  • uBuffer: Pointer to the U buffer pointer in the YUV data.
  • vBuffer: Pointer to the V buffer pointer in the YUV data.
  • width: Video pixel width.
  • height: Video pixel height.
  • yStride: Line span of the Y buffer in the YUV data.
  • uStride: Line span of the U buffer in the YUV data.
  • vStride: Line span of the V buffer in the YUV data.
Return Value None
struct  VideoFrame  {
VIDEO_FRAME_TYPE  type;
int  width;
int  height;
int  yStride;
int  uStride;
int  vStride;
void*  yBuffer;
void*  uBuffer;
void*  vBuffer;
};

Other User Muted the Audio Callback (onUserMuteAudio)

public void onUserMuteAudio(int uid, boolean muted);

This callback indicates that some other user has muted/unmuted his/her audio stream.

Note

Currently, this callback returns invalid when the number of broadcasters in a channel exceeds 20, which will be improved in the future.

Name Description
uid User ID.
muted
  • true: User has muted his/her audio.
  • false: User has unmuted his/her audio.

RtcEngine Statistics Callback (onRtcStats)

public void onRtcStats(RtcStats stats);

The SDK updates the application on the statistics of the RtcEngine runtime status once every two seconds.

Name Description
stats Refer to the descriptions in Leave a Channel Callback (onLeaveChannel).

Network Quality Callback (onLastmileQuality)

public void onLastmileQuality(int quality);

This callback reports the network quality of the local user. It is triggered once every 2 seconds after you have called enableLastmileTest().

Name Description
quality

Quality of the last mile network:

  • QUALITY_UNKNOWN(0): The network quality is unknown.
  • QUALITY_EXCELLENT(1): The network quality is excellent.
  • QUALITY_GOOD(2): The network quality is quite good, but the bitrate may be slightly lower than excellent.
  • QUALITY_POOR(3): Users can feel the communication slightly impaired.
  • QUALITY_BAD(4): Users can communicate only not very smoothly.
  • QUALITY_VBAD(5): The network is so bad that users can hardly communicate.
  • QUALITY_DOWN(6): Users cannot communicate at all.

Channel Network Quality Callback (onNetworkQuality)

public void onNetworkQuality(int uid, int txQuality, int rxQuality);

This callback is triggered every 2 seconds to update the application on the network quality of each user in a communication or live broadcast channel.

Name Description
uid User ID. The network quality of the user with this UID will be reported. If uid is 0, it reports the local network quality.
txQuality

The transmission quality of the user:

  • QUALITY_UNKNOWN(0): The network quality is unknown.
  • QUALITY_EXCELLENT(1): The network quality is excellent.
  • QUALITY_GOOD(2): The network quality is quite good, but the bitrate may be slightly lower than excellent.
  • QUALITY_POOR(3): Users can feel the communication slightly impaired.
  • QUALITY_BAD(4): Users can communicate only not very smoothly.
  • QUALITY_VBAD(5): The network is so bad that users can hardly communicate.
  • QUALITY_DOWN(6): Users cannot communicate at all.
rxQuality

The receiving quality of the user:

  • QUALITY_UNKNOWN(0): The network quality is unknown.
  • QUALITY_EXCELLENT(1): The network quality is excellent.
  • QUALITY_GOOD(2): The network quality is quite good, but the bitrate may be slightly lower than excellent.
  • QUALITY_POOR(3): Users can feel the communication slightly impaired.
  • QUALITY_BAD(4): Users can communicate only not very smoothly.
  • QUALITY_VBAD(5): The network is so bad that users can hardly communicate.
  • QUALITY_DOWN(6): Users cannot communicate at all.

Remote Video State Changed Callback (onRemoteVideoStateChanged)

public void onRemoteVideoStateChanged(int uid, int state)

This callback indicates that the state of the remote video has changed.

Name Description
uid ID of the user whose remote video state has changed.
state

The state of the remote video:

  • 1: The remote video is normal.
  • 2: The remote video is frozen.

Connection Interrupted Callback (onConnectionInterrupted)

public void onConnectionInterrupted();

This method indicates that the SDK has lost connection with the server.

This method is triggered by a connection lost, while the onConnectionLost method (stated below) is triggered when the SDK attempts to reconnect after losing connection. Once the connection is lost, and if the application does not call leaveChannel, the SDK automatically tries to reconnect repeatedly.

Connection Lost Callback (onConnectionLost)

public void onConnectionLost();

This callback indicates that the SDK has lost connection with the network, and it has remained unconnected for a period of time (10 seconds by default) despite that it attempts to reconnect. It is also triggered when the SDK fails to join the channel 10 seconds after it calls joinChannel. The SDK will keep trying to reconnect after this callback is triggered. Upon reconnection, an onRejoinChannelSuccess callback will then be triggered.

Connection Banned Callback (onConnectionBanned)

public void onConnectionBanned();

This callback is triggered when your connection is banned by the Agora Server.

First Local Video Frame Displayed Callback (onFirstLocalVideoFrame)

public void onFirstLocalVideoFrame(int width, int height, int elapsed);

This callback is triggered when the first local video frame is displayed on the screen.

Name Description
width Width (pixels) of the video stream.
height Height (pixels) of the video stream.
elapsed Time elapsed (ms) from calling joinChannel until this callback is triggered.

First Remote Audio Frame Received Callback (onFirstRemoteAudioFrame)

public void onFirstRemoteAudioFrame(int uid, int elapsed)

This callback method is triggered when the first remote audio frame is received.

Name Description
uid The UID of the remote user whose audio frame is received.
elapsed Time elapsed (ms) from calling joinChannel until this callback is triggered.

First Remote Video Frame Displayed Callback (onFirstRemoteVideoFrame)

public void onFirstRemoteVideoFrame(int uid, int width, int height, int elapsed);

This method is triggered when the first frame of the remote video appears in the user’s video window. The application can retrieve the data of time elapsed from user joining the channel until the first video frame is displayed.

Name Description
uid User ID of the user whose video streams are received.
width Width (pixels) of the video stream.
height Height (pixels) of the video stream.
elapsed Time elapsed (ms) from calling joinChannel until this callback is triggered.

First Remote Video Frame Received and Decoded Callback (onFirstRemoteVideoDecoded)

public void onFirstRemoteVideoDecoded(int uid, int width, int height, int elapsed);

This callback is triggered upon receiving and successfully decoding the first frame of the remote video. The application can configure the user view settings in this callback.

Name Description
uid User ID of the user whose video streams are received.
width Width (pixels) of the video stream.
height Height (pixels) of the video stream.
elapsed Time elapsed (ms) from calling joinChannel until this callback is triggered.

Other User Paused/Resumed Video Callback (onUserMuteVideo)

public void onUserMuteVideo(int uid, boolean muted);

This callback indicates that some other user has paused/resumed his/her video stream.

Name Description
uid User ID
muted
  • true: User has paused his/her video.
  • false: User has resumed his/her video.

Other User Enabled/Disabled Video Callback (onUserEnableVideo)

public void onUserEnableVideo(int uid, boolean enabled);

This callback indicates that some other user has enabled/disabled the video function. Disabling the video function means that the user can only use voice call, can neither show/send their own videos nor receive or display videos from other people.

Name Description
uid User ID
enabled
  • true: The user has enabled the video function and can enter a video call or video live broadcast.
  • false: The user has disabled the video function and can only enter a voice session where the user can neither send or receive any video streams.

Other User Enabled/Disabled Local Video Callback (onUserEnableLocalVideo)

public void onUserEnableLocalVideo(int uid, boolean enabled)

This callback indicates that some other user has enabled/disabled the local video function.

Name Description
uid User ID
enabled
  • true: The user has enabled the local video function. Other users in the channel can see the video of this user.
  • false: the user has disabled the local video function. Other users in the channel can no longer receive the video stream from this user, while this user can still receive the video streams from other users.

Local Publish Fallback to Audio Only Callback (onLocalPublishFallbackToAudioOnly)

public void onLocalPublishFallbackToAudioOnly(boolean isFallbackOrRecover);

If you call the Set the Local Publish Fallback Option (setLocalPublishFallbackOption) API and set option as STREAM_FALLBACK_OPTION_AUDIO_ONLY, this callback will be triggered when the local publish stream falls back to audio-only mode due to poor uplink conditions, or when the audio stream switches back to the video when the uplink condition improves.

Note

Once the local publish stream falls back to audio-only, the remote app will receive the onUserMuteVideo callback.

Name Description
isFallbackOrRecover

The local publish stream fell back to audio-only or switched back to video:

  • true: The local publish stream fell back to audio-only due to poor network conditions.
  • false: The local publish stream switched back to the video when the network condition improved.

Remote Subscribe Fallback To Audio Only Callback (onRemoteSubscribeFallbackToAudioOnly)

public void onRemoteSubscribeFallbackToAudioOnly(int uid, boolean isFallbackOrRecove);

If you call the Set the Remote Subscribe Fallback Option (setRemoteSubscribeFallbackOption) API and set option as STREAM_FALLBACK_OPTION_AUDIO_ONLY, this callback will be triggered when the remote subscribe stream falls back to audio-only due to poor downlink conditions, or when the audio stream switches back to the video when the downlink network conditions improve.

Note

Once the remote subscribe stream is switched to the low stream due to poor network conditions, you can monitor the stream switch between a high and low stream in the remoteVideoStats callback.

Name Description
uid The UID of the remote user.
isFallbackOrRecover

Remote subscribe stream fell back to audio only or recovered to video:

  • true: The remote subscribe stream fell back to audio-only due to poor network conditions.
  • false: The remote subscribe stream switched back to the video stream when the network conditions improved.

Stream Injected Status Callback (onStreamInjectedStatus)

public void onStreamInjectedStatus(String url, int uid, int status)

This callback indicates the status of the injected stream.

Name Description
url URL address added into the broadcast.
uid User ID.
status

Status of the injected stream:

  • public final static int INJECT_STREAM_STATUS_START_SUCCESS = 0
  • public final static int INJECT_STREAM_STATUS_START_ALREADY_EXIST = 1
  • public final static int INJECT_STREAM_STATUS_START_UNAUTHORIZED = 2
  • public final static int INJECT_STREAM_STATUS_START_TIMEDOUT = 3
  • public final static int INJECT_STREAM_STATUS_START_FAILED = 4
  • public final static int INJECT_STREAM_STATUS_STOP_SUCCESS = 5
  • public final static int INJECT_STREAM_STATUS_STOP_NOT_FOUND = 6
  • public final static int INJECT_STREAM_STATUS_STOP_UNAUTHORIZED = 7
  • public final static int INJECT_STREAM_STATUS_STOP_TIMEDOUT = 8
  • public final static int INJECT_STREAM_STATUS_STOP_FAILED = 9
  • public final static int INJECT_STREAM_STATUS_BROKEN = 10

Local Video Statistics Callback (onLocalVideoStats)

public void onLocalVideoStats(LocalVideoStats stats) {
};

The SDK updates the application on the statistics of uploading local video streams once every two seconds.

Definition of LocalVideoStats

public static class LocalVideoStats {
    public int sentBitrate;
    public int sentFrameRate;
}
Name Description
stats

The statistics of the local video, including:

  • sentBitrate: Data sending bitrate (kbit/s) since last count.
  • sentFrameRate: Data sending frame rate (fps) since last count.

Remote Video Statistics Callback (onRemoteVideoStats)

public void onRemoteVideoStats(RemoteVideoStats stats) {
};

The SDK updates the application on the statistics of receiving remote video streams for each host once every 2 seconds. If there are muptiple remote hosts, this callback is triggered for multiple times every 2 seconds.

Definition of RemoteVideoStats

public static class RemoteVideoStats {
       public int uid;
       public int delay;
       public int width;
       public int height;
       public int receivedBitrate;
       public int receivedFrameRate;
       public int rxStreamType;
 }
Name Description
stats

The statistics of the remote video, including:

  • uid: User ID of the user whose video streams are received.
  • delay: Time delay (ms).
  • width: Width of the remote video.
  • height: Height of the remote video.
  • receivedBitrate: Data receiving bitrate (kbit/s).
  • receivedFrameRate: Data receiving frame rate (fps).
  • rxStreamType: High Video Stream or Low Video Stream.

Camera Ready Callback (onCameraReady)

public void onCameraReady();

This callback indicates that the camera is turned on and ready to capture video. If the camera fails to turn on, handle the error in the onError() method.

Video Stopped Callback (onVideoStopped)

public void onVideoStopped();

This callback indicates that the video has stopped. The application can use this callback to change the configuration of the view (for example, displaying other pictures on the view) after the video stops.

Data Stream Received Callback (onStreamMessage)

public void onStreamMessage(int uid, int streamId, byte[] data);

This callback indicates that the local user has received the data stream from the other user within five seconds.

Name Description
uid User ID
streamId Stream ID
data The data received by the recipients.

Data Stream Sent Failure Callback (onStreamMessageError)

public void onStreamMessageError(int uid, int streamId, int error, int missed, int cached);

This callback indicates that the local user has not received the data stream from the other user within five seconds.

Name Description
uid User ID
streamId Stream ID
error
  • ERR_OK = 0, No error
  • ERR_NOT_IN_CHANNEL=113, the user is not in a channel
  • ERR_BITRATE_LIMIT=115, limited bitrate

For more error code descriptions, see Error Codes and Warning Codes

missed The number of lost messages
cached The number of incoming cached messages when the data stream is interrupted.

Token Privilege Will Expire Callback (onTokenPrivilegeWillExpire)

public void onTokenPrivilegeWillExpire(String token);

If the Token you specified when calling joinChannel expires, you will become offline. This callback is triggered 30 seconds before the Token expires to remind the app to renew the Token. Upon receiving this callback, the user needs to generate a new Token on your server and call renewToken to pass the new Token on to the SDK.

Name Description
token The Token that will expire in 30 seconds.
Return Value
  • 0: Method call succeeded.
  • < 0: Method call failed.

Active Speaker Detected Callback (ActiveSpeaker)

public void onActiveSpeaker(int uid);

If you used the enableAudioVolumeIndication API, this callback is triggered when the volume detecting unit detects an active speaker in the channel, and returns the uid of the active speaker.

Name Description
uid The UID of the active speaker. By default, 0 is the local user. You can also add relative functions on your app, for example, the active speaker, once detected, will have his/her head portrait zoomed in.

Note

  • You need to call enableAudioVolumeIndication to receive this callback.
  • The active speaker has the uid of the speaker who speaks at the highest volume during a certain period of time.

Token Expired Callback (onRequestToken)

public void onRequestToken();

If a Token is specified when calling joinChannel, and due to Token will be expired after a certain amount of time, and if SDK lost the connection with the Agora server out of network issues, it might need a new Token to reconnect to the server.

This callback notifies the application of generating a new Token, and calling renewtoken to specify the newly generated Token.

This function was previously provided in the callback report of onError(): ERR_TOKEN_EXPIRED(109), ERR_INVALID_TOKEN(110). Starting from v1.7.3, the old method is still working, but it is recommended for you to put the related logic in this callback.

Audio Mixing File Playback Finished Callback (onAudioMixingFinished)

public void onAudioMixingFinished();

This callback is triggered when the audio mixing file playback is finished after calling startAudioMixing. If you failed to execute the startAudioMixing method, it returns the error code in the onError callback.

Local Audio Effect Playback Finished Callback (onAudioEffectFinished)

public void onAudioEffectFinished(int soundId)

This callback is triggered when the local audio effect playback is finished.

Name Description
soundId ID of the audio effect. Each audio effect has a unique ID.

User Role Changed Callback (onClientRoleChanged)

public void onClientRoleChanged(int oldRole, int newRole);

This callback is triggered when the user role is switched, for example, from a host to an audience or vice versa.

Name Description
oldRole Role that you switched from.
newRole Role that you switched to.

Camera Focus Area Changed (onCameraFocusAreaChanged)

public void onCameraFocusAreaChanged (Rect rect){};

This callback is triggered when the camera focus area has changed.
Name Description
rect The rectangle area in the camera zoom that specifies the focus area

Video Size Changed Callback (onVideoSizeChanged)

public void onVideoSizeChanged(int uid, int width, int height, int rotation)

This callback indicates that the size of the video frame has changed.

Parameter Description
uid User ID.
width Width (pixels) of the video frame.
height Height (pixels) of the video frame.
rotation New rotational angle of the video. Can be 0, 90, 180, or 270. It is set as 0 by default.

Publishing Succeeded Callback (onStreamUrlPublished)

public void onStreamUrlPublished(String url)

This callback notifies the host that publishing to a specific URL has succeeded.

Parameter Description
url URL to which the host publishes the stream.

Unpublished Callback (onStreamUrlUnpublished)

public void onStreamUrlUnpublished(String url)

This callback notifies the publisher that the unpublish action has succeeded.

Parameter Description
url URL to which the host publishes the stream.

Transcoding was Updated (onTranscodingUpdated)

public void onTranscodingUpdated()

This callback notifies the host that the transcoding has been successfully updated.

Is this page helpful?