Release Notes

v4.0.0-beta.2

v4.0.0-beta.2 was released on June 21, 2022.

Compatibility changes

Video snapshot

This release deleted the redundant channel parameter in the takeSnapshot method and the snapshotTaken callback.

New features

1. Media Player
This release adds the openWithMediaSource method and the AgoraRtcMediaPlayerCacheManagerProtocol and introduces real-time caching and autoplay functions for Media Player.

• Real-time caching: When you call the openWithMediaSource method to open an online media file, you can choose whether to cache part of the file through the enableCache parameter in the AgoraMediaSource. If you enable this function, the Media Player caches part of the media file being played on your local device for you to enjoy a better viewing experience. You can also get the statistics about the cached media file through AgoraMediaPlayerCacheStatistics and manage the storage space for cached media files through the methods under the AgoraRtcMediaPlayerCacheManagerProtocol.
• Autoplay: When you call the openWithMediaSource method to open a media file, you can choose whether to enable autoplay once the media file is opened through the autoplay parameter in AgoraMediaSource.

This release deprecates the openWithCustomSourceAtStartPos method; Agora recommends you using the openWithMediaSource method instead.

2. In-call monitoring
To improve the experience of the local user when receiving a remote audio stream, this release adds qualityChangedReason to AgoraRtcRemoteAudioStats and the AgoraExperiencePoorReason enum to report the possible reason for a poor audio experience during a call.

3. Video event notification management
This release adds a videoSizeChangedOfUid callback, which is triggered when the size and rotation information of the local or remote video changes, and reports the changed video information.

4. Raw video data
This release adds the getVideoFormatPreference callback, where you can set the format of the original video data you want to get, including the original video format or other video formats.

5. Raw audio data of in-ear monitoring
This release adds an onEarMonitoringAudioFrame callback to enable you to get the raw audio data of the on-ear monitoring after you register the audio frame observer. To set the desired audio data format, such as sample rate, number of channels, you can call the setEarMonitoringAudioFrameParametersWithSampleRate method, which is also added in this release.

6. Extensions
This release adds the getExtensionProperty method, where you can get the properties of the extension.

API changes

Added

• getExtensionProperty

• setEarMonitoringAudioFrameParametersWithSampleRate

• onEarMonitoringAudioFrame

• openWithMediaSource

• createMediaPlayerCacheManager

• AgoraRtcMediaPlayerCacheManagerProtocol：

  • sharedInstance
  • removeAllCaches
  • removeOldCache
  • removeCacheByUri
  • setCacheDir
  • setMaxCacheFileCount
  • setMaxCacheFileSize
  • enableAutoRemoveCache
  • cacheDir
  • maxCacheFileCount
  • maxCacheFileSize
  • cacheFileCount

• AgoraMediaSource

• cacheStatistics in AgoraMediaPlayerUpdatedInfo

• AgoraMediaPlayerCacheStatistics

• AgoraMediaPlayerEvent：

  • AgoraMediaPlayerEventReachCacheFileMaxCount
  • AgoraMediaPlayerEventReachCacheFileMaxSize

• qualityChangedReason in AgoraRtcRemoteAudioStats

• AgoraExperiencePoorReason

• getVideoFormatPreference

• AgoraVideoFormat：

  • AgoraVideoFormatDefault
  • AgoraVideoFormatNV21
  • AgoraVideoFormatRGBA
  • AgoraVideoFormatI422
  • AgoraVideoFormatCVPixelNV12
  • AgoraVideoFormatCVPixelI420
  • AgoraVideoFormatCVPixelBGRA

• videoSizeChangedOfUid

Modified

• takeSnapshot
• snapshotTaken

Deprecated

• openWithCustomSourceAtStartPos
• setAudioProfile[2/2]

v4.0.0-beta.1

v4.0.0-beta.1 was released on April 11, 2022.

Compatibility changes

This release has optimized the implementation of some features, resulting in incompatibility with v3.6.2. The following are the main features with compatibility changes:

• Multiple channel
• Media stream publishing control
• Custom video capture and renderer (Media IO)
• Error codes and warning codes

After upgrading the SDK, you need to update the code in your app according to the business scenarios. For details, see Migrate from v3.6.2 to v4.0.0 Beta.

New features

1. Extensions from the Agora extensions marketplace

In order to enhance the real-time audio and video interactive activities based on the Agora SDK, this release supports the one-stop solution for the extensions from the Agora extensions marketplace:

• Easy to integrate: The integration of modular functions can be achieved simply by calling an API, and the integration efficiency is improved by nearly 95%.
• Extensibility design: The modular and extensible SDK design style endows the Agora SDK with good extensibility, which enables developers to quickly build real-time interactive apps based on the Agora extensions marketplace ecosystem.
• Build an ecosystem: A community of real-time audio and video apps has developed that can accommodate a wide range of developers, offering a variety of extension combinations. After integrating the extensions, developers can build richer real-time interactive functions. For details, see Use an Extension.
• Become a vendor: Vendors can integrate their products with Agora SDK in the form of extensions, display and publish them in the Agora extensions marketplace, and build a real-time interactive ecosystem for developers together with Agora. For details on how to develop and publish extensions, see Become a Vendor.

2. Multiple media tracks

This release supports one AgoraRtcEngineKit instance to collect multiple audio and video sources at the same time and publish them to the remote users, adapting to various business scenarios, including the following:

• Simultaneously publish a video stream from a media player, a screen sharing stream, and a video stream captured by a camera.
• Simultaneously publish the audio stream captured by a single-channel microphone, self-captured, and the audio stream of the media player.

You can also experience the following features with the multi-channel capability:

• Publish multiple sets of audio and video streams to the remote users through different user IDs (uid).
• Mix multiple audio streams and publish to the remote users through a user ID (uid).
• Combine multiple video streams and publish them to the remote users through a user ID (uid).

3. Enhanced channel management

To meet the channel management requirements of various business scenarios, this release adds the following functions to the AgoraRtcChannelMediaOptions structure:

• Sets or switches the release of multiple audio and video sources.
• Sets or switches channel profile and user role.
• Sets or switches the stream type of the subscribed video.
• Controls audio publishing delay.

Set AgoraRtcChannelMediaOptions when calling joinChannelByToken or joinChannelExByToken to specify the publishing and subscription behavior of a media stream, for example, whether to publish video streams captured by cameras or screen sharing, and whether to actively subscribe to the audio and video streams of remote users. After joining the channel, call updateChannelWithMediaOptions to update the settings in AgoraRtcChannelMediaOptions at any time, for example, to switch the published audio and video sources.

4. Agora media player

To make it easier for users to integrate the Agora SDK and reduce the SDK's package size, this release introduces the Agora media player. After calling the createMediaPlayerWithDelegate method to create a media player object, you can then call the methods in the AgoraRtcMediaPlayerProtocol class to experience a series of functions, such as playing local and online media files, preloading a media file, changing the CDN route for playing according to your network conditions, or sharing the audio and video streams being played with remote users.

5. Streaming module

This release introduces the streaming module that supports the SDK directly pushing media streams published by the hosts to a CDN. During this process, media streams are pushed in the RTMP protocol, and you can set the video and audio profiles according to your scenarios. The streaming module also supports media enhancement supplementary information (SEI), customizing the live streaming watermark, and adjusting video encoder configurations based on your network conditions. You can try this feature in live streaming events where there is only one host, such as fashion show, game, and e-commerce live streaming.

6. Real-time chorus

This release gives real-time chorus the following abilities:

• Two or more choruses are supported.
• Each singer is independent of each other. If one singer fails or quits the chorus, the other singers can continue to sing.
• Very low latency experience. Each singer can hear each other in real time, and the audience can also hear each singer in real time.

This release adds the AgoraAudioScenarioChorus enumeration in AgoraAudioScenario. With this enumeration, users can experience ultra-low latency in real-time chorus when the network conditions are good.

7. Ultra-high audio quality

To make the audio clearer and restore more details, this release adds the AgoraVoiceBeautifierUltraHighQuality enumeration. In scenarios that mainly feature the human voice, such as chat or singing, you can call setVoiceBeautifierPreset and use this enumeration to experience ultra-high audio quality.

8. Stereo processing

In scenarios that require enhanced realism, such as concerts, local users might need to capture stereo audio and send stereo signals to remote users.
To meet the requirement for stereo audio, this release adds the setAdvancedAudioOptions method. Agora recommends the following settings:

1. Preprocessing: Call setAdvancedAudioOptions, and set audioProcessingChannels to AgoraAudioProcessChannelsStereo in AdvancedAudioOptions.
2. Post-processing: Call setAudioProfile, and set profile to AgoraAudioProfileMusicStandardStereo or AgoraAudioProfileMusicHighQualityStereo.

Improvements

1. Fast channel switching

This release can achieve the same switching speed as switchChannelByToken in v3.6.2 by switching channels through leaveChannel and joinChannelByToken, without additionally calling switchChannelByToken.

2. External video source

This release adds the sourceType parameter in the setExternalVideoSource method, so that you can push the encoded external video data to the SDK.

3. Voice pitch of the local user
This release adds voicePitch in AgoraRtcAudioVolumeInfo of reportAudioVolumeIndicationOfSpeakers. You can use voicePitch to get the local user's voice pitch and perform business functions such as rating for singing.