Video Stream Fallback

Introduction

The audio and video quality of interactive live video streaming or a video call deteriorates under poor network conditions. To improve the efficiency of interactive live video streaming or a video call, the SetLocalPublishFallbackOption and SetRemoteSubscribeFallbackOption methods are used for the SDK to automatically switch the high-video stream to low-video stream and disable the video stream under these conditions.

Implementation

Before proceeding, ensure that you implement interactive live video streaming or call in your project. See Start a Video Call or Start Live Interactive Video Streaming for details.

Refer to the following steps to set the stream fallback under poor network conditions:

1. After joining the channel, the publisher calls the EnableDualStreamMode method to enable dual stream mode.
2. The publisher calls the SetLocalPublishFallbackOption method and set AUDIO_ONLY (2) to enable the fallback option for the locally published media stream based on the network conditions. The SDK will disable the upstream video and enable audio only when the network conditions deteriorate and cannot support both video and audio streams.

When the local video stream falls back to audio only or when the audio-only stream switches back to the video, the SDK triggers the OnLocalPublishFallbackToAudioOnlyHandler callback to report current stream state to the local publisher.

3. The subscriber in the channel calls the SetRemoteSubscribeFallbackOption method to set the subscribed stream fallback under poor network conditions.

   • Sets STREAM_LOW (1) to only subscribe to the low-video stream sent from the publisher under poor network conditions.

   • Sets AUDIO_ONLY (2) to subscribe to the low-video stream sent or even audio stream from the publisher under poor network conditions.

     Once the remote media stream is switched to the low stream due to poor network conditions, you can monitor the stream change between a high and low stream in the OnRemoteVideoStatsHandler callback. When the remotely subscribed video stream falls back to audio only or when the audio-only stream switches back to the video stream, the SDK triggers the OnRemoteSubscribeFallbackToAudioOnlyHandler callback.

Sample code

// Enables the dual-stream mode (Configuration for the local).
mRtcEngine.EnableDualStreamMode(true);

// Configuration for the publisher. When the network condition is poor, send audio only. 
mRtcEngine.SetLocalPublishFallbackOption(STREAM_FALLBACK_OPTIONS.STREAM_FALLBACK_OPTION_AUDIO_ONLY);

// Configuration for the subscriber. Try to receive low stream under poor network conditions. When the current network conditions are not sufficient for video streams, receive audio stream only.    
mRtcEngine.SetRemoteSubscribeFallbackOption(STREAM_FALLBACK_OPTIONS.STREAM_FALLBACK_OPTION_AUDIO_ONLY);

API reference

• EnableDualStreamMode
• SetLocalPublishFallbackOption
• SetRemoteSubscribeFallbackOption
• OnLocalPublishFallbackToAudioOnlyHandler
• OnRemoteSubscribeFallbackToAudioOnlyHandler
• OnRemoteVideoStatsHandler