Skip to main content

Enable adaptive bitrate

Adaptive Bitrate (ABR) streaming delivers multiple video quality levels from a single source, allowing viewers to automatically switch between different bitrates based on their network conditions and device capabilities. This ensures optimal viewing experience by reducing buffering and maintaining the highest possible quality for each viewer's situation.

This guide explains how to configure Adaptive Bitrate (ABR) encoding for video transcoding in Media Gateway.

Understand the tech

ABR works by encoding the same video content at multiple resolutions and bitrates, called layers. Viewers can then select or automatically switch between these layers based on their available bandwidth and device performance. Media Gateway supports ABR through the simulcastStreamLayers configuration in stream templates. When enabled, the system creates multiple encoding layers from your source stream.

Following are the key ABR concepts:

  • Source stream: The original stream pushed by the client without transcoding.

  • Main stream: The primary transcoded stream output by Media Gateway when video transcoding is enabled. Resolution does not exceed the source stream resolution.

  • Layer: Additional streams with lower bitrates that Media Gateway outputs, based on your configuration. These streams are in addition to the main stream and are available when video transcoding is enabled. Each layer maintains consistent aspect ratios while providing different quality levels. The system automatically handles encoding parameters and ensures proper bitrate gradients between layers for smooth switching.

  • Layer ID: A unique identifier for each layer stream, ranging from 1 to 7.

  • Low stream: A lower-quality stream option that was available before full ABR support. Previously used to provide basic two-level quality switching. It is now implemented as Layer ID 7 to maintain compatibility with existing applications that expect this stream type.

    info
    • ABR layers are independent - enabling ABR doesn't automatically create a low stream. You can selectively enable Layer ID 7 based on your needs.
    • Use Layer ID 7 for backward compatibility with older applications that don't support full ABR functionality.

ABR Configuration

Configure ABR encoding in stream configuration templates using the settings.transcoding.video.simulcastStreamLayers parameter.

Request parameters

  • transcoding.video Object (required): Video transcoding configuration
    • enabled Boolean (required): Set to true to enable video transcoding
    • codec String (optional): Video encoding format. "H.264" (default), "VP8"
    • width Number (optional): Main stream encoding width in pixels. If unspecified or 0, follows source stream width when height is also unspecified. If height is specified, implements "short side fixed, long side adaptive"
    • height Number (optional): Main stream encoding height in pixels. If unspecified or 0,Follows source stream height when width is also unspecified. If width is specified, implements "long side fixed, short side adaptive"
    • fps Number (optional): Main stream frame rate in fps. If unspecified or 0, follows source stream fps
    • bitrate Number (optional): Main stream bitrate in kbps. If unspecified, follows source stream bitrate. If set to 0, automatically calculated based on resolution and fps.
    • simulcastStreamLayers Array (optional): ABR layer definitions. When specified, enables ABR functionality. Each array element contains:
      • id Number (required): Layer identifier (range: 1-7)
      • width Number (optional): Layer encoding width in pixels. Must decrease monotonically as Layer ID increases. If unspecified or 0: height must be specified for aspect ratio calculation
      • height Number (optional): Layer encoding height in pixels. Must decrease monotonically as Layer ID increases. If unspecified or 0: width must be specified for aspect ratio calculation
      • fps Number (optional): Layer frame rate in fps. If unspecified or 0: Uses main stream fps
      • bitrate Number (required): Layer bitrate in kbps. Must decrease monotonically as Layer ID increases

Request example

The following example creates a transcoding template with ABR encoding. It configures a main stream (1080p) and four additional layers with decreasing quality:


_51
$ curl --request PUT \
_51
--url http://${host}/${region}/v1/projects/${appid}/rtls/ingress/stream-templates/${templateId} \
_51
--header 'Content-Type: application/json' \
_51
--header 'Authorization: Basic XXXXXX' \
_51
--data '{
_51
"settings": {
_51
"transcoding": {
_51
"video": {
_51
"enabled": true,
_51
"width": 1920,
_51
"height": 1080,
_51
"fps": 30,
_51
"bitrate": 3800,
_51
"simulcastStreamLayers": [
_51
{
_51
"id": 1,
_51
"width": 1280,
_51
"height": 720,
_51
"fps": 30,
_51
"bitrate": 2400
_51
},
_51
{
_51
"id": 2,
_51
"width": 960,
_51
"height": 540,
_51
"fps": 30,
_51
"bitrate": 1500
_51
},
_51
{
_51
"id": 3,
_51
"width": 854,
_51
"height": 480,
_51
"fps": 30,
_51
"bitrate": 800
_51
},
_51
{
_51
"id": 7,
_51
"width": 640,
_51
"height": 360,
_51
"fps": 30,
_51
"bitrate": 500
_51
}
_51
]
_51
},
_51
"audio": {
_51
"enabled": true,
_51
"profile": 3
_51
}
_51
}
_51
}
_51
}'

Configuration requirements

Follow these requirements to ensure proper ABR functionality and avoid configuration errors:

Layer limitations:

  • Maximum 5 simultaneous layers per stream (despite Layer ID range of 1-7)
  • Choose 1-5 layers based on your use case

Resolution requirements:

  • Main stream: Specify at least one dimension (width or height) when ABR is enabled
  • ABR layers: Specify at least one dimension for each layer using consistent fields across all layers

Bitrate requirements:

  • Main stream: Must specify bitrate when ABR is enabled (cannot be unspecified or 0)
  • ABR layers: Must specify bitrates in decreasing order by Layer ID
  • Use proper bitrate gradients to ensure smooth quality switching

Client integration:

  • Clients must specify the Layer ID when subscribing to receive the corresponding quality layer

Layer ID selection

When specifying layer IDs, best practice is to bind each ID to a specific resolution. This enables you to dynamically add or delete ABR layers without needing to modify the mapping between layer ID and resolution.

Recommended Layer ID mapping:

Layer IDResolution
13840x2160 (4K)
22560x1440 (2K)
31920x1080 (1080p)
41280x720 (720p)
5960x540 (540p)
6854x480 (480p)
7640x360 (360p)

Example configuration:

If your source stream is 1080p and you want to output 1080p, 720p, 480p, and 360p video streams, configure the ABR layer IDs as follows:

Layer IDResolution
Main Stream1080p
4720p
6480p
7360p

Webhook notifications

To help you monitor the ABR configuration for each stream, Media Gateway provides this information through webhooks. When video transcoding is enabled, the live_stream_connected event includes the transcoding configuration details for your stream.

Example notification payload

The transcoding.video.simulcastStreamLayers array shows the active ABR layers for this stream, including their resolution, bitrate, and Layer ID configuration.


_47
{
_47
"beginAt": "2025-03-11T17:56:23.60Z",
_47
"domain": "rtls-ingress-test-cn.agoramdn.com",
_47
"region": "dev-cn",
_47
"rtcInfo": {
_47
"channel": "abrtest0224",
_47
"uid": "1000"
_47
},
_47
"sid": "66ff5811-7baa-0298-8244-f2de5ae2a047",
_47
"streamKey": "123456rlg-RSQ_TqkhqgF373zAll4vz",
_47
"transcoding": {
_47
"audio": {
_47
"enabled": true,
_47
"profile": 3
_47
},
_47
"video": {
_47
"enabled": true,
_47
"bitrate": 3000,
_47
"fps": 30,
_47
"width": 1920,
_47
"height": 1080,
_47
"simulcastStreamLayers": [
_47
{
_47
"id": 4,
_47
"bitrate": 2000,
_47
"fps": 30,
_47
"height": 720,
_47
"width": 1280
_47
},
_47
{
_47
"id": 5,
_47
"bitrate": 1200,
_47
"fps": 30,
_47
"height": 540,
_47
"width": 960
_47
},
_47
{
_47
"id": 7,
_47
"bitrate": 700,
_47
"fps": 30,
_47
"height": 360,
_47
"width": 640
_47
}
_47
]
_47
}
_47
}
_47
}

Subscriber-side ABR stream

This section explains how to implement ABR streaming on the subscriber side using Agora Native and Web SDKs. ABR automatically adjusts video quality based on network conditions to ensure smooth playback.

Integration guidelines

The following Video SDK versions include the necessary ABR APIs and optimizations. Using the recommended versions ensures access to the latest performance improvements and bug fixes.

  • Web SDK:

  • Native SDK:

    • Minimum version: 4.3.2 or later
    • Recommended: Use the latest SDK from the official website

Stream quality levels

ABR uses multiple quality layers to adapt to different network conditions. The table below shows how RTMPG layer IDs correspond to SDK-specific stream types:

RTMPG Layer IDNative SDK (VideoStreamType)Web SDK (RemoteStreamType)
1VIDEO_STREAM_LAYER_1HIGH_STREAM_LAYER1
2VIDEO_STREAM_LAYER_2HIGH_STREAM_LAYER2
3VIDEO_STREAM_LAYER_3HIGH_STREAM_LAYER3
4VIDEO_STREAM_LAYER_4HIGH_STREAM_LAYER4
5VIDEO_STREAM_LAYER_5HIGH_STREAM_LAYER5
6VIDEO_STREAM_LAYER_6HIGH_STREAM_LAYER6
7VIDEO_STREAM_LOWLOW_STREAM
Push streamingVIDEO_STREAM_HIGHHIGH_STREAM

Native SDK: Subscriber-side APIs

These APIs enable you manage how video streams are handled under varying network conditions.

Configure remote subscriber fallback

  • API: setRemoteSubscribeFallbackOption(StreamFallbackOptions option)

  • Description: Sets the fallback option for subscribed audio and video streams under poor network conditions. In suboptimal network environments, real-time audio and video quality may degrade. Use this method to set a StreamFallbackOptions value. When the downlink network is weak and the quality is severely affected, the SDK automatically reduces the video stream to a lower layer to maintain smooth playback. The SDK continuously monitors the network and restores the higher layer when conditions improve.

    Specify the minimum acceptable fallback level:

    • To fall back to a lower video stream, use STREAM_FALLBACK_OPTION_VIDEO_STREAM_LOW.
    • To fall back to audio only, use STREAM_FALLBACK_OPTION_AUDIO_ONLY.
  • Enum: StreamFallbackOptions


    _11
    enum StreamFallbackOptions {
    _11
    STREAM_FALLBACK_OPTION_DISABLED,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LOW,
    _11
    STREAM_FALLBACK_OPTION_AUDIO_ONLY,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LAYER_1,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LAYER_2,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LAYER_3,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LAYER_4,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LAYER_5,
    _11
    STREAM_FALLBACK_OPTION_VIDEO_STREAM_LAYER_6
    _11
    }

Manual stream selection

  • API: setRemoteVideoStreamType(int uid, VideoStreamType streamType)
  • Description: Sets the specified video stream type for subscription

Set the default stream type

  • API: setRemoteDefaultVideoStreamType(VideoStreamType streamType)

  • Description: Sets the default video stream type for subscription

  • Enum: VideoStreamType


    _10
    enum VideoStreamType {
    _10
    VIDEO_STREAM_HIGH,
    _10
    VIDEO_STREAM_LOW,
    _10
    VIDEO_STREAM_LAYER_1,
    _10
    VIDEO_STREAM_LAYER_2,
    _10
    VIDEO_STREAM_LAYER_3,
    _10
    VIDEO_STREAM_LAYER_4,
    _10
    VIDEO_STREAM_LAYER_5,
    _10
    VIDEO_STREAM_LAYER_6;
    _10
    }

Web SDK: Subscriber-side APIs

These APIs enables subscribers to control stream behavior and optimize performance based on network conditions.

Enable bandwidth estimation

  • API: .setParameter("ENABLE_AUT_CC", true);
  • Description: AUT-CC is a server-side Bandwidth Estimation (BWE) mechanism that relies on packet loss. Its main inputs include the number of packets, packet size, feedback on packet loss, and the timing of that feedback. The output is the currently available bandwidth.

Configure remote subscriber fallback

  • API: setStreamFallbackOption(uid: UID, fallbackType: RemoteStreamFallbackType): Promise<void>

  • Description: Enable automatic switching, the subscriber side adapts to the network conditions.

  • Example:


    _2
    // the range supports up to LOW_STREAM
    _2
    client.setStreamFallbackOption(uid, 1);

Manual stream selection

  • API: setRemoteVideoStreamType(uid: UID, streamType: RemoteStreamType): Promise<void>

  • Description: Manual switching, set the specified video stream type for subscription.

  • Example:


    _1
    client.setRemoteVideoStreamType(uid, 0); // HIGH_STREAM = 0

  • Enum: VideoStreamType


    _10
    enum VideoStreamType {
    _10
    HIGH_STREAM,
    _10
    LOW_STREAM,
    _10
    HIGH_STREAM_LAYER1,
    _10
    HIGH_STREAM_LAYER2,
    _10
    HIGH_STREAM_LAYER3,
    _10
    HIGH_STREAM_LAYER4,
    _10
    HIGH_STREAM_LAYER5,
    _10
    HIGH_STREAM_LAYER6;
    _10
    }

For manual switching, consider the network conditions. The following example shows how to obtain the network status:


_14
const NetworkQualityDesc = [
_14
"Unknown",
_14
"Excellent",
_14
"Good",
_14
"Poor",
_14
"Bad",
_14
"Very Bad",
_14
"Down",
_14
];
_14
client.on("network-quality", (quality) => {
_14
uplinkQuality = NetworkQualityDesc[quality.uplinkNetworkQuality];
_14
downlinkQuality = NetworkQualityDesc[quality.downlinkNetworkQuality];
_14
console.log(uplinkQuality, downlinkQuality);
_14
});

Set the default stream type

  • API: setRemoteDefaultVideoStreamType(streamType: RemoteStreamType): Promise<void>
  • Description: Set the default video stream type for subscription