Configure video encoding

Updated

Customer satisfaction with your Video Calling integrated game depends on the quality of video and audio it provides. The sharpness, smoothness, and overall quality of the video is directly linked to the frame rate, bitrate, and other video encoder settings. Choosing improper settings can result in poor video quality. Conversely, if the settings are too demanding, the available bandwidth quickly gets choked, leading to a suboptimal experience for your users.

This page guides you on configuring the video encoder settings to ensure optimal video quality in your Interactive Live Streaming game.

Understand the tech

In Video SDK you can set the video dimensions, framerate, bitrate, orientation mode, and mirror mode by specifying a video profile. You can also set the degradation preference to specify how video quality is degraded under suboptimal network conditions.

Resolution, frame rate, and bitrate

  • Resolution: Defines the video encoding resolution in pixels. The default value is 960 × 540. Higher resolutions generally result in better video clarity. Note that this parameter does not determine the final video orientation. Refer to Video orientation for setting the video output orientation.

  • Frame rate: Represents the number of frames encoded per second (FPS). The default value is 15 FPS. A higher frame rate contributes to smoother video playback. For use-cases demanding high video smoothness, consider setting this parameter to 20 or 25. It is advised not to exceed a frame rate of 30.

  • Bitrate: Indicates the video encoding bitrate in Kbps. The default mode is set to Standard Bitrate. In this mode, the Video SDK dynamically sets an appropriate bitrate based on the channel profile, resolution, and frame rate.

  • Minimum bitrate: Sets the minimum video encoding bitrate in Kbps. The SDK automatically adjusts bitrate based on network conditions. The default value for this parameter allows the SDK to determine the minimum bitrate automatically, which is recommended for most use cases. Setting this parameter higher than the default value forces the video encoder to maintain higher quality, but may increase packet loss and affect video playback smoothness.

Video SDK offers a variety of resolutions and frame rates to choose from. To specify your own configuration, refer to the Video profiles table.

To achieve high video quality, it's crucial to maintain a balanced relationship between resolution, bitrate, and frame rate. Higher resolutions necessitate a higher bitrate. If the bitrate is fixed, an excessively high frame rate can reduce the resolution.

The configured parameter settings represent maximum values under ideal conditions. In cases where video quality cannot reach the set maximum values due to network constraints or other factors, the actual values are adjusted to match the specified maximum resolution, frame rate, or bitrate as closely as possible.

Billing is based on actual video resolution, not configured settings. For example, if network adaptation reduces your video from 1280×720 to 640×360, you are billed for the lower resolution that users actually receive.

Video orientation

The way video is displayed on the playing device depends on orientationMode used on the encoding device, orientation of the capturing device, orientation of the playing device, and whether screen rotation is enabled on the playing device. On the capturing device, you can set the orientationMode to:

  • Adaptive

    In the this mode, the direction of the video output is consistent with the direction of the captured video. The receiving end rotates the video based on the received video rotation information. This mode is suitable for use-cases where the receiving end can adjust the video direction. No matter which mode you choose, Video SDK ensures that the relative position of the video and the status bar is always consistent at the capturing end and playback end.

  • Fixed Portrait

    In this mode, the output video is always in portrait mode relative to the Status Bar. If the captured video is in landscape mode, the video encoder crops it. This method is suitable for situations where the receiving end cannot process the rotation information.

  • Fixed Landscape

    In this mode, the output video is always in landscape mode relative to the Status Bar. If the captured video is in portrait mode, the video encoder crops it. This method is suitable for situations where the receiving end cannot process the rotation information, such as web browsers or legacy applications.

The following table shows how video orientation behaves under different mode and device configurations:

Orientation modeScreen rotationCapturing deviceRecording serverReceiving end display
AdaptiveDisabledLandscapeLandscapeLandscape. Does not change with device rotation
AdaptiveDisabledPortraitPortraitPortrait. Does not change with device rotation
AdaptiveEnabledLandscapeLandscapeLandscape. Rotates with device orientation
AdaptiveEnabledPortraitPortraitPortrait. Rotates with device orientation
Fixed landscapeLandscapeLandscapeAlways landscape
Fixed landscapePortraitLandscape (cropped)Always landscape (cropped)
Fixed portraitPortraitPortraitAlways portrait
Fixed portraitLandscapePortrait (cropped)Always portrait (cropped)

The following images illustrate the orientation behavior:

Adaptive orientation mode

  • Screen rotation: Disabled

  • Capturing device orientation: Landscape

  • Screen rotation: Disabled

  • Capturing device orientation: Portrait

  • Screen rotation: Enabled

  • Capturing device orientation: Landscape

  • Screen rotation: Enabled

  • Capturing device orientation: Portrait

Landscape orientation mode

  • Capturing device orientation: Landscape

  • Capturing device orientation: Portrait

Portrait orientation mode

  • Capturing device orientation: Portrait

  • Capturing device orientation: Landscape

Degradation preference

To enhance the video experience for users in low-bandwidth conditions, the Video SDK offers the degradationPreference parameter. This parameter determines how video adapts when bandwidth is limited: prioritize frame rate for smooth playback, prioritize resolution for visual clarity, balance both qualities, or let the SDK decide automatically based on the scenario.

Mirror mode

By default, Video SDK does not mirror the video during encoding. You use the mirrorMode parameter to decide whether to mirror the video that remote users see.

Prerequisites

Ensure that you have implemented the SDK quickstart in your project.

Implementation

Use the SetVideoEncoderConfiguration method to configure video encoding parameters. You can call this method at any point after initialization, whether it's before or after joining a channel. For optimal performance, best practice is to enable the video module by calling EnableVideo before calling SetVideoEncoderConfiguration. This approach reduces the time required for the first frame to be generated.

To set the video encoder configuration, take the following steps:

  1. Create UI

In BP_VideoWidget, create a Video Encoder Configuration area containing the following controls:

  • ET_FPS: Set the frame rate of video encoding.
  • ET_Width: Set the width of the video.
  • ET_Height: Set the height of the video.
  • ET_BitRate: Set the bitrate of the video.
  • ET_MinBitRate: Set the minimum encoding bitrate of the video.

Sample UI Interface:

  1. Bind events

Follow the steps below to bind the corresponding events in the node.

  1. In the Bind UIEvent node, bind the OnVideoConfigConfirmClicked callback. This callback is triggered when the user sets the video encoding parameters in the video encoder configuration.

  2. In the Bind Event node, bind the OnVideoSizeChanged callback. This callback is triggered when there is a change in the video size or rotation information.

  3. Implement the OnVideoConfigConfirmClicked callback

To change the video encoder configuration, take the following steps:

  1. In the My Blueprint panel of the Level Blueprint, follow these steps to create an OnVideoConfigConfirmClicked UI event node:

  2. Open the Level Blueprint.

  3. Locate the My Blueprint panel.

  4. Create a new UI event node named OnVideoConfigConfirmClicked.

  5. Connect the created UI element to the Make VideoEncoderConfiguration node. Pass in the encoding parameters entered by the user to configure the VideoEncoderConfiguration. Follow the steps below:

  6. Retrieve the user-specified values from the UI elements.

  7. Use these values as inputs to the Make VideoEncoderConfiguration node.

  8. Connect the output of the Make VideoEncoderConfiguration node to the appropriate input of the video encoding process.

  9. Call the SetVideoEncoderConfiguration method and configure the following parameters in the Make VideoEncoderConfiguration node according to your specific requirements:

  10. Orientation Mode: Set the rotation direction of the video.

  11. Degradation Preference: Specify the degradation preference for video encoding when bandwidth is limited.

  12. Mirror Mode: Define whether the video screen seen by the remote user should be mirrored.

  13. Compression Preference: Set the compression preference for video encoding.

Refer to the following sample interface:

  1. Implement the OnVideoSizeChanged callback

Follow the steps below to implement the OnVideoSizeChanged callback function and adjust the local view based on the video change information obtained in the callback:

  1. In the My Blueprint panel of the level blueprint, create an OnVideoSizeChanged event node.
  2. Within the OnVideoSizeChanged callback, obtain the key parameters of the video change information: sourceType, uid, width, height, rotation.
  3. Pass these parameters into the Make FVideoViewIdentity node to adjust both the local and remote views accordingly.

Sample implementation:

  • Connect the obtained parameters (sourceType, uid, width, height, rotation) to the corresponding inputs of the Make FVideoViewIdentity node.
  • Adjust the local and remote views using the output of the Make FVideoViewIdentity node.

Troubleshooting

The following are some common issues and their solutions:

IssuePossible solution
Blurry videoIncrease resolution or bitrate
Choppy/laggy playbackReduce resolution, increase minimum bitrate, or use maintain framerate degradation
High bandwidth usageLower resolution or frame rate
Video orientation incorrectUse fixed portrait or fixed landscape orientation mode

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Video profiles table

Video SDK provides a selection of video dimensions and frame rates to choose from. You can also customize the values according to the following table:

Resolution (width × height)Frame rate (fps)
160 × 12015
120 × 12015
320 × 18015
180 × 18015
240 × 18015
320 × 24015
240 × 24015
424 × 24015
640 × 36015
360 × 36015
640 × 36030
360 × 36030
480 × 36015
480 × 36030
640 × 48015
480 × 48015
640 × 48030
480 × 48030
848 × 48015
848 × 48030
640 × 48010
960 × 54015
960 × 54030
1280 × 72015
1280 × 72030
960 × 72015
960 × 72030
1920 × 108015
1920 × 108030
1920 × 108060
2560 × 144030
2560 × 144060
3840 × 216030
3840 × 216060

After you set the resolution and frame rate, the SDK automatically sets the corresponding bitrate. Agora does not recommend that you modify the bitrate manually.

Sample project

Agora provides open source sample projects for your reference. Download or view the following projects for more detailed examples.

API reference