Video extension

Understand the tech

A video filter accesses video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and playback. To modify the voice or video data in the transmission pipeline, video extensions are inserted into either the pre-processing or post-processing stage.

Prerequisites

• Android Studio 4.1 or higher.
• Android SDK API Level 24 or higher.
• A mobile device that runs Android 4.1 or higher.
• A project to develop in.

Project setup

In order to integrate an extension into your project:

• Follow the SDK setup flow in Quickstart - integrate an extension.
• Prepare the Android project structure and Gradle dependencies required for the extension package.

You are now ready to develop your extension.

Create a video extension

After encapsulating the filter into an extension, you need to register and package it into a .aar or .so file, and submit it together with a file that contains the extension name, vendor name, and filter name to Agora.

Package the extension

1. Register the extension.

   Register the extension with the REGISTER_AGORA_EXTENSION_PROVIDER macro in AgoraExtensionProviderEntry.h. Use this macro at the entrance of the extension implementation. When the SDK loads the extension, this macro automatically registers it to the SDK. For example:

   REGISTER_AGORA_EXTENSION_PROVIDER(ByteDance, agora::extension::ExtensionProvider);

2. Link the libagora-rtc-sdk-jni.so file.

   In CMakeLists.txt, specify the path to save the libagora-rtc-sdk-jni.so file in the downloaded SDK package according to the following table:

   File	Path
   64-bit libagora-rtc-sdk-jni.so	AgoraWithByteDanceAndroid/agora-bytedance/src/main/agoraLibs/arm64-v8a
   32-bit libagora-rtc-sdk-jni.so	AgoraWithByteDanceAndroid/agora-bytedance/src/main/agoraLibs/arm64-v7a

3. Provide extension information.

   Create a .java or .md file to provide the following information:

   • EXTENSION_NAME: The name of the target link library used in CMakeLists.txt. For example, for a .so file named libagora-bytedance.so, the EXTENSION_NAME should be agora-bytedance.
   • EXTENSION_VENDOR_NAME: The name of the extension provider, which is used for registering in the agora-bytedance.cpp file.
   • EXTENSION_FILTER_NAME: The name of the filter, which is defined in ExtensionProvider.h.

Test your implementation

• Run functional and performance tests.
• Validate the extension listing in Agora's test environment.
• Prepare the public integration document and update it after end-to-end verification.

You are then ready to Publish your extension.

Reference

• RtcEngineConfig.addExtension
• enableExtension
• setExtensionProperty
• IMediaExtensionObserver

Understand the tech

A video filter accesses video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and playback. To modify the voice or video data in the transmission pipeline, video extensions are inserted into either the pre-processing or post-processing stage.

Prerequisites

• Xcode 12.0 or higher.
• A device running iOS 9.0 or higher.
• A project to develop in.

Project setup

• Follow the iOS SDK setup flow in Quickstart - integrate an extension.
• Prepare the project structure, frameworks, and signing settings needed for your extension package.

Create a video extension

Package the extension as an iOS framework or xcframework, register it with the SDK, and provide the metadata Agora needs for Marketplace integration.

Reference

• RTC iOS API reference
• RTC iOS API reference

Understand the tech

A video filter accesses video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and playback. To modify the voice or video data in the transmission pipeline, video extensions are inserted into either the pre-processing or post-processing stage.

Prerequisites

• Xcode 12.0 or higher.
• A device running macOS 10.11 or higher.
• An Apple developer account.
• A project to develop in.

Project setup

• Follow the macOS SDK setup flow in Quickstart - integrate an extension.
• Prepare the framework packaging, code signing, and runtime loading flow for your extension.

Understand the tech

A video filter accesses video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and playback. To modify the voice or video data in the transmission pipeline, video extensions are inserted into either the pre-processing or post-processing stage.

Prerequisites

• A supported browser.
• Physical media input devices such as a camera and a microphone.
• A JavaScript package manager such as npm.
• A project to develop in.

Project setup

• Start from the Web flow in Quickstart - integrate an extension.
• Prepare your extension package, bundler integration, and runtime registration flow.

Understand the tech

A video filter accesses video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and playback. To modify the voice or video data in the transmission pipeline, video extensions are inserted into either the pre-processing or post-processing stage.

Prerequisites

• A device running Windows 7 or higher.
• Microsoft Visual Studio 2017 or higher.
• C#.
• A project to develop in.

Project setup

• Follow the Windows SDK setup flow in Quickstart - integrate an extension.
• Prepare the extension DLL packaging and loading flow.

Understand the tech

A video filter accesses video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Prerequisites

• Physical media input devices such as a camera and a microphone.
• A JavaScript package manager such as npm.
• A project to develop in.

Project setup

• Follow the Electron SDK setup flow in Quickstart - integrate an extension.
• Prepare the native extension packaging and runtime loading flow.

We are still developing extensions for this platform. The docs will magically appear here as soon as it is released.

We are still developing extensions for this platform. The docs will magically appear here as soon as it is released.

We are still developing extensions for this platform. The docs will magically appear here as soon as it is released.