Integrate an extension

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Download the extension

   In the extension detail page, click Download, then unzip the extension in a local directory.

4. Install the extension in your project

   • Android Archive file (.aar)

   1. Save the extension .aar file to /app/libs in your project.

   2. In /Gradle Scripts/build.gradle(Module: <ProjectName> app), add the following line under dependencies:

      implementation fileTree(include: ['*.jar', '*.aar'], dir: 'libs')

   • Shared Library (.so)

   Save the .so file to the following paths in your project:

   1. /app/src/main/jniLibs/arm64-v8a

   2. /app/src/main/jniLibs/armeabi-v7a

You are now ready to integrate the extension in your app.

Integrate the extension into your project

The watermark extension adds a watermark on video streamed to your local client. This section shows you how to implement the watermark extension in your Agora project:

1. Import the necessary classes

   1. Download the watermark extension and follow the steps for .aar files in setup.

   2. In app/src/main/java/com.example.<projectname>/MainActivity:

      1. Add the following lines to import the Android classes used by the extension:
      Java

      Kotlin

      import org.json.JSONException;
          import org.json.JSONObject;

      2. Add the following lines to import the Agora classes used by the extension:
      Java

      Kotlin

      // ExtensionManager is used to pass in basic info about the extension
          import io.agora.extension.ExtensionManager;
          import io.agora.rtc2.IMediaExtensionObserver;

2. Add the extension and register the event handler

   In setupVideoSDKEngine, add the following code before agoraEngine = RtcEngine.create(config);:

   Java

   Kotlin

   config.addExtension(ExtensionManager.EXTENSION_NAME);
     // Register IMediaExtensionObserver to receive events from the extension.
     config.mExtensionObserver = new IMediaExtensionObserver() {
       @Override
       public void onEvent(String vendor, String extension, String key, String value) {
         // Add callback handling logics for extension events.
         showMessage("Extension: " + extension + "
   Key: " + key + "
   Value:" + value);
       }
   
       @Override
       public void onStarted(String provider, String extension) {
         showMessage("Extension started");
       }
   
       @Override
       public void onStopped(String provider, String extension) {
         showMessage("Extension stopped");
       }
   
       @Override
       public void onError(String provider, String extension, int error, String message) {
         showMessage(message);
       }
     };

3. Enable the extension

   Call enableExtension to enable the extension. To enable multiple extensions, call enableExtension as many times. The sequence of enabling multiple extensions determines the order of these extensions in the transmission pipeline. For example, if you enable extension A before extension B, extension A processes data from the SDK before extension B.

   In setupVideoSDKEngine, add the following code before agoraEngine = RtcEngine.create(config);:

   Java

   Kotlin

   agoraEngine.enableExtension(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_VIDEO_FILTER_NAME, true);

4. Set extension properties

   In the joinChannel(View view) method, add the following code after agoraEngine.joinChannel:

   Java

   Kotlin

   JSONObject o = new JSONObject();
     try {
       // Pass in the key-value pairs defined by the extension provider to configure the feature you want to use.
       o.put("plugin.watermark.wmStr", "Agora");
       o.put("plugin.watermark.wmEffectEnabled", true);
   
       // Call setExtensionProperty to use the watermark feature.
       agoraEngine.setExtensionProperty(ExtensionManager.EXTENSION_VENDOR_NAME,
           ExtensionManager.EXTENSION_VIDEO_FILTER_NAME, "key", o.toString());
     } catch (JSONException e) {
       e.printStackTrace();
     }

Test your implementation

To ensure that you have integrated the extension in your app:

1. Connect the Android device to the computer.

2. Click Run app on your Android Studio. A moment later you will see the project installed on your device.

3. When the app launches, you can see yourself and the watermark Agora on the local view.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

• RtcEngineConfig.addExtension

• enableExtension

• getExtensionProperty

• setExtensionProperty

• IMediaExtensionObserver

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Download the extension

   In the extension detail page, click Download, then unzip the extension in a local directory.

4. Install the extension in your project

   1. Save the extension .framework or .xcframwork files to your project folder.

   2. In Xcode, open your project and select your target. Go to General > Frameworks, Libraries, and Embedded Content.

   3. Click + > Add Other… > Add Files and select the .framework or .xcframwork files you saved earlier.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

To use an extension in your Agora project:

• The implementation procedure varies according to extensions. Each extension vendor provides their own implementation guides, which is validated by Agora before the official release of the extension.

• To implement the extension in your project, go to the detail page of the extension on Agora Console, click Implementation guides, and follow the steps on the page.

Test your implementation

To ensure that you have integrated the extension in your app:

1. Connect a device to the computer.

2. In Xcode, Click Run. A moment later you will see the project installed on your device.

3. Connect to a channel and test your effect.

Agora provides an open-source sample project SimpleFilter on GitHub for your reference.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Download the extension

   In the extension detail page, click Download, then unzip the extension in a local directory.

4. Install the extension in your project

   1. Save the extension .framework or .xcframwork files to your project folder.

   2. In Xcode, open your project and select your target. Go to General > Frameworks, Libraries, and Embedded Content.

   3. Click + > Add Other… > Add Files and select the .framework or .xcframwork files you saved earlier.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

To use an extension in your Agora project:

• The implementation procedure varies according to extensions. Each extension vendor provides their own implementation guides, which is validated by Agora before the official release of the extension.

• To implement the extension in your project, go to the detail page of the extension on Agora Console, click Implementation guides, and follow the steps on the page.

Test your implementation

To ensure that you have integrated the extension in your app:

1. In Xcode, Click Run. A moment later you will see the project installed on your device.

2. Connect to a channel and test your effect.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart.

• A supported browser.

• Physical media input devices, such as a camera and a microphone.

• A JavaScript package manager such as npm.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Download the extension

   In the extension detail page, click Download, you are taken to the npm page for the extension.

4. Install the extension in your project

   1. In the extension npm page, copy the Install command.

   2. In the root of your project, install the extension package. For example, for AI Noise Suppression:

      npm i agora-extension-ai-denoiser

You are now ready to integrate the extension in your app.

Integrate the extension into your project

To use an extension in your Agora project:

• The implementation procedure varies according to extensions. Each extension vendor provides their own implementation guides, which is validated by Agora before the official release of the extension.

• To implement the extension in your project, go to the detail page of the extension on Extensions Marketplace, click Implementation guides, and follow the steps on the page.

To learn how to use a Web extension, see AI Noise Suppression.

Test your implementation

To ensure that you have integrated the extension in your app:

1. Generate a temporary token in Agora Console.

2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

3. In main.js, update appID, channel and token with your values.

4. Start the dev server

   Execute the following command in the terminal:

   npm run dev

   Use the URL displayed in the terminal to open the app in your browser.

5. Try out the extension that you have integrated.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

• AgoraRTC.registerExtensions

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Same setup as the Voice SDK quickstart prerequisites for Voice Calling.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Download the extension

   In the extension detail page, click Download, then unzip the extension in a local directory.

4. Install the extension in your project

   Add the extension .dll to your project.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

Integrating an extension into your project involves a few important steps to ensure seamless interaction between the extension and Agora Video SDK. Each of these steps plays a crucial role to ensure that the extension works correctly and efficiently within your Agora project:

1. Define variables to integrate an extension with Agora Video SDK

   In AgoraImplementationDlg.h, add the following declarations to the CAgoraImplementationDlg class before setupVideoSDKEngine();:

   // File path to the dynamic library (.dll file on Windows) that contains the implementation of the extension provider.
   const char* extensionLibrarayPath = "<extension dynamic library path and name>"; // For example : "/ library / libagora_segmentation_extension.dll"
   const char * extensionProvider = "<extension library path and name>"; // For example : "agora.io";
   const char* extensionName = "<extension name>"; // For example : "libagora_segmentation_extension.dll";
   
   // The key is a string that identifies a specific property of the extension,
   // and value is the new value for that property.
   // What these keys are and what values they can take is entirely up to the extension.
   // For example, an extension might have a property that controls the quality of a video stream, with a key named "videoQuality".
   // To set this property to "high", you would call setExtensionProperty() with "videoQuality" as the key and "high" as the value.
   const char* extensionKey = "<name of the extension property you want to set>";
   const char* extensionValue = "<new value of the property>";
   
   // When using methods that take a MEDIA_SOURCE_TYPE parameter, you would specify the value that corresponds to the type of media source
   // you want the method to operate on. For example, if you want to use the second camera to capture video, set this parameter to "SECONDARY_CAMERA_SOURCE".
   // The default value is UNKNOWN_MEDIA_SOURCE.
   agora::media::MEDIA_SOURCE_TYPE extensionMediaType = agora::media::UNKNOWN_MEDIA_SOURCE; // insert the media souce type as per your context

2. Register extension callbacks to the IRtcEngineEventHandler event handler

   1. In AgoraImplementationDlg.h, add the following declarations to the AgoraEventHandler class after onLeaveChannel:

      virtual void onExtensionEvent(const char* provider, const char* extension, const char* key, const char* value)override;
      virtual void onExtensionStopped(const char* provider, const char* extension)override;
      virtual void onExtensionStarted(const char* provider, const char* extension)override;
      virtual void onExtensionError(const char* provider, const char* extension, int error, const char* errorMessage)override;

   2. Define these callbacks in AgoraImplementationDlg.cpp:

      // Extension Callbacks
      void AgoraEventHandler::onExtensionEvent(const char* provider, const char* extension, const char* key, const char* value)
      {
        // Handle the event
        CString extension_provider(provider);
        CString extension_name(extension);
        CString extension_key(key);
        CString extension_value(value);
      
        CString message;
        message.Format(_T("Extension Provider: %s\nExtension Name: %s\nKey: %s\nValue: %s\n is enabled to run."), extension_provider, extension_name);
        AfxMessageBox(message);
      }
      
      void AgoraEventHandler::onExtensionStopped(const char* provider, const char* extension)
      {
        AfxMessageBox(L"Extension stopped.");
      }
      
      void AgoraEventHandler::onExtensionStarted(const char* provider, const char* extension)
      {
        AfxMessageBox(Extension started);
      }
      
      void AgoraEventHandler::onExtensionError(const char* provider, const char* extension, int error, const char* errorMessage)
      {
        // Handle the error event
        CString error_message(errorMessage);
        CString message;
        message.Format(_T("Error: %s\n with error code %d"), error_message, error);
        AfxMessageBox(message);
      }

3. Enable the extension and set the extension properties

   To use the extension you downloaded in your app:

   1. Load the extension.
   2. Register the extension.
   3. Enable the extension.
   4. Set the extension property.

   To accomplish these tasks, do the following:

   1. In AgoraImplementationDlg.h, declare setupMarketExtension() before setupVideoSDKEngine();:

      // Function to load, register and enable the extension. Here you also set the extension propertties for your specific extension
      void setupMarketExtension(IRtcEngine* agoraEngine);

   2. In AgoraImplementationDlg.cpp, put the following in the setupVideoSDKEngine function just after initializing Agora Engine agoraEngine->initialize(context);:

   // Integrate extension to the Agora SDK.
   setupMarketExtension(agoraEngine);

   1. Define setupMarketExtension():

      void CAgoraImplementationDlg::setupMarketExtension(IRtcEngine* agoraEngine)
      {
        // Load the extension with the file path to the dynamic library (.dll file on Windows) and a boolean flag 'unload_after_use' is true here mean
        // unload the dynamic library after it has been used.
        agoraEngine->loadExtensionProvider(extensionLibrarayPath,true);
      
        // Register the extension
        agoraEngine->registerExtension(extensionProvider, extensionName, extensionMediaType);
      
        // Enable the extension
        agoraEngine->enableExtension(extensionProvider, extensionName, true, extensionMediaType);
      
        // Set Extension property. The extension key and extension value would be specific to the extension you are using.
        // This function is used to set the value of a specific property of an extension
        agoraEngine->setExtensionProperty(extensionProvider, extensionName, extensionKey, extensionValue, extensionMediaType);
      }

4. Disable the extension before closing the application

   You must unload and disable all extensions before the app closes. As you set unload_after_use to true in loadExtensionProvider(), the extension dynamic link library unloads automatically after it has been used. You disable the extension to prevent any potential memory leaks or unanticipated effects. To do this, put the following in OnClose() before agoraEngine->release();:

   // Disable extension by making 'eanbled' flag as false .
   agoraEngine->enableExtension(extensionProvider, extensionName, false);

Test your implementation

To ensure that you have integrated the extension in your app:

1. Generate a temporary token in Agora Console .

2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

3. In CAgoraImplementationDlg.h, update appId, channelName and token with the values for your temporary token.

4. Set the values for variables and properties in the framework code for your extension in use.

5. In Visual Studio, click Local Windows Debugger. A moment later you see the project running on your development device.

   If this is the first time you run the project, you need to grant microphone and camera access to your app.

6. Experience the features of your new extension.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

• loadExtensionProvider

• registerExtension

• enableExtension

• setExtensionProperty

• MEDIA_SOURCE_TYPE

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.
• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Physical media input devices, such as a camera and a microphone.
• A JavaScript package manager such as npm.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

This section shows you how to implement the video filter extension in your app:

1. Add the required variables

   To integrate the extension, in preload.js, add the following variables to declarations:

   var path = "agora_segmentation_extension";
   var provider = "agora_video_filters_segmentation";
   var extension = "portrait_segmentation";
   var enableExtension = false;

2. Implement the logic to load and enable the extension

   To load the extension in your app, call loadExtensionProvider and pass the extension path. To enable the extension, call enableExtension and pass the provider name and extension name. To implement this workflow, in preload.js, add the following method before window.onload = () =>:

   function enableExtension()
   {
     if (!path) {
       console.log('path is invalid');
       return;
     }
     if (!provider) {
       console.log('provider is invalid');
       return;
     }
     if (!extension) {
       console.log('extension is invalid');
       return;
     }
     agoraEngine.loadExtensionProvider(path);
     agoraEngine.enableExtension(provider, extension, enableExtension);
   };

3. Implement the logic to disable the extension

   To disable the extension, call enableExtension and pass the provider, extension, and enableExtension variables as parameters. The enableExtension variable is set to false to disable the extension. To implement this logic, in preload.js, add the following method before window.onload = () =>:

   function disableExtension()
   {
     agoraEngine.enableExtension(provider, extension, enableExtension);
   };

4. Setup an event listener to enable and disable the extension with a button

   When the user presses Enable Extension, your app loads and starts the extension with a call to the enableExtension method. When the user presses the button again, your app disables the extension with a call to the disableExtension method. To implement this workflow, in preload.js, add the following method before document.getElementById("leave").onclick = async function ():

   document.getElementById("extension").onclick = async function ()
   {
     enableExtension = !enableExtension;
     if(isExtension)
     {
       enableExtension();
       document.getElementById("extension").innerHTML = "Disable Extension";
     }
     {
       disableExtension();
       document.getElementById("extension").innerHTML = "Enable Extension";
     }
   }

5. Implement the required callbacks to manage the extension

   Video SDK provides the following callbacks to manage an extension:

   • onExtensionStarted: Occurs when the extension is enabled.
   • onExtensionError: Occurs when the extension runs incorrectly.
   • onExtensionStopped: Occurs when the extension is disabled.
   • onExtensionEvent: The event callback of the extension.

   To implement these callbacks, in preload.js, add the following callbacks before onUserJoined:

   onExtensionError:( provider, extName, error, msg) =>
   {
     console.log("Error :" + error, "Error detail :" + msg);
   },
   onExtensionEvent:( provider, extName, key, value) =>
   {
   },
   onExtensionStarted:(provider, extName) =>
   {
   },
   onExtensionStopped:(provider, extName) =>
   {
   }

Test your implementation

To ensure that you have integrated the extension in your app:

1. In preload.js, update appID, channel and token with your values.

2. Run the app

   Execute the following command in the terminal:

   npm start

   You see your app opens a window named Get started with Voice Calling.

3. Press Enable Extension. Your app start using the Agora video filter extension.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

• loadExtensionProvider

• setExtensionProperty

• enableExtension

• onExtensionStarted

• onExtensionStopped

• onExtensionError

• onExtensionEvent

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart.

• Flutter 2.0.0 or higher

• Dart 2.15.1 or higher

• Android Studio, IntelliJ, VS Code, or any other IDE that supports Flutter, see Set up an editor.

• If your target platform is iOS:

  • Xcode on macOS (latest version recommended)
  • A physical iOS device
  • iOS version 12.0 or later

• If your target platform is Android:

  • Android Studio on macOS or Windows (latest version recommended)
  • An Android emulator or a physical Android device.

• If you are developing a desktop application for Windows, macOS or Linux, make sure your development device meets the Flutter desktop development requirements.

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Open your Flutter project

   Load the SDK quickstart Voice Calling project you created previously.

4. Get the extension

   Visit the Agora Extensions Marketplace and follow the procedure to download and install the desired extension.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

This section presents the framework code you add to your Flutter project to integrate an extension.

1. Load the extension provider

   You call loadExtensionProvider during Agora Engine initialization to specify the extension library path. To do this, add the following code to the setupVideoSDKEngine method after you initialize the engine with agoraEngine.initialize:

   agoraEngine.loadExtensionProvider("<extensionLibraryPath>");

2. Enable the extension

   To enable the extension, add the following code to the setupVideoSDKEngine method before await agoraEngine.enableVideo();:

   // Extensions marketplace hosts both third-party extensions as well as those developed by Agora. To use an Agora extensions, you do not need to call addExtension or enableExtension.
   agoraEngine.enableExtension(
     provider: "<The name of the extension provider>",
     extension: "<extensionName>",
     type: MediaSourceType.unknownMediaSource,
     enable: true
   );

   If the extension uses the second camera to capture video, set the type parameter to secondaryCameraSource.

   To enable multiple extensions, call enableExtension for each extension. The calls to enableExtension determines the order of each extension in the transmission pipeline. For example, if you enable extension A before extension B, extension A processes data from Video SDK before extension B.

3. Set extension properties

   To customize the extension for your particular app, set suitable values for the extension properties. Refer to the extension documentation for a list of available property names and allowable values. To set a property, add the following code to the setupVideoSDKEngine method after agoraEngine.enableExtension:

   agoraEngine.setExtensionProperty(
     provider: "<providerName>",
     extension: "<extensionName>",
     key: "<keyName>",
     value: "<keyValue>");

Test your implementation

To ensure that you have integrated the extension in your app:

1. Set the variables and properties in the framework code to values suitable for your chosen extension.

2. In your IDE, open /lib/main.dart and update appId, channelName and token with the values from Agora Console.

3. Connect a test device to your development device.

4. In your IDE, click Run app or execute the flutter run command in the terminal. A moment later, you see the project installed on your device.

   If this is the first time you run the app, grant camera and microphone permissions.

5. Join the same channel from another test device or the Agora web demo.

6. Experience the features of your new extension.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

• enableExtension

• loadExtensionProvider

• setExtensionProperty

• onExtensionStarted

• onExtensionStopped

• onExtensionError

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart.

• React Native 0.60 or later. For more information, see Setting up the development environment.
• Node 10 or later
• For iOS
  • A machine running macOS
  • Xcode 10 or later
  • CocoaPods
  • A physical or virtual mobile device running iOS 9.0 or later. If you use React Native 0.63 or later, ensure your iOS version is 10.0 or later.
• For Android
  • A machine running macOS, Windows, or Linux
  • Java Development Kit (JDK) 11 or later
  • Android Studio
  • A physical or virtual mobile device running Android 5.0 or later

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Open your React-native project

   Load the SDK quickstart Voice Calling project you created previously.

4. Get the extension

   Visit the Agora Extensions Marketplace and follow the procedure to download and install the desired extension.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

This section presents the framework code you add to your React Native project to integrate an extension.

Implement the user interface

You can use a switch to enable and disable the extension you wish to integrate. To do this, follow these steps:

1. Import the necessary modules

   Add the following import from react-native before Text,:

   Switch,

2. Render the switch

   Add the following code in the return statement, after Leave </Text>:

   <Switch
     onValueChange={() => {
       if (enable_extension) {
         disableExtension();
         setEnableExtension(false);
       } else {
         enableExtension();
         setEnableExtension(true);
       }
     }}
   />

Integrate the extension

1. Declarations for the extension

   You can create constants for the extension provider and extension name that you need to integrate.

   const extprovider = 'agora_segmentation';
   const extension = 'PortraitSegmentation';

   You also need a state variable to keep track of the status of your extension. Inside App, add the following code along with the other declarations:

   const [enable_extension, setEnableExtension] = useState(false);

2. Enable the extension

   You call enableExtension to enable your extension. To do this using the switch, add the following code after the leave function:

   const enableExtension = () => {
     if (Platform.OS === 'android') {
       agoraEngineRef.current?.loadExtensionProvider(`${extprovider}_extension`);
     }
     agoraEngineRef.current?.enableExtension(extprovider, extension, true);
     setEnableExtension(true);
   };

3. Disable the extension

   You call disableExtension to disable your extension. To do this using the switch, add the following code after the enableExtension function:

   const disableExtension = () => {
     agoraEngineRef.current?.enableExtension(extprovider, extension, false);
     setEnableExtension(false);
   };

4. Add the necessary callbacks

   To get notified of important events, add the following callbacks to agoraEngine.registerEventHandler({:

   onExtensionErrored: (
     provider: string,
     extName: string,
     error: number,
     msg: string,
   ) => {
     console.log(
     'onExtensionErrored',
     'provider',
     provider,
     'extName',
     extName,
     'error',
     error,
     'msg',
     msg,
     );
   },
   onExtensionEvent: (
     provider: string,
     extName: string,
     key: string,
     value: string,
   ) => {
     console.log(
     'onExtensionEvent',
     'provider',
     provider,
     'extName',
     extName,
     'key',
     key,
     'value',
     value,
     );
   },
   onExtensionStarted: (provider: string, extName: string) => {
     console.log(
     'onExtensionStarted',
     'provider',
     provider,
     'extName',
     extName,
     );
     setEnableExtension(true);
   },
   onExtensionStopped: (provider: string, extName: string) => {
     console.log(
     'onExtensionStopped',
     'provider',
     provider,
     'extName',
     extName,
     );
     setEnableExtension(false);
   },

Test your implementation

To ensure that you have integrated the extension in your app:

1. Run your app:

   • Android
   1. Enable Developer options on your Android device, and then connect it to your computer using a USB cable.
   2. Run npx react-native run-android in the project root directory.
   • iOS:
   1. Run your app with Xcode.

   If this is the first time you run the project, you need to grant microphone and camera access to your app.

2. Join the same channel from another test device or the Agora web demo.

3. Experience the features of your new extension.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Extensions are add-ons designed to rapidly extend the functionality of your game. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your game. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

• An Agora account and project.

• A computer with Internet access. Ensure that no firewall is blocking your network communication.

• Implemented the SDK quickstart

• Unity Hub
• Unity Editor 2017.X LTS or higher
• Microsoft Visual Studio 2017 or higher

Project setup

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.
   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

   You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

3. Open your Unity project

   Load the SDK quickstart Voice Calling project you created previously.

4. Get the extension

   Visit the Agora Extensions Marketplace and follow the procedure to download and install the desired extension.

You are now ready to integrate the extension in your game.

Integrate the extension into your project

This section presents the framework code you add to your Unity project to integrate an extension.

1. Load the extension provider

   You call loadExtensionProvider during Agora Engine initialization to specify the extension library path. To do this, add the following code to the SetupVideoSDKEngine method after you initialize the engine with RtcEngine.Initialize:

   RtcEngine.LoadExtensionProvider("<extensionLibraryPath>");

2. Enable the extension

   To enable the extension, add the following code to the Join method before RtcEngine.EnableVideo();:

   RtcEngine.EnableExtension(
     provider: "<The name of the extension provider>",
     extension: "<extensionName>",
     enable: true,
     type: MEDIA_SOURCE_TYPE.UNKNOWN_MEDIA_SOURCE
   );

   Extensions marketplace hosts both third-party extensions as well as those developed by Agora. To use an Agora extension, you do not need to call EnableExtension.

   To enable multiple extensions, call EnableExtension for each extension. The calls to EnableExtension determines the order of each extension in the transmission pipeline. For example, if you enable extension A before extension B, Video SDK processes data from extension A before extension B.

3. Set extension properties

   To customize the extension for your particular game, set suitable values for the extension properties. Refer to the extension documentation for a list of available property names and allowable values. To set a property, add the following code to the Join method after RtcEngine.EnableExtension:

   RtcEngine.SetExtensionProperty(
     provider: "<providerName>",
     extension: "<extensionName>",
     key: "<keyName>",
     value: "<keyValue>"
   );

Test your implementation

To ensure that you have integrated the extension in your game:

1. Set the variables and properties in the project code to values suitable for your chosen extension.

2. Generate a temporary token in Agora Console.

3. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

4. In Unity Editor, in Assets/Agora-RTC-Plugin/Agora-Unity-RTC-SDK/Code/JoinChannelVideo.cs, update _appID, _channelName, and _token with the values for your temporary token.

5. In Unity Editor, click Play. A moment later you see the game installed on your device.

6. Experience the features of your new extension.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference

• EnableExtension

• LoadExtensionProvider

• SetExtensionProperty