Integrate an extension

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

• Vendors create and publish extensions to provide functionality such as audio and video processing.
• App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

To test the code used in this page you need to have:

• Implemented either of the following:
  • UI Kit quickstart
  • SDK quickstart

• React Native 0.60 or later. For more information, see Setting up the development environment.
• Node 10 or later
• For iOS
  • A machine running macOS
  • Xcode 10 or later
  • CocoaPods
  • A physical or virtual mobile device running iOS 9.0 or later. If you use React Native 0.63 or later, ensure your iOS version is 10.0 or later.
• For Android
  • A machine running macOS, Windows, or Linux
  • Java Development Kit (JDK) 11 or later
  • Android Studio
  • A physical or virtual mobile device running Android 5.0 or later

• An Agora account and project.

• A computer with Internet access.

  Ensure that no firewall is blocking your network communication.

In order to integrate an extension into your project:

1. Activate an extension

   1. Log in to Agora Console.

   2. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

      You are now on the extension detail page.

   3. Select a pricing plan and click Buy and Activate.

      • If you have already created an Agora project:

        The Projects section appears and lists all of your projects.

      • If you have not created an Agora project:

        Create a new project, the project appears in the Projects section.

   4. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

2. Get the apiKey and apiSecret for the extension

   If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

1. Open your React-native project

   Load the SDK quickstart Interactive Live Streaming project you created previously.

2. Get the extension

   Visit the Agora Extensions Marketplace and follow the procedure to download and install the desired extension.

You are now ready to integrate the extension in your app.

This section presents the framework code you add to your React Native project to integrate an extension.

You can use a switch to enable and disable the extension you wish to integrate. To do this, follow these steps:

1. Import the necessary modules

   Add the following import from react-native before Text,:

   Switch,
   

2. Render the switch

   Add the following code in the return statement, after Leave </Text>:

   <Switch
   

   onValueChange={() => {
   

   if (enable_extension) {
   

   disableExtension();
   

   setEnableExtension(false);
   

   } else {
   

   enableExtension();
   

   setEnableExtension(true);
   

   }
   

   }}
   

   />
   

1. Declarations for the extension

   You can create constants for the extension provider and extension name that you need to integrate.

   const extprovider = 'agora_segmentation';
   

   const extension = 'PortraitSegmentation';
   

   You also need a state variable to keep track of the status of your extension. Inside App, add the following code along with the other declarations:

   const [enable_extension, setEnableExtension] = useState(false);
   

2. Enable the extension

   You call enableExtension to enable your extension. To do this using the switch, add the following code after the leave function:

   const enableExtension = () => {
   

   if (Platform.OS === 'android') {
   

   agoraEngineRef.current?.loadExtensionProvider(`${extprovider}_extension`);
   

   }
   

   agoraEngineRef.current?.enableExtension(extprovider, extension, true);
   

   setEnableExtension(true);
   

   };
   

3. Disable the extension

   You call disableExtension to disable your extension. To do this using the switch, add the following code after the enableExtension function:

   const disableExtension = () => {
   

   agoraEngineRef.current?.enableExtension(extprovider, extension, false);
   

   setEnableExtension(false);
   

   };
   

4. Add the necessary callbacks

   To get notified of important events, add the following callbacks to agoraEngine.registerEventHandler({:

   onExtensionErrored: (
   

   provider: string,
   

   extName: string,
   

   error: number,
   

   msg: string,
   

   ) => {
   

   console.log(
   

   'onExtensionErrored',
   

   'provider',
   

   provider,
   

   'extName',
   

   extName,
   

   'error',
   

   error,
   

   'msg',
   

   msg,
   

   );
   

   },
   

   onExtensionEvent: (
   

   provider: string,
   

   extName: string,
   

   key: string,
   

   value: string,
   

   ) => {
   

   console.log(
   

   'onExtensionEvent',
   

   'provider',
   

   provider,
   

   'extName',
   

   extName,
   

   'key',
   

   key,
   

   'value',
   

   value,
   

   );
   

   },
   

   onExtensionStarted: (provider: string, extName: string) => {
   

   console.log(
   

   'onExtensionStarted',
   

   'provider',
   

   provider,
   

   'extName',
   

   extName,
   

   );
   

   setEnableExtension(true);
   

   },
   

   onExtensionStopped: (provider: string, extName: string) => {
   

   console.log(
   

   'onExtensionStopped',
   

   'provider',
   

   provider,
   

   'extName',
   

   extName,
   

   );
   

   setEnableExtension(false);
   

   },
   

To ensure that you have integrated the extension in your app:

1. Run your app:

   • Android

     1. Enable Developer options on your Android device, and then connect it to your computer using a USB cable.
     2. Run npx react-native run-android in the project root directory.

   • iOS:

     1. Run your app with Xcode.

   If this is the first time you run the project, you need to grant microphone and camera access to your app.

2. Select an option and click Join to start a session.

   • When you join as a Host, the local video is published and played in the app.

3. Join the same channel from another test device or the Agora web demo.

4. Experience the features of your new extension.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.