Quickstart - integrate an extension

Updated

Integrate extensions from Extensions Marketplace directly into your app.

Extensions are add-ons designed to rapidly extend the functionality of your app. Extensions Marketplace is home to extensions that make your app more fun. Extensions provide features such as Audio effects and voice changing, Face filters and background removal, and Live transcription and captioning.

In the Agora Extensions Marketplace:

  • Vendors create and publish extensions to provide functionality such as audio and video processing.
  • App developers use extensions to quickly implement fun and interactive functionality.

This page shows you how to integrate an extension from Agora Extensions Marketplace into your app. There can be specific guidance for each extension.

Understand the tech

An extension accesses voice and video data when it is captured from the user's local device, modifies it, then plays the updated data to local and remote video channels.

Extension call workflow

A typical transmission pipeline consists of a chain of procedures, including capture, pre-processing, encoding, transmitting, decoding, post-processing, and play. Audio or video extensions are inserted into either the pre-processing or post-processing procedure, in order to modify the voice or video data in the transmission pipeline.

Prerequisites

To test the code used in this page you need to have:

Project setup

In order to integrate an extension into your project:

  1. Activate an extension

  2. Log in to Agora Console.

  3. In the left navigation panel, click Extension Marketplace, then click the extension you want to activate.

You are now on the extension detail page.

  1. Select a pricing plan and click Buy and Activate.

    • If you have already created an Agora project:

      The Projects section appears and lists all of your projects.

    • If you have not created an Agora project:

      Create a new project, the project appears in the Projects section.

  2. Under Projects on the extension detail page, find the project in which you want to use the extension, then turn on the switch in the Action column.

  3. Get the apiKey and apiSecret for the extension

If required for the extension, to get the extension apiKey and apiSecret, in the Projects extension detail page, click View in the Secret column.

  1. Download the extension

In the extension detail page, click Download, then unzip the extension in a local directory.

  1. Install the extension in your project

Add the extension .dll to your project.

You are now ready to integrate the extension in your app.

Integrate the extension into your project

Integrating an extension into your project involves a few important steps to ensure seamless interaction between the extension and Agora Video SDK. Each of these steps plays a crucial role to ensure that the extension works correctly and efficiently within your Agora project:

  1. Define variables to integrate an extension with Agora Video SDK

    In AgoraImplementationDlg.h, add the following declarations to the CAgoraImplementationDlg class before setupVideoSDKEngine();:

    // File path to the dynamic library (.dll file on Windows) that contains the implementation of the extension provider.
    const char* extensionLibrarayPath = "<extension dynamic library path and name>"; // For example : "/ library / libagora_segmentation_extension.dll"
    const char * extensionProvider = "<extension library path and name>"; // For example : "agora.io";
    const char* extensionName = "<extension name>"; // For example : "libagora_segmentation_extension.dll";
    
    // The key is a string that identifies a specific property of the extension,
    // and value is the new value for that property.
    // What these keys are and what values they can take is entirely up to the extension.
    // For example, an extension might have a property that controls the quality of a video stream, with a key named "videoQuality".
    // To set this property to "high", you would call setExtensionProperty() with "videoQuality" as the key and "high" as the value.
    const char* extensionKey = "<name of the extension property you want to set>";
    const char* extensionValue = "<new value of the property>";
    
    // When using methods that take a MEDIA_SOURCE_TYPE parameter, you would specify the value that corresponds to the type of media source
    // you want the method to operate on. For example, if you want to use the second camera to capture video, set this parameter to "SECONDARY_CAMERA_SOURCE".
    // The default value is UNKNOWN_MEDIA_SOURCE.
    agora::media::MEDIA_SOURCE_TYPE extensionMediaType = agora::media::UNKNOWN_MEDIA_SOURCE; // insert the media souce type as per your context
  2. Register extension callbacks to IRtcEngineEventHandler event handler

  3. In AgoraImplementationDlg.h, add the following declarations to the AgoraEventHandler class after onLeaveChannel:

    virtual void onExtensionEvent(const char* provider, const char* extension, const char* key, const char* value)override;
    virtual void onExtensionStopped(const char* provider, const char* extension)override;
    virtual void onExtensionStarted(const char* provider, const char* extension)override;
    virtual void onExtensionError(const char* provider, const char* extension, int error, const char* errorMessage)override;
  4. Define these callbacks in AgoraImplementationDlg.cpp:

    // Extension Callbacks
    void AgoraEventHandler::onExtensionEvent(const char* provider, const char* extension, const char* key, const char* value)
    {
      // Handle the event
      CString extension_provider(provider);
      CString extension_name(extension);
      CString extension_key(key);
      CString extension_value(value);
    
      CString message;
      message.Format(_T("Extension Provider: %s\nExtension Name: %s\nKey: %s\nValue: %s\n is enabled to run."), extension_provider, extension_name);
      AfxMessageBox(message);
    }
    
    void AgoraEventHandler::onExtensionStopped(const char* provider, const char* extension)
    {
      AfxMessageBox(L"Extension stopped.");
    }
    
    void AgoraEventHandler::onExtensionStarted(const char* provider, const char* extension)
    {
      AfxMessageBox(Extension started);
    }
    
    void AgoraEventHandler::onExtensionError(const char* provider, const char* extension, int error, const char* errorMessage)
    {
      // Handle the error event
      CString error_message(errorMessage);
      CString message;
      message.Format(_T("Error: %s\n with error code %d"), error_message, error);
      AfxMessageBox(message);
    }
  5. Enable the extension and set the extension properties

    To use the extension you downloaded in your app:

    1. Load the extension.
    2. Register the extension.
    3. Enable the extension.
    4. Set the extension property.

    To accomplish these tasks, do the following:

    1. In AgoraImplementationDlg.h, declare setupMarketExtension() before setupVideoSDKEngine();:

      // Function to load, register and enable the extension. Here you also set the extension propertties for your specific extension
      void setupMarketExtension(IRtcEngine* agoraEngine);
    2. In AgoraImplementationDlg.cpp, put the following in the setupVideoSDKEngine function just after initializing Agora Engine agoraEngine->initialize(context);:

    // Integrate extension to the Agora SDK.
    setupMarketExtension(agoraEngine);
    1. Define setupMarketExtension():

      void CAgoraImplementationDlg::setupMarketExtension(IRtcEngine* agoraEngine)
      {
        // Load the extension with the file path to the dynamic library (.dll file on Windows) and a boolean flag 'unload_after_use' is true here mean
        // unload the dynamic library after it has been used.
        agoraEngine->loadExtensionProvider(extensionLibrarayPath,true);
      
        // Register the extension
        agoraEngine->registerExtension(extensionProvider, extensionName, extensionMediaType);
      
        // Enable the extension
        agoraEngine->enableExtension(extensionProvider, extensionName, true, extensionMediaType);
      
        // Set Extension property. The extension key and extension value would be specific to the extension you are using.
        // This function is used to set the value of a specific property of an extension
        agoraEngine->setExtensionProperty(extensionProvider, extensionName, extensionKey, extensionValue, extensionMediaType);
      }
  6. Disable the extension before closing the application

You must unload and disable all extensions before the app closes. As you set unload_after_use to true in loadExtensionProvider(), the extension dynamic link library unloads automatically after it has been used. You disable the extension to prevent any potential memory leaks or unanticipated effects. To do this, put the following in OnClose() before agoraEngine->release();:

// Disable extension by making 'eanbled' flag as false .
agoraEngine->enableExtension(extensionProvider, extensionName, false);

Test your implementation

To ensure that you have integrated the extension in your app:

  1. Generate a temporary token in Agora Console .

  2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

  3. In CAgoraImplementationDlg.h, update appId, channelName and token with the values for your temporary token.

  4. Set the values for variables and properties in the framework code for your extension in use.

  5. In Visual Studio, click Local Windows Debugger. A moment later you see the project running on your development device.

If this is the first time you run the project, you need to grant microphone and camera access to your app.

  1. Experience the features of your new extension.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API reference