LiveData Conversation Intelligence

This page shows you how to integrate and use the extension in your app.

Understand the tech

The extension is an encapsulation of a cloud real-time speech recognition and translation API. You can integrate its capabilities by passing the specified key and value parameters to the setExtensionProperty method provided by the Video SDK.

Prerequisites

• Android Studio 4.1 or later.
• A physical device (not an emulator) running Android 5.0 or later.
• Integrated the Video SDK and implemented basic video calling, see SDK quickstart.

Project setup

The extension provides a sample project on GitHub:

1. Clone the repository:

   git clone https://github.com/highras/rtvt-agora-marketplace.git

2. Open the sample project in Android Studio.

3. Sync the project with Gradle files.

4. Connect a real Android device and run the project.

Integrate the extension

Purchase and activate the extension

Visit Extensions Marketplace and purchase the extension. Save the appKey and appSecret used to initialize it.

Add the extension package

1. Download the extension package from Extensions Marketplace.

2. Save all .aar files to /app/libs.

3. Add the Gradle dependency:

   implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

Enable the extension

RtcEngineConfig config = new RtcEngineConfig();
config.addExtension("agora-iLiveData-filter");
engine = RtcEngine.create(config);
engine.enableExtension("iLiveData", "RTVT", true);

Start transcription and translation

JSONObject jsonObject = new JSONObject();
jsonObject.put("appKey", "80001000");
jsonObject.put("appSecret", "qwerty");
jsonObject.put("srclang", "zh");
jsonObject.put("dstLang", "en");

engine.setExtensionProperty(
  EXTENSION_VENDOR_NAME,
  EXTENSION_AUDIO_FILTER_VOLUME,
  "startAudioTranslation",
  jsonObject.toString()
);

Get transcription and translation results

After successful initialization, the extension returns transcription and translation results through the onEvent callback.

Stop using the extension

engine.setExtensionProperty(
  EXTENSION_VENDOR_NAME,
  EXTENSION_AUDIO_FILTER_VOLUME,
  "closeAudioTranslation",
  "end"
);

Reference

• RtcEngineConfig.addExtension
• enableExtension
• setExtensionProperty

This page shows you how to integrate and use the extension in your app.

Understand the tech

The extension is an encapsulation of a cloud real-time speech recognition and translation API. You can integrate its capabilities by passing the specified key and value parameters to the setExtensionPropertyWithVendor method provided by the Video SDK.

Prerequisites

• Xcode 9.0 or later.
• A physical device (not an emulator) running iOS 9.0 or later.
• Integrated the Video SDK and implemented basic video calling, see SDK quickstart.

Project setup

1. Clone the sample repository:

   git clone https://github.com/highras/rtvt-agora-marketplace.git

2. Enter iOS/SW_Test and run:

   pod install

3. Open SW_Test.xcworkspace in Xcode.

4. Connect a real iOS device and run the project.

Integrate the extension

Add the extension package

1. Download the extension package from Extensions Marketplace.

2. Save all .framework files to your project folder.

3. Add the dynamic libraries in Xcode and make sure Embed is set to Embed & Sign.

4. Add -ObjC to Other Linker Flags.

   

Enable the extension

AgoraRtcEngineConfig *config = [AgoraRtcEngineConfig new];
config.eventDelegate = self;
self.agoraKit = [AgoraRtcEngineKit sharedEngineWithConfig:config delegate:self];
[self.kit enableExtensionWithVendor:[iLiveDataSimpleFilterManager companyName]
                          extension:[iLiveDataSimpleFilterManager plugName]
                            enabled:YES];

Start transcription and translation

Use setExtensionPropertyWithVendor with the startAudioTranslation key and a JSON string containing appKey, appSecret, srcLanguage, and destLanguage.

Get transcription and translation results

After successful initialization, the extension returns transcription and translation results through the onEvent callback.

Stop using the extension

Use setExtensionPropertyWithVendor with the closeAudioTranslation key.

Reference

• enableExtensionWithVendor
• setExtensionPropertyWithVendor