# Optimize audio quality (/en/ai/platform/best-practices/audio-setup)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

> Documentation: Conversational AI.
> Documentation version: Preview.
> This document describes only this version. [Version index](/llms/doc-versions-convoai-guides-v2.txt).


In real-time audio interactions, the rhythm, continuity, and intonation of conversations between humans and AI often differ from those between humans. To improve the AI–human conversation experience, it's important to optimize audio settings.

When users talk to a Conversational AI Platform agent from an Android, iOS, or Web app built with the RTC SDK, follow the best practices in this guide to improve conversation fluency and reliability, especially in complex network environments.

## Server configuration

When you [start a session](/en/ai/platform/get-started/start-stop-agent#start-a-session), the session's audio scenario sets how the agent's audio is processed in the channel. Keep the default, `aiserver`, which is optimized for latency and network resilience in conversations with an agent. Use `chorus` only when participants have good network conditions and need ultra-low latency, or when you record the conversation with Cloud Recording. For details, see [Record agent conversations](/en/ai/platform/best-practices/record-agent-conversation).

For other audio-related settings, use the default values.

## Client configuration

To configure the client app, implement the following:

### Integrate the required dynamic libraries

For the best audio experience with an agent, integrate and load the following dynamic libraries in your project:

**Android**

- AI noise suppression plugin: `libagora_ai_noise_suppression_extension.so`
- AI echo cancellation plug-in: `libagora_ai_echo_cancellation_extension.so`

For integration details, refer to [App size optimization](/en/realtime-media/rtc/build/optimize-and-operate/app-size-optimization/android).

**iOS**

- AI noise suppression plugin: `AgoraAiNoiseSuppressionExtension.xcframework`
- AI echo cancellation plug-in: `AgoraAiEchoCancellationExtension.xcframework`

For integration details, refer to [App size optimization](/en/realtime-media/rtc/build/optimize-and-operate/app-size-optimization/ios).

**Web**

- Integrate the `agora-extension-ai-denoiser` extension. Refer to [AI Noise Suppression](/en/realtime-media/rtc/build/enhance-the-audio-experience/ai-noise-suppression/web).

> **Info**
>
> Optimizing audio uses AI Noise Suppression, which is a paid feature.

## Optimize audio for optimal performance

You can optimize audio settings in the following ways:

- **(Recommended) Use the Toolkit APIs**

  Supported in Video/Voice SDK version 4.5.1 and above.

- **Use the Video/Voice SDK APIs directly**

  Supported in SDK version 4.3.1 and above.

### Use the toolkit APIs

In this solution, you use the toolkit APIs to optimize audio settings.

**Android**

1. **Integrate the toolkit**

   Add the toolkit to your project using Maven or source code. See [Install the Android toolkit](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#installation).

2. **Create a toolkit instance**

   ```kotlin
   val config = ConversationalAIAPIConfig(
       rtcEngine = rtcEngine,
       rtmClient = rtmClient,
       renderMode = TranscriptRenderMode.Word,
       enableLog = true,
       enableRenderModeFallback = true
   )

   val api = ConversationalAIAPIImpl(config)
   ```


3. **Set optimal audio settings**

   Before joining the RTC channel, call `loadAudioSettings()` to apply the optimal audio parameters.

   ```kotlin
   api.loadAudioSettings()
   rtcEngine.joinChannel(token, channelName, null, userId)
   ```


4. **Release resources**

   ```kotlin
   api.destroy()
   ```

**iOS**

1. **Integrate the toolkit**

   Add the toolkit to your project using CocoaPods, Swift Package Manager, or source code. See [Install the iOS toolkit](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#installation).

2. **Create a toolkit instance**

   ```swift
   let config = ConversationalAIAPIConfig(
       rtcEngine: rtcEngine,
       rtmEngine: rtmEngine,
       renderMode: .words,
       enableLog: true,
       enableRenderModeFallback: true
   )

   convoAIAPI = ConversationalAIAPIImpl(config: config)
   ```


3. **Set optimal audio settings**

   Before joining the RTC channel, call `loadAudioSettings()` to apply the optimal audio parameters.

   ```swift
   convoAIAPI.loadAudioSettings()
   rtcEngine.joinChannel(rtcToken: token, channelName: channelName, uid: uid, isIndependent: independent)
   ```


4. **Release resources**

   ```swift
   convoAIAPI.destroy()
   ```

**Web**

- **Integrate the transcript processing toolkit**

  Add the toolkit to your project using a package manager or source code. See [Install the Web toolkit](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#installation). Refer to [Folder structure](#folder-structure) to understand the role of each file.

### Use the SDK APIs

In this solution, you use the RTC SDK to optimize audio settings.

#### Set audio parameters

The settings in this section apply to Video/Voice SDK versions 4.3.1 and above. If you are using an earlier version, upgrade to version 4.5.1 or above or [Contact Technical Support](mailto:support@agora.io).

**Android**

For the best conversational AI audio experience, apply the following settings:

1. **Set the audio scenario**: When initializing the engine, set the audio scenario to the AI client scenario. You can also set the scenario before joining a channel by calling the `setAudioScenario` method.

2. **Configure audio parameters**: Call `setParameters` before joining a channel and whenever the `onAudioRouteChanged` callback is triggered. This configuration sets audio 3A plug-ins (acoustic echo cancellation, noise suppression, and automatic gain control), the audio sampling rate, the audio processing mode, and other settings. For recommended parameter values, refer to the sample code.

   > **Info**
   >
   > Since Video/Voice SDK versions 4.3.1 to 4.5.0 do not support the AI client audio scenario, set the scenario to `AUDIO_SCENARIO_CHORUS` to improve the audio experience. However, the audio experience cannot be aligned with versions 4.5.1 and above. To get the best audio experience, upgrade the SDK to version 4.5.1 or higher.

The following sample code defines a `setAudioConfigParameters` function to configure audio parameters. Call this function before joining a channel and whenever the audio route changes.

```kotlin
private var rtcEngine: RtcEngineEx? = null
private var mAudioRouting = Constants.AUDIO_ROUTE_DEFAULT

// highlight-start
// Set audio configuration parameters
private fun setAudioConfigParameters(routing: Int) {
    mAudioRouting = routing
    rtcEngine?.apply {
        setParameters("{\"che.audio.aec.split_srate_for_48k\":16000}")
        setParameters("{\"che.audio.sf.enabled\":true}")
        setParameters("{\"che.audio.sf.stftType\":6}")
        setParameters("{\"che.audio.sf.ainlpLowLatencyFlag\":1}")
        setParameters("{\"che.audio.sf.ainsLowLatencyFlag\":1}")
        setParameters("{\"che.audio.sf.procChainMode\":1}")
        setParameters("{\"che.audio.sf.nlpDynamicMode\":1}")

        if (routing == Constants.AUDIO_ROUTE_HEADSET // 0
            || routing == Constants.AUDIO_ROUTE_EARPIECE // 1
            || routing == Constants.AUDIO_ROUTE_HEADSETNOMIC // 2
            || routing == Constants.AUDIO_ROUTE_BLUETOOTH_DEVICE_HFP // 5
            || routing == Constants.AUDIO_ROUTE_BLUETOOTH_DEVICE_A2DP) { // 10
            setParameters("{\"che.audio.sf.nlpAlgRoute\":0}")
        } else {
            setParameters("{\"che.audio.sf.nlpAlgRoute\":1}")
        }
        
        setParameters("{\"che.audio.sf.ainlpModelPref\":10}")
        setParameters("{\"che.audio.sf.nsngAlgRoute\":12}")
        setParameters("{\"che.audio.sf.ainsModelPref\":10}")
        setParameters("{\"che.audio.sf.nsngPredefAgg\":11}")
        setParameters("{\"che.audio.agc.enable\":false}")
    }
}
// highlight-end

// Create and initialize the RTC engine
fun createRtcEngine(rtcCallback: IRtcEngineEventHandler): RtcEngineEx {
    val config = RtcEngineConfig()
    config.mContext = AgentApp.instance()
    config.mAppId = ServerConfig.rtcAppId
    config.mChannelProfile = Constants.CHANNEL_PROFILE_LIVE_BROADCASTING
    // highlight-start
    // Set the audio scene to AI dialogue scene (supported by 4.5.1 and above)
    // Version 4.3.1 ~ 4.5.0 is set to chorus scene AUDIO_SCENARIO_CHORUS
    config.mAudioScenario = Constants.AUDIO_SCENARIO_AI_CLIENT
    // Register audio route change callback
    config.mEventHandler = object : IRtcEngineEventHandler() {
        override fun onAudioRouteChanged(routing: Int) {
            super.onAudioRouteChanged(routing)
            // Set audio related parameters
            setAudioConfigParameters(routing)
        }
    }
    // highlight-end
    try {
        rtcEngine = (RtcEngine.create(config) as RtcEngineEx).apply {
            // highlight-start
            // Load the audio plugin
            loadExtensionProvider("ai_echo_cancellation_extension")
            loadExtensionProvider("ai_noise_suppression_extension")
            // highlight-end
        }
    } catch (e: Exception) {
        Log.e("CovAgoraManager", "createRtcEngine error: $e")
    }
    return rtcEngine!!
}

// Join the channel
fun joinChannel(rtcToken: String, channelName: String, uid: Int, isIndependent: Boolean = false) {

    // highlight-start
    // Initialize audio configuration parameters
    setAudioConfigParameters(mAudioRouting)
    // highlight-end

    // Configure channel options and join the channel
    val options = ChannelMediaOptions()
    options.clientRoleType = CLIENT_ROLE_BROADCASTER
    options.publishMicrophoneTrack = true
    options.publishCameraTrack = false
    options.autoSubscribeAudio = true
    options.autoSubscribeVideo = false       
    val ret = rtcEngine?.joinChannel(rtcToken, channelName, uid, options)
}
```

**iOS**

For the best conversational AI audio experience, apply the following settings:

1. **Set the audio scenario**: When initializing the engine, set the audio scenario to the AI client scenario. You can also set the scenario before joining a channel by calling the `setAudioScenario` method.

2. **Configure audio parameters**: Call `setParameters` before joining a channel and whenever the `rtcEngine:didAudioRouteChanged:` callback is triggered. This configuration sets audio 3A plug-ins (acoustic echo cancellation, noise suppression, and automatic gain control), the audio sampling rate, the audio processing mode, and other settings. For recommended parameter values, refer to the sample code.

   > **Info**
   >
   > Since Video/Voice SDK versions 4.3.1 to 4.5.0 do not support the AI client audio scenario, set the scenario to `AgoraAudioScenarioChorus` to improve the audio experience. However, the audio experience cannot be aligned with versions 4.5.1 and above. To get the best audio experience, upgrade the SDK to version 4.5.1 or higher.

The following sample code defines a `setAudioConfigParameters` function to configure audio parameters. Call this function before joining a channel and whenever the audio route changes.

```swift
class RTCManager: NSObject {
    private var rtcEngine: AgoraRtcEngineKit!
    private var audioDumpEnabled: Bool = false
    private var audioRouting = AgoraAudioOutputRouting.default
    
    // highlight-start
    // Set audio related parameters
    private func setAudioConfigParameters(routing: AgoraAudioOutputRouting) {
        audioRouting = routing
        rtcEngine.setParameters("{\"che.audio.aec.split_srate_for_48k\":16000}")
        rtcEngine.setParameters("{\"che.audio.sf.enabled\":true}")
        rtcEngine.setParameters("{\"che.audio.sf.stftType\":6}")
        rtcEngine.setParameters("{\"che.audio.sf.ainlpLowLatencyFlag\":1}")
        rtcEngine.setParameters("{\"che.audio.sf.ainsLowLatencyFlag\":1}")
        rtcEngine.setParameters("{\"che.audio.sf.procChainMode\":1}")
        rtcEngine.setParameters("{\"che.audio.sf.nlpDynamicMode\":1}")
        if routing == .headset ||
            routing == .earpiece ||
            routing == .headsetNoMic ||
            routing == .bluetoothDeviceHfp ||
            routing == .bluetoothDeviceA2dp {
            rtcEngine.setParameters("{\"che.audio.sf.nlpAlgRoute\":0}")
        } else {
            rtcEngine.setParameters("{\"che.audio.sf.nlpAlgRoute\":1}")
        }
        rtcEngine.setParameters("{\"che.audio.sf.ainlpModelPref\":10}")
        rtcEngine.setParameters("{\"che.audio.sf.nsngAlgRoute\":12}")
        rtcEngine.setParameters("{\"che.audio.sf.ainsModelPref\":10}")
        rtcEngine.setParameters("{\"che.audio.sf.nsngPredefAgg\":11}")
        rtcEngine.setParameters("{\"che.audio.agc.enable\":false}")
    }
    // highlight-end
}

extension RTCManager: RTCManagerProtocol {
    
    func createRtcEngine(delegate: AgoraRtcEngineDelegate) -> AgoraRtcEngineKit {
        let config = AgoraRtcEngineConfig()
        config.appId = AppContext.shared.appId
        config.channelProfile = .liveBroadcasting
        // highlight-start
        // Set the audio scene to AI dialogue scene (supported by 4.5.1 and above)
        // Versions 4.3.1 ~ 4.5.0 support chorus scenes .chorus
        config.audioScenario = .aiClient
        rtcEngine = AgoraRtcEngineKit.sharedEngine(with: config, delegate: delegate)
        // Register audio route change callback
        rtcEngine.addDelegate(self)
        // highlight-end
        return rtcEngine
    }
    
    func joinChannel(rtcToken: String, channelName: String, uid: String) {
        
        // highlight-start
        // Initialize audio configuration parameters
        setAudioConfigParameters(routing: audioRouting)
        // highlight-end

        // Configure channel options and join the channel
        let options = AgoraRtcChannelMediaOptions()
        options.clientRoleType = .broadcaster
        options.publishMicrophoneTrack = true
        options.publishCameraTrack = false
        options.autoSubscribeAudio = true
        options.autoSubscribeVideo = false
        let ret = rtcEngine.joinChannel(byToken: rtcToken, channelId: channelName, uid: UInt(uid) ?? 0, mediaOptions: options)           
    }
}

// highlight-start
// Implement the AgoraRtcEngineDelegate interface to handle audio route change callbacks
extension RTCManager: AgoraRtcEngineDelegate {
    public func rtcEngine(_ engine: AgoraRtcEngineKit, didAudioRouteChanged routing: AgoraAudioOutputRouting) {
        setAudioConfigParameters(routing: routing)
    }
}
// highlight-end
```

**Web**

Integrate the `agora-extension-ai-denoiser` extension. Refer to [AI Noise Suppression](/en/realtime-media/rtc/build/enhance-the-audio-experience/ai-noise-suppression/web).

> **Info**
>
> Use Web SDK version 4.15.0 or later.

## Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

### Sample project

Refer to the following open-source sample code to set audio-related parameters.

**Android**

- [`CovRtcManager.kt`](https://github.com/AgoraIO-Community/Conversational-AI-Demo/blob/main/Android/scenes/convoai/src/main/java/io/agora/scene/convoai/rtc/CovRtcManager.kt)

**iOS**

- [`RTCManager.swift`](https://github.com/AgoraIO-Community/Conversational-AI-Demo/blob/main/iOS/Scenes/ConvoAI/ConvoAI/ConvoAI/Classes/Manager/RTCManager.swift)

### Folder structure

**Android**

- `IConversationalAIAPI.kt`: API interface and related data structures and enumerations
- `ConversationalAIAPIImpl.kt`: ConversationalAI API main implementation logic
- `ConversationalAIUtils.kt`: Tool functions and event callback management
- `subRender/`
  - `v3/`: Transcript module
    - `TranscriptionController.kt`: Transcript Controller
    - `MessageParser.kt`: Message Parser

**iOS**

- `ConversationalAIAPI.swift`: API interface and related data structures and enumerations
- `ConversationalAIAPIImpl.swift`: ConversationalAI API main implementation logic
- `Transcription/`
  - `TranscriptionController.swift`: Transcript Controller

**Web**

- `index.ts`: API Class
- `type.ts`: API interface and related data structures and enumerations
- `utils/`
  - `index.ts`: API utility functions
  - `events.ts`: Event management class, which can be extended to easily implement event monitoring and broadcasting
  - `sub-render.ts`: Transcript module

### API reference

**Android**

- SDK
  - `setAudioScenario`
  - `setParameters`
  - `onAudioRouteChanged`
- Toolkit
  - [`loadAudioSettings`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#loadaudiosettings)
  - [`destroy`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#destroy)

**iOS**

- SDK
  - `setAudioScenario`
  - `setParameters`
  - `rtcEngine:didAudioRouteChanged:`
- Toolkit
  - [`loadAudioSettings`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#loadaudiosettings12)
  - [`destroy`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#destroy)

**web**

- Toolkit
  - [`IConversationalAIAPIEventHandlers interface`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#iconversationalaiapieventhandlers-interface)
  - [`EConversationalAIAPIEvents`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#econversationalaiapievents)
  - [`subscribeMessage`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#subscribemessage)
  - [`unsubscribeMessage`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#unsubscribe)
  - [`destroy`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#destroy)