Skip to main content

AI Noise Suppression (beta)

With its AI-powered noise reduction algorithm, you use AI Noise Suppression to suppress hundreds of types of noise and reduce distortion for human voices when multiple people speak at the same time. In scenarios such as online meetings, online chat rooms, video consultations with doctors, and online gaming, AI Noise Suppression makes virtual communication as smooth as face-to-face.

Want to test AI Noise Suppression? Try the online demo.

AI Noise Suppression strategies work on the following:

  • Noise types suppressed

    • Television.
    • Kitchen.
    • Street, such as birds chirping, traffic, and subway sounds.
    • Machine, such as fans, air conditioners, vacuum cleaners, and copiers.
    • Office, such as keyboard and mouse clicks.
    • Household, such as doors opening, creaking chairs, crying babies, and house renovations.
    • Constant knocking.
    • Beeps and clapping.
    • Music.
  • Noise reduction strategies

    • Default: Reduces noise to a comfortable level without distorting the human voice.
    • Custom: If you need a more enhanced or custom noise reduction strategy for your business scenario, contact support@agora.io.

Understand the tech

The typical transmission pipeline for extensions in Video SDK consists of a chain of procedures, including capture, preprocessing, encoding, transmitting, decoding, post-processing, and playback. In the preprocessing stage, AI Noise Suppression modifies data in the pipeline to and uses AI to reduce noise.

Prerequisites

To use the AI Noise Suppression extension, you must meet the following requirements:

  • Implemented Get Started with Video Calling.
  • Noise type matches your business scenario. For example, if you want the microphone to collect background music, the extension is not applicable because it categorizes such background music as noise.

This section explains how to use the latest version of the AI Noise Suppression extension. Implementation for previous versions might be different. For details, see the release notes.

Implementation

To integrate the AI Noise Suppression extension and implement the noise reduction feature, follow these steps:

  1. Integrate AI Noise Suppression into your app

    If you installed the io.agora.rtc:full-sdk when you setup your app, AI Noise Suppression is already included. If not, add the latest version of https://search.maven.org/search?q=a:ains as an implementation in /Gradle Scripts/build.gradle (Module: <project name>.app).

  2. Import the necessary classes

    In app/src/main/java/com.example.<projectname>/MainActivity, add the following lines to import the Android classes used by the extension:


    _1
    import io.agora.rtc2.video.VideoDenoiserOptions;

  3. Initialize the variable to set AI Noise Suppression options

    Add the following line to the top of the MainActivity class:


    _1
    private VideoDenoiserOptions denoiserOptions;

  4. Set the AI Noise Suppression mode

    Auto mode automatically enables or disables the video noise reduction feature according to the ambient light. To set the AI Noise Suppression mode to auto, in setupVideoSDKEngine, add the following code after config.mEventHandler = mRtcEventHandler;:


    _1
    denoiserOptions.denoiserMode = VideoDenoiserOptions.VIDEO_DENOISER_AUTO;

  5. Adjust the AI Noise Suppression level

    By default, AI Noise Suppression is set to 'HighQuality' to balance the AI Noise Suppression quality and performance.

    • To set the value to 'Fast' and prioritize AI Noise Suppression performance over quality, in setupVideoSDKEngine, put this line after denoiserOptions.denoiserMode = VideoDenoiserOptions.VIDEO_DENOISER_AUTO;:


      _1
      denoiserOptions.denoiserLevel = VideoDenoiserOptions.VIDEO_DENOISER_LEVEL_FAST;

    • To give more focus to AI Noise Suppression quality rather than performance, change the level to 'Strength':


      _1
      denoiserOptions.denoiserLevel = VideoDenoiserOptions.VIDEO_DENOISER_LEVEL_STRENGTH;

  6. Set AI Noise Suppression on Agora Engine

    Put the following code after agoraEngine = RtcEngine.create(config);:


    _1
    agoraEngine.setVideoDenoiserOptions(true,denoiserOptions );

Reference

Considerations

Currently, AI Noise Suppression has the following limitations:

  • If the sample rate of the input signal is not 16 KHz, AI Noise Suppression:

    1. Downsamples the signal to 16 KHz.
    2. Removes noise
    3. Resamples the output signal to the original sample rate. This means that audio data above 8 KHz is removed in the output signal.
  • In some unknown scenarios, AI Noise Suppression could cause audio quality to decrease by a certain degree.

  • When multiple people speak at the same time, the audio quality of lowest human voices could be decreased by a certain degree.

  • If only some of the audio tracks on the current web page enable AI Noise Suppression, audio tracks that do not enable AI Noise Suppression could be affected. This is because AI Noise Suppression turns on AEC and AGC and turns off NS in the browser.

  • Best practice is to not support Safar. Although AI Noise Suppression supports Safari v14.1 and greater, there are Safari performance issues for AI Noise Suppression.

  • AI Noise Suppression does not support browsers on mobile devices.

API reference

Page Content

Video Calling