Audio mixing and sound effects

In Agora Web SDK, if you publish multiple local audio tracks, the SDK automatically mixes these audios. To achieve the effect of mixing audio files with voice, create and publish multiple local audio tracks.

Try out the online demo for Audio effects and music files.

This page shows you how to implement audio mixing and playing sound effects in your app.

Understand the tech

RTC SDK provides APIs that enable you to implement:

• Audio mixing

  Mix in music file such as background music with microphone audio. Using this feature, you can play only one file at a time.

• Sound effects

  Play audios with a short duration. For example, applause, cheers, or gunshots. You can play multiple sound effects at the same time.

Prerequisites

Ensure that you have:

• Implemented the SDK quickstart in your project.

Implement audio mixing

To implement audio mixing in your project, take the following steps:

1. Create a buffer source track

   RTC SDK offers the createBufferSourceAudioTrack method that enables you to read a local or online audio file, and create a corresponding local audio track object BufferSourceAudioTrack:

   // Create an audio track using online music
   const audioFileTrack = await AgoraRTC.createBufferSourceAudioTrack({
     source: "https://web-demos-static.agora.io/agora/smlt.flac",
   });
   console.log("create audio file track success");

   Audio tracks created from audio files follow a different audio data processing flow than microphone audio tracks. After creating a track using an audio file, if you directly call audioFileTrack.play or client.publish([audioFileTrack]), you do not hear the music either locally or remotely.

   You can publish the following audio tracks in the channel.

   1. MicrophoneAudioTrack

      For microphone audio tracks, the SDK continuously collects the latest audio data (AudioBuffer) from the target microphone device.

      

      • After you call the play method, the audio data is sent to the LocalPlayback component, allowing the local user to hear it.

      • When the publish method is called, the audio data is transmitted to Agora SDRTN® and eventually heard by the remote user. The microphone audio track continues to acquire audio data until the close method is invoked, at which point the audio track becomes unavailable.

   2. BufferSourceAudioTrack

      For audio files, the RTC SDK does not collect audio data; instead, it achieves a similar effect by reading files, shown in the following figure.

      

      The key distinctions between collecting audio data and reading files are as follows:

      1. Continuous vs. controlled reading:

         • Collection of audio data is continuous and cannot be paused, focusing on capturing the latest audio data.
         • File reading provides more flexibility. You can pause, jump to specific positions, or loop playback. These operations are core functions of BufferSourceAudioTrack.

      2. Initiating file reading:

         • After creating an audio track from an audio file, the SDK does not automatically read the file. You need to use BufferSourceAudioTrack's startProcessAudioBuffer method to initiate the reading and audio data processing. You then call the play and publish methods to make the audio file audible locally and remotely.

2. Enable audio mixing

   RTC SDK supports publishing multiple audio tracks. You can mix audio by publishing BufferSourceAudioTrack together with the audio tracks created through the microphone. To enable audio mixing, refer to the following code:

   // Create a microphone audio track
   const microphoneTrack = await AgoraRTC.createMicrophoneAudioTrack();
   
   // Start processing audio data from the audio file
   audioFileTrack.startProcessAudioBuffer();
   
   // Publish both the audio file track and the microphone track to begin mixing
   await client.publish([microphoneTrack, audioFileTrack]);
   
   // To stop mixing, either stop processing audio data from the file
   audioFileTrack.stopProcessAudioBuffer();
   
   // Or directly unpublish the audio file track
   await client.unpublish([audioFileTrack]);

3. Manage audio mixing

   To handle audio mixing, refer to the following code:

   // Pause processing audio data
   audioFileTrack.pauseProcessAudioBuffer();
   // Resume processing audio data
   audioFileTrack.resumeProcessAudioBuffer();
   // Stop processing audio data
   audioFileTrack.stopProcessAudioBuffer();
   // Start processing audio data in the loop mode
   audioFileTrack.startProcessAudioBuffer({ loop: true });
   
   // Get the current playback progress (seconds)
   audioFileTrack.getCurrentTime();
   // Total duration of the current audio file (seconds)
   audioFileTrack.duration;
   // Jump to the position at 50 seconds
   audioFileTrack.seekAudioBuffer(50);

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Development notes

Consider the following when developing your app:

• To play online audio files, you also need to configure CORS.
• Supported audio formats are MP3, AAC, and other audio formats supported by the browser.
• Local files only support the browser's native File object.
• Safari versions below 12 do not support mixing, so publishing multiple audio tracks is not possible.
• Irrespective of the number of audio tracks published locally, the SDK automatically mixes them into one audio track, so the remote user only receives one RemoteAudioTrack.

API reference

• createBufferSourceAudioTrack
• BufferSourceAudioTrack
• publish