Skip to main content
Android
iOS
macOS
Web
Windows
Electron
Flutter
React Native
React JS
Unity
Unreal Engine
Unreal (Blueprint)

Audio mixing and sound effects

Video SDK makes it simple for you to publish audio captured through the microphone to subscribers in a channel. In some real-time audio and video scenarios, such as games or karaoke, you need to play sound effects or mix in music files to enhance the atmosphere and add interest. Video SDK enables you to add sound effects and mix in pre-recorded audio.

This page shows you how to implement audio mixing and playing sound effects in your app.

Understand the tech

Video SDK provides APIs that enable you to implement:

  • Audio mixing

    Mix in music file such as background music with microphone audio. Using this feature, you can play only one file at a time.

  • Sound effects

    Play audios with a short duration. For example, applause, cheers, or gunshots. You can play multiple sound effects at the same time.

Prerequisites

Ensure that you have:

  • Audio files in one of the supported formats.

  • Added the required permission to your project

    If your project's targetSdkVersion is greater than 20, add the following code to the AndroidManifest.xml file:


    _5
    <application>
    _5
    <!-- Other application settings -->
    _5
    android:usesCleartextTraffic="true"
    _5
    android:requestLegacyExternalStorage="true"
    _5
    </application>

Play sound effects and music

This section shows you how to implement playing sound effects and add audio mixing in your app.

To manage audio mixing and sound effects, Video SDK provides the following APIs:

FunctionSound effectAudio mixing
Play or stop playing a specific audio filepreloadEffect
unloadEffect
playEffect
stopEffect
stopAllEffects
startAudioMixing
stopAudioMixing
Pause or resume playing an audio filepauseEffect
pauseAllEffects
resumeEffect
resumeAllEffects
pauseAudioMixing
resumeAudioMixing
Get and adjust playback position and volumesetEffectPosition
getEffectCurrentPosition
getEffectsVolume
setEffectsVolume
setVolumeOfEffect
getAudioMixingCurrentPosition
setAudioMixingPosition
getAudioMixingPublishVolume
adjustAudioMixingPublishVolume
getAudioMixingPlayoutVolume
adjustAudioMixingPlayoutVolume
Report playback status of audio filesonAudioEffectFinishedonAudioMixingStateChanged

Set up file access permissions

For Android projects with targetSdkVersion greater than or equal to 20, add the following to the project's AndroidManifest.xml file:


_4
<application>
_4
android:usesCleartextTraffic="true"
_4
android:requestLegacyExternalStorage="true"
_4
</application>

Play sound effects

To play sound effects, refer to the following code example:


_51
//Import the IAudioEffectManger class
_51
import io.agora.rtc.IAudioEffectManager;
_51
_51
// Call getAudioEffectManager to get the IAudioEffectManager class
_51
private IAudioEffectManager audioEffectManager;
_51
audioEffectManager = mRtcEngine.getAudioEffectManager();
_51
_51
// Set the sound effect ID as a unique identifier for identifying sound effect files
_51
int id = 0;
_51
// If you want to play the sound effect repeatedly, you can preload the file into memory
_51
// If the file is large, do not preload it
_51
// You can only preload local sound effect files
_51
audioEffectManager.preloadEffect(id++, "Your file path");
_51
_51
// Call playEffect to play the specified sound effect file
_51
// Call playEffect multiple times, set multiple sound effect IDs, and play multiple sound effect files at the same time
_51
audioEffectManager.playEffect(
_51
0, // Set the sound effect ID
_51
"Your file path", // Set the sound effect file path
_51
-1, // Set the number of times the sound effect loops. -1 means infinite loop
_51
1, // Set the tone of the sound effect. 1 represents the original pitch
_51
0.0, // Set the spatial position of the sound effect. 0.0 means the sound effect appears directly in front
_51
100, // Set the sound effect volume. 100 represents the original volume
_51
true, // Set whether to publish sound effects to the remote end
_51
0 // Set the playback position of the sound effect file (in ms). 0 means start at the beginning
_51
);
_51
_51
// Pause or resume playing the specified sound effect file
_51
audioEffectManager.pauseEffect(id);
_51
audioEffectManager.resumeEffect(id);
_51
_51
// Set the playback position of the specified local sound effect file
_51
audioEffectManager.setEffectPosition(id, 500);
_51
_51
// Set the playback volume of all sound effect files
_51
audioEffectManager.setEffectsVolume(50.0);
_51
_51
// Set the playback volume of the specified sound effect file
_51
audioEffectManager.setVolumeOfEffect(id, 50.0);
_51
_51
// Release the preloaded sound effect file
_51
audioEffectManager.unloadEffect(id);
_51
_51
// Stop playing all sound effect files
_51
audioEffectManager.stopAllEffects;
_51
_51
@Override
_51
// This callback is triggered when the sound effect file ends playing
_51
public void onAudioEffectFinished(int soundId) {
_51
super.onAudioEffectFinished(soundId);
_51
}

Incorporate audio mixing

Before or after joining a channel, call startAudioMixing to play the audio file. When the audio mixing status changes, the SDK triggers the onAudioMixingStateChanged callback and reports the reason for the change.

To mix in an audio file, refer to the following code example:


_30
// Start playing a music file
_30
mRtcEngine.startAudioMixing(
_30
"Your file path", // Specify the path of the local or online music file
_30
false, // Set whether to play the music file only locally. false means both local and remote users can hear the music
_30
-1, // Set the number of times the music file should be played. -1 indicates infinite loop
_30
0 // Set the starting playback position of a music file
_30
);
_30
_30
@Override
_30
// Triggered when the playback state of the music file changes
_30
// After receiving the onAudioMixingStateChanged callback, call other mixing APIs, such as pauseAudioMixing or getAudioMixingDuration
_30
public void onAudioMixingStateChanged(int state, int errorCode) {
_30
super.onAudioMixingStateChanged(state, errorCode);
_30
}
_30
_30
// Pause or resume playing the music file
_30
rtcEngine.pauseAudioMixing();
_30
rtcEngine.resumeAudioMixing();
_30
_30
// Get the total duration of the current music file
_30
rtcEngine.getAudioMixingDuration();
_30
_30
// Set the playback position of the current music file. 500 indicates starting playback from the 500th ms of the music file
_30
rtcEngine.setAudioMixingPosition(500);
_30
_30
// Adjust the playback volume of the current music file for remote users
_30
rtcEngine.adjustAudioMixingPublishVolume(50);
_30
_30
// Adjust the playback volume of the current music file locally
_30
rtcEngine.adjustAudioMixingPlayoutVolume(50);

Control playback using the following methods:

  • pauseAudioMixing: Pause playback.
  • resumeAudioMixing: Resume playback.
  • stopAudioMixing: Stop playing.
  • setAudioMixingPosition: Set the playing position of the current audio file.
  • adjustAudioMixingPlayoutVolume: Adjust the volume of the current audio file played locally.
  • adjustAudioMixingPublishVolume: Adjust the volume of the current audio file played at the remote end.
caution

If you play a short sound effect file using startAudioMixing, or a long music file using playEffect, the playback may fail.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Interactive Live Streaming