Audio mixing and sound effects

Video SDK makes it simple for you to publish audio captured through the microphone to subscribers in a channel. In some real-time audio and video use-cases, such as games or karaoke, you need to play sound effects or mix in music files to enhance the atmosphere and add interest. Video SDK enables you to add sound effects and mix in pre-recorded audio.

This page shows you how to implement audio mixing and playing sound effects in your app.

Video SDK provides APIs that enable you to implement:

• Audio mixing

  Mix in music file such as background music with microphone audio. Using this feature, you can play only one file at a time.

• Sound effects

  Play audios with a short duration. For example, applause, cheers, or gunshots. You can play multiple sound effects at the same time.

Ensure that you have:

• Implemented the SDK quickstart in your project.

• Audio files in one of the formats supported by the target development platform:
  • macOS
  • Windows

This section shows you how to implement playing sound effects and add audio mixing in your app.

To manage audio mixing and sound effects, Video SDK provides the following APIs:

Function	Sound effect	Audio mixing
Play or stop playing a specific audio file	preloadEffect unloadEffect playEffect stopEffect stopAllEffects	startAudioMixing stopAudioMixing
Pause or resume playing an audio file	pauseEffect pauseAllEffects resumeEffect resumeAllEffects	pauseAudioMixing resumeAudioMixing
Get and adjust playback position and volume	setEffectPosition getEffectCurrentPosition getEffectsVolume setEffectsVolume setVolumeOfEffect	getAudioMixingCurrentPosition setAudioMixingPosition getAudioMixingPublishVolume adjustAudioMixingPublishVolume getAudioMixingPlayoutVolume adjustAudioMixingPlayoutVolume
Report playback status of audio files	onAudioEffectFinished	onAudioMixingStateChanged

After joining a channel, call playEffect to play the specified sound effect file. Call playEffect multiple times to set multiple sound effect IDs and play multiple files simultaneously. When the playback is finished, the SDK triggers the onAudioEffectFinished callback.

Refer to the following code example:

// Register the sound effect playback completion callback

const EventHandler = {

onAudioEffectFinished: (soundId) => {

console.log(`soundId:${soundId}`);

}

}

rtcEngine.registerEventHandler(EventHandler);

// Playing sound files

rtcEngine.playEffect(

soundId: 0, // Sets the sound ID

filePath: 'your file path',

loopCount: 1, // Sets the number of times the sound effect is looped. 1 means looped once.

pitch: 1.0, // Sets the pitch of the sound effect. 1.0 indicates the original pitch.

pan: 0, // Sets the spatial position of the sound effect. 0.0 means the sound effect appears directly in front

gain: 100, // Sets the sound volume. 100 indicates the original volume.

publish: false, // Sets whether or not to publish the sound to the remote users

startPos: 0, // Sets the playback position of the sound effect file in ms. 0 means playback starts at the beginning.

);

Call the startAudioMixing method to play a music file. When the music mixing state changes after a successful call to this method, the SDK triggers the onAudioMixingStateChanged callback to report the changed music file playback status and the reason for the change.

Refer to the following code example:

// Register the music file's playback state has changed callback

const EventHandler = {

onAudioMixingStateChanged: (state, reason) => {

console.log(`state:${state}, reason:${reason}`);

}

}

rtcEngine.registerEventHandler(EventHandler);

// Playing Music Files

rtcEngine.startAudioMixing(

filePath: 'your file path', // Sets the music file path

loopback: false, // Sets whether music files are only played locally. 'false' indicates that files are published to the remote end, and both local and remote users can hear the music

cycle: -1, // Sets the number of times the music file is played. -1 means infinite loop playback

startPos: 0, // Sets the playback position of the music file in ms. 0 means playback starts st the beginning.

);

caution

If you play a short sound effect file using startAudioMixing, or a long music file using playEffect, the playback may fail.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

• playEffect
• onAudioEffectFinished
• startAudioMixing
• onAudioMixingStateChanged