Audio mixing and sound effects

Video SDK makes it simple for you to publish audio captured through the microphone to subscribers in a channel. In some real-time audio and video use-cases, such as games or karaoke, you need to play sound effects or mix in music files to enhance the atmosphere and add interest. Video SDK enables you to add sound effects and mix in pre-recorded audio.

This page shows you how to implement audio mixing and playing sound effects in your app.

Video SDK provides APIs that enable you to implement:

• Audio mixing

  Mix in music file such as background music with microphone audio. Using this feature, you can play only one file at a time.

• Sound effects

  Play audios with a short duration. For example, applause, cheers, or gunshots. You can play multiple sound effects at the same time.

Ensure that you have:

• Implemented the SDK quickstart in your project.

• Audio files in one of the supported formats.

This section shows you how to implement playing sound effects and add audio mixing in your app.

To manage audio mixing and sound effects, Video SDK provides the following APIs:

Function	Sound effect	Audio mixing
Play or stop playing a specific audio file	preloadEffect unloadEffect playEffect stopEffect stopAllEffects	startAudioMixing stopAudioMixing
Pause or resume playing an audio file	pauseEffect pauseAllEffects resumeEffect resumeAllEffects	pauseAudioMixing resumeAudioMixing
Get and adjust playback position and volume	setEffectPosition getEffectCurrentPosition getEffectsVolume setEffectsVolume setVolumeOfEffect	getAudioMixingCurrentPosition setAudioMixingPosition getAudioMixingPublishVolume adjustAudioMixingPublishVolume getAudioMixingPlayoutVolume adjustAudioMixingPlayoutVolume
Report playback status of audio files	rtcEngineDidAudioEffectFinish	audioMixingStateChanged

Before joining a channel, call preloadEffectto preload the sound effect file. After joining the channel, call playEffect to play the specified sound effect. Call playEffect multiple times to set multiple sound effect IDs and play multiple files simultaneously. After the sound effect has finished playing, the SDK triggers the rtcEngineDidAudioEffectFinish callback.

// Set the effect ID as a unique identifier for the audio effect file

let EFFECT_ID:Int32 = 1

// Specify the path to the audio effect file

let filePath = "your filepath"

// If you want to play the effect repeatedly, you need to preload the audio effect file into memory. Do not preload if the file size is large

// You can only preload local audio effect files

agoraKit.preloadEffect(EFFECT_ID, filePath: filePath)

// Set the number of times the effect will loop. -1 means infinite loop

int loopCount = -1;

// Set the pitch of the effect. 1 means the original pitch

let pitch = 1

// Set the spatial position of the effect. 0.0 means the effect is in front

let pan = 1

// Set the volume of the effect. 100 means the original volume

let gain = 100.0

// Set whether to publish the effect to the remote end

let publish = true

// Call playEffect to play the specified audio effect file

// Multiple calls to playEffect can set multiple effect IDs to play multiple audio effect files simultaneously

agoraKit.playEffect(EFFECT_ID, filePath: filePath, loopCount: Int32(loopCount), pitch: pitch, pan: pan, gain: gain, publish: publish)

// Pause or resume playing the audio effect file

agoraKit.pauseEffect(EFFECT_ID)

agoraKit.resumeEffect(EFFECT_ID)

// Set the playback position of the specified local audio effect file

agoraKit.setEffectPosition(EFFECT_ID, pos: 500)

// Set the playback volume for all audio effect files

agoraKit.setEffectsVolume(volume: 50.0)

// Set the playback volume for the specified audio effect file

agoraKit.setVolumeOfEffect(EFFECT_ID, volume: 50.0)

// Unload the preloaded audio effect file

agoraKit.unloadEffect(EFFECT_ID)

// Pause playing all audio effect files

agoraKit.stopAllEffects()

// Trigger this callback when the audio effect file finishes playing

func rtcEngineDidAudioEffectFinish(_engine: AgoraRtcEngineKit soundId: soundId)

Before or after joining a channel, call startAudioMixing to play the audio file. When the audio mixing status changes, the SDK triggers the audioMixingStateChanged callback and reports the reason for the change.

To mix in an audio file, refer to the following code example:

// Specify the path to the local or online music file

let filePath = "your file path"

// Set whether to play the music file only locally. false means both local and remote users can hear the music file

let loopback = false

// The number of times the music file will loop. 1 means play only once

let cycle = 1

// Set the starting position of the music file playback

let startPos = 0

// Call the startAudioMixing method to play the music file

agoraKit.startAudioMixing(filePath, loopback: loopback, cycle: cycle, startPos: startPos)

// This callback is triggered when the music file playback status changes

// After receiving the localAudioMixingStateDidChanged callback, Agora recommends calling other music mixing APIs, such as pauseAudioMixing or getAudioMixingDuration

func rtcEngine(_ engine: AgoraRtcEngineKit,

audioMixingStateChanged state: AgoraAudioMixingStateType,

reasonCode: AgoraAudioMixingReasonCode) {

}

// Pause or resume playback of the music file

agoraKit.pauseAudioMixing()

agoraKit.resumeAudioMixing()

// Get the total duration of the current music file

agoraKit.getAudioMixingDuration()

// Set the playback position of the current music file. 500 means start playing from 500 ms into the music file

agoraKit.setAudioMixingPosition(pos: 500)

// Adjust the playback volume of the current music file on the remote end

agoraKit.adjustAudioMixingPublishVolume(volume: 50)

// Adjust the playback volume of the current music file locally

agoraKit.adjustAudioMixingPlayoutVolume(volume: 50)

Control playback using the following methods:

• pauseAudioMixing: Pause playback.
• resumeAudioMixing: Resume playback.
• stopAudioMixing: Stop playing.
• setAudioMixingPosition: Set the playing position of the current audio file.
• adjustAudioMixingPlayoutVolume: Adjust the volume of the current audio file played locally.
• adjustAudioMixingPublishVolume: Adjust the volume of the current audio file played at the remote end.

caution

If you play a short sound effect file using startAudioMixing, or a long music file using playEffect, the playback may fail.

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Agora provides an open source AudioMixing project on GitHub for your reference. Download or view the source code for a more detailed example.

• preloadEffect
• playEffect
• rtcEngineDidAudioEffectFinish
• startAudioMixing
• audioMixingStateChanged
• pauseAudioMixing
• resumeAudioMixing
• stopAudioMixing
• adjustAudioMixingPlayoutVolume
• adjustAudioMixingPublishVolume