Skip to main content

Spatial audio

Spatial audio brings theater-like effects to Voice Calling making it seem as if the sound originates from all around the user. Voice SDK provides spatial audio effects that give users an immersive audio experience in scenarios such as e-sports competitions and online conferences. Using Voice SDK.

You can configure the following spatial audio effects:

  • Spatial audio effects for users:

    By setting the local and remote user's spatial positions, you can create an environment that enables users to experience changes in the distance, position, and orientation of other users in real-time. You can also enable your users to experience multiple audio effects such as audio blurring and air absorption.

  • Spatial audio effects for media player:

    By updating the spatial position of the media player, you can add a sense of space to media resources such as background sounds and musical accompaniment. Agora provides local cartesian coordinate system calculation solution for the media player. This solution calculates the relative positions of the local user and the media player through the Voice SDK. You update the spatial coordinates of the local user and the media player, respectively, so that the local user can hear the spatial audio effect of the media player.

Understand the tech

The following figure shows the workflow you need to integrate spatial audio into your app.

Spatial Audio

Prerequisites

In order to follow this procedure you must have:

  • Android Studio 4.1 or higher.
  • Android SDK API Level 24 or higher.
  • A mobile device that runs Android 4.1 or higher.
  • An Agora account and project.

  • A computer with Internet access.

    Ensure that no firewall is blocking your network communication.

  • To ensure a true spatial experience, Agora recommends using an audio device that supports true binaural playback.

Project setup

To create the environment necessary to implement spatial audio into your app, open the project you created in Get Started with Voice Calling.

Add spatial audio to your app

This section shows how to use the Video SDK to implement spatial audio into your app, step-by-step.

Implement the user interface

In a real-word application, you report your local spatial position to a server in your environment and retrieve positions of remote users in the channel from your server. In this simple example, you use a single Button to set the spatial position of a remote user.

To add the button to the UI, in /app/res/layout/activity_main.xml, add the following code before </RelativeLayout>:


_9
<Button
_9
android:id="@+id/SpatialAudioButton"
_9
android:layout_width="wrap_content"
_9
android:layout_height="wrap_content"
_9
android:layout_below="@+id/JoinButton"
_9
android:layout_alignEnd="@id/LeaveButton"
_9
android:layout_alignStart="@id/JoinButton"
_9
android:onClick="updateSpatialAudioPosition"
_9
android:text="Update Spatial Audio Position" />

Handle the system logic

  1. Add the required libraries

    To set up and configure an instance of the spatial audio engine, import the corresponding Agora libraries. In /app/java/com.example.<projectname>/MainActivity, add the following to the list of import statements:


    _3
    import io.agora.spatialaudio.ILocalSpatialAudioEngine;
    _3
    import io.agora.spatialaudio.LocalSpatialAudioConfig;
    _3
    import io.agora.spatialaudio.RemoteVoicePositionInfo;

Implement spatial audio

To implement these features in your app, take the following steps:

  1. Declare the variables you need

    You create an instance of ILocalSpatialAudioEngine to configure spatial audio and set self and remote user positions. In /app/java/com.example.<projectname>/MainActivity, add the following declarations to the MainActivity class:


    _2
    private ILocalSpatialAudioEngine localSpatial;
    _2
    private int remoteUid = 0; // Stores the uid of the remote user

  2. Instantiate and configure the spatial audio engine

    To create an instance of ILocalSpatialAudioEngine at startup take the following steps:

    1. When a user launches the app, you create an instance of ILocalSpatialAudioEngine, configure it and update the user's self position. To do this, add the following method to the MainActivity class:


      _24
      private void configureSpatialAudioEngine() {
      _24
      LocalSpatialAudioConfig localSpatialAudioConfig = new LocalSpatialAudioConfig();
      _24
      localSpatialAudioConfig.mRtcEngine = agoraEngine;
      _24
      localSpatial = ILocalSpatialAudioEngine.create();
      _24
      localSpatial.initialize(localSpatialAudioConfig);
      _24
      _24
      // By default Agora subscribes to the audio streams of all remote users.
      _24
      // Unsubscribe all remote users; otherwise, the audio reception range you set
      _24
      // is invalid.
      _24
      localSpatial.muteLocalAudioStream(true);
      _24
      localSpatial.muteAllRemoteAudioStreams(true);
      _24
      _24
      // Set the audio reception range, in meters, of the local user
      _24
      localSpatial.setAudioRecvRange(50);
      _24
      // Set the length, in meters, of unit distance
      _24
      localSpatial.setDistanceUnit(1);
      _24
      _24
      // Update self position
      _24
      float[] pos = new float[]{0.0F, 0.0F, 0.0F};
      _24
      float[] forward = new float[]{1.0F, 0.0F, 0.0F};
      _24
      float[] right = new float[]{0.0F, 1.0F, 0.0F};
      _24
      float[] up = new float[]{0.0F, 0.0F, 1.0F};
      _24
      localSpatial.updateSelfPosition(pos, forward, right, up);
      _24
      }

    2. To execute this method at startup, add the following line after setupVideoSDKEngine(); to the onCreate method:


      _1
      configureSpatialAudioEngine();

  3. Set the spatial position of a remote user

    To update the spatial position of a remote user:

    1. Define the RemoteVoicePositionInfo and call updateRemotePosition.


      _17
      public void updateSpatialAudioPosition(View view){
      _17
      RemoteVoicePositionInfo positionInfo = new RemoteVoicePositionInfo();
      _17
      _17
      // Set the coordinates in the world coordinate system.
      _17
      // This parameter is an array of length 3
      _17
      // The three values represent the front, right, and top coordinates
      _17
      positionInfo.position = new float[]{2.0F, 4.0F, 0.0F};
      _17
      _17
      // Set the unit vector of the x axis in the coordinate system.
      _17
      // This parameter is an array of length 3,
      _17
      // The three values represent the front, right, and top coordinates
      _17
      positionInfo.forward = new float[]{1.0F, 0.0F, 0.0F};
      _17
      _17
      // Update the spatial position of the specified remote user
      _17
      localSpatial.updateRemotePosition(remoteUid, positionInfo);
      _17
      showMessage("Remote user spatial position updated.");
      _17
      }

    2. To update the spatial position of a specific remote user, you need the uid of that user. Add the following line to the beginning of the onUserJoined callback, to store the remoteUid:


      _1
      remoteUid = uid;

Spatial audio is currently experimental. Contact sales-us@agora.io to activate this function. If you need further support, please contact Agora technical support.

Test your implementation

To ensure that you have implemented spatial audio features into your app:

  1. Generate a temporary token in Agora Console .

  2. In your browser, navigate to the Agora web demo and update App ID, Channel, and Token with the values for your temporary token, then click Join.

  1. In Android Studio, open app/java/com.example.<projectname>/MainActivity and update appId, channelName and token with the values from Agora Console.

  2. Connect an Android device to your development device.

  3. In Android Studio, click Run app. A moment later, you see the project installed on your device.

    If this is the first time you run your app, grant camera and microphone permissions.

  1. Press Join to connect to the same channel as your web demo.
  1. Test spatial audio effects for users

    1. Press Update Spatial Audio Position. Your app updates the position of the remote user in the spatial audio engine.

    2. Listen to the audio of the remote user. You feel that the location of the remote user has shifted.

  2. Test spatial audio effects for media player

    1. To setup spatial audio position of your media player, add Media Playing to your app.

    2. Replace the call to localSpatial.updateRemotePosition in updateSpatialAudioPosition with the following:


      _2
      //To configure special audio for media player use the following statement
      _2
      localSpatial.updatePlayerPositionInfo(mediaPlayer.getMediaPlayerId(), positionInfo);

    3. Press Update Spatial Audio Position

      Your app updates the spatial position of the media player in the spatial audio engine. Listen to the audio of the media player. You feel that the location of the media player has shifted.

Reference

This completes the information in this page, or points you to documentation that explains other aspects about this product.

Page Content

Voice Calling