Use this guide to quickly start the live interactive video streaming with the Agora Video SDK for Unity.

Prerequisites

  • Unity 2017 or later

  • Operating system and IDE requirements:

    Target Platform Operating system version IDE version
    Android Android 4.1 or later Android Studio 2.0 or later
    iOS iOS 8.0 or later Xcode 8.0 or later
    macOS macOS 10.0 or later Xcode 8.0 or later
    Windows Windows XP SP3 or later Visual Studio 2013 or later
  • A valid Agora account and an App ID

Open the specified ports in Firewall Requirements if your network has a firewall.

Set up the development environment

In this section, we create a Unity project and integrate the SDK into the project.

Create a Unity project

Use the following steps or follow the Unity official manual to build a project from scratch. Skip to Integrate the SDK if you have already created a Unity project.

Create a Unity Project
  1. Ensure that you have downloaded and installed Unity before the following steps. If not, click here to download.

  2. Open Unity and click New.

  3. Fill in and select the options in the following fields, and click Create project.

    • Project name
    • Location
    • Organization
    • Template: Choose 3D

Integrate the SDK

Choose either of the following approaches to integrate the Agora Unity SDK into your project.

Approach 1: Automatically integrate the SDK with Unity Asset Store

  1. Open Asset Store in Unity, search for Agora Video SDK for Unity and click it.
  2. Click Download to download the SDK.
  3. When the download completes, the button becomes Import. Click Import to show the packages of the downloaded SDK.
  4. All packages of the SDK are chosen by default. Uncheck the packages you do not need and click Import.

Approach 2: Manually add the SDK files

  1. Go to SDK Downloads, download the Agora SDK for Unity under Video SDK, and extract the files from the downloaded SDK package.

  2. Create a Plugins folder under the Assets folder in your project.

  3. Copy the following file or subfolder from the libs folder of the downloaded SDK to the corresponding directory of your project.

    Target Platform File or subfolder Directory of your project
    Android /Android/AgoraRtcEngineKit.plugin /Assets/Plugins/Android
    iOS /iOS/AgoraRtcEngineKit.framework /Assets/Plugins/iOS
    macOS /macOS/agoraSdkCWrapper.bundle /Assets/Plugins/macOS
    Windows /x86 /Assets/Plugins/x86
    Windows /x86_64 /Assets/Plugins/x86_64
    • Android or iOS developers using Unity Editor for macOS or Windows must also copy the SDK file for macOS or the subfolder for Windows X86_64 to the specified directory.
    • Android developers also need to copy the AndroidManifest.xml and project.properties from the directory of the sample project (./Assets/Plugins/Android/AgoraRtcEngineKit.plugin) to the directory of the Unity project (./Assets/Plugins/Android). The former is for adding project permissions; the latter is for setting project properties.
    • When integrate the SDK with the version earlier than v3.0.1, iOS developers also need to do the following steps: (You can find the sample code logic in BL_BuildPostProcess.cs, or copy this file to directory of your Unity project)
      • Link the following libraries:
        • CoreTelephony.framework
        • VideoToolbox.framework
        • libresolv.tbd
        • libiPhone-lib.a
        • CoreText.framework
        • Metal.framework
        • CoreML.framework
        • Accelerate.framework
      • Ask for the following permissios:
        • NSCameraUsageDescription
        • NSMicrophoneUsageDescription

Implement a basic live interactive streaming

This section provides instructions on using the Agora Video SDK to start the live interactive video streaming, as well as an API call sequence diagram.

1. Create the UI

Create the user interface (UI) for the interactive video streaming in your project. If you already have one UI in your project, skip to Get the device permission (Android only) or Initialize IRtcEngine.

We recommend adding the following elements to the UI:

  • The view of the host
  • An exit button

If you use the Unity Editor to build your UI, ensure that you bind VideoSurface.cs to the GameObjects designated for local and remote videos.

2. Get the device permission (Android only)

If you build for Android, you should add in APIs to check and request the device permission. For all other platforms, this is handled by the engine, and you can skip to Initialize IRtcEngine.

Unity versions later than UNITY_2018_3_OR_NEWER do not automatically request camera and microphone permissions from your Android device. If you are using one of these versions, call the CheckPermission method to request access to the camera and microphone of your Android device.

#if(UNITY_2018_3_OR_NEWER) 
using UnityEngine.Android; 
#endif 
void Start () 
{  
#if(UNITY_2018_3_OR_NEWER) 
permissionList.Add(Permission.Microphone);  
permissionList.Add(Permission.Camera);  
#endif  
} 
private void CheckPermission()
{ 
#if(UNITY_2018_3_OR_NEWER) 
foreach(string permission in permissionList) 
{ 
if (Permission.HasUserAuthorizedPermission(permission)) 
{ 
} 
else 
{  
Permission.RequestUserPermission(permission); 
} 
} 
#endif 
} 
void Update () 
{  
#if(UNITY_2018_3_OR_NEWER) 
// Ask for your Android device's permissions.
CheckPermission(); 
#endif  
}

3. Initialize IRtcEngine

Initialize the IRtcEngine object before calling any other Agora APIs.

Call the GetEngine method and pass in the App ID to initialize the IRtcEngine object.

If you want to exit the app or release the memory of IRtcEngine, call Destroy to destroy the IRtcEngine object. See details in Destroy the IRtcEngine object.

Listen for callback events, such as when the local user joins the channel, and when the first video frame of a remote user is decoded.

// Pass an App ID to create and initialize an IRtcEngine object.
mRtcEngine = IRtcEngine.GetEngine (appId); 
// Listen for the OnJoinChannelSuccessHandler callback.
// This callback occurs when the local user successfully joins the channel.
mRtcEngine.OnJoinChannelSuccessHandler = OnJoinChannelSuccessHandler; 
// Listen for the OnUserJoinedHandler callback.
// This callback occurs when the first video frame of a remote user is received and decoded after the remote user successfully joins the channel.
// You can call the SetForUser method in this callback to set the remote video.
mRtcEngine.OnUserJoinedHandler = OnUserJoinedHandler; 
// Listen for the OnUserOfflineHandler callback.
// This callback occurs when the remote user leaves the channel or drops offline.
mRtcEngine.OnUserOfflineHandler = OnUserOfflineHandler;

4. Set the channnel profile

After initializing the IRtcEngine object, call the SetChannelProfile method to set the channel profile as LIVE_BROADCASTING.

One IRtcEngine object uses one profile only. If you want to switch to another profile, destroy the current IRtcEngine object with the Destroy method and create a new one before calling the SetChannelProfile method.

// Set the channel profile as LIVE_BROADCASTING.
mRtcEngine.SetChannelProfile(CHANNEL_PROFILE.CHANNEL_PROFILE_LIVE_BROADCASTING);

5. Set the client role

An interactive streaming channel has two client roles: BROADCASTER and AUDIENCE, and the default role is AUDIENCE. After setting the channel profile to LIVE_BROADCASTING, your app may use the following steps to set the client role:

  1. Allow the user to set the role as BROADCASTER or AUDIENCE.
  2. Call the SetClientRole method and pass in the client role set by the user.

Note that in an live interactive streaming, only the host can be heard and seen. If you want to switch the client role after joining the channel, call the SetClientRole method.

// Set the client role as the host.
mRtcEngine.SetClientRole(CLIENT_ROLE.BROADCASTER);

6. Set the local video

After setting the channel profile and client role, set the local video view before joining the channel so that the host can see the local video in the interactive streaming. Follow these steps to configure the local video:

  • Call EnableVideo to enable the video module.

  • Call EnableVideoObserver to get the local video.

      // Enable the video module.
      mRtcEngine.EnableVideo();
      // Get the local video and pass it on to Unity.
      mRtcEngine.EnableVideoObserver();`
  • Choose an object to display the local video, and drag the VideoSurface.cs file to Script, so that you can bind the VideoSurface.cs file to the object and see the local video. Unity renders 3D objects by default, such as Cube, Cylinder and Plane. To render the object in other types, modify the renderer in the VideoSurface.cs file.

      // The default renderer is Renderer.
      Renderer rend = GetComponent(); 
      rend.material.mainTexture = nativeTexture; 
      // Change the renderer to RawImage.
      RawImage rend = GetComponent(); 
      rend.texture = nativeTexture;`

7. Join a channel

After setting the client role and the local video view (for the interactive video streaming), you can call JoinChannelByKey to join a channel. Set the following parameters when calling this method::

  • channelKey: The token for identifying the role and privileges of a user. Set it as one of the following values:

    • NULL.
    • A temporary token generated in Agora Console. A temporary token is valid for 24 hours. For details, see Get a temporary token.
    • A token generated at the server. This applies to scenarios with high-security requirements. For details, see Generate a token from Your Server.
    If your project has enabled the app certificate, ensure that you provide a token.
  • channelName: The unique name of the channel to join. Users that input the same channel name join the same channel.

  • uid: Integer. The unique ID of the local user. If you set uid as 0, the SDK automatically assigns one user ID and returns it in the OnJoinChannelSuccessHandler callback.

If an interactive streaming channel uses both the RTC Native SDK and the RTC Web SDK, ensure that you call the EnableWebSdkInteroperability method before joining the channel.

// Call this method to enable interoperability between the RTC Native SDK and the RTC Web SDK if the RTC Web SDK is in the channel.
mRtcEngine.EnableWebSdkInteroperability(true);
// Join a channel. 
mRtcEngine.JoinChannelByKey(null, channel, null, 0);

8. Set the remote video

In the live interactive video streaming you should also be able to see all the hosts, regardless of your role. This is achieved by calling SetForUser in VideoSurface.cs after joining a channel.

Shortly after a remote host joins the channel, the SDK gets the host's user ID in the OnUserJoinedHandler callback, and you can get the host’s uid.

  • In the VideoSurface.cs file, call the SetForUser method in the callback, and pass in the uid to set the video of the remote user.

  • Drag the VideoSurface.cs script to the target object, so that you can bind the VideoSurface.cs to the object and see the remote video.

  • The video capture and render frame rate of Agora Unity SDK is 15 fps by default. Call SetGameFps in the VideoSurface.cs file to adjust the video refresh rate based on your scenario.

    The default value of uid in the SetForUser method is 0. To see the remote host, ensure that you input the uid of the remote host in SetForUser. Otherwise, the SDK uses the default value and you can only see the local video.
// This callback occurs when the first video frame of a remote host is received and decoded after the remote host successfully joins the channel.
// You can call the SetForUser method in this callback to set up the remote video view.
private void OnUserJoinedHandler(uint uid, int elapsed)
    {
        Debug.Log ("OnUserJoinedHandler: uid = " + uid)
        GameObject go = GameObject.Find (uid.ToString ());
        if (!ReferenceEquals (go, null)) {
            return; 
        }
        go = GameObject.CreatePrimitive (PrimitiveType.Plane);
        if (!ReferenceEquals (go, null)) {
            go.name = uid.ToString ();
            VideoSurface remoteVideoSurface = go.AddComponent<VideoSurface> ();
          // Set the remote video.
          remoteVideoSurface.SetForUser (uid);
          // Adjust the video refreshing frame rate. The video capture and render frame rate of Agora Unity SDK is 15 fps by default.
          // For example, in the game scenario, the refreshing frame rate of the game is 60 fps, which causes 3 times redundancy compared to the video render frame rate. Call the SetGameFps to adjust the refreshing frame rate of the game to 15 fps.
          remoteVideoSurface.SetGameFps (60);
          remoteVideoSurface.SetEnable (true);
        }
        mRemotePeer = uid;
    }

To remove the VideoSurface.cs script from the object, see the following sample codes.

private void OnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
    {
        // Remove video stream.
        // Call this in the main thread.
        GameObject go = GameObject.Find (uid.ToString());
        if (!ReferenceEquals (go, null)) {
            Destroy (go);
        }
    }

9. Leave the channel

According to your scenario, such as when the live interactive streaming ends and when you need to close the app, call LeaveChannel to leave the current channel, and call DisableVideoObserver to disable the video.

public void leave()
    {
        Debug.Log ("calling leave");
        if (mRtcEngine == null)
            return;
        // Leave the channel.
        mRtcEngine.LeaveChannel();
        // Disable the video.
        mRtcEngine.DisableVideoObserver();
    }

10. Destroy the IRtcEngine object

After leaving the channel, if you want to exit the app or release the memory of IRtcEngine, call Destroy to destroy the IRtcEngine object.

void OnApplicationQuit()
{
    if (mRtcEngine != null)
    {
    // Destroy the IRtcEngine object.
    IRtcEngine.Destroy();
    mRtcEngine = null;
    }
}

Run the project

Run the project in Unity. When you set the role as the host and successfully start the live interactive video streaming, you can see the video view of yourself in the app. When you set the role as the audience and successfully join the live interactive video streaming, you can see the video view of the host in the app.

Reference

How can I listen for an audience joining or leaving a live interactive streaming channel?