Agora Native SDK for Windows Reference Manual

Introduction

The Agora Native SDK for Windows allows your code to perform the following operations:

  • Session setup. Join and leave shared Agora conferencing sessions (identified by unique channel names), where there may be many global users conferenced together or simply one other user for one-to-one communication. Your application code should create and manage unique channel names; these usually originate in user, session, and date/time information that your application is already managing.
  • Media control. Enable and disable voice and video (allowing muting) and set various voice and video parameters that help the Agora Native SDK optimize communications. Many of the SDK operations are automated and do not require developer intervention if these parameter settings are not provided.
  • Device control. Access the microphone or speakerphone, set the volume, select from alternative cameras, and set the video window display.
  • Session control. Receive events when other users join or leave a conferencing session (channel), and understanding who’s speaking, who’s muted, and who’s viewing the session. These events allow the application to decide when to make changes to the delivery of video and audio streams, and other application-specific user and status information.
  • Quality management. Obtain statistics and quality indicators about network conditions, run built-in tests, submit ratings and complaints about sessions and enable different levels of error logging.
  • Recording: Record audio or video and audio in one or multiple channels simultaneously. This function is only applicable to the users who have adopted the Recording Key schema.For details, refer to Agora Recording Server.
  • Data management: Encrypt the audio and video packets, modify the audio or video raw data and receive reliable and ordered packets via data channels.
  • Screen Sharing. Share the desktop or window with another user. When the screen sharing function is enalbed, the other person’s screen
  • Whiteboard: Provide a simple collaboration platform on the whiteboard where users from different locations can draw, annotate, and share PDF documents to visualize and simplify the communication. For details, refer to Agora Whiteboard SDK.

The Agora Native SDK for Windows provides a number of C++ classes to deliver various features. For detailed API information, refer to the API Reference - Agora Native SDK for Windows.

  • IRtcEngine class provides all the primary methods that can be invoked by your application.
  • RtcEngineParameters is an auxiliary class that allows various SDK parameters to be easily set.
  • IRtcEngineHandler class enables callback event notifications to your application.
  • IAudioDeviceManager and IAudioDeviceCollection classes help enumerate and test all the audio devices on a system.
  • IVideoDeviceManager and IVideoDeviceCollection classes help enumerate and test all the video devices on a system.

For the following features, refer to Raw Data API.

  • IAudioFrameObserver class modifies the audio raw data.
  • IVideoFrameObserver class modifies the video raw data.

The Agora Native SDK may return an error or warning message when calling API or running. For details, see Agora Native SDK in Error and Warning Messages.

Required Development Environments

  • Microsoft Visual C++ 2008 SP1 or later
  • Microsoft Visual Studio recommended
  • Microsoft Windows VISTA or above

Required Libraries

Agora’s Native SDK for Windows does not rely on any other library. To install and use it:

  • Add the AgoraRtcEngineSDK/include folder to the INCLUDE directory of your project.
  • Add the AgoraRtcEngineSDK/lib folder to the LIB directory of your project, and make sure that mediasdk.lib is linked with your project.
  • Copy all .dll files under AgoraRtcEngineSDK/dll to the directory where your executable file is located.

Getting Started

Obtaining SDK

Download the latest SDK from http://www.agora.io/downloads/ or contact sales@agora.io.

About keys

This section describes the concepts and use of App ID, App Certificate, Dynamic Key, Channel Key and Recording Key. For details, see Agora Keys User Guide.

Creating a New Project

Note

While the examples here show a Microsoft Foundation Class (MFC) project, the Agora Native SDK for Windows does not depend on MFC, and applications can be built in other frameworks without MFC.

  1. Create a project in Visual Studio.
  2. Select the type of application, for example, dialog-based as shown below.
../_images/windows-project-2.png
  1. The Visual Studio project becomes available, as shown below.
../_images/windows-project-3.png
  1. Use the following Agora Native SDK header files and library files to enable the use of Agora Native SDK for Windows.

Relative paths vary depending on how you have structured your directories.

#include ”../SDK/include/IAgoraRtcEngine.h”

#pragma lib (lib, ”../SDK/lib/agorartc.lib”)

Refer to API Reference - Agora Native SDK for Windows for a complete reference of the voice and video SDK methods now available to your applications.

Using the Demo Application

Agora provides a demo application in the Agora Native SDK for Windows zip file. AgoraDemo includes basic methods of joining and leaving a call, demonstrates the core operations of Agora Native SDK for Windows and enables audio and video calls with simple method calls. The demo consists of the following files.

  • AgoraDemoSDKDemoAgoraDemo: The source file folder of the demo program
  • AgoraDemoSDKDemoSDKDemo.sln: The project file of the demo program
  • SDK: The SDK framework including DLLs and include files

Note

Visit https://github.com/AgoraLab/ for more Open Source demo applications.

Compiling the Demo Program

  1. Run Visual C++.
  2. Open the .AgoraDemoSDKDemoSDKDemo.sln project file.
../_images/windows-demo-2.png
  1. Press the F5 key. The compiler automatically compiles and runs the demo application.

Executing the Demo Program

The compiler automatically compiles and runs the demo application. The AgoraDemo requires two clients for voice and video communication. One is your Windows demo application, and the other is another Windows system running the same demo. It could also be the AgoraDemo running on an iOS or Android phone, if you have also built the demo for these platforms (from their own Native SDK packages).

  1. Make a voice call:
  1. On both clients (Computer + Computer or Computer + Phone), enter the same App ID and ChannelName (such as 2222, displayed as “room” in the phone demos), and select AudioMode.

    ../_images/windows-demo-3.png
  2. Click OK to enter the voice call. After a successful login, the screen displays the current channel number and UID.

    ../_images/windows-demo-4.png

Because this is a voice call, most of the screen is unused – the other screen areas are there for the video part of the demo.

Several functions are now available to you: Mute / off mute, turn on / off the speaker, leave the channel, and so on.

  1. Make a video call:
  1. On both clients(Computer + Computer or Computer + Phone), enter the same App ID and ChannelName (such as 2222, displayed as “room” in the phone demos) and select VideoMode. You need a camera to send video.
  2. Click OK to enter the video call. After successful login the screen displays the current channel number and UID.
../_images/windows-demo-5.png

Your own video is shown in the left-hand window, as shown, and video from other demo clients connected to the same channel is displayed on in the other windows on your screen when they join the channel.

Several functions are now available to you: mute / off-mute, turn on / off the speaker, turn on / off the camera, leave the channel, and so on.

Encrypting Data

The Agora Native SDK allows your application to encrypt audio and video packets in one of the following ways:

Note

If your application has integrated the built-in encryption of Agora SDK, then users are able to record an encrypted channel.

Using Agora Native SDK Built-in Encryption

Agora Native SDKs support built-in encryption using the AES-128 or AES-256 encryption algorithm. You can call setEncryptionSecret to enable the encryption function and then call setEncryptionMode to set the mode to be used. For details, see API Reference - Agora Native SDK for Windows.

The following diagram depicts the built-in encryption/decryption process:

../_images/agora-encryption.png

Implementing Your Customized Data Encryption Algorithm

The Agora Native SDK allows your application to encrypt audio and video packets by implementing the customized data encryption algorithm of your application.

The following diagram below depicts the data encryption/decryption process:

../_images/developer-encryption.png

These are the steps:

Registering a Packet Observer

Agora Native SDK allows your application to register a packet observer to receive events whenever an audio or video packet is transmitting.

Register a packet observer on your application using the following API:

virtual int registerPacketObserver(IPacketObserver* observer);

The observer must inherit from agora::rtc::IPacketObserver and be implemented in C++. The following is the definition of the IPacketObserver class:

class IPacketObserver
{
public:

        struct Packet
        {
                const unsigned char* buffer;
                unsigned int size;
        };
        /**
        * called by sdk before the audio packet is sent to other participants
        * @param [in,out] packet:
        *      buffer *buffer points the data to be sent
        *      size of buffer data to be sent
        * @return returns true to send out the packet, returns false to discard the packet
        */
        virtual bool onSendAudioPacket(Packet& packet) = 0;
        /**
        * called by sdk before the video packet is sent to other participants
        * @param [in,out] packet:
        *      buffer *buffer points the data to be sent
        *      size of buffer data to be sent
        * @return returns true to send out the packet, returns false to discard the packet
        */


virtual bool onSendVideoPacket(Packet& packet) = 0;
        /**
        * called by sdk when the audio packet is received from other participants
        * @param [in,out] packet
        *      buffer *buffer points the data to be sent
        *      size of buffer data to be sent
        * @return returns true to process the packet, returns false to discard the packet
        */
        virtual bool onReceiveAudioPacket(Packet& packet) = 0;
        /**
        * called by sdk when the video packet is received from other participants
        * @param [in,out] packet
        *      buffer *buffer points the data to be sent
        *      size of buffer data to be sent
        * @return returns true to process the packet, returns false to discard the packet
        */
        virtual bool onReceiveVideoPacket(Packet& packet) = 0;
Implementing a Customized Data Encryption Algorithm

Inherit from agora::rtc::IPacketObserver to implement the customized data encryption algorithm on your application.

The following example uses XOR for data processing. For Agora Native SDK, sending and receiving packets are handled by the different threads, which is why encryption and decryption can use different buffers:

class AgoraPacketObserver : public agora::rtc::IPacketObserver
 {
 public:
     AgoraPacketObserver()
     {
         m_txAudioBuffer.resize(2048);
         m_rxAudioBuffer.resize(2048);
         m_txVideoBuffer.resize(2048);
         m_rxVideoBuffer.resize(2048);
     }
     virtual bool onSendAudioPacket(Packet& packet)
     {
         int i;
         //encrypt the packet
         const unsigned char* p = packet.buffer;
         const unsigned char* pe = packet.buffer+packet.size;


                  for (i = 0; p < pe && i < m_txAudioBuffer.size(); ++p, ++i)
         {
             m_txAudioBuffer[i] = *p ^ 0x55;
         }
         //assign new buffer and the length back to SDK
         packet.buffer = &m_txAudioBuffer[0];
         packet.size = i;
         return true;
     }

     virtual bool onSendVideoPacket(Packet& packet)
     {
         int i;
         //encrypt the packet
         const unsigned char* p = packet.buffer;
         const unsigned char* pe = packet.buffer+packet.size;
         for (i = 0; p < pe && i < m_txVideoBuffer.size(); ++p, ++i)
         {
             m_txVideoBuffer[i] = *p ^ 0x55;
         }
         //assign new buffer and the length back to SDK
         packet.buffer = &m_txVideoBuffer[0];
         packet.size = i;
         return true;
     }

     virtual bool onReceiveAudioPacket(Packet& packet)
     {
         int i = 0;
         //decrypt the packet
         const unsigned char* p = packet.buffer;
         const unsigned char* pe = packet.buffer+packet.size;
         for (i = 0; p < pe && i < m_rxAudioBuffer.size(); ++p, ++i)
         {
             m_rxAudioBuffer[i] = *p ^ 0x55;
         }
         //assign new buffer and the length back to SDK
         packet.buffer = &m_rxAudioBuffer[0];
         packet.size = i;
         return true;
     }

     virtual bool onReceiveVideoPacket(Packet& packet)
     {
         int i = 0;
         //decrypt the packet
         const unsigned char* p = packet.buffer;
         const unsigned char* pe = packet.buffer+packet.size;


                 for (i = 0; p < pe && i < m_rxVideoBuffer.size(); ++p, ++i)
         {
             m_rxVideoBuffer[i] = *p ^ 0x55;
         }
         //assign new buffer and the length back to SDK
         packet.buffer = &m_rxVideoBuffer[0];
         packet.size = i;
         return true;
     }

 private:
     std::vector<unsigned char> m_txAudioBuffer; //buffer for sending audio data
     std::vector<unsigned char> m_txVideoBuffer; //buffer for sending video data

     std::vector<unsigned char> m_rxAudioBuffer; //buffer for receiving audio data
     std::vector<unsigned char> m_rxVideoBuffer; //buffer for receiving video data
 };
Registering Instance

Call registerPacketObserver to register the instance of the agora::rtc::IPacketObserver class implemented by your application.

Modifying Agora Raw Data

Agora raw data interface is an advanced feature provided in the SDK library for users to obtain the audio/video raw data of the SDK engine. You, as a developer, can modify the audio or video data and create special effects to meet special needs of their applications.

You can insert a pre-processing stage before sending the data to the encoder, modifying the captured of video frames or audio signals. You can also insert a post-processing stage before sending the data to the decoder, modifying the received video frames or audio signals.

Agora raw data interface is a C++ interface.

Modifying Audio Raw Data

See IAudioFrameObserver Interface Class for detailed API description. For details, see Raw Data API.

  1. Define AgoraAudioFrameObserver by inheriting IAudioFrameObserver(The IAudioFrameObserver Class is defined in IAgoraMediaEngine.h). You need to realize the following virtual interfaces:

For example,

class AgoraAudioFrameObserver : public agora::media::IAudioFrameObserver
{
  public:
    virtual bool onRecordAudioFrame(AudioFrame& audioFrame) override
    {
      return true;
    }
    virtual bool onPlaybackAudioFrame(AudioFrame& audioFrame) override
    {
      return true;
    }
    virtual bool onPlaybackAudioFrameBeforeMixing(unsigned int uid, AudioFrame& audioFrame) override
    {
      return true;
    }
};

The above example returns true for audio pre-processing or post-processing interfaces. Users can modify the data if necessary:

class IAudioFrameObserver
{
  public:
    enum AUDIO_FRAME_TYPE {
    FRAME_TYPE_PCM16 = 0,  //PCM 16bit little endian
    };
  struct AudioFrame {
    AUDIO_FRAME_TYPE type;
    int samples;  //number of samples in this frame
    int bytesPerSample;  //number of bytes per sample: 2 for PCM 16
    int channels;  // number of channels (data are interleaved if stereo)
    int samplesPerSec;  //sampling rate
    void* buffer;  //data buffer
    };
  public:
    virtual bool onRecordAudioFrame(AudioFrame& audioFrame) = 0;
    virtual bool onPlaybackAudioFrame(AudioFrame& audioFrame) = 0;
    virtual bool onPlaybackAudioFrameBeforeMixing(unsigned int uid, AudioFrame& audioFrame) = 0;
};
  1. Register the audio frame observer to the SDK engine. After creating the IRtcEngine object, and before joining a channel, you can register the object of audio observer.
AgoraAudioFrameObserver s_audioFrameObserver;

agora::util::AutoPtr<agora::media::IMediaEngine> mediaEngine;
mediaEngine.queryInterface(*engine, agora::rtc::AGORA_IID_MEDIA_ENGINE);
if (mediaEngine)
{
  mediaEngine->registerAudioFrameObserver(&s_audioFrameObserver);
}

Note

Here *engine can be passed in by loadAgoraRtcEnginePlugin in Creating Agora Native SDK Plugin.

Modifying Video Raw Data

See IVideoFrameObserver Interface Class for detailed API explanation. For details, see Raw Data API.

  1. Define AgoraVideoFrameObserver by inheriting IVideoFrameObserver(The IVideoFrameObserver Class is defined in IAgoraMediaEngine.h). You need to realize the following virtual interfaces:

    For example,

    class AgoraVideoFrameObserver : public agora::media::IVideoFrameObserver
    {
      public:
        virtual bool onCaptureVideoFrame(VideoFrame& videoFrame) override
        {
          return true;
        }
        virtual bool onRenderVideoFrame(unsigned int uid, VideoFrame& videoFrame) override
        {
          return true;
        }
    };
    

The above example returns true for audio pre-processing or post-processing interfaces. Users can modify the data if necessary:

class IVideoFrameObserver
{
  public:
    enum VIDEO_FRAME_TYPE {
    FRAME_TYPE_YUV420 = 0,  //YUV 420 format
    };
  struct VideoFrame {
    VIDEO_FRAME_TYPE type;
    int width;  //width of video frame
    int height;  //height of video frame
    int yStride;  //stride of Y data buffer
    int uStride;  //stride of U data buffer
    int vStride;  // stride of V data buffer
    void* yBuffer;  //Y data buffer
    void* uBuffer;  //U data buffer
    void* vBuffer;  //V data buffer
    };
  public:
    virtual bool onCaptureVideoFrame(VideoFrame& videoFrame) = 0;
    virtual bool onRenderVideoFrame(unsigned int uid, VideoFrame& videoFrame) = 0;
};
  1. Register the video frame observer to the SDK engine. After creating the IRtcEngine object and enabling the video mode, and before joining a channel, you can register the object of video observer.
AgoraVideoFrameObserver s_videoFrameObserver;

agora::util::AutoPtr<agora::media::IMediaEngine> mediaEngine;
  mediaEngine.queryInterface(*engine,agora::rtc::AGORA_IID_MEDIA_ENGINE);
  if (mediaEngine)
  {
     mediaEngine->registerVideoFrameObserver(&s_videoFrameObserver);
  }

Note

Here *engine can be passed in by loadAgoraRtcEnginePlugin in Creating Agora Native SDK Plugin.

Creating Agora Native SDK Plugin

Agora Native SDK supports loading the third-party plugin by using dynamic link libraries. The usage is as follows:

  1. Create a Windows DLL project, and the DLL file must be created with libapm- as prefix and dll as suffix. For example, libapm-encryption.dll.
  2. Implement and export the related interface functions.
  • The following plugin interface function is mandatory which is to be called when the Agora SDK loads the plugin. It returns 0 upon a successful registration, and the registration fails when it does not return 0.

    extern "C" __declspec(dllexport) __cdecl int loadAgoraRtcEnginePlugin(agora::rtc::IRtcEngine* engine);
    

Sample Code:

extern "C" __declspec(dllexport) __cdecl int loadAgoraRtcEnginePlugin(agora::rtc::IRtcEngine* engine)
{
    agora::util::AutoPtr<agora::media::IMediaEngine> mediaEngine;
    mediaEngine.queryInterface(*engine,agora::rtc::AGORA_IID_MEDIA_ENGINE);
    if (mediaEngine)
    {
        mediaEngine->registerVideoFrameObserver(&s_videoFrameObserver);
    }
  return 0;
}


* The following plugin interface function is optional which is to be called when the Agora SDK removes the plugin.
extern "C" __declspec(dllexport) __cdecl void unloadAgoraRtcEnginePlugin(agora::rtc::IRtcEngine* engine);

Sample Code:

extern "C" __declspec(dllexport) __cdecl void unloadAgoraRtcEnginePlugin(agora::rtc::IRtcEngine* engine)
{
    agora::util::AutoPtr<agora::media::IMediaEngine> mediaEngine;
    mediaEngine.queryInterface(*engine,agora::rtc::AGORA_IID_MEDIA_ENGINE);
    if (mediaEngine)
    {
        mediaEngine->registerVideoFrameObserver(NULL);
    }
}

The process of Agora Native SDK loading the plugin is listed as follows:

  1. The Agora SDK engine searches all matched dynamic link libraries in the format of libapm-xxx.dll in the directory that contains the native libraries during the initialization. For example, libapm-encryption.dll.
  2. The Agora SDK loads the plugin using LoadLibrary.
  3. The user must implement the plugin and export the loadAgoraRtcEnginePlugin interface according to the above instructions. The SDK obtains and calls loadAgoraRtcEnginePlugin for each plugin found. If it returns 0 after calling the loadAgoraRtcEnginePlugin function, then the execution succeeds. Otherwise the SDK will remove the plugin (FreeLibrary). The input parameter of the interface is agora:rtc:: IRtcEngine interface object, which can be used by the plugin in the loadAgoraRtcEnginePlug function to register IPacketObserver, IAudioFrameObserver, IVideoFrameObserver and etc.
  4. When the SDK engine is destroyed, call the optional entry function unloadAgoraRtcEnginePlugin of the plugin.