Terminology and Acronym

Introduction

This document lists all the terminologies and acronyms mentioned in all Agora documents.

Agora Caas

The overall service that Agora products provide is called as Agora Communication as a Service(CaaS).

It provides ensured Quality of Experience for worldwide, Internet-based voice and video communications through the Agora Global Software-Defined Real-time Network(SD-RTN). The network optimizes real-time, mobile communications and solves quality of experience challenges for mobile devices, such as 3G/4G/Wi-Fi networks that perform erratically and Internet bottlenecks worldwide.

It also provides signaling systems with account information system, channel system and call system to establish connections between peers, control conferences and send reliable messages.

Channel

The following figure depicts the channel relations when using Agora services:

../../_images/channel_relation.png

The signal channel and media channel are independent:

Signal Channel It establishes the network session connection between different clients, controls conference, sends reliable callback events, supports adding attributes and etc.
The channel name is set when defining the parameter channelID in calling signaling API channelJoin.
Media Channel It transfers audio and video data for audio/vide communication and live broadcast between different clients.
The channel name is set when defining the parameter channelName or channel in calling media(Agora Native SDK) API joinChannel or joinChannelByKey. [1]

Footnotes

[1]Different platforms may use different parameter names and API names, refer to each platform API references for details.

For example,

  • Users use the Signaling System to send messages, for example, User A has something important to discuss with User B, so A sends an instant message to B: I want to discuss with you face to face. Please join the channel: AgoraTalk. Then B agrees.
  • Both users use the Agora Media SDK to join the AgoraTalk channel(media channel) for audio/video communication(or live broadcast).
../../_images/overview_relation1.png

Account and User ID

When you are calling media and signaling API, you will find keywords like account and uid, the following table explains them in details:

Parameter/Keyword Media Signal
account The account ID used to log onto the client application. The account ID used to log onto the client application.
uid

The unique identifier in a channel on the client application, which is like a room in a house. For example,

If two users in the channel have the same uid, and both of them will be kicked out of channel.

If the users set it as 0, the agora server will automatically assigned a UINT 32 uid.

Two ways to the developers to manage the UID:

  • If the account ID of the user meets the 32-bit UINT requirement, the users can use the account ID directly as uid.
  • If the account ID is a phone number or an email address, not acceptable. Then the users can maintain a mapping relations between the account ID and 32-bit UINT uid.

Set it as 0.

But after the users set it as 0, for example,

when user A calls the signaling API channelJoin, all the other users will receive the callback onChannelUserJoined.

The callback indicates that user A has joined the channel with a internal uid of user A displayed.

The uid is an internal uid generated according to the account ID of the user.

It is recommended that the other users record the the uid of the user A for easy debugging later if any issues occur.

API Method and Callback

Be sure that you have the basic understanding of call mechanism

Agora SDK provides API methods(Engine Interface) and callbacks(Event Callback):

  • Method: The client call the methods to enable certain functions provided by the Agora SDK.
  • Callback: The feedback sent from the Agora SDK to the client when some events happened, including local and remote event callbacks.

Here the remote event callback means the callbacks from the remote user events in the channel. The remote event callbacks are transferred through UDP channel which is not 100% reliable.

It is recommended that the developers use the reliable signaling to pay attention to certain events and status. All the API methods and local events callbacks are reliable except for the remote event callbacks, for example,

Call API Method Local Event Callback Remote Event Callback Description
joinChannel onJoinChannelSuccess onUserJoined The user joined the channel.
leaveChannel onLeaveChannel onUserOffline The user left the channel.
muteLocalAudioStream   onUserMuteAudio The user muted the local audio.
muteLocalVideoStream   onUserMuteVideo The user paused sending local video stream.
disableVideo/enableVideo   onUserEnableVideo The user disabled or enabled the video.

Aside from the callback mechanism provided by Agora, the developers can also do the following:

  • Detecting the connection status between signaling service and client through heartbeat mechanism.
  • Maintaining a client status list at the signaling server side to sync up the client status for the client to query easily later.

Advanced Audience

Users can watch the live broadcast using a mobile/PC application or watch it simply in a web browser.

The audience who watch the live broadcast using an application is called Advanced Audience, while the rest is called as URL Audience.

The Advanced Audience can switch the user role from audience to host, but the URL Audience can not.

Analytics Data

Clients can essentially check all the analytics data on the customer dashboard.

App ID

An static ID that identifies each unique project which you can obtain from https://dashboard.agora.io/projects

For details on the usages, refer to Agora Keys User Guide.

App Certificate

A unique signature for each project that is used together with an App ID to generate a Dynamic Key.

For details on how to get the App Certificate, refer to Agora Keys User Guide.

ARS

Agora Recording Server. It records the communication channel(s).

For details, refer to Recording Guide - Service Beta.

Bypass Live

The users in a live broadcast channel can generate a URL from the PC or Mobile application, and any one can open the URL in a web browser to watch the live broadcast, which is called as Bypass Live.

The users can also share the generated URL on any social media/platform, which is called as Social Sharing.

Call and Hang up

Call and hang-up are the concepts of the signaling layer, that is, the process of two clients negotiating and joining the same channel. The negotiation process is implemented by the signaling, either Agora’s or the developer’s.

After the negotiation, the client calls SDK to join or leave the agreed channel which involves the following APIs:

Interface Description
joinChannel This method allows the user to join a channel.
leaveChannel This method allows the user to leave the channel.
onJoinChannelSuccess This callback indicates the user joined the channel.
onLeaveChannel This callback indicates the user left the channel.

Channel Key

A Dynamic Key generated based on App ID, App Certificate, and etc for the users to join a Agora channel.

For details on the usages, refer to Agora Keys User Guide.

Dynamic Key

If users have high security requirements, it is recommended for the users to use a new key generated dynamically whenever accessing the Agora service. Depending on what service the user accesses, the name for the Dynamic Key varies, for example, Channel Key, Recording Key and etc.

For details on the usages, refer to Agora Keys User Guide.

Error Reporting Mechanism

Agora SDK will return some error codes or warning codes if any issues occur when calling APIs or during runtime:

  • Error Message means that the SDK encountered an error that can be not recovered automatically without application intervention. For example, the SDK returns an error if it fails to open a camera, and then the application needs to remind the user not to use the camera.
  • Warning Message means that the SDK encountered an error that might be recovered automatically. A Warning Code is just for notification, which can usually be ignored.

For details, refer to Error and Warning Messages.

Image Processing Mechanism

According to Agora SDK mechanism, the image will go through the following processes with encoding and render modules to be configured manually if necessary:

Module Description
Capture The camera captures the video raw data.
Encoding The users call the API setVideoProfile to set the resolution.
It is recommended the resolution you set here is similar to the one you select with the camera.
Transmission The users receive and send the encoded data.
Decoding The SDK decodes the encoded data at the recipients’ side.
Render The users call the API setLocalRenderMode or setRemoteRenderMode to set the video display mode in the local or remote. [2]

Footnotes

[2]For Android users, however, they must call the API createRendererView to create the view for video renderer. For details, refer to Android API Reference documentation.

The images are mainly processed at:

  • Capture: Rotation
  • Encoding/Render: Crop or/and Zoom

Capture

The image comes out of the camera may contain rotation information. Rotation only changes the direction without image content or quality loss.

For example, the following image is rotated 90 degrees from 640x480 to 480x640, but the size of the image stays the same.

How much the image can be rotated is not fixed, but it must be multiplied by 90 depending on the physical information of the camera.

../../_images/rotation.png

Encoding/Render

Encoding and Render are similar when it comes to image processing mechanism. Take encoding for example.

The input for encoder is the video or image with certain resolution, which might be quite different.

Use the aspect ratio(width/height) to categorize the resolution, for example,

Aspect Ratio Possible Actual Resolution
16:9 1280x720, 640x360, 320x180, 160x90 …
4:3 1280x960, 640x480, 320x240 …

When the resolution required by the encoder is different from the input, then the input will be resized properly: Crop or/and Zoom.

Step 1: Crop

If the aspect ratio of the input is 16:10, for example, and the encoder requires 16:9, the height of the original image will be shortened according to the following:

../../_images/16.10.png

If the aspect ratio of the input is 16:10 and the encoder requires 16:12, the width of the original image will be shortened according to the following:

../../_images/16.12.png

Step 2: Zoom

After Step 1: Crop , the image meets the aspect ratio requirements of the encoder, but if the actual resolution is not the same as the targeted resolution, then zoom it to the proper size.

For example,

The aspect ratio of both images is 16:9, but the actual sizes are different:

../../_images/16.9.png

The aspect ratio of both images is 16:12, but the actual sizes are different:

../../_images/16.12_new.png

In Channel Permission Key

A Dynamic Key generated based on App ID, App Certificate, and etc for host-in authentication in the live broadcast scenario.

For how to generate and use an In Channel Permission Key, refer to Agora Keys User Guide.

Host in

In the live broadcast scenario, the Advanced Audience can switch the user role from audience to host so that they can interact with the existing hosts in a Agora channel. This process is called as Host in.

With this function, the live broadcast Agora provides is called as interactive broadcast.

Media Server Logs

Logs generated from media servers include items such as call quality metrics (packet loss, bitrates), stream state (added, removed, archived etc), number of subscribers. Entries will be maintained, containing date, time, operation performed (connect, publish, etc.) and etc.

PKI

Public Key Infrastructure.

A public key infrastructure (PKI) is a set of roles, policies, and procedures needed to create, manage, distribute, use, store, and revoke digital Certificates[1] and manage public-key encryption.

Production Machines

They are the servers which are accessible by customer applications in initiating sessions, transmitting media, managing in-session state and recording sessions.

Push Stream

In the live broadcast scenario, by pushing stream to the CDN side, the users can watch the Bypass Live in a web browser, and at the same time, record and store the live broadcast at the CDN side.

Push stream is triggered upon the first host joining a channel, and ended when no stream sent from any host in the channel for over 30 seconds. Push stream is triggered only when the user joins the channel as a host, and it won’t be triggered if the user joined the channel as an audience, then switch to a host in the channel.

Reconnection

The Agora SDK attempts to reconnect the server once it loses connection with the server, and notify the client in the event callbacks. The client reminds of the local user upon the notification, and it will also notify the remote users through signaling, and waits for SDK to reconnect automatically.

The client itself does not need to attempt the reconnection, and the following callbacks will be probably involved:

Local Event Remote Event Description
onJoinChannelSuccess onUserJoined Joined the channel.
onLeaveChannel onUserOffline Left the channel.
onConnectionInterrupted   Lost connection with the server.
onConnectionLost onUserOffline Continual connection loss for over 10 seconds.
onRejoinChannelSuccess onUserJoined Reconnection succeeded.

Recording Key

A Dynamic Key generated based on App ID, App Certificate, and etc for the users to record a communication channel.

For details on the usages, refer to Agora Keys User Guide.

Real-time Interactive Live

Real-time live broadcast with low latency, users from different locations join the same channel from their mobile or PC clients to host in for interaction, which is also called as Real-time Interactive Broadcast or Interactive Broadcast.

Signaling Key

A Dynamic Key generated based on App ID, App Certificate, and etc for the users to log onto the Agora signaling system if the users want to access the signaling system using the Agora Signaling SDK.

For details on the usages, refer to Agora Keys User Guide.

SD-RTN

Software-Defined Realtime Network is a real-time transmission virtual network based on the UDP protocol and is the specific usage of Software Defined Network(SDN) in the real-time transmission scenario.

SD-RTN is mainly used to meet the real-time transmission requirement of the Internet, especially in the case of unstable Internet signal, poor transmission efficiency, to ensure stable transmission and low latency.

SD-RTN is manifested with the following features:

Minimal Latency SD-RTN is essentially a real-time transmission network with the user data in the network unit and transmission lines are transmissed in realtime to ensure the minimal latency.
Transmission Guranteed Arrival Rate

SD-RTN uses the UDP protocol which is designed for real-time transmission to:

  • Gurantee the transmission arrival;
  • Avoid the shortcomings of TCP, for example, uncontrollable latency;
  • Reduce the interaction latency to milliseconds;
Data Security Assurance SD-RTN is based on customized routing. By selecting the optimal transmission path, the content is directly transmissed from end to end. The data in the network unit is not cached to ensure an enhanced security.
Fully Adaptation to Realtime Interactive Requirements The SD-RTN usage scenario is suitable for real-time interactive scenarios that require very low latency, such as Internet telephony, video conferencing, interactive live broadcast with anchor and audience interaction requirements and etc.

Comparison with other networks in realtime

Basic Parameters

  Other Network Agora SD-RTN
Latency The latency is around 5~20 seconds (RTMP) The latency is usually around 200~600 milliseconds, and the maximun delay is no more than 2 seconds.
Interaction One-way interaction with usually one host and many audiences in a single channel. Fully interactive, with up to 7 video hosts and many audiences in a single channel.
Host-in Do not support audio or video communication in real time. The users can host in anytime with audio or video.
Packet Loss Recovery No packet loss recovery mechanism, and when the packet loss rate reaches 30%, the video is frozen completely. Super packet loss mechanism, and when the packet loss rate reaches 30%, the video is still smooth.

Network Transmission Parameters

  Other CDN+RTMP Agora SD-RTN
Transport Layer Protocol TCP protocol, and if reliability is optimized, there will be extra latency. UDP protocol, no extra latency.
Transport Layer Algorithm Open-source algorithm Private algorithm.
Transnational Transmission Different operators, not optimized for realtime transmission. Private virtual network routing based on nearly 100 global network nodes.
Adjust and Monitor Network Change The transmission quality is not reliable, and the user experience is affected when the network is challenging. Network quality monitoring and automatic scheduling system. Optimized transmission quality in real time.

UDP

User Datagram Protocol, which is one of the core members of the Internet protocol suite.

Agora uses UDP as the basic transmission protocol instead of TCP because:

TCP is a reliable transport protocol, which means that when packet loss occurs, the operating system stack will try to retransmit until the transmission is successful or timeout, which is why it is difficult to optimize the network latency using TCP.

UDP is not reliable, even when the network device has lost the message during the transmission, the operating system won’t re-send it. In real-time communication, latency is more important than reliability. When the UDP is used as the basic transmission protocol, Agora uses the private algorithm to determine when to or when not to re-send the message.

For an internet link with packet loss(almost all internal links have packet loss), the latency of TCP varies from hundreds of milliseconds(almost no packet loss) to tens of minutes(over 30% packet loss). For the same internet link, the latency of UDP varies from tens of milliseconds(almost no packet loss) to several seconds(under terrible routing). UDP continues transmissing data when the packet loss reaches 30%, while TCP can not.

TCP is not optimized for realtime communication, while Agora has been continuously optimizing the UDP transmission for realtime communication.