Product Description

Product Description

Introduction

This page provides an overview of Agora.io real-time communications products, including:

The information on this page is specifically intended for use by:

  • Agora.io prospective partners.
  • Software developers interested in Agora.io products and services.

Major Features

This section describes the major features of Agora.io products.

One-to-one Call

One-to-one call is a term used in social apps such as WhatsApp, Hike, and Snapchat, and involves two users communicating with each other.

With Agora.io products, you can easily implement the one-to-one call feature on any app. If user A and user B want to set up a one-to-one call, they only need to enter the same channel. A channel is created when you call an API to join a channel, and is destroyed automatically when both users leave the channel. The following figure shows a typical one-to-one call scenario:

../_images/sei_2host.png

This is a video call, but you can disable video and proceed with a voice call only.

Group Call

A group call is when more than two users chat with one another, like when you chat with a selected group of friends together on WhatsApp. The following figure shows a group call scenario, where three users are involved:

../_images/sei_3host.png

This is a video call, but you can disable video and proceed with a voice call only.

Live Broadcast

Live broadcast is the broadcast of a live performance with an app, where the viewers are called the audience and the one who gives the performance is the host.

Agora.io products allow you to easily implement the live broadcast feature on any app:

  • To give a live broadcast, create a channel and enter it as the host.
  • To view a live broadcast, enter the channel created by the host as the audience.

The following terms are associated with live broadcast:

Term Description
Singleton Broadcast A live broadcast with only one host.
Host In An audience can apply to become a host to interact directly with the existing hosts.
Real-time Interactive Broadcast Global users can join the same channel to “Host In” from their mobile or PC clients.
Stream Pushing [1] The process of pushing a media stream onto the Internet.
CDN live The process of publishing stream onto the CDN (Content Delivery Network) where users can view live broadcast through a web browser.

Footnotes

[1]Stream pushing is triggered when the host first joins the channel. When there is no host in a channel for more than 30 seconds, stream pushing terminates. Stream pushing will not be triggered when an audience switches to a host.

The following figure shows a live broadcast scenario:

../_images/group_live.png

This is a live video broadcast, but you can disable video and proceed with a live voice broadcast only.

Signaling

Signaling implements the following processes:

  • Texting
  • Calling and hanging up
  • Retrieving the user list and sending public messages

Texting

You may also send text messages to each other This process is implemented by signaling; while the actual voice or video call, beginning from joining a channel, falls into the category of communications.

The following figure shows a typical texting scenario:

../_images/p2p_message.png

Calling and Hanging Up

If you are making a call using an app, signaling implements the processes such as sending out a call invitation, receiving a call invitation, taking a call, and hanging up.

The following figure shows a calling scenario.

../_images/call.png

The following figure shows a channel messaging scenario:

../_images/agora_project.png

In this scenario, signaling implements:

  • The retrieval of the user list, which shows how many users there are online
  • Group texting, that is visible to all users in the channel

Note

Terms such as calling, meeting control, and texting refer to signaling. For more information about the differences between the signaling channel and the live broadcast channel, see Channels.

Recording

Communications and live broadcasts are in real time and only available to the users present in the corresponding channels. With the recording feature, you can record and share the contents of the communications or live broadcasts at a later time.

Product Lines

With just one SDK and a set of APIs, you can take advantage of Agora.io globally deployed real-time virtual network.

Communication or Live Broadcast SDK

  • Communication and live broadcast on the same platform share one SDK and a set of APIs. The only difference is whether you set the channel mode to Communication or Live Broadcast when calling the API, setChannelProfile().

  • The Gaming Voice SDK derives from the Media SDK, and enables the implementation of the voice feature in your gaming app with two modes:

    • Free-talk mode (communication)
    • Command mode (live broadcast)
  • Agora.io also provides a web-based SDK to implement voice and video features on the web.

Utilities

You can incorporate the following utilities in the communications or live broadcast SDKs:

  • Whiteboard (PC client)
  • Image Enhancement (mobile client)
  • Recording (all platforms)
  • Signaling (all platforms)

Service Features

This section describes the major features of Agora.io services.

Security

Agora.io provides the following security mechanisms:

  • Login authentication
  • Call encryption

For more information about the security mechanisms associated with Agora.io products and services, refer to security.

Login Authentication

User login normally requires authentication. Agora.io provides static and dynamic App IDs to meet different needs.

After signing up at Dashboard you can create multiple projects. Each of your projects will be assigned a unique identity called an App ID. If someone illegally obtained your App ID, he could use it on any SDK provided by Agora.io. Furthermore, leakage of both your App ID and channel name may put your communication at risk. Therefore, Agora.io recommends that you use a Channel Key, a more secure user identity authentication scheme, on your projects before the official launch.

Note

For information about App ID and Dynamic Key, refer to key.

Call Encryption

Simply by calling one API, you can apply the AES-128 or AES-256 algorithm to Agora.io channels between the clients or between clients and servers.

Quality

Agora.io high-quality services feature:

  • 99.99% service availability.
  • 99.9% throughput rate.
  • End-to-end latency down to 76 ms.
  • A globally-deployed, real-time virtual network that enables transnational and cross-domain transmission, coupled with Agora.io proprietary transmission algorithm, packet-loss countermeasures, and self-adaptive bandwidth control.
  • High-definition graphics, superior speed, and low image freeze (smooth graphics rendering), which are prominent on weak networks in particular. For more information about Agora.io technical advantages in transmission, see Transmitting.

Ease of Use

  • One SDK
  • One set of APIs
  • Support on all-platforms, including Android, iOS, macOS, Windows, and the Web.
  • Support for multiple programming languages, including Java, Objective-C, Swift, and C++.
  • Compatible with more than 5,000 types of devices. (Incompatibility may lead to problems such as chirping, echoes, CPU overload, and high power consumption.)

Core Concepts

This section describes the core concepts to get started with Agora.io products and services.

Security Keys

After signing up at Agora.io official site, you can create multiple projects. Each of your projects will be assigned a unique identity called an App ID. If someone illegally obtained your App ID, he could use it on any SDK provided by Agora.io. Furthermore, leakage of both your App ID and channel name may put your communication at risk. Therefore, Agora.io recommends that you use a Channel Key, a more secure user identity authentication scheme, on your projects before the official launch.

Note

For information about the App ID and Dynamic Key, refer to key.

Channels

A channel is created when you call an API to join a channel, and is destroyed automatically when all users have left the channel.

You will see the following channels when using Agora.io products and services:

Channel Description
Signaling

To transmit signaling, such as to set up Internet sessions between different clients, control meetings, send reliable callback events, and support additional attributes. The channel name is set by the parameter, channelID, in the signaling API.

The signaling channel is a separate channel even if it has the same name as the communications or live broadcast channel.

Media

To transmit media data in communications or live broadcast between different clients. The channel name is set by the parameter, channelName, in the communications or live broadcast API.

The communications and live broadcast channels under the media channel are virtually the same. The only difference is whether you set the channel mode to communications or live broadcast when calling the API, setChannelProfile().

Account and User ID

You may need to set the account and uid parameters when using Agora.io media and signaling channels. The account parameter is the user account of the app in either the communications or the live broadcast service, while the uid parameter varies with the channels:

The uid parameter is a unique identity in a single media channel of a client app. If two users in the same channel have identical uids, both users will drop out of the channel. If a user sets uid to 0, Agora.io server will randomly assign a UINT32 uid to it. You can use the following ways to manage the uid:

  • If a user’s account meets the requirement of the 32-bit UINT, the account can be used directly as the uid.
  • If a user’s account is a phone number or mail account that does not meet the requirements of the 32-bit UNIT, the user can maintain a mapping table between the account and the 32-bit UNIT.

The uid parameter is set to 0 by default. When user A successfully calls the signaling API, channelJoin, all the other users in the corresponding channel will receive an onChannelUserJoined callback, which suggests that user A has joined the channel and returns the internal uid automatically assigned by the system. Agora.io recommends the other users to record user A’s uid to quickly locate and fix issues when they occur.

Voice and Video Processing

Agora.io voice and video processing include:

  • Capturing
  • Pre-processing
  • Encoding
  • Transmitting
  • Decoding
  • Post-processing
  • Rendering

Capturing

Recording sound using a cellphone is a process of voice capturing; taking pictures or video is a process of image or video capturing. Simply put, capturing is the process of collecting information from unprocessed raw voice and video data.

Pre-processing

Pre-processing implies face-glow, face-swapping, and other image enhancement processing.

Encoding

Encoding is the process where the sampled or pre-processed data is transformed into a different format.

Transmitting

Agora.io voice and video transmission relies on its self-built SD-RTN (Software Defined Real-time Network), a virtual and UDP (User Datagram Protocol)-based network architecture designed specifically for real-time communications. By deploying software networking units, which work in synergy with one another, at different data centers across the Internet, Agora.io manages to add a virtual layer. To ensure stable transmission and low latency, particularly on weak networks, the SD-RTN automatically assigns an optimal path according to the following node conditions in a real-time way:

  • Transmission status
  • Load conditions
  • Distance to the users
  • Response time

The following lists the features of the SD-RTN:

  • Ultra-low latency: User data transmits in a real-time way both within the network elements and on the transmission lines to ensure the lowest latency.

  • Guaranteed packet arrival rate: Adopts UDP, a protocol specifically designed for real-time transmissions.

    • Guaranteed transmission arrival rate
    • No uncontrollable latency as in TCP (Transmission Control Protocol)
    • Latency down to within 1 ms
  • Guaranteed data security: Chooses the optimal route based on self-defined routing, transmits data directly from end-to-end without caching data in the network units, and therefore ensures greater data security.

  • Perfect fit for real-time communications: Applies to real-time interactive scenarios requiring ultra-low latency, such as Internet calls, video conferences, and live broadcast involving interaction between the host and the audience.

The following table compares the basic parameters of other networks and Agora SD-RTN:

Parameters Other Networks Agora SD-RTN
Latency Normally between 5 to 20 sec (RTMP). Normally between 200 to 600 ms, no more than 2 sec.
Interaction One-way interaction between the host and a number of audience. Supports host-in between up to seven video hosts and a number of audience.
Host In No support for both voice and video host-in. Supports both voice and video host-in.
Packet Loss No packet loss countermeasures. Image freezes completely when packet loss hits 30%. Superior packet loss countermeasures. Smooth and fluid image when packet loss hits 30%.

The following table compares the network transmission parameters of other networks and Agora SD-RTN:

Parameters Content Delivery Network Agora SD-RTN
Transport Layer Protocol TCP protocol. Can be optimized in terms of reliability, but may introduce additional latency. UDP protocol without additional latency.
Transport Layer Algorithm Open source. Proprietary.
Transnational Transmission Involves different web operators and not optimized for real-time transmission. Proprietary virtual network with nearly 100 networking nodes deployed worldwide. Monitors and adjusts the network in real-time.
Packet Loss Countermeasures Unreliable transmission quality. Poor user experience on weak networks. Monitors the real-time network quality and optimizes traffic accordingly.

Decoding

Decoding is the reverse process of encoding.

Rendering

Rendering is the process that transforms decoded voice or video data into playable audio or graphics. When it comes to graphics rendering, the process may crop or scale the images.

API Methods and Callbacks

The Agora SDK includes a set of API methods (engine interfaces) and callbacks (event callbacks).

  • API Methods: The client calls the API methods to implement the features provided by the Agora SDK.
  • Callback: Feedback is sent from the SDK to the client on a local or remote event that occured. A remote event callback is intiated by a remote user in a channel, and transmitted through the UDP channel which is not 100% reliable.

Note

It is recommended that you use reliable signaling for events and status. All API methods and callbacks listed in the following table are reliable.

API Methods Local Event Callback Remote Event Callback Description
joinChannel onJoinChannelSuccess onUserJoined A user joined the channel.
leaveChannel onLeaveChannel onUserOffline A user left the channel.
muteLocalAudioStream   onUserMuteAudio A user muted the local audio.
muteLocalVideoStream   onUserMuteVideo A user paused sending the local video stream.
disableVideo/enableVideo   onUserEnableVideo A user disabled or enabled the video.

UDP

Agora.io uses UDP rather than TCP as its underlying transport protocol.

TCP is a reliable protocol. When packet loss occurs, the operation system will keep resending the packets until transmission is completed or timeout is reached. Applications using TCP have almost no way of optimizing the resend process to reduce latency.

UDP is an unreliable protocol. When packet loss occurs, the operating system will not resend the packets.

In real-time communications, latency is more important than reliability. Using UDP as the underlying transport protocol, Agora.io can decide on when to resend the packets or not.

On an Internet connection with packet loss, TCP has a latency that ranges from hundreds of milliseconds (in the case of almost no packet loss) to dozens of minutes (when the packet loss rate hits 30%), and a connection that easily breaks (when the packet loss rate hits 50%); on similar links, UDP has a latency that ranges from dozens of milliseconds (in the case of little packet loss) to several seconds (when transmitting through poor routers). To sum up, UDP still works when packet loss exceeds 30%, while TCP does not. TCP is not optimized for real-time communications, but UDP transmission is persistently optimized by Agora.io.