A

Agora Analytics

Agora Analytics is a site for developers to track and analyze the usage and quality of calls.

Agora Analytics provides an intuitive interface where developers can locate quality issues across several dimensions, including call quality, call duration, and other factors. Developers can also discover root causes and fix them to improve user experience.

Agora also provides RESTful APIs for developers to retrieve the statistics of their calls and use the data in their own applications.

Agora Console

Agora Console is a site for developers to manage Agora projects and services.

Agora Console provides an intuitive interface for developers to make payments, query, manage and accomplish other operations when using Agora services. After registering an Agora account, developers can use the following main features:

  • Manage the account
  • Check and manage Agora projects and services
  • Get an App ID
  • Check call quality and usage
  • Check billing and make payments
  • Manage members and roles

Agora also provides RESTful APIs, so developers can implement some of the above features directly, such as create a project and fetch usage numbers.

Agora RTC SDK

Agora provides the Agora RTC (Real-time Communication) SDK for enabling real-time audio and video communications. By integrating the Agora RTC SDK, developers can add voice call, video call, audio broadcast, and video broadcast functions in their projects.

Based on the different functions and platforms, the RTC SDK is also categorized as follows:

SDK Platform Feature
Voice SDK
  • Native: Android, iOS, macOS, and Windows
  • Third-party framework: Unity
  • Voice call
  • Audio broadcast
Video SDK
  • Native: Android, iOS, macOS, Web, and Windows
  • Third-party framework: Unity, Electron, React Native, and Flutter
  • Voice call
  • Audio broadcast
  • Audio and video call
  • Audio and video broadcast

RTC SDKs for Android, iOS, macOS, and Windows are known as RTC Native SDK. Agora maintains and releases repositories on GitHub for both React Native and Flutter.

In addition to basic real-time audio and video communication, RTC SDK also supports advanced features such as audio mixing, screen sharing, modifying raw data, using external audio and video data, and pushing streams to the CDN.

Developers can use other Agora SDKs or services to implement the following:

  • Enable real-time audio and video recording with the Agora On-premise Recording SDK or Cloud Recording service.
  • Enable playback of online media resource playback with the Agora MediaPlayer Kit plug-in.
  • Enable real-time messaging or signaling with the Agora RTM SDK.

Agora RTM SDK

You can use the Agora RTM (Real-time Messaging) SDK to implement real-time messaging scenarios that require low latency and high concurrency for a global audience.

Agora RTM SDK supports the following platforms:

  • Android
  • iOS
  • macOS
  • Linux Java
  • Linux C++
  • Windows C++
  • Web

Agora RTM SDK is also known as RTM Native SDK for all of these platforms except for Web.

You can use the Agora RTM SDK along with the Agora RTC SDK or a third-party audio-and-video SDK. The Agora RTM SDK can find uses in live streaming, social, education, and IoT scenarios. Integrating the Agora RTM SDK enables the following functions:

  • Real-time commentaries
  • Whiteboard
  • Call invitation
  • Privilege management
  • Subscription
  • Remote control.

App ID

App ID is a random string created within Agora Console and is the unique identifier of an app.

Agora uses App ID to identify each app and provides billing and other statistical data services based on it. After signing up within Agora Console, you can create multiple apps, each with a unique App ID. When initializing a client, you need to pass an App ID as an argument. Clients created by different App IDs cannot communicate with each other.

For situations requiring high security, such as in a production environment, you must use a token for user authentication; otherwise, your environment is open to anyone who has your App ID.

App Certificate

An App Certificate is a randomly-generated string. It enables token authentication and is one of the required arguments for generating a token.

Choose either of the following methods to enable an App Certificate:

  • If you choose the App ID + App Certificate + Token schema when creating a new project within Agora Console, then your App Certificate is automatically enabled and you must use token authentication for your app.
  • If you choose the App ID schema when creating a new project within Agora Console and wish to use the token schema at a later point, find your project on the project management page and then enable its App Certificate. See Enable App Certificate.
Please keep your App Certificate safe. Store it on your business server, not on the client. If it is stolen, we cannot guarantee the security of your app.

audience

In a live-broadcast channel, an audience is a group of users who can only subscribe to streams. The audience cannot publish streams.

audience (becoming)

Becoming an audience describes a scenario within a live-broadcast channel (the channel profile is live broadcast) when a host switches the user role and becomes an audience.

This former host can no longer publish audio and video streams, and all users in the channel can no longer hear and see this former host.

audio mixing

Audio mixing is the process that developers mix multiple audio streams into only one audio stream. Common audio mixing scenarios are as follows:

  • During real-time communication, when a user simultaneously speaks and plays a music file in a channel, after audio mixing, all users in the channel will hear the user's mixed voice.
  • In a group call, any voice heard by the local user is the mixed audio stream of all remote users.
  • When use composite recording mode to record the audio or video, the recorded file contains all users' audio streams after mixing.

C

CDN live streaming

The process of publishing streams into the CDN (Content Delivery Network) is called CDN live streaming, where users can view the live broadcast through a web browser.

channel

A channel is created by a developer calling the methods provided by Agora for transmitting real-time data.

Agora uses different channels to transmit different types of data. The RTC channel transmits audio or video data, while the RTM channel transmits messaging or signaling data. RTC channels and RTM channels are independent of each other.

Additional components provided by Agora, such as On-premise Recording and Cloud Recording, can join the RTC channel and provide real-time recording, transmission acceleration, media playback, and content moderation.

Agora identifies channels by channel name. Users with the same channel name join the same channel and interact with each other. A channel no longer exists when the last user leaves the channel.

To ensure communication security, users should provide a token for authentication when joining a channel. When the token expires, the user can no longer access Agora services.

channel profile

The SDK applies different optimization methods according to the channel profile. Agora supports the following channel profiles:

Channel profile Description
Communication One-on-one or group calls, where all users in the channel can talk freely.
Live Broadcast In a live broadcast channel, users have two client roles: Host and audience. The host sends and receives audio/video, and the audience receives audio/video with the sending function disabled.
Gaming Any user in the channel can talk freely. This profile uses the codec with low-power consumption and low bitrate by default.
The gaming profile applies to the Agora Gaming SDK only.

Cloud Recording

Cloud Recording is a component provided by Agora for recording and saving voice and video calls and interactive broadcasts on a third-party cloud storage through RESTful APIs.

Developers can quickly and flexibly record one-to-one or one-to-many voice and video calls or interactive broadcasts through integration. Unlike Agora On-premise Recording, Agora Cloud Recording does not require a Linux server. Developers can use Cloud Recording for scenarios such as online classrooms, interactive broadcasts, customer service, and more.

Cloud Recording RESTful APIs are compatible with the Agora RTC SDK. Cloud Recording supports two recording modes: individual and composite.

co-hosting

Co-hosting describes a scenario with more than one host. All hosts can interact with each other. The audience can hear and see these hosts.

There are two possible scenarios:

Co-hosting in a channel. It can either be multiple hosts joining a channel, or an audience switching user role and becoming a co-host when another host is already present.
Co-hosting across channels. The Agora RTC SDK supports relaying the media stream of a host in one channel to other channels. For details, see Co-host across Channels. For the maximum number of co-hosts possible in a channel, see How many users can Agora RTC SDK support at the same time? A channel with too many co-hosts may cause network latency and packet loss. Refer to Video for 7+ Users to ensure communication quality.

composite recording mode

Composite recording mode generates a single mixed audio and video file for all UIDs in a channel. It is one of the recording modes for both On-premise Recording and Cloud Recording.

For example, if there are three UIDs sending audio and video in a channel, composite recording mode generates a single mixed audio and video file.

On-premise recording also enables developers to generate a separate audio and video file for all UIDs in a channel, instead of combining audio and video into a single file.

Reference:
  • Composite Recording (Cloud Recording)
  • Composite Recording (On-premise Recording)
  • custom rendering

    Custom rendering is the process where developers collect raw data from the SDK and process it according to specific needs.

    When the default audio or video renderer cannot meet requirements, developers can use an external audio or video renderer to render raw data. Common custom rendering scenarios are as follows:

    • When storing the raw data in other engine for rendering, developers need to use the custom rendering function.
    • When rendering animation, developers can use the custom rendering function.
    • To avoid any conflict between real-time communication and other processes that may be working in parallel, developers can use an external audio or video renderer to render raw data.

    D

    dual-stream mode

    In the dual-stream mode, the SDK transmits a high-quality and a low-quality video stream from the sender. The high-quality video stream has a higher resolution and bitrate, and the low-quality video stream has a lower resolution and bitrate.

    Subscribing to low-quality streams improves communication continuity because this reduces bandwidth consumption. Developers can choose to receive low-quality video streams when network condition are unreliable, or when multiple users publish streams.

    The SDK sets the default video profile of the low-quality video stream based on that of the high-quality video stream.

    When customizing the video profile of the low-quality video stream, the aspect ratio of the low-quality video stream must match that of the high-quality video stream.

    H

    high-quality video stream

    In the dual-stream mode, the SDK transmits two video streams of differing quality at the same time. The high-quality video stream has a higher resolution and bitrate than the low-quality video stream. See dual-stream mode for details.

    host

    In a live broadcast channel, the broadcasters or hosts are users who can publish and subscribe to streams.

    host (becoming)

    Becoming a host describes a scenario within a live-broadcast channel (the channel profile is live broadcast) when an audience switches the user role and becomes a host (broadcaster).

    If there is already a host in the channel, the new host becomes a co-host. This new host or co-host can publish audio and video streams. All users in the channel hear and see this co-host.

    I

    individual recording mode

    Individual recording mode records audio and video of each UID as separate files. It is one of the recording modes for both On-premise Recording and Cloud Recording.

    For example, if there are three UIDs sending audio and video in a channel, individual recording mode generates three audio files and three video files.

    Developers can use Agora's Audio & Video File Merging script to merge the audio and video files of each UID. See Merge Audio and Video Files (Cloud Recording) or Use the Transcoding Script (On-premise Recording) for details.

    Reference:
  • Individual Recording (Cloud Recording)
  • Individual Recording (On-premise Recording)
  • L

    low-quality video stream

    In the dual-stream mode, the SDK transmits two video streams of differing quality at the same time. The low-quality video stream has a lower resolution and bitrate than the high-quality video stream. See dual-stream mode for details.

    O

    On-premise Recording

    On-premise Recording is a component provided by Agora for recording and saving voice and video calls and interactive broadcasts on a Linux server.

    On-premise Recording enables developers to record, transfer and store audio and video data on a local server. For this reason, On-premise Recording is useful in high-security scenarios, such as surveillance and internal government or enterprise communications.

    The On-premise Recording SDK is compatible with the Agora RTC SDK. It supports two recording modes: Individual and composite. Developers can record communications either directly from the command lineor by calling APIs. On-premise Recording offers additional advanced features such as adding watermarks and capturing screenshots.

    P

    publish

    Publishing is the action of sending the user's audio and/or video data to the channel. Usually, the published object is a media stream created by the audio data sampled from a microphone and/or the video data captured by a camera. Developers can also publish media streams from other sources, including an online music file and the user's screen.

    After publishing succeeds, the SDK continues sending media data to other users in the channel. By publishing one's stream and subscribing to others' streams, users have real-time voice or video calls with each other.

    Reference:
  • Channel
  • Subscribe
  • R

    raw data

    Raw data, including raw audio data and raw video data, is the unprocessed data which developers can collect during real-time communication.

    • Raw data collected from a sender is the unprocessed data coming directly from the data source such as microphone or camera.
    • Raw data retrieved from a receiver is the data sent by remote users after decoding.

    Generally, the data format of raw audio is PCM, and the data formats of raw video are RGB and YUV420.

    RTM Native SDK

    Agora RTM SDK for the following platforms is also known as RTM Native SDK:

    • Android
    • iOS
    • macOS
    • Linux Java
    • Linux C++
    • Windows C++.
    Reference: Agora RTM SDK

    S

    SD-RTN™

    SD-RTN™, or Software Defined Real-time Network, is a real-time transmission network built by Agora and is the only network infrastructure specifically designed for real-time communications in the world.

    All audio and video services provided by the Agora SDK are deployed and transmitted through the Agora SD-RTN™. Agora deploys over 250 data centers worldwide that use intelligent dynamic routing algorithms to achieve millisecond latency and ensure high availability of Agora's service.

    stream

    A stream is an object that contains audio and/or video data. Users in a channel can publish the local stream and subscribe to the remote streams from other users.

    stream fallback

    In scenarios where multiple users engage in real-time audio and video communication, user experience can be impaired if the network condition is too poor to guarantee both audio and video at the same time.

    Stream fallback minimizes the impact of poor network connections by ensuring that some form of communication continues between the sender and receiver. Audio and video streams change to audio-only, or from high-quality to low-quality video.

    If the network connection improves, the low-quality video stream or audio-only stream return to the original quality.

    subscribe

    Subscribing is the action of receiving media streams published to the channel. A user can receive other users' audio and/or video data by subscribing to their streams. A user can subscribe to one or more streams published by other users.

    Developers can either directly play the subscribed streams or process them, for example by taking screenshots, or recording the streams.

    Reference:
  • Channel
  • Publish
  • T

    token

    A token, also known as a dynamic key, is used in situations requiring high security, such as in a production environment. You need a token for authentication when joining an RTC channel or when logging into the Agora RTM system.

    • For users of the Agora RTC SDK, Agora On-premise Recording SDK, or Agora Cloud Recording service, the Agora server uses a token to verify each user's App ID, privileges (for joining a channel and for publishing different types of streams), privilege expiration period, and token validity.
    • For users of the Agora RTM SDK, the Agora RTM server uses a token to verify each user's App ID, user ID, and token validity.

    In a production environment, an app user must generate a token on a business server and pass the token when joining an RTC channel or when logging in the Agora RTM system. If you are still testing your app or do not wish to take the trouble of setting up your own business server for generating tokens, have Agora Console generate a temporary token for you when creating your project. In this case, a temporary token suffices because it has the same function.

    Each token has a privilege expiration period and a validity period. The privilege expiration period is when the user privilege expires; the validity period is when the token itself expires (24 hours by default).

    Users of the Agora RTC SDK, Agora On-premise SDK, or Agora Cloud Recording service are kicked out of the channel immediately when their token expires. Users of the Agora RTM SDK are not immediately kicked out of the Agora RTM system but will not be able to log in the RTM system the next time their client connects to the system. Therefore, when notified that your privilege will soon expire or that the token validity has expired, generate a new token at your earliest convenience and save it for the next channel-join or login.

    transcoding

    Transcoding is used in CDN live streaming when multiple hosts are in the channel.

    In CDN live streaming, the audio and video streams sent to the SD-RTN™ are transferred into RTMP (Real-Time Messaging Protocol) protocol and pushed to the CDN. If there are multiple hosts, their streams need to be combined into a single stream by transcoding.

    Transcoding sets the audio/video profiles and the picture-in-picture layout for the stream to be pushed to the CDN.

    Agora does not recommend transcoding in the case of a single host.

    U

    user ID

    A user ID identifies a user in the channel. Each user in the same channel should have a unique user ID.

    V

    video layout

    Video layout arranges the display of users when multiple users are mixed into one stream, such as in CDN live streaming or a composite recording. Video layout sets the relative size and position of each user on the canvas. It is sometimes referred to as picture-in-picture layout.

    In the following image, the background of the video is the canvas, and each user occupies a region on the canvas.

    img

    Developers may have to set video layout when using Cloud Recording, On-premise Recording, or pushing streams to CDN.

    • Recording: When using On-premise Recording or Cloud Recording to record in the composite recording mode, developers can choose from three predefined video layouts: floating, best fit, and vertical. Developers can also customize the video layout by setting the size and position of each user's region on the canvas.

    • Push Streams to CDN: When multiple hosts are in a CDN live streaming channel, transcoding combines the streams of all hosts into a single stream. Use TranscodingUser to set the video layout for each user.

    Reference:
  • Set Video Layout(Cloud Recording)
  • Set Video Layout(On-premise Recording)
  • Push Streams to CDN