Local screenshot upload

Updated

Take screenshots of the video stream and upload images to third-party cloud storage.

To meet the needs of video content supervision, Agora Video SDK enables you to take screenshots of the video stream in a channel and then upload these images to third-party cloud storage.

This page shows you how to enable screenshot uploads on your app and handle screenshot upload callbacks.

Local screenshot upload is a paid service. For billing details, refer to Pricing.

Understand the tech

When you enable screenshot upload, you set the interval between screenshots. After each interval, Video SDK takes a screenshot of the video stream sent by local users, and uploads the image to Agora SDRTN®. Agora SDRTN® uploads each screenshot to your third-party cloud storage and sends a callback notification to your server in the form of an HTTPS request.

Screenshot upload

Prerequisites

Before implementing the Agora local screenshot upload feature, complete the following configuration.

Activate local screenshot upload service

Log in to Agora Console, select the project for which you want to activate the local screenshot upload service, and then click Configure. Scroll down to the features section to find Video Screenshot Upload and click Config.

Configure third-party cloud storage

In the Video Screenshot Upload Configuration, update the screenshot upload settings:

Currently, the Agora local screenshot upload service supports Amazon S3 and Alibaba Cloud OSS as the third-party cloud storage providers. If you want to use other provider, contact technical support.

Fill in the following information:

  • Screenshot Callback Address: The server address to receive callback notifications of local screenshot upload results.

  • Storage: Third-party cloud storage provider. Select AWS.

  • Region: The data center region setting for third-party cloud storage. See AWS Official Documentation for details.

  • Access Key: String type, the access key of the third-party cloud storage. Best practice is to provide a write-only access key.

  • Secret Key: String type, the secret key of the third-party cloud storage.

  • Bucket Name: String type, the bucket name of the third-party cloud storage. The bucket name must comply with the naming rules of the corresponding third-party cloud storage service.

  • Endpoint: Specify the access domain (Endpoint) of the bucket. See AWS Official Documentation for details.

  • Filename Prefix: (Optional) JSON Array, an array composed of multiple strings, indicating the storage location of screenshots in the third-party cloud storage. For example, if fileNamePrefix = ["directory1","directory2"], the prefix directory1/directory2/ will be added to the screenshot filename, that is, directory1/directory2/xxx.jpg. The prefix length, including slashes, must not exceed 128 characters. The string should not contain symbols such as slashes, underscores, parentheses, and so on. The supported character set is as follows:

    • 26 lowercase English letters a-z
    • 26 uppercase English letters A-Z
    • 10 digits 0-9

Sample configuration

  • Snapshot Callback Address: https://webhook.site/2e75da2d-xxxx-xxxx-xxx-a0ef44a5259d
  • Storage: AWS
  • Region: cn-north-1
  • Access Key: 12345678
  • Secret Key: abcd123
  • Bucket Name: test-artifacts
  • Endpoint: s3.cn-north-1.amazonaws.com.cn
  • Filename Prefix: test
  • Screenshot Callback Address: The server address to receive callback notifications of local screenshot upload results.

  • Storage: Third-party cloud storage provider. Select Alibaba Cloud.

  • Access Key: String type, the access key of the third-party cloud storage. Best practice is to provide a write-only access key.

  • Secret Key: String type, the secret key of the third-party cloud storage.

  • Bucket Name: String type, the bucket name of the third-party cloud storage. The bucket name must comply with the naming rules of the corresponding third-party cloud storage service.

  • Endpoint: Specify the access domain (Endpoint) of the bucket. See Alibaba Cloud Official Documentation for details.

  • Filename Prefix: (Optional) JSON Array, an array composed of multiple strings, indicating the storage location of screenshots in the third-party cloud storage. For example, if fileNamePrefix = ["directory1","directory2"], the prefix directory1/directory2/ will be added to the screenshot filename, that is, directory1/directory2/xxx.jpg. The prefix length, including slashes, must not exceed 128 characters. The string should not contain symbols such as slashes, underscores, parentheses, and so on. The supported character set is as follows:

    • 26 lowercase English letters a-z
    • 26 uppercase English letters A-Z
    • 10 digits 0-9

Sample configuration

  • Snapshot Callback Address: https://webhook.site/2e75da2d-xxxx-xxxx-xxx-a0ef44a5259d
  • Storage: Alibaba Cloud
  • Region: cn-north-1
  • Access Key: 12345678
  • Secret Key: abcd123
  • Bucket Name: test-artifacts
  • Endpoint: oss-accelerate.aliyuncs.com
  • Filename Prefix: test

Integrate Video SDK

Integrate the Video SDK (including the local screenshot upload dynamic library) into your project, join a channel, and publish a video stream. See SDK quickstart for details.

Implement local screenshot upload

You enable screenshot upload in your Interactive Live Streaming client and handle the notification callback from Agora on your callback server.

Enable screenshot upload

Call the enableContentInspect method to enable the local screenshot upload feature. After successfully enabling the feature, the SDK will capture screenshots at the specified frequency and upload them to the Agora cloud server. To implement this logic, refer to the following code:

const inspectConfig = {
  // Set the interval for local screenshot to 2000 milliseconds (2 seconds), with a minimum value of 1000.
  interval: 2000,
  // (Optional) Set the address prefix for third-party cloud storage.
  ossFilePrefix: 'https://your-cloud-storage.com/',
  // (Optional) Add additional information.
  extraInfo: 'This is some additional information for content inspection.',
  // Set the function type to \"supervise\".
  inspectType: ['supervise'],
};

client
  .enableContentInspect(inspectConfig)
  .then(() => {
    console.log('Content inspection enabled successfully.');
  })
  .catch((error) => {
    console.error('Failed to enable content inspection:', error);
  });

HTTP notification callback

After a screenshot is successfully uploaded to the Agora SDRTN®, Agora sends a message notification callback to your application server in the form of an HTTPS POST request. The data format is JSON, and the character encoding is UTF-8.

Upon receiving the message notification, your application server must respond with status code 200. The expected response body format is JSON. In any of the following cases, the notification is considered to have failed:

  • No response is received from your server within 5 seconds after sending the message notification.
  • The HTTP status code of the response is not 200, or the response body format is not JSON.

The Agora service immediately retries after the first notification failure, and sends the message three times in total.

Request header

Field nameValue
Content-Typeapplication/json;charset=utf-8

Request body

The request body is a JSON object containing the following fields:

FieldTypeDescription
requestIdStringCallback notification ID that identifies an event notification from the Agora business server.
callbackParamJSON objectCustom fields passed in the callback. Currently, this object contains cname, which represents the channel name.
callbackDataStringString data passed through by the SDK local screenshot upload module.
checksumStringMD5 value calculated from callbackAddr, code, object, and requestId. Use it to verify that the callback comes from the local screenshot upload service.
objectStringName of the screenshot file. See the filename format below.
codeNumberStatus code of the screenshot. 200 indicates that the screenshot is complete.
msgStringStatus of the screenshot. "Supervise complete" indicates that the screenshot is complete.
channelNameStringChannel name of the user being screenshotted. This value is consistent with object.cname.
userIdStringUser ID of the user being screenshotted. This value is consistent with object.uid.
timestampIntegerUnix timestamp in milliseconds of the screenshot. This value uses UTC and is consistent with object.utc.

The object filename follows this format:

<OSS prefix>/<year month day>/<sid>_<cname>__uid_s_<uid>__uid_e_<type>_utc.jpg

  • <sid>: Screenshot ID, a unique identifier for a screenshot.
  • <cname>: Channel name of the user being screenshotted.
  • <uid>: User ID set when joining the channel.
  • <type>: File type. Only video is supported.
  • <utc>: UTC timestamp of the screenshot, including year, month, day, hour, minute, second, and millisecond. For example, 20190611073246073 represents June 11, 2019, at 07:32
    .073 UTC.

Request example

{
  "requestId": "38f8e3cfdcXXXXXXXXX1ceba380d7e1a_1652693284_b5813fe2ae4fa5cdfe5abd8fef82526f",
  "callbackParam": {
    "cname": "httpClient463224"
  },
  "callbackData": "",
  "checksum": "75ee988XXXXXXc2ad4ec2aef58f178fd8",
  "object": "test/20201216/38f8e3cfdc474cd56fc1ceba380d7e1a_httpClient463224__uid_s_91__uid_e_video_20200413081128672.jpg",
  "code": 200,
  "msg": "Supervise complete",
  "channelName": "httpClient463224",
  "userId": "91",
  "timestamp": 20190611073246070
}

Response

After receiving the message notification, your application server must return an HTTP status code of 200 and a JSON response body. The Agora service considers the notification unsuccessful in either of the following cases:

  • Your server does not respond within 5 seconds.
  • The response status code is not 200, or the response body is not valid JSON.

After the first unsuccessful notification, the Agora service retries immediately. It sends the notification up to three times in total.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Pricing

Agora charges for local screenshot upload based on two metrics:

  • Screenshot fee
  • QPS overage fee (queries per second)

At the end of each month, Agora calculates:

  • The total number of screenshots generated.
  • The monthly peak QPS, if it exceeds the free quota.

Billing formula

Monthly fee = (Number of screenshots - free quota) x Screenshot unit price
            + (Monthly peak QPS - free QPS) x QPS overage unit price
Billing typeUnit priceDiscount policy
Screenshot fee$0.30 per 10,000 imagesThe first 10,000 images each month are free.
QPS overage fee$1.50 per QPSThe first 500 QPS each month are free. Usage above 500 QPS is charged according to the monthly peak QPS.

Example 1

A project generates 910,000 screenshots in one month and reaches a peak QPS of 100:

  • Billable screenshots: 910,000 - 10,000 = 900,000 images.
  • Screenshot fee: 900,000 / 10,000 x $0.30 = $27.00.
  • QPS fee: 100 is within the free 500 QPS quota, so the fee is $0.
  • Total fee: $27.00.

Example 2

A project generates 2.91 million screenshots in one month and reaches a peak QPS of 600:

  • Billable screenshots: 2,910,000 - 10,000 = 2,900,000 images.
  • Screenshot fee: 2,900,000 / 10,000 x $0.30 = $87.00.
  • QPS fee: (600 - 500) x $1.50 = $150.00.
  • Total fee: $237.00.

local screenshot upload dynamic library

You can find details about the name of the local screenshot upload dynamic library and the increase in app size after integration in App size optimization.

HTTP status codes

The HTTP status codes for the application server responses are as follows:

Status CodeDescription
200Successful request.
201Successful request and creation of a new resource.
206No user streamed during the entire screenshot process, or some screenshot files were not uploaded to the third-party cloud storage.
400Syntax error in the request (for example, parameter error), and the server cannot understand it.
401Unauthorized (App ID or Customer Certificate mismatch).
404The server cannot find the requested resource (webpage).
500Internal server error, unable to complete the request.
504Internal server error. The server acting as a gateway or proxy did not receive a timely response from the upstream server.