Send and receive video metadata

With Server Gateway, you can attach metadata to the video stream you send and receive the metadata that remote users send with their video. Use video metadata when you need to deliver information related to the video stream. This page shows you how to send and receive video metadata.

PrerequisitesCopy Anchor Link

Before you start, make sure that you have:

• Server Gateway Linux C++ SDK v4.4.33 or later.
• A project that creates and publishes a local video track. For more information, see Send and receive media streams.

ImplementationCopy Anchor Link

Take the following steps to send and receive video metadata:

1. Implement agora::rtc::IMetadataObserver and override the following callbacks:

   • getMaxMetadataSize: Returns the maximum metadata buffer size in bytes. The default is 512 and the maximum is 1024. Override this callback only to change the default.
   • onReadyToSendMetadata: Write the data to send to metadata.buffer, set metadata.size, and return true. The SDK provides the buffer. Make sure the data doesn't exceed the size that getMaxMetadataSize returns. To skip metadata for this callback, return false. Returning false doesn't stop video transmission.
   • onMetadataReceived: Read the metadata that a remote user sends. The buffer is valid only during the callback. To use the data after the callback returns, copy the buffer contents.

   Copy Text

   #include <cstring>
   #include <memory>
   #include <string>
   
   #include "NGIAgoraVideoTrack.h"
   
   class SampleMetadataObserver : public agora::rtc::IMetadataObserver {
    public:
     int getMaxMetadataSize() override { return DEFAULT_METADATA_SIZE_IN_BYTE; }
   
     bool onReadyToSendMetadata(Metadata& metadata,
                                agora::rtc::VIDEO_SOURCE_TYPE source_type) override {
       const std::string payload = "your-metadata";
       if (payload.size() > static_cast<size_t>(getMaxMetadataSize())) {
         return false;  // Skip metadata for this callback. Video still sends.
       }
       memcpy(metadata.buffer, payload.data(), payload.size());
       metadata.size = static_cast<unsigned int>(payload.size());
       return true;
     }
   
     void onMetadataReceived(const Metadata& metadata) override {
       // Copy the data. The buffer is valid only during this callback.
       std::string received(reinterpret_cast<const char*>(metadata.buffer),
                            metadata.size);
       // Process received, metadata.uid, and metadata.timeStampMs.
     }
   };

2. After you create the local video track, call registerVideoMetadataObserver on the track to register the observer. Then enable and publish the track.

   Copy Text

   auto metadataObserver = std::make_unique<SampleMetadataObserver>();
   customVideoTrack->registerVideoMetadataObserver(metadataObserver.get());
   customVideoTrack->setEnabled(true);
   connection->getLocalUser()->publishVideo(customVideoTrack);

3. When you no longer need the observer, call unregisterVideoMetadataObserver, then release the observer object.

   Copy Text

   customVideoTrack->unregisterVideoMetadataObserver(metadataObserver.get());
   metadataObserver.reset();

ReferenceCopy Anchor Link

This section describes the metadata structure.

Metadata fieldsCopy Anchor Link

IMetadataObserver::Metadata contains the following fields:

The source_type parameter of onReadyToSendMetadata is the video source type, of type VIDEO_SOURCE_TYPE.