Custom video source

Custom video capture refers to the collection of a video stream from a custom source. Unlike the default video capture method, custom video capture enables you to control the capture source, and precisely adjust video attributes. You can dynamically adjust parameters such as video quality, resolution, and frame rate to adapt to various application use-cases. For example, you can capture video from high-definition cameras, and drone cameras.

Agora recommends default video capture for its stability, reliability, and ease of integration. Custom video capture offers flexibility and customization for specific video capture use-cases where default video capture does not fulfill your requirements.

Try out the online demo for Custom video source.

Understand the tech

RTC SDK provides support for creating local video tracks by passing in a MediaStreamTrack object to createCustomVideoTrack.

Prerequisites

Ensure that you have implemented the SDK quickstart in your project.

Implement the logic

This section shows you how to create and publish a custom video track from a media streams or a canvas stream.

Create a custom video track a media stream

You can implement custom video capture or preprocessing by obtaining a MediaStreamTrack object. For example, manually call the getUserMedia method to obtain a MediaStreamTrack, then use the createCustomVideoTrack method to create a local video track object for use in RTC SDK.

async function createAndPublishCustomVideoTrack() {
  try {
    // Get the video media stream
    const mediaStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: false });

    // Extract the video track from the media stream
    const videoMediaStreamTrack = mediaStream.getVideoTracks()[0];

    // Create a custom video track
    const customVideoTrack = await AgoraRTC.createCustomVideoTrack({
      mediaStreamTrack: videoMediaStreamTrack,
    });

    // Store the custom video track for later use in a shared object
    rtc.localVideoTrack = customVideoTrack;

    // Publish the custom video track to the RTC channel
    await rtc.client.publish([rtc.localVideoTrack]);

    console.log("Custom video track published successfully!");
  } catch (error) {
    console.error("Failed to create or publish custom video track:", error);
  }
}

You can also use HTMLMediaElement.captureStream or HTMLCanvasElement.captureStream to obtain the MediaStreamTrack object.

The MediaStreamTrack object refers to the browser-native MediaStreamTrack API. For details on usage and browser support, see th MediaStreamTrack API documentation.

Create a custom video track from a canvas stream

To create and publish a custom video track generated from a canvas stream, follow these steps:

Create a canvas element and set its dimensions.
Continuously draw content on the canvas using requestAnimationFrame to ensure the stream updates in real time.
Capture a media stream from the canvas.
Create a custom video track using the canvas stream.
Publish the custom video track to the channel.

Refer to the following example:

// Function to create and publish a custom video track
async function createAndPublishCustomVideoTrack() {
  try {
    // Create and draw on the canvas
    const canvas = createAndDrawCanvas();

    // Get the media stream from the canvas
    const canvasStream = canvas.captureStream();
    const canvasMediaStreamTrack = canvasStream.getVideoTracks()[0];

    // Create a custom video track from the canvas stream
    rtc.localVideoTrack = await AgoraRTC.createCustomVideoTrack({
      mediaStreamTrack: canvasMediaStreamTrack,
    });

    // Publish the custom video track to the RTC channel
    await rtc.client.publish([rtc.localVideoTrack]);

    console.log("Custom video track published successfully!");
  } catch (error) {
    console.error("Failed to create or publish custom video track:", error);
  }
}

// Function to create a canvas and start drawing
function createAndDrawCanvas() {
  const canvas = document.createElement("canvas");
  canvas.width = 640;
  canvas.height = 480;
  const context = canvas.getContext("2d");

  // Continuously draw on the canvas
  function drawCanvas() {
    // Clear previous drawings
    context.clearRect(0, 0, canvas.width, canvas.height);
    context.fillStyle = "blue";
    context.fillRect(0, 0, canvas.width, canvas.height);

    // Add your code to draw on the canvas

    // Keep refreshing the canvas
    requestAnimationFrame(drawCanvas);
  }
  drawCanvas();

  return canvas;
}

// Call the function to create and publish the track
createAndPublishCustomVideoTrack();

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Understand the tech

Prerequisites

Implement the logic

Create a custom video track a media stream

Create a custom video track from a canvas stream

Reference

API reference

Contact Us

Contact Us