# Raw video processing (/en/realtime-media/video/build/capture-and-render-video/raw-video-processing/web)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

In certain use-cases, it is necessary to process raw video captured through the camera to achieve desired functionality or enhance the user experience. Video SDK provides the capability to pre-process and post-process the captured video data, allowing you to implement custom playback effects.

      
  
      
  
      
  
      
  
      
  
      
    ## Understand the tech [#understand-the-tech-5]

    Video SDK allows you to process video frames using the [Canvas API](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Manipulating_video_using_canvas#manipulating_the_video_frame_data). Use the API to modify video frames before publishing to the channel.

    This guide explains how to extract video frames from the local video track, modify them, and publish the processed frames to the channel.

    The following figure shows the video data processing flow in the SDK video module.

    **Process raw video**

    ![image](https://assets-docs.agora.io/images/video-sdk/process-raw-video-web.svg)

    ## Prerequisites [#prerequisites-5]

    Ensure that you have implemented the [SDK quickstart](/en/realtime-media/video/get-started-sdk) in your project.

    ## Implement raw video processing [#implement-raw-video-processing-5]

    To implement raw video data functionality in your project, refer to the following steps:

    1. Extract video frames:

       1. Create a local video track to capture the raw video stream from the camera.
       2. Assign the local video stream to a hidden `<video>` element to enable frame extraction.
       3. Call `processFrame` on `onplay` to modify frames before rendering or publishing.
       4. Create a `canvas` to process the extracted video frames.
       5. Obtain a 2D drawing context from the `canvas` to manipulate pixel data.

          ```js
          // Declare global variables for video processing
          let cameraTrack, canvas, ctx;
          let isProcessing = false;

          async function extractVideoFrames() {
            // Create a video track from the camera
            cameraTrack = await AgoraRTC.createCameraVideoTrack();

            // Create a hidden video element to play the camera feed
            const videoElement = document.createElement('video');
            videoElement.srcObject = new MediaStream([cameraTrack.getMediaStreamTrack()]);
            videoElement.autoplay = true;
            videoElement.playsInline = true;
            videoElement.style.display = 'none';
            document.body.appendChild(videoElement);

            isProcessing = true;

            videoElement.onloadedmetadata = () => {
              canvas.width = videoElement.videoWidth;
              canvas.height = videoElement.videoHeight;
            };

            videoElement.onplay = () => processFrame(videoElement);

            // Create a canvas element to manipulate video frames
            canvas = document.createElement('canvas');
            ctx = canvas.getContext('2d', { willReadFrequently: true });
            document.body.appendChild(canvas);
          }
          ```

    2. Process video frames:

       To modify video frames in real time, follow these steps:

       1. Draw the video frame onto the `canvas` to capture pixel data.
       2. Retrieve the pixel data from the `canvas` and modify it as needed.
       3. Apply the modified pixel data back to the `canvas`.
       4. Call `requestAnimationFrame` to continuously process frames in a loop.

          ```js
          function processFrame(videoElement) {
            // Exit if processing is stopped or required elements are missing
            if (!isProcessing || !canvas || !ctx) return;

            // Ensure the video has loaded and has valid dimensions
            if (!videoElement.videoWidth || !videoElement.videoHeight) return;

            // Update canvas size to match video dimensions if necessary
            if (canvas.width !== videoElement.videoWidth || canvas.height !== videoElement.videoHeight) {
              canvas.width = videoElement.videoWidth;
              canvas.height = videoElement.videoHeight;
            }

            // Draw the current video frame onto the canvas
            ctx.drawImage(videoElement, 0, 0, canvas.width, canvas.height);

            // Get pixel data from the canvas
            const frame = ctx.getImageData(0, 0, canvas.width, canvas.height);

            // Iterate through each pixel and remove the green channel
            for (let i = 0; i < frame.data.length; i += 4) {
              frame.data[i + 1] = 0;
            }

            // Apply the modified pixel data back to the canvas
            ctx.putImageData(frame, 0, 0);

            // Request the next frame to continue processing in a loop
            requestAnimationFrame(() => processFrame(videoElement));
          }
          ```

    3. Publish processed frames:

       To publish the processed frame to the channel:

       1. Create a custom video track from the modified frames using `createCustomVideoTrack`.
       2. Publish the custom track.

          ```js
          async function extractVideoFrames() {
            // Create a custom video track from the canvas stream
            const processedTrack = await AgoraRTC.createCustomVideoTrack({
              // Capture video frames from the canvas at 30 FPS and extract the video track
              mediaStreamTrack: canvas.captureStream(30).getVideoTracks()[0],
            });

            // Publish the processed video track to the RTC channel
            await rtc.client.publish([processedTrack]);
          }
          ```

    ## Reference [#reference-5]

    This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

    ### API reference [#api-reference-5]

    * [`createCameraVideoTrack`](https://api-ref.agora.io/en/video-sdk/web/4.x/interfaces/iagorartc.html#createcameravideotrack)
    * [`createCustomVideoTrack`](https://api-ref.agora.io/en/video-sdk/web/4.x/interfaces/iagorartc.html#createcustomvideotrack)
    * [`publish`](https://api-ref.agora.io/en/video-sdk/web/4.x/interfaces/iagorartcclient.html#publish)