# Alpha transparency effect (/en/realtime-media/interactive-live-streaming/build/apply-effects-and-enhancements/alpha-transparency-effect/unity)
> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).
In various real-time video interaction use-cases, segmenting the host's background and applying transparent special effects can make the interaction more engaging, enhance immersion, and improve the overall interactive experience.
Consider the following sample use-cases:
* **Broadcaster background replacement**: The audience sees the broadcaster's background in the video replaced with a virtual scene, such as a gaming environment, a conference, or a tourist attractions.
* **Animated virtual gifts**: Display dynamic animations with a transparent background to avoid obscuring live content when multiple video streams are merged.
* **Chroma keying during live game streaming**: The audience sees the broadcaster's image cropped and positioned within the local game screen, making it appear as though the broadcaster is part of the game.
## Prerequisites [#prerequisites-5]
Ensure that you have implemented the [SDK quickstart](../../index.mdx) in your project.
## Implement alpha transparency [#implement-alpha-transparency-5]
Choose one of the following methods to implement the alpha transparency effect based on your specific business use-case.
### Custom video capture use-case [#custom-video-capture-use-case-1]
The following figure illustrates the implementation process for this use-case:
**Alpha transparency custom video capture use-case**
Agora provides the following options to push self-captured video frames with alpha information to the SDK for publishing to the channel. Choose the method that best fits your needs.
Due to limitations of the Unity SDK, it is currently not possible to locally preview the custom captured video streams to be pushed through the `VideoSurfaceYUVA` component. You must view the effects of the push streams on the receiving end.
#### Set alpha information for video frames [#set-alpha-information-for-video-frames]
Refer to the following steps to process the self-captured video data, push the self-captured video stream within the channel, and set the alpha information for each frame individually to realize dynamic visual effects. The following sample code uses the following picture as an example:
1. Process texture data. Extract and convert the color data of the `Texture2D` object into two byte arrays, one containing the RGBA data and the other containing the alpha channel data for subsequent processing and transmission.
```csharp
// The Read/Write enable property is set to true and bound to the TextureWithAlphaBuffer variable
public Texture2D TextureWithAlphaBuffer;
private byte[] _rgbaData = null;
private byte[] _alphaData = null;
private int _textureWidth;
private int _textureHeight;
// Customized video track IDs
private uint _videoTrackId = 0;
// Convert color data from the Color32 array to rgbaData and alphaData arrays
private void ConvertColor32(Color32[] colors, byte[] rgbaData, byte[] alphaData)
{
int i = 0;
int l = 0;
foreach (var color in colors)
{
rgbaData[i++] = color.r;
rgbaData[i++] = color.g;
rgbaData[i++] = color.b;
rgbaData[i++] = color.a;
alphaData[l++] = color.a;
}
}
// Read the color data from the texture and initialize the _rgbaData and _alphaData arrays, as well as record the width and height of the texture
public void InitTextureDataWithAlphaBuffer()
{
Color32[] color32s = TextureWithAlphaBuffer.GetPixels32(0);
_rgbaData = new byte[color32s.Length * 4];
_alphaData = new byte[color32s.Length];
ConvertColor32(color32s, _rgbaData, _alphaData);
_textureWidth = TextureWithAlphaBuffer.width;
_textureHeight = TextureWithAlphaBuffer.height;
this.Log.UpdateLog(string.Format("AlphaBuffer width: {0}, height: {1}", _textureWidth, _textureHeight));
}
```
2. Configure the video encoding parameters to send alpha data to the remote end. Call `SetVideoEncoderConfiguration` to set the video encoding properties and set `encodeAlpha` to `true` to encode and send alpha data to the remote end.
3. Create a video track for posting custom captured video to the channel. Call `CreateCustomVideoTrack` to create a custom video track.
4. Call `JoinChannel`\[2/2] to join the channel and set the channel media options:
* Set `customVideoTrackId` to the video track ID obtained using `CreateCustomVideoTrack`.
* Set `publishCameraTrack` to `false` to stop publishing the video stream captured by the camera.
* Set `publishCustomVideoTrack` to true to publish the custom captured video stream.
Refer to the following sample code:
```csharp
public void OnStartPushVideoFrameWithAlphaBuffer()
{
// Read RGBA data and alpha data from an image
InitTextureDataWithAlphaBuffer();
// Set video encoding properties
VideoEncoderConfiguration videoEncoderConfiguration = new VideoEncoderConfiguration();
videoEncoderConfiguration.dimensions.width = _textureWidth; videoEncoderConfiguration.dimensions.height = _textureHeight;
// Set the Transmit Code alpha Data
videoEncoderConfiguration.advanceOptions.encodeAlpha = true;
RtcEngine.SetVideoEncoderConfiguration(videoEncoderConfiguration);
// Create custom video tracks for posting custom captured video streams within the channel
_videoTrackId = RtcEngine.CreateCustomVideoTrack();
// Set Channel Media Options
var options = new ChannelMediaOptions();
// No release of video streams captured by cameras
options.publishCameraTrack.SetValue(false);
// Publish audio streams captured by microphones
options.publishMicrophoneTrack.SetValue(true);
// Publish custom captured video streams
options.publishCustomVideoTrack.SetValue(true);
// Set customVideoTrackId to the video track ID you returned when calling the CreateCustomVideoTrack method.
options.customVideoTrackId.SetValue(_videoTrackId);
// Join the channel.
RtcEngine.JoinChannel(_token, _channelName, 0, options)
InvokeRepeating("UpdateForPushVideoFrameWithAlphaBuffer", 0.1f, 0.1f);
}
```
5. Push a custom captured video stream to the SDK. Call the `PushVideoFrame` method to push the video frame and set the alpha data for the video frame. Set the alpha channel data for the video frame by setting the `alphaBuffer` or `fillAlphaBuffer` parameter.
* `alphaBuffer`: Alpha channel data. This data is consistent with the size of the video frame, and each pixel point takes the value in the range of \[0,255], where 0 represents the background; 255 represents the foreground (portrait).
* `fillAlphaBuffer`: When set to true, the SDK can extracts and fills the alpha channel data automatically, and you don't need to additionally set an `alphaBuffer`. `fillAlphaBuffer` parameter currently only supports video data in BGRA or RGBA format.
Make sure the `alphaBuffer` is exactly the same size (width × height) as the video frame, otherwise the game may crash.
```csharp
public void UpdateForPushVideoFrameWithAlphaBuffer()
{
var timetick = System.DateTime.Now.Ticks / 10000;
ExternalVideoFrame externalVideoFrame = new ExternalVideoFrame();
// Video buffer type is raw data
externalVideoFrame.type = VIDEO_BUFFER_TYPE.VIDEO_BUFFER_RAW_DATA;
// Video pixels in RGBA format
externalVideoFrame.format = VIDEO_PIXEL_FORMAT.VIDEO_PIXEL_RGBA;
externalVideoFrame.buffer = _rgbaData;
externalVideoFrame.alphaStitchMode = ALPHA_STITCH_MODE.NO_ALPHA_STITCH
externalVideoFrame.alphaBuffer = null;
// Extracts and fills alpha channel data, currently only available for video data in RGBA or BGRA format.
externalVideoFrame.fillAlphaBuffer = true;
// Extracts and fills alpha channel data, currently only available for video data in RGBA or BGRA format.
//externalVideoFrame.alphaBuffer = _alphaData;
//externalVideoFrame.fillAlphaBuffer = false
externalVideoFrame.stride = _textureWidth;
externalVideoFrame.height = _textureHeight;
externalVideoFrame.rotation = 180;
externalVideoFrame.timestamp = timetick;
// Push custom captured video to the SDK
var ret = RtcEngine.PushVideoFrame(externalVideoFrame,
_videoTrackId);
Debug.Log("PushVideoFrame ret = " + ret + " time: " + timetick);
}
```
#### Merge video data and alpha data [#merge-video-data-and-alpha-data]
Refer to the following steps to merge the captured video data with the alpha channel data and push the combined video data to the SDK for publishing to the channel. The following steps show how to integrate the alpha channel data with the video data.
1. Merge the captured video data with the alpha data.
```csharp
public void InitTextureDataWithAlphaStitchMode()
{
// Read raw texture data
Color32[] color32s = TextureWithAlphaBuffer.GetPixels32(0);
// Get the width and height of the original texture
var width = TextureWithAlphaBuffer.width;
var height = TextureWithAlphaBuffer.height;
// Create a new texture that is twice the height of the original texture.
TextureWithAlphaStitchMode = new Texture2D(width, height * 2, TextureFormat.RGBA32, false);
// Initialize a new color array to store the spliced pixel data
Color32[] newColor32s = new Color32[color32s.Length * 2];
// Copy the original color data to the top half of the new color array Array.Copy(color32s, newColor32s, color32s.Length);
// Splice the alpha channel data into the lower half of the new color array
for (int i = color32s.Length; i < newColor32s.Length; i++) {
newColor32s[i].r = color32s[i - color32s.Length].a;
newColor32s[i].g = newColor32s[i].r;
newColor32s[i].b = newColor32s[i].r;
newColor32s[i].a = 255;
}
// Apply the new color array to the new texture
TextureWithAlphaStitchMode.SetPixels32(newColor32s); TextureWithAlphaStitchMode.Apply();
// Convert color data and stores it in _rgbaData and _alphaData arrays
_rgbaData = new byte[newColor32s.Length * 4];
_alphaData = new byte[newColor32s.Length];
ConvertColor32(newColor32s, _rgbaData, _alphaData);
// Update texture width and height
_textureWidth = TextureWithAlphaStitchMode.width;
_textureHeight = TextureWithAlphaStitchMode.height;
this.Log.UpdateLog(string.Format("AlphaStitchMode width: {0}, height: {1}", _textureWidth, _textureHeight));
// Bind the new texture to the RawImage component in the UI.
GameObject.Find("RawImage").GetComponent().texture = TextureWithAlphaStitchMode;
GameObject.Find("RawImage").GetComponent().sizeDelta = new Vector2(_textureWidth, _textureHeight);
}
```
2. Configure the video encoding parameters to send alpha data to the remote end. Call `SetVideoEncoderConfiguration` to set the video encoding properties and set `encodeAlpha` to `true` to encode and send alpha data to the remote end.
3. Create a video track for posting custom captured video to the channel. Call `CreateCustomVideoTrack` to create a custom video track.
4. Call `JoinChannel`\[2/2] to join the channel and set the channel media options:
* Set `customVideoTrackId` to the video track ID obtained by calling `CreateCustomVideoTrack`.
* Set `publishCameraTrack` to `false` to stop publishing the video stream captured by the camera.
* Set `publishCustomVideoTrack` to `true` to publish a custom captured video stream.
The sample code is as follows:
```csharp
public void OnStartPushVideoFrameWithAlphaStitchMode()
{
InitTextureDataWithAlphaStitchMode();
VideoEncoderConfiguration videoEncoderConfiguration = new VideoEncoderConfiguration();
videoEncoderConfiguration.dimensions.width = _textureWidth;
// The actual height sent is half the height of the stitched texture
videoEncoderConfiguration.dimensions.height = _textureHeight / 2;
// Encodes and sends Alpha data to the remote end
videoEncoderConfiguration.advanceOptions.encodeAlpha = true;
// Set video encoding properties
RtcEngine.SetVideoEncoderConfiguration(videoEncoderConfiguration);
// Create custom video tracks for posting custom captured video streams within the channel
_videoTrackId = RtcEngine.CreateCustomVideoTrack();
var options = new ChannelMediaOptions();
// Set not to publish video streams captured by the camera
options.publishCameraTrack.SetValue(false);
// Setting Up Audio Streams for Publishing Microphone Capture
options.publishMicrophoneTrack.SetValue(true);
// Setting up to publish custom captured video streams
options.publishCustomVideoTrack.SetValue(true);
// Set customVideoTrackId to the video track ID you returned when calling the CreateCustomVideoTrack method.
options.customVideoTrackId.SetValue(_videoTrackId);
// Join the channel
RtcEngine.JoinChannel(_token, _channelName, 0, options);
InvokeRepeating("UpdateForPushVideoFrameWithAlphaStitchMode", 0.1f, 0.1f);
}
```
5. Push the merged video frame to the SDK. Call the `PushVideoFrame` method and set the relative position of the alpha data and the video frame with the `alphaStitchMode` parameter.
Refer to the following sample code:
```csharp
public void UpdateForPushVideoFrameWithAlphaStitchMode()
{
var timetick = System.DateTime.Now.Ticks / 10000;
ExternalVideoFrame externalVideoFrame = new ExternalVideoFrame();
// Video buffer type is raw data
externalVideoFrame.type = VIDEO_BUFFER_TYPE.VIDEO_BUFFER_RAW_DATA;
// Video pixels in RGBA format
externalVideoFrame.format = VIDEO_PIXEL_FORMAT.VIDEO_PIXEL_RGBA;
externalVideoFrame.buffer = _rgbaData;
// highlight-start
// Since Unity texture coordinates and SDK coordinates are different, if you need to splice the alpha data to the top of the original data, you need to set the splice position to the bottom when the SDK reads it here.
externalVideoFrame.alphaStitchMode = ALPHA_STITCH_MODE.ALPHA_STITCH_BELOW;
// highlight-end
externalVideoFrame.stride = _textureWidth;
externalVideoFrame.height = _textureHeight;
externalVideoFrame.rotation = 180;
externalVideoFrame.timestamp = timetick;
// Push video frames to SDK
var ret = RtcEngine.PushVideoFrame(externalVideoFrame, _videoTrackId);
Debug.Log("PushVideoFrame ret = " + ret + " time: " + timetick);
}
```
### SDK Capture use-case [#sdk-capture-use-case-1]
The following figure illustrates the implementation process for this use-case:
**Alpha transparency SDK capture use-case**
Take the following steps to implement this logic:
1. When rendering local video using the `VideoSurface` component, call `SetLocalVideoDataSourcePosition` to get the position of the video frame. Setting the position to `POSITION_POST_CAPTURER` means getting the video frame after the local video capture is done.
```csharp
// Set the local video frame position to POSITION_POST_CAPTURER
RtcEngine.SetLocalVideoDataSourcePosition(VIDEO_MODULE_POSITION.POSITION_POST_CAPTURER);
```
2. Call `EnableVirtualBackground` on the host to enable the background segmentation algorithm and get the alpha data of the portrait area. The recommended parameters are as follows:
* `enabled`: Set to `true` to enable virtual background.
* `backgroundSourceType`: Set to `BACKGROUND_NONE` to split the portrait and background and process the background as an alpha channel.
```csharp
var source = new VirtualBackgroundSource();
source.background_source_type = BACKGROUND_SOURCE_TYPE.BACKGROUND_NONE;
var segproperty = new SegmentationProperty();
var nRet = RtcEngine.EnableVirtualBackground(true, source, segproperty, MEDIA_SOURCE_TYPE.PRIMARY_CAMERA_SOURCE);
this.Log.UpdateLog("EnableVirtualBackground true :" + nRet);
```
3. Call `SetVideoEncoderConfiguration` to set `encodeAlpha` to `true`. This encodes the alpha information and sends it to the remote end. Call `JoinChannel`\[2/2] to join the channel.
```csharp
VideoEncoderConfiguration videoEncoderConfiguration = new VideoEncoderConfiguration();
videoEncoderConfiguration.advanceOptions.encodeAlpha = true;
RtcEngine.SetVideoEncoderConfiguration(videoEncoderConfiguration);
RtcEngine.JoinChannel(_token, _channelName, "", 0);
```
4. Render views with alpha transparency effects using the `VideoSurfaceYUVA` component.
```csharp
private VideoSurface MakeImageSurface(string goName)
{
// ......
// Render with VideoSurfaceYUVA
VideoSurface videoSurface = go.AddComponent();
return videoSurface;
}
```