Stream media to a channel

Updated

Play local or online media files locally or to remote users in an Agora channel.

Playing media files during online business presentations, educational sessions, or casual meetups heightens user engagement. Video SDK enables you to add media playing functionality to your game.

This page shows you how to use media player-related APIs to play local or online media resources with remote users in Interactive Live Streaming channels.

Understand the tech

To play a media file in a channel, you open the file using a media player instance. When the file is ready to be played, you set up the local video container to display the media player output. You update channel media options to start publishing the media player stream, and stop publishing the camera and microphone streams. The remote user sees the camera and microphone streams of the media publishing user replaced by media streams.

Media player flow

Prerequisites

Ensure that you have implemented the SDK quickstart in your project.

Implement the logic

To implement a media player in your game, follow these steps:

Create a media player object

After initializing an IRtcEngine instance, call createMediaPlayer to create an IMediaPlayer object.

MediaPlayer = RtcEngine.CreateMediaPlayer();
if (MediaPlayer == null)
{
  this.Log.UpdateLog("Failed to create media player.");
  return;
}

If you need multiple media player objects, call createMediaPlayer multiple times.

Register callback events and implement callbacks

Call InitEventHandler from the IMediaPlayer class to register media player callback events.

MpkEventHandler handler = new MpkEventHandler(this);
MediaPlayer.InitEventHandler(handler);
this.Log.UpdateLog("Player ID: " + MediaPlayer.GetId());

Implement IMediaPlayerSourceObserver callbacks as needed.

class MpkEventHandler : IMediaPlayerSourceObserver
{
  // Report player status changes
  public override void OnPlayerSourceStateChanged(MEDIA_PLAYER_STATE state, MEDIA_PLAYER_REASON reason)
  {

  }

  // Report player events
  public override void OnPlayerEvent(MEDIA_PLAYER_EVENT @event, Int64 elapsedTime, string message)
  {
    Debug.Log($"OnPlayerEvent state: {@event}");
  }

  // Report preload events
  public override void OnPreloadEvent(string src, PLAYER_PRELOAD_EVENT @event)
  {
    Debug.Log($"OnPreloadEvent src: {src}, event: {@event}");
  }

  // Report media resource playback progress
  public override void OnPositionChanged(long positionMs, long timestampMs)
  {

  }
}

Set the video rendering window

To play a video, set the video renderer after opening the media resource. Destroy the video view when playback stops. Use GetId to retrieve the player ID and set VIDEO_SOURCE_TYPE to VIDEO_SOURCE_MEDIA_PLAYER.

// Handle player status changes
public override void OnPlayerSourceStateChanged(MEDIA_PLAYER_STATE state, MEDIA_PLAYER_REASON reason)
{
  Debug.Log($"State: {state}, Reason: {reason}, Player ID: {_sample.MediaPlayer.GetId()}");
  if (state == MEDIA_PLAYER_STATE.PLAYER_STATE_OPEN_COMPLETED)
  {
    // Create a view to display the video stream when the media resource opens
    MakeVideoView((uint)_sample.MediaPlayer.GetId(), "", VIDEO_SOURCE_TYPE.VIDEO_SOURCE_MEDIA_PLAYER);
  }
  else if (state == MEDIA_PLAYER_STATE.PLAYER_STATE_STOPPED)
  {
    // Destroy the video view after playback stops
    DestroyVideoView((uint)_sample.MediaPlayer.GetId());
  }
}

To set the video display, call SetForUser, and start rendering with SetEnable.

// Create a video view
static void MakeVideoView(uint uid, string channelId = "", VIDEO_SOURCE_TYPE videoSourceType = VIDEO_SOURCE_TYPE.VIDEO_SOURCE_CAMERA)
{
  var go = GameObject.Find(uid.ToString());
  if (go != null) return;

  // Create a GameObject for rendering video
  var videoSurface = MakeImageSurface(uid.ToString());
  if (videoSurface == null) return;

  // Set up video display
  videoSurface.SetForUser(uid, channelId, videoSourceType);
  videoSurface.SetEnable(true);

  // Adjust video surface size
  videoSurface.OnTextureSizeModify += (int width, int height) =>
  {
    var transform = videoSurface.GetComponent<RectTransform>();
    if (transform != null)
    {
      transform.sizeDelta = new Vector2(width / 2, height / 2);
      transform.localScale = Vector3.one;
    }
    else
    {
      float scale = (float)height / (float)width;
      videoSurface.transform.localScale = new Vector3(-1, 1, scale);
    }
    Debug.Log($"Texture size modified: {width} x {height}");
  };
}

// Destroy the video view
static void DestroyVideoView(uint uid)
{
  var go = GameObject.Find(uid.ToString());
  if (go != null)
  {
    Destroy(go);
  }
}

// Create a video view with a RawImage component
private static VideoSurface MakeImageSurface(string goName)
{
  GameObject go = new GameObject(goName);
  go.AddComponent<RawImage>();
  go.AddComponent<UIElementDrag>();

  var canvas = GameObject.Find("VideoCanvas");
  if (canvas != null)
  {
    go.transform.SetParent(canvas.transform);
    Debug.Log("Added video view");
  }
  else
  {
    Debug.Log("Canvas not found for video view");
  }

  go.transform.Rotate(0f, 0f, 180f);
  go.transform.localPosition = Vector3.zero;
  go.transform.localScale = new Vector3(4.5f, 3f, 1f);

  return go.AddComponent<VideoSurface>();
}

Open media resources

Call Open to load a local or online media file. You can also set the playback start position. For supported formats, see Supported Formats.

public void OnOpenButtonPress()
{
  var ret = MediaPlayer.Open("Your File Path", 0);
  Debug.Log("Open returns: " + ret);
}

Alternatively, use OpenWithMediaSource to open media resources and configure playback settings, like automatic playback and real-time caching.

Publish the audio and video streams

When joining a channel, set the ChannelMediaOptions as follows:

ChannelMediaOptions options = new ChannelMediaOptions();
// Enable automatic subscription to audio and video streams
options.autoSubscribeAudio.SetValue(true);
options.autoSubscribeVideo.SetValue(true);
options.publishCustomAudioTrack.SetValue(false);
options.publishCameraTrack.SetValue(false);
// Enable publishing audio and video streams from the media player
options.publishMediaPlayerAudioTrack.SetValue(true);
options.publishMediaPlayerVideoTrack.SetValue(true);
// Provide the media player ID
options.publishMediaPlayerId.SetValue(MediaPlayer.GetId());
// Enable audio playback
options.enableAudioRecordingOrPlayout.SetValue(true);
// Set the user role to broadcaster
options.clientRoleType.SetValue(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);

var ret = RtcEngine.JoinChannel(_token, _channelName, 0, options);
this.Log.UpdateLog($"JoinChannel returns: {ret}");

Playback control

  1. Call Play to start playing a local or online media resource.

    public void OnPlayButtonPress()
    {
      var ret = MediaPlayer.Play();
      Debug.Log($"Play return: {ret}");
    }

If you use Open to load a media resource, wait for the OnPlayerSourceStateChanged callback to report the state as PLAYER_STATE_OPEN_COMPLETED before calling Play. If you use OpenWithMediaSource with autoPlay enabled, playback starts automatically.

  1. To loop playback, call SetLoopCount. Set the value to -1 for infinite looping. The default value of 0 means no loop playback.
public void OnPlayButtonPress()
{
  MediaPlayer.SetLoopCount(this.IsLoop() ? -1 : 0);
  var ret = MediaPlayer.Play();
  this.Log.UpdateLog($"Play return: {ret}");
}
  1. To play from a specific position, use Seek. Ensure Seek is called after receiving the PLAYER_STATE_OPEN_COMPLETED state in the OnPlayerSourceStateChanged callback.

    If you call Seek while playback is paused, the media remains paused after the call. Call Resume or Play to continue playback.

    1. Call SetPlaybackSpeed to control the playback speed.
public void OnSetPlaybackSpeedButtonPress()
{
  var ret = MediaPlayer.SetPlaybackSpeed(2);
  this.Log.UpdateLog($"SetPlaybackSpeed return: {ret}");
}

Pause, resume, and stop

To control playback, use the following methods:

  • Pause: Pauses playback.

    public void OnPauseButtonPress()
    {
      var ret = MediaPlayer.Pause();
      this.Log.UpdateLog($"Pause return: {ret}");
    }
  • Resume: Resumes playback after pausing.

    public void OnResumeButtonPress()
    {
      var ret = MediaPlayer.Resume();
      this.Log.UpdateLog($"Resume returns: {ret}");
    }
  • Stop: Stops playback. Reopen the media resource to play again.

    public void OnStopButtonPress()
    {
      var ret = MediaPlayer.Stop();
      this.Log.UpdateLog($"Stop return: {ret}");
    }

Adjust the volume

To adjust the local playback volume, call AdjustPlayoutVolume. To adjust the volume of the audio stream sent to the remote end, call AdjustPublishSignalVolume.

var ret = MediaPlayer.AdjustPlayoutVolume(30);
this.Log.UpdateLog($"AdjustPlayoutVolume return: {ret}");

ret = MediaPlayer.AdjustPublishSignalVolume(50);
this.Log.UpdateLog($"AdjustPublishSignalVolume return: {ret}");

Preload media resources

To play multiple media resources continuously, preload the media resources to ensure optimal audience experience when switching resources.

  1. Call PreloadSrc to preload media resources. To load multiple resources, call this method multiple times. Listen to the OnPreloadEvent callbacks to receive preload related events.

    public void OnPreloadSrcButtonClick()
    {
    
      var nRet = MediaPlayer.PreloadSrc(PRELOAD_URL, 0);
      this.Log.UpdateLog("PreloadSrc: " + nRet);
    }
  2. Call PlayPreloadedSrc to play the preloaded media resource. After you call this method successfully, you receive a OnPlayerStateChanged callback reporting status PLAYER_STATE_PLAYING.

    public void OnPlayPreloadButtonClick()
    {
      var nRet = MediaPlayer.PlayPreloadedSrc(PRELOAD_URL);
      this.Log.UpdateLog("PlayPreloadedSrc: " + nRet);
    }
  3. When you no longer need the loaded media resources, call UnloadSrc to release the preloaded resources.

    var nRet = MediaPlayer.UnloadSrc(PRELOAD_URL);
    this.Log.UpdateLog("UnloadSrc: " + nRet);

Leave channel

Follow these steps to leave a channel:

  1. Call DestroyMediaPlayer to destroy the media player object.

  2. Call InitEventHandler to unregister the player event observer.

  3. Call LeaveChannel to leave the channel.

  4. Call Dispose to destroy IRtcEngine object and releases resources used by the SDK.

    public void OnLeaveChannelButtonPress()
    {
      if (RtcEngine == null) return;
      // Destroy the media player object
      if (MediaPlayer != null) RtcEngine.DestroyMediaPlayer(MediaPlayer);
      // Unregister the media player event handler
      RtcEngine.InitEventHandler(null);
      // Leave the channel
      RtcEngine.LeaveChannel();
      // Destroy the engine and release resources
      RtcEngine.Dispose();
      RtcEngine = null;
      MediaPlayer = null;
    }

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Supported formats and protocols

The media player supports the following media formats and protocols:

Video encoding formats

  • H.263, H.264, H.265, MPEG-4, MPEG-2, RMVB, Theora, VP3, VP8, AVS, WMV

Audio coding formats

  • WAV, MP2, MP3, AAC, OPUS, FLAC, Vorbis, AMR-NB, AMR-WB, WMA v1, WMA v2

Container formats

  • WAV, FLAC, OGG, MOV, ASF, FLV, MP3, MP4, MPEG-TS, Matroska (MKV), AVI, ASS, CONCAT, DTS, AVS

Supported protocols

  • HTTP, HTTPS, RTMP, HLS, RTP, RTSP

Sample project

Agora provides an open source sample project MediaPlayer on GitHub. Download it or view the source code for a more detailed example.

API reference