Screen sharing

Agora screen sharing offers the following advantages:

• Ultra HD quality experience: Supports Ultra HD video (4K resolution, 60 FPS frame rate), giving users a smoother, high-definition, ultimate picture experience.
• Multi-app support: Compatible with many mainstream apps such as WPS Office, Microsoft Office Power Point, Visual Studio Code, Adobe Photoshop, Windows Media Player, and Scratch. This makes it convenient for users to directly share specific apps.
• Multi-device support: Supports multiple devices sharing at the same time. Screen sharing is compatible with Windows 8 systems, devices without independent graphics cards, dual graphics card devices, and external screen devices.
• Multi-platform adaptation: Supports iOS, Android, macOS, Windows, Web, Unity, Flutter, React Native, Unreal Engine, and other platforms.
• High security: Supports sharing only a single app or part of the screen. Also supports blocking specified app windows, effectively ensuring user information security.

This page shows you how to implement screen sharing in your app.

Understand the tech

The screen sharing feature provides the following screen sharing modes for use in various use-cases:

Screen sharing use-cases

• Share the entire screen: Share your entire screen, including all the information on the screen. This feature supports collecting and sharing information from two screens at the same time.
• Share an app window: If you don't want to share the entire screen with other users, you can share only the area within an app window.
• Share a designated screen area: If you only want to share a portion of the screen or app window, you can set a sharing area when starting screen sharing.

Screen sharing modes are available on different platforms as follows:

• Desktop (Windows and macOS): Supports all screen sharing features listed above.

• Mobile (Android and iOS): Only supports sharing the entire screen.

Advanced features

You can enable the following advanced screen sharing features by adjusting the parameters when starting screen sharing:

• Shield a specified app window: When sharing the screen, if you don't want to expose a specific app window, you can choose to shield it so that the window will not appear in the shared screen.
• Stroke: If you need to outline the area being shared, you can stroke a specified app window or screen and customize the width, color, and transparency of the stroke.
• Pin an app window to the front: If you share multiple app windows at the same time, the windows may block each other. You can specify an app window and bring it to the front to prevent it from being blocked by other windows.
• Set the sharing scenario: The SDK automatically adjusts the Quality of Experience (QOE) policy according to the scenario you choose:
  • When sharing documents, slides, tables, or in remote control applications, set the sharing scene to either document or remote control. The SDK prioritizes optimizing image quality and minimizing latency, thus enhancing the viewing experience for the recipient of the shared video.
  • When sharing games, movies, or live video broadcasts, set the shared scene to game or video. The SDK prioritizes smoothness of video at the receiving end.

Prerequisites

• To implement screen sharing, please make sure not to manually remove the screen sharing dynamic library AgoraScreenCaptureExtension.xcframework when integrating the SDK, otherwise the function will not work properly.

• Ensure that you have implemented the SDK quickstart in your project.

Set up your project

Implement screen sharing

This section shows you how to implement screen sharing in your project. The basic API call sequence is shown in the figure below:

Screen share process

Depending on your business use-case, you can choose either of the following methods to implement screen sharing:

• Call startScreenCaptureByDisplayId or startScreenCaptureByWindowId before joining the channel, then call joinChannelByToken[2/4] to join the channel and set publishScreenTrack or publishSecondaryScreenTrack to true to start screen sharing.

• Call startScreenCaptureByDisplayId or startScreenCaptureByWindowId after joining the channel, then call updateChannelWithMediaOptions to set publishScreenTrack or publishSecondaryScreenTrack to true to start screen sharing.

The flow diagram and implementation steps in this article demonstrate the first use-case.

Get resource list

Call getScreenCaptureSourcesWithThumbSize to get an object list of screens and windows that can be shared. The list contains important information such as window ID and screen ID. This enables users to choose a certain screen or a window, through the thumbnails in the list, for sharing.

// Get available screen information
@IBAction func onScreentThumbnailButton(_ sender: NSButton) {
  let result = agoraKit.getScreenCaptureSources(withThumbSize: NSScreen.main?.frame.size ?? .zero, iconSize: .zero, includeScreen: true)
  saveThumbnailToDesktop(result: result, type: .screen)
}

// Get information about available windows
@IBAction func onWindowThumbnailButton(_ sender: NSButton) {
  let result = agoraKit.getScreenCaptureSources(withThumbSize: selectedResolution?.size() ?? .zero, iconSize: .zero, includeScreen: true)
  saveThumbnailToDesktop(result: result, type: .window)
}

Enable screen capture

Depending on the actual application use-case, choose one of the following three screen sharing methods.

Share the entire screen

Call startScreenCaptureByDisplayId with the following parameters to start sharing the whole screen:

• Set displayId to the sourceId (screen ID) obtained in the previous step.
• Set your desired video encoding properties in captureParams:
  • In document scenes or remote control scenes, best practice is to set dimensions to 1920 × 1080 and frameRate as 10 FPS.
  • In game scenes or video scenes, best practice is to set dimensions to 960 × 720 and frameRate to 15 FPS.

// Start sharing the specified screen
let result = agoraKit.startScreenCapture(byDisplayId: UInt32(screen.id), regionRect: .zero, captureParams: params)

Share app window

Call startScreenCaptureByWindowId with the following parameters to start sharing an entire app window:

• Set windowId to the sourceId (window ID) obtained in the previous step.
• Set your desired video encoding properties in captureParams:
  • In document scenes or remote control scenes, best practice is to set dimensions to 1920 × 1080 and frameRate as 10 FPS.
  • In game scenes or video scenes, best practice is to set dimensions to 960 × 720 and frameRate to 15 FPS.

// Start sharing a specified app window
let result = agoraKit.startScreenCapture(byWindowId: UInt32(window.id), regionRect: .zero, captureParams: params)

Share a designated area

Call startScreenCaptureByDisplayId or startScreenCaptureByWindowId method to start sharing, and set regionRect to the position of the area you want to share relative to the whole screen or window. The sample code is as follows:

// Set the parameters of the area you want to share
// The next line of code specifies a rectangle with a length and width of 100px and an x-coordinate and y-coordinate of 0 in the upper-left corner of the rectangle.
let captureRect = CGRect(x: 0, y: 0, width: 100, height: 100)
let result = agoraKit.startScreenCapture(byWindowId: UInt32(window.id), regionRect: captureRect, captureParams: params)

Set up a screen sharing scenario

Call the setScreenCaptureScenario method to set the screen sharing scenario. Set the scenarioType to any one of the following according to your usage scenario:

• AgoraScreenScenarioDocument: Document scenario
• AgoraScreenScenarioGaming: Game scenario
• AgoraScreenScenarioVideo: Video scenario
• AgoraScreenScenarioRDC: Remote control scenario

The sample code is as follows:

agoraKit.setScreenCaptureScenario(.video)

Capture system audio (Optional)

To capture and publish the audio played in the shared screen or window simultaneously, call enableLoopbackRecording to start sound card capture.

After you call this method, the audio played by other processes in the system is published to the remote end. To turn off sound card acquisition, call the method again.

Join a channel and publish a screen sharing video stream

Call joinChannelByToken[2/4] to join the channel and set the channel media options.

// Set the channel profile
agoraKit.setChannelProfile(.liveBroadcasting)
// Set the user role
agoraKit.setClientRole(.broadcaster)
// Set channel media options
// Publish screen capture video in the channel
// To publish a second screen capture video in the channel, replace the next line of code with the following: mediaOptions.publishSecondaryScreenTrack = true
mediaOptions.publishScreenTrack = true
// Join the channel
let result = self.agoraKit.joinChannel(byToken: token, channelId: channel, uid: 0, mediaOptions: option)

Additional notes

This example demonstrates publishing the screen sharing stream directly in the channel without publishing the stream captured by the camera. In a real-world scenario, the requirements may be different. Adjust the code logic based on your scenario.

To publish the stream captured by the camera and the screen sharing stream at the same time, refer to the following steps:

• Call joinChannelByToken[2/4] to join the first channel and publish the video stream recorded by the camera in that channel.
• Call joinChannelEx to join a second channel in which to publish the screen sharing stream.

Update screen sharing area or parameters

To dynamically update the screen sharing area or parameters in the channel, call the following methods:

• To update the shared region of the screen, call the updateScreenCaptureRegion method to reset the regionRect parameter and define a new sharing area.

• To update the screen sharing parameters, such as video encoding resolution, frame rate, or bitrate, to stroke the screen or app window, or to block a specified window, call the updateScreenCaptureParameters method and update the captureParams parameter configuration.

// Update screen sharing area
let region = NSMakeRect(0, 0, !toggleRegionalScreening ? rect!.width/2 : rect!.width, !toggleRegionalScreening ? rect!.height/2 : rect!.height)
agoraKit.updateScreenCaptureRegion(region)
// Update screen sharing parameters
agoraKit.updateScreenCaptureParameters(capParam)

Stroke the screen

Depending on your scenario, you may stroke the shared screen or window in one of the following ways:

• Stroke when enabling screen sharing: Call startScreenCaptureByDisplayId or startScreenCaptureByWindowId, set highLighted in captureParams to true, and set both highLightColor and highLightWidth to specify the stroke color and width.

• Stroke after screen sharing is enabled: Call updateScreenCaptureParameters, set highLighted in captureParams to true, and set both highLightColor and highLightWidth to specify the stroke color and width.

let params = AgoraScreenCaptureParameters()
params.frameRate = FPS
params.dimensions = resolution.size()
// Set the stroke width of the screen
params.highLightWidth = 5
// Set the stroke color of the screen
params.highLightColor = .green
// Enable screen stroke
params.highLighted = true
let result = agoraKit.startScreenCapture(byDisplayId: UInt32(screen.id), regionRect: .zero, captureParams: params)

Block windows

Depending on your usage scenario, you may block a specified window in one of the following ways:

• Block windows when enabling screen sharing: Call startScreenCaptureByDisplayId, set excludeWindowList in captureParams to the list of windows you want to block.

• Block windows after screen sharing is enabled: Call updateScreenCaptureParameters, set excludeWindowList in captureParams to the list of windows you want to block.

let captureParams = AgoraScreenCaptureParameters()
captureParams.excludeWindowList = windowlist.filter({ $0.id != window.id })
agoraKit.updateScreenCaptureParameters(captureParams)

Stop screen sharing

Call stopScreenCapture to stop screen sharing within a channel.

// Stop screen sharing
agoraKit.stopScreenCapture()

Limitations

Be aware of the following limitations:

• After turning on screen sharing, Agora uses the resolution of the screen sharing video stream as the billing standard. Please see the billing instructions for details. The default resolution is 1280 × 720, but you can adjust it according to your business needs.
• To share 4K resolution Ultra-HD video during screen sharing, your device needs to meet certain requirements. The minimum device specifications recommended by Agora are: 2021 M1 MacBook Pro 16-inch.

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

Sample project

Agora provides an open source macOS sample project on GitHub. Download and explore this project for a more detailed example.

API reference

• startScreenCaptureByDisplayId
• startScreenCaptureByWindowId
• updateScreenCaptureRegion
• updateScreenCaptureParameters
• stopScreenCapture