Picture-in-Picture

Updated

Keep video visible in a floating overlay when your app moves to the background or the user switches to another app.

When your application moves to the background or the user switches to another app, the Picture-in-Picture (PiP) feature in Video SDK keeps the current video visible as a floating overlay window on screen. This lets users multitask without interrupting an ongoing call or viewing session, improving the multitasking experience and user retention.

PiP is useful in the following scenarios:

Use caseDescription
Video callingKeeps the call visible when the user switches apps or returns to the home screen. Useful for social calls, remote meetings, and online customer service.
Online educationLets students watch a teacher's video explanation while reviewing study materials, taking notes, or using other apps.
Interactive live streamingLets viewers browse products, chat, or use other apps while the stream continues in the background. Useful for e-commerce and content live streaming.
Screen sharingKeeps shared content visible when the user switches apps during remote collaboration or a presentation. Useful for remote work, technical support, and product demos.

Prerequisites

Before implementing Picture-in-Picture, make sure you have the following:

  • Flutter 3.7.0 or later
  • Video SDK for Flutter v6.6.2 or later integrated into your project. See SDK quickstart.
  • To use PiP, the target device must run one of the following:
    • Android 8.0 or later
    • iOS 15.0 or later

Set up your project

Before using PiP, complete the required configuration for each target platform.

  1. In AndroidManifest.xml, declare that the current Activity supports PiP mode:

    <activity
        android:name=".MainActivity"
        android:supportsPictureInPicture="true"
        android:configChanges="screenSize|smallestScreenSize|screenLayout|orientation|uiMode"
        android:exported="true"
        android:launchMode="singleTop">
    </activity>
  2. Make MainActivity extend AgoraPIPFlutterActivity or AgoraPIPFlutterFragmentActivity.

    import io.agora.agora_rtc_ng.AgoraPIPFlutterActivity
    
    class MainActivity : AgoraPIPFlutterActivity()

How PiP is triggered differs by Android version:

  • Android 12 and later: The system can automatically enter PiP when the user leaves the current screen in appropriate scenarios. You can use this with autoEnterEnabled.
  • Earlier than Android 12: Automatic PiP entry via autoEnterEnabled is not supported. You must trigger PiP entry explicitly, for example by calling pipStart() in response to a user action.

When MainActivity extends AgoraPIPFlutterActivity or AgoraPIPFlutterFragmentActivity, the SDK automatically handles the onPictureInPictureModeChanged and onUserLeaveHint Activity callbacks required for PiP.

In Xcode, add the Background Modes capability to your app:

  1. Open your target's Signing & Capabilities tab.
  2. Click + Capability.
  3. Select Background Modes.
  4. Check Audio, AirPlay, and Picture in Picture.

Information

If your use case requires capturing local camera video in multitasking mode, contact support@agora.io for guidance on enabling iOS multitasking capture.

Implement Picture-in-Picture

Starting from Video SDK v6.6.2, Agora provides the AgoraPipController and related configuration options for enabling and managing PiP on Android and iOS. This section describes how to implement PiP in your Flutter project.

Get the PiP controller and register a listener

After initializing the RTC engine, call createPipController() to create a PiP controller, then register a state change listener:

late final RtcEngine _engine;
late final AgoraPipController _pip;

_engine = createAgoraRtcEngine();
await _engine.initialize(RtcEngineContext(appId: appId));

_pip = _engine.createPipController();
await _pip.registerPipStateChangedObserver(
  AgoraPipStateChangedObserver(
    onPipStateChanged: (state, error) {
      debugPrint('[PiP] state=$state error=$error');
    },
  ),
);

Check device support

Check whether the current device and system support the required capabilities:

final pipSupported = await _pip.pipIsSupported();
final pipAutoEnterSupported = await _pip.pipIsAutoEnterSupported();
  • pipIsSupported() checks whether the current device supports PiP.
  • pipIsAutoEnterSupported() checks whether the current device and system support automatic PiP entry.

Configure PiP options

Call pipSetup() to configure PiP options. Use AgoraPipOptions to set platform-specific options for Android and iOS.

On Android, you typically need to configure the aspect ratio, the initial source region, and seamless resize:

final options = AgoraPipOptions(
  autoEnterEnabled: pipAutoEnterSupported,
  aspectRatioX: 16,
  aspectRatioY: 9,
  sourceRectHintLeft: 0,
  sourceRectHintTop: 0,
  sourceRectHintRight: 0,
  sourceRectHintBottom: 0,
  seamlessResizeEnabled: true,
  useExternalStateMonitor: false,
  externalStateMonitorInterval: 100,
);

await _pip.pipSetup(options);

If your MainActivity is configured for PiP as described, but PiP state callbacks are unreliable on certain devices or in certain scenarios, set useExternalStateMonitor to true and configure the polling interval using externalStateMonitorInterval to supplement PiP state synchronization.

On iOS, you can configure the PiP window size, video stream layout, and system control style:

final options = AgoraPipOptions(
  preferredContentWidth: 960,
  preferredContentHeight: 540,
  sourceContentView: 0,
  contentView: 0,
  contentViewLayout: const AgoraPipContentViewLayout(
    padding: 0,
    spacing: 2,
    row: 1,
    column: 2,
  ),
  videoStreams: [
    AgoraPipVideoStream(
      connection: RtcConnection(
        channelId: channelId,
        localUid: localUid,
      ),
      canvas: const VideoCanvas(
        uid: 0,
        sourceType: VideoSourceType.videoSourceCamera,
        setupMode: VideoViewSetupMode.videoViewSetupAdd,
        renderMode: RenderModeType.renderModeHidden,
        mirrorMode: VideoMirrorModeType.videoMirrorModeEnabled,
      ),
    ),
    ...remoteUsers.entries.map(
      (entry) => AgoraPipVideoStream(
        connection: entry.value,
        canvas: VideoCanvas(
          uid: entry.key,
          sourceType: VideoSourceType.videoSourceRemote,
          setupMode: VideoViewSetupMode.videoViewSetupAdd,
          renderMode: RenderModeType.renderModeHidden,
        ),
      ),
    ),
  ],
  controlStyle: 2,
);

await _pip.pipSetup(options);

In this example:

  • When contentView is set to 0, the SDK manages the native views inside the PiP window. If you pass a non-zero value, you are responsible for managing the native view layout.
  • videoStreams defines the local and remote streams to display.
  • contentViewLayout configures the grid layout for video streams inside the PiP window. Streams are arranged from left to right, then top to bottom.
  • controlStyle: 2 hides the play/pause button and progress bar, keeping only the essential window control buttons. This is recommended for real-time interactive scenarios.

Listen for PiP state changes

Implement AgoraPipStateChangedObserver to listen for PiP state changes. Use the AgoraPipState value to update the UI or handle errors:

await _pip.registerPipStateChangedObserver(
  AgoraPipStateChangedObserver(
    onPipStateChanged: (state, error) async {
      if (state == AgoraPipState.pipStateFailed) {
        await _pip.pipDispose();
      }
    },
  ),
);

AgoraPipState defines the following states:

  • pipStateStarted: PiP has started successfully.
  • pipStateStopped: PiP has stopped.
  • pipStateFailed: PiP failed to start or encountered an error while running.

Start, stop, and release PiP

Call pipStart() to start PiP. The recommended approach for stopping and releasing PiP differs by platform:

// Start PiP
await _pip.pipStart();

// Release the current PiP session
await _pip.pipDispose();

// Destroy the controller
await _pip.dispose();

On Android, pipStop() is not the standard path for exiting PiP and restoring the foreground page. If you no longer need the current PiP session, call pipDispose() to release resources. Restoring full-screen view is typically handled by the user through the system PiP window controls.

Call pipStop() to stop PiP and restore the app interface. If the page is about to be destroyed or you no longer need PiP, call pipDispose() and dispose() to release resources.

// Start PiP
await _pip.pipStart();

// Stop PiP and restore the app interface
await _pip.pipStop();

// Release the current PiP session
await _pip.pipDispose();

// Destroy the controller
await _pip.dispose();

Development considerations

This section covers platform-specific behavior and best practices to keep in mind when integrating PiP.

  • After entering PiP on Android, you typically need to update the app UI to match the PiP state. Many apps show only the video area while PiP is active.
  • Android 12 and later support automatic PiP entry when the app moves to the background. On earlier versions, you must trigger PiP entry explicitly, for example by calling pipStart() in response to a user action.
  • pipStart() does not guarantee success in all timing scenarios. Calling it when the app moves to the background is generally more reliable.
  • PiP does not bypass Android background restrictions. If you need to continue capturing or playing audio and video in the background, combine PiP with a foreground service. For details, see Android background restrictions.
  • On Android, pipStop() moves the task to the background rather than exiting PiP and restoring the foreground page. Full-screen restoration is typically handled by the user through the system PiP window controls.
  • Call pipDispose() or dispose() promptly when leaving a channel, destroying the page, or when PiP is no longer needed, to avoid resource leaks.
  • PiP must be triggered by explicit user action only, such as tapping a button or swiping up to the home screen. Triggering PiP programmatically without a user action may result in App Store review rejection.
  • The PiP window is managed by the system and the SDK. Your app does not need to render a custom floating window.
  • To display multiple video streams in PiP mode, configure the layout explicitly using videoStreams and contentViewLayout. When remote users join or leave, call pipSetup() again with the updated videoStreams; you do not need to call pipDispose() first.
  • AgoraPipOptions.controlStyle supports four control styles. Style 2 is recommended for video call scenarios. Style 3 hides all controls, including the close and restore buttons.
  • Call pipDispose() or dispose() promptly when leaving a channel, destroying the page, or when PiP is no longer needed, to avoid resource leaks.

Reference

This section includes links to related API reference and additional resources.

API reference

Sample project

Agora provides a sample project demonstrating Picture-in-Picture. Download or view the source code.