Appliance plugin

The Appliance Plugin is a high-performance whiteboard tool that enhances teaching aids with advanced annotation features. It includes laser pen support, customizable strokes and shapes, split-screen display, minimap navigation, element filtering, and searchable icon stickers.

This feature is for Web only.

This page describes how to integrate and use the Appliance Plugin in your app.

Understand the tech

The plugin uses the 2D functionality of SpriteJS, supports WebGL2 rendering, and remains compatible with WebGL and Canvas2D as fallback options. It employs a dual web Worker and OffscreenCanvas mechanism to handle drawing calculations and rendering in a separate worker thread, minimizing CPU usage on the main thread.

Prerequisites

Before you begin, ensure that you have:

Integrated the Fastboard SDK into your project.
Implemented the logic to join a whiteboard room.

Integrate the plugin

Follow these steps to integrate the Appliance Plugin:

Install the package

To install the plugin, open a terminal in your project folder and execute the following command:

npm install @netless/appliance-plugin

Import the dependencies

The plugin supports two modes:

Single whiteboard mode

import { ApplianceSinglePlugin, ApplianceSingleWrapper } from '@netless/appliance-plugin';

Multi-window mode

import { ApplianceMultiPlugin } from '@netless/appliance-plugin';

Load the Appliance Plugin workers

The Appliance Plugin requires Web Workers to handle specific tasks efficiently. You can load these workers in two ways:

Using a CDN (Recommended): No need to import local worker files.
Using local distribution (dist): Manually import the worker files and configure them properly.

The following code demonstrates how to load Web Workers from local distribution and integrate them with Fastboard:

// Import worker scripts as raw strings from local distribution

// Convert to Blob URLs for worker usage
const fullWorkerBlob = new Blob([fullWorkerString], { type: 'text/javascript' });
const fullWorkerUrl = URL.createObjectURL(fullWorkerBlob);
const subWorkerBlob = new Blob([subWorkerString], { type: 'text/javascript' });
const subWorkerUrl = URL.createObjectURL(subWorkerBlob);

Once the workers are loaded, integrate them with Fastboard as follows:

Fastboard React integration

import { useFastboard, Fastboard } from "@netless/fastboard-react";

const app = useFastboard(() => ({
    sdkConfig: { ... },
    joinRoom: { ... },
    managerConfig: {
      cursor: true,
      enableAppliancePlugin: true,
      ...
    },
    enableAppliancePlugin: {
      cdn: {
          fullWorkerUrl,
          subWorkerUrl,
      }
    }
}));

Fastboard standalone integration

import { createFastboard, createUI } from "@netless/fastboard";

const fastboard = await createFastboard({
    sdkConfig: { ... },
    joinRoom: { ... },
    managerConfig: {
      cursor: true,
      supportAppliancePlugin: true,
      ...
    },
    enableAppliancePlugin: {
      cdn: {
          fullWorkerUrl,
          subWorkerUrl,
      }
    }
});

Integrate the Appliance Plugin with Whiteboard and WindowManager

The Appliance Plugin extends the functionality of the Netless Whiteboard by enabling additional interactive tools. This section demonstrates how to:

Set up WhiteWebSdk and join a room
Mount WindowManager for multi-window support
Enable the Appliance Plugin with Web Workers

Import required dependencies

To use WindowManager and the Appliance Plugin, import the necessary styles and modules:

import '@netless/window-manager/dist/style.css';
import '@netless/appliance-plugin/dist/style.css';

import { WhiteWebSdk } from "white-web-sdk";
import { WindowManager } from "@netless/window-manager";
import { ApplianceMultiPlugin } from '@netless/appliance-plugin';

Load Web Workers for the Appliance Plugin

If using local distribution (dist), manually import worker files and convert them into Blob URLs:

import fullWorkerString from '@netless/appliance-plugin/dist/fullWorker.js?raw';
import subWorkerString from '@netless/appliance-plugin/dist/subWorker.js?raw';

const fullWorkerBlob = new Blob([fullWorkerString], { type: 'text/javascript' });
const fullWorkerUrl = URL.createObjectURL(fullWorkerBlob);
const subWorkerBlob = new Blob([subWorkerString], { type: 'text/javascript' });
const subWorkerUrl = URL.createObjectURL(subWorkerBlob);

Initialize WhiteWebSdk and join a room

Create a WhiteWebSdk instance and join a whiteboard room:

const whiteWebSdk = new WhiteWebSdk(...);

const room = await whiteWebSdk.joinRoom({
    ...
    invisiblePlugins: [WindowManager, ApplianceMultiPlugin],
    useMultiViews: true,
});

Mount WindowManager and Enable the Appliance Plugin

After joining the room, mount WindowManager and enable the ApplianceMultiPlugin:

const manager = await WindowManager.mount({
    room,
    container: elm,
    chessboard: true,
    cursor: true,
    supportAppliancePlugin: true
});

if (manager) {
    await ApplianceMultiPlugin.getInstance(manager, {
        options: {
            cdn: {
                fullWorkerUrl,
                subWorkerUrl,
            }
        }
    });
}

Using the Appliance Plugin with a single Whiteboard

The Appliance Plugin enhances the WhiteWebSdk by adding interactive tools for a single whiteboard instance. This section explains how to:

Set up WhiteWebSdk and join a room
Enable the Appliance Plugin for a single whiteboard
Load Web Workers (if needed)

Import the required dependencies

import { WhiteWebSdk } from "white-web-sdk";
import { ApplianceSinglePlugin, ApplianceSigleWrapper } from '@netless/appliance-plugin';

Load Web Workers for the Appliance Plugin

If using local distribution (dist), manually import worker scripts and convert them into Blob URLs:

import fullWorkerString from '@netless/appliance-plugin/dist/fullWorker.js?raw';
import subWorkerString from '@netless/appliance-plugin/dist/subWorker.js?raw';

const fullWorkerBlob = new Blob([fullWorkerString], { type: 'text/javascript' });
const fullWorkerUrl = URL.createObjectURL(fullWorkerBlob);
const subWorkerBlob = new Blob([subWorkerString], { type: 'text/javascript' });
const subWorkerUrl = URL.createObjectURL(subWorkerBlob);

If using a CDN, you do not need to manually import worker scripts. :::

Initialize WhiteWebSdk and join a room

Create a WhiteWebSdk instance and join a whiteboard room:

const whiteWebSdk = new WhiteWebSdk(...);

const room = await whiteWebSdk.joinRoom({
    ...
    invisiblePlugins: [ApplianceSinglePlugin],
    wrappedComponents: [ApplianceSigleWrapper]
});

Enable the Appliance Plugin

After joining a room, initialize the ApplianceSinglePlugin:

await ApplianceSinglePlugin.getInstance(room, {
    options: {
        cdn: {
            fullWorkerUrl,
            subWorkerUrl,
        }
    }
});

Implement features

This section describes how to use various features of the Appliance Plugin.

Use the laser pen feature

The laser pen feature allows users to annotate with a pencil tool and draw temporary content that disappears after a second.

To implement this feature:

Import the required dependencies:

import { EStrokeType, ApplianceNames } from '@netless/appliance-plugin';

Set stroke type:

// Solid line
setMemberState({strokeType: EStrokeType.Normal });

// Line with pen edge
setMemberState({strokeType: EStrokeType.Stroke });

// Dotted line
setMemberState({strokeType: EStrokeType.Dotted });

// Long dotted line
setMemberState({strokeType: EStrokeType.LongDotted });

Set stroke opacity:
```
setMemberState({strokeOpacity: 0.5 });
```

Set the text properties:

setMemberState({textOpacity: 0.5, textBgOpacity: 0.5, textBgColor:[0, 0, 0]});

Use different shapes

Polygons:

// Regular pentagon
setMemberState({currentApplianceName: ApplianceNames.shape, shapeType: ShapeType.Polygon, vertices: 5});

Stars:

// Fat hexagonal star
setMemberState({currentApplianceName: ApplianceNames.shape, shapeType: ShapeType.Star, vertices: 12, innerVerticeStep: 2, innerRatio: 0.8});

Speech Balloon:

// The dialog box in the lower left corner
setMemberState({currentApplianceName: ApplianceNames.shape, shapeType: ShapeType.SpeechBalloon, placement: 'bottomLeft'});

Set shape fill color and fill opacity:

setMemberState({fillOpacity: 0.5, fillColor:[0, 0, 0]});

Split the display screen

To implement a split screen display, integrate the @netless/app-little-white-board (Version >=1.1.3).

Use the mini map feature

The mini-map feature displays a certain part of the whiteboard at a different zoom level than the main map control, allowing easier navigation.

// Version >=1.1.6
/** Create a minimap
 * @param viewId ID of the whiteboard under windowManager. The ID of the main whiteboard is mainView, and the ID of other whiteboards is the appID of addApp() return
 * @param div Small map DOM container
 */
createMiniMap(viewId: string, div: HTMLElement): Promise<void>;
/** Destroy minimap */
destroyMiniMap(viewId: string): Promise<void>;

Filter annotations

This feature allows you to control the visibility of annotations on a whiteboard for different users.

/** Filter Elements
 * @param viewId ID of the whiteboard under windowManager. The ID of the main whiteboard is mainView, and the ID of other whiteboards is the appID of addApp() return
 * @param filter filter condition
 *  render: Whether notes can be rendered, [uid1, uid2,...] Or true. true, that is, both render, [uid1, uid2,...] The collection of user Uids rendered for the specified
 *  hide: Note is hidden, [uid1, uid2,...] Or true. true, that is to hide, [uid1, uid2,...] To specify a hidden user uid collection
 *  clear: Whether notes can be cleared, [uid1, uid2,...] Or true. true, that is, can be cleared, [uid1, uid2,...] Specifies a collection of user Uids that can be cleared
 * @param isSync Whether to synchronize data to other users. The default value is true, that is, the data will be synchronized to other users
 */
filterRenderByUid(viewId: string, filter: { render?: _ArrayTrue, hide?: _ArrayTrue, clear?: _ArrayTrue}, isSync?:boolean): void;
/** Filter Elements
 * @param viewId ID of the whiteboard under windowManager. The ID of the main whiteboard is mainView, and the ID of other whiteboards is the appID of addApp() return
 * @param isSync Whether to synchronize data to other users. The default value is true, that is, the data will be synchronized to other users. Keep it the same as the filterRenderByUid setting
 */
cancelFilterRender(viewId: string, isSync?:boolean): void;

Use the auto draw feature

The auto draw feature is designed to automatically recognize and suggest related graphical elements based on a user's hand-drawn sketches on a whiteboard.

export type AutoDrawOptions = {
    /** Automatically associate rest api addresses */
    hostServer: string;
    /** A container that holds a list of associated icons */
    container: HTMLDivElement;
    /** How long does the drawing end start activating the association */
    delay?: number;
};
const plugin = await ApplianceMultiPlugin.getInstance(...);
const autoDrawPlugin = new AutoDrawPlugin({
    container: topBarDiv,
    hostServer: 'https://autodraw-white-backup-hk-hkxykbfofr.cn-hongkong.fcapp.run',
    delay: 2000
});
plugin.usePlugin(autoDrawPlugin);

Reference

This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

API References

cleanCurrentScene: Clears the current whiteboard scene.
setMemberState: Updates the whiteboard member's state, such as stroke type and color.
undo: Reverts the last action performed on the whiteboard.
redo: Restores the last undone action on the whiteboard.
callbacks: Registers event callbacks for appliance-plugin interactions.
insertImage: Inserts an image into the whiteboard.
lockImage: Locks an inserted image to prevent modifications.
completeImageUpload: Finalizes the image upload process to the whiteboard.
getImagesInformation: Retrieves metadata about images added to the whiteboard.
destroy: Destroys the instance of the appliance-plugin.
addListener: Adds an event listener to the appliance-plugin.
removeListener: Removes an event listener from the appliance-plugin.

Deprecated

The following APIs have been renamed or changed:

room.getBoundingRect: Use getBoundingRectAsync instead.
room.screenshotToCanvas: Use screenshotToCanvasAsync instead.
room.scenePreview: Use scenePreviewAsync instead.

Incompatible

With the appliance Plugin, these APIs are incompatible:

exportScene: When the appliance-plugin is enabled, notes cannot be exported in room mode.
Server-side screenshot: After enabling the appliance-plugin, notes cannot be obtained using the server-side screenshot API. Use screenshotToCanvasAsync instead.

Webpack configuration for `?raw` imports

The Appliance Plugin requires loading Web Worker scripts. If you're using Vite, it supports ?raw imports out of the box. However, Webpack needs additional configuration to handle ?raw as a raw text asset. If your project uses Webpack, add the following rule to your webpack.config.js file:

module.exports = {
    module: {
        rules: [
            // Default rule for JavaScript files
            {
                test: /\.m?js$/,
                resourceQuery: { not: [/raw/] }, // Exclude '?raw' imports
                use: [ ... ] // Your existing loaders
            },
            // Custom rule for handling '?raw' imports
            {
                resourceQuery: /raw/,
                type: 'asset/source', // Treats it as a raw text asset
            }
        ]
    }
};

Configuration parameters

getInstance(wm: WindowManager, adaptor: ApplianceAdaptor)

wm: WindowManager\room\player. In multi-window mode, you pass WindowManager, and in single-window mode, you pass room or player(whiteboard playback mode).
adaptor: configures the adapter.
- options: AppliancePluginOptions; The cdn addresses of both workers must be configured.
```
export type AppliancePluginOptions = {
    /** cdn Configuration item */
    cdn: CdnOpt;
    /** Synchronize data configuration items */
    syncOpt? : SyncOpt;
    /** Canvas configuration item */
    canvasOpt? : CanvasOpt;
    /** stroke width range */
    strokeWidth?: {
        min: number,
        max: number,
    }
}
```
  - cursorAdapter?: CursorAdapter; This parameter is optional. In single whiteboard mode, customize the mouse style.
  - logger?: Logger; This parameter is optional. Configure the log printer object. The default output is on the local console. To uploaded the log to the specified server, manually set the configuration.

Front-end debugging

To understand and track the internal status of the plug-in during the interconnection process, you can view the internal data using the following console commands.

const applianPlugin = await ApplianceSinglePlugin.getInstance(...)
applianPlugin.CurrentManager  // can see the package version number, internal state, etc
applianPlugin.CurrentManager.ConsoleWorkerInfo () // can check information to draw on the worker