electron-window-rtc

Inspired by electron-peer-connection, electron-window-rtc is a zero-dependency package that allows sharing medias between Electron windows through WebRTC with (almost) zero-latency, depending on application configuration.

It works by creating a main process events hub that acts as a signaling server for the windows through Electron's IPC. Once windows are registered, each renderer process creates a WindowRTCPeerConnection to another window and begins to send/receive media streams.

This package was primarily released to handle video/canvas manipulation before sending it to a window rendered offscreen passed to the Syphon framework with same author's node-syphon package.

Donate

If you find this package useful, contribute to the author's open source work by donating here! Thank you!

Table of Contents

• electron-window-rtc
  • Donate
  • Table of Contents
  • Install
  • Usage
    • Main process
    • Renderer process
      • Sender
      • Receiver
  • API
    • Main process
      • WindowRTCMain.register
      • WindowRTCMain.unregister
      • WindowRTCMain.dispose
    • Renderer process
      • defineIpc
      • WindowRTCPeerConnection.with
      • windowPeerConnectionInstance.addStream
      • windowPeerConnectionInstance.requestOffer
      • windowPeerConnectionInstance.on
      • windowPeerConnectionInstance.off
      • windowPeerConnectionInstance.dispose
    • Events
    • Types
      • IpcObject
      • WindowRTCEvent
  • Using canvas
    • Setting frameRequestRate
    • Without setting frameRequestRate
  • Using WebAudio API
  • Known Issues
  • License

Install

npm i -s electron-window-rtc

yarn add electron-window-rtc

Usage

See Electron+Vue example for a complete integration example.

Main process

// WindowRTCMain is a singleton.
import { WindowRTCMain } from 'electron-window-rtc';

const senderWindow: BrowserWindow = createWindow(); // Left to application.
const receiverWindow: BrowserWindow = createWindow();

// Note that windows can both send and receive by creating two connection (see below in 'renderer process' section).

WindowRTCMain.register('sender', senderWindow);
WindowRTCMain.register('receiver', receiverWindow);

Renderer process

Sender

import {
  WindowRTCPeerConnection,
  defineIpc,
} from 'electron-window-rtc/renderer';

// Important: define early how to access to IPC object, according to application 'preload' script.
// The IPC object must at least expose `on`, `removeListener`, `send` and `invoke` methods (see IPCObject below).
defineIpc(window.electron.ipcRenderer);

document.addEventListener('DOMContentLoaded', (event) => {
  const windowConnection = new WindowRTCPeerConnection('receiver');

  // Canvas for example...
  const canvas: HTMLCanvasElement = document.getElementById('canvas');

  // Add track from canvas: this will create an 'offer' for 'receiver' window.
  // Note the '240' fps framerate: leaving it empty creates latency in the receiver.
  windowConnection.addStream(canvas.captureStream(240));
});

Receiver

import {
  WindowRTCPeerConnection,
  defineIpc,
} from 'electron-window-rtc/renderer';

// Important: define early how to access to IPC object, according to application 'preload' script.
// The IPC object must at least expose `on`, `removeListener`, `send` and `invoke` methods (see IPCObject below).
defineIpc(window.electron.ipcRenderer);

document.addEventListener('DOMContentLoaded', (event) => {
  const windowConnection = new WindowRTCPeerConnection('sender');

  // Listen to 'track' added by 'sender' window.
  windowConnection.on('track', (event: EventManagerDTO) => {
    const video: HTMLVideoElement = document.getElementById('video');

    const trackEvent: RTCTrackEvent = event.payload;
    const streams = trackEvent.streams;
    for (const stream of streams) {
      // For the sake of this example, keep only one stream, we could also get `streams[0]`.
      video.srcObject = null;
      video.srcObject = stream;
    }
  });
});

API

Main process

Import the singleton.

import { WindowRTCMain } from 'electron-window-rtc';

WindowRTCMain.register

Registers a BrowserWindow for message passing with a unique name.

Parameters

| Name | Type | Default | Optional | | -------- | --------------- | ----------- | -------- | | name | string | undefined | false | | window | BrowserWindow | undefined | false |

Returns void

Throws Error if a window with this name or this window has already been registered.

WindowRTCMain.unregister

Unregister a BrowserWindow from message passing. Fails silently if window can not be found.

Parameters

| Name | Type | Default | Optional | | ------ | -------- | ----------- | -------- | | name | string | undefined | false |

Returns void

WindowRTCMain.dispose

Dispose the WindowRTCMain singleton by unregistering windows and removing IpcMain listeners.

Returns void

Renderer process

import { WindowRTCPeerConnection, defineIpc } from 'electron-window-rtc/renderer';

defineIpc

Define the global IpcObject to use for thiw window for communicating with main process.

Parameters

| Name | Type | Default | Optional | | ----- | ----------- | ----------- | -------- | | ipc | IpcObject | undefined | false |

Returns void

WindowRTCPeerConnection.with

Create a WindowRTCPeerConnection with window of given name.

Parameters

| Name | Type | Default | Optional | | ------ | -------- | ----------- | -------- | | name | string | undefined | false |

Returns Promise<WindowRTCPeerConnection> An instance of WindowRTCPeerConnection.

Throws Error if IpcObject was not defined with defineIpc, or if the window with name was not registered, or if this window was not registered.

windowPeerConnectionInstance.addStream

Add a stream to send to connected window and create an offer sent to receiving window.

Parameters

| Name | Type | Default | Optional | | ------------ | ----------------------------------- | ---------------------- | -------- | | stream | MediaStream | undefined | false | | maxBitrate | { audio: number; video: number; } | 5000 (Kbps) for both | true |

Returns Promise<void>

windowPeerConnectionInstance.requestOffer

Request the peer window to send an offer.

Returns Promise<void>

windowPeerConnectionInstance.on

Register a listener to WindowRTCPeerConnection instance's events.

Parameters

| Name | Type | Default | Optional | | ---------- | --------------------------------- | ----------- | -------- | | channel | WindowRTCEventChannel | undefined | false | | listener | (event: WindowRTCEvent) => void | undefined | false |

Returns void

windowPeerConnectionInstance.off

Unregister a listener or a whole channel from WindowRTCPeerConnection instance's events. If listener is left undefined, unregisters all listeners for this channel.

Parameters

| Name | Type | Default | Optional | | ----------- | --------------------------------- | ----------- | -------- | | channel | WindowRTCEventChannel | undefined | false | | listener? | (event: WindowRTCEvent) => void | undefined | true |

Returns void

windowPeerConnectionInstance.dispose

Close the connection with the other window and remove all listeners.

Returns void

Events

WindowRTCEvent (see below) sent by WindowRTCPeerConnection with different payloads according to the event emitted.

| Channel | Payload Type | Emitted | | -------------------------- | --------------------------------------------------------- | ----------------------------------------------------------- | | * | any | For each event. | | icecandidate | RTCIceCandidate | On icecandidate RTCPeerConnection event. | | iceconnectionstatechange | Event | On iceconnectionstatechange RTCPeerConnection event. | | icecandidateerror | RTCPeerConnectionIceErrorEvent | On icecandidateerror RTCPeerConnection event. | | icegatheringstatechange | Event | On icegatheringstatechange RTCPeerConnection event. | | negotiationneeded | Event | On negotiationneeded RTCPeerConnection event. | | signalingstatechange | Event | On signalingstatechange RTCPeerConnection event. | | track | RTCTrackEvent | On track RTCPeerConnection event. | | leave | undefined | Error | When local window leaves. | | peer-left | undefined | Error | When remote window leaves. | | request-offer | undefined | When a window requests an offer from its peer window. | | sent-offer | RTCSessionDescriptionInit | When a window has sent an offer. | | received-offer | { offer: SdpObject, answer: RTCSessionDescriptionInit } | When a window has received an offer and sent an answer. | | received-answer | SdpObject | When a window has received an answer. | | received-candidate | RTCIceCandidate | When a window has received an ICE candidate. | | error | Error | When an error occurred in IPC communication. |

Types

IpcObject

Describes the IPC object used by electron-window-rtc

interface IpcObject {
  on: (
    channel: string,
    callback: (event: IpcRendererEvent, ...args: any[]) => void
  ) => void;
  removeListener: (
    channel: string,
    callback: (event: IpcRendererEvent, ...args: any[]) => void
  ) => void;
  send: (channel: string, ...args: any[]) => void;
  invoke: (channel: string, ...args: any[]) => Promise<any>;
}

WindowRTCEvent

Describes the generic event data sent and received by WindowRTCPeerConnection.

interface WindowRTCEvent {
  /**
   * This window name the event was emitted from.
   */
  local: string;
  /**
   * Peer window name.
   */
  remote: string;
  payload: any;
}

// Example.
windowConnection.on('track', (event: WindowRTCEvent) => {
  const local: string = event.local;
  const receiveremoter: string = event.remote;
  const trackEvent: RTCTrackEvent = event.payload;
});

Using canvas

When using canvas to get an image to send to other windows, application should set the frameRequestRate parameter of canvas.captureStream to a high framerate to avoid latency on the receiver side.

Setting frameRequestRate

Fig. 1: const stream = canvas.captureStream(240); in Sender window doesn't induce latency. Look at the result of performance.now() sent by the Sender.

Without setting frameRequestRate

Fig. 2: Without setting frameRequestRate: a latency of ~30ms is induced.

Using WebAudio API

See example's SenderWindow and ReceiverWindow for the setup of the final stream.

Fig. 3: Sending audio for visualization with or without playing it in the Sender window.

Known Issues

• Electron example doesn't handle very well reloading windows and introduces latency. We may explore recreating stream on reload.
• Reloading Sender window takes a lot of time for Receiver window to reconnect, whereas the requestOffer method allows reconnecting quickly on Receiver window's reload.
• Closing and opening again windows has not been tested: it may involve some logic in the main process to be integrated in WindowRTCMain.
• Latency differs when using the example in development mode or production mode.

License

MIT