@promiseqube/webrtc-core

Framework-agnostic WebRTC client for promiseQ. It gives a third-party app live view, DVR playback, PTZ, and alert images/clips from a qube's cameras over WebRTC — the same wire protocol the promiseQ operator UIs use, with no framework dependency (plain TypeScript, works in Angular, React, Vue, or vanilla).

How it works

• One RTCPeerConnection per qube, multiplexing every granted camera's video track. Each inbound track is routed to a <video> by stream.id (= camera_id).
• The agent (qube / nvr-core) is the offerer; this client answers. ICE is trickled both ways.
• A single agent-created ctl data channel carries all app control — DVR (seek/speed/play/pause/live), PTZ, push-to-talk, and alert image/clip transfer (chunked base64, reassembled to blob URLs). It is orthogonal to signaling.
• Signaling is pluggable (SignalingTransport). Two transports ship:
  • CloudWSTransport — via cloud-backend \/ws/signaling, multi-qube, sends the access token as the first WS frame.
  • LocalWSTransport — direct to a qube's \/signaling/viewer on the LAN.

One connection per instance

A handler instance opens exactly one signaling connection and reuses it for everything. connect() establishes it (or the first connectQube() does, lazily); every connectQube(qubeId) afterwards multiplexes a new viewer session over that same socket. disconnectQube(qubeId) ends one qube's session but leaves the shared connection open for the others; destroy() closes the connection and frees all resources. If the socket drops, the handler reconnects it once and re-joins every still-wanted qube. So: initialise the class once, then call operations on that instance — you never manage sockets yourself.

Install

npm install @promiseqube/webrtc-core

Quick start (cloud / third-party)

import { createCloudClient } from "@promiseqube/webrtc-core";

// token: an Auth0 JWT OR a promiseQ system-generated access token (OAuth2).
const client = createCloudClient({
  apiBaseUrl: "https://cloud.example.com",
  token: () => myAccessToken,        // getter → fresh token on every (re)connect
});

await client.connect();              // optional: open the one signaling connection up front

const qubeId = "…";

client.on(qubeId, (e) => {
  switch (e.type) {
    case "status":  /* e.status: connecting | connected | reconnecting | … */ break;
    case "cameras": /* e.data: WebRTCCameraInfo[] — attach a <video> per camera */ break;
    case "pos":     /* DVR position tick */ break;
    case "alerts":  /* alert list */ break;
    case "alert-image": /* { id, image } blob URL */ break;
  }
});

await client.connectQube(qubeId);
// once you have <video> elements (e.g. from the `cameras` event):
client.attachVideoElement(cameraId, videoEl, qubeId);

Control surface (all on WebRTCP2PHandler)

| Area | Methods | |---|---| | Lifecycle | connectQube(qubeId), disconnectQube(qubeId), destroy() | | Video | attachVideoElement(camId, el, qubeId), detachVideoElement, setAudio, setVideoPaused | | DVR | seek(camId, time), goLive, play, pause, setSpeed(camId, x), requestSegments | | PTZ | ptz(camId, dir) where dir ∈ up/down/left/right/zoom-in/zoom-out, ptzStop(camId) | | Talk | talkStart(camId), talkStop(camId) | | Alerts | requestAlertEvents(camId, from, to), requestAlerts(query), requestAlertImage(camId, id), requestAlertClip(camId, id) | | Bandwidth | hibernate(camId), wake(camId) | | Stats | getStats(camId, qubeId) |

PTZ is click-to-nudge: fire a direction, then ptzStop after ~500ms (the agent runs ONVIF ContinuousMove until stop).

Recipes

All snippets are framework-agnostic (vanilla TS/DOM) so they drop straight into Angular, React, or plain JS. They share this setup:

import { createCloudClient } from "@promiseqube/webrtc-core";

const client = createCloudClient({ apiBaseUrl, token });   // token: Auth0 or system token
const qubeId = "…";

Everything the agent reports (camera list, positions, segments, alerts, media) arrives through client.on(qubeId, cb). The camera list — with the ptz and recording_watch capability flags — comes as a cameras event shortly after the connection is up.

1. Live streams for ALL cameras in a qube

// Render every granted camera into its own <video>. One shared connection is
// opened for the qube and all camera tracks are multiplexed over it.
const grid = document.getElementById("grid")!;

client.on(qubeId, (e) => {
  if (e.type === "cameras") {
    for (const cam of e.data as import("@promiseqube/webrtc-core").WebRTCCameraInfo[]) {
      let video = document.getElementById(`cam-${cam.id}`) as HTMLVideoElement | null;
      if (!video) {
        video = document.createElement("video");
        video.id = `cam-${cam.id}`;
        video.autoplay = video.muted = video.playsInline = true;
        grid.appendChild(video);
      }
      client.attachVideoElement(cam.id, video, qubeId);   // routes that camera's track here
    }
  }
});

await client.connectQube(qubeId);

2. Live stream for ONE selected camera in a qube

const cameraId = "…";
const video = document.getElementById("player") as HTMLVideoElement;

client.on(qubeId, (e) => {
  if (e.type === "cameras") {
    const cams = e.data as import("@promiseqube/webrtc-core").WebRTCCameraInfo[];
    client.attachVideoElement(cameraId, video, qubeId);
    // Optional: stop paying for the streams you're not showing.
    cams.filter((c) => c.id !== cameraId).forEach((c) => client.hibernate(c.id, qubeId));
  }
});

await client.connectQube(qubeId);
// later, to switch: client.wake(otherId, qubeId); client.attachVideoElement(otherId, video, qubeId);

3. PTZ (pan / tilt / zoom)

// Only cameras whose `cameras` entry has ptz === true accept these.
type Dir = "up" | "down" | "left" | "right" | "zoom-in" | "zoom-out";

// Click-to-nudge: move, then stop after a beat (agent runs ONVIF ContinuousMove until stop).
function nudge(cameraId: string, dir: Dir) {
  client.ptz(cameraId, dir, qubeId);
  setTimeout(() => client.ptzStop(cameraId, qubeId), 500);
}

// Press-and-hold: start on mousedown, stop on mouseup.
btn.addEventListener("mousedown", () => client.ptz(cameraId, "left", qubeId));
btn.addEventListener("mouseup", () => client.ptzStop(cameraId, qubeId));

nudge(cameraId, "zoom-in");

4. Recordings / DVR playback

// Cameras with recording_watch === true have a DVR timeline.
client.on(qubeId, (e) => {
  if (e.type === "segments") {
    // e.data: { start: number; end: number }[]  (epoch ms) — the recorded ranges
    console.log("recorded ranges for", e.cameraId, e.data);
  }
  if (e.type === "pos") {
    // e.data: { mode: "live" | "playback"; position: number | null }
    console.log("playhead", e.cameraId, e.data);
  }
  if (e.type === "paused") console.log("paused?", e.cameraId, e.data);
  if (e.type === "speed") console.log("speed", e.cameraId, e.data);
});

client.requestSegments(cameraId, qubeId);                 // -> "segments" event

// Jump to a point in the recording (video switches on the SAME element, no re-attach):
client.seek(cameraId, new Date("2026-07-29T09:30:00Z"), qubeId);
client.setSpeed(cameraId, 2, qubeId);                     // 2x fast-forward
client.pause(cameraId, qubeId);
client.play(cameraId, qubeId);
client.goLive(cameraId, qubeId);                          // back to the live edge

5. Alerts (list + thumbnail + clip)

client.on(qubeId, (e) => {
  if (e.type === "alerts") {
    // e.data: { alerts: WebRTCAlert[]; total: number; offset: number }
    for (const a of (e.data as any).alerts) {
      client.requestAlertImage(a.camera_id, a.id, qubeId);  // lazy thumbnail
    }
  }
  if (e.type === "alert-image") {
    // e.data: { id, image }  — `image` is a ready-to-use blob URL
    const { id, image } = e.data as { id: string; image: string };
    (document.getElementById(`thumb-${id}`) as HTMLImageElement).src = image;
  }
  if (e.type === "alert-clip") {
    const { id, url } = e.data as { id: string; url: string };  // blob URL (mp4)
    (document.getElementById(`clip-${id}`) as HTMLVideoElement).src = url;
  }
  if (e.type === "alert-events") {
    // Per-camera timeline metadata: e.data: WebRTCAlertEvent[] (id, timestamp, duration, alertTypes)
  }
});

// Cross-camera, filtered + paginated list (omit `cameras` for all granted cameras):
client.requestAlerts(
  { startMs: Date.now() - 24 * 3600_000, sort: "-timestamp", limit: 50, offset: 0 },
  qubeId,
);

// Fetch a specific alert's clip on demand:
client.requestAlertClip(cameraId, alertId, qubeId);       // -> "alert-clip" event

// Or a single camera's alert markers for a time window (for a timeline overlay):
client.requestAlertEvents(cameraId, Date.now() - 3600_000, Date.now(), qubeId);

Angular usage

The package is framework-free; wrap it in a service. Example:

import { Injectable, NgZone, OnDestroy } from "@angular/core";
import { Subject } from "rxjs";
import {
  createCloudClient,
  WebRTCP2PHandler,
  ConnectionEvent,
} from "@promiseqube/webrtc-core";

@Injectable({ providedIn: "root" })
export class WebrtcService implements OnDestroy {
  private client?: WebRTCP2PHandler;
  readonly events$ = new Subject<ConnectionEvent>();

  constructor(private zone: NgZone) {}

  connect(apiBaseUrl: string, token: () => string, qubeId: string) {
    this.client = createCloudClient({ apiBaseUrl, token });
    this.client.on(qubeId, (e) =>
      // WebRTC callbacks fire outside Angular; re-enter the zone so bindings update.
      this.zone.run(() => this.events$.next(e)),
    );
    return this.client.connectQube(qubeId);
  }

  attach(cameraId: string, el: HTMLVideoElement, qubeId: string) {
    this.client?.attachVideoElement(cameraId, el, qubeId);
  }

  ptz(cameraId: string, dir: "up" | "down" | "left" | "right" | "zoom-in" | "zoom-out", qubeId: string) {
    this.client?.ptz(cameraId, dir, qubeId);
    setTimeout(() => this.client?.ptzStop(cameraId, qubeId), 500);
  }

  ngOnDestroy() {
    this.client?.destroy();
  }
}

React usage

The same package works unchanged in React — the demo in examples/oauth-client uses it directly. Typical shape:

import { useEffect, useRef, useState } from "react";
import { createCloudClient, WebRTCCameraInfo } from "@promiseqube/webrtc-core";

function useQubeStreams(apiBaseUrl: string, token: () => string, qubeId: string) {
  const clientRef = useRef<ReturnType<typeof createCloudClient>>();
  const [cameras, setCameras] = useState<WebRTCCameraInfo[]>([]);

  useEffect(() => {
    const client = createCloudClient({ apiBaseUrl, token });
    clientRef.current = client;
    const off = client.on(qubeId, (e) => {
      if (e.type === "cameras") setCameras(e.data as WebRTCCameraInfo[]);
    });
    void client.connectQube(qubeId);
    return () => { off(); client.destroy(); };   // one connection, closed on unmount
  }, [qubeId]);

  const attach = (cameraId: string, el: HTMLVideoElement) =>
    clientRef.current?.attachVideoElement(cameraId, el, qubeId);

  return { cameras, attach, client: clientRef };
}

Attach each <video ref={el => el && attach(cam.id, el)} autoPlay muted playsInline />.

Authentication

createCloudClient sends the token as the first WS frame. cloud-backend's \/ws/signaling accepts either an Auth0 JWT or a promiseQ system-generated access token (OAuth2 authorization-code + PKCE). A token minted with the qube:read scope already carries view_live, so it can open live streams for qubes the granting user owns or has been granted access to. See examples/oauth-client for an end-to-end OAuth-to-stream demo.

Building

npm run build      # tsc → dist/ (ESM + .d.ts)
npm run type-check