@luxonis/remote-connection

Browser WebRTC client for connecting to Luxonis devices via the signaling server.

Part of the device-connection monorepo.

Installation

npm install @luxonis/remote-connection

Quick Start

import { WebRtcClient } from "@luxonis/remote-connection";

const client = new WebRtcClient({
  clientId: "device-42",
  applicationIdentifier: "my-app",
  authPayload: "auth-token",
});

client.on("connection_established", (connection) => {
  // Reliable channel — default, no framing
  const channel = connection.createDataChannel("comms");
  channel.on("message:string", (data) => {
    console.log("Received:", data);
  });
});

client.on("error", (code, error) => {
  console.error(`Error [${code}]:`, error);
});

Lossy / Framed Channels

The SDK supports explicit per-channel framing using the v1 lossy data channel protocol. Configure local (browser-created) channels with framing: "v1" and remote (device-created) channels via remoteDataChannelOptions.

Local channel

import { LOSSY_MEDIA_CHANNEL } from "@luxonis/remote-connection";

const lossy = connection.createDataChannel("message", LOSSY_MEDIA_CHANNEL);
const outcome = lossy.trySendMessage(new Uint8Array([1, 2, 3]));
if (outcome.status === "dropped") {
  console.debug("Browser admission drop", outcome.reason);
}
lossy.on("message:binary", (data: DataView) => {
  console.log("Received framed payload", data);
});

LOSSY_MEDIA_CHANNEL maps to ordered: false, maxRetransmits: 0, framing: "v1", and the shared 64 KiB native buffered-amount admission default. For a custom profile, pass the options explicitly:

connection.createDataChannel("message", {
  ordered: false,
  maxRetransmits: 0,
  framing: "v1",
  reassembly: { maxMessageBytes: 4 * 1024 * 1024 },
  outbound: {
    maxLogicalMessageBytes: 16 * 1024 * 1024,
    maxLocalQueuedBytes: 32 * 1024 * 1024,
    maxLocalQueuedMessages: 64,
    // Optional per-channel override; the lossy profile defaults to 64 KiB.
    maxBufferedAmountBeforeDrop: configuredNativeBufferLimit,
  },
});

Remote channels

For data channels created by the device, provide a resolver that maps labels to framing/reassembly options. The SDK never guesses framing from payload bytes. Selecting framing: "v1" automatically applies the same 64 KiB local outbound admission default; specify outbound only to override it.

const client = new WebRtcClient({
  clientId: "device-42",
  applicationIdentifier: "my-app",
  authPayload: "auth-token",
  remoteDataChannelOptions: (label, _channel) => {
    if (label === "message") return { framing: "v1" };
    return { framing: "none" }; // system, comms, ping-pong stay reliable
  },
});

See PROTOCOL.md for the v1 wire format and breaking change details.

Reliable vs lossy channels

The SDK supports two channel profiles:

• Reliable (default): ordered: true, no retransmit limit, framing: "none". Use for control, system, and text traffic where every message must arrive.
• Lossy: ordered: false, maxRetransmits: 0, framing: "v1", with a 64 KiB native buffered-amount admission default. Use for low-latency live media where freshness matters more than completeness.

// Reliable control channel
const control = connection.createDataChannel("comms");
control.sendMessage("hello");

// Lossy media channel
const media = connection.createDataChannel("message", LOSSY_MEDIA_CHANNEL);
media.sendMessage(new Uint8Array([...]));

Key semantics

• LOSSY_MEDIA_CHANNEL profile: ordered: false, maxRetransmits: 0, framing: "v1", and a 64 KiB native buffered-amount admission limit. This is the recommended lossy media configuration; override outbound per channel when a different ceiling is required.
• One sendMessage / try_send call = one complete opaque message: the entire serialized payload (e.g. one Foxglove CompressedVideo message) must be passed in a single call. The transport does not assemble multi-call fragments.
• Accepted is not delivery: on the Rust sender, try_send returns Accepted when the complete message passes local admission checks and is queued. This does not promise SCTP delivery or even later native submission. Later native-buffer-full drops are observable through separate events.
• Browser bounded admission: trySendMessage immediately returns a typed accepted/dropped outcome. The v1 queue retains at most 64 complete logical messages and 32 MiB of encoded bytes by default, with a 16 MiB payload safety ceiling. The v1 limit of 1,024 chunks can impose a smaller representable payload maximum for the configured chunk size (16,744,448 bytes at the default 16 KiB wire chunk); such offers are dropped synchronously as message-exceeds-local-budget. Every v1 header is included in accounting; overflow drops the newest offer. Connecting channels use the same bounds, while closing/closed channels throw DataChannelStateError and retain nothing. sendMessage delegates to this bounded path on v1 channels; reliable/unframed sends offered while connecting retain the legacy queue-and-replay-on-open behavior and do not acquire lossy drop semantics.
• Native browser admission: every framing: "v1" channel uses the shared 64 KiB outbound.maxBufferedAmountBeforeDrop default, whether locally or remotely created. Callers only specify this field for a per-channel override. Admission checks live bufferedAmount before queueing and again before the first native chunk; lossy sending never waits for drainage.
• Exact gap diagnostics: reassembly:gap.missingCount is a decimal string. Parse it as bigint when arithmetic is required; do not convert it to number.
• SCTP vs local admission vs native post-admission vs receiver expiry:
  • SCTP reliability comes from the channel's negotiated configuration (maxRetransmits, maxPacketLifeTime, ordered). maxRetransmits: 0 means SCTP will not retransmit lost chunks.
  • Local admission is the sender's pre-queue check: if the local bounded queue or native bufferedAmount snapshot is full, the whole message is dropped before queuing (Dropped(LocalQueueFull) or Dropped(BufferedAmountSnapshotFull)).
  • Native post-admission is the outbound worker's pre-SCTP check: if bufferedAmount + encoded_size exceeds the limit, the queued message is discarded before any chunk enters SCTP (DroppedBeforeNativeSend).
  • Receiver expiry: incomplete reassembly entries expire after 2 seconds by default (receiver monotonic time). This is a memory ceiling, not a playback delay.
• Unordered MaxRetransmits(0) loss: with ordered: false and maxRetransmits: 0, SCTP does not retransmit and does not guarantee order. Lost chunks mean the message is never completed — the bounded reassembler holds an incomplete entry that expires, and newer complete messages are delivered immediately without head-of-line blocking.
• Complete-only / bounded reassembly: the TypeScript receiver emits only completely reassembled payloads. Missing chunks never produce partial application messages. Incomplete state is bounded by TTL (2 s default), count (32 default), and aggregate bytes (32 MiB default).
• Codec / keyframe deferral: the transport is payload- and codec-agnostic. It does not parse Foxglove schemas, H.264/H.265/AV1 NAL units, or identify IDR/CRA frames. Codec-aware queueing, dependent-chain dropping, and keyframe requests are deferred to a future work package.

v1 breaking change and coordinated deployment

Framing v1 replaces the legacy 12-byte fragmentation header with a 32-byte versioned header (LDC1 magic, 64-bit sequence/timestamp, total payload length, consistent chunk count). Old and new framed peers are not interoperable. There is no fallback — both Rust/Python/oak producers and TypeScript consumers must be deployed as a compatible set.

Configuration

WebRtcConfig options:

| Option | Type | Default | Description | |---|---|---|---| | clientId | string | — | Agent (device) ID to connect to | | applicationIdentifier | string | — | Application identifier for auth | | authPayload | string | — | Auth payload for session | | signalingServerUrl | string? | wss://signal.cloud.luxonis.com/session/ | Signaling server WS URL | | iceServers | RTCIceServer[]? | STUN defaults | ICE servers for WebRTC | | turnCredentialsUrl | string? | https://signal.cloud.luxonis.com/api/v1/turn-credentials | HTTP URL for TURN credentials when useLuxonisIceServers is enabled | | turnCredentialsToken | string? | — | Optional bearer token for TURN credentials requests | | signalingConnectionRetries | number \| false | 3 | Retries on signaling connection failure | | remoteDataChannelOptions | RemoteDataChannelOptionsResolver? | none (no framing) | Resolver mapping incoming channel labels to framing/reassembly options | | useLuxonisIceServers | boolean? | false | Fetch TURN credentials from the Luxonis signaling service | | sessionRecovery | SessionRecoveryOptions? | — | Opt-in resolver-driven recovery of stale/closed sessions |

Session recovery

Recovery is supported only while the original signaling clientId remains valid. The SDK does not discover a replacement device identity. If an oak-webrtc restart, upgrade, or re-registration assigns a new ID, recovery is terminal and the user must reopen or reload with newly issued credentials. Static clients retain the existing behavior: a rejected session is terminal.

const client = new WebRtcClient({
  clientId: initial.agentId,
  applicationIdentifier: initial.applicationIdentifier,
  authPayload: initial.authPayload,
  sessionRecovery: {
    maxAttempts: 3,
    initialDelayMs: 1_000,
    maxDelayMs: 10_000,
    resolveSession: async () => ({
      clientId: initial.agentId,
      applicationIdentifier: initial.applicationIdentifier,
      authPayload: getCurrentCachedAuthPayload(),
    }),
  },
});

The resolver must return a non-empty same clientId and current authPayload. Recovery uses exponential backoff (1 s, 2 s, 4 s by default, capped at 10 s). maxAttempts is the maximum number of consecutive attempts in one recovery episode, not a lifetime limit. The episode completes and its budget resets only when the replacement peer reaches connection_established; successful credential resolution alone does not reset it. Each healthy replacement produces a new WebRtcConnection. Dispose old application subscriptions and channel references, then attach them exactly once to the new connection_established value.

Events

WebRtcClient events

| Event | Payload | Description | |---|---|---| | connection_established | WebRtcConnection | P2P WebRTC connection is ready | | signaling_connection_closed | — | Signaling WS closed | | connection_closed | — | P2P connection closed | | session_recovery_started | attempt, reason | A resolver-backed recovery cycle started | | session_recovery_succeeded | attempt, previousClientId, clientId | Fresh metadata was resolved and a new negotiation began | | session_recovery_failed | attempt, Error \| string | A resolver attempt failed (credentials are never emitted) | | error | WebRtcClientErrorCode, Error? | Error occurred; includes session_recovery_exhausted |

WebRtcConnection events

| Event | Payload | Description | |---|---|---| | data_channel | WebRtcDataChannel | Remote peer opened a data channel | | connection_closed | — | Peer connection closed |

WebRtcDataChannel events

| Event | Payload | Description | |---|---|---| | message | MessageEvent | Raw message event | | message:binary | DataView | Binary message | | message:string | string | Text message | | message:json | Record<string, any> | Parsed JSON message | | open | Event | Channel opened | | close | Event | Channel closed | | error | Event | Channel or legacy send error | | send:accepted | { sequence, bytes, chunks } | v1 message admitted locally | | send:dropped | { sequence, bytes, chunks, reason } | v1 message dropped before native submission | | send:submitted | { sequence, bytes, chunks } | all chunks accepted by native send | | send:error | { sequence, bytes, chunks, chunksSent } | conversion/native error; partial submission may have occurred | | reassembly:gap | { sequence, missingCount: string } | exact missing logical-sequence count |

License

Proprietary © Luxonis Corp.