@furious.luke/argus-js

Browser client for Argus WebRTC sessions. It establishes the reliable agent text channel and optionally publishes camera, screen, or microphone media, handling gateway selection, signaling, and ICE negotiation for you.

Zero runtime dependencies — it uses only the browser's built-in WebSocket and RTCPeerConnection.

Where this fits

Argus is a distributed video ingestion service. A stream flows through three parties:

• Your server mints a short-lived join token using its secret API key (see the Go client, published as the github.com/furious-luke/argus-go module, or the control-plane HTTP API).
• The browser (this library) uses that token to establish a text-only or media-backed session with the nearest Argus media server. It never sees the API key.
• The browser relays the winning gateway URL and its region-scoped read token back to your server. The URL routes both frame reads and notifications to the selected region; the read token authorizes frame reads only, while notifications use the server-held control token.

This library is only the browser publishing half. It deliberately does not talk to the control plane or mint tokens — that requires your secret API key and must stay server-side.

Voice catalogue discovery and voice selection follow the same boundary. Your server obtains the catalogue and includes its selected voice configuration when creating the stream; the browser receives only its join token and gateway URLs. argus-js therefore exposes the resulting assistant speech track but never a voice configuration or control-plane credential.

┌──────────┐  1. request join token   ┌────────────┐
│  browser │ ───────────────────────▶ │ your server│ ──(API key)──▶ Argus control plane
│          │ ◀─────────────────────── │            │ ◀── token + gateway_urls ──
│ argus-js │  2. token + gateway_urls └────────────┘
│          │
│          │  3. WebRTC text + optional media ────────▶ Argus media server
│          │  4. frameReadToken + selectedGatewayURL ─▶ your server
└──────────┘

Installation

npm install @furious.luke/argus-js

Usage

Your server first creates a stream and returns the join token bundle to the browser (the token and gateway_urls from Argus's POST /api/streams response). Then:

import { Publisher, captureCamera } from "@furious.luke/argus-js";

// `join` is the { token, gateway_urls } bundle your server obtained from Argus.
const publisher = new Publisher({
  gatewayURLs: join.gateway_urls,
  token: join.token,
  callbacks: {
    onConnected: async () => {
      console.log("streaming live");
      // Relay these as a pair; the read token is valid in the selected region.
      await fetch("/api/stream-credentials", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
          stream_id: join.stream_id,
          read_token: publisher.frameReadToken,
          gateway_url: publisher.selectedGatewayURL,
        }),
      });
    },
    onConnectionStateChange: (state) => console.log("state:", state),
    onConnectionQualityChange: ({ level }) => console.log("connection quality:", level),
    onRecoveryStateChange: (event) => console.log("media recovery:", event),
    // Show a button or prompt. From its click handler, call captureScreen()
    // and then publisher.publish(newStream, "screen").
    onRecoveryRequired: (event) => console.warn(`${event.track} must be restarted`, event),
    onError: (err) => console.error("publish error:", err),
  },
});

// captureCamera() applies sensible capture defaults; any MediaStream works too.
// The second argument labels the track type (defaults to "camera").
const stream = await captureCamera();
await publisher.start(stream, "camera");

For a typed agent that needs no initial media or capture permission, start the same WebRTC session with only its ordered text data channel:

await publisher.startTextOnly();
publisher.sendUserText(crypto.randomUUID(), "Hello");

Camera, screen, microphone, and the optional inbound speech track can still be added later through publish, publishMicrophone, and enableSpeech.

Each published track is labelled with a track type ("camera" or "screen"). The label is declared to Argus during signaling, so frame reads and change-notification subscriptions can address a specific track by type. A stream may carry one track of each type at a time.

Capture helpers

captureCamera() and captureScreen() wrap getUserMedia / getDisplayMedia with defaults tuned for streaming — a capped resolution and modest frame rate, audio off. They matter most for screen sharing: on HiDPI/Retina displays a raw getDisplayMedia captures at native resolution (often 3456px+ / effectively 4k), wasting upload bandwidth and downstream decode for no benefit. captureScreen() caps the width instead.

import { captureScreen } from "@furious.luke/argus-js";

const stream = await captureScreen();
await publisher.start(stream, "screen");

Both accept overrides (shallow-merged over the defaults), and any MediaStream you build yourself still works if you'd rather manage constraints directly.

Adding and removing tracks live

Start with one track, then add or remove others without reconnecting. publish adds a track of a given type (renegotiating in place); unpublish removes it. Each supplied stream must contain exactly one video track. Publishing a second track of a type replaces the first.

// Live camera already publishing from start(stream, "camera")...
const screen = await captureScreen();
await publisher.publish(screen, "screen"); // now streaming camera + screen

// Later, stop sharing the screen but keep the camera live.
await publisher.unpublish("screen");

replaceStream(stream, type?) remains available as a convenience — it delegates to publish and defaults to the "camera" type. It is handy from onRecoveryRequired to swap in a freshly reacquired screen share:

const screen = await captureScreen();
await publisher.replaceStream(screen, "screen");

Stopping

publisher.stop(); // stops all tracks and tears down the peer connection

API

Assistant speech and text

Every publisher creates an ordered, reliable argus.text data channel. Enable the persistent outbound speech track only after the user opts in:

const publisher = new Publisher({
  gatewayURLs: join.gateway_urls,
  token: join.token,
  callbacks: {
    onSpeechTrack(track) {
      const audio = new Audio();
      audio.srcObject = new MediaStream([track]);
      void audio.play();
    },
    onAssistantText({ utteranceId, text }) {
      appendCaption(utteranceId, text); // APPEND to the bubble keyed by utteranceId
    },
    onAssistantTextFinished({ utteranceId }) {
      finalizeCaption(utteranceId); // this utterance's caption is complete
    },
    onUserTextResult(result) {
      console.log(result.messageId, result.accepted);
    },
  },
});

await publisher.startTextOnly();
await publisher.enableSpeech(); // call from the application's user action
publisher.sendUserText(crypto.randomUUID(), "Stop and explain that again");

enableSpeech() is idempotent. The remote track is labelled speech and stays attached but silent between utterances. sendUserText is admitted only while the customer server owns a live control-token notify subscription.

One assistant reply arrives as many onAssistantText chunks paced to the audio, all sharing the same utteranceId. Group by that id and append each chunk to the one caption element — creating a new element per callback is what scatters a single reply across the UI. Chunks travel a reliable, ordered data channel, so appending in arrival order reconstructs the text (there is no sequence number). Treat onAssistantTextFinished (same utteranceId, emitted on that same ordered channel after the last chunk) as the completion boundary — not the utterance_finished lifecycle event, which can overtake the last chunk.

new Publisher(options)

| Option | Type | Description | | --- | --- | --- | | gatewayURLs | string[] | Required. The gateway_urls from the join response. All are opened at once; the region whose acknowledgement (accepted) returns first is selected on network path, and only that region places the stream. The rest are held as standbys to fail over to. | | token | string | Required. The short-lived join token from the join response. | | iceServers | RTCIceServer[] | Optional extra ICE servers (e.g. your own STUN). TURN is supplied automatically by the winning gateway. | | iceTransportPolicy | RTCIceTransportPolicy | Passed to the underlying RTCPeerConnection. Defaults to "all". Set "relay" to force media through TURN only (verifies the relay path end to end). | | turnTransportPolicy | "all" \| "udp" \| "tls" | Restricts gateway TURN URLs. Defaults to "all"; use "tls" with relay-only ICE to verify TURN over TLS. Startup fails if the required transport was not advertised. | | gatewayHandshakeTimeoutMs | number | Overall deadline for the whole gateway race — selection, placement, and any failovers. Unaccepted sockets are replaced after 3 seconds so a blackholed TCP flow cannot consume the full deadline. Defaults to 20 seconds. | | gatewayFailoverTimeoutMs | number | How long the selected region has to return ready before the publisher fails over to the next-fastest standby. A socket that closes or errors fails over immediately regardless; this is the backstop for a region that goes silent. Kept generous so a slow-but-good region is not abandoned for one that is merely closer to the control plane. Defaults to 8 seconds, capped at 20. | | peerConnectionTimeoutMs | number | Deadline after the initial offer for WebRTC to reach connected. Defaults to 30 seconds. | | signalingReconnectTimeoutMs | number | How long to retry a dropped signaling socket against the selected regional gateway. Defaults to 20 seconds. | | preferredVideoCodecs | string[] | Preferred video codecs, most-preferred first, as RTP MIME types (e.g. "video/VP9", "video/H264"). Each published video track offers these ahead of the rest, so the browser sends the first one the media server also accepts. Defaults to ["video/VP9"]. Pass [] to leave the browser's native order untouched. Codecs the browser lacks (or setCodecPreferences support, e.g. older Safari) are ignored — negotiation always falls back cleanly. | | connectionStatsIntervalMs | number | How often to poll RTCPeerConnection.getStats() for connection-quality assessment while connected. Defaults to 2000. Set to 0 to disable stats polling entirely (onConnectionStats/onConnectionQualityChange will not fire). | | connectionQualityThresholds | Partial<ConnectionQualityThresholds> | Overrides for the quality classification thresholds. Packet loss is the primary axis; RTT and jitter can only push the level worse, never better. Any omitted field keeps its default (see below). | | connectionQualityDebounceSamples | number | Consecutive worse-than-current samples required before a downgrade is committed, damping transient blips. Improvements are reported on the first better sample. Defaults to 2. | | callbacks | PublisherCallbacks | Optional lifecycle callbacks (see below). |

Methods & properties

| Member | Description | | --- | --- | | start(stream, type?) | Races the gateways, completes the handshake, and sends the SDP offer. The stream must contain exactly one video track; type labels it, defaulting to "camera". Resolves once the offer is sent — use onConnected to know when media is actually flowing. | | startTextOnly() | Starts with only the reliable ordered argus.text data channel and requests no capture permission. Media or speech may be added later. | | startAudioOnly(stream) | Starts with the stream's single microphone track plus the text data channel. | | publish(stream, type) | Adds the stream's single video track under type and renegotiates in place. Replaces any existing track of the same type. | | publishMicrophone(stream) | Adds or replaces the session's microphone track and renegotiates in place (turns on transcription). | | unpublish(type) | Removes the live track(s) of type, stops their capture, and renegotiates so Argus ends that track. | | unpublishMicrophone() | Removes the microphone track and renegotiates (ends transcription). | | enableSpeech() | Opts into the inbound speech track for text-to-speech. Idempotent; call from a user action. | | sendUserText(messageId, text) | Sends typed input over the argus.text data channel (≤4 KiB). | | replaceStream(stream, type?) | Convenience wrapper over publish; defaults to "camera". | | stop() | Stops all local tracks and closes the peer connection. | | frameReadToken | The one-hour read token from the gateway's ready message, or null before connecting. The publisher uses it internally to resume signaling; hand it to your server to fetch frames. | | selectedGatewayURL | The signaling URL that won the regional race, or null before connecting. Relay it so your server uses the same region for frame reads and notifications; frameReadToken authorizes only the frame reads. | | peerConnection | The underlying RTCPeerConnection, or null if not started. | | isConnected | true when the peer connection state is "connected". |

PublisherCallbacks

| Callback | When | | --- | --- | | onConnected() | The peer connection reached "connected" — media is flowing. | | onConnectionStateChange(state) | The RTCPeerConnectionState changed. | | onRecoveryStateChange(event) | Argus detected stalled media and the publisher started, escalated, completed, or failed automatic recovery. | | onRecoveryRequired(event) | Automatic recovery could not restore media, or capture ended and the host must ask the user for a new screen share. | | onSpeechTrack(track, streams) | The inbound speech track arrived after enableSpeech() — attach it to an <audio> element to play text-to-speech. | | onAssistantText({ utteranceId, text }) | One caption chunk arrived; paced with synthesized speech when speech is enabled, immediate in text-only mode. An utterance emits many chunks sharing one utteranceId — append them to the bubble keyed by that id rather than rendering each separately. | | onAssistantTextFinished({ utteranceId }) | The utterance's caption stream is complete. Emitted on the same ordered channel after its last onAssistantText chunk; finalize the visible caption here rather than on utterance_finished. | | onUserTextResult({ messageId, accepted, reason }) | The server accepted or rejected a sendUserText message. | | onConnectionStats(sample) | A periodic ConnectionStatsSample (loss ratio, RTT, jitter, send/available bitrate, encoder limitation) — the raw feed behind onConnectionQualityChange. Fires every connectionStatsIntervalMs while connected. | | onConnectionQualityChange({ level, sample }) | The derived connection-quality level ("good" \| "fair" \| "poor" \| "critical") changed, including the first assessment after connecting. Downgrades are debounced; upgrades fire immediately. React to a degrading uplink here — warn the user, or drop a secondary track. | | onSpeechQualityChange({ degraded, realtimeFactor }) | Server-side text-to-speech generation crossed the realtime boundary: degraded: true when the TTS provider dropped below realtime (realtimeFactor < 1.0), synthesizing slower than it plays and starving playout, degraded: false once it climbs comfortably back above realtime. The degraded transition is raised live during synthesis (a long, slow utterance is reported while it happens, not at its end). Edge-triggered (no flapping at the boundary). A distinct axis from onConnectionQualityChange (that is network health) — react by warning the user or offering a text fallback. | | onError(error) | A fatal error occurred (signaling error, WebRTC connection failure/timeout, or signaling resume timed out). |

How start() works

1. Gateway race. Every URL in gatewayURLs is opened at once with the token in the query string. Selection and placement are two separate steps: the region whose accepted returns first is selected — purely on network path, without waiting on any placement work — and the browser sends proceed to that one only, holding the rest open as standbys. This picks the lowest-latency region without a separate probe, and keeps a region's distance to the control plane out of the choice.
2. Placement, with failover. The selected region does its placement work and returns ready. If its socket closes or errors, the browser fails over to the next-fastest standby immediately; if it just goes silent, it fails over after gatewayFailoverTimeoutMs. A region that answers unavailable (transiently unable to serve) is retried after a backoff rather than counted out. A region that reports the stream is already bound elsewhere sends a placement_redirect, and the browser reconnects to the region that holds it — so a mistimed failover self-heals.
3. TURN + read token. The selected gateway's ready message carries per-session TURN credentials (merged into the ICE configuration) and the read token exposed as frameReadToken.
4. WebRTC. A peer connection and the text data channel are created. Any initial media track is added and labelled, then an offer is sent. Remote ICE candidates that arrive before the SDP answer are buffered and flushed once the answer is applied. Locally gathered candidates remain queued until their signaling write succeeds and are replayed after signaling resume.

After this initial race, the publisher is pinned to the selected regional gateway. If signaling drops, it reconnects to that same gateway with the one-hour read token and waits for resumed; it does not race regions, rebuild the peer connection, or repeat the ready handshake. If the retry deadline expires, the publisher closes the stream and calls onError.

Leave both transport policies at their defaults for normal use. To prove the firewall-friendly TURN TLS path from a VPN or restrictive network, force both relay ICE and TLS TURN:

const publisher = new Publisher({
  gatewayURLs: join.gateway_urls,
  token: join.token,
  iceTransportPolicy: "relay",
  turnTransportPolicy: "tls",
});

This diagnostic configuration fails startup when the winning gateway did not advertise a TLS TURN URL instead of silently testing another path.

Automatic media recovery

Argus measures freshness from complete encoded samples received by the media server, not from pixel changes. An unchanged screen therefore remains healthy. If samples stop for 15 seconds, the media server tells the publisher that the track has stalled and stops serving its retained frame as current. The publisher first detaches and reattaches the live sender and renegotiates. If samples do not resume within four seconds, it performs an ICE restart and waits another eight seconds. These fixed timings deliberately are not application configuration.

onRecoveryStateChange reports the recovery transitions, and the event's track field names the affected track type. If both automatic steps fail, onRecoveryRequired is called and the host can acquire a replacement stream and pass it to publish (or replaceStream) for that track type. A browser cannot silently reacquire a screen share after the user or operating system ends it, so capture_ended always requires host UI and a fresh captureScreen() call.

Connection quality

While connected, the publisher polls the peer connection's WebRTC stats every connectionStatsIntervalMs (default 2s) and derives a coarse quality level so you can react to a degrading uplink without parsing getStats() yourself. Each poll delivers a raw ConnectionStatsSample to onConnectionStats; whenever the derived level changes it delivers a ConnectionQuality to onConnectionQualityChange.

const pub = new Publisher({
  gatewayURLs,
  token,
  callbacks: {
    onConnectionQualityChange: ({ level, sample }) => {
      if (level === "poor" || level === "critical") {
        showBanner(`Weak connection — ${Math.round(sample.lossRatio * 100)}% packet loss`);
      } else {
        hideBanner();
      }
    },
  },
});

Packet loss is the primary signal; round-trip time, jitter, and a bandwidth-limited encoder can only push the level worse, never better. To avoid flapping on a momentary blip, downgrades are debounced — they require connectionQualityDebounceSamples (default 2) consecutive worse samples before committing — while upgrades are reported on the first improved sample so recovery is reflected promptly. The first sample after connecting always emits the baseline level.

The default ConnectionQualityThresholds (all overridable via connectionQualityThresholds):

| Level | Packet loss | RTT | Jitter | | --- | --- | --- | --- | | fair | ≥ 2% | ≥ 300 ms | ≥ 50 ms | | poor | ≥ 5% | ≥ 600 ms | ≥ 150 ms | | critical | ≥ 12% | ≥ 1000 ms | — |

Loss ratio and send bitrate are windowed over each interval; RTT, jitter, and available bitrate are point-in-time. A sample field is null when the browser did not report the underlying stat. This is a detection surface only — the publisher does not itself lower bitrate or resolution in response.

Browser support

Requires a browser with WebRTC (RTCPeerConnection) and WebSocket — all current evergreen browsers. There is no Node.js runtime support; this is a browser-only package.

Development

npm install
npm run build      # bundle ESM + CJS + types into dist/ via tsup
npm test           # run the vitest suite (jsdom)
npm run typecheck  # tsc --noEmit

Publishing

The package is published to npm under the @furious.luke scope. prepublishOnly runs typecheck, tests, and build first, so a release is:

npm version <patch|minor|major>
npm publish

(The scope is configured for public access via publishConfig.)