@voicethere/client

Browser and Node client for VoiceThere voice sessions.

Website: voicethere.io

Modes

| Mode | When | Signaling URL | | --------- | ------------------------------------- | --------------------------------------- | | local | Local dev — agent runner on localhost | ws://127.0.0.1:8080/ws | | cloud | Hosted VoiceThere sessions | wss://signaling…/ws?token=<joinToken> |

Wire protocol: @node-webrtc-rust/signaling.

Local (runner direct)

import { connectVoiceSession } from "@voicethere/client";

const client = await connectVoiceSession({
  mode: "local",
  signalingUrl: "ws://127.0.0.1:8080/ws",
  sessionId: "local-dev",
});

client.on("peer-joined", (peerId) => console.log("peer", peerId));

Cloud (hosted VoiceThere)

// Use a **client** API key (prefix vthc_) — safe to embed in web/mobile apps.
// Create one in the dashboard (/api-keys) or: voicethere api-keys create --kind client --project-id <uuid> --name "Web app"
const res = await fetch("https://sessions.voicethere.dev/v1/sessions", {
  method: "POST",
  headers: {
    Authorization: "Bearer vthc_…",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ project_id: "<uuid>" }),
});

const credentials = await res.json();

const client = await connectVoiceSession({
  mode: "cloud",
  credentials: {
    sessionId: credentials.session_id,
    joinToken: credentials.join_token,
    signalingUrl: credentials.signaling_url,
    roomId: credentials.room_id,
    iceServers: credentials.ice_servers,
  },
});

Local stack verify

With the VoiceThere API and session stack running locally:

cd client && npm run demo:cloud

npm publish via GitHub Actions on release/* tags — see scripts/RELEASE.md.

Browser test page

For manual cloud testing with a client API key in the browser:

npm run demo:browser

See examples/browser-test/README.md.

Latency benchmark

examples/latency-bench runs counting-echo turns against your own deployed agent and writes per-turn end-to-end latency in the node-webrtc-rust benchmark format.

Ending a session

| Action | API | Server end_reason | | --------------------- | --------------------------------------------------------------------------------- | --------------------- | | Local teardown | session.disconnect() (sync) or session.disconnectAsync() (await native close) | client_disconnected | | Graceful close signal | session.sendCloseSignal(reason?) | client_close_signal | | Server idle timeout | (automatic) | idle_timeout |

Reconnect and billing

| UI / API | Behavior | | ------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | Dashboard Reconnect or embed Connect after disconnect | Calls startSession() → new orchestrator session id, new Supabase row, new billing period once WebRTC connects | | Unintentional drop (network, signaling close) | Default reconnectPolicy: "same-session" re-joins with the same credentials and peerId (auto-retry with backoff) | | Manual session.reconnect() | Same orchestrator session — re-opens signaling only |

Billing starts when the runner reports a billable WebRTC leg (voice: connected PC + open control channel + agent; data-only: PC + DC). Provision alone does not bill.

Pass reconnectPolicy: "new-session" to disable automatic same-session retry.

Connection drops and reconnect

A short network outage must not end the conversation. After the session was ready once, the client watches the transport and keeps trying to restore it for the reconnect budget, counted from the first loss. The budget follows the project's reconnect window (15 to 30 s, set in the dashboard), which the server sends with the reconnect token; without it the default is 15000. Passing reconnectBudgetMs overrides the server's window.

• Detection. ICE disconnected lasting more than 2 s (the status shows "Reconnecting…"), ICE/peer failed, or the signaling WebSocket closing unexpectedly. A disconnected connection gets rejoinAfterDisconnectedMs (default 5000) to come back on its own before the client rejoins, because a rejoin replaces the old connection on the server; failed and a signaling close with the transport down rejoin at once.
• Recovery. The client closes the old signaling socket itself, opens a new one with the latest session_reconnect_token (or the join token if none arrived yet) and rejoins the same session. The first action is always this same-session reconnect, with no relay detour. Each socket gets 4 s to open and each attempt 6 s to become ready. Attempts start after 1 s, then retry after 2 s each until the budget is spent.
• Giving up. When the budget runs out the session emits WEBRTC_RECONNECT_EXHAUSTED. If the gateway answers a reconnect with HTTP 401 (Node runtime, where the status is readable) the session is gone and the client stops at once with SESSION_ENDED_DURING_RECONNECT.
• Compatibility. maxAutoReconnectAttempts is still accepted and acts as an extra cap on attempts. Without it, a session that was ready is limited only by the budget.

onConnectionStatus receives a recovery field on the status object while this happens:

| recovery.state | Extra fields | Default message | | ---------------- | ----------------------------------------------- | ------------------------------------------------------------------------------- | | interrupted | sinceMs | Connection lost. Reconnecting… | | restored | downtimeMs, via (ice-restart/reconnect) | Reconnected. | | lost | reason (session error code) | The connection could not be restored. Please start a new conversation. | | lost | reason: "SESSION_ENDED_DURING_RECONNECT" | The conversation ended while the connection was down. Please start a new one. |

The default texts are exported as CONNECTION_INTERRUPTED_MESSAGE, CONNECTION_RESTORED_MESSAGE, CONNECTION_LOST_MESSAGE and CONNECTION_SESSION_ENDED_MESSAGE; pass connectionStatusMessages to override them. The embed widget shows them in its status line and re-enables the Connect button on lost.

Every signaling WebSocket failure is logged as one signaling/ws_error debug line: signaling ws failed attempt=<n> elapsed=<ms> phase=<connect|open> code=<closeCode> reason=<closeReason> httpStatus=<status|n/a> wasClean=<bool> url=<redacted>. It appears in createDebugConsole().exportText().

When you end a session with disconnect() or disconnectAsync(), the client first sends { "type": "client_hangup" } (CLIENT_HANGUP_MESSAGE_TYPE) on the control channel so the runner closes right away instead of waiting out its drop grace period. It is never sent on reconnects or after an unintentional loss.

Signaling peerId (voice sessions)

When connecting to a VoiceThere runner or any server using @node-webrtc-rust/helpers VoiceAgentSessionHost:

• Omit peerId — the SDK generates client-<random> (recommended).
• Or pass an explicit id that starts with client- (e.g. client-tab1).
• Do not use bare labels like user-1 or steady-worker-3 — signaling join succeeds but the server ignores the peer and never sends a WebRTC offer.

Same peerId must be reused for reconnectPolicy: "same-session".

Library reference: node-webrtc-rust/docs/signaling-peer-ids.md.

In @voicethere/agent, call disconnectClient(sessionId, { reason }) to kick a peer from agent code (e.g. stale multiplayer state).

Session time limit. When a session reaches its maximum length (the project's max_session_minutes, or a limit the agent set for that session), the server sends one session_close with code max_session_length. The client reports a terminal session error SESSION_MAX_LENGTH and does not reconnect. Its message is the close reason the project or the agent configured, or Session reached its time limit when none was sent.

Configure idle timeouts per project in the dashboard Session settings panel or voicethere projects session-settings set.

Session error events

Pass onSessionError to startSession and connectBrowserSession for a unified handler across provisioning failures, WebRTC errors, and runner session_error data-channel events:

import {
  startSession,
  connectBrowserSession,
} from "@voicethere/client/browser";

const provision = await startSession({
  apiBase: "https://sessions.example/v1",
  projectId,
  headers: { Authorization: `Bearer ${apiKey}` },
  // Optional. Sent when the session starts (so a canary deployment can route on
  // one of its fields) and again after connect. At most 4096 bytes as JSON.
  customerContext: { userId: "u_123" },
  onSessionError: (event) => console.error(event.code, event.message),
});

if (provision.ok) {
  console.log(provision.buildId); // build the session runs on, or null
  await connectBrowserSession({
    mode: "voice",
    credentials: provision.credentials,
    customerContext: { userId: "u_123" },
    onSessionError: (event) => console.error(event.code, event.message),
  });
}

Legacy { type: "agent_error" } payloads are mapped to AGENT_CHILD_CRASHED. See platform docs for the full error code catalog.

End-user identity

Tell the platform who is signed in to your app, so the agent can load that person's account and history.

const provision = await startSession({
  apiBase: "https://sessions.example/v1",
  projectId,
  headers: { Authorization: `Bearer ${apiKey}` },
  // A signed token your backend issues. Use `endUser: { id }` instead for an
  // unverified id. Setting both throws a TypeError before any request.
  endUserToken,
});

const session = await connectBrowserSession({
  mode: "voice",
  credentials: provision.credentials,
  // Needed for session.identify().
  apiBase: "https://sessions.example/v1",
  projectId,
  apiHeaders: { Authorization: `Bearer ${apiKey}` },
  onIdentityConflict: (error) => console.warn(error.code),
});

// The user signs in during the call:
const user = await session.identify({ endUserToken: freshToken });
// -> { id, verified, result: "updated" | "unchanged" }

// The user signs out: closes the session with reason "end_user_logout".
await session.logout();

identify rejects with EndUserRejectedError (token not accepted), EndUserConflictError (the session already belongs to another user), or EndUserAuthUnavailableError (retryable: true; the project's verifier could not be reached). All extend EndUserIdentityError, which carries code and httpStatus.

Root connection error handler

For process-wide logging (signaling socket drops, WebRTC failures, and session errors) without attaching .on('error') on every connection, register once:

import {
  setRootConnectionErrorHandler,
  connectBrowserSession,
} from "@voicethere/client/browser";

setRootConnectionErrorHandler((error) => {
  console.error(error.source.subsystem, error.source, error.message);
});

onSessionError still runs for per-session UI. Details: node-webrtc-rust docs/connection-errors.md.

Agent state and interruptions

Pass onAgentState and onAgentInterrupted to connectBrowserSession (or connectBrowserVoiceSession) to follow what the agent is doing:

await connectBrowserSession({
  mode: "voice",
  credentials: provision.credentials,
  onAgentState: ({ state, detail }) => {
    // state: "listening" | "thinking" | "tool_call" | "speaking"
    statusEl.textContent =
      state === "thinking" ? "thinking…" : (detail ?? state);
  },
  onAgentInterrupted: ({ replyPlayedMs, playedTextExact }) => {
    console.log(
      `caller interrupted after ${replyPlayedMs} ms`,
      playedTextExact,
    );
  },
});

Agents send these events with @voicethere/agent 0.11+ on current runners. Older agents send none, so the callbacks simply never fire. onControlMessage still receives the raw payloads.

Links from the agent

Agents built on @voicethere/agent 0.22+ take links out of what they speak and send each one as an assistant_content message. Pass onAgentContent to show them, for example as a clickable link under the reply:

await connectBrowserSession({
  mode: "voice",
  credentials: provision.credentials,
  onAgentContent: ({ turnId, spoken, content }) => {
    if (content.kind !== "link") return;
    const a = document.createElement("a");
    a.href = content.url;
    a.target = "_blank";
    a.rel = "noopener noreferrer";
    a.textContent = content.label ?? content.url;
    appendToTurn(turnId, a);
  },
});

• Only https: links are reported. Other schemes, malformed URLs and content kinds this client does not know are skipped, and onControlMessage still receives the raw message.
• spoken is true when the link belongs to a voice reply. A voice turn that held only a link may have no spoken text at all.
• parseAgentContentEvent(payload) applies the same checks to a raw control message.
• The embed widget renders each link as its own button bubble (link icon plus label) at the moment it arrives, and opens it in a new tab. It never moves when later replies arrive.

Filler phrases from the agent

Agents built on @voicethere/agent 0.25+ may speak a short filler such as "one moment" while a tool or lookup runs, and send the phrase as an assistant_filler message. Pass onAgentFiller to show it, for example as a status line:

await connectBrowserSession({
  mode: "voice",
  credentials: provision.credentials,
  onAgentFiller: ({ turnId, text }) => showStatus(turnId, text),
});

• The callback fires once per spoken filler with { turnId, text }.
• A filler is not part of the reply. onAgentText never reports it and it is never joined to the reply text for the turn, so a transcript built from onAgentText stays clean.
• Messages without a string turnId and a non-empty string text are ignored, and onControlMessage still receives the raw message.
• parseAgentFillerEvent(payload) applies the same checks to a raw control message.

Spoken text reveal

When the agent speaks, its text arrives as assistant_delta (one per sentence, spoken: true) and assistant_done messages, sent before the audio is heard. Pass onAgentText and the session reveals that text word by word while the agent talks, instead of showing every sentence the moment it is sent.

const session = await connectBrowserSession({
  credentials,
  onAgentText: ({ turnId, text, spoken, final, inReplyTo }) => {
    // `text` is the whole turn so far. Replace the line for `turnId`.
    renderAgentLine(turnId, text);
  },
});

• onAgentText is called once per visible change. text is the full turn so far, keyed by turnId; final is true on the last update of a turn.
• spokenTextReveal is "playback" by default: the reveal follows agent_speaking_start / agent_speaking_end, pauses between spoken chunks, and stops at what was heard on barge_in or when assistant_done has status other than completed. Set spokenTextReveal: "immediate" to get each sentence at once.
• Typed turns (assistant_delta / assistant_done without spoken) are reported at once with spoken: false. The chat_reply that follows them repeats the same text and is not reported.
• onControlMessage still receives every raw message.
• A spoken turn is never shown before its audio: it waits for agent_speaking_start. If a completed reply never gets audio, its full text is shown 5 s after it ends when no speech was queued for it. Once its speech is queued (agent_speak), the text waits for the audio and is shown in full when the server reports agent_speak_failed, or at most 40 s after the speech was queued.
• The embed widget option streamSpokenText has three values: unset (default) paces spoken replies and shows plain chat_reply text at once; true paces spoken replies and chat_reply text; false paces nothing and shows spoken replies at once.

Embed widget customization

Import from @voicethere/client/embed. For production snippets, pass only a client API key (vthc_…) to createVoiceThereWidgetAsync({ clientKey }); the client fetches bootstrap JSON from the CDN (project id, session API base, and optional published appearance). Staging pages can pass apiBase: "https://sessions.voicethere.dev/v1" so the client uses the staging CDN. Legacy integrations may still pass configUrl (stable config.json) or inline projectId / apiBase.

CDN JSON (VoiceThereWidgetConfigV1) and inline options support preset, position (bottom-right, top-left, custom, …), positionOffset, theme (including theme.chat bubble fonts/colors), and customCss. Call widget.updateConfig({ … }) to restyle without remounting during a live session.

Opening the panel hides the launcher; use widget.close() or the header close control to show it again.

Set display: "inline" to embed the chat in your page instead: no launcher, the panel fills mount and is always open. A host that runs its own session endpoint can pass startSession: (signal) => Promise<StartSessionResult> instead of clientKey (programmatic only, never in CDN config).

Programmatic host hooks (never in CDN config; each runs after the widget's own handling, and a throwing hook is logged and ignored): onSession(session | null) (live session after each connect and reconnect, null when gone), onSessionEnd({ reason, error? }) (once per session), onAgentAudioTrack(track), onControlMessage(message) (read-only tap), onStatus({ text, loading, error }), audio: { deviceId?, silent? } (preferred microphone, or a silent track with no microphone request) and runtime (an injectable WebRtcRuntime for tests). isLinkAllowed(url: URL) => boolean limits which agent links are clickable: a rejected link (or a throwing callback) is shown as plain text. onDebugEvent(event) receives every debug event (level, source, name, detail?, raw?, ts) of the widget's session pipeline. While a session is starting the button reads Cancel; clicking it cancels the start (a session that arrives late is closed and never passed to onSession), and Connect works again once the start has settled.

Message times show under each bubble by default. Hide them or change them with theme.chat.timestamps: { show?: boolean; format?: "auto" | "12h" | "24h"; color?: string } ("auto" follows the browser locale). Every bubble has data-voicethere-role, data-voicethere-kind (reply, filler, link, speech, chat, greeting) and, when the message has one, data-voicethere-turn, for tests and custom CSS.

| Class | Role | | -------------------------------------------------------------- | ------------------------------------------------------ | | vt-widget | Root fixed container; CSS variables below are set here | | vt-widget-launcher | Closed-state pill button | | vt-widget-panel | Open chat panel | | vt-widget-header / vt-widget-close | Panel title row and close button | | vt-widget-transcript | Scrollable message list | | vt-widget-msg--incoming / --outgoing / --system | Message bubbles | | vt-widget-msg--filler / --link | Filler and link bubbles | | vt-widget-msg-text / vt-widget-msg-time | Bubble text and its time | | vt-widget-composer / vt-widget-input / vt-widget-connect | Message input and Connect |

CSS variables on .vt-widget include --vt-color-primary, --vt-color-bg, --vt-color-text, --vt-font-ui, --vt-font-incoming, --vt-font-outgoing, --vt-bubble-incoming-bg, --vt-bubble-outgoing-bg, --vt-panel-width, --vt-panel-height, and --vt-panel-radius. Export constants: WIDGET_CSS_CLASSES, WIDGET_CSS_VARIABLES.