@pouchy_ai/companion-sdk

Embed the Pouchy companion — chat, voice, tools, memory, live world-state, Instant UI, and agent-to-agent messaging — in any app, game, or site.

All six companion capabilities are reachable over the SDK (memory · reasoning · skills · A2A/social · wallet · instant UI), and each rides a platform-neutral contract — so the same companion runs on web, native iOS/Android, or a CLI. See docs/companion-capabilities.md.

Install

npm i @pouchy_ai/companion-sdk

No bundler? Pull it from a CDN via an import map instead — see docs/companion-integration-faq.md.

The package ships ESM-only ("type": "module", no CJS build). Every modern bundler and Node ≥ 18 consumes it as-is; on Node ≥ 20.17 even require() of the ESM entry works natively. Changes per release are tracked in CHANGELOG.md; the pinnable CDN releases (immutable URL + SRI per version) are indexed at pouchy.ai/sdk/versions.

React? @pouchy_ai/react wraps this SDK in a <CompanionProvider> + useCompanion / useMessages / useTyping / useCall hooks so you don't wire the handshake, stream, or teardown yourself.

Svelte / Vue? This package ships thin adapters as subpath exports — @pouchy_ai/companion-sdk/svelte and @pouchy_ai/companion-sdk/vue (0.31.0).

Quickstart

Two steps. 1) Your backend exchanges the project Secret Key (create one in the dashboard → Keys) for a per-user session token — first-seen external_user_ids are provisioned automatically:

POST https://pouchy.ai/v1/sessions
Authorization: Bearer pchy_sk_…
{ "agent": "<agentId>", "external_user_id": "user_4211" }

→ { "session_token": "pchy_…", "expires_in": 3600, "instance": { … } }

2) The client connects with the session token (never ship the Secret Key):

import { createCompanion } from '@pouchy_ai/companion-sdk';

const companion = createCompanion({ baseUrl: 'https://pouchy.ai', token: sessionToken });
await companion.connect();
companion.onMessage((t) => console.log(t));
companion.start();
await companion.sendText('hey');
await companion.connectCall();              // live voice (browser)
companion.sendWorldState({ type: 'game.player.hp', data: { hp: 12 }, retained: true });

(Personal/single-user integrations can pass a server-minted Personal Access Token as token instead — same API. See the docs' authentication section.)

• Guide: docs/companion-quickstart.md
• Spec: docs/companion-sdk-protocol.md
• API reference: docs/companion-api-reference.md
• Capabilities map: docs/companion-capabilities.md
• Instant UI renderer contract: docs/companion-instant-ui.md

Framework adapters (Svelte / Vue)

Thin, logic-free bindings over one shared view controller (createCompanionView, also exported from the root for any other framework). The view tracks { streamState, transcript, draft, typing, pendingConfirms } as immutable snapshots, wraps sendText (optimistic user turn — rolled back if the send rejects, 0.32.1) and confirmAction (confirm-card bookkeeping; a confirm_resolved/confirm_not_found rejection drops the stale card, since a confirm can be resolved elsewhere — hosted page or an IM keyword reply), and can restore recent history with { restore: N }. You own the client lifecycle — create/connect/start it once, hand it to the adapter, close it when done. React hosts: use the standalone @pouchy_ai/react package instead (it owns the client lifecycle and adds a voice-call hook; this view owns the richer render-state).

<script>
  import { companionStore } from '@pouchy_ai/companion-sdk/svelte';
  const companion = companionStore(client, { restore: 20 });
</script>
{#each $companion.transcript as turn}<p>{turn.role}: {turn.text}</p>{/each}
{#if $companion.typing}<p>…</p>{/if}
<button onclick={() => companion.sendText('hey')}>send</button>

// Vue 3
import { useCompanion } from '@pouchy_ai/companion-sdk/vue';
const { snapshot, sendText } = useCompanion(client, { restore: 20 });
// snapshot.value.transcript / .draft / .streamState — auto-disposes with scope

Constructor options (beyond baseUrl + token)

createCompanion({ … }) also takes: surface (one resumable session per surface), modalities / handles / contextKinds / tools / appContext (the capability handshake), visitor (representative mode, below), onAuthError (401 → return a fresh token and the client retries transparently), stream: 'sse' | 'websocket' (receive transport — 'websocket' opts into the lower-latency WS plane when the deployment serves one and falls back to SSE automatically, observable as streamState === 'degraded_sse'), debug (0.32.0 — true logs structured instrumentation events via console.debug; a function receives every CompanionDebugEvent: HTTP request/response with method/path/status/ms, each delivered envelope post-dedup, stream-state transitions, synthesized errors — never the token, headers, or bodies), queueOffline (0.34.0 — opt-in offline send queue, below) with outboxStore (custom persistence for it; default localStorage), requestTimeoutMs (0.35.0 — deadline for any request to produce response HEADERS, default 30 000 ms, 0 disables; expiry rejects code: 'request_timeout'; never bounds reading a streaming body), replayPendingToolCalls (0.37.0, default true — when connect() resumes a session paused on YOUR tools, re-emit each outstanding call to onToolCall with replayed: true; see "Resuming a paused turn" below), and two injection points for Node/tests: fetch (custom fetch implementation) and webSocketImpl (WebSocket constructor when globalThis.WebSocket is absent).

Events

Subscribe with on(type, fn) (or '*'), or these typed convenience helpers. Since 0.30.0 on() narrows the envelope payload per event name (OutboundPayloadMap) — e.g. on('companion.typing', (env) => env.payload.active) type-checks without a cast; '*' keeps the untyped envelope:

| Helper | Event | Use | | --- | --- | --- | | onMessage(fn) | companion.message | the assistant reply (fires exactly once per turn, streaming or not) | | onDelta(fn) | (POST-response SSE) | token streaming — registering makes sendText stream the reply as it is generated: fn(chunk, {reset?}) fires per text chunk (reset = clear the partial — that text was tool-call deliberation); onMessage then fires once with the authoritative final text (replay-deduplicated). Force per call with sendText(text, { stream: true \| false }) | | onToolCall(fn) | companion.tool_call | the companion asks your app to run a declared tool → sendToolResult. The call carries argsJson — args pre-parsed as JSON (undefined on empty/malformed args). replayed: true (0.37.0) marks a re-delivery of a still-outstanding call after a mid-pause reconnect — treat id as an idempotency key for side-effecting tools (see "Resuming a paused turn") | | onRender(fn) | companion.ui_action | Instant UI — draw payload.interface (platform-neutral genui schema) with your own renderer (web / iOS / Android / CLI). Needs ui.render | | onInterfaceUpdate(fn) | companion.ui_update | live {key,value} update to an already-rendered panel (no rebuild) | | onSocialMessage(fn) | companion.social_message | inbound A2A friend message, delivered cross-app. Needs social.message | | onConfirmRequest(fn) | companion.confirm_request | a sensitive-op approval request. Platform session tokens (your end users): show your own confirm card and resolve it with confirmAction — this is how confirm-gated custom skills (POST / credentialed) run. First-party user tokens: observe-only — approval stays first-party (where the stepUp:true biometric gate lives) | | onAudio(fn) | companion.audio | TTS clip (non-call modality). (reserved — not emitted yet) | | onExpression(fn) | companion.expression | avatar viseme / expression / gesture. (reserved — not emitted yet) | | onVoiceInject(fn) | companion.voice_inject | fn({text, speak}) — a voiceRelevant world-state line to say aloud during a live call; route text to your voice session when speak | | onTyping(fn) | companion.typing | activity indicator — fn({active}) fires true when a turn starts working and false when it finishes / pauses, spanning the tool-loop / thinking phase before the first text delta. Drive a "typing…" state | | on('control.call_ready', fn) | control.call_ready | a voice call is ready — the stream echo of startCall's accept. Deliberately secret-free ({ provider, agentId/model, voice, … } — the actual WebRTC credentials only ride the startCall HTTP response); useful for UI state on surfaces that didn't initiate the call | | onUsage(fn) | control.usage | per-token metering echo. (reserved — not emitted yet) | | onError(fn) | control.error | agent / stream errors — agent_error (server-side turn failed after accept; safe to re-send), call_mint_failed (voice-credential mint failed — retry later or fall back to text) or the SDK-synthesized stream_unauthorized (stream 401 exhausted the onAuthError refresh retries, or the stream got a 403 — immediately terminal, commonly a token missing the events.subscribe scope; refresh/fix the token, start() again). Since 0.30.0 the vocabulary is EXPORTED TYPED: CONTROL_ERROR_CODES (runtime list, drift-tested against the server) + the ControlErrorCodeValue union fn's err.code now carries — switch on it with autocomplete |

Instant UI

import { renderInterface } from './instant-ui'; // your renderer (web example in examples/web-instant-ui)
let panel: ReturnType<typeof renderInterface> | null = null;
companion.onRender(({ interface: ui }) => { panel = renderInterface(ui, mountEl); });
companion.onInterfaceUpdate(({ update }) => panel?.applyUpdate(update));

Reference renderers ship in examples/: web-instant-ui (vanilla), native-instant-ui (SwiftUI + Jetpack Compose), and cli-skills (terminal + local/device skills).

Client methods (beyond the quickstart)

| Method | What it does | |---|---| | sendText(text, { awaitReply: true, replyTimeoutMs? }) | Request/response mode: the promise resolves with the completed turn's reply { seq, text, envelope } (SendTextReply) — no onMessage wiring or hand-rolled turn gate. Works without start() when the turn completes in text (the reply rides the same streaming request). A turn that pauses on YOUR declared tools resolves via the event stream, so it DOES need start() — without it the promise rejects immediately with code: 'needs_event_stream'. replyTimeoutMs (default 60s) bounds the whole operation (0.28.0); on expiry it aborts the in-flight stream and rejects with code: 'reply_timeout'. Each turn carries a minted turnId the server echoes as replyTo on the reply (API ≥ 1.1), so the fallback resolves on THE reply — proactive messages can't steal it. | | recall({ query?, limit? }) / remember({ content, … }) | Read / write the token-scoped memory namespace. query runs SEMANTIC search over the facts server-side (embedding-based, ranked fallback); omit it for the plain importance/recency ranking. | | ingestKnowledge({ text, name, … }) / ingestFile(file) | Push documents into the user's knowledge base ("My materials"); needs memory.write:core. | | history({ limit? }) | Fetch the session transcript — restore chat on reload. | | setModalities([...]) | Switch active I/O mid-session (within the token's grant). | | ping() | Keepalive for long-idle embeds. | | setToken(token) | Swap the bearer for every subsequent request + stream reconnect — session tokens expire (1h default), so call this when your backend re-mints one. Prefer the onAuthError constructor option to refresh on demand instead. | | pendingConfirms() / confirmAction(id, approve) | List / resolve pending sensitive-action confirms (platform session tokens). On { status: 'exec_failed', retryable: true } you MAY re-call the same confirmId — idempotent actions only. | | endSession() | Distill the session into durable memory now; returns { skipped?: 'no_session' \| 'no_content' \| 'throttled' }. | | close() | One-call teardown: stop() + endSession() — the text-session mirror of the call handle's close(). | | startCall(opts) / connectCall(opts) | Voice plane: raw credentials / fully-wired call handle (call.close() ends + folds the transcript into memory). | | streamState / onStreamStateChange(fn) | Receive-stream lifecycle (0.30.0): 'idle' \| 'connecting' \| 'connected' \| 'reconnecting' \| 'degraded_sse' \| 'stopped' (CompanionStreamState). fn(state, prev) fires on change only — drive a connection indicator; degraded_sse = the WS transport errored and delivery continues on the SSE fallback; stopped = stop()/close() or a permanent stream_unauthorized failure. | | getAvatar() / brandIconUrl(size?) | The user's avatar (VRM/portrait) and the Pouchy brand icon for your UI. Standalone (no client/token): pouchyBrandIconUrl(baseUrl, size?) derives the same URL before/without connecting. | | getWallet() | Read the instance's own wallet — { balances: [{ currency, amount }], totalUsd, currency }. Read-only + receive-only; needs the wallet.read scope. | | flushOutbox() / pendingOutbox() / onOutboxChange(fn) | Offline send queue (0.34.0, opt-in queueOffline: true). A sendText whose fetch fails at the NETWORK level (or navigator.onLine === false) resolves { seq: null, queued: true, turnId } instead of rejecting; with awaitReply: true it rejects with code: 'queued_offline' (the item is still queued). HTTP errors (4xx/5xx incl. 429) are never queued — the server saw those. The queue replays FIFO — automatically after connect() and whenever the stream reconnects, or manually via flushOutbox() → { sent, remaining } — each item a buffered POST reusing its ORIGINAL turnId, which the server dedupes (completed turnIds, 10 min / last 8 per session), so a retried ambiguous failure can never double-run a turn. pendingOutbox() lists the queued sends (QueuedSend[]); onOutboxChange(fn) fires on every enqueue/flush/drop (drive a "N pending" badge). Persistence: localStorage by default (in-memory page-lifetime fallback), or bring your own via the outboxStore option. |

Tool calling beyond your own declared tools — voice calls only: on connectCall() sessions the SDK automatically offers HOST_CONTROL_TOOLS (universal verbs — invoke_action, set_feature, set_value, navigate, highlight) once you declare tools/handles, and AVATAR_VISUAL_TOOLS (play_gesture / play_expression) let avatar-rendering embeds receive expression cues — declare them or they no-op silently. Text turns expose exactly the tools[] you declared (nothing is auto-added), so declare the verbs there yourself if you want them outside calls. Full contract: docs/companion-host-control.md.

Cancellation (0.29.0, completed in 0.29.1): every request-performing method accepts an optional AbortSignal — including sendToolResult(callId, result, { signal }) and startCall({ signal }) since 0.29.1. Methods with an options object take a signal? field (sendText(text, { signal }), recall({ signal }), …); the rest take a trailing opts (getAvatar({ signal }), ping({ signal }), …). Aborting rejects with CompanionError code: 'aborted', and for sendText({ awaitReply }) it also tears down the in-flight event stream.

const ac = new AbortController();
const p = companion.sendText('summarize this', { awaitReply: true, signal: ac.signal });
// …user navigated away
ac.abort(); // p rejects with code: 'aborted'

Offline send queue (0.34.0, opt-in): queueOffline: true turns a network-level send failure (fetch rejected / navigator.onLine === false) into a queued send instead of a lost one — replayed FIFO on reconnect with its original turnId (server-deduped, so a retry can never double-run a turn; see the methods table above for the full per-item semantics):

const companion = createCompanion({ baseUrl, token, queueOffline: true });
await companion.connect();
companion.onOutboxChange((items) => render(items.length)); // "N pending" badge
const r = await companion.sendText('hey');
if (r.queued) render('queued — sends on reconnect', r.turnId);

Resuming a paused turn (0.37.0): a turn that pauses on YOUR declared tools used to be unrecoverable if the embed lost its state (tab reload / crash mid-pause) — the only unblock was abandoning it via endSession() (/end). Now connect() on a session resumed mid-pause returns the outstanding calls as HelloAck.pendingToolCalls (PendingToolCall[] — { id, name, args, turnId?, pausedAt? }; always present, empty when none), and — unless you pass replayPendingToolCalls: false — re-emits each to your onToolCall handler with replayed: true (one tick after connect() resolves, at most once per client instance), so your existing tool loop completes the turn instead of losing it. Register onToolCall before connect()/start(). Because the app may have already PERFORMED a replayed call and died before posting the result, side-effecting tools should treat id as an idempotency key when replayed is set (the server's result apply is idempotent per id too — a double sendToolResult for the same id is safe); use pausedAt to judge staleness before re-running something expensive:

declare function runTool(name: string, args: unknown): Promise<string>; // yours

const performed = new Set<string>(); // your durable "already ran" ledger
companion.onToolCall(async ({ id, name, argsJson, replayed }) => {
  if (replayed && performed.has(id)) {
    await companion.sendToolResult(id, { ok: true, result: 'already applied' });
    return;
  }
  performed.add(id);
  const result = await runTool(name, argsJson);
  await companion.sendToolResult(id, { ok: true, result });
});
const ack = await companion.connect(); // resumed mid-pause → ack.pendingToolCalls
companion.start();

Scopes (typed)

The full capability-scope vocabulary ships as typed constants — no more copying scope strings from prose:

import {
  COMPANION_SCOPES, SENSITIVE_SCOPES, DEFAULT_SCOPES,
  hasScope, isSensitiveScope, type CompanionScope
} from '@pouchy_ai/companion-sdk';

const ack = await c.connect();
if (!hasScope(ack.grantedScopes, 'skills.execute')) {
  // hide the "run a skill" affordance instead of eating a 403
}

COMPANION_SCOPES lists every scope a key can carry; SENSITIVE_SCOPES are the ones that are never on by default (money / skills / social / core memory / the representative plane); DEFAULT_SCOPES is what a fresh key gets. Also exported: COMPANION_MODALITIES, REPRESENT_SCOPES, DEFAULT_MODALITIES, isRepresentScope(). The list mirrors the server vocabulary and is guarded by the SDK's drift test.

The wire-protocol runtime constants are exported too — PROTOCOL_VERSION, INBOUND_TYPES, OUTBOUND_TYPES, and COMPANION_ERROR_CODES (the append-only HTTP code vocabulary behind CompanionError.code, e.g. missing_scope / session_not_found / turn_pending) — so you can assert the version, validate the event vocabulary, or switch on error codes without hard-coding strings. Every helper populates CompanionError.code when the server names a cause (0.24.0 — previously only the POST-backed calls did). 0.27.0 adds rate_limited for the runtime's 429 cost gates (turn burst ceiling, demo daily budget); on those failures CompanionError.retryAfter carries the server's Retry-After in seconds, so back off with if (e.code === 'rate_limited') await sleep(e.retryAfter * 1000). The SDK also synthesizes a few client-side codes outside that HTTP vocabulary: reply_timeout (awaitReply fallback gave up), stream_unauthorized (event-stream 401/403 — a 401 exhausted the token-refresh retries; a 403 is immediately terminal, commonly a missing events.subscribe scope), and — 0.26.0 — the voice connect-step codes call_unsupported (no WebRTC/mic in this environment), call_connect_failed (mic timeout / SDP exchange failed) and call_dependency_missing (@elevenlabs/client not installed). 0.28.0 adds the confirm/step-up codes (confirm_not_found, confirm_resolved, step_up_required, step_up_failed), the client-side needs_event_stream, and types CompanionError.code as the CompanionErrorCodeValue union (autocomplete in your switch; still open for newer servers). 0.34.0 adds queued_offline (a send parked in the offline outbox for replay) and 0.35.0 adds request_timeout (the requestTimeoutMs headers deadline fired). The authoritative reference is the exported CompanionErrorCodeValue union on CompanionError.code (the internal SDK_SYNTHESIZED_ERROR_CODES list is drift-tested but deliberately not part of the public index surface). (stream_unauthorized above is a control.error/onError code, not a thrown CompanionError.code.) Every outbound payload has a named type; ToolCallPayload ({ id, name, args, replayed? }) types the companion.tool_call event, MessagePayload ({ text, replyTo? }) types companion.message, PendingToolCall (0.37.0) types the HelloAck.pendingToolCalls entries, and onToolCall's callback receives ToolCallEvent — the payload plus argsJson (pre-parsed args).

For advanced voice hosts, the lower-level openCompanionCall(creds, options?, bridge?) primitive is exported too — it consumes the CallCredentials that startCall() mints (bring-your-own call lifecycle; see docs/companion-integration-faq.md).

Representative mode (代聊)

Pass a visitor and the session flips from owner-facing to representative: the companion answers that visitor on the owner's behalf (代聊 / customer-service) using only screened owner context — never the owner's private memory, system prompt, or PII. Works over text and voice.

const c = createCompanion({
  baseUrl: 'https://pouchy.ai',
  token: OWNER_PAT,                 // must hold the `represent` scope
  surface: 'support-widget',
  appContext: { name: 'AcmeShop', description: 'Order support' },
  visitor: { id: stableVisitorId, displayName: 'Sam' }  // stable per end-user
});
const ack = await c.connect();      // → { representative: true, visitorPaired, … }
await c.sendText('do you ship to Canada?');
const call = await c.connectCall(); // same, over voice

Scopes (granted by the owner's "Let it represent you to visitors" key toggle):

| Scope | Adds | | --- | --- | | represent | required — open a visitor-facing session | | expose:knowledge | answer from the owner's knowledge base ("My materials") | | expose:facts | share a wider set of screened facts | | represent:remember | durable per-visitor notes across visits (isolated store) | | represent:pair | c.pairVisitor(visitorPAT) — pair a visitor who's also a Pouchy user → unlock A2A |

Without represent, supplying a visitor is rejected (403). First-party tools (wallet / skills / social) are withheld; your own declared tools still work. Full walkthrough: API reference §3.3–3.4 and the source README.

Voice (optional dependency)

Zero-code alternative: if you just want a chat box, skip the SDK entirely — <iframe src="https://pouchy.ai/embed?token=…&theme=dark&accent=%23ff6b81"> is a hosted drop-in widget with a postMessage control plane (docs: docs/companion-widget.md in the repo / pouchy.ai/sdk → Drop-in widget).

connectCall({ bargeIn: true }) opts into full-duplex voice: the mic stays open while the companion speaks so the user can interrupt mid-utterance (the providers' native interruption takes over). The default is half-duplex — on phone speakers echo cancellation can't fully remove the companion's own voice and it would interrupt itself. Enable it only where AEC holds (headphones, desktop, or devices you've verified).

call.interrupt() (0.36.0) is the silent cut: stop the utterance currently being spoken WITHOUT generating new speech — for real-time commentary going stale at a beat boundary (next hand starts while last hand's remark is still playing). Idempotent; check call.provider for the tier: openai-realtime is guaranteed (response.cancel + output-buffer clear — playback stops within ~an audio frame, the session continues); elevenlabs-convai is best-effort (the EL client exposes no public stop-playback API — we signal user_activity and attempt the client's internal buffer cut; worst case a no-op). For guaranteed preemption on EL, injectEvent(text, true) — which does speak new content — remains the reliable lever.

connectCall() for the ElevenLabs Convai provider needs the optional peer dependency:

npm i @elevenlabs/client

OpenAI Realtime needs no extra dependency (self-contained WebRTC).

Build & publish

The source lives in the app repo at src/lib/companion-sdk and is decoupled from any $lib import, so this package builds with plain tsc:

cd packages/companion-sdk
npm run build      # tsc → dist/ (ESM + .d.ts + source/declaration maps) + src/ (shipped sources)
npm run build:cdn  # cdn/companion-sdk{,.min}.js + versions/<v>/ + integrity.json (SRI)
npm publish        # public scope (publishConfig.access: "public"); runs build via prepublishOnly

The CDN artifacts are committed and served from pouchy.ai: /sdk/companion-sdk{,.min}.js (floating, 1h cache) and /sdk/v<version>/companion-sdk{,.min}.js (pinned, immutable) — pin the versioned URL together with the sha384 hash from cdn/integrity.json (also published at pouchy.ai/sdk) to opt out of silent upgrades.

The wire contract (protocol.ts) is a self-contained mirror of the server's; protocol.drift.test.ts (run by the app's vitest) fails CI if they diverge.

Full release steps (version bump, CHANGELOG.md, drift test, tag) live in PUBLISHING.md.

License

Proprietary — © 2026 Pouchy.ai. The published package may be installed and used to build integrations with Pouchy's official services; all other rights are reserved. See LICENSE.