Reactor JS SDK

The JavaScript/TypeScript SDK for Reactor, the developer platform for real-time world models.

In a few lines of code you can connect a browser app to a Reactor session over WebRTC, render live model video at 30–60 FPS, send typed commands to steer what generates, and receive structured messages back.

The SDK ships a React API (ReactorProvider, ReactorView, useReactor, …) for browser apps, and an imperative Reactor class for everything else running in a browser context — Electron, a game engine's webview, and so on.

Full reference and guides live at docs.reactor.inc.

Install

npm install @reactor-team/js-sdk

No bundler configuration needed: the WebAssembly binding the SDK runs on ships inside its own JavaScript module, fetched the first time a Reactor connects rather than when the SDK is imported. Each release is checked against Vite 6, 7 and 8 and Next.js 15 and 16 (webpack and Turbopack), in both dev and production builds, except Next.js 15 with Turbopack, checked in dev only since its production build is still beta.

If your page sets a Content Security Policy, its script-src needs 'wasm-unsafe-eval' for the binding to compile.

Quickstart

import { Reactor } from "@reactor-team/js-sdk";

const reactor = new Reactor({
  modelName: "my-model",
  jwt: fetchToken, // your resolver, or a plain string; omit for an unauthenticated local runtime
});

reactor.on("statusChanged", (status) => {
  if (status === "ready") {
    // commands and media are live
  }
});
reactor.on("error", (error) => console.error(error.code, error.message));

await reactor.connect();
await reactor.disconnect();

The token can also be passed directly to connect(jwt, options?) instead of the constructor.

Authentication

fetchToken() above is your own function — mint the JWT server-side so your API key never reaches the browser. The authorization_details block scopes the token: it can create a bounded number of sessions for one model and act only on the sessions it created, so a leaked token exposes nothing else on the account:

// e.g. a Next.js route handler
const result = await fetch("https://api.reactor.inc/tokens", {
  method: "POST",
  headers: {
    "Reactor-API-Key": process.env.REACTOR_API_KEY!,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    authorization_details: [
      {
        type: "session",
        resources: { models: { match: ["my-model"] } },
        constraints: { max_sessions: 5 },
      },
    ],
  }),
});
const { jwt } = await result.json();

See Authentication for the full request shape, including max_session_duration_seconds and other constraints.

Refreshing a token mid-session

A scoped token reaches only the sessions it created or was bound to. When a session outlives its token, a fresh token minted the same way cannot reach it: uploads, reconnects, the session delete and clip reads all return 403. So the SDK tells the resolver what each request is for:

async function fetchToken({ sessionId }: JwtRequestContext = {}): Promise<string> {
  // ask your backend for a token, and pass sessionId along when it is set
}

sessionId is set for every request that targets an existing session and unset for session creation. Keep returning the token that created the session until shortly before it expires. Then mint a replacement bound to that session, which lets it act on the session and create nothing else:

body: JSON.stringify({
  authorization_details: [
    {
      type: "session",
      resources: { models: { match: ["my-model"] }, sessions: { bind: [sessionId] } },
    },
  ],
}),

Reactor only binds a session the API key owns, so the question your backend must answer is whether this browser may hold a token for that session. The simplest answer is to create the session on the backend and hand the browser { jwt, sessionId } to adopt with connect(jwt, { sessionId }): the backend then knows the owner. When the browser creates the session itself, have it register the session id as soon as the SDK first names it to the resolver, which is right after creation, with the token that created it; check that token against GET /sessions/{id} (Reactor answers only for a token bound to it), and remember the session for that browser. Registering only after connect() returns is too late when a slow model keeps the connect open past the token's expiry. The bundled examples do exactly that in examples/shared/token-server.ts.

Events

on(event, handler) / off(event, handler) / once(event, handler). The ones you'll reach for most:

| Event | Payload | | --- | --- | | statusChanged | ReactorStatus | | error | ReactorError | | message | ReactorMessage from the model | | trackReceived | a new media track from the model |

Full list — including schemaReceived, capabilitiesReceived, and statsUpdate — in Events.

Model schema and capabilities

reactor.getSchema();        // the model's command/webhook schema
reactor.getCapabilities();  // negotiated tracks and command set

Both arrive automatically after connect() — no request needed — via the getters above or the matching schemaReceived/capabilitiesReceived events. See getSchema() and Capabilities for the full shape.

Stepped generation

A model that generates one step at a time — rather than streaming continuously — is driven with step(). Create the client with generation: "stepped"; connect() fails with a GenerationModeMismatchError when the runtime does not start the session that way.

import { Reactor, StepRefusedError, stepFrameToImageData } from "@reactor-team/js-sdk";

const reactor = new Reactor({ modelName: "my-stepped-model", generation: "stepped", jwt });
await reactor.connect();

// Input frames never travel over RTP in a stepped session: claim the input
// track, push frames through it, and the next step carries them.
const camera = await reactor.publishTrack("camera");
await camera.pushFrame(canvasCtx.getImageData(0, 0, 640, 360));

try {
  const result = await reactor.step({
    state: { prompt: "a cat" },
    commands: [{ command: "jump", data: { height: 2 } }],
  });
  if (result.error) console.warn(`step ${result.step} failed:`, result.error);
  const frame = result.tracks.find((t) => t.track === "main_video")?.frames[0];
  if (frame) outputCtx.putImageData(stepFrameToImageData(frame), 0, 0);
} catch (error) {
  if (error instanceof StepRefusedError) console.warn("refused:", error.entry, error.message);
  else throw error;
}

// Every part of a step is optional: this one runs on the model's current state.
await reactor.step();

• step({ state, commands }) sends state, commands and every frame pushed since the previous step in one request. The runtime applies them in order — each state key as the model's set_<key> command, then commands, then the frames — and then generates. The runtime checks the whole request before it applies any of it, so a refused step changes nothing; past that check they are separate calls, and a handler that fails does not undo the ones before it. sendCommand() outside a step still works and changes the state the next step uses. A file input goes in the way it does for a command: uploadFile() it, then put the FileRef in a command's data or a state value.
• It resolves with { step, tracks: [{ track, kind, rate, frames }], elapsed, files, error? }, step being the step's number in the session, counted from 1. rate is the rate a track's frames play at as the runtime reported it — frames per second for video, samples per second for audio, 0 when the runtime did not say. elapsed is the seconds the step's generate() took (or the time the model reported for the step); each frame has dtype, shape, typed-array data (Uint8Array RGB (height, width, 3) for video, Int16Array planar (channels, samples) for audio), and optional metadata / captureTimeUs. files lists { name, contentType, size } for each file the model kept with the step; the bytes are served over HTTP from the session's step results (GET /sessions/{id}/steps/{step}), not carried in the result. error is set when the step ran and the model reported a failure — such a step resolves, it does not reject.
• publishTrack(name) resolves with a handle whose pushFrame(frame, { metadata?, captureTimeUs? }) buffers a frame for the next step (reactor.pushFrame(name, frame) does the same, for a track published on this connection). The handle stops working once the track is unpublished or its connection ends; publish again for a new one. It takes a VideoFrame, an ImageData, { data, width, height, format? } raw "rgba"/"rgb"/"bgra" pixels, or { samples, sampleRate, channels? } interleaved Int16Array PCM for an audio track. Pushes keep their order, and step() waits for every push called before it. A refused push rejects, so await it, or .catch() it in a frame loop. At most one message's worth of frames waits for the next step: pushing faster than steps run drops the oldest frames, across tracks. clearStepInputs() drops whatever is waiting.
• The MediaStreamTrack argument of publishTrack() is optional in a stepped session and ignored there, with a console warning: no RTP is sent, frames go through pushFrame(). In a streamed session it is still required, and publishTrack(name) without one rejects with an InvalidStateError.
• Refusals are typed: StepRefusedError (step_refused: nothing the step carried was applied, and entry names the part of the request at fault), StepNotReadyError (step_not_ready: the session cannot take a step yet, and the step did not run), StepCommandFailedError (step_command_failed: a state key or command failed, the calls before it stay applied, and the step did not run; entry names the call and command_code the code its handler raised), StepSessionEndedError (step_session_ended: the session ended before the step's result came back), StepResultTooLargeError (step_result_too_large), UnsupportedFrameEncodingError, and MessageTooLargeError when the request is too large to send. The frames pushed for a step belong to it: when its request cannot be sent, the step rejects and its frames are dropped with it, not kept for the next step.

Recording and clips

const clip = await reactor.requestClip(10); // last 10 seconds
await reactor.downloadClipAsFile(clip, "clip.mp4", { jwt: reactor.getJwtResolver() });

jwt takes a token or a resolver. A resolver is called before every poll of the manifest with { sessionId: clip.sessionId }, so a clip that takes a while to become ready still downloads after the token that created the session has expired. The same goes for getJwt on <ClipPlayer>, <ClipDownloadButton> and useClipDownload.

requestClip()/requestRecording()/downloadClipAsFile() live directly on Reactor, and the same three are bound on the React store. For a preview or download button instead, drop in <ClipPlayer>/<ClipDownloadButton>:

import { ClipPlayer, ClipDownloadButton } from "@reactor-team/js-sdk";

<ClipPlayer clip={clip} getJwt={fetchToken} />
<ClipDownloadButton clip={clip} getJwt={fetchToken} filename="clip.mp4" />

Full API, playback details, and the useClipDownload hook: Recordings.

Stats and timings

reactor.on("statsUpdate", (stats) => console.log(stats.rtt, stats.packetLossRatio));

getStats()/getConnectionTimings() give you the same data on demand. Field list (RTT, bitrate, jitter, handshake timing, ...) in Types.

Errors

One class, ReactorError, for both the error event payload and a rejected call's error — carries code, message, and recoverable. It's the base of a typed hierarchy, one subclass per failure kind (UnauthorizedError, RateLimitedError, NetworkError, ...) — full list in Types. instanceof and matching error.code are equivalent:

import { ReactorError, UnauthorizedError } from "@reactor-team/js-sdk";

try {
  await reactor.connect();
} catch (error) {
  if (error instanceof UnauthorizedError) {
    await refreshToken();
  } else if (error instanceof ReactorError && error.recoverable) {
    await reactor.reconnect();
  }
  throw error;
}

React

import { ReactorProvider, ReactorView, useReactor } from "@reactor-team/js-sdk";

function App() {
  // A module-level fetchToken is stable across renders — see the note on
  // ReactorProvider below for why this matters.
  return (
    <ReactorProvider modelName="my-model" jwtToken={fetchToken}>
      <Status />
    </ReactorProvider>
  );
}

function Status() {
  const { status, sendCommand } = useReactor((s) => ({
    status: s.status,
    sendCommand: s.sendCommand,
  }));

  return (
    <div>
      <ReactorView className="w-full aspect-video" videoObjectFit="cover" />
      {status}
      <button onClick={() => sendCommand("set_image", { url: "..." })}>Send</button>
    </div>
  );
}

react is a peer dependency — install it yourself, matching your app's own version.

ReactorProvider's props are live: changing jwtToken, modelName, or connectOptions tears down the current connection and builds a fresh one. Pass a stable jwtToken (useCallback, or hoist it outside the component) so an unrelated re-render doesn't reconnect you.

useReactor(selector) binds connection state (status, lastError, lastMessage, tracks, ...) and the common actions (connect, sendCommand, publish, requestClip, ...) — enough for most components. For anything else — schema, capabilities, stats, track lookups, the raw event emitter — useReactor((s) => s.internal.reactor) gets you the underlying Reactor instance. Full field list: React hooks.

Typed model SDKs

For models with a published typed SDK, prefer @reactor-models/<name>. It re-exports everything here and adds typed commands, messages, and hooks for one specific model. Use this base SDK when the model doesn't have a typed package yet, or when you want to stay model-agnostic. See Typed model SDKs.

API surface

| Surface | Where it lives | | --- | --- | | Reactor | Reactor class | | <ReactorProvider>, <ReactorView>, <WebcamStream> | React components | | useReactor, useReactorMessage, useReactorInternalMessage, useStats | React hooks | | <ClipPlayer>, <ClipDownloadButton>, useClipDownload | Recordings | | ReactorError and its subclasses, Clip, Capabilities, ConnectionStats, ... | Types |

Development

mise run install:js   # npm install
mise run lint:js      # eslint + tsc --noEmit
mise run build:js     # tsup -> dist/, then copies reactor-wasm's pkg/ into dist/wasm, binary inlined
mise run test:js      # vitest — pure-JS unit tests against a mocked ReactorClient, no wasm build needed
mise run test:js:consumer-tests  # the packed SDK in real Vite/Next.js apps, in a real browser — see consumer-tests/README.md

build:js fails fast with a clear message if crates/reactor-wasm/pkg doesn't exist yet — run mise run build:wasm first.

See the repo-wide CONTRIBUTING.md for the rest (DCO, commit conventions, opening a PR).

Documentation

The full documentation covers platform concepts and the other language SDKs. See CHANGELOG.md for what changed release to release.

License

Apache-2.0 — see LICENSE.