ai-sdk-relay

A Durable Object relay for agent sessions: one DO per session, many client sockets (phone, laptop tabs), exactly one runner driving the agent. It supplies the network layer that the in-process harness stack (@ai-sdk/harness, ACP) leaves open.

graph LR
  ClientA["Client (phone)"] <--> DO["AgentSessionDO<br/>sequence authority + replay log"]
  ClientB["Client (laptop)"] <--> DO
  Runner["Runner (exactly one)"] <--> DO
  Host["Tool host (optional)"] <--> DO
  DO --> SSE["SSE readers"]

The base class AgentSessionDO owns the generic core:

• Sequence authority: every fan-out frame gets a monotonic seq; a DO is single-threaded, so this is correct without locks.
• Full-history replay log with delta compaction: sequenced frames are appended to an ai-sdk-session-log on the DO's own SQLite storage, so reconnecting clients read forward from any lastSeq cursor (drift-reload never fires) and the transcript doubles as a durable archive for later analysis or distillation. A lastSeq query parameter on the upgrade URL starts replay before the hello frame arrives. Live text-deltas stream as ephemeral text-chunks and compact into one persisted assistant-message per group. Declare the DO under new_sqlite_classes.
• Hibernating WebSockets: idle sessions cost nothing; role survives eviction as a hibernation tag; a new runner supersedes a stale one.
• Prompt buffering and remote wake: prompts queue while no runner is attached; an overridable wakeRunner can relaunch a scaled-to-zero sandbox, supervised by boot and stop-grace watchdog alarms.
• Permission lifecycle: requests fan out to clients and flip the session to awaiting-input; every request resolves with a sequenced outcome (answered, replayed from a receipt, withdrawn, superseded, or unavailable), so no client is left showing a dead prompt. A decision made once for a tool and input is replayed if the agent asks again in the same turn.
• Fencing and receipts: each runner attach takes a new generation and older generations' frames are dropped; prompts get turn receipts written before dispatch. See DURABILITY.md for the invariants.
• Tool host broker: a local process can attach as the session's tool host (ai-sdk-relay-host) and serve tools to a runner running anywhere, with per-call deadlines and result budgets. Results are journaled, so a repeated call replays instead of running twice.
• Export, import, and fork: a session's total state exports as one archive; import into a fresh session moves it, ?until=N forks it at a point in history, ?seal=1 makes the export a fenced cutover.
• SSE reads: the same log is readable as a server-sent event stream with Last-Event-ID cursors, for consumers without a WebSocket.
• In-process executor: attachInProcessExecutor runs the agent inside the Durable Object over the same frames, instead of across a runner socket. Its callback receives each frame with the generation to echo back through deliverExecutorFrame.
• Per-session ticket auth: clients present short-lived HMAC tickets minted by your control plane via /_internal/mint-ticket; the key never leaves the DO.

Application concerns are protected override points: verifyRunner (the one abstract member), loadInitialStatus, onEvent, onFrame (archive sink for unbounded history beyond the replay window), onStatusChange, onMetric, sessionCreatedAt, wakeRunner, and storeArtifact. The base class never touches a database.

import { AgentSessionDO } from 'ai-sdk-relay';
import type { SessionToClient } from 'ai-sdk-relay-protocol';

export class SessionDO extends AgentSessionDO<Env> {
  protected async verifyRunner(sessionId: string, token: string | null) {
    const machine = await machineOwning(this.env, sessionId);
    return machine !== null && machine.token === token;
  }

  protected override onFrame(frame: SessionToClient & { seq: number }) {
    // archive, index, or notify on every sequenced frame
  }
}

Wire shapes live in ai-sdk-relay-protocol. The reference product is dispatch, where this was extracted from; its relay adds D1 rows, Analytics Engine metrics, semantic search, notifications, and cloud-sandbox wake on top of these hooks.

Tests run on real workerd (@cloudflare/vitest-pool-workers): auth, replay, compaction, prompt buffering, supersede and generation fencing, permission lifecycle and decision replay, turn receipts, seal and fork, SSE, the tool host broker, and the in-process executor. pnpm load runs a load harness against a dev worker and reports attach, fan-out, and replay percentiles; measured numbers are in DURABILITY.md.