Spectest SDK

Spectest runs your application and its dependencies in isolated environments. Tests can fork a parent's live state, including memory and files, then drive user flows with recorded assertions and graphical replays.

Install the SDK in your project's spectest/ directory:

mkdir -p spectest
npm install --prefix spectest --save-exact @specific.dev/spectest

Install the Spectest CLI and run spectest docs /installation for project setup and authentication. spectest docs contains the complete offline authoring guide.

For browser tests, read spectest docs /tests/browser; for phone-emulated Expo apps, read spectest docs /tests/mobile. The shared test model is in /tests. Prefer unique, scoped role/name or test-ID locators and awaited browser assertions: await expect(browser.getByRole("status")).toHaveText("Saved"). A one-time read inside expect does not retry. Observe the exact saved state before dependent work; do not replace readiness with sleeps or arbitrary .first() selections. SDK 0.123.0 uses native Playwright assertions while retaining Spectest's API and recording. Ambiguous single-element selectors fail immediately; make them unique.

For terminal readiness, use terminal.waitFor directly. It retries the condition and throws on timeout, so neither an outer ctx.wait nor an extra assertion is needed to check that text appears:

await terminal.waitFor("Rack is initialized", /Current rack status:\s*Initialized/, {
  timeoutMs: 35 * 60_000,
  intervalMs: 5_000,
  failOn: [
    { description: "Setup was rejected", matcher: "Start Rack Setup Failed" },
    { description: "Initialization failed", matcher: "Initialization Failed" },
    { description: "Initialization panicked", matcher: "Initialization Panicked" },
  ],
});
await terminal.press("Ctrl+C"); // Separate cleanup after the condition succeeds.

SDK 0.129.0 records the string/regex matcher (or identifies a custom predicate), timeout, and start/end positions for this wait. The dashboard shows its condition and outcome and replays up to the result. Later inputs belong to their own steps; the full terminal session can be opened separately. ctx.wait remains useful when a retry must repeat actions, rather than just observe terminal state.

Terminal waits record matched, timed-out, failure-condition, predicate-error, process-exit and cancelled outcomes. failOn checks the rendered screen before checking success; string and regex conditions are recorded explicitly. For a custom predicate, provide condition: "Expected sled has a valid bootstrap IPv6 address" to explain the actual check separately from its step title. Async predicates are awaited, and their rejection is recorded. A final matching screen succeeds even if the process has exited; otherwise exit fails the wait after final output is drained. No Promise.race with terminal.exited is needed.

Pass signal: controller.signal to cancel a wait without closing its terminal. Cancellation stops SDK polling; it cannot forcibly stop work started inside a custom predicate. terminal.close({ timeoutMs: 10_000 }) cancels pending waits, sends EOF, escalates to a kill, and bounds cleanup including output drain. The default close deadline is 10 seconds; repeated calls share the same result or failure. terminal.exited records the exit when observed after completion, not when its promise is first read. A stalled output drain fails after 5 seconds instead of hanging indefinitely. Test code should preserve its original failure if diagnostic or cleanup operations also fail.

Browser navigation accepts public Playwright-style options, without a backend cast or pageOp call:

await browser.goto(consoleUrl, { timeout: 120_000, waitUntil: "domcontentloaded" });
await browser.reload({ timeout: 120_000 });

These options also apply to goBack and goForward. Navigation deadlines do not cancel a surrounding ctx.poll, and retrying a navigation after a visible error is a recovery action that should remain visible in the test results.

Pub/Sub integrations are project-owned as of SDK 0.98.0. The former pubsub/pubsubClient exports and /components/gcp entry point are removed. Use a local service definition or serviceGroup() wrapper; the repository’s examples/google-pubsub demonstrates this using public SDK APIs.

k3s registry images are pulled when the environment starts (0.138.2)

The registry strings in k3s({ images }) are now pulled when bootstrap starts, alongside service images and the environment's images, instead of after the cluster is running. Setup loads them into every node from that local copy, so services that depend on the cluster start sooner. Each pull gets up to three attempts; an image that cannot be pulled at the start is pulled by the nodes during setup, also with up to three attempts. References to the cluster's own registry are still pulled once that registry is up.

Image pulls retry network failures (0.138.1)

A service image pull that fails at the network level (DNS, refused or reset connections, TLS handshake timeouts, or an upstream 5xx) is retried twice, after 2 and 5 seconds, before environment setup fails. A missing image, denied access, a registry rate limit or the pull's own timeout still fails at once.

Browser coverage through navigation away from the app (0.137.1)

A click that navigates from a covered app to another site (an OAuth or GitHub installation page, for example) no longer fails coverage capture with "Browser script identity changed". Chromium delivers a coverage take sent during that navigation to the incoming page; the SDK now attributes it to the renderer whose scripts it reports, and uses sources it retained before the old page went away. Genuinely ambiguous takes still report incomplete coverage.

Browser coverage across navigation (0.131.17)

Browser coverage keeps script identities separate when Chromium changes renderers during cross-site navigation, including login redirects. Anonymous browser code cannot inherit an earlier script’s URL or source map. No application configuration change is needed. A script identity change during capture is reported as incomplete coverage instead of publishing uncertain attribution.

Next.js coverage (0.131.16)

Use coverage.next({ appDir: "ops", sourceRoot: "." }) in your Spectest service configuration to collect server and browser coverage. Keep Next's server and browser source maps in the runtime image. appDir is relative to the container working directory; sourceRoot identifies the repository directory represented by the build's source-map root.

Standalone servers launched through Node wrappers can change working directory: the adapter preserves the launcher's generated dependency identity in children. Turbopack's generated image modules, including SVG data-URL wrappers, are excluded automatically wherever the asset lives. Authored asset and application sources are not excluded.

For Next 16.4.3, the adapter also corrects a reproduced Turbopack Edge source-map path defect using Next's resolved build manifest (appDir, distDir, and turbopack.root). This happens while the collector reads a map, before it creates the coverage record. No application build-script workaround is required. The correction is scoped to that verified version and layout; correctly emitted maps and other versions retain normal source-map interpretation. Unknown source paths remain unresolved and cannot authorize subset selection.

Supabase function coverage (0.122.0)

supabase({ dir: "supabase" }) automatically collects coverage for its edge functions, including local shared modules outside that directory. No application instrumentation or coverage option is needed. Custom component names and folder layouts work the same way. Stacks without functions have no extra collector.

Coverage starts before user code in both shared and per-function isolates. Setup and individual test captures are separate, with source maps checked against the uploaded TypeScript/JavaScript. Generated startup gates let functions warm concurrently while preserving their Deno configuration and import maps. Isolate captures, source metadata reads, and report writes run concurrently. Lost workers or unusable source metadata fail collection rather than silently reporting no execution. Changes to the function catalogue or runtime configuration conservatively invalidate subset selection, including newly added functions. This collects edge function code only: SQL and application frontend coverage are separate, and subset enforcement remains a project setting.

Interactive entrypoints (0.114.0)

Choose the links shown in the interactive dashboard and PR preview comment:

const env = defineEnvironment({
  name: "my-app",
  services: { web, api },
  interactive: {
    instructions: "Sign in as [email protected] and use 000000 as the code",
    entrypoints: [
      { name: "App", url: "https://app.test" },
      { name: "Admin", url: "https://app.test/admin" },
      { name: "API shell", terminal: "api" },
    ],
  },
});

List order is preserved. URLs must match declared service hostnames and are translated to public preview URLs, retaining paths, queries and fragments. Terminal targets are typed service names and use dashboard member permissions. Entrypoints are display metadata; they do not change exposure or test execution. With no entries, the only preview link is Open environment. Optional interactive.instructions is a concise, direct plain-text sign-in description, shown in a collapsed Instructions section in the dashboard and PR comment. Blank instructions are omitted.

Live APIs in interactive environments

Add live to a scripted fake to forward its entire HTTP and WebSocket surface in interactive environments, without recording requests or responses:

const soniox = defineFake({
  name: "soniox",
  hostnames: ["api.eu.soniox.com", "stt-rt.eu.soniox.com"],
  state: createState,
  helpers: createHelpers,
  handler: handleHttp,
  websocket: handleWebSocket,
  live: {
    inject: [{
      headers: { authorization: "Bearer {{secret:SONIOX_API_KEY}}" },
    }],
  },
});

Tests and shared baseline setup always use the scripted handlers, even when credentials are available. Interactive forwarding starts after the shared baseline and before interactive seeding. Every declared hostname forwards to that same real host over HTTPS/WSS on port 443. Scripted handlers are bypassed; helpers remain available but do not observe or control live provider traffic.

live: {} forwards without credential overrides. live.inject uses the same first-matching header rules as replayFake for HTTP requests and WebSocket handshakes; matches select credentials, not which traffic is forwarded. live.sign supports the same AWS SigV4 signing for HTTP (not WebSockets). Secret references are resolved from the project's Secrets store. Missing credentials and upstream failures surface as errors without scripted fallback.

For Soniox, use header authentication and omit api_key from the initial WebSocket message. The application can use a dummy key; Spectest overwrites its outbound Authorization header with the real one. Provider callbacks and provider-fetched audio URLs must separately be reachable in the environment.

Unlike replayFake, live fakes have no cassette state or recording helpers. HTTP payloads may be buffered during forwarding and WebSocket opening buffers are bounded; neither is retained as a recording. Application-level storage and logs remain under application control.

Microphone fixtures and WebSocket fakes (0.103.0)

Supply a 16-bit PCM mono/stereo WAV when creating a browser session:

const browser = await ctx.browser({
  microphone: { path: "spectest/tests/fixtures/speech.wav" },
});

The microphone uses Chromium's native capture pipeline. It plays once and then supplies silence; add loop: true to repeat. Paths are relative to the repository root. Set the fixture on the first call for that browser session; reattaching preserves its source. Fixture sessions use a dedicated Chromium process to keep named browsers' audio isolated.

A fake can serve HTTP and WebSockets on the same hostname, sharing state and helpers:

const speech = defineFake({
  name: "speech",
  hostnames: ["speech.example.com"],
  state: () => ({ messages: [] as string[] }),
  handler: () => Response.json({ ready: true }),
  websocket(req, state) {
    if (new URL(req.url).pathname !== "/realtime") {
      return new Response("Not found", { status: 404 });
    }
    return {
      message(socket, message) {
        if (typeof message === "string") state.messages.push(message);
        socket.send(message);
      },
    };
  },
  helpers: ({ state }) => ({ messages: () => [...state.messages] }),
});

The handshake returns a rejection Response or callbacks (open, message, close), with an optional selected protocol. Callbacks may be async and run in order per connection. WS and trusted WSS support text/binary messages; observe traffic through the fake's helpers.

Chat reactions (0.105.0)

A chat annotation's message can carry the tapbacks a messaging app puts on a bubble. Each reaction is { emoji, from?, new? }: from is self (the end user) or other (the default), and new marks one that arrived in this step, exactly like a message's new. A removed reaction simply stops being listed.

return annotate(turns, "chat", {
  messages: turns.map((t) => ({
    side: t.fromCustomer ? "self" : "other",
    text: t.body,
    reactions: t.acked ? [{ emoji: "✅", new: t.ackedNow }] : [],
  })),
});

The dashboard draws the chips on the bubble's lower corner, opposite the sender, and slides a new one in after the bubbles that arrived with it.

Read receipts and typing (0.106.0)

A chat message can carry its delivery status (delivered or read) and an optional statusAt — free text such as "23:14" — drawn as the caption a messaging app shows under the bubble, on the bubble's own side. The annotation itself takes typing: "self" | "other", drawn as the three-dot bubble at the end of the thread on that side. Both are the fake's to set, like new; typing is transient, so a step either shows it or not.

return annotate(turns, "chat", {
  typing: state.agentTyping ? "other" : undefined,
  messages: turns.map((t) => ({
    side: t.fromCustomer ? "self" : "other",
    text: t.body,
    status: t.readAt ? "read" : "delivered",
    statusAt: t.readAt,
  })),
});

Inline replies (0.107.0)

A chat message can carry an id — any non-empty string the fake chooses, the provider's message handle usually; it is never drawn — and a reply names an earlier message's id as replyTo. The dashboard draws the reply the way a messaging app threads one: an excerpt of the earlier message in a muted mini-bubble directly above the reply, on the reply's side, with a short connector between them. A reply to the bubble right above it draws only the connector, since the message it answers is already on screen. A run of consecutive replies to one message quotes it once, above the first of them, and the rest draw only the connector so they read as one thread; a reply to a different message, or an ordinary message, ends the run. A replyTo that names no earlier message of the same call draws nothing extra; the bubble is an ordinary message.

return annotate(turns, "chat", {
  messages: turns.map((t) => ({
    id: t.id,
    side: t.fromCustomer ? "self" : "other",
    text: t.body,
    replyTo: t.inReplyTo,
  })),
});

HTTP assertions

ctx.fetch() returns a wrapped response. Status fields, header lookups, and all body readers retain the HTTP call's provenance, so assertions appear under that call on the timeline:

const response = await ctx.fetch("http://app:3000/image.png");
expect(response.status).toBe(200);
expect(response.headers.get("content-type")).toBe("image/png");
const bytes = await response.bytes();
expect(bytes).toHaveLength(70);
expect(bytes[0]).toBe(0x89);
expect(bytes.slice(0, 8)).toEqual(new Uint8Array([0x89, 0x50, 0x4e, 0x47, 13, 10, 26, 10]));

arrayBuffer() wraps buffer metadata and slices. blob() wraps metadata and its body readers. formData() wraps get(), getAll(), and has(), including uploaded files and their metadata. Headers also support wrapped getSetCookie() and Bun's getAll(), toJSON(), and count. Missing fields remain null, with provenance recovered by an immediately following expect.

Use .unwrap() to pass a wrapped value to a native API, such as new TextDecoder().decode(bytes.unwrap()). Use .transform("text", bytes => new TextDecoder().decode(bytes)) when the derived value should retain provenance. Iteration and callback arguments remain raw, as do array/typed-array length and stream control flags (done, locked). Use expect(bytes).toHaveLength(n) for a linked length assertion. bodyUsed is wrapped: unwrap it for control flow.

For an open-ended stream, pass { captureBody: false } to return without waiting for recorder body capture. SSE (text/event-stream) skips capture automatically. The HTTP event and assertions are still recorded; the event omits the body. response.body.getReader().read() returns a raw done flag and wrapped value. BYOB readers, tee(), and pipeThrough() preserve provenance too. Release reader locks and cancel streams when finished, as with native fetch.

Electron apps

Use electron() to build and run a real Linux Electron app under Xvfb, with its main process, preload, IPC and renderer. Spectest supplies the container setup; no custom Dockerfile is needed.

Create spectest/index.ts:

import { defineEnvironment } from "@specific.dev/spectest";
import { electron } from "@specific.dev/spectest/components";

export const env = defineEnvironment({
  name: "desktop-app",
  services: { app: electron() },
});

export default env.project();

Create spectest/tests/notes.ts, using your app's labels:

import { expect } from "@specific.dev/spectest";
import { env } from "../index";

env.test("Save a note", async (ctx) => {
  const app = await ctx.desktop(ctx.svc.app);
  await app.getByRole("textbox", { name: "New note" }).fill("Hello desktop");
  await app.getByRole("button", { name: "Add note", exact: true }).click();
  await expect(app.getByRole("listitem")).toHaveText(["Hello desktop"]);
});

Run spectest test . from the project root. The dashboard records desktop interactions and replays the app with window chrome derived from its Electron configuration. Child tests using dependsOn inherit the live app state, including unsaved input, in their own isolated environment.

The app must declare Electron in package.json (its own or, with context set to a monorepo root, the root one). Defaults are npm ci with a package lock (otherwise npm install), npm run build, and the package's main entry. Use appDir, installCommand, buildCommand, entry, and nodeVersion to customize the build. env supplies runtime variables; dependsOn waits for backend services in the same environment. Expose the backend through the built-in proxy (tls: [{ hostname, port }]) and give the app its https:// URL: a renderer Content Security Policy usually refuses a plain http:// service address.

Applications need Linux-compatible dependencies and binaries. Native OS dialogs and macOS-specific APIs are outside this basic support. Replays capture the renderer DOM from acquisition onward, not native OS surfaces.

Run spectest docs /components/electron for the full configuration reference, backend setup, session behavior, and replay scope (CLI 0.2.124 or later).

k3s({ registry: { host: "registry.test" }, images: [...] }) exposes the component's writable registry over verified HTTPS through the VM proxy. Dockerfile builds in images targeting registry.test/... are published and preloaded from the same cached build, preserving digests without an export/push step. Other images keep direct preloading. Registry writes stay isolated across test forks; client and containerd CA trust are configured automatically.

HTTP/2 and gRPC ingress

For a service listening on plaintext HTTP/2 (h2c), declare tls: [{ hostname: "grpc.example.test", port: 50051, http2: true }]. Clients use trusted TLS on port 443. The ingress selects HTTP/2 with ALPN, routes each stream by :authority, and preserves streaming bodies, metadata, trailers, and cancellation. HTTP/1.1 requests still use the existing HTTP proxy. Runtime services support the same option. HTTP/2 streams currently bypass ctx.intercept and HTTP request recording.

CORS at ingress

Fake and service ingress default to cors: "passthrough": OPTIONS requests reach the handler or upstream through ctx.intercept, and response CORS headers are preserved, including their absence on rejections or responses with no CORS policy. Allowed credentialed requests depend on the application's own headers.

Automatic CORS is opt-in per fake or service hostname:

defineFake({ name: "api", hostnames: ["api.test"], cors: "permissive", handler });
// On a boot or runtime service:
tls: [{ hostname: "api.test", port: 3000, cors: "permissive" }]
// On a Supabase gateway (including responses from interceptors):
supabase({ hostname: "supabase.example.com", cors: "permissive" });
// On a low-level component proxy:
proxy("api.test", { service: "api", port: 3000, cors: "permissive" });

Permissive HTTP/1 ingress answers CORS preflights before interceptors and adds reflected allow-origin plus allow-credentials to responses without allow-origin, including errors. Ordinary OPTIONS still reaches the handler. Use passthrough when testing application CORS policies. HTTP/2 streams always pass CORS through.

Prefer the application's production hostnames for the frontend and API inside the environment: Spectest resolves declared names to its test services and provides certificates trusted inside the environment. This preserves existing origin allowlists and cookie relationships. Different subdomains are still different origins; a synthetic response must still supply the intended CORS headers. Permissive mode is a convenience override and should not be used to verify application CORS restrictions.

Annotated service helpers (0.108.0)

Service helpers support the same return annotate(value, "email" | "chat", options?) contract as fake helpers. The SDK opens the annotation box, records one named service call, and returns a wrapped value whose assertions link to that call. The annotated call's internal operations are suppressed; concurrent work is unaffected. Ordinary helper returns retain their values and underlying events.

helpers: ({ fetch }) => ({
  async latest() {
    const response = await fetch("http://mail:8080/messages/latest");
    const mail = (await response.json<{ subject: string; text: string }>()).unwrap();
    return annotate(mail, "email");
  },
})

The classic viewer shows mail.latest() with Email/JSON tabs; JSON contains helper arguments and the returned value. The SPA retains its nested presentation. annotate always returns a box for the helper boundary; it does not attach views to arbitrary recorded results. See spectest docs /services for a complete example.

SIP phone calls

const phone = await ctx.phone({
  endpoint: "sip:5060",
  from: "sip:[email protected]",
  language: "sv",
});
await phone.call("receptionist");
await phone.say("Hej! Jag vill boka en tid.");
await phone.hangup();

The phone offers all installed codecs, preferring Opus then higher sample rates, with G.711 and other legacy codecs available as fallbacks. Recordings use 48 kHz mono PCM for capture and continuously encode it as 48 kbps VBR Opus/WebM. Encoding runs off the SIP callback thread and writes no intermediate WAV; the original audio timeline and waveform data are preserved.

Supertonic 3 runs locally using models installed in the golden image. Each call, say, and hangup is recorded. One continuous recording captures both sides from dialing through hangup, including replies and pauses between steps. Each say clip starts when speech is transmitted, skipping generation time, and ends when that step completes. Both call(destination, { waitFor }) and say(text, { waitFor }) poll application state, then wait for incoming silence. For call, polling starts after the call connects, allowing the greeting to finish within the call step. For say, polling starts after outgoing playback. Without waitFor, call ends when connected and say ends after outgoing playback. The wait has no timeout of its own: the test case's timeoutMs is the only ceiling, and a bound on the reply belongs inside the predicate (for example a ctx.poll with its own timeoutMs), where it appears in the timeline. The quiet period (700 ms) and poll interval (100 ms) are internal defaults for now. waitFor may return a nonempty transcript string instead of true; after the silence wait, it appears as Response below Caller in an indented chat row beneath the waveform. Return false or "" to keep waiting. Your test supplies the text from application state or a fake; an async callback can wait and then return it. Playback starts muted and remembers your mute choice across steps, with overlapping blue caller and violet recipient waveforms and replay controls. Both sides play together from the same recording. Step playback trims silence at the edges of utterances and waited greetings; greetings also show the approximate time to first audio. Hangup steps have no player. The complete recording remains stored. Outgoing text remains a plain transcript; incoming audio is not transcribed automatically. Calls close and recordings finalize automatically at case end. This initial API requires SDK 0.112.0 and a golden image with phone support.