llm-cassette

VCR for LLM API calls. Record real HTTP responses from LLM providers into human-readable fixture files ("cassettes") and replay them in tests — deterministic, zero-cost, offline CI for AI apps.

• Provider-agnostic by construction. Interception happens at the fetch level, not by wrapping SDKs. Anything that accepts a custom fetch works: @anthropic-ai/sdk, openai, and every OpenAI-compatible service (Azure OpenAI, Together, Groq, OpenRouter, …) with zero provider-specific code.
• Zero runtime dependencies. Node >= 18, TypeScript, ESM + CJS.
• Secrets never land in fixtures. Auth headers are dropped, response headers are allowlisted, and known key formats are scrubbed before writing.

Install

npm install --save-dev @jamal-0x1/llm-cassette
# or
pnpm add -D @jamal-0x1/llm-cassette
# or
yarn add -D @jamal-0x1/llm-cassette

Node >= 18. Ships ESM + CJS with types; zero runtime dependencies. vitest is an optional peer, only needed for the /vitest helper.

Quickstart

Both official SDKs accept a fetch option in the client constructor — hand them the cassette's fetch and you're done:

import Anthropic from "@anthropic-ai/sdk";
import OpenAI from "openai";
import { cassette } from "@jamal-0x1/llm-cassette";

const c = cassette("test/cassettes/summarize.json");

const anthropic = new Anthropic({ fetch: c.fetch });
const openai = new OpenAI({ fetch: c.fetch });

// first run: real API calls, responses recorded to the file
// later runs: served from the file, no network
const msg = await anthropic.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Summarize this article…" }],
});

const completion = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Summarize this article…" }],
});

One cassette can hold interactions from multiple providers — matching is keyed on URL + body, so the Anthropic and OpenAI calls above coexist in the same file.

For code that cannot inject fetch there is a global fallback (prefer the constructor option whenever possible):

import { patchGlobalFetch, unpatchGlobalFetch } from "@jamal-0x1/llm-cassette";

patchGlobalFetch(c);
// … code that calls global fetch …
unpatchGlobalFetch();

Modes

Resolved in priority order: explicit option > LLM_CASSETTE_MODE env var > default auto.

| mode | behavior | |---|---| | auto | replay if a matching interaction exists in the cassette, otherwise hit network and record. Default. | | replay | replay only. No match = throw CassetteMissError with the request summary and a closest-match hint. Never touches the network. CI mode. | | record | always hit network, overwrite/append recordings for matched requests. | | bypass | pass through to real fetch, record nothing. |

replay mode works with no API key set — that's the whole point. (SDK clients still require a key string to construct; any placeholder like "ci-placeholder" works since the request never leaves the cassette.)

const c = cassette("test/cassettes/summarize.json", {
  mode: "replay",              // explicit; or set LLM_CASSETTE_MODE=replay
  match: {
    ignoreBodyPaths: ["metadata.user_id"], // volatile fields dropped before hashing
  },
  allowDuplicates: false,      // see "Duplicate requests" below
});

c.stats();       // { interactions, replayed, recorded, passed }
await c.flush(); // force a write (also auto-flushed after each recording)

Vitest helper

import { useCassette } from "@jamal-0x1/llm-cassette/vitest";

test("summarizes", async () => {
  // derives the path from the test file + test name:
  // test/cassettes/<testfile>/<test-name>.json
  const c = useCassette();
  const client = new Anthropic({ fetch: c.fetch });
  // …
});

Request matching

A request matches a recording when method + URL + body hash to the same key:

• Query parameters are sorted, so param order never matters.
• JSON bodies are compared with sorted keys, so key order never matters.
• Headers never participate in matching. Auth headers vary per environment; that is exactly what breaks naive VCRs.
• Non-JSON bodies are compared as exact text.

Volatile body fields (request IDs, per-user tags) break exact matching — drop them with ignoreBodyPaths:

| provider | recommended ignoreBodyPaths | |---|---| | OpenAI | ["user", "metadata"] | | Anthropic | ["metadata.user_id"] | | any | whatever your app injects per-request (trace IDs, timestamps) |

Duplicate requests

By default, identical requests all replay the same single recording — call the same prompt three times, get the same response three times.

With allowDuplicates: true, recordings for the same request are kept as an ordered list and consumed sequentially in replay: the Nth identical call gets the Nth recording. Exhausting the list throws CassetteMissError.

Cassette files

One human-readable, git-diffable JSON file per cassette (2-space indent, stable key order). JSON bodies are stored parsed, everything else as raw text:

{
  "version": 1,
  "interactions": [
    {
      "key": "a1b2c3d4e5f60718",
      "recordedAt": "2026-07-19T10:00:00Z",
      "request": {
        "method": "POST",
        "url": "https://api.anthropic.com/v1/messages",
        "body": { "model": "claude-sonnet-5", "messages": ["…"] }
      },
      "response": {
        "status": 200,
        "headers": { "content-type": "application/json" },
        "body": { "id": "msg_x", "content": ["…"] }
      }
    }
  ]
}

A corrupt cassette file throws a clear error naming the file — it is never silently overwritten.

Sanitization guarantees

Before any interaction is written to disk:

• Request headers are never persisted at all — authorization, x-api-key, cookie, and friends cannot leak because they are not stored.
• Response headers are reduced to an allowlist (content-type only); request IDs, rate-limit state, and cookies are dropped.
• The recorded URL and both bodies are scanned for common secret formats — sk-ant-…, sk-proj-…, generic sk-…, Bearer …, AWS AKIA…, GitHub ghp_…, and SOME_API_KEY=…-style env dumps — and replaced with <REDACTED>.
• If a secret was redacted inside the request body, a one-line warning is printed to stderr: the match key is derived from the raw body, so replay keys may differ across environments with different keys. Don't put secrets in prompts.

CI recipe

Commit your cassettes, then run tests in replay mode with no keys configured:

# .github/workflows/test.yml
- run: npm test
  env:
    LLM_CASSETTE_MODE: replay

Any request that misses the cassette fails the build with a CassetteMissError describing the request and the closest recorded match — no accidental API spend, no flaky model output.

Re-recording

When prompts or models change, re-record against the real APIs:

LLM_CASSETTE_MODE=record ANTHROPIC_API_KEY=… OPENAI_API_KEY=… npm test

Review the cassette diff like any other fixture change and commit it.

Other providers

Any fetch-injectable SDK works the same way — Azure OpenAI, Together, Groq, OpenRouter, and other OpenAI-compatible services are all just new OpenAI({ baseURL, fetch: c.fetch }). No special support needed; a different base URL is simply a different match key.

Limitations (v0.1)

• No streaming/SSE replay yet — planned for v0.2. Record non-streaming variants of your calls for now.
• Exact matching only (with configurable ignoreBodyPaths). No fuzzy or semantic matching: a changed prompt is a changed request, by design.
• No response templating/mutation, no proxy mode, Node >= 18 only.

License

MIT