npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@duskel/cassette

v0.1.0

Published

Deterministic record/replay for LLM and tool calls, recorded at the SDK boundary instead of the socket.

Readme

cassette

Deterministic record/replay for LLM and tool calls. Record your agent once against a live model, then replay it forever: the same prompts return the same completions, the same tool calls run against recorded results, and a readable diff tells you the moment a prompt or a tool signature drifts away from its recording.

Agent tests become fast, free and reproducible. No API key in CI, no flakes, no bill.

Model calls served from the cassette    3
Tools executed for real                2
Completed in 2.7ms

The problem

Testing an agent means running a loop that calls a model, executes tools, feeds the results back, and calls the model again. Every run of that loop is slow, costs money and returns something slightly different. So agent test suites end up in one of two bad places: nobody runs them, or everybody mocks the model so heavily that the test no longer resembles the thing being shipped.

Why HTTP mocking is the wrong layer

The obvious fix is the one HTTP fixtures libraries have used for a decade: intercept the socket, replay the response body. vcrpy and friends do this well, for the problem they were designed for.

It is structurally wrong for agents.

When you mock at the HTTP layer, you are replacing the transport. But an agent's interesting behaviour is not in the transport, it is in the loop around it:

  • The model returns a tool_use block. Your code has to parse it, dispatch it, run the tool, and format the result back into the next request. That is where agents break.
  • Mock the HTTP response and your tool never actually runs. The dispatch path, the argument coercion, the error handling in the tool itself: all untested.
  • SSE framing, retries and base-URL changes are transport details. Recording them couples your fixtures to the SDK's internals, so an SDK upgrade invalidates cassettes that describe conversations that did not change.

cassette records at the SDK boundary, not the socket. It patches client.messages.create, not fetch. The consequence is the important part:

Under replay, your agent loop, your tool dispatch and your tools all still execute for real. Only the model's answers come off the tape.

If convertTemperature has an off-by-one, the replayed test fails, exactly as production would. An HTTP-level mock cannot tell you that, because it never called the function.

You can still record the tool boundary when you want to, for tools that are slow, paid or non-deterministic. That is what the tools adapter is for, and it is opt-in per tool.

Install

npm install --save-dev @duskel/cassette

Node 20.11+, ESM, TypeScript types included, zero runtime dependencies. The Anthropic and OpenAI SDKs are optional peer dependencies; you only need the one you actually use.

Quickstart

import Anthropic from '@anthropic-ai/sdk';
import { withCassette, anthropicAdapter } from '@duskel/cassette';
import { expect, test } from 'vitest';

test('the agent converts the temperature it looked up', async () => {
  const client = new Anthropic();

  const run = await withCassette('weather-agent', async (tape) => {
    tape.use(anthropicAdapter, client);
    return runAgent(client, 'What is the weather in Paris, in fahrenheit?');
  });

  expect(run.answer).toContain('69.8');
  expect(run.toolCalls.map((c) => c.name)).toEqual(['get_weather', 'convert_temperature']);
});

The default mode is auto: the first run has no cassette, so it records against the live API and writes cassettes/weather-agent.cassette.json. Commit that file. Every run after that replays it, offline, in milliseconds.

Run the example

The repository ships a working agent with two tools and a committed cassette. It uses the real @anthropic-ai/sdk and needs no API key:

git clone <this repo> && cd cassette
npm install
npm run example

Real output:

Replaying from the committed cassette. No network, no API key.

Question   What is the weather in Paris right now, in fahrenheit?
Answer     It is currently 69.8 degrees fahrenheit and clear in Paris.

Model calls served from the cassette    3
Tools executed for real                2
  get_weather({"city":"Paris"}) -> {"city":"Paris","temperature":21,"unit":"celsius","conditions":"clear"}
  convert_temperature({"from":"celsius","to":"fahrenheit","value":21}) -> {"value":69.8,"unit":"fahrenheit"}

Completed in 2.7ms

Three model calls served from disk. Both tools executed for real, in the replayed run. That is the difference from HTTP mocking, in one command.

Drift detection

When a request stops matching its recording, you get a diff, not a shrug. Change the example's question and re-run:

cassette: no recording matched this request.

  cassette   weather-agent
  adapter    anthropic
  operation  messages.create
  model      claude-sonnet-4-5
  recorded   3 interactions

Closest recording is #1 (98% payload match, 1 field differs):

  ~ payload.messages[0].content
      recorded: "What is the weather in Paris right now, in fahrenheit?"
      current:  "What is the weather in Lisbon right now, in fahrenheit?"

Re-record with CASSETTE_MODE=record, or relax matching with
`ignorePaths` / `fuzzy` if the difference is not meaningful.

This is the feature that makes recorded fixtures survive contact with a real codebase. Edit a system prompt six months from now and the failing test tells you which prompt, on which line, changed how.

Modes

| Mode | Behaviour | | ------------- | ---------------------------------------------------------------- | | auto | Replay if a cassette exists, otherwise record it. The default. | | record | Always call through and rewrite the cassette from scratch. | | replay | Never call through. An unmatched request throws. | | passthrough | Call through and record nothing. Useful for a one-off live check. |

Set per call site, or globally with CASSETTE_MODE:

CASSETTE_MODE=record npm test     # refresh every cassette
CASSETTE_MODE=replay npm test     # what CI should run: fails rather than dialling out

replay is the mode you want in CI. A miss throws CassetteMissError instead of falling through to the network, so a stale fixture can never turn into a surprise API bill.

Matching

By default a request matches a recording when the adapter, operation, model, tool name and full payload all agree. Loosen it when something legitimately varies:

await withCassette('agent', run, {
  // Drop volatile fields before comparing.
  ignorePaths: ['metadata.request_id', 'messages[*].timestamp'],

  // Normalize non-deterministic values on both sides.
  fuzzy: { timestamps: true, ids: true, whitespace: true, numberPrecision: 2 },
});

Or replace the matcher stack entirely:

import { matchers } from '@duskel/cassette';

await withCassette('agent', run, {
  matchers: [
    matchers.adapter,
    matchers.operation,
    matchers.messages, // conversation only; temperature and max_tokens may vary
    matchers.custom((request, recorded) => request.payload.tenant === recorded.payload.tenant),
  ],
});

Built-in matchers: adapter, operation, model, toolName, payload, messages, toolArguments, field(path), payloadIgnoring(paths) and custom(fn).

Redaction

Redaction is on by default and runs before a cassette touches disk. A leaked key in a committed fixture is the failure mode that would make this library not worth using, so the built-in rules err towards over-redacting: Anthropic, OpenAI, Stripe, GitHub, Slack, Google and AWS credentials, JWTs, bearer tokens, private key blocks, e-mail addresses, and the value of any field named like a secret (authorization, api_key, cookie, password, and others).

await withCassette('agent', run, {
  redact: {
    patterns: [/CUST-\d{6}/, 'acme-internal-tenant'],
    keys: ['x-internal-trace'],
    transform: (value) => value.replaceAll(realCustomerName, 'CUSTOMER'),
  },
});

Redaction also runs on the incoming request during replay, so a redacted recording still matches a live request carrying the real credential.

CLI

npx cassette list
npx cassette inspect weather-agent
npx cassette prune --delete
npx cassette verify
CASSETTE       CALLS  ADAPTERS   SIZE     RECORDED
-------------  -----  ---------  -------  ----------
weather-agent  3      anthropic  11.2 KB  2026-07-24

1 cassette in /path/to/examples/cassettes
  • list - every cassette, its call count and which adapters it used.
  • inspect <name> - the interactions in one cassette; --index n for a full payload.
  • prune - cassettes no longer referenced from any source file. Dry run by default.
  • verify - fails if any committed cassette contains something that looks like a credential. Put this in CI:
- run: npx cassette verify
Checked 1 cassette in /path/to/examples/cassettes.
No credentials found.

Adapters

The adapter registry is the architecture. One file per SDK, and every adapter is validated by the same shared contract suite.

| Adapter | Covers | | ----------- | ---------------------------------------------------------------------------- | | anthropic | @anthropic-ai/sdk: messages.create (streaming and not), countTokens | | openai | openai: chat.completions.create, responses.create, embeddings.create | | tools | Any tool registry: bare functions or { execute } descriptors | | fetch | Generic HTTP fallback for providers without a dedicated adapter |

Adapters use structural typing against the SDK surfaces, so the SDKs stay optional peer dependencies and the whole test suite runs with neither installed, with no network and no API key.

Adding one is one file plus one call to runAdapterContract. See CONTRIBUTING.md.

Test output

Run from a clean checkout with npm test, offline:

 RUN  v2.1.9 /Users/.../cassette

 ✓ test/matchers.test.ts (20 tests) 3ms
 ✓ test/redact.test.ts (23 tests) 5ms
 ✓ test/drift.test.ts (18 tests) 3ms
 ✓ test/store.test.ts (26 tests) 39ms
 ✓ test/example.test.ts (10 tests) 16ms
 ✓ test/session.test.ts (18 tests) 44ms
 ✓ test/contract/anthropic.test.ts (33 tests) 80ms
 ✓ test/cli.test.ts (24 tests) 110ms
 ✓ test/contract/tools.test.ts (33 tests | 3 skipped) 84ms
 ✓ test/contract/openai.test.ts (33 tests) 134ms
 ✓ test/contract/fetch.test.ts (33 tests | 3 skipped) 136ms

 Test Files  11 passed (11)
      Tests  265 passed | 6 skipped (271)
   Duration  440ms

The four test/contract/* files are the same 33 assertions applied to four different adapters. The six skips are streaming assertions on the two adapters that do not stream.

Documentation

Known limitations

Stated plainly, because finding these out later is worse:

  • client.messages.stream(), the Anthropic event-emitter helper, is not intercepted yet. messages.create({ stream: true }) is. See the good-first-issues list.
  • Recording a stream drains it fully before returning, so record mode does not reproduce live token timing. Replay and record then return the same shape, which is deliberate.
  • The fetch adapter buffers response bodies, so it is not suitable for very large downloads, and it stores binary bodies as a size marker rather than base64.
  • prune finds cassette names written as string literals. Names built at runtime cannot be detected, which is why it dry-runs by default.

Built and maintained by Duskel

We build and audit production AI agents. cassette came out of needing our own agent test suites to run in CI without an API key and without flaking, and we maintain it because we use it.

If you are shipping an agent and want a second pair of eyes on it, or you want one built properly the first time, talk to us: duskel.com.

Issues and adapter pull requests are welcome regardless.

License

MIT, copyright Duskel. See LICENSE.


Built and maintained by Duskel — an AI automation & software studio. We build LLM apps and needed deterministic tests that didn't depend on the network to run. Work with us.