depug

depug turns TypeScript test failures into evidence a coding agent can read, and answers questions about a run by measuring it rather than by reading the source.

An agent debugging a failed test usually infers what a value was from the code around it. depug reruns the test with instrumentation and reports what the value actually was.

Works with vitest and with Node's own test runner. Development used Node v26.7.0, vitest 4.1.11, and typescript 6.0.3.

Install

npm install --save-dev @meganemura/depug

depug carries its own copy of typescript, so the version your project is on does not have to match and does not have to change.

To move an exact pin to a later release, name the version. npm install @meganemura/depug finds the pin already satisfied, changes nothing, and exits 0, and npm update does the same; npm outdated is what reports the gap.

npm install --save-dev --save-exact @meganemura/depug@latest

Or from a checkout: npm install && npm run build, then point a dependency at the directory.

Add the reporter to your vitest config. Nothing else has to change:

import { defineConfig } from "vitest/config";
import DepugReporter from "@meganemura/depug/reporter";

export default defineConfig({
  test: {
    reporters: ["default", new DepugReporter()],
  },
});

The verbs below need no configuration at all. Each one generates its own config from yours, runs one test, and throws the generated file away.

Start from the failure

A failed test now prints two lines:

depug evidence: /abs/path/tmp/depug/run-20260903-010701-94176/001-expandipv6-should-result-be-valid.json (the value's source already returned; rerun to reach it)
depug rerun: npx vitest run "src/utils/ipaddr.test.ts" -t "^expandIPv6 Should result be valid$"

The parenthetical says whether that file can answer:

• the failing call is in these frames — the code that produced the wrong value is still on the stack. Read the file.
• the value's source already returned; rerun to reach it — it is not, and no deeper capture of that run would find it. Use a verb.

Across 155 real failures from one project's history, 94.2% were the second kind. That is the case the rest of this tool exists for.

depug never runs the rerun command for you. Every verb starts a process only when you invoke it, so a green suite pays nothing.

The verbs

Each verb takes the command the failure printed, with a verb in front:

depug frames    -- npx vitest run "src/utils/ipaddr.test.ts" -t "^expandIPv6 Should result be valid$"
depug preflight -- ...
depug probe "src/utils/ipaddr.ts:expandIPv6@13:14"    -- ...
depug flt   "src/utils/ipaddr.ts:expandIPv6@13:14#7"  -- ...
depug exec  "src/app.ts:sumUntil@18:17#1" --line 21 --statement "total = 100" -- ...

frames — what did this test call?

depug frames: tmp/depug/frames-P8OxUK/frames-3222.jsonl
depug calls: 15

One JSONL record per call, each named by a function id.

You usually arrive knowing a line, not an id. --at converts one:

depug frames --at src/utils/url.ts:208 -- npx vitest run ...

depug at src/utils/url.ts:208, innermost first:
  src/utils/url.ts:parse@201:12#1
  src/utils/url.ts:hc@151:14#1

The innermost function comes first, because its locals are what that line touched. Only calls the run actually recorded are listed.

Where the application boundary is

frames and preflight instrument the files under one or more directories and treat everything else as not the application. The default is src/. A project that keeps its code elsewhere names it once in package.json:

{ "depug": { "include": ["cli", "viewer/lib"] } }

--include <path> on the command line overrides that, and can be repeated. A test file is never instrumented, whichever set it falls in: a function you want to watch has to live under the boundary, not in the test.

When an index comes back empty, the notes say how many files each directory holds that depug would instrument, where the boundary came from, and whether the test's own file sits outside it:

depug calls: 0
depug note: no application calls were recorded
depug note: src/ holds 61 file(s) depug would instrument (--include from the default)
depug note: the test's own file tests/probe.test.ts is outside that set, and a test file is never instrumented; a function to watch has to live under --include

Zero then says whether depug was watching anything, which is the difference between a wrong path and a test that called nothing.

preflight — is this test safe to address by call index?

depug preflight: deterministic (app calls: 214)

Every other verb names a call by its index inside the test. An index that moves between runs names a different call. Run this first. Measured on 80 tests across 53 files of one project, 80 matched their own second run.

probe — what did this function receive and return?

src/utils/ipaddr.ts:expandIPv6@13:14  calls: 7, threw: 0
  ipV6: "1::1", "::1", "2001:2::", "2001:0:0:db8::1", "::ffff:127.0.0.1", "::ffff:0.0.0.1"
  returns: ..., "0000:0000:0000:0000:0000:ffff:7f00:0001", "0000:0000:0000:0000:0000:0000:ffff:0001"
  observed: string
  declared: string

Use this when a function ran several times and you do not know which call went wrong. Every call's argument and return sit beside each other.

It also puts what a value was beside what it was declared to be:

  observed: {id: string, email: undefined (3 / 5 calls) | string (2 / 5 calls)}
  declared: {email: string, id: number}
  mismatch: id was string, declared number (5 of 5 calls)
  mismatch: email was absent, declared string (3 of 5 calls)

That output came from a suite that passes. JSON.parse, an as cast, process.env, and an any parameter each let a value through without checking it, and where the annotation stopped making a claim, only a run can say what came through.

flt — where inside this call did the value go wrong?

call  locals: {"ipV6": "::ffff:0.0.0.1"}
line  14  new:     {"sections": ["", "", "ffff", "0.0.0.1"]}
line  16  changed: {"sections": {"old": ["", "", "ffff", "0.0.0.1"], "new": ["", "", "ffff", "1"]}}

Line 16 is the answer: an embedded IPv4 address collapsed into one group instead of two.

Records arrive in completion order, not source order, because each is written after its statement finishes. Read the line field. A loop keeps its first and last iteration and folds the middle into a marker carrying the count.

rerun — run it again, under a clock

The rerun line a failure prints already carries this:

npx depug rerun -- npx vitest run "test/user.test.ts" -t "^parses a user$"

It runs the command and, if nothing comes back in time, stops it. The default is 120 seconds; --timeout <s> changes it.

This is here because a test that stops returning takes a core with it and neither runner will stop it. --test-timeout cannot: the timer it needs runs on the worker's event loop, and the worker is what is blocked. One machine carried three runs of a printed rerun line for 8 to 12 days at a load average of 4-5. What does work is a signal to the runner, whose own loop is free, and that is what this sends.

The re-execution verbs carry their own clock, so a printed line pasted after one is unwrapped rather than supervised twice:

depug frames -- npx depug rerun -- npx vitest run "test/user.test.ts" -t "^parses a user$"

exec — what would happen if the value were different?

depug value: 100
depug result: fail (exit 1)

The expression runs in that line's own scope, so total = 100 assigns the function's own total. It can change what the test does and perform side effects; the run's own outcome is reported beside the value for that reason. Only the launcher arms it.

A worked example

A truncation bug in one project's IPv6 handling, reproduced at the commit before its fix, diagnosed in three commands and about four seconds:

1. The suite fails and prints the two lines.
2. probe lists seven calls; the seventh's return has ffff one group further along than the sixth's.
3. flt on #7 shows ["", "", "ffff", "0.0.0.1"] becoming ["", "", "ffff", "1"] at line 16.

No step required reading the implementation to guess a value.

What depug does not observe

These are absences the files declare, not facts about the program.

• A suspend with no resume after it: the awaited value rejected.
• parameters_not_observed in a probe: a destructured parameter has no single binding to read, so the value is unknown, not undefined.
• skipped_iterations in a trace: a loop's folded middle went unobserved.
• Events under test.concurrent: two concurrent tests in one worker share one current-test pointer, so test on a record can name the wrong one. The call ids stay correct.
• A trace carries no suspend or resume in v0.1.

Cost

The always-on layer writes a file when a test fails and does nothing otherwise. Measured on node:test, three suite shapes run nine times each -- 700 trivial tests, 200 tests each spawning a shell, and 200 spawning a shell that does real work -- the reporter cost nothing above the noise, with --experimental-test-coverage on. A second reporter alongside it cost nothing either.

One project reported about a fifth on a suite where each of 700 tests takes about 226 ms, spawning a shell per gate check and contending real processes against a lock. That measurement turned out not to separate the reporter from the order it ran in: depug ran second in every pair, and running the same configuration three times in a row on that machine gave 137 s, 171 s, and 199 s. A 45 % drift with nothing changed swallows the 25-35 s the comparison had shown, so it is not evidence either way.

If your suite is large and process-heavy, measure it rather than trusting any of these numbers, and beware the same trap: alternating A then B still runs A first every time, so a machine drifting through a sitting charges the difference to whichever arm goes second. Run each pair in both orders, take several, and check the drift first by running one arm against itself.

A verb's instrumentation is a fixed cost of about 67 ms for each worker process, dominated by loading typescript, plus about 46 ns for each recorded event. That is a formula rather than a multiplier because a ratio only describes the workload it was measured on: the same instrumentation measured 1.12x on a workload with heavy function bodies and 2.04x on one with four million trivial calls.

Instrumenting one project's whole suite ran 4961 tests to the same result as without it, in 14.31 s against a 14 s baseline.

Monorepos

A repository that splits its run into vitest projects works without extra arguments, as long as --include points inside the project holding the code:

depug frames --include packages/zod/src -- npx vitest run "packages/zod/src/.../array.test.ts" -t "^array min/max$"

vitest does not apply a root config's plugins to a project's own config, so depug reaches the project itself. Where the projects resolve to a config per package, it names the one holding your code, extends that config, and keeps that package's directory as the root, so every relative path in it resolves where it did before. Where the projects are written inline in the root config, it rewrites each of them to carry the instrumentation.

Verified against two real repositories of each shape.

Test runners

The verbs read the command you hand them and set themselves up accordingly:

depug frames -- npx vitest run "test/user.test.ts" -t "^parses a user$"
depug frames -- node --test test/user.test.ts

The instrumentation is the same either way, because depug rewrites TypeScript before it executes rather than hooking a runner. vitest owns that step through a vite plugin; Node owns it through module.registerHooks, which the verbs reach with NODE_OPTIONS. The ids, the coordinates, and the files are identical.

For node:test the reporter is a --test-reporter:

node --test --test-reporter=@meganemura/depug/node-test-reporter --test-reporter-destination=stdout \
     --test-reporter=spec --test-reporter-destination=stderr

One difference is worth knowing. --test-name-pattern matches a test's own name and not the suites around it, and repeating the flag widens the selection rather than narrowing it, so a rerun command there names the test and any same-named test elsewhere in the same file. That is node:test's CLI; the printed command says what it selects.

Scope of version 0.1

vitest and node:test. jest comes later. The files are the interface, and the evidence schema is the contract. The design decisions explain each choice with the measurement behind it.

The bundled skill tells an agent how to read all of this.

depug is the TypeScript sibling of bulldogger, a Ruby gem by the same owner.

License

MIT. See LICENSE.