Gauntlet

One review engine, four checkpoints. A provider-neutral code review harness that runs at your agent's stop-hook, your pre-commit hook, your branch CI, and your whole-project audit — and remembers, across all four, what you already decided.

MIT. Node 20+. Near-zero dependencies.

Status: published. It reviews real code with a real agent at all four checkpoints, and as of 0.3.0 it reviews a running UI in a real browser. The pipeline, the BYO agent-CLI backend, the Assay backend (local binary and hosted transport), the prove backend and its Chrome driver, the four trigger adapters (stop-hook config, pre-commit hook, CI workflow, project audit) and init/doctor are built and tested. Verified end to end against the live claude CLI, the live hosted Assay API, and two live websites — where it found a real invisible-state-flip defect. See docs/implementation-plan.md for what was measured rather than assumed.

Why

Review happens at four disconnected points, each with its own tool, its own output format, and no shared memory. The same finding nags at every checkpoint. A dismissal at one is invisible to the rest. Gauntlet is one engine invoked at all four, with one findings contract and one verdict memory.

It analyzes. It never writes fixes, never edits source, and never writes outside .gauntlet/. A reviewer that cannot edit cannot rationalize its own fixes — and a read-only tool is what a governed environment can actually approve.

Install

npm install -g gauntlet-review     # or: npx gauntlet-review@latest doctor

cd your-repo
gauntlet init        # wire the four checkpoints (prints a plan first)
gauntlet doctor      # what is configured, what is missing, no secrets printed

It sends nothing unless you ask it to — see Telemetry, where every field it would send is listed by name.

Try the pipeline

The fake backend drives the whole engine with deterministic fixtures, so you can watch each stage react without paying for an agent:

gauntlet review --stage=project --fixture=findings --json
gauntlet review --stage=pre-commit --fixture=critical-finding   # exits 1
gauntlet review --stage=pre-commit --fixture=malformed          # exits 2, never "clean"

# Then dismiss one and watch it disappear from every stage
gauntlet dismiss <finding-id> --reason "the id is internal, never user input"

From a clone instead:

npm install
npm test              # 488 tests
npm run typecheck
npm run build
npm run prove:silence # 0 outbound calls with telemetry off, and the control proves the probe works
node dist/cli.js review --stage=project --fixture=findings --json

Use a real reviewer

Point Gauntlet at the coding agent you already pay for. Presets ship for the CLIs whose flags were read from their own --help; anything else is an argv array you supply.

{ "version": 1, "backend": "cli", "cli": { "provider": "claude" } }

| Provider | Invocation | Verified | |---|---|---| | claude | claude --print --output-format text --disallowedTools Edit,Write,… | yes | | codex | codex exec --sandbox read-only --skip-git-repo-check --color never - | yes | | copilot | copilot -p | no — flags unread; the report says so on every run | | custom | your own cli.command argv array | — |

node dist/cli.js review --stage=pre-commit --file src/login.ts

What the child process gets: the prompt on stdin (never argv, which shows up in ps), an allowlisted environment rather than yours, its own process group so a timeout kills anything it spawned, a capped stdout, and a worktree photographed before and after. If it edits a file, the review is inconclusive and the report names the file.

Review a running UI

Reading code cannot tell you whether a button does anything a person can see. A handler can be perfect, its state consumed, and the panel it reveals can render 1,900px below the fold — no line of source is wrong. So Gauntlet also drives.

# a Chrome that is already listening. Gauntlet never installs or launches one.
chrome --remote-debugging-port=9222 --headless=new --window-size=1440,900

{ "version": 1, "backend": "prove", "prove": { "url": "http://localhost:3000/checkout" } }

gauntlet review --stage=project

It enumerates every affordance, declares the promise each one makes before pressing it, restores the page between presses, presses for real, and measures whether the effect became perceptible — then classifies:

perceptible · not-rendered · zero-size · transparent · clipped · occluded · outside-viewport · sliver · invisible-state-flip

Findings carry runtime evidence — route, role, accessible name, and the measurement in the units a person experiences — and flow through the same identity, verdict memory, gates and exit codes as code findings. Identity is keyed on the promise, never on coordinates, so a dismissal survives the layout moving.

Zero new dependencies. CDP is a WebSocket carrying JSON, and Node has both. BYO browser, for the same reason as BYO reviewer: a review harness that drags a 300MB dependency behind it is one a governed environment cannot approve.

It will not report clean on a page it did not fully examine. Declare 40 affordances and measure 12, and the other 28 are named and the review is inconclusive. Skip the call to action and it is inconclusive. Find zero affordances — almost always a sweep that ran before the app rendered — and it is inconclusive. Silence is never available.

| Key | What it does | |---|---| | prove.url | the page to sweep | | prove.followNavigation | drive links that leave the page (default off: following one ends the sweep of this surface) | | prove.maxAffordances | cap the sweep; whatever it cuts off is named, never silently dropped | | prove.cdpEndpoint | default http://127.0.0.1:9222 | | prove.reportPath | ingest a report produced elsewhere instead of driving |

This is the prove-every-action discipline as tested code rather than as instructions a run can forget: enumerate, declare, restore, press, measure, classify. Driving the page's actual call to action end to end is still yours to supply — report it through the journey verdict, and note that omitting it is itself inconclusive.

Commands

gauntlet review --stage=<stop-hook|pre-commit|branch|project> [options]
gauntlet review --format=<terminal|json|github>
gauntlet review --url=<url>   sweep this page (prove backend), overriding config
gauntlet dismiss <finding-id> --reason "<why>" [--scope=…] [--expires=…]
gauntlet stop-hook            reads a Stop-hook payload on stdin
gauntlet init [--force] [--dry-run] [--merge-precommit] [--backend=<name>]
gauntlet doctor [--repo=<path>]
gauntlet telemetry <on|off|status>

Wiring the four checkpoints

gauntlet init wires all four. It prints a plan, creates what is missing, and never replaces a file it did not write — an existing pre-commit hook is left alone and reported, because a tool that silently overwrites someone's hook to install itself has broken their repository. Run it twice and the second run writes nothing. --dry-run shows the plan and touches nothing; --force replaces.

gauntlet init --dry-run     # see the plan
gauntlet init               # write it
gauntlet doctor             # check the result

Each adapter is thin: it assembles stage context, calls the one runner, renders, and maps to an exit code. If you would rather wire them by hand:

Agent stop-hook. gauntlet stop-hook speaks the Stop-hook protocol that Claude Code and Codex CLI share — a JSON payload on stdin, and {"decision":"block","reason":"…"} on stdout when the gate blocks. It always exits 0, and every failure path (not a repository, malformed payload, reviewer crashed) lets the turn end with a note on stderr. A reviewer that wedges an agent session is worse than one that misses a finding.

For Claude Code, in settings.json:

{ "hooks": { "Stop": [{ "hooks": [{ "type": "command", "command": "gauntlet stop-hook", "timeout": 90 }] }] } }

stop_hook_active is honoured, so a blocked turn is never blocked twice by this hook.

Pre-commit. Generate the hook with preCommitScript() from src/triggers/precommit.ts and write it to .git/hooks/pre-commit (mode 755). It reviews the staged diff, skips when nothing is staged, and fails closed if Gauntlet cannot run — printing the bypass on the way out:

GAUNTLET_SKIP=1 git commit ...     # skip this hook
git commit --no-verify ...         # skip every pre-commit hook

A gate that waves commits through when its own binary is missing looks configured and does nothing.

Branch / CI. branchWorkflow() in src/triggers/branch.ts emits a GitHub Actions workflow. --format=github puts one annotation per finding on the changed lines of the PR diff and writes a summary table to $GITHUB_STEP_SUMMARY. The workflow checks out with fetch-depth: 0, which is load-bearing: without full history there is no merge base and the branch scope silently degrades to the wrong diff.

Project. projectWorkflow() in src/triggers/project.ts emits a scheduled sweep. It is report-only — it never fails a build over a pre-existing finding — but it does fail loudly on exit 3 (bad configuration) and exit 2 (the reviewer failed), because a 3am job that cannot tell those from "clean" is worse than no job. Run it by hand any time with gauntlet review --stage=project.

Exit codes

| Code | Meaning | |---|---| | 0 | clean, or a non-blocking result according to the invoking stage's policy | | 1 | blocking findings | | 2 | inconclusive review or operational failure | | 3 | invalid invocation or configuration |

CI can tell all four apart. That is the point: "the reviewer crashed" and "your code is fine" must never produce the same exit code.

The two fields people conflate

reviewOutcome  what happened during the review   clean | findings | inconclusive
gateDecision   what this stage does about it     pass  | block    | warn

They are separate on purpose. One unverifiable finding among six makes the outcome inconclusive without forcing a fast stage to block; a strict stage can still block on it. A non-blocking stop-hook may exit 0 on an inconclusive review — it may not rewrite the report to clean.

Verdict memory

gauntlet dismiss <id> --reason "…" appends one auditable record to .gauntlet/memory.jsonl. From then on, that finding is filtered at every stage. The file is committed, so the team and CI share one brain; it is append-only JSONL, so two branches that both appended merge as a union.

Guardrails that are not optional:

• A dismissal with no reason is refused.
• Conflicting verdicts never silently favour dismissal — the finding stays visible with a warning.
• A change may not create a dismissal and then use it to hide its own finding. At the branch stage, records introduced by the change under review do not suppress.
• Security findings are not suppressed by blanket path- or rule-scoped dismissals.
• Corrupt memory never blocks a commit: bad lines are skipped with diagnostics, git conflict markers are tolerated, a missing file is empty memory.

Finding identity

A finding's logical id survives lines inserted above it, reformatting, a renamed file, and a moved function — because it is anchored on the enclosing symbol and the normalized code, never on the reviewer's prose. Its occurrenceId says where that issue currently lives, so one defect copied into two places is one issue with two occurrences.

Measured, not asserted. Three independent live claude runs over the same file:

run A  f_e35231e1f1873e22  "Password check compares plaintext values with ==="
run B  f_e35231e1f1873e22  "Password comparison uses non-constant-time equality"
run C  f_e35231e1f1873e22  "Password comparison uses non-constant-time equality"

Three different titles. One id. That is what a dismissal has to survive.

A model's own ruleId is deliberately thrown away, for the same reason its prose is: in those runs one SQL injection was named security/sql-injection on one pass and security/sql-injection-string-interpolation on another. A backend that declares ruleIdsAreStable — a static analyzer — keeps its rule id.

When identity confidence is low, Gauntlet creates a new finding rather than letting a weak match inherit someone else's dismissal.

Evidence is checked, not trusted

Every finding's evidence is validated against the actual review bundle: the path must resolve inside the repo and be in the bundle, the range must be positive and ordered, and the snippet must match — or safely re-anchor to where it really is. A fabricated path, or a snippet that appears nowhere in the cited file, is discarded and the review is marked inconclusive. Valid findings in the same report are still emitted.

Backends

| Backend | What it is | Network | |---|---|---| | fake | Deterministic fixtures. Proves wiring, never a real review. | none | | cli | Your own agent CLI (Claude, Codex, Copilot, or any configured argv) as a contained subprocess. | whatever your agent does | | assay (local) | The local Assay binary's formal checks, over MCP stdio. | none by default | | assay (hosted) | The hosted 3-phase pipeline — claim extraction, verification (LLM + formal), remediation — at <baseUrl>/api/v1/forward. Opt-in, ~85s per file. | your source is sent to that server |

The Assay backend deserves its specifics stated, because they were measured rather than assumed:

• Assay returns no file path, so Gauntlet verifies one file at a time and attributes the path itself.
• Its line number points at the enclosing unit, not the offending line — a function on line 1 whose SQL injection sits on line 5 is reported as line 1. Gauntlet treats it as a search hint and locates the real lines, so the cited snippet is always genuine source read from the bundle, never Assay's prose.
• Its checkName is stable across runs, so unlike a model backend it keeps its rule id in finding identity (ruleIdsAreStable).
• Its formal/LLM metadata maps into the core's optional verification structure. Nothing Assay-shaped reaches the contract.
• Its learned-rule findings are excluded by default. A live call returned 48 of them for a three-line function, carrying literal {line} placeholders and an i18n rule that "matched" the tokens export, function, id. They cannot be anchored to real code. Set assay.includeLearnedRules if you want them.
• It only covers TypeScript, JavaScript and Python. Files in other languages are skipped and the count is reported, because a sweep that examined 2 of 40 files and said "clean" is the exact failure this project exists to stop.

The hosted Assay transport

Set assay.baseUrl and assay.sendSource: true and the backend talks to the hosted 3-phase pipeline instead of the local binary. It finds more, and it costs three things that are stated here so you can decide before you run it, not after:

• It sends your source code to a server. One HTTPS POST per file, body {code, language}. Gauntlet refuses to start unless assay.sendSource: true says that out loud — setting a URL is not consent.
• It takes at least 85 seconds per file, and longer as files grow. Measured 2026-08-26, one call per file: 69.6-84.5s across five calls on a three-line function, 82.5s on a 77-line file, and 144.6s on a 177-line file. Treat 85s as the floor, not the typical. Because of it, Gauntlet refuses the hosted transport at any stage whose budget is under 180,000 ms — which by default means the stop-hook (60s) and pre-commit (120s), where it could only ever time out and report inconclusive. Raise stages.<stage>.timeoutMs if you want it there anyway. gauntlet doctor tells you which stages currently refuse it.
• Its output does not reproduce. Two calls on byte-identical source returned 14 claims and 12 claims; one description of fourteen was identical; the verdict split moved from 5 FAIL / 5 PARTIAL / 4 PASS to 2 / 4 / 6. So the hosted path emits no rule id and declares ruleIdsAreStable: false — folding an unstable id into finding identity would mean a dismissal recorded today stops matching tomorrow.

How the pipeline's three phases map onto a Gauntlet finding:

| Hosted verdict | Becomes | verification.status | confidence | |---|---|---|---| | FAIL | a finding — the claim about the code does not hold | verified | high if formally verified, medium if the LLM judged it | | PARTIAL | a finding, deliberately kept | unverified | low, so the stage's minConfidence floor decides whether it surfaces | | PASS | nothing — a satisfied claim is not a defect | — | — |

An unrecognised verdict, or a verification_method other than formal/llm, is never guessed at: the first is dropped with a warning, the second degrades to method: 'none', status: 'unverified'.

The one thing that would quietly ruin this: the hosted pipeline reports its evidence as prose, with a truncated code fragment embedded in it — "Code line: db.query(...) - If id = "1 OR 1=1"..." — and gives no file path and no line number anywhere in the payload. An adapter that trusted those strings positionally would emit plausible-looking findings that fail evidence validation and turn every review inconclusive. Gauntlet runs the same locateEvidence anchoring the local path uses: the fragments are searched for in the bundle, the snippet is read back out of the file, and a claim that cannot be anchored is dropped with a warning rather than cited at a line someone made up — and counted, because a review that had to drop what it found is inconclusive, never clean. It quotes code with backticks or single quotes depending on the file, so both are searched. Measured over seven live responses (43 non-PASS claims, across a 3-line fixture and real 77- and 177-line files): 40 anchored to the correct line, 3 were dropped for carrying no code fragment at all, and 0 produced a snippet the file does not contain.

The hosted key is read from the environment by name (ASSAY_API_KEY, or whatever assay.apiKeyEnv points at). It is never placed in argv, never handed to a subprocess — there isn't one, the hosted path is straight fetch — and it is scrubbed out of every warning and error this backend produces, including a body the server echoes back.

{
  "version": 1,
  "backend": "assay",
  "assay": {
    "baseUrl": "https://api.tryassay.ai",
    "sendSource": true,
    "apiKeyEnv": "ASSAY_API_KEY"
  }
}

Telemetry (off, and off is silent)

Gauntlet sends nothing. Not "nothing important" — nothing. Off is the default, and while it is off no identifier is minted, no file is written outside your repository, and no outbound request is made at all. There is nothing to disable because nothing was ever started.

That is checked, not asserted. npm run prove:silence runs a full gauntlet review --stage=project over this repository with a probe wrapping fetch, net.Socket#connect and dns.lookup, and fails unless the count is zero — and it fails just as hard if the control run (telemetry on, pointed at a closed local port) also reports zero, because a probe that measures nothing would look exactly like a tool that sends nothing:

telemetry off : 0 network attempt(s), exit 0
telemetry on  : 1 network attempt(s) (fetch http://127.0.0.1:9/prove-silence), exit 0

If you turn it on

gauntlet telemetry on       # opt in
gauntlet telemetry off      # opt out; deletes the id and the counters
gauntlet telemetry status   # what is on, what would be sent, and where

There is one question worth measuring, and it is the one this project could be wrong about: is the verdict memory load-bearing, or just a nice idea? A first dismissal only proves someone tried the tool. A second dismissal, weeks later, from the same install, means someone is curating a memory they intend to keep relying on. So each dismissal carries its own ordinal and the days since that install's first one.

Every field, by name. There is no other field, and no free-text one.

Sent with both event kinds:

| Field | What it is | |---|---| | v | payload schema version — currently 1 | | event | review or dismissal | | installId | a random UUID, made when you opt in, destroyed when you opt out | | gauntletVersion | e.g. 0.2.0 | | at | the timestamp of the event |

Sent with a review:

| Field | What it is | |---|---| | stage | stop-hook | pre-commit | branch | project | | backend | fake | cli | assay | other — the family, never your configured provider name | | outcome | clean | findings | inconclusive | | gate | pass | block | warn | | findings | how many findings were reported (a number) | | suppressed | how many the verdict memory filtered (a number) |

Sent with a dismissal:

| Field | What it is | |---|---| | stage | stop-hook | pre-commit | branch | project | manual | other | | verdict | dismissed | accepted-risk | fixed | reopened | stale | | scope | repository | path | rule | occurrence | | ordinal | 1 for your first ever dismissal, 2 for the second, … | | daysSinceFirst | whole days since your first dismissal |

What is never sent, and cannot be: source code, file paths, finding titles, finding ids, rule ids, dismissal reasons, repository names, your git identity, branch names, or anything derived from them. Every string above is either a random UUID or a value from a fixed list in src/core/signal.ts — free-text config like backend is clamped to that list before it is put in a payload, so a value from your gauntlet.json cannot ride along. A test drives a full review and a dismissal in a repository whose file names, finding titles and dismissal reason are all distinctive, and asserts that none of them appear anywhere in the bytes sent.

Where it goes and who decides.

• Endpoint: https://seatrial.io/api/gauntlet/signal. Change it with telemetry.endpoint in gauntlet.json, or GAUNTLET_TELEMETRY_ENDPOINT.
• Consent is yours, not your repository's. It lives in ~/.gauntlet/telemetry.json, so a committed config file cannot opt your contributors in. A repository may set "telemetry": { "allowed": false } to forbid it outright — the config schema has no way to express the opposite.
• DO_NOT_TRACK=1 or GAUNTLET_TELEMETRY=0 turns it off no matter what the state file says. GAUNTLET_TELEMETRY=1 turns it on for one shell, for CI.
• Everything sent is also appended to ~/.gauntlet/sent.jsonl. "Show me exactly what you sent" is cat, not trust.
• It cannot bite: a 2-second timeout, every failure swallowed, and a test that a dead endpoint changes neither the exit code nor a single verdict.

Sharing the pre-commit slot

.git/hooks/pre-commit is one slot, and more than one tool wants it. Gauntlet will not take a hook it did not write — but "exists, left unchanged" is a useless thing to be told, so it names what it found and what to do:

exists    .git/hooks/pre-commit — shipsafe-mcp (Seatrial Sentinel)  (left unchanged)

shipsafe-mcp (Seatrial Sentinel) already owns .git/hooks/pre-commit. It scans
staged source files with Sentinel and blocks on critical-or-above.

These two do not do the same job, and neither replaces the other:
  shipsafe-mcp (Seatrial Sentinel)   one checkpoint  — the staged diff, at commit time
  gauntlet                           four checkpoints — agent turn, commit, branch, project

The disagreement worth knowing about: shipsafe-mcp lets a commit through when its
own scan cannot run, and gauntlet refuses one.

To run both:

gauntlet init --merge-precommit

The existing hook moves to .git/hooks/pre-commit.local and a dispatcher takes its place. The incumbent runs first — it is the cheap gate, and paying an agent to learn what a static scan already knew is a bad trade on every commit. If it blocks, Gauntlet never runs. Undo the whole thing with mv .git/hooks/pre-commit.local .git/hooks/pre-commit.

GAUNTLET_SKIP=1 skips Gauntlet only; the chained hook still runs. Gauntlet's bypass is not a master switch for somebody else's gate.

shipsafe-mcp is recognised because it ships from the same house as Gauntlet and does a genuinely different job: a static scan of the staged diff, in under a second, with no model involved. If you want one, you probably want both — its side of the story is at seatrial.io/docs/mcp/workflow. Any other hook is left alone and reported as unrecognised, with the same merge offer.

Trust boundaries (stated plainly)

• Repository content is untrusted data. Every request carries explicit prompt-injection guard rules. Source, comments, docs, fixtures and file names are treated as an injection surface.
• Secrets are redacted from the bundle before the reviewer or the report sees them. Redaction is best-effort pattern matching, not a guarantee.
• Gauntlet does not sandbox your CLI agent. An invoked agent runs with the permissions your organisation already grants it. Gauntlet contains what it can — no shell, argv arrays only, an allowlisted environment, its own process group, a killed group on timeout — and it detects worktree mutations and returns inconclusive. But "Gauntlet did not edit" is not the same claim as "the agent could not edit", and only the first one is ours to make. The codex preset asks that CLI for its own read-only sandbox; the claude preset denies the edit tools. Neither is a substitute for a real sandbox.
• Your source leaves this machine only if you say so. By default the assay backend uses the local binary and makes no network calls. Setting assay.baseUrl sends the full text of every reviewed file to that server, one POST per file, and Gauntlet refuses to start until assay.sendSource: true says that out loud — a URL on its own is not consent. That hosted path takes 85 seconds per file at minimum and about 145s on a 177-line file (measured; see above), so a 40-file review means 40 files uploaded and one to two hours of wall clock. The cli backend sends whatever your agent sends — that is between you and your agent.
• gauntlet doctor never prints a credential. It reports which credential variables are visible, by name, and nothing else. It is meant to be safe to paste into an issue. The same rule binds the hosted Assay transport: the key is read from the environment by name, never enters argv or a child process, and is scrubbed from every message the backend emits.
• Gauntlet phones home only if you tell it to. Telemetry is off by default, and off means zero outbound calls — proven by npm run prove:silence, not by this bullet. See Telemetry for every field by name.
• Gauntlet does not guarantee your code is safe. It is a review harness with evidence checks and policy controls. It finds what the configured reviewer finds.

Configuration

gauntlet.json, versioned. Precedence: built-in defaults → gauntlet.json → stage override → CLI flags → environment variables.

{
  "version": 1,
  "backend": "fake",
  "reviewTypes": ["quality", "security"],
  "ignore": ["node_modules/**", "dist/**", "coverage/**"],
  "stages": {
    "stop-hook": { "blockingSeverity": "critical" },
    "pre-commit": { "blockingSeverity": "high", "inconclusiveBehavior": "block" }
  },
  "fake": { "fixture": "critical-finding" }
}

fake.fixture pins what the fake backend reports. It is how you confirm a freshly installed hook actually fires — pin a fixture, make a commit, watch the gate react — without paying for a real agent.

Unknown keys are rejected. A typo that silently reviews with defaults is how a gate ends up doing nothing while looking configured.

Documents

• docs/superpowers/specs/2026-08-25-gauntlet-design.md — the design
• docs/build-brief.md — the implementation authority
• docs/build-brief-review.md — the review that shaped it
• docs/implementation-plan.md — build order, status, and decisions taken