Shipcraft

A seatbelt for test-driven development with coding agents.

Code is cheap now. Judgment isn't. Shipcraft makes sure the judgment you put into a test can't be quietly undone by the code that is supposed to satisfy it, and it stays out of the way the rest of the time.

$ git commit -m "fix rounding"
shipcraft refused: locked test tests/test_invoice.py changed after RED (index)
Why: RED is the contract: GREEN changes the code, not the test that judges it
Next: undo the change in tests/test_invoice.py; if the test looks wrong, stop and tell the RED author

Why

A coding agent asked to make a failing test pass has two ways to do it: fix the code, or "fix" the test. Most of the time it fixes the code. Sometimes, under pressure from a hard bug, a confusing assertion or a long session, it edits the assertion, skips the case, deletes the file, or changes the runner config so the suite goes green. Every one of those looks like progress in the diff summary.

Prose rules in AGENTS.md ("never edit the tests") help, but nothing enforces them. Shipcraft turns that rule into a mechanical check that runs where agents can't argue with it: in the git hook and again in CI.

What it catches

| The problem | What Shipcraft does | |---|---| | An agent edits, skips or deletes a failing test, or changes the test config, so the suite goes green | The git hook refuses the commit, in three lines: what happened, why, and the next step | | Someone skips the hook (git commit --no-verify, an edited hook) | shipcraft ci re-checks every commit of the branch from a clean checkout; the full suite runs in CI too, on your machine or in your GitHub checks (docs/github-ci.md), except on docs-only PRs | | A PR changes the config to switch off its own checks | The PR is judged by the config on the base branch, not by its own | | The code that passes the tests was never read by anyone else | A different agent (Codex, Claude or Gemini) reviews the PR against your spec; a blocker counts only if it reproduced it | | A merge goes in before CI and review are green | shipcraft merge refuses; --no-review "<reason>" waives the review, never CI, and the reason is recorded in the PR and in the numbers | | Agents that don't know the process | init writes the cycle into AGENTS.md, and every refusal names the next step, often the exact command | | No way to tell whether the process pays for itself | Every hook, CI and review run is logged on your machine; shipcraft report gives the times and refusals |

How it works

Work is split into two roles, and they should be different agents (or an agent and you):

• RED writes the tests that describe the change and runs shipcraft red. Shipcraft runs the suite, requires it to fail on assertions, and writes a lock with a hash of every test file, the config and the failing cases. The lock is committed with the tests.
• GREEN writes the code. While the lock is open, any commit that changes, deletes or skips a locked test, or touches the config, is refused. The first commit where every locked case passes closes the lock by itself.

flowchart LR
    A[RED writes tests] --> B["shipcraft red<br/>suite must fail"]
    B --> C[lock committed: open]
    C --> D[GREEN writes code]
    D --> E{git commit}
    E -- locked test changed --> F[refused, 3 lines]
    F --> D
    E -- locked cases still red --> D
    E -- all locked cases pass --> G[lock closes in a commit right after]
    G --> H["shipcraft pr<br/>CI + review"]
    H --> I["shipcraft merge"]

The hook only runs the locked cases, so commits stay fast. shipcraft ci then repeats every check on every commit of the branch, from a clean checkout, and the full suite runs in CI, on your machine or in your GitHub checks (docs-only PRs skip it). A commit made with --no-verify or with an edited hook is caught there.

Every refusal is three lines: what happened, why it matters, and the next step. Most "Next" lines contain the exact command to run, so an agent that has never heard of Shipcraft can recover on its own.

What it guarantees, and what it doesn't

It guarantees that a branch cannot reach your main through shipcraft merge (or through required status checks on GitHub) if any commit changed, removed or skipped a test that was locked at RED, or changed the config that decides how tests run, while the lock was open.

The threat model is the honest agent that makes mistakes or takes shortcuts. It is not an adversary rewriting Shipcraft itself. Shipcraft does not sandbox anything, and a determined human with admin rights can always merge around it. When that happens, shipcraft ci on main reports the merges that skipped the gate.

It does not judge whether your tests are good. That's the RED's job, and the review layer's. Shipcraft protects the judgment once it has been made.

Quick start

What you need:

• macOS (Apple silicon or Intel) for the npm package and install.sh. On Linux, build from source (below). Windows is not supported.
• git, and a project with tests: Python with pytest, TypeScript/JavaScript with vitest or jest, Go with gotestsum (go install gotest.tools/gotestsum@latest), or any runner that writes JUnit XML (see Configuration).
• Optional, for shipcraft pr and shipcraft merge: a GitHub remote named origin and gh logged in.
• Optional, for the review: the codex, claude or gemini CLI. The review sends the diff to that agent's provider; without one, init turns the review off.

Installing is two steps, and step 2 is required: add the package (or the binary), then run init at the repository root. init sets up Shipcraft and wires its hooks; until it runs, Shipcraft checks none of your commits. From the npm package, step 2 is pnpm exec shipcraft init or npx -- shipcraft init.

# A Node/TypeScript project (macOS): pin Shipcraft in the project, like any dev dependency
pnpm add -D shipcraft          # step 1, or: npm i -D shipcraft
pnpm exec shipcraft init       # step 2 (required), or: npx -- shipcraft init; the hooks call this project's install

# macOS (Apple silicon or Intel), any language: the latest release, checksum-verified, into ~/.local/bin
gh repo clone victorassisalves/shipcraft && bash shipcraft/install.sh

# or build from source (macOS or Linux, Rust toolchain)
git clone https://github.com/victorassisalves/shipcraft && cd shipcraft
cargo install --path nucleo

install.sh --version <v> pins a version and install.sh --uninstall removes the binary. It needs gh logged in. With the npm install, shipcraft is not on your PATH: run it through the project (pnpm exec shipcraft … or npx -- shipcraft …).

With install.sh or cargo install, step 2 is, in your project (install.sh puts the binary in ~/.local/bin; if that folder is not on your PATH, run export PATH="$HOME/.local/bin:$PATH" first, or add it to your shell profile):

shipcraft init          # detects the language, writes .shipcraft.toml, wires the hook

Either way, init ends with the files to commit (shipcraft: ready. Commit …): .shipcraft.toml, .shipcraft/, and the hook manager's file and AGENTS.md/CLAUDE.md when it wrote them. Commit all of them, plus package.json and its lockfile with the npm install:

git add .shipcraft.toml .shipcraft    # plus every other file init listed (AGENTS.md, CLAUDE.md, the hook manager's file),
                                      # and package.json with pnpm-lock.yaml or package-lock.json
git commit -m "add shipcraft"

Git does not commit hooks: every other clone installs the dependencies and runs init once (or the hook manager's install). Until then nothing runs at its commits, and CI is the backstop.

A cycle (with the npm install, run each shipcraft command through the project's install, from the repository root: pnpm exec shipcraft red or npx -- shipcraft red; git stays as is):

# work on a branch: `shipcraft pr` refuses the default branch
git switch -c invoice-rounding

# RED: write the failing tests, then lock them
shipcraft red
git add -A && git commit -m "red: invoice rounding"

# GREEN: write the code; commit as often as you like
git commit -am "fix rounding"      # the lock closes on the first fully green commit

# open the PR (needs a GitHub remote named origin and gh logged in): CI runs, then the review
# when an agent is configured (init turns review off if it finds no codex, claude or gemini),
# and their statuses are published
shipcraft pr --title "Invoice rounding"

# merge only when CI is green and the review approved (or is off)
shipcraft merge 42

shipcraft status shows the open locks and how protected the repository really is.

Without a GitHub remote, the cycle still works up to the commits: red, the hook and shipcraft ci --base <ref> run locally; pr and merge need GitHub.

See it refuse

After the RED commit, before you write the code, try the four shortcuts an agent takes. Make each change, run git commit -am "fix rounding", read the refusal, and undo it with git restore --staged --worktree <file> before the next one:

1. change the expected value in a locked test: shipcraft refused: locked test <file> changed after RED;
2. skip the test (t.Skip, pytest.mark.skip, it.skip): the same refusal;
3. delete the test file: shipcraft refused: locked file <file> was deleted;
4. change .shipcraft.toml: shipcraft refused: .shipcraft.toml changed … with lock … open.

Each refusal goes on with a Why: line and a Next: line that says how to recover.

Removing Shipcraft

1. Remove the binary: pnpm remove shipcraft (or npm rm shipcraft) with the npm install, install.sh --uninstall with the script, or cargo uninstall shipcraft from source.
2. Delete .shipcraft.toml and the .shipcraft/ folder.
3. Remove the block between <!-- shipcraft --> and <!-- /shipcraft --> in AGENTS.md and in CLAUDE.md (delete either file if init created it and nothing else is in it).
4. Remove the hooks: with plain git hooks, commit-msg and post-commit in the hooks folder (git rev-parse --git-path hooks prints it), the two files marked # shipcraft hook; with lefthook, husky or pre-commit, the shipcraft lines in the hook manager's file.

Upgrading: with the npm install, pnpm add -DE shipcraft@latest (or npm i -D --save-exact shipcraft@latest) and commit package.json with the lockfile; every clone then gets the same version. To go back, pin the previous one exactly: pnpm add -DE shipcraft@<version>, and commit both files again. With install.sh, run it again, or install.sh --version <version>. A version older than the config's shipcraft_min refuses to run: going back past it also means lowering shipcraft_min in .shipcraft.toml, with no lock open. Release notes say which version to go back to when a release changes something you may want to undo.

Commands

| Command | What it does | |---|---| | shipcraft init [--lang python\|node\|go] [--github] | Detects the language and test runner, writes .shipcraft.toml, and wires the hook into your hook manager. | | shipcraft --version | Prints the version and build commit. | | shipcraft red | Runs the suite, requires assertion failures, and writes the lock in .shipcraft/locks/<id>.json. | | shipcraft check [--commit-msg <file>] | Called by the hook. Judges the commit being made against open locks and closes a lock when its cases pass. | | shipcraft close | Closes a lock in its own commit; the post-commit hook runs it, and you can run it by hand. | | shipcraft ci --base <ref> [--pr] | Re-checks every commit in <ref>..HEAD and runs the full suite on HEAD, unless the PR is docs only or [execution] suite_checks leaves the suite to your GitHub checks. Runs locally or in GitHub Actions. | | shipcraft pr [--title <t>] [--body <file>] | Pushes the branch, opens or updates the PR, runs CI and review, and publishes the shipcraft/ci and shipcraft/review statuses. | | shipcraft review [--pr <n>] | Runs the review on the local HEAD (to catch blockers before opening the PR) or on any PR. | | shipcraft merge <pr> [--no-review "<reason>"] | Merges only when CI and review are green on the PR's head. --no-review "<reason>" is the recorded way out, and the reason shows up in the PR and in the numbers. | | shipcraft status | The open locks, and how protected the repository really is (branch protection or only shipcraft merge). |

Languages, runners and hook managers

| | Supported | |---|---| | Languages | Python, TypeScript/JavaScript, Go, and any runner that writes JUnit XML through a generic suite | | Runners | pytest, vitest, jest, go test / gotestsum, anything else via [[suite]] | | Hook managers | plain git hooks, lefthook, husky, pre-commit | | Repositories | single package or monorepo (several [[suite]] entries), TypeScript project references ([build]) | | CI | local (on your machine, no runner needed) or GitHub Actions | | Review | a separate agent session: Codex, Claude or Gemini CLI |

When init finds a hook manager, it integrates with it instead of overwriting it. Shipcraft never edits your test files, your runner config or your existing hooks.

Configuration

shipcraft init writes .shipcraft.toml. In command, {junit} is replaced by the path where the runner writes its JUnit report. A typical file:

shipcraft_min = "2.0"

[[suite]]
name = "api"
root = "packages/api"
command = "npx --no -- vitest run --reporter=junit --outputFile={junit}"
globs = ["**/*.test.ts", "vitest.config.*"]

[execution]
ci = "local"        # local | github | off
review = "local"    # local | off

[roles]
review = "codex"    # codex | claude | gemini

Two rules keep the config honest:

• The config is part of the lock. It can't change while a lock is open.
• A PR is judged by the config on the base branch, not by its own. A PR can't switch off its own review.

Adding a package with a new suite. Once the base branch has a .shipcraft.toml, a [[suite]] added in the same PR as its first tests doesn't run in CI, and shipcraft ci refuses its locked cases as missing. Bring the suite first, in a PR of its own: the package, its [[suite]] entry and one passing test. Once that's merged, rebase and write the RED. shipcraft red warns when changed test files match no suite's globs, or when the lock has cases in a suite the base doesn't have.

A layer that is switched off is labeled "off in .shipcraft.toml" on every PR, so nobody mistakes it for a check that ran.

The layers

| Layer | When it runs | What it catches | |---|---|---| | Hook | every commit | edits to locked tests or config, skipped or missing cases; closes the lock when green | | CI | every PR | everything the hook checks, on every commit, from a clean checkout, plus the full suite | | Review | every PR, after CI is green | a different agent reads the diff against the spec and the PR body, and runs its own checks. A blocker counts only if the reviewer reproduced it and it breaks a quoted spec rule or crashes. Errors, timeouts and quota failures are never an approval. | | Merge gate | shipcraft merge | refuses unless CI and review are green on the PR head |

A PR that changes only docs (a *.md at the root, or anything under docs/) still gets every commit checked, but skips the build and the suites, so specs and docs reach main in seconds. Any doubt, such as a Markdown file a suite reads as a test, runs everything.

On a public repository (or a paid GitHub plan), shipcraft init --github also marks both statuses as required, so the platform enforces the gate too.

Measured, not asserted

Shipcraft records every hook, CI and review run in .git/shipcraft/runs.jsonl: layer, result, refusal code and time. Nothing is sent anywhere. shipcraft pr puts the branch's numbers in the PR summary. shipcraft report reads the same log and gives, per lock, the times from RED to close and to merge, with daily runs, refusals and hook times (--json for scripts).

Every rule in Shipcraft has to pay for itself. It ships with the metric that justifies it and the condition that removes it. Rules that don't catch anything in practice get deleted.

Working with coding agents

Agents don't need a plugin, an MCP server or a skill. They find Shipcraft the same way they find every other project constraint:

• git commit runs the hook, and refusals say exactly what to do next;
• shipcraft init writes a short block in AGENTS.md that tells agents the cycle: write the failing test and stub, run shipcraft red, commit the tests with the lock, then implement. In our trials (four runs per arm, a small sample) the earlier block took agents from 0/4 to 4/4 running shipcraft red before the first commit.
• Since 2.17, the block also asks for the RED and the GREEN in separate sessions (a subagent per phase in Claude Code, a new codex exec per phase in Codex), and init adds an import of it to CLAUDE.md when Claude Code is installed or the file exists, since Claude Code reads CLAUDE.md. init upgrades a block it wrote earlier and keeps one you edited; while AGENTS.md holds an edited or older block, CLAUDE.md is left alone and init says so.
• shipcraft --version and shipcraft --help say when the repository has no .shipcraft.toml yet: one line names init as the required next step, with the one command to run (pnpm exec shipcraft init, npx -- shipcraft init or shipcraft init, by how Shipcraft is installed).

Give the RED and the GREEN to different sessions. They can be the same model; separate context is what matters.

Status and limits

Shipcraft 2 is a rewrite of an earlier internal version, built and measured in public from here on.

• Binaries: prebuilt for macOS (Apple silicon and Intel) through install.sh; on Linux, build from source with cargo install --path nucleo.
• Windows: not supported.
• Fork PRs are refused by the local review. The good-faith model doesn't hold for code from outside.
• Not yet: a hosted review layer (review = "github"), and other CI providers.

Development

cargo build --release --manifest-path nucleo/Cargo.toml -j 4
bash acceptance/run.sh nucleo/target/release/shipcraft all   # the full acceptance battery

Shipcraft is developed with Shipcraft: specs in docs/, RED and GREEN by separate agents, and every change gated by the battery and a review.

Contributing

Pull requests from outside the project are not accepted for now; bug reports and field reports are welcome as issues. Tried it? A field report helps most: where you got stuck in this README, how long the hook took, and any refusal that looked wrong, with its three lines. See CONTRIBUTING.md, and SECURITY.md for vulnerabilities.

License

Apache License 2.0