WIP: not published in npm yet.

Harness

Harness is a simple diff based testing framework, written in pure native powershell/bash script. Simple and flexible to install, works with any kind of project.

You write a program, hit record to save what it prints, and from then on test runs it again and yells if anything changed. No clever assertions, no DSL, no framework to learn. If your program prints stuff, you can test it.

After the npx scaffold, there's no Node dependency. Just scripts. Wipe Node off the machine and the harness keeps working.

This project was made with agentic coding in mind and the templates carry their own CLAUDE.md (generated by LLMs).

Why

Most testing frameworks make you learn the framework before you can test anything. This one only knows three things: what came out on stdout, what came out on stderr, what the exit code was. That's it. Diff the new output against the saved one. Match means pass, mismatch means fail. Done.

Works for cpp, java, python, scala, node, rust, go, a bash script you wrote last tuesday, whatever. The harness doesn't care what language you used or how you built it. If it runs and prints, it tests.

Quick start

npx create-snap-harness my-project
cd my-project
# PowerShell
.\scripts\install.ps1 -All
.\scripts\build.ps1 -All
.\scripts\record.ps1 -All
.\scripts\test.ps1 -All
# Bash
./scripts/install.sh --all
./scripts/build.sh --all
./scripts/record.sh --all
./scripts/test.sh --all

Run those four scripts in order the first time. After that you mostly just need test. Use record again whenever you intentionally changed the expected output.

Adding harness to an existing project

Already have a tree of programs and don't want a full scaffold? Drop scripts into a project root with install:

npx create-snap-harness install            # current dir, .harness/scripts/, detected flavor
npx create-snap-harness install ./repo --shell both
npx create-snap-harness install --dir scripts --shell ps1   # legacy flat layout

install only touches <target>/<scriptsDir> — your harness.toml files, __tests__/, and sources are untouched. Re-run it any time to pick up template updates.

Scripts run a recursive scan from wherever you invoke them, so every <dir>/harness.toml (or convention install/build/run/clean.<ext> script) in the tree gets picked up. Drop the scripts at the root, run .\.harness\scripts\test.ps1 -All from there, and every program below gets tested.

Want to target one subtree? Use -Recurse (PowerShell) or --recurse (Bash) — points the scan at a single directory and implies -All inside it:

.\.harness\scripts\build.ps1 -Recurse .\chapter2
.\.harness\scripts\test.ps1  -Recurse .\chapter2 -Filter default

./.harness/scripts/build.sh --recurse ./chapter2
./.harness/scripts/test.sh  --recurse ./chapter2 --filter default

Hidden dirs (.git, .harness, …), node_modules, __tests__, and common build-output dirs (dist, build, out, bin, obj, target) are skipped automatically.

Flags

npx create-snap-harness [<name>] [options]
  --shell <ps1|sh|both>           shell flavor (default: detected from OS)
  --samples <list>                comma-separated subset of: cpp,java,scala,python,node,none
  --no-git                        skip git init
  --no-commit                     skip initial commit

Skip any of these and you'll get asked. Pass --samples none if you don't want the hello-world examples cluttering things up.

What gets scaffolded

<proj>/
├── scripts/                       chosen shell flavor(s)
│   ├── install.{ps1,sh}           run install step per program
│   ├── build.{ps1,sh}             run build step
│   ├── test.{ps1,sh}              run cases, diff stdout/stderr/exit
│   ├── record.{ps1,sh}            capture current output as golden
│   ├── clean.{ps1,sh}             run clean; --refs nukes expect files
│   └── lib/{toml,common}.{ps1,sh}
├── samples/<lang>-hello/          example programs (delete or replace)
├── README.md
├── CLAUDE.md                      LLM authoring guide
├── .gitignore
└── .github/workflows/test.yml     only when --shell=both

The samples are there so you can see a working setup before you wire up your own programs. Delete them when you don't need them anymore.

Program manifest

Each program gets a harness.toml. Only run is required, the rest is optional.

name    = "hello"
install = "npm ci"            # optional
build   = "tsc -p ."          # optional
run     = "node dist/main.js" # required
clean   = "rimraf dist"       # optional

Hate config files? Skip the toml entirely and drop install.<ext>, build.<ext>, run.<ext>, or clean.<ext> scripts straight into the program folder. The runner finds them either way. Pick whichever feels less annoying for the program you're writing.

Test cases

Each case is a folder under <program>/__tests__/<name>/:

| File | Role | Written by record | |---|---|---| | in.txt | piped to stdin | never | | args.txt | extra CLI args appended to run cmd | never | | expect.txt | golden stdout (required for case to run) | always | | expect.err.txt | golden stderr | only when stderr non-empty | | exit.txt | expected exit code | only when ≠ 0 |

The flow: write in.txt and args.txt yourself, or skip them if your program doesn't need any input. Run record once you trust the output. Then test diffs every future run against what got saved.

expect.err.txt and exit.txt only get created when they actually matter, so your test folders stay clean and you can tell at a glance which cases care about stderr or exit codes.

When a diff fails, test prints both sides next to each other so you can see what changed. If the change was intentional, run record again. If it wasn't, you have a bug.

Development

git clone https://github.com/abh80/harness.git
npm install
npm run build
npm test
node dist/cli.js <args>

Tests live in tests/ (vitest). Templates in templates/ ship verbatim, the build never touches them. If you change a template, the change goes straight into whatever the next scaffold produces.

License

Creative Commons License