peirad

πεῖρα — Greek for the trial that puts a thing to the test; the root of empirical.

Know the moment your agent integration stops holding.

You wired a tool into your coding agent months ago — a hook, a settings key, a transcript reader, a script that passes the right flags. Since then the harness shipped a dozen updates. Your integration might still work, or it might have quietly stopped the day a flag was renamed — and nothing told you, because the failure is silent.

peirad contract-tests your integration against the harness you actually have installed, right now, and prints a dated verdict that names the version it checked.

$ npx peirad --manifest peirad.json

my integration — harness claude 2.1.223 · 2026-08-15
  ok    command-exists(claude): claude is on PATH
  ok    version: 2.1.223
  ok    flag-accepted(-p,--allowedTools): all flags present in --help
  ok    config-key(settings.json): keys present: hooks.PreToolUse
  DEGR  transcript-field(projects/**/*.jsonl): schema drift — missing: message
  1 degraded — drift detected

It needs Node ≥ 18.17 and must run on the machine where the harness is installed (it checks the live install). It is a checker, not a fixer — it tells you what drifted; changing it is yours.

Quick start

Write a small peirad.json describing what your integration relies on, then run:

npx peirad --manifest peirad.json

Exit code is non-zero on any drift, so it drops straight into CI or a scheduled check.

What it checks

You declare probes; each runs against the live harness:

| Probe | Confirms | | ------------------ | ------------------------------------------------------------- | | command-exists | the harness binary is on PATH | | version | the installed version (stamped into the verdict) | | flag-accepted | the CLI flags your automation passes still parse | | config-key | the settings keys you rely on still exist | | hook-registered | your hook is still wired for its event | | transcript-field | the fields your tool reads from transcripts are still present | | script | a repo-provided check still passes |

A non-critical probe that drifts reports degraded; a probe marked critical reports blocked; a probe its harness profile says cannot apply reports n/a. Nothing throws — one drift never hides the next.

The script probe runs an executable from the repo the manifest lives in: exit 0 passes, exit 1 fails with the script's stdout as the finding, and exit 2 reports n/a — no verdict. Any other outcome (not executable, killed by the timeout, crashed) is read the same way: the probe fails open, so a broken probe can only report n/a, never claim your integration broke — even when marked critical. Running a manifest's scripts is running that repo's code, the same trust as its npm scripts: a manifest is only as trustworthy as the repo that ships it.

How it works

peirad is a generic engine plus a per-project manifest. The manifest is data — which probes to run, and the flags/keys/paths your integration depends on — so adding a check or a new harness is a manifest edit, not an engine change. The engine resolves the harness, runs each probe against the live install, and returns a dated verdict.

{
  "name": "my integration",
  "harness": "claude",
  "probes": [
    { "type": "command-exists", "critical": true },
    { "type": "version" },
    { "type": "flag-accepted", "flags": ["-p", "--allowedTools"] },
    {
      "type": "config-key",
      "file": "settings.json",
      "keys": ["hooks.PreToolUse"],
      "critical": true
    },
    {
      "type": "transcript-field",
      "glob": "projects/**/*.jsonl",
      "fields": ["type", "message"]
    }
  ]
}

Relative file/glob paths resolve against the manifest's directory, or pass --config-dir to point at your harness config location. Add --json for a machine-readable verdict.

Harness profiles

Agent CLIs disagree on how to be driven headless: one takes -p <prompt> --output-format json and prints a single envelope; another wants a subcommand, the prompt as a positional, --json for an event stream, and reports usage at the end of the turn. A harness profile holds that shape — how to pass a one-shot prompt, how to ask for machine-readable output, how to read the reply and the token/cost numbers back out.

Two are built in:

• claude — -p <prompt> --output-format json; reply from the envelope's result, usage from its usage block.
• codex — codex exec --skip-git-repo-check --sandbox read-only --color never <prompt> --json; reply from the final agent_message event, usage from turn.completed.

The manifest selects one with harnessProfile; when it is absent, the harness name decides ("harness": "codex" gets the codex profile) and an unknown harness falls back to the claude convention. For a CLI neither profile fits, replace the argv templates directly — promptArgs must contain one {prompt} element (a standalone argument, never spliced into a flag), outputArgs is appended after it:

{
  "harness": "my-cli",
  "harnessProfile": "codex",
  "promptArgs": ["exec", "--no-banner", "{prompt}"],
  "outputArgs": ["--json"],
  "probes": []
}

Profiles also say which probes can apply. The settings-file probes (config-key, hook-registered) assume a JSON settings layout that not every CLI family has; under a profile where they have no counterpart they report n/a with the profile named — declared inapplicable, never silently passed — and flag-accepted reads the help the profile points at (root --help, or a subcommand's exec --help for codex).

Triage — does the drift matter?

run tells you that something drifted; triage tells you whether it matters. It reads an alarm — a changelog excerpt, a CI failure, a drift report — against a rubric and prints a structured pre-assessment for the person who has to decide. The assessment is machine-written and labelled unverified by construction; nothing is edited or closed on its say-so.

npx peirad triage --alarm changelog.md --rubric changelog --manifest peirad.json

The model call goes through the harness named in your manifest, headless (no tools, default model) — the same binary the probes exercise, invoked through the manifest's harness profile. --rubric takes a markdown file, or the built-in changelog, which builds the rubric from your own manifest — "did anything I declared a dependency on change?" — listing every flag, settings key, hook and transcript field your probes rely on.

## Pre-assessment (machine, unverified)
Verdict: action
Confidence: high
Reasoning:
- declared flag --allowedTools is renamed — "The `--allowedTools` flag is now `--allowed-tools`; the old spelling is no longer accepted."
Draft resolution:
Rename --allowedTools to --allowed-tools in the launch script; retest the PreToolUse hook.
usage: in 4 / cached 1850 / out 220 tokens · model claude-sonnet-5 · cost $0.0123

Two guards keep it honest:

• Quote guard — every reasoning point must quote a line found verbatim in the alarm; points that cannot be traced are dropped and counted in a trailing dropped: N unquotable point(s) line.
• Loud failure — if the harness call fails or times out, the command exits 2 with pre-assessment unavailable: <reason> instead of guessing.

The harness's own accounting for the call — tokens in/cache/out, model, cost — ends up in a trailing usage: line, or a usage object with --format json (null when the harness reports none). Pass --usage-log <file> to also append one JSON row per call to that file, so triage runs you schedule or script can be tallied afterwards; a log that cannot be written fails the command (exit 2) rather than pass silently.

Exit 0 on any verdict — a verdict is information, not a failure. --format json emits {verdict, confidence, reasoning, draft, dropped, assessed_at, harness, harness_version, profile, usage, rubric}. A PEIRAD_HARNESS environment variable overrides the manifest's harness binary (the profile still comes from the manifest), which is handy for testing.

Precedent — has this been ruled on before?

triage asks whether drift matters; precedent asks whether it has already been decided. Give it a work-queue entry (markdown with a # title and a from: frontmatter line), a decisions ledger in the ### D-00n — title + **Scope:** style, and any directories of resolved entries:

npx peirad precedent --entry queue/014-canary-drift.md --ledger DECISIONS.md --resolved queue/resolved --json

It derives the entry's class — the title's stem (up to the first : or —) plus its from: source, with dates, versions and parentheticals normalized away so recurrences collapse — then looks for prior rulings: resolved entries of the same class (a past done: line is a paste-ready resolution) and ledger entries whose title or scope covers the class.

{
  "schema": "precedent/1",
  "matched": true,
  "class": "canary drift · nightly sweep",
  "source": "resolved",
  "id": "013-canary-drift-2026-09-03.md",
  "resolution": "re-ran the sweep twice — known clock skew; closed without changes",
  "confidence": "high"
}

Three properties keep it safe to run unattended. It is read-only — prints, never writes, never resolves anything itself. Matching is deterministic text work — no model call; the same inputs give the same answer, and every match names the prior artefact it rests on (confidence: high = a resolved sibling, medium = a ledger ruling only). And entries whose text trips a rail keyword list — credentials, guarded material (plus any words you pass with --rail-words), machine surfaces, registries, releases, outward actions — always come back matched: false with the rail named. The list is deliberately over-broad: a false "no match" costs a person a glance, a false "matched" would cost a wrong auto-resolution, so the tool fails toward the first.

Exit 0 whether or not precedent matched — the answer is information, not a failure. Exit 2 when inputs are unreadable (missing entry, ledger or --resolved directory) or the entry has no # title to derive a class from.

What it is NOT

• Not a sandbox or a security tool. It reports whether your wiring still works, including whether a security hook you rely on is still registered — it does not stop an attacker.
• Not a one-shot scanner. Run it on every harness upgrade and on a schedule; drift is continuous.
• Not magic. It checks what your manifest declares. A gap you do not declare is one it will not catch.

Reference

• Verdict statuses: pass, degraded (non-critical drift), blocked (critical drift), n/a (the probe has no counterpart under the manifest's harness profile — declared, never a pass). Exit 0 when all pass, 1 on any drift.
• --json emits the full verdict object (name, harness, profile, version, date, and per-probe results) for programmatic use.

Roadmap: ROADMAP.md · Decisions: DECISIONS.md

License

MIT