cliguard

Snapshot testing for CLI contracts.

The problem

REST and GraphQL APIs have contract testing (Pact, Specmatic, oasdiff) baked into every serious CI pipeline. CLIs don't. So a CLI's breaking changes ship silently: a flag quietly goes from optional to required, a subcommand gets renamed, a default value changes - and the first place anyone finds out is a Slack message from whoever's automation script just started failing in production.

The solution

cliguard captures your CLI's real contract - every command, flag, default, and required argument - straight from your CLI framework's own object graph, not by parsing --help text. Commit that contract like a snapshot test. From then on, cliguard check fails your build the moment a change would break an existing caller, and passes straight through anything additive or cosmetic.

🔴 [root -> build -> option[--target]] Option "--target" was removed.
🟢 [root -> build -> option[--dry-run]] New optional option "--dry-run" was added.

The first line fails your CI. The second one doesn't - --dry-run is new and optional, so nothing that already calls your CLI can break because of it.

Quick start

npm install --save-dev cliguard

Point cliguard straight at your existing CLI's entry file - most real CLIs work unmodified, since cliguard automatically captures the framework instance they build at load time even if they never export it (see "Entry files that build the CLI lazily" below). Exporting the instance is still the cleanest way to adopt cliguard where you can, since it never risks running any of your CLI's real logic:

// bin/cli.js
const { Command } = require("commander");

const program = new Command();
program.command("build").requiredOption("-t, --target <target>", "build target");
// ...

module.exports = { program }; // <- cliguard reads this, never runs it

ESM entry files work the same way - export default program (or a named export) instead of module.exports. cliguard loads your entry through a real dynamic import(), so this works even for a target CLI with a top-level await.

Built with CAC instead? Export the CAC instance the same way (module.exports = { cli } / export default cli) and pass --adapter cac:

// bin/cli.js
const { cac } = require("cac");

const cli = cac("mycli");
cli.command("build <entry>", "build target").option("-t, --target <target>", "build target");

module.exports = { cli };

npx cliguard init ./bin/cli.js --adapter cac

cac itself is an optional dependency of cliguard - only installed if you actually use --adapter cac.

Built with Yargs instead? Export the instance the same way and pass --adapter yargs:

// bin/cli.js
const yargs = require("yargs/yargs");

const cli = yargs([])
  .command("build <entry>", "build the project", (y) =>
    y.option("target", { alias: "t", describe: "build target", type: "string" }).demandOption("target"),
  );

module.exports = { cli };

npx cliguard init ./bin/cli.js --adapter yargs

Like cac, yargs itself is an optional dependency of cliguard - only installed if you actually use --adapter yargs.

Built with Python's Click instead of a JS framework? Point cliguard straight at the .py file that defines your root command or group - no export needed, since Click's own @click.command()/@click.group() decorators already bind it as a module-level name:

# cli.py
import click

@click.group()
def cli():
    pass

@cli.command()
@click.option("--target", "-t", required=True, help="build target")
def build(target):
    ...

npx cliguard init ./cli.py --adapter click

Unlike the JS adapters, click needs a real Python interpreter: cliguard shells out to python3 (falling back to python) with click installed in that same environment - there's no in-process way to introspect a Python object from Node. pip install click in whichever Python cliguard's shell can already reach is all that's required; nothing npm-installable covers this one.

Built with oclif instead? Point cliguard at the plugin's own root directory (wherever its package.json lives) and pass --adapter oclif:

npx cliguard init ./ --adapter oclif

Unlike every other adapter, this one doesn't need to load your CLI's code at all - oclif already ships a first-class oclif manifest command that dumps a complete, structured description of every command's flags and arguments as oclif.manifest.json. cliguard reads that file directly if your project already has one (some oclif projects commit it, or produce it as part of their own build), or runs oclif manifest itself and cleans up afterward if it doesn't. Either way, oclif needs to be a devDependency of the target project - the same one your own npm run prepack (or similar) would already need.

Entry files that build the CLI lazily

Not every real CLI exports its instance - plenty build it inside a function that only runs when something actually calls it, or just never had a reason to export it. Pointing cliguard straight at a file like that would fail with "no instance found" under the rule above alone.

So when the direct export lookup finds nothing, cliguard tries one more thing automatically, no flag needed: it patches the exact copy of commander/cac/yargs your entry file will itself require(), so any new Command() (or CAC's cac(), or a yargs(...) call) anywhere in your file's own top-level code is captured - even though nothing was ever exported. This covers most real CLIs, since even ones that never bother exporting still build (and often .parse()) at the top of their own file as a matter of course.

It can't reach an instance built strictly inside a function that's only invoked later, never automatically at load time (a main() some other file calls, not the file cliguard is pointed at) - there's no safe, generic way for cliguard to know which function to call or with what arguments. For that shape, write a small wrapper file that reaches into the target's own internals to get (or construct) the instance, and point cliguard at the wrapper instead of the original entry file. The exact shape of that wrapper is inherently project-specific - it's standing in for whatever that project's own entry point would otherwise do - but the command stays the same either way: cliguard init ./your-wrapper.mjs.

Either way, if the target's top-level code has its own real side effects when loaded - a .parse() call that matches a real command and runs it, a network request, spawning a process - running cliguard against it (directly or through a wrapper) triggers those too, exactly as node ./bin/cli.js would. cliguard neutralizes one specific danger this creates (a target calling process.exit() can't kill cliguard's own process or override its exit code), but doesn't sandbox anything else - see SECURITY.md.

Then:

# Capture the current contract - commit .cliguard/contract.json
npx cliguard init ./bin/cli.js

# Same, plus scaffold .github/workflows/cliguard.yml so CI is wired up too
npx cliguard init ./bin/cli.js --with-ci

# In CI: fail the build on any breaking change
npx cliguard check ./bin/cli.js

# You changed something on purpose? Accept the new contract.
npx cliguard update ./bin/cli.js

cliguard check exits 1 if it finds even one BREAKING change, and 0 otherwise - safe to drop straight into any CI pipeline.

Pass --json to check for a machine-readable result instead of the emoji lines above - useful for a bot that comments on the PR, a dashboard, or any other script consuming the result instead of a human reading it:

npx cliguard check ./bin/cli.js --json

{
  "ok": false,
  "changes": [
    { "type": "BREAKING", "path": "root -> build -> option[--target]", "message": "Option \"--target\" was removed." }
  ],
  "summary": { "breaking": 1, "acknowledgedBreaking": 0, "additive": 0, "patch": 0 },
  "suggestedBump": "major"
}

suggestedBump is the semver bump this diff implies ("major", "minor", "patch", or null if nothing changed) - a direct read of the same BREAKING/ADDITIVE/PATCH classification the emoji output already uses, so a release script never has to re-derive it.

Marking something unstable right where it's declared

cliguard.config.js is a separate file - useful for a blanket rule, but one more place to keep in sync as flags get renamed or removed. For a single command/option/argument that isn't stable yet, mark it in its own description instead:

program.option("--fast", "skip checks [unstable]");

Any BREAKING change to a path whose own description contains [unstable] reports as PATCH instead - the marker travels with the code, so it can't silently point at a flag that no longer exists the way an external ignore list can.

Project-wide policy: ignoring or downgrading a whole class of change

accept/deprecate handle one breaking change at a time. For a rule that applies to a whole class of changes - "alias changes are never breaking for us," "ignore everything under the debug subcommand" - write cliguard.config.js (or .cjs) instead:

// cliguard.config.js
module.exports = {
  // Dropped from the report entirely - never shown, never fails the build.
  ignore: ["root -> debug -> *"],

  // Reclassified, not dropped - still visible, just not BREAKING anymore.
  severityOverrides: [{ pattern: /alias/, severity: "PATCH" }],
};

pattern in either field is a RegExp or a glob string (* matches any run of characters) matched against a change's path (the same string check's own output shows, e.g. "root -> build -> option[--target]"). Applied before accept/deprecate ever run, so a change this config already downgraded has nothing left for either of those to act on. No cliguard.config.js present is a no-op - every project behaves exactly as it always has.

Monorepos with multiple CLI entry points

A monorepo shipping more than one CLI - a packages/* layout where two or three packages each have their own bin - doesn't need N separate .cliguard/ directories and N hand-written CI steps. Declare every entry point once, as targets in cliguard.config.js:

// cliguard.config.js
module.exports = {
  targets: [
    { name: "cli-a", entry: "packages/cli-a/bin/index.js", adapter: "commander" },
    { name: "cli-b", entry: "packages/cli-b/bin/index.js", adapter: "yargs" },
  ],
};

init/check/update/accept all pick it up - omit the entry argument to run against every configured target, or pass a target's name in its place to run just that one:

npx cliguard init            # initializes every target: .cliguard/cli-a/contract.json, .cliguard/cli-b/contract.json, ...
npx cliguard check           # checks every target - exits 1 if ANY of them has an unacknowledged breaking change
npx cliguard check cli-a     # just one, by name
npx cliguard accept cli-a "root -> build -> option[--target]" --reason "..."  # accept is always single-target - see below

Each target's contract, accepted breaks, and deprecations live under their own .cliguard/<name>/ directory, so two targets never collide on disk. Running more than one target at once prints a == <name> == banner between them; checking a single named target (or the classic single-CLI flow below) prints exactly as it always has, with no banner at all.

This is additive, not a replacement for the single-CLI flow that's still most of cliguard's actual usage. An explicit file-path entry (npx cliguard check ./bin/cli.js) keeps working exactly as it does today, cliguard.config.js or not - targets is only ever consulted when the entry argument is omitted, or when it exactly matches a configured target's name.

accept is the one exception to "omit entry to run every target": accepting a specific breaking change is inherently a one-target operation (a changePath on one CLI's contract has nothing to do with another CLI's), so its entry argument stays required - a literal path or a target's name, same as check.

Comparing against a git ref instead of a local file

check normally diffs against .cliguard/contract.json on disk, but a CI runner checking out a PR branch often doesn't have a freshly-updated one - --against <ref> reads the contract straight out of git instead, no local file required:

npx cliguard check ./bin/cli.js --against origin/main

Works with any ref git show understands - a branch, a tag, a commit sha. Combine with --json the same way as the file-based path.

Accepting an intentional breaking change

Sometimes a BREAKING change is exactly what you meant to ship - a flag genuinely needed to go away in a major version. Running cliguard update after a real, intentional break re-baselines the entire contract silently; it doesn't leave a record of what changed or why. cliguard accept does:

npx cliguard accept ./bin/cli.js "root -> build -> option[--target]" --reason "removed in v2.0, replaced by --targets"

This only works against a change check would currently report as BREAKING - it reads the exact path from your own check output (text or --json), so there's nothing to guess. It writes .cliguard/accepted-breaks.json (commit this file); from then on, check still shows that change - now as a 🟣 acknowledged line with the reason attached - but stops counting it toward the BREAKING total that fails your build. Any other, un-accepted breaking change still fails CI as normal. Once you're done, cliguard update still re-baselines the contract to match reality, same as always.

Deprecating something ahead of its removal

accept forgives a break that already happened. deprecate is the other half - announce a removal before it happens, so when it eventually does, it's a PATCH instead of a BREAKING change:

# The option still exists today - deprecate marks it for a future removal
npx cliguard deprecate ./bin/cli.js "root -> build -> option[--target]" \
  --remove-by 2.0.0 --reason "replaced by --targets"

This only works against a path that currently exists (it reads the same path shape check/accept use) - it writes .cliguard/deprecations.json (commit this file). From then on, whenever that command/option/argument actually gets removed, check reports it as PATCH, with the deprecation's reason and --remove-by folded into the message, instead of failing the build. Removing anything that was never deprecated first still fails exactly as before - deprecation has to be announced ahead of the break, not applied retroactively.

--remove-by is informational only (a version or a date, whichever fits your release process) - cliguard never checks it against the clock or your package.json version, it's just carried through into the message so whoever's reading a changelog or a PR comment knows the plan.

Comparing two contracts directly

cliguard diff <old.json> <new.json> runs the same comparison as check, but reads both sides straight off disk instead of running any CLI - useful for comparing two tags' committed contracts (git show v1.0.0:.cliguard/contract.json > old.json), or reviewing a contract change in a PR without a working copy of the target CLI at all:

npx cliguard diff old-contract.json new-contract.json --json

It respects .cliguard/accepted-breaks.json the same way check does, and exits 1 on an un-acknowledged BREAKING change.

Previewing a contract without committing it

cliguard preview <entry> runs the same extraction init would, but prints the contract to stdout instead of writing .cliguard/contract.json - useful for sanity-checking what a new adapter or a lazily-built target CLI actually captures before you commit to it as the baseline:

npx cliguard preview ./bin/cli.js --adapter yargs

Generating CLI reference docs that can't drift

cliguard docs <entry> walks the same extracted contract check already fails CI over, and renders it as Markdown - one section per command, a table of its arguments, a table of its options (flag, default, description, whether it's required):

npx cliguard docs ./bin/cli.js > CLI.md

Commit CLI.md and add --check to catch it going stale the same way check catches the contract going stale - it exits 1 the moment the generated docs and the committed file disagree, instead of drifting silently the way hand-written or copy-pasted-from---help docs do:

npx cliguard docs ./bin/cli.js --check CLI.md

Checking an adapter's real limitations, or sanity-checking one against your CLI

Every adapter has a couple of real, framework-shape gaps (see "Supported frameworks" below) - cliguard doctor surfaces them directly instead of leaving them to a code comment only a maintainer would read:

npx cliguard doctor

Pass an entry file to also run a real extraction against it and get a quick structural summary (or the real failure, if extraction doesn't work) instead of a full contract dump:

npx cliguard doctor ./bin/cli.js --adapter yargs

Catching a breaking change before it reaches CI

cliguard install-hook <entry> installs a git hook (pre-push by default) that runs cliguard check automatically, so a breaking change is caught locally instead of waiting for CI to say so:

npx cliguard install-hook ./bin/cli.js
# or, to gate every commit instead of every push:
npx cliguard install-hook ./bin/cli.js --hook pre-commit

Never overwrites a hook that's already there - if you're already using husky or a similar tool, add the same npx cliguard check ... line to your existing hook instead.

--strict: catching changes the default rules can't see

The default rules match commands/options/arguments by name, so a change that keeps every name the same is invisible to them - even when it can still break a caller. --strict adds rules for exactly that gap. Today, one: a pure reorder of a command's positional arguments.

// before
program.command("copy").argument("<src>").argument("<dest>");
// after - same two arguments, swapped order
program.command("copy").argument("<dest>").argument("<src>");

The default rules see no change at all here (<src> still exists, <dest> still exists). But cli copy a.txt b.txt now copies b.txt over a.txt, not the reverse - a real, silent break for anyone calling it positionally:

npx cliguard check ./bin/cli.js --strict

Off by default so it never changes behavior for an existing CI config - opt in per project.

A --strict-only change is real BREAKING output, so cliguard accept needs the same flag to find it - cliguard accept ./bin/cli.js "root -> copy" --strict --reason "..." - without it, accept compares in default (non-strict) mode and won't see the change at all.

Architecture

The interesting part isn't the diff - it's getting the CLI's true shape out of code that may never export it. See "Entry files that build the CLI lazily" below for how the adapter reaches a Commander/CAC/Yargs instance that's never assigned to anything exported.

Programmatic API

Everything above is the CLI. The same extraction and diff logic is also available as a library, for a custom build script, monorepo tool, or bot that wants to embed a contract check without spawning npx cliguard as a subprocess:

const { extractContract, compareContracts, ChangeType } = require("cliguard");

const oldContract = await extractContract("./bin/cli.js"); // or read one off disk yourself
const newContract = await extractContract("./bin/cli.js");
const diff = compareContracts(oldContract, newContract, { strict: true });

const breaking = diff.filter((change) => change.type === ChangeType.BREAKING);

listAdapters() returns every name extractContract's second argument accepts. DiffEngine, every adapter class (CommanderAdapter/CacAdapter/YargsAdapter/ClickAdapter/CobraAdapter/OclifAdapter), and the toJUnitXml/toGitLabCodeQuality/toRdjsonl formatters are all exported too, for anything more custom than the two convenience functions cover.

How changes get classified

| | Removed | Added | Required flipped | Value type / default changed | |---|---|---|---|---| | Command | 🔴 BREAKING | 🟢 ADDITIVE | - | - | | Option / argument | 🔴 BREAKING | 🟢 ADDITIVE (optional) / 🔴 BREAKING (required) | 🔴 optional→required · 🟡 required→optional | 🔴 BREAKING | | Alias | 🔴 BREAKING | 🟡 PATCH | - | - | | Description | - | - | - | 🟡 PATCH |

Every rule cliguard enforces - including how environment variable bindings and each escape hatch (accept/deprecate/[unstable]) factor in - is documented in full in RULES.md, the same spirit as oasdiff documenting its own ~755 OpenAPI checks separately from its source. The underlying implementation and its own tests are src/core/diff.engine.ts and src/__tests__/diff-engine.test.ts, if RULES.md and the code ever disagree.

Reports for non-GitHub CI

The bundled GitHub Action is the recommended path on GitHub, but check/diff can also emit two other formats directly, no Action or bespoke reporter needed:

# JUnit XML - understood natively by Jenkins, CircleCI, Azure DevOps, and GitLab's own JUnit widget
npx cliguard check ./bin/cli.js --format junit > cliguard-report.xml

# GitLab Code Quality JSON - surfaced as inline annotations on a GitLab merge request
npx cliguard check ./bin/cli.js --format gitlab-codequality > gl-code-quality-report.json

A third format, --format rdjsonl, emits reviewdog's own Diagnostic Format instead of a report cliguard renders itself - hand it off to whichever platform reviewdog already has a reporter for:

npx cliguard check ./bin/cli.js --format rdjsonl | reviewdog -f=rdjsonl -reporter=github-pr-review

Same exit code either way - 1 on an unacknowledged BREAKING change, 0 otherwise - so any of the three drops straight into a CI job that already fails the build on a non-zero exit.

Webhook reporter for SaaS integrations

--webhook <url> (or a CLIGUARD_WEBHOOK_URL environment variable) POSTs the same diff check just computed as JSON to a URL of your choice - the first building block toward the hosted dashboard/Slack-alert roadmap below, v1 scoped to just the POST itself:

npx cliguard check ./bin/cli.js --webhook https://example.com/cliguard-hook

{
  "entry": "./bin/cli.js",
  "repo": "[email protected]:you/your-cli.git",
  "commit": "a1b2c3d4e5f6...",
  "changes": [
    { "type": "BREAKING", "path": "root -> option[--target]", "message": "Option \"--target\" was removed." }
  ]
}

repo/commit are best-effort (git config --get remote.origin.url / git rev-parse HEAD) - null outside a git repository. A webhook that's unreachable or slow to respond never fails check or changes its exit code - it just prints a warning and moves on.

--open-diff: opening a real difference in your editor

Inspired by how ApprovalTests' reporters open an external diff tool the moment a test fails - --open-diff does the same the moment check finds a real difference: it writes the expected and actual contracts to temp files and, if VS Code's own code CLI is on PATH, opens its built-in two-pane diff view on them.

npx cliguard check ./bin/cli.js --open-diff

No supported editor found on PATH? cliguard never fails hard over it - it just prints both files' paths instead, so you can open them with whatever you have. Either way, --open-diff never changes check's own exit code, and the editor (when one opens) is launched detached from cliguard's own process, so check still exits immediately rather than waiting on you to close it.

CI integration

cliguard init --with-ci scaffolds the workflow below for you - git add .github/workflows/cliguard.yml and you're done. Prefer to see it first, or wire it up by hand? Read on.

The bundled GitHub Action (Bryandero98/cliguard@v0) is the recommended way to run this in CI: on top of the same exit-code gate as npx cliguard check, it posts the diff as a PR comment - updated in place on every push, not a new one each time - so a reviewer sees exactly what changed without opening the CI log:

# .github/workflows/cliguard.yml
name: CLI contract
on: [pull_request]
permissions:
  pull-requests: write # needed for the PR comment
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22.x }
      - run: npm ci
      - uses: Bryandero98/cliguard@v0
        with:
          entry: ./bin/cli.js
          # adapter: yargs        # default: commander
          # comment-on-pr: false  # default: true
          # strict: true          # default: false

Set comment-on-pr: false to keep the exit-code gate without the comment, or use the raw CLI directly for a non-GitHub CI provider:

- run: npx cliguard check ./bin/cli.js

Supported frameworks

Commander.js (default), CAC (--adapter cac), Yargs (--adapter yargs), Python's Click (--adapter click), and oclif (--adapter oclif, via its own oclif manifest command) today, all requiring zero changes to the target CLI itself.

Go's Cobra (--adapter cobra) also exists, but as a proof of concept only (see issue #9) - unlike every adapter above, it needs the target CLI's own author to wire in a hidden dump subcommand (there's no published cliguard-go package yet to do that for them). See examples/cobra-dump/ for the full example and src/adapters/cobra.adapter.ts for exactly what it does and doesn't cover. Rust's Clap (--adapter clap) would follow the same pattern (clap::Command is introspectable before parsing, same as clap_complete/clap_mangen already rely on) but isn't built yet - see issue #10.

The core (types + diff engine) is 100% framework-agnostic by design: every framework-specific detail lives behind the CliAdapter interface in src/adapters/, so adding a new adapter never touches the diffing logic.

A couple of OptionContract/ArgumentContract fields carry real, framework-specific limitations rather than a mapping gap - see src/adapters/cac.adapter.ts's own doc comment for exactly which ones and why (CAC has no declarative "this flag must be passed" concept, and no per-argument description). Yargs's own real limitation is the opposite kind - see src/adapters/yargs.adapter.ts's doc comment for why each command's options are read from a fresh, isolated instance rather than the shared one the target CLI actually built.

The Commander/CAC/Yargs adapters load the target CLI's entry file into the Node process (import()/require()) and read its object graph directly - that only works for other Node frameworks. Click and Cobra instead run a small extractor as a subprocess and parse JSON off its stdout, since a Python object graph or a compiled Go binary can't be require()'d into Node the way a JS CLI can.

Security

Extracting a contract runs the target entry file's own top-level code, the same as node ./bin/cli.js would - see SECURITY.md for what that means in practice.

Roadmap

The CLI and core diffing engine are, and will stay, free and open-source. Planned next: a hosted add-on for teams that want more than a CI exit code - a dashboard with the history of contract changes across releases, and Slack/webhook alerts the moment a breaking change lands. See issue: Webhook reporter for SaaS integration for the first building block.

Support this project

cliguard is free and will stay free. If it's saving you from a broken release, a small tip helps keep it going:

• Ko-fi: ko-fi.com/bryandero98
• USDT (TRC20): TEG4Kk2qXYMQ4mHNd7dPhSPRyT14CGr2or — double-check the network is set to TRC20 before sending; a transfer on the wrong network can't be recovered.

Contributing

See CONTRIBUTING.md.

License

MIT