ai-diff-check

Your AI wrote the code. This runs the vibe check.

An open-source CLI that reviews only your git diff before you commit — catching what AI coding agents leave behind. Offline, deterministic, no account, no LLM required. JavaScript/TypeScript, Python, and Go.

npx ai-diff-check   # that's it — reviews your working tree against main

Why

• 66% of developers say their #1 frustration is AI code that's "almost right, but not quite" (Stack Overflow 2025)
• 96% don't fully trust AI-generated code — yet only 48% verify it before committing (Sonar 2025)
• The existing tools in this space (Sonar, CodeRabbit, Qodo) are paid and cloud-based

ai-diff-check is the free, local, diff-scoped alternative: it never scans your whole repo, only what just changed — so legacy code stays grandfathered and adoption costs nothing.

vs. the paid tools

| | ai-diff-check | CodeRabbit | SonarQube Cloud | Qodo | |---|---|---|---|---| | Price | Free, MIT | per-seat | per-LoC tiers | per-seat | | Runs | Local + CI, offline | cloud | cloud/server | cloud | | Your code leaves your machine | Never | yes | yes | yes | | LLM required | No — deterministic | yes | no | yes | | Scope | Only the diff (legacy grandfathered) | PR | whole project | PR | | Custom house rules | One JSON line | prompt config | quality profiles | prompt config | | Setup | npx ai-diff-check init | GitHub app | server + scanner | app |

Not a replacement for deep static analysis or a human reviewer — it's the fast, free gate that catches AI-agent mess before the commit, deterministically, with zero noise on a clean diff.

What it checks

| Check | Catches | Status | |-------|---------|--------| | SECRET | Hardcoded credentials in new code — AWS/GitHub/npm/Anthropic/Stripe/Google keys, private keys, connection strings | ✅ built | | TEST-FOCUS | Focused tests left in (.only/fdescribe/fit) that silently disable the rest of the suite | ✅ built | | STUB | Empty catch blocks, TODOs, "not implemented" throws, leftover console.log | ✅ built | | OVERSIZE | Monolith files / 200+ line components, with a proposed split | ✅ built | | PHANTOM-DEP | Imports of packages that don't exist or aren't installed, with a "did you mean" correction (stirpe → did you mean 'stripe'?) — and a guard for typo names that DO exist on npm (squatted lookalikes) | ✅ built | | DUPLICATE | New functions that repeat code already in your codebase | ✅ built | | DELETION | Removed validation, error handling, or tests the task didn't ask to remove | ✅ built | | UNREFERENCED | New exports with zero references — generated "just in case" | ✅ built | | NO-TEST | Source logic changed but the matching test file didn't | ✅ built | | STANDARD | Your house rules from aidiff.config.json — each pointing at the approved alternative | ✅ built | | REUSE | New components whose markup forks an existing component — same JSX skeleton, different wiring | ✅ built | | DEP-ADVISOR | New deps the platform already covers (axios, moment, uuid…) or that duplicate an existing dep's purpose | ✅ built | | CONSISTENCY | New code that breaks the repo's own inferred conventions — file naming, named-vs-default exports, import extensions | ✅ built | | DOCS-AUDIT | Opt-in LLM audit of the diff against your own CLAUDE.md/standards docs (BYO Anthropic key) | ✅ built |

Languages: all checks run on JavaScript/TypeScript. Python and Go get the core set — STUB, DELETION, NO-TEST, OVERSIZE, PHANTOM-DEP, STANDARD, and DEP-ADVISOR — routed by file extension (fmt.Println / panic("not implemented"), missing if err != nil handling, go.mod phantom imports, pkg/errors→stdlib advice, and so on). Language is detected per file, so a mixed repo is reviewed correctly in one pass.

Usage

npx ai-diff-check                  # review working tree vs the nearest trunk (or --base <ref>)
npx ai-diff-check --staged         # review only staged changes (what your commit will contain)
npx ai-diff-check --watch          # re-review continuously as you edit (Ctrl+C to stop)
npx ai-diff-check init             # starter config + pre-commit hook (husky-aware, uses --staged)
npx ai-diff-check ignore DUP-104   # permanently silence a false positive by id
npx ai-diff-check baseline         # adopt on a messy branch: snapshot today's debt, report only what's new
npx ai-diff-check --fix            # auto-apply the provably-safe fixes; everything else still reports
npx ai-diff-check --ci             # CI mode — GitHub annotations or GitLab Code Quality, auto-detected
npx ai-diff-check --badge          # shields.io endpoint JSON for a "vibe check: 94/100" README badge

Exit code 1 on any error-severity finding (and on warnings too with "strict": true) — wire it into husky/pre-commit or CI to gate merges.

Pre-commit gate

init wires the hook for you: .husky/pre-commit if your repo uses husky, plain .git/hooks/pre-commit otherwise — never overwriting what's already there. Every commit then gets the vibe check; findings with error severity block it.

CI

One step in any workflow — findings land as inline annotations on the PR diff:

jobs:
  vibe-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # full history so the diff base resolves
      - uses: prashnat-MSBC/ai-diff-check@main

Under the hood that's --ci mode: ::error file=…,line=…::message → fix workflow commands plus a JSON report for anything else to consume. This repo runs it on its own PRs — the reviewer reviews itself.

For coding agents

MCP server mode — let the agent vibe-check its own diffs before declaring a task done:

{ "mcpServers": { "ai-diff-check": { "command": "npx", "args": ["ai-diff-check", "mcp"] } } }

Three tools: vibe_check (review the diff, return the trust score + scoped fix list), vibe_fix (apply the safe fixes, report what remains), and baseline (snapshot existing debt so only new findings report). Zero extra dependencies — the MCP server is part of the same lean codebase.

--fix-prompt — the same fix list as a copy-pasteable prompt for any agent:

ai-diff-check --fix-prompt | pbcopy   # paste into your agent, let it clean up after itself

Configuration

Optional aidiff.config.json in the repo root (defaults shown):

{
  "maxFileLines": 400,       // OVERSIZE: file line limit
  "maxComponentLines": 200,  // OVERSIZE: component line limit (.tsx/.jsx)
  "strict": false,           // true → warnings also fail the run (exit 1)
  "ast": true,               // use tree-sitter when installed (see below); false forces heuristics
  "ignore": [],              // finding ids to silence, e.g. ["STUB-441"]
  "checks": {                // per-check tuning (all checks on by default)
    "no-test": false,                  // false disables a check entirely
    "oversize": { "severity": "info" } // or re-grade everything it finds
  },
  "standards": [             // your house rules — diff-scoped, so legacy is grandfathered
    {
      "ban": "fetch(",                        // substring to ban ("pattern" takes a regex instead)
      "in": "src/**",                         // optional glob scope
      "use": "apiClient from @/lib/api",      // the approved alternative, shown as the fix
      "docs": "docs/api-standards.md",        // optional pointer to your standards doc
      "severity": "error"                     // error | warn | info (default warn)
    }
  ]
}

New code that breaks a rule gets flagged with your own alternative as the fix; old violations stay quiet until touched — a ratchet, not a rewrite mandate.

Optional AST precision (tree-sitter)

By default the structural checks use fast text heuristics (column-0 declaration boundaries) — zero dependencies, works everywhere. If your project already has tree-sitter and the matching grammar installed, OVERSIZE, DUPLICATE, and UNREFERENCED automatically upgrade to real parses — exact declaration spans, precise function boundaries, and export detection that ignores lookalikes inside strings/comments:

npm i -D tree-sitter tree-sitter-typescript   # and/or tree-sitter-python, tree-sitter-go

It's resolved from your node_modules, never bundled — ai-diff-check itself stays at two dependencies. If the grammar isn't installed (or its native build fails on your Node version), the tool silently falls back to heuristics; nothing breaks. Set "ast": false to force heuristics even when tree-sitter is present.

Write your own check (plugins)

When a STANDARD one-liner isn't enough, write a plugin — the same Check interface the built-ins use, in ~20 lines. Point config at it:

{ "plugins": ["./checks/no-console-error.mjs", "@myorg/aidiff-house-checks"] }

A plugin module exports a Check (default export or named check):

export const check = {
  name: 'no-console-error',
  async run(ctx) {              // ctx = { repoRoot, baseRef, files, config }
    const findings = [];
    for (const file of ctx.files) {
      for (const [line, text] of file.added) {
        if (/\bconsole\.error\(/.test(text)) {
          findings.push({ id: `NO-CE-${file.path}:${line}`, check: 'no-console-error',
            severity: 'warn', file: file.path, line, message: 'console.error left in', fix: 'use the logger' });
        }
      }
    }
    return findings;
  },
};

Plugin findings flow through everything the built-ins do — ignore, per-check checks config, trust score, CI. A plugin that fails to load or throws is skipped with an info note and never blocks your commit. (Listing a plugin runs its code, so only add plugins you trust.) Full example: examples/plugins/no-console-error.mjs.

Every finding prints a stable id like [OVERSIZE-377] — add it to ignore to permanently silence a false positive. A malformed config never falls back silently: the run stops with exit code 2 and the reason.

docs-audit (opt-in, the one LLM check)

Everything above is deterministic and offline. docsAudit is the single opt-in exception: it sends the diff plus your own standards docs to the Anthropic API and reports violations of rules you wrote:

{ "docsAudit": { "enabled": true, "docs": ["CLAUDE.md"], "model": "claude-opus-4-8" } }

Bring your own everything: npm i -D @anthropic-ai/sdk in your project and set ANTHROPIC_API_KEY (the SDK is never a dependency of ai-diff-check). Any failure — SDK missing, no key, offline, refusal — degrades to a single info note and never blocks the commit.

License

MIT © Prashant Parmar