Nitpick

Nitpick is an agent-agnostic, BYOK QA agent that acts as an independent adversarial tester for code produced by AI coding agents (Claude Code, Cursor, Codex, Aider, …) or humans. It plans and executes tests, then emits a machine-readable Verdict (QVP) that any coding agent can consume in a fix→retest loop. Findings are called nits.

Install

npm install -g nitpick-qa   # provides the `nit` CLI

Requires Node ≥ 20. Docker is recommended (sandboxed test execution); without it Nitpick falls back to a restricted subprocess and marks verdict confidence as degraded.

Quick start

cd your-project
nit init        # detect stack, scaffold .nit/
nit plan        # see what Nitpick intends to run
nit run         # execute + emit verdict (exit code mirrors it)
nit verdict     # re-print the last verdict (--json for QVP)

Exit codes: 0 pass / pass-with-warnings, 1 fail, 2 inconclusive.

CLI

nit init                      # detect stack, scaffold .nit/
nit plan [--diff <base>]      # produce/print test plan
nit run [--only dims] [--diff <base>] [--smoke] [--json]
nit verdict [--json|--md]     # print last verdict
nit repro <nit-id>            # re-run a finding's exact repro
nit corpus [list|run|export]  # regression bug corpus
nit baseline update perf      # accept new perf baselines
nit mcp                       # start MCP server (stdio)
nit ci                        # CI-optimized: run + GitHub annotations
nit config                    # print effective config + hash

MCP

Register nit mcp as a stdio MCP server to give any agent the tools qa_plan, qa_run, qa_verdict, qa_repro, qa_corpus_search, and qa_await_fix (L3 loop handshake), plus nitpick://verdict/latest and nitpick://plan/latest resources. Example (Claude Code):

claude mcp add nitpick -- nit mcp

BYOK models

Configure in .nit/config.yaml — any Anthropic or OpenAI-compatible endpoint (OpenRouter, Ollama, vLLM, LM Studio). Keys are read from env vars only and never written to disk or verdicts. Without a configured model, adjudication falls back to deterministic heuristics.

models:
  judge: { provider: anthropic, model: claude-sonnet-5, api_key_env: ANTHROPIC_API_KEY }
  # worker: { provider: openai-compatible, base_url: http://localhost:11434/v1, model: qwen2.5-coder }
budget: { max_tokens_per_run: 500000, max_usd_per_run: 3.0 }

Autonomy levels

| Level | Behavior | |---|---| | L0 | Observe: runs and reports, never fails a build | | L1 | Plan-approval: interactive sign-off before execution | | L2 | Autonomous (default): plans, executes, gates | | L3 | Loop: enables the qa_await_fix fix→retest handshake |

Status / roadmap

This is M0 — the walking skeleton: stack ingest, existing jest/vitest/pytest suites executed in a sandbox, QVP verdicts with exit codes, CLI + MCP frontends, BYOK router (Anthropic + OpenAI-compatible), autonomy levels, corpus replay plumbing.

Coming next: M1 contract extraction + black-box test generation + bug corpus distillation; M2 integration/system/security layers + risk graph; M3 performance baselines, mutation spot-checks, usability audits.

License

Apache-2.0 — see LICENSE. The "Nitpick" name and logo are trademarks of the project owner; see TRADEMARK.md.