Caliper

Code review for the agentic coding era.

AI agents write code fast — but they drift. They skip conventions, use wrong patterns, and repeat the same mistakes turn after turn. One wrong choice in turn 3 becomes the template for turns 4 through 30.

Caliper keeps agents on the rails with three feedback loops, from tightest to broadest:

1. After every agent turn — your CLAUDE.md rules, compiled into convention checks that run in sub-second with no AI calls
2. Before every commit — lightweight AI review of staged changes for logic, security, and design issues
3. Before every merge — deep AI review posted as inline GitHub comments

Each layer narrows what the next needs to evaluate. The agent writes code, Caliper catches drift, the agent fixes it — no human in the loop until the code already meets your standards.

Three Ways to Use Caliper

1. Convention enforcement (Claude Code stop hook)

caliper refresh reads your CLAUDE.md files and compiles every mechanically-checkable rule into a convention check — grep patterns, AST analysis, file scans. This is a one-time step that requires ANTHROPIC_API_KEY. From that point on, the compiled checks run after every Claude Code turn in sub-second with no AI calls. Rules that require judgment become conventions for the AI review layers instead. Nothing is dropped.

• Setup: caliper init --agent — installs a stop hook in .claude/settings.json
• Runtime: sub-second, no API calls
• Exit codes: 0 = clean, 2 = violations (Claude fixes them)

Here's what the compilation looks like in practice:

| Your CLAUDE.md says | Caliper compiles to | | ---------------------------------------------- | --------------------------------------------- | | "No classes — use functions and plain objects" | Convention check — flags class declarations | | "Keep functions under ~30 lines" | AST check — measures each function's length | | "Never use execSync with template strings" | Convention check — flags execSync( calls | | "Every migration needs a test file" | File-exists check — ensures .test.ts exists | | "Scripts must import dotenv/config" | File-contains check — flags missing import | | "Scripts must use kebab-case filenames" | File-path check — flags uppercase in path |

2. Local AI review

AI reviews your staged changes before you commit — no GitHub PR needed.

• Usage: caliper review
• Requires: ANTHROPIC_API_KEY
• Variants: --fix (auto-apply fixes), --branch (diff vs base branch)
• Pre-commit hook: caliper gate — non-interactive, exit 1 on blockers

3. PR review

Deep AI review posted as inline GitHub comments.

• Interactive mode: run locally, review each finding, approve/edit/skip before posting

  caliper 42

• GitHub Action / CI mode: lights-out — auto-approves and posts findings

  caliper 42 --ci --min-severity recommendation --max-cost 2.00 --fail-on-blocking

• Requires: ANTHROPIC_API_KEY + GitHub CLI (gh)
• Small PRs (<100 lines, ≤3 files, no high-risk files) automatically skip expensive phases

Prerequisites

| Prerequisite | Convention enforcement | Local AI review | PR reviews | | --------------------- | :--------------------: | :-------------: | :--------: | | Node.js 20+ | required | required | required | | Anthropic API key | setup only (init) | required | required | | GitHub CLI (gh) | — | — | required |

• GitHub CLI: install, then gh auth login
• Anthropic API key: get one, then export ANTHROPIC_API_KEY="sk-ant-..."

Installation

npm install --save-dev @caliperai/caliper
# or: pnpm add -D @caliperai/caliper

Quick Start

1. Initialize

export ANTHROPIC_API_KEY="sk-ant-..."
npx caliper init

This auto-detects your framework, compiles your CLAUDE.md rules into convention checks, and installs the Claude Code stop hook. Convention checks run after every agent turn with no AI calls.

2. Review local changes

caliper review               # AI review of staged changes (requires ANTHROPIC_API_KEY)

3. Review a PR

caliper 42                   # interactive review of PR #42 (requires ANTHROPIC_API_KEY + gh)

Commands

Convention checks (stop hook)

| Command | Description | | ------------------------------- | ------------------------------------------------- | | caliper check | Run convention checks (exit 2 on violations) | | caliper check --all-files | Check all source files, not just recently changed | | caliper init --agent | Install Claude Code stop hook | | caliper refresh | Recompile convention checks from your CLAUDE.md | | caliper refresh --interactive | Recompile with manual approval of each check |

Local AI review

| Command | Description | | ------------------------------- | ---------------------------------------------------------- | | caliper review | Review staged changes interactively | | caliper review --fix | Auto-apply suggested fixes to working tree | | caliper review --branch | Review all commits on this branch vs base branch | | caliper review --full | Force deep review even for small changesets | | caliper review --resume | Resume the last review without re-running AI | | caliper review --severity <l> | Minimum severity to report (blocking, recommendation, nit) | | caliper review --verbose | Show Claude API request/response details |

Pre-commit hook

| Command | Description | | -------------- | ---------------------------------------------------------------- | | caliper gate | Lightweight AI check for git hooks (exit 1 on blocking findings) |

PR review

| Command | Description | | ----------------------------- | ----------------------------------------- | | caliper | Show PR picker (interactive PR selection) | | caliper <pr> | Run deep AI review and post findings | | caliper <pr> --resume | Resume from the last saved phase | | caliper <pr> --no-post | Run the review but skip posting to GitHub | | caliper <pr> --post-only | Post findings from a previous review | | caliper <pr> --skip-refresh | Skip the staleness check on startup | | caliper <pr> --fast | Force fast path (skip expensive phases) | | caliper <pr> --full | Force deep review even for small PRs | | caliper <pr> --verbose | Show AI request/response details |

PR review (CI mode)

| Flag | Description | | ------------------------ | -------------------------------------------------------- | | --ci | Non-interactive mode (auto-approve and post findings) | | --min-severity <level> | Minimum severity to post (blocking, recommendation, nit) | | --max-cost <amount> | Cost ceiling in USD (skip review if exceeded) | | --fail-on-blocking | Exit 1 if blocking findings found |

Example:

caliper 42 --ci --min-severity recommendation --max-cost 2.00 --fail-on-blocking

Utilities

| Command | Description | | -------------------------------- | ----------------------------------------------------- | | caliper init | Scaffold .caliper/ config (non-interactive default) | | caliper init --interactive | Interactive init with full control over settings | | caliper doctor | Run health checks (Node.js, gh, API key, config) | | caliper config | Show resolved configuration | | caliper learn | Mine Claude Code history to propose new conventions | | caliper stats | Show review history analysis and patterns | | caliper trace <pr> | View the pipeline trace for a completed review | | caliper clean | Remove old review state and trace files | | caliper clean <pr> | Remove state for a specific PR | | caliper clean --older-than <N> | Remove reviews older than N days |

CI Integration

Run Caliper in any CI environment that supports Node.js 20+:

# .github/workflows/caliper.yml
name: Caliper Review
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      - run: npm install -g @caliperai/caliper
      - run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Configuration

After caliper init, your .caliper/ directory contains:

| File | Purpose | | ------------------ | ------------------------------------------------------------------- | | config.yaml | Project configuration (preset, source dirs, toolchain, preferences) | | policy.md | Review policy (what the AI evaluates against) | | conventions.yaml | Judgment-based conventions (for AI review layers) |

Example config.yaml:

preset: typescript
srcDirs:
  - src
  - lib
toolchain:
  format: npm run format:check
  lint: npm run lint
  test: npm test

Supported Presets

TypeScript, Python, Go, Ruby, Rust, Java. Framework is auto-detected during caliper init.

Preferences

Control how findings are written:

preferences:
  tone: direct # direct | friendly | terse
  strictness: precise # precise | balanced | thorough
  nits: true # report minor style nits
  suggestedFixes: always # always | when-clear | never

Standards

Set the bar for what the reviewer enforces:

standards:
  testing: complex-only # always | complex-only | none
  documentation: complex-only # public-api | complex-only | none
  errorHandling: important-paths # strict | important-paths | discretionary
  complexity: moderate # strict | moderate | none
  logging: important-paths # strict | important-paths | none
  dependencies: justified # strict | justified | open
  magicValues: non-obvious # strict | non-obvious | none
  deadCode: strict # strict | comments-only | none

Lenses

Focused domain-expert passes that activate based on risk signals in the code. Available lenses: security, data-integrity, api-contracts, concurrency, design.

lenses:
  - security
  - data-integrity
  - api-contracts
  - concurrency
  - design

See the User Guide for the full configuration reference.

API Usage

Caliper is free. You bring your own Anthropic API key.

Convention checks use no API calls at runtime. AI review (local and PR) uses your ANTHROPIC_API_KEY to call the Claude API. Usage scales with diff size. Control it with --max-cost in CI mode and the costWarningThreshold config option.

State and Resume

Review state is saved to /tmp/caliper/<pr>/state.json after each phase. If a review is interrupted, use --resume to pick up where it left off. Use caliper clean to remove old state files.

caliper 42 --resume

Troubleshooting

| Error | Fix | | --------------------------------- | --------------------------------------------------------------------- | | GitHub CLI (gh) not found | Install from https://cli.github.com | | GitHub CLI not authenticated | Run gh auth login | | Failed to fetch PR #N metadata | Check the PR number and that gh has access to this repo | | ANTHROPIC_API_KEY is not set | Get a key at https://console.anthropic.com and export it | | Invalid API key | Check that ANTHROPIC_API_KEY is correct and not expired | | Anthropic rate limit hit | Wait a minute, then retry with --resume | | Anthropic API overloaded | Wait a few minutes, then retry with --resume | | GitHub rate limit hit | Wait a few minutes, then retry with --resume | | gh CLI timed out | Check your network connection and retry with --resume | | Diff exceeds 50 MB buffer limit | The PR is too large — split it into smaller PRs | | Error: not a git repository | Run caliper from inside a git repo | | Cost warning exceeds threshold | Adjust costWarningThreshold in .caliper/config.yaml (default: $2) |

Set CALIPER_DEBUG=1 to show full stack traces on errors.