Caliper

Code review for the agentic coding era.

AI agents write code fast — but they drift. They skip conventions, use wrong patterns, and repeat the same mistakes turn after turn. One wrong choice in turn 3 becomes the template for turns 4 through 30.

Caliper keeps agents on the rails with three feedback loops, from tightest to broadest:

1. After every agent turn — your CLAUDE.md rules, compiled into convention checks that run in sub-second with no AI calls
2. Before every commit — lightweight AI review of staged changes for logic, security, and design issues
3. Before every merge — deep AI review posted as inline GitHub comments

Each layer narrows what the next needs to evaluate. The agent writes code, Caliper catches drift, the agent fixes it — no human in the loop until the code already meets your standards.

Three Ways to Use Caliper

1. Convention enforcement (Claude Code stop hook)

npx caliper refresh reads your CLAUDE.md files and compiles every mechanically-checkable rule into a convention check — grep patterns, AST analysis, file scans. Re-run it whenever your CLAUDE.md changes. Requires ANTHROPIC_API_KEY. The compiled checks then run after every Claude Code turn in sub-second with no AI calls. Rules that require judgment become conventions for the AI review layers instead. Nothing is dropped.

• Setup: npx caliper init --agent — installs a stop hook in .claude/settings.json
• Runtime: sub-second, no API calls
• Exit codes: 0 = clean, 2 = violations (Claude fixes them)

Here's what the compilation looks like in practice:

| Your CLAUDE.md says | Caliper compiles to | | ---------------------------------------------- | --------------------------------------------- | | "No classes — use functions and plain objects" | Convention check — flags class declarations | | "Keep functions under ~30 lines" | AST check — measures each function's length | | "Never use execSync with template strings" | Convention check — flags execSync( calls | | "Every migration needs a test file" | File-exists check — ensures .test.ts exists | | "Scripts must import dotenv/config" | File-contains check — flags missing import | | "Scripts must use kebab-case filenames" | File-path check — flags uppercase in path |

2. Local AI review

AI reviews your staged changes before you commit — no GitHub PR needed.

• Usage: npx caliper review
• Requires: ANTHROPIC_API_KEY
• Variants: --fix (auto-apply fixes), --branch (diff vs base branch), --severity <level> (filter by severity), --full (force deep review)
• Pre-commit hook: npx caliper gate — deterministic + AI checks, non-interactive, exit 1 on blockers

3. PR review

Deep AI review posted as inline GitHub comments.

• Interactive mode: run locally, review each finding, approve/edit/skip before posting

  npx caliper 42

• GitHub Action / CI mode: lights-out — auto-approves and posts findings

  npx caliper 42 --ci --min-severity recommendation --max-cost 2.00 --fail-on-blocking

• Requires: ANTHROPIC_API_KEY + GitHub CLI (gh)
• Small PRs (<100 lines, ≤3 files, no high-risk files) automatically skip expensive phases

Prerequisites

| Prerequisite | Convention enforcement | Local AI review | PR reviews | | --------------------- | :--------------------: | :-------------: | :--------: | | Node.js 20+ | required | required | required | | Anthropic API key | setup only (init) | required | required | | GitHub CLI (gh) | — | — | required |

• GitHub CLI: install, then gh auth login
• Anthropic API key: get one, then export ANTHROPIC_API_KEY="sk-ant-..."

Installation

npm install --save-dev @caliperai/caliper
# or: pnpm add -D @caliperai/caliper

Quick Start

1. Initialize

export ANTHROPIC_API_KEY="sk-ant-..."
npx caliper init

This auto-detects your framework, compiles your CLAUDE.md rules into convention checks, and installs the Claude Code stop hook. Convention checks run after every agent turn with no AI calls.

2. Review local changes

npx caliper review               # AI review of staged changes (requires ANTHROPIC_API_KEY)

3. Review a PR

npx caliper 42                   # interactive review of PR #42 (requires ANTHROPIC_API_KEY + gh)

Commands

Convention checks (stop hook)

| Command | Description | | ----------------------------------- | ------------------------------------------------- | | caliper check | Run convention checks (exit 2 on violations) | | caliper check --all-files | Check all source files, not just recently changed | | caliper init --agent | Install Claude Code stop hook | | caliper refresh | Recompile convention checks from your CLAUDE.md | | caliper refresh --interactive | Recompile with manual approval of each check | | caliper refresh --force | Bypass cache and re-extract everything | | caliper refresh --concurrency <N> | Run N file extractions in parallel (default: 4) | | caliper refresh --eslint | Also generate .caliper/eslint.config.js | | caliper refresh --trace | Write extraction trace to /tmp/caliper/refresh/ |

Local AI review

| Command | Description | | ----------------------------------- | ---------------------------------------------------------- | | caliper review | Review staged changes interactively | | caliper review --fix | Auto-apply suggested fixes to working tree | | caliper review --branch | Review all commits on this branch vs base branch | | caliper review --pre-push | Review only commits not yet pushed (reads pre-push stdin) | | caliper review --full | Force deep review even for small changesets | | caliper review --resume | Resume the last review without re-running AI | | caliper review --fail-on-blocking | Exit 1 if blocking findings are found | | caliper review --severity <level> | Minimum severity to report (blocking, recommendation, nit) | | caliper review --verbose | Show Claude API request/response details |

Pre-commit hook

| Command | Description | | --------------------------------- | ------------------------------------------------------------------- | | caliper gate | Non-interactive AI + convention checks, exit 1 on blocking findings | | caliper gate --severity <level> | Minimum severity to report (blocking, recommendation, nit) | | caliper gate --verbose | Show AI request/response details |

PR review

| Command | Description | | ----------------------------- | ----------------------------------------- | | caliper | Show PR picker (interactive PR selection) | | caliper <pr> | Run deep AI review and post findings | | caliper <pr> --resume | Resume an interrupted review | | caliper <pr> --no-post | Run the review but skip posting to GitHub | | caliper <pr> --post-only | Post findings from a previous review | | caliper <pr> --sequential | Use sequential finding reviewer | | caliper <pr> --skip-refresh | Skip the staleness check on startup | | caliper <pr> --fast | Force fast path (skip expensive phases) | | caliper <pr> --full | Force deep review even for small PRs | | caliper <pr> --verbose | Show AI request/response details |

PR review (CI mode)

| Flag | Description | | ------------------------ | -------------------------------------------------------- | | --ci | Non-interactive mode (auto-approve and post findings) | | --min-severity <level> | Minimum severity to post (blocking, recommendation, nit) | | --max-cost <amount> | Cost ceiling in USD (skip review if exceeded) | | --fail-on-blocking | Exit 1 if blocking findings found |

Example:

npx caliper 42 --ci --min-severity recommendation --max-cost 2.00 --fail-on-blocking

Utilities

| Command | Description | | -------------------------------- | ----------------------------------------------------- | | caliper init | Scaffold .caliper/ config (non-interactive default) | | caliper init --interactive | Interactive init with full control over settings | | caliper init --ci | Generate .github/workflows/caliper.yml | | caliper init --hooks-only | Reinstall git hooks (no API key required) | | caliper doctor | Run health checks (Node.js, gh, API key, config) | | caliper config | Show resolved configuration | | caliper stats | Show review history analysis and patterns | | caliper trace <pr> | View the pipeline trace for a completed review | | caliper clean | Remove old review state and trace files | | caliper clean <pr> | Remove state for a specific PR | | caliper clean --older-than <N> | Remove reviews older than N days |

CI Integration

Run caliper init --ci to generate a .github/workflows/caliper.yml automatically. Or use the pattern below for any CI environment that supports Node.js 20+:

# .github/workflows/caliper.yml
name: Caliper Review
on:
  pull_request:
    types: [opened, synchronize]
    paths-ignore:
      - "**.md"
      - "pnpm-lock.yaml"
      - "package-lock.json"
      - "yarn.lock"
      - ".caliper/**"

permissions:
  pull-requests: write
  contents: read

jobs:
  review:
    runs-on: ubuntu-latest
    # Skip fork PRs — secrets are unavailable for forks and cannot post PR comments
    if: github.event.pull_request.head.repo.full_name == github.repository
    steps:
      - name: Check required secrets
        run: |
          if [ -z "$ANTHROPIC_API_KEY" ]; then
            echo "Error: ANTHROPIC_API_KEY secret is not set."
            exit 1
          fi
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      # Adjust for your package manager: pnpm install --frozen-lockfile, yarn install --immutable
      - run: npm ci
      - run: npx --no-install caliper "$PR_NUMBER" --ci --skip-refresh --fail-on-blocking
        env:
          PR_NUMBER: ${{ github.event.pull_request.number }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Configuration

After npx caliper init, your .caliper/ directory contains:

| File | Purpose | | ------------------ | ------------------------------------------------------------------- | | config.yaml | Project configuration (preset, source dirs, toolchain, preferences) | | policy.md | Review policy (what the AI evaluates against) | | conventions.yaml | Judgment-based conventions (for AI review layers) |

Example config.yaml:

preset: typescript
srcDirs:
  - src
  - lib
toolchain:
  format: npm run format:check
  lint: npm run lint
  test: npm test

Supported Presets

TypeScript, Python, Go, Ruby, Rust, Java. Framework is auto-detected during npx caliper init.

Preferences

Control how findings are written:

preferences:
  tone: direct # direct | friendly | terse
  strictness: precise # precise | balanced | thorough
  nits: true # report minor style nits
  suggestedFixes: always # always | when-clear | never

Standards

Set the bar for what the reviewer enforces:

standards:
  testing: complex-only # always | complex-only | none
  documentation: complex-only # public-api | complex-only | none
  errorHandling: important-paths # strict | important-paths | discretionary
  complexity: moderate # strict | moderate | none
  logging: important-paths # strict | important-paths | none
  dependencies: justified # strict | justified | open
  magicValues: non-obvious # strict | non-obvious | none
  deadCode: strict # strict | comments-only | none

Lenses

Focused domain-expert passes that activate based on risk signals in the code. Available lenses: security, data-integrity, api-contracts, concurrency, design.

lenses:
  - security
  - data-integrity
  - api-contracts
  - concurrency
  - design

API Usage

Caliper is free. You bring your own Anthropic API key.

Convention checks use no API calls at runtime. AI review (local and PR) uses your ANTHROPIC_API_KEY to call the Claude API. Usage scales with diff size. Control it with --max-cost in CI mode and the costWarningThreshold config option.

State and Resume

Review state is saved automatically as it runs. If a review is interrupted, use --resume to pick up where it left off. Use npx caliper clean to remove old state files.

npx caliper 42 --resume

What Caliper Modifies

When you run npx caliper init, Caliper writes to the following locations:

| Path | Purpose | Created by | | ----------------------------------- | -------------------------------------------------------------------- | ---------------------------------- | | .caliper/config.yaml | Project configuration | caliper init | | .caliper/policy.md | AI review policy template | caliper init | | .caliper/checks.js | Compiled convention checks | caliper init / caliper refresh | | .caliper/conventions.yaml | Extracted conventions | caliper init / caliper refresh | | .claude/settings.json | Registers the Claude Code stop hook (npx caliper check) | caliper init | | .git/hooks/pre-commit | Pre-commit gate hook (npx caliper gate) | caliper init (optional) | | .git/hooks/pre-push | Pre-push review hook (npx caliper review --pre-push) | caliper init (optional) | | ~/.caliper/telemetry-id | Anonymous installation ID (prefixed UUIDv4) | First run | | ~/.caliper/telemetry-notice-shown | Sentinel so the telemetry notice isn't repeated | First run | | ~/.caliper/telemetry-org | GitHub owner slug detected via gh repo view for telemetry labeling | First run (if gh available) | | ~/.caliper/telemetry-queue/ | Daily telemetry summary files awaiting upload | Each day of usage | | ~/.caliper/telemetry-opt-out | Opt-out sentinel — create this file to disable telemetry | On opt-out |

Telemetry: On first run, Caliper runs gh repo view --json owner (5-second timeout) to detect your GitHub owner name for telemetry labeling. This is skipped entirely when telemetry is disabled. Usage data is anonymous (command counts, errors — no code or file paths) and sent to getcaliper.dev on each command. Disable at any time:

export CALIPER_TELEMETRY=0                    # environment variable
touch ~/.caliper/telemetry-opt-out            # persistent opt-out sentinel

See getcaliper.dev/telemetry for details.

Network calls:

• Anthropic API (api.anthropic.com) — only when running AI review commands (caliper review, caliper <pr>)
• Caliper telemetry (getcaliper.dev) — anonymous usage events on each command, opt-out above
• No network calls during caliper check (convention enforcement runs entirely locally)

Troubleshooting

| Error | Fix | | --------------------------------- | --------------------------------------------------------------------- | | GitHub CLI (gh) not found | Install from https://cli.github.com | | GitHub CLI not authenticated | Run gh auth login | | Failed to fetch PR #N metadata | Check the PR number and that gh has access to this repo | | ANTHROPIC_API_KEY is not set | Get a key at https://console.anthropic.com and export it | | Invalid API key | Check that ANTHROPIC_API_KEY is correct and not expired | | Anthropic rate limit hit | Wait a minute, then retry with --resume | | Anthropic API overloaded | Wait a few minutes, then retry with --resume | | GitHub rate limit hit | Wait a few minutes, then retry with --resume | | gh CLI timed out | Check your network connection and retry with --resume | | Diff exceeds 50 MB buffer limit | The PR is too large — split it into smaller PRs | | Error: not a git repository | Run npx caliper from inside a git repo | | Cost warning exceeds threshold | Adjust costWarningThreshold in .caliper/config.yaml (default: $2) |

Set CALIPER_DEBUG=1 to show full stack traces on errors.