Calibra

Automatic per-prompt model routing for Claude Code. Calibra intercepts every prompt via a local HTTP proxy, classifies its complexity, and rewrites the model field in-flight — so cheap prompts use lighter models automatically, without any manual switching.

Tiers

| Tier | Default Model | When | |------|--------------|------| | light | Haiku | Greetings, recall/lookup, trivial one-liners (add log, rename, fix typo, format) | | mid | Sonnet | Concrete single-component work (fix, build, write, implement, explain) | | deep | Opus | Synthesis and judgment scoped to one system (design, audit, optimize, diagnose) | | ultra | Opus | Multi-system or org-wide programs (≥2 named subsystems, comprehensive, multi-quarter) |

How It Works

1. The Proxy

Calibra installs a local HTTP proxy (saka-proxy.js) and sets ANTHROPIC_BASE_URL to point at it. Every API request from Claude Code passes through before reaching the upstream server.

Claude Code
    │
    ▼  ANTHROPIC_BASE_URL → http://127.0.0.1:{port}
saka-proxy.js                   ← reads prompt, classifies, rewrites model
    │
    ▼  CALIBRA_REMOTE_HOST
Upstream AI Server

See docs/diagrams/proxy-architecture.drawio for the full intercept flow.

Two classification engines are available. The active engine is controlled by the calibra-engine flag file (absent = heuristic).

2. Heuristic Engine (default)

The heuristic engine scores every prompt across five independent axes — no signal appears in more than one axis — then maps the total to a tier.

Step-by-step:

0. Typo correction (correctTypos) — applied once before scoring. Hunspell dictionaries (dictionary-en, dictionary-en-gb, dictionary-tr) plus Damerau-Levenshtein distance correct single-character typos (e.g. secuity → security, analz → analiz). Code fences and identifiers are never touched. Fail-soft: if dictionaries are missing, this step is a no-op.

1. Early exits — checked in order, short-circuit immediately:

   • Greeting or social acknowledgement → light
   • Trivial one-liner (add console.log, rename variable, fix typo, add null check) → light
   • Slash command (/) → mid
   • Short prompt (≤55 chars) with no actionable signal → light

2. 5-axis scoring:

   | Axis | Signal | Points | |------|--------|--------| | 1 — Length | > 500 chars | +3 | | | > 200 chars | +2 | | | ≥ 80 chars | +1 | | 2 — Intent | Deep verbs: architect, design, analyse, audit, investigate, diagnose, review, optimize, harden, evaluate, compare, strategy, plan, … | +3 | | | Mid verbs: implement, build, create, write, fix, debug, add, update, migrate, explain, configure, … | +1 | | 3 — Scope | Breadth words: comprehensive, entire, full, end-to-end, exhaustive, detailed, overall, holistic, company-wide, org-wide, genelinde, … | +2 | | 4 — Domain | Technical vocabulary: distributed, microservices, kubernetes, graphql, grpc, authentication, event-driven, sharding, circuit breaker, scalability, … | +2 | | 5 — Structure | Multiple code blocks or block > 52 lines | +2 | | | Single code block | +1 | | | Step-by-step / multi-part markers | +1 |

3. Floor rules:

   • Any deep-intent verb → minimum deep regardless of score
   • Any mid-intent verb → minimum mid

4. Threshold mapping (max realistic score ≈ 13):

   score 0–2, no intent      → light
   score 0–7, mid intent     → mid
   deep intent present       → deep   (floor)
   score 8+                  → ultra

5. Model rewrite — model field in the request body is replaced with the tier's configured model before forwarding.

See docs/diagrams/heuristic-engine.drawio for the full flow.

3. ML Engine (opt-in — /calibra ml on)

The ML engine uses a rule-first cascade: deterministic rules handle the clearly-decidable cases; a MiniLM neural model owns the genuinely ambiguous residual under a cost objective.

Step-by-step:

1. Rule layer (ruleClassify) — evaluated in order, first match wins and commits immediately:

   | Rule | Condition | Result | |------|-----------|--------| | 1 | Empty prompt or slash command | mid | | 2 | Pure greeting / social | light | | 3 | Trivial single-edit EN/TR (add null check, rename symbol, fix indentation, add return statement, add null guard, "yeniden isimlendir", "girintiyi duzelt", …) | light | | 4 | ≥2 breadth words or 3+ item enumeration or ≥2 distinct named subsystems joined by a coordinator (auth and payments, mobile app and backend, …) | ultra | | 5–7 | Intent verb (deep/mid) or short-no-signal | not confident → defer to ML |

   Rules 1–4 are 100% precise on every eval set and commit without calling the model. Rules 5–7 defer because an intent verb alone does not pin the tier — that is the irreducible ambiguity the ML must own.

2. Typo correction (correctTypos) — same EN/TR hunspell correction as the heuristic engine, applied before the rule layer. The ONNX embedding also receives corrected text, so a typo variant and its correct form produce the same (or very close) sentence vector.

3. MiniLM ONNX pipeline (rules 5–7 residual only):

   1. Tokenize prompt with BERT WordPiece (bert-base-uncased). MiniLM's trained max sequence length is 256 tokens, so a longer prompt is split into overlapping 256-token windows (56-token stride) instead of tail-truncated — up to 4 windows, keeping the head + tail windows and dropping the middle if there are more (task statement is usually up front, constraints/edge-cases at the end).
   2. Run all-MiniLM-L6-v2 ONNX on each window → last_hidden_state [1 × seq × 384]
   3. Mean-pool with attention mask → sentence vector [384] per window; L2-normalize
   4. Append lexical feature axes (deepC, midC, scopeC, domainC) and classify each window independently
   5. Ordinal regression head → tier posterior distribution [light, mid, deep, ultra], per window
   6. expectedCostDecision(policy I) — choose the tier minimising expected routing cost over the posterior, not raw argmax, per window. The cost matrix biases the ambiguous boundary toward the safer tier (never severely under-route). Across windows, the most severe tier wins (tie-break: higher score) — a prompt is as complex as its hardest part, not its average. A single window (the common case) skips straight to that window's decision.

4. Fail-soft — if the ONNX model is absent, times out (CALIBRA_ML_TIMEOUT_MS), or throws, the system silently falls back to the heuristic engine. No error is shown to the user.

See docs/diagrams/ml-engine.drawio for the full cascade.

Accuracy: ~93% on the dev benchmark; ~88% on independent holdouts labeled by separate models. The ~5% gap is the irreducible label-noise ceiling — terse, typo-heavy, and multi-system prompts sit at genuine tier boundaries where labelers themselves disagree. See docs/RESULTS.md for the full cross-labeler analysis.

Requirements

• Node.js ≥ 18
• Claude Code CLI
• An enterprise wrapper that sets CALIBRA_REMOTE_HOST and ANTHROPIC_BASE_URL before launching Claude Code

Installation

Option A — npx (recommended)

npx calibra install

Option B — global install

npm install -g calibra

If you get EACCES: permission denied:

mkdir -p ~/.npm-global
npm config set prefix '~/.npm-global'
echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.zshrc
source ~/.zshrc
npm install -g calibra

If you use nvm or fnm, global installs already work without this step.

The postinstall script:

1. Copies saka-proxy.js to ~/.claude-corp/
2. Copies hooks to ~/.claude/hooks/
3. Copies the /calibra command to ~/.claude/commands/
4. Creates ~/.claude-corp/calibra/calibra-models.json (first install only — never overwritten on upgrade)
5. Copies ML runtime files (including spellcorrect.js) to ~/.claude-corp/calibra/ml/
6. Installs onnxruntime-node, nspell, dictionary-en, dictionary-en-gb, dictionary-tr into ~/.claude-corp/calibra/node_modules/ (skips packages already present)
7. Registers hooks in ~/.claude/settings.json

Configuration

Model tiers — ~/.claude-corp/calibra/calibra-models.json

Edit to change which model each tier uses:

{
  "light": "claude-haiku-4-5-20251001",
  "mid":   "claude-sonnet-4-6",
  "deep":  "claude-opus-4-7",
  "ultra": "claude-opus-4-7",
  "nonAnthropicModels": []
}

Never overwritten on upgrade. nonAnthropicModels lists model IDs that need special request sanitisation (strip thinking blocks, set min max_tokens).

Remote host

export CALIBRA_REMOTE_HOST="your-litellm-server.example.com"

ML environment variables

| Variable | Purpose | Default | |----------|---------|---------| | CALIBRA_ML_MODEL_PATH | Path to a local .onnx file (air-gapped installs) | ~/.claude-corp/calibra/models/router.onnx | | CALIBRA_ML_TIMEOUT_MS | Max inference time before falling back to heuristic | 250 |

Spell-check dependencies (nspell, dictionary-en, dictionary-en-gb, dictionary-tr) are installed automatically into ~/.claude-corp/calibra/node_modules/ by the install script. If they are absent, typo correction silently no-ops — routing still works.

Usage

Calibra runs silently. A context note appears on each prompt:

calibra: >> claude-sonnet-4-6 . mid

/calibra command

/calibra status    → show routing state for Claude Code and Codex separately
/calibra on        → enable routing for Claude Code
/calibra off       → disable routing for Claude Code (original model used)
/calibra toggle    → flip Claude Code state
/calibra ml on     → switch to ML engine (downloads model on first use)
/calibra ml off    → switch back to heuristic
/calibra rules     → alias for ml off

Natural-language phrases also work: disable calibra, enable calibra.

Routing state is per-environment — Claude Code and Codex each have their own flag. /calibra on|off from a Claude Code session only affects Claude Code; the equivalent phrases in Codex only affect Codex.

Codex CLI users

Codex has no slash command system — /calibra will produce Unrecognized command '/calibra' in the Codex TUI. Use plain-text phrases instead; codex-proxy.js intercepts them at the wire before they reach the model:

| Intent | Type this in Codex | |--------|-------------------| | Enable routing | enable calibra | | Disable routing | disable calibra | | Check status | status calibra | | Turn on | turn on calibra | | Turn off | turn off calibra | | Enable ML engine | calibra ml on (or enable calibra ml) | | Disable ML engine | calibra ml off (or disable calibra ml, calibra rules) |

Each environment maintains its own enable/disable and engine state — toggling in Codex does not affect a running Claude Code session, and vice versa. Codex reads its own calibra-engine-codex flag; Claude Code reads calibra-engine.

Note: The ML engine is available on both Claude Code and Codex, each switched independently. As on the Claude side, Codex's ML path is fully fail-soft — if the ONNX model is absent, times out, or errors, it silently falls back to the heuristic. The routing decision is shown inline at the top of each Codex reply: » [calibra: <model> · <tier>].

ML Engine: First Activation

The first time you run /calibra ml on, a ~22 MB quantized ONNX model is downloaded to ~/.claude-corp/calibra/models/router.onnx and verified against a SHA-256 checksum.

Improving ML Accuracy

Add labeled prompts to tools/eval_prompts.jsonl then retrain:

node tools/train_tier_classifier.js     # re-fits the ordinal head
node tools/tune_thresholds.js tools/final_holdout_opus_500.jsonl --write
node tools/evaluate_classifier.js tools/adversarial_eval.jsonl   # honest number

Labeling rubric (keep human-assigned, never circular):

• light — no judgment: recall/lookup, single-statement mechanical edit, or social
• mid — one bounded component with chosen logic: implement/fix/debug/refactor a single function or feature
• deep — synthesis/judgment scoped to ONE system: design/architect/analyze/audit/optimize
• ultra — multi-system OR org/platform-wide OR long program: ≥2 named subsystems joined, comprehensive/company-wide, multi-quarter

Evaluation protocol — keep eyes separate:

| Role | File | Used for | |------|------|----------| | fit | eval_prompts.jsonl + targeted_train_opus_800.jsonl | gradient fit | | dev | final_holdout_opus_500.jsonl | threshold tuning only | | test | calibra_eval_set.jsonl, adversarial_eval.jsonl | report only — never trained/tuned on |

Upgrade

npx calibra upgrade

• saka-proxy.js and hooks are updated
• calibra-models.json is never overwritten
• calibra-ml.json is never overwritten

Uninstall

npx calibra uninstall

Removes all installed files, hooks, and hook entries from settings.json.

Files Installed

| File | Location | Purpose | |------|----------|---------| | saka-proxy.js | ~/.claude-corp/ | Proxy — classifies prompts, rewrites model | | calibra-models.json | ~/.claude-corp/calibra/ | Tier → model mapping (user config) | | calibra-ml.json | ~/.claude-corp/calibra/ | ML metadata and local model settings | | calibra-disabled-claude | ~/.claude-corp/calibra/ | Flag file — routing off for Claude Code when present | | calibra-disabled-codex | ~/.claude-corp/calibra/ | Flag file — routing off for Codex when present | | ml/ | ~/.claude-corp/calibra/ | ML classifier, tokenizer, vocab, centroids, spellcorrect | | models/router.onnx | ~/.claude-corp/calibra/ | Downloaded ONNX model (ML mode) | | calibra-notify.js | ~/.claude/hooks/ | Shows routing decision in context bar | | calibra-debug.js | ~/.claude/hooks/ | Logs raw hook input to <tmpdir>/calibra-debug.log | | calibra-toggle.js | ~/.claude/hooks/ | Handles Claude Code toggle commands | | calibra.md | ~/.claude/commands/ | /calibra slash command definition |

Publishing a New Version

npm version patch   # or minor / major
npm publish
git push && git push --tags

Platforms

| Platform | Status | |----------|--------| | macOS | Supported | | Linux | Supported | | Windows (native) | Supported |

Quick Reference

| Action | Command | |--------|---------| | Install | npx calibra install | | Check status | /calibra status in Claude Code | | Upgrade | npx calibra upgrade | | Uninstall | npx calibra uninstall | | Enable ML engine | /calibra ml on | | Disable routing | /calibra off |