model-router

A Pi coding agent extension that routes each coding request to the cheapest model capable of handling it. The idea: not every task needs the same model. A quick "what does this function return?" costs a fraction of what an open-ended refactor costs, and routing everything to one model wastes money on simple tasks without saving it on hard ones.

Every tier's provider and model is configured via .env. Install it, edit .env, and point each tier at OpenAI, Anthropic, or a local OpenAI-compatible server (Ollama, LM Studio, vLLM, and so on) in any mix. See .env.example for the full set of overrides and a worked example running two tiers local and two tiers hosted.

How it works

Before each turn, a lightweight planner call (default: gpt-5-mini) reads the request and classifies it on three axes:

• editScope: does the task change code? none (read-only), single-file, or multi-file. File count is not a difficulty signal: a mechanical rename across 20 files stays at Tier 2.
• needsUnderstanding: does correctness require tracing behavior across multiple functions or files? Debugging lives here. Reading one function to answer a question does not.
• openEndedness: specified (clear instructions), some (moderate ambiguity), or open (key decisions left to the model).

Those three axes map deterministically to a tier. The planner classifies the work; the routing table picks the model.

Tiers

Reference tier table, this is what .env.example ships with, but there is no in-code default; .env is the only source of truth (see below):

| Tier | Reference model | What it handles | |------|-------|-----------------| | T1 | gpt-5-nano | Read-only tasks: questions, explanations, summaries | | T2 | gpt-5.4-nano | Well-specified single-file edits and mechanical refactors | | T3 | gpt-5.4-mini | Work requiring understanding of existing behavior, debugging, multi-file changes | | T4 | gpt-5.5 | Open-ended tasks, large designs, problems where key decisions are left to the model |

Configuring providers and models

Every tier (and the planner) resolves its provider and model entirely from .env, there is no hardcoded default in the code. TIER_N_PROVIDER, TIER_N_MODEL, TIER_N_COST_INPUT, TIER_N_COST_OUTPUT, TIER_N_CONTEXT_WINDOW, TIER_N_MAX_TOKENS (N = 1-4), and PLANNER_PROVIDER/PLANNER_MODEL/PLANNER_REASONING_EFFORT are all required. The router refuses to start if any is missing, naming exactly which var.

• Getting started: .env.example ships with a complete, ready-to-use OpenAI configuration (the reference table above). Copy it to .env, fill in OPENAI_API_KEY, and you're running, nothing to derive, it's already the whole config.
• Changing a tier: edit its six TIER_N_* vars directly in .env. There's no partial-override behavior, every field for that tier comes from what you set.
• Any provider name works: add {PROVIDER}_BASE_URL / {PROVIDER}_API_KEY for the provider name you reference (e.g. LOCAL_BASE_URL/LOCAL_API_KEY for a local provider). openai and anthropic are recognized by name with sensible base-URL/wire-format defaults; anything else needs {PROVIDER}_BASE_URL set explicitly.
• Mixed setups are first-class: for example two tiers on a local OpenAI-compatible server (Ollama, LM Studio, vLLM) and two on a hosted API (OpenAI, Anthropic) in the same run. See the worked example in .env.example.
• Cost tracking: TIER_N_COST_INPUT/TIER_N_COST_OUTPUT feed the daily-budget/logging cost estimate directly. Set them to 0 for local models or anything else without a real per-token price.

Full details and every override live in .env.example.

Setup

1. Install the Pi coding agent CLI (skip if you already have it):

npm install -g @earendil-works/pi-coding-agent

2. Install this package in your repo:

npm install @usmanghani23/model-router

3. This creates or updates the following, automatically:

| File | What happens | |------|--------------| | .env | Copied from the bundled template if you don't have one yet. If you already have a .env, the model-router variables are appended to the end instead, your existing content is left alone. | | budget-state.json | Created empty if it doesn't already exist. | | routing.log.json | Created empty if it doesn't already exist. | | .pi/settings.json | Created or updated so Pi loads the extension automatically. No manual wiring needed. |

4. Open .env and fill in the *_API_KEY value(s) it asks for (at minimum OPENAI_API_KEY if you're using the default setup).

5. Run it:

npx model-router

Budget/logging env vars (all have defaults):

| Variable | Default | Description | |----------|---------|-------------| | MAX_BUDGET_TIER_1 | 20,000 | Daily token cap for T1 | | MAX_BUDGET_TIER_2 | 30,000 | Daily token cap for T2 | | MAX_BUDGET_TIER_3 | 40,000 | Daily token cap for T3 | | MAX_BUDGET_TIER_4 | 80,000 | Daily token cap for T4 | | BUDGET_STATE_PATH | ./budget-state.json | Where daily usage is persisted | | LOG_FILE_PATH | ./routing.log.json | Where request logs are written |

Budget management

Each tier has a daily token cap. Once a tier hits 85% of its cap, the router stops routing to it for the rest of the day and substitutes the next available tier (cheaper first, then more expensive if cheaper tiers are also exhausted). Caps reset at midnight local time.

Escalation

The router escalates in two situations.

Retry escalation. When the same files come up again and the prior attempt's bash command failed, the router treats it as evidence the cheaper tier did not solve the problem. It bumps one tier per two failed attempts, up to T4. At T4 it stops escalating and notifies you instead of looping.

Mid-run escalation. During a run the router watches for three signals and bumps the tier immediately if any appear:

• Context grows past the current model's window
• Output truncates at the model's max-token limit
• The model repeats the same response across consecutive turns

Logging

Every completed request appends a JSON entry to routing.log.json:

{
  "query": "rename the `foo` variable to `bar` in utils.ts",
  "tier": 2,
  "selectedModel": "gpt-5.4-nano",
  "reason": "editScope=single-file, needsUnderstanding=no, openEndedness=specified",
  "escalated": false,
  "requestType": "simple-code",
  "contextTokens": 8200,
  "inputTokens": 1540,
  "outputTokens": 320,
  "estimatedCost": 0.0012,
  "budgetPercentUsed": 0.42,
  "budgetAction": "ok",
  "latencyMs": 4230
}

Working on this repo directly

Cloning this repo instead of installing it as a package? Copy .env.example to .env yourself. Pi auto-discovers .pi/extensions/routing-layer.ts as the extension entry point, no separate wiring needed.

Project layout

core/
  models.ts              tier definitions and per-model pricing
  planner.ts             demand-profile classification and tier mapping
  escalation-manager.ts  retry escalation logic
  budget-manager.ts      daily token caps and state persistence
  logger.ts              structured request logging

.pi/extensions/
  routing-layer.ts       Pi extension entry point, wires everything together

tests/                   unit tests for all core modules