pi-model-router

Smart per-turn model router extension for the pi-coding-agent that optimizes your AI budget and usage limits without sacrificing quality by dynamically routing each turn to the optimal LLM tier. It automatically selects between high, medium, and low-tier models based on task intent, session budget, context size, and custom rules — complete with automatic fallbacks and phase awareness.

What it does

• Logical Router Provider: Registers a router provider that exposes stable profiles (e.g., router/balanced) as models.
• Per-Turn Routing: Intelligently chooses between high, medium, and low tiers for every turn based on task intent and complexity.
• Task-Aware Heuristics: Detects planning vs. implementation vs. lightweight tasks using keyword analysis, word count, and conversation history.
• Advanced Controls: Includes built-in support for:
  • LLM Intent Classifier: Optionally use a fast model to categorize intent (overrides heuristics).
  • Custom Rules: Define keyword-based tier overrides for specific patterns (e.g., deploy → high).
  • Cost Budgeting: Set a session spend limit; high tier downgrades to medium once exceeded.
  • Fallback Chains: Automatic retry with alternative models if the primary choice fails.
• Phase Memory: Biased stickiness to keep you in the same tier during multi-turn planning or implementation work.
• Thinking Control: Full control over reasoning/thinking levels per tier and profile. Changing pi's thinking level (e.g. via shift+tab) automatically applies as an all-tier override for the active router profile.
• Persistent State: Pins, profiles, costs, and debug history are remembered across agent restarts and conversation branches.

Installation

As a user

Install from npm:

pi install npm:@yeliu84/pi-model-router

For development

Clone this repo and install from source:

pi install .

Or load directly for one run:

pi -e ./extensions/index.ts

Configuration

Copy the example config to one of:

• ~/.pi/agent/model-router.json (Global)
• .pi/model-router.json (Project-specific)

Basic Config Shape

{
  "classifierModel": "google/gemini-flash-latest",
  "maxSessionBudget": 1.0,
  "profiles": {
    "auto": {
      "high": { "model": "openai/gpt-5.4-pro", "thinking": "high" },
      "medium": { "model": "google/gemini-flash-latest", "thinking": "medium" },
      "low": { "model": "openai/gpt-5.4-nano", "thinking": "low" }
    }
  }
}

Configuration Fields

| Field | Description | | ----------------------- | --------------------------------------------------------------------------------- | | classifierModel | (Optional) Model used to categorize intent. Supports model aliases. If omitted, fast heuristics are used. | | maxSessionBudget | (Optional) USD budget for the session. Forces medium tier once exceeded. | | phaseBias | (0.0 - 1.0) Stickiness of the current phase. Higher = more stable. Default 0.5. | | rules | List of custom keyword rules (e.g. { "matches": "deploy", "tier": "high" }). | | models | (Optional) Map of model aliases to definitions with model, contextWindow, maxTokens. | | profiles | Map of profile definitions, each containing optional high, medium, and low tiers (at least one required). Tier models can reference aliases from models. |

Commands

| Command | Description | | --------------------------- | ------------------------------------------------------------------------------- | | /router | Show detailed status, current profile, spend, and settings. | | /router status | Alias for /router (show current status). | | /router profile [name] | Switch to a profile or list available ones (enables router if off). | | /router pin <t\|a> | Pin a tier (high/medium/low/auto) for the active profile. | | /router fix <tier> | Correct the last decision and pin that tier for the current profile. | | /router thinking <level> | Override thinking level for all tiers (e.g. /router thinking xhigh). Not all tier models may support every level. | | /router thinking <tier> <level> | Override thinking level for a specific tier (e.g. /router thinking low off). | | /router disable | Disable the router and switch back to the last non-router model. | | /router widget <on\|off> | Toggle the persistent state widget (supports toggle). | | /router debug <on\|off> | Toggle turn-by-turn routing notifications (supports toggle, clear, show). | | /router reload | Hot-reload the configuration JSON. | | /router help | Show usage help for all subcommands. |

Documentation

• Architecture Guide: Deep dive into the routing logic and modular design.
• Sample Configuration: Diverse profile examples (cheap, deep, balanced).