pi-model-router

Intelligent per-turn model router extension for the pi coding agent. Automatically selects between high, medium, and low-tier LLMs based on task intent, session budget, context size, and custom rules — with automatic fallbacks, phase awareness, Ollama sync, and transparent rate-limit recovery.

✨ Features

| Feature | Description | |---------|-------------| | Logical Router Provider | Registers a router provider that exposes stable profiles (e.g., router/auto) as models. | | Per-Turn Routing | Intelligently chooses between high, medium, and low tiers for every turn based on task intent and complexity. | | Task-Aware Heuristics | Detects planning vs. implementation vs. lightweight tasks using keyword analysis, word count, and conversation history. | | LLM Intent Classifier | Optionally use a fast model to categorize intent (overrides heuristics). | | Custom Rules | Define keyword-based tier overrides for specific patterns (e.g., deploy → high). | | Context Trigger | Automatically upgrade to high-tier when token usage exceeds a threshold. | | Cost Budgeting | Set a session spend limit; high tier downgrades to medium once exceeded. | | Fallback Chains | Automatic retry with alternative models if the primary choice fails. | | Phase Memory | Biased stickiness to keep you in the same tier during multi-turn planning or implementation work. | | Thinking Control | Full control over reasoning/thinking levels per tier and profile. | | Persistent State | Pins, profiles, costs, and debug history are remembered across agent restarts and conversation branches. | | Ollama Auto-Sync | Auto-detect and register local Ollama models. | | Rate-Limit Fallback | Detect HTTP 402/429/503/529 and transparently fall back to local models. | | Feature Toggles | Enable/disable features at user or project level. | | Progressive Enhancement | Detects installed plugins (qmd-ledger, agent-bus) and integrates conditionally. |

📦 Installation

From npm

pi install npm:@kylebrodeur/pi-model-router

From source

git clone https://github.com/kylebrodeur/pi-model-router.git
cd pi-model-router
pi install .

Quick test

pi -e ./extensions/index.ts

🚀 Quick Start

# 1. Install the extension
pi install npm:@kylebrodeur/pi-model-router

# 2. Create default config
/router init

# 3. Reload to apply
/reload

# 4. Check status
/router status

⚙️ Configuration

Copy the example config to one of:

• ~/.pi/agent/model-router.json (Global)
• .pi/model-router.json (Project-specific)

Basic Config Shape

{
  "defaultProfile": "auto",
  "classifierModel": "google/gemini-flash-latest",
  "maxSessionBudget": 1.0,
  "profiles": {
    "auto": {
      "high": { "model": "openai/gpt-5.4-pro", "thinking": "high" },
      "medium": { "model": "google/gemini-flash-latest", "thinking": "medium" },
      "low": { "model": "openai/gpt-5.4-nano", "thinking": "low" }
    }
  }
}

Configuration Fields

| Field | Description | |-------|-------------| | defaultProfile | The profile to use when starting a new session. | | classifierModel | (Optional) Model used to categorize intent. If omitted, fast heuristics are used. | | maxSessionBudget | (Optional) USD budget for the session. Forces medium tier once exceeded. | | largeContextThreshold | (Optional) Token count trigger to force high tier for large contexts. | | phaseBias | (0.0 - 1.0) Stickiness of the current phase. Higher = more stable. Default 0.5. | | rules | List of custom keyword rules (e.g. { "matches": "deploy", "tier": "high" }). | | profiles | Map of profile definitions, each containing high, medium, and low tiers. |

Feature Toggles

{
  "features": {
    "ollamaSync": true,
    "rateLimitFallback": true,
    "scopeShim": true,
    "perTurnRouting": true,
    "intentClassifier": false,
    "costBudgeting": true,
    "phaseMemory": true,
    "contextCompression": true,
    "ledgerIntegration": false,
    "agentBusIntegration": false
  }
}

Priority: Project config .pi/model-router.json overrides user config ~/.pi/agent/model-router.json. Both override defaults.

Rate Limit Interception & Fallback

The router can gracefully handle 429 Rate Limit and Quota errors. If the error specifies a wait time (e.g., "reset after 58s"), the router will pause and automatically retry the prompt if the wait time is under your threshold. If it exceeds the threshold or is unparseable, it fails over to the next available model in your fallback sequence.

{
  "rateLimitFallback": {
    "enabled": true,
    "shortDelayThreshold": 60,
    "autoFallback": true,
    "autoRestore": true,
    "restoreCheckInterval": 300,
    "fallbackSequence": ["anthropic/claude-3-haiku-20240307", "ollama/*"]
  }
}

| Field | Description | |-------|-------------| | shortDelayThreshold | Maximum time (in seconds) the router will pause and wait to retry when encountering a rate limit. If the cooldown is longer than this, it triggers a fallback. | | fallbackSequence | Array of model IDs (or wildcards like ollama/*) to try if the primary model fails or the wait time is too long. | | autoFallback | (Optional) Automatically switch session to the fallback model globally after a hard failure. | | autoRestore | (Optional) If fallback was triggered, automatically try to restore the original cloud model after restoreCheckInterval seconds. |

Progressive Enhancement Configs

After installing optional extensions, copy one of these to .pi/model-router.json:

| File | Feature | |------|---------| | model-router.ledger.json | Log routing decisions to qmd-ledger | | model-router.agent-bus.json | Publish model changes to agent-bus | | model-router.essential.json | All integrations enabled |

⌨️ Commands

Core Router Commands

| Command | Description | |---------|-------------| | /router | Show detailed status, current profile, spend, and settings. | | /router status | Alias for /router (show current status). | | /router profile [name] | Switch to a profile or list available ones (enables router if off). | | /router pin [prof] <t\|a> | Pin a tier (high/medium/low/auto) for the current or specified profile. | | /router fix <tier> | Correct the last decision and pin that tier for the current profile. | | /router thinking ... | Override thinking levels (e.g., /router thinking low xhigh). | | /router disable | Disable the router and switch back to the last non-router model. | | /router widget <on\|off> | Toggle the persistent state widget (supports toggle). | | /router debug <on\|off> | Toggle turn-by-turn routing notifications (supports toggle, clear, show). | | /router reload | Hot-reload the configuration JSON. | | /router help | Show usage help for all subcommands. |

Fork-Added Commands

| Command | Feature | Description | |---------|---------|-------------| | /router ollama-sync | ollamaSync | Manually sync Ollama models | | /router fallback | rateLimitFallback | Switch to fallback model sequence | | /router restore | rateLimitFallback | Restore cloud model | | /router init | | Scaffold default config file | | /router scope apply | scopeShim | Sync router profiles to Pi enabled models | | /router scope reset | scopeShim | Clear router profiles from Pi enabled list | | /router scope show | scopeShim | Show current Pi scoped models settings |

🔒 Requirements

• Pi >= 0.70.2 — Required for after_provider_response event (rate limit detection)
• Ollama — Running locally for Ollama sync and fallback
• Node.js >= 18 — For TypeScript compilation

📚 Documentation

• Architecture Guide: Deep dive into routing logic and modular design.
• Testing Guide: Step-by-step testing checklist.
• Contributing Guide: How to contribute back to upstream.
• Quick Start: 6-step install guide.
• Sample Configuration: Diverse profile examples (cheap, deep, balanced).
• Learnings: Development insights and best practices.

🤝 Contributing

See CONTRIBUTING.md for:

• How to split changes into upstream-friendly PRs
• Commit message conventions
• Testing requirements
• What to say in PR descriptions

📜 License

MIT — See LICENSE.

🙏 Acknowledgements

• yeliu84/pi-model-router: The original author and architecture behind the router provider.
• shouvik12/trooper: Inspiration for robust, transparent HTTP rate-limit fallback triggers.