omp-llm-role

Ranks today's LLM leaderboard into best-fit picks for each omp model role (default, smol, slow, vision, plan, commit, tiny, task, advisor, plus the opt-in designer) and ships an omp plugin that applies those picks to ~/.omp/agent/config.yml daily. Every role weights price and throughput, so the shipped default is the best-value pick, not the strongest one; set selection: "cheapest" to make it literally the cheapest model that can do the job.

Example output

The report is a full worked example — per-role top-10 tables (value, q, $/M, tok/s, ctx, weighted metric contributions, ★ Pareto marker) plus the suggested modelRoles block. See docs/llm-role-rankings.md.

Install

Any one of three routes:

omp plugin install omp-llm-role                          # npm
omp plugin install github:Gott50/omp-llm-role            # git (public repo)
omp plugin marketplace add Gott50/omp-llm-role           # marketplace
omp plugin install omp-llm-role@gott50-plugins

• npm and git installs shell out to bun install, so bun must be on PATH; marketplace installs need no package manager.
• No runtime dependencies.
• Restart the session after install so the extension module loads.
• Upgrades: omp plugin upgrade omp-llm-role (npm/git); for the marketplace, omp plugin marketplace update gott50-plugins then omp plugin upgrade omp-llm-role@gott50-plugins.
• Dev: omp plugin link ~/Documents/omp-llm-role.

Daily behaviour

• UTC-day trigger. The first omp session of each UTC day rewrites modelRoles (and retry.fallbackChains) to that day's best key-eligible models; later same-day sessions no-op. /refresh-roles forces a run anytime.
• Awaited session-start run. The session-start run is awaited before the session's first prompt is dispatched, so the day's write lands even in a short-lived session (omp -p, a subagent).
• Session model activation. When the day's run switches default and the triggering session still has an empty conversation, the plugin also switches the live session model to the new selector — the first prompt then runs on the freshly ranked pick. Guards: main sessions only, empty conversation, the session actually booted on the previous default selector (an explicit omp --model X is never clobbered), and only on a real change. Disable with activateDefaultOnEmptySession=false.
• Provider in the model display. When the live session model carries a provider pin — a hand-set providerPin, or the automatic pin the providerPinning capability writes — the status line's model segment shows the route next to the model name, e.g. DeepSeek V4.1 Flash @coreweave/fp8. The suffix is applied on session start, refreshed when a fallback switches the model, and re-derived at the start of every turn, so it self-heals after omp silently restores the primary model following a fallback cooldown. An unpinned model's display is unchanged, and the suffix is display-only: it rides the model object's name, so attribution, usage recording and model comparison are unaffected.
• Surgical writes. Only managed role lines, managed chain keys, and the plugin-managed task.disabledAgents names change; comments, blank lines and unknown keys stay byte-identical, and a no-change run writes nothing. ~/.omp/agent/llm-role-state.json snapshots the previous modelRoles block on every write as a rollback aid.
• Usage ledger. The plugin appends one row per assistant turn (the model, the role, the provider, the token counts and the cost) to llm-role-usage.jsonl beside its other state — <project>/.omp in project mode, ~/.omp/agent otherwise. It is append-only, capped at the newest 2000 rows, and best-effort (a recording failure never breaks a session). The explorer's $/task column reads it to learn your own token mix; nothing else uses it.

Roles

| Role | Purpose | |---|---| | default | Main workhorse: strong general coding-agent quality, sane cost | | smol | Fast lightweight model: cheap and quick, still competent | | slow | Most capable model for hard problems; cost and speed as tiebreakers | | vision | Image understanding: vision index dominates | | plan | Planning: reasoning and long-context coherence, priced so the cheapest capable planner wins | | commit | Commit messages: cheap and fast with decent general quality | | tiny | Background tasks (titles, memory): cheapest and fastest wins | | task | Subagents: agentic + tool calling, moderate cost sensitivity | | advisor | Advisor/watchdog: deep reasoning over long context | | designer | Design work: visual/UX judgement on image-capable models — opt-in |

designer is the only non-built-in shipped role and ships disabled: a stock run ranks the nine built-in roles and fetches only their sources. Enable it with

omp plugin config set omp-llm-role config '{"roles":{"designer":{"enabled":true}}}'

The plugin then ranks it and fetches Design Arena. The shipped designer agent is opt-in too: the plugin keeps it in task.disabledAgents until roles.designer.enabled=true, so it is absent from the roster and refuses spawns by default. Its model: "@designer, @default" chain only matters if a user re-enables the agent by hand — with the role disabled the unresolved first entry is skipped and the child runs on @default.

Agent pins. The plugin derives each agent's pinned role from the first @<role> in its model: frontmatter, across the shipped agents/ dir, the user dir (~/.omp/agent/agents/) and the project dir (<project>/.omp/agents/). Disabling a role disables the agents that pin it (they are kept in task.disabledAgents); enabling re-enables them. This sync is not day-gated — it runs on every session start, so enabling a role takes effect on the next session. An agent pinning a role the plugin does not know is left alone, and other task.disabledAgents entries and their order are preserved.

Settings

Settings live in ~/.omp/plugins/omp-plugins.lock.json → settings["omp-llm-role"], deep-merged over the shipped defaults. The plugin also exposes the whole shipped surface in omp's /settings → Plugins → omp-llm-role tab (booleans as toggles, thinking as a picker, required as a comma-separated string); an edit there takes effect on the next /refresh-roles (or the next day-gated session start). The explorer's Settings panel edits the global knobs below (and the capability flags) with the effective value, the shipped default and a reset, and its per-role editor edits the role's description, weights, required set, filters (the five endpoint filters as first-class fields), thinking, λ, providerPin, enabled/locked and the capability flags. Both write the same lock file through the same validated path.

Global knobs:

| Knob | Default | Meaning | |---|---|---| | switchMargin | 0.02 | Value margin a challenger must beat the incumbent by before switching; 0 = always take the best. /refresh-roles --no-margin treats this as 0 for that one run without changing the setting | | priceSwitchFraction | 0.5 | Inside the margin, adopt a challenger at least this fraction cheaper (0.5 = half the $/M); 0 disables | | writeFallbackChains | true | Write per-role fallback chains into config.yml | | fallbackChainDepth | 2 | Fallback models per role chain | | fallbackQualityFloor | 0.9 | A cheaper-or-equal fallback must retain at least this fraction of the chosen model's quality; 0 disables | | activateDefaultOnEmptySession | true | Reapply the new default selector to an empty session's active model | | tokenProfile | unset | { inputTokens, outputTokens } per task for the explorer's $/task column, used when a role has no recorded usage. Display-only; the recorded usage takes precedence. Unset = the column reads — | | errorReporting | off | Opt-in reporting of unexpected plugin errors as GitHub issues on Gott50/omp-llm-role; one of off, ask, auto (see below) | | selection | value | Model selection rule: value picks the best-value model (today's behaviour); cheapest picks the cheapest model that clears qualityFloor. Opt-in — an upgrade changes nothing until you ask | | qualityFloor | 0.9 | In cheapest mode, the fraction of the pool's best model's quality a cheaper candidate must retain; 0 disables the bar, 1 requires the anchor's exact q. Inert in "value" mode |

Per-role knobs (roles.<name>.*):

| Knob | Default | Meaning | |---|---|---| | enabled | true (designer: false) | Opt a role in/out; false drops it from the resolved set and deletes its modelRoles.<role> line | | description | shipped one-liner | Role description | | weights | shipped set | Metric weights; must sum to 1.0 ± 0.01 | | required | shipped set | Eligibility gate (metrics a model must have); independent of weights | | filters.image | false (vision/designer: true) | Restrict the pool to image-capable models | | filters.tools | false | Restrict the pool to models with a tool-capable route | | filters.minContextTokens | 0 (off) | Drop routes whose endpoint context is below this (tokens) | | filters.minOutputTokens | 0 (off) | Drop routes whose endpoint output ceiling is below this (tokens) | | filters.maxPriceUsdPerM | 0 (off) | Drop a model whose thinking-adjusted blend exceeds this ($/M) | | filters.preferredMinThroughput | 0 (off) | Prefer routes at or above this output throughput (tok/s); a below-floor route is deprioritized, not dropped (the soft floor the providerPinning preset sets to 50) | | thinking | shipped level | Thinking level appended to the selector (off…max, auto) | | providerPin | "" (unset) | Pin the role's requests to one OpenRouter provider route (@<slug>, may be tiered like deepinfra/fp8). The role is then priced and gated by that route — its billed price and p50 throughput, not the 1/price² blend — and a model with no matching route is ineligible. A manual pin overrides the automatic pin the providerPinning capability would write | | cacheHitRate | 0 (off) | Assumed cache-hit rate (0–1) for the role's input tokens; each route's effective input price becomes h·cacheRead + (1−h)·input before the 3:1 billed blend and the thinking factor. The 1/price² weight basis stays the listed input price (the router's sort key), so a cheap cache on a lightly-weighted route cannot dominate; a route with no cache-read price keeps its full input price. 0/absent = off (byte-identical ranking) | | lambda | derived | Explicit λ ($ per quality point) override | | selection | global | Per-role selection rule (value/cheapest); overrides the global selection | | qualityFloor | global | Per-role quality floor for cheapest mode; overrides the global qualityFloor | | locked | false | Leave the role alone: rank it but never rewrite its selector or chain (explorer toggle) |

Capability flags (features.<id> global, roles.<role>.features.<id> per role). One named flag per opt-in capability; turning it on applies the plugin's recommended settings for that capability to the role. Every individual knob stays settable, and an explicit knob always beats the flag. Every flag ships OFF — with no flags set, the resolved roles are byte-identical to the shipped defaults.

| Flag | Recommended settings | What it buys | |---|---|---| | endpointCeilings | filters.tools: true, filters.minOutputTokens: 16384 | Routes that cannot call tools and routes whose output ceiling is below 16 384 tokens leave the pool — a reasoning model is not cut off mid-thought | | cachePricing | cacheHitRate: 0.5 | The role is priced on a cache-heavy agent loop's invoice (the endpoint's cacheReadPrice blended in) rather than the sticker price | | providerPinning | preferOwnProvider: true, filters.preferredMinThroughput: 50 | Each request is bound to a provider route that OpenRouter actually serves: the updater walks the model's value-for-money route order (the vfm score, with a soft 50 tok/s floor that deprioritizes a pathologically slow route without dropping it) and probes each candidate with a one-token request bound to it (provider: { only: [slug] }), pinning the first route the request does not reject and dropping the pin (bare selector, OpenRouter's own routing) when no candidate is routable — so a role never 404s on a route the endpoints data still called healthy. The selector gains @<slug>, and the fallback chain carries the primary model on every gate-passing route except the pinned one, then each of the next fallbackChainDepth fallback models on every gate-passing route — each drawn from the role's own ranking below the primary, the cheaper-or-equal ones that keep at least fallbackQualityFloor of its quality first, then the pricier — so a down provider fails over to another provider of the same model before the chain switches models, and a failover never silently multiplies the spend or drops the role onto a much weaker model. The model is priced and measured on its score-max route (its billed price and p50 throughput, so q moves too), falling back to the default blend only when it has no candidate route | | costCap | filters.maxPriceUsdPerM: 10 | The priciest tail of the eligible pool leaves it, so the pick cannot be an expensive outlier |

Precedence is one rule:

effective def = shipped default  ⊕  preset(enabled flags)  ⊕  explicit user keys

A flag is on when roles.<role>.features.<id> is present (it wins either way) and otherwise when the global features.<id> is true — so a per-role false beats a global true. The preset fills only knobs the role does not already set, and an explicit key is merged last, so it always wins. providerPinning's recommended setting is a soft per-model policy (preferOwnProvider), not a hard providerPin: it prices a model on its best route (the max-vfm route — the value-for-money score, with the soft 50 tok/s floor — among the filter-passing, non-degraded, priced routes the account's allowed-providers whitelist permits) — the route's billed price and its p50 throughput, so q moves as well as priceEff — and never makes a model ineligible (the hard providerPin stays available by hand, and overrides the automatic choice). minContextTokens is deliberately not part of endpointCeilings — the complaint it would answer is an endpoint advertising a fraction of the model's own window, which an absolute token floor cannot express, so it stays a hand-set knob.

The flags are settable from every role-authoring surface: --feature <id,...> (comma-separated) on node src/cli/create-role.ts, /create-agent and /project-roles (--list-features lists them on the first two), plus the explorer's Features panel. An unknown flag id or a non-boolean value aborts the run with the offending key and no write.

Error reporting (errorReporting). This knob ships off — installing or upgrading the plugin never starts uploading anything. When something genuinely unexpected happens (a /refresh-roles or authoring command throws, or the updater hits an internal error), the plugin can file one structured report — the plugin version, runtime, platform, the command that failed, the error name/message/stack and a timestamp — as a GitHub issue on Gott50/omp-llm-role. The three values:

• off — never files; behaviour is exactly today's (one warning line).
• ask — shows you the exact text and files it only if you confirm. In a session with no UI (omp -p, RPC, a subagent) it never uploads: it notifies a prefilled new-issue URL instead, so you can submit it yourself.
• auto — files without asking.

The payload is redacted before it leaves the machine (your home directory → ~, token-shaped strings → <redacted>) and carries no config contents, model selectors, prompts or history rows. Filing goes through the GitHub API when a token is available (GITHUB_TOKEN → GH_TOKEN → gh auth token — the token is never stored in a settings file); a token that cannot write issues here, or no token at all, falls through to a prefilled …/issues/new URL (opened in your browser, or printed). A repeated crash comments on the existing open issue instead of opening a new one, and the path is capped (one open issue per fingerprint, one creation per fingerprint per UTC day, 3 per day) so a crash loop cannot spam the tracker. Reporting is best-effort: a failure in it only adds one notification line and never affects the run it reports on. Expected environment conditions (no key budget, a concurrent refresh holding the lock, write conflicts, unavailable ranking data, invalid settings) are never reported. In project mode the project lock file's value applies.

Validation. A run aborts (notify, no write) on: a weight ≤ 0, or a role's weights not summing to 1.0 ± 0.01; a required entry or a weighted metric outside the known-metric set; a thinking level outside off|minimal|low|medium|high|xhigh|max|auto; switchMargin/priceSwitchFraction outside [0, 1]; a non-boolean enabled; a cacheHitRate outside [0, 1]; a selection outside value|cheapest; a non-finite or out-of-range qualityFloor (never silently clamped); a providerPin that is empty or contains @/: (the selector's own delimiters); an unknown capability id (global or per-role) or a non-boolean flag value; a non-boolean preferOwnProvider. weights: null opts a role out; the legacy suffixes.* keys are rejected with a migration hint (moved into roles.<role>.thinking).

Typed writes. omp plugin config set <plugin> <key> <value> stores every value as a string, which the validator rejects for numbers/arrays/booleans, so it is only usable for string-valued keys. Write typed role defs with node src/cli/create-role.ts or the explorer's Export (both validate → merge → backup → atomic write); for a whole-object override use the config escape hatch, omp plugin config set omp-llm-role config '<json>' (JSON-parsed by the plugin). The lock file stores flat dotted keys (roles.slow.weights.code); the reader also accepts nested objects, so an existing nested lock file keeps working and normalizes on the next role write. Invalid settings abort the run with the offending role/key and no write.

The blast radius of selection: "cheapest". The rule is opt-in, so an upgrade changes nothing until you set it. On the 2026-10-20 cache, with the live resolved defs and the as-written pool, the prospective pick at the default bar (qualityFloor: 0.9) differs from today's value leader for 6 of 10 roles: smol, slow, plan, commit and advisor move to deepseek-v4.1-flash-1017, and tiny moves to glm-5.3-flash; default, vision, task and designer are unchanged. The written change set is smaller — the cost-side margin (priceSwitchFraction) holds a role whose challenger is not meaningfully cheaper, so today's dry run leaves default and task as kept-margin. The bar value is the differentiation knob: distinct picks across the ten roles by bar value are 0.80 → 1, 0.85 → 2, 0.90 → 2, 0.95 → 3, 0.98 → 5, 1.00 → 6 — at the default 0.9, nine of the ten roles collapse onto one model. Raise qualityFloor to spread the picks; lower it to concentrate them.

Commands

# in-session (omp)
/refresh-roles [--no-margin]            # force a run now; --no-margin ignores switchMargin for this run
/explore-roles [--port N] [--no-open]   # interactive ranking UI
/create-agent <request> [options]       # agent + role + wiring (free text or flags)
/create-agent --name <n> --purpose <text> [options]   # agent + role + wiring
/remove-agent --name <n> [--scope user|project] [--lock PATH] [--yes] [--dry-run]   # delete an agent and its role
/project-roles [--purpose <text>] [--roles <spec>] [--feature <id,...>] [--force] [--yes] [--dry-run] [--json]   # project-scoped role set

# CLI
node src/cli/llm-role-rank.ts [--top N] [--json] [--out FILE] [--refresh] [--all] [--url URL]
node src/cli/create-role.ts --name <role> --weights m=w,... [--required m,...] [--thinking <level>] [--description <text>] [--image] [--lambda N] [--feature <id,...>] [--list-features] [--lock PATH] [--dry-run] [--json]
node --test tests/

• /refresh-roles — force the daily pipeline now (same path as the session-start run). --no-margin treats switchMargin as 0 for that run only — the README's own "always take the best" reading: every role whose current model is in its candidate pool adopts the pool leader, while a role already holding the leader, a locked or disabled role, and a role whose current model is not in the pool (key-blocked, probe-blocked, unranked) are unchanged. The override is never written to the settings or the state — the next session-start run uses the configured margin again — and priceSwitchFraction is untouched, so a role can move onto a pricier model (the margin is a value band, not a price band). The notification says so (switchMargin ignored for this run (--no-margin)) and the run's history row records the effective margin. A project-scoped run honours the flag too; an unrecognised token prints a usage line and runs nothing.
• /explore-roles — boot the explorer in-process (see below); --port picks a preferred port (default 5177, a busy port falls back to a free one) and --no-open skips the browser.
• /create-agent — create an agent and its role and wire them in one command (see Add a specialist agent).
• /remove-agent — the inverse of /create-agent: deletes the agent .md and its role, then runs the updater in-process so modelRoles.<n> is dropped; refuses shipped default roles (see Remove a specialist agent).
• /project-roles — discover the project's usecase from its own artifacts and apply a project-scoped role set to <cwd>/.omp/config.yml (see Project roles).
• src/cli/llm-role-rank.ts — markdown report to stdout (per-role tables + suggested modelRoles). --top N rows per role (default 10); --json machine payload; --out FILE write instead of stdout; --refresh bypass both caches; --all include opt-in roles (designer) — the default ranks the nine built-in roles only; --url override the llm-stats page URL.
• src/cli/create-role.ts — add/update one role in the settings lock file (validated, backup + atomic write); --dry-run validates without writing, --json prints the payload.
• node --test tests/ — the unit suite.

Add a specialist agent

One command does all of it. Describe the agent in plain language — the architect names it, any benchmark you mention is folded into the weights, and any benchmark link you point at is resolved:

/create-agent i want an agent for writing. use the writing related Benchmarks in the Leaderboard https://llm-stats.com/leaderboards/best-ai-for-writing

A link to any llm-stats benchmark page (llm-stats.com/benchmarks/<id>) resolves to that benchmark; a link to a provider the plugin has never seen is authored into a source declaration — the plugin fetches it, proposes a declaration, shows the coverage and leader, and asks you to confirm before it is saved. A named benchmark takes a decisive share of the weights, so it actually drives the ranking. You do not have to name every benchmark: when you give a purpose, the command also discovers the llm-stats catalog benchmarks relevant to it (a typed judgment through your configured judge role) and folds them in — so a writing agent is ranked on WritingBench, Creative Writing v3, COLLIE and the other writing benchmarks, not just the one you named. Discovery is non-fatal and skippable with --no-discover; an explicit --benchmarks list wins over it.

Discovery is quality-gated: each candidate is assessed on nine axes — coverage (share of the field carrying the metric), dispersion (does it separate models), provider composition, freshness, trust (the share of the payload's entries that are self-reported, so a mostly vendor-submitted source is flagged), modality (the catalog row's modality and the payload's multimodal share — an annotation, never a gate), maintenance (the catalog row's dataset age against a 12-month bar — an annotation, never a gate), provenance (the benchmark's owner and the source's own per-entry org mix, cross-checked against the composition axis — an annotation, never a gate) and cross-source (the zeroeval per-entry price/context against the plugin's own, flagging a divergence past a tolerance — an annotation, never a gate, and never a change to the 1/price² blend) — and a discovered benchmark that fails a gating axis is dropped rather than folded in (a benchmark you name yourself is never dropped, only warned). The focus set is capped at three benchmarks so one keeps a decisive share. The report prints the nine signals and warns per below-bar axis, and warns when the new role's top pick is the same as the default role's (the role adds nothing). The agent file is written before the role, so a partial failure never leaves a dangling role.

The flag form is equivalent and gives you the name explicitly:

/create-agent --name review --purpose "review pull requests for correctness and security"

It fits the weights to the purpose from the archetype table (--list-archetypes prints the ten sets), authors ~/.omp/agent/agents/review.md with model: "@review, @default", writes the validated role into the settings lock file, and runs the updater in-process so modelRoles.review lands in config.yml — nothing to remember. Options: --archetype <id>, --weights m=w,..., --required, --thinking, --tools, --benchmarks m,... (extra metrics to fold in; --list-benchmarks prints the weightable set, including any external benchmark in use), --feature <id,...> (comma-separated capability flags; --list-features prints the capability set), --scope user|project, --body/--body-file, --force (overwrite an existing agent file), --yes (accept a proposed benchmark source without prompting), --no-discover (skip catalog discovery), --dry-run, --json. In the free-text form the flags still apply after the request, e.g. /create-agent i want an agent for writing --dry-run. There is no out-of-session CLI for this path — run /create-agent in an omp session.

By hand, the same three artifacts:

1. Define the role with the plugin's validated write path:

   node ~/.omp/plugins/node_modules/omp-llm-role/src/cli/create-role.ts --name review \
     --weights reasoning=0.30,general=0.24,code=0.20,agents=0.10,price=0.10,throughput=0.06 \
     --required general,price,throughput --thinking high \
     --description "Code review: agentic depth with cost awareness"

2. Author the agent that pins model: "@review, @default" (the routing), or pin an existing agent through task: { agentModelOverrides: { <agent>: "@review" } }.
3. Wire it: /refresh-roles in a session.

The shipped skill omp-llm-role-create-agent is the hand-driven equivalent, for when the body needs real authoring rather than the archetype scaffold.

Remove a specialist agent

One command deletes an agent and its role:

/remove-agent --name review

It deletes the agent .md (user and/or project scope) and the role's lock-file keys, then runs the updater in-process so modelRoles.review is dropped from config.yml — nothing to remember. Options: --scope user|project, --lock PATH, --yes (skip the in-session confirmation), --dry-run. Shipped default roles (e.g. designer) cannot be deleted — the message points at the explorer's "Reset to shipped default". A role created any way (the explorer's "+ new role", create-role.ts, or /create-agent) is removable.

By hand, the same two artifacts:

1. Delete the role from the settings lock file (the explorer's Export, or edit the lock file directly).
2. Delete the agent file: rm ~/.omp/agent/agents/review.md.
3. Wire it: /refresh-roles in a session.

Project roles

/project-roles discovers a project's usecase from its own artifacts (README, package.json, AGENTS.md, docs listing, git log, file tree) and applies a project-scoped role set — the roles the project needs, with agents authored for the new ones — to <cwd>/.omp/config.yml:

/project-roles
/project-roles --purpose "a Python data pipeline repo" --dry-run
/project-roles --roles "data=build pipelines,-vision,default"

It prints the plan (kept / dropped / added roles, each new role's archetype, weights, focus benchmarks and top pick) before writing. --purpose overrides the discovered usecase; --roles overrides the role set (-name drops a shipped role, name keeps or adds one, name=purpose adds a new role); --dry-run writes nothing; without --force an existing project role config is refused.

When a project carries its own omp-llm-role settings entry, the updater scopes the whole run to <cwd>/.omp — the config, the state/history/lock files and the day gate — so the project's modelRoles are refreshed independently of the global ~/.omp/agent/config.yml, which is never written in project mode. A successful apply registers the project, so the explorer's scope switcher lists it (see below).

Explorer

/explore-roles in omp boots a loopback-only web UI (http://127.0.0.1:5177) that answers "why is model X at rank 7 for @slow?" and "what happens if I care more about price than agents?" without editing source and re-running the CLI.

• Scope switcher — the header lists every role-config source: the user-level scope, plus every project where /project-roles was used (the plugin records each project root in a small global registry, and the current session's project is listed immediately). The explorer opens on the session's project when the cwd has a project role config, else user-level. Switching re-reads that scope's roles and universe from disk and points Export at that scope's lock file — a project scope resolves as the project lock merged over the user-level lock, exactly what the updater ranks. A project whose lock file is gone is shown disabled. Switching with unsaved edits asks first.
• Pool toggle — the header switches the ranked pool between raw (the account-agnostic fetch) and as-written (the pool the updater actually writes from: the catalog/tier gate plus your account's allowed-providers whitelist plus the keyed-catalog prune — models your key cannot run are dropped). The as-written pool is the pool the next /refresh-roles ranks, but its leader is not necessarily the model you will get: the hysteresis can keep the incumbent, and in cheapest mode the pick is the cheapest bar-clearing model, not the leader. For the decision itself — the model and chain the updater would write — use the Preview panel, which runs the updater's own candidate set and hysteresis regardless of the toggle. The raw pool shows the account-agnostic ranking and keeps every row — key-blocked models included — with their verdicts. The toggle is best-effort (a failed whitelist fetch degrades to the raw pool) and per-scope, and it never affects Export — Export always writes the role you authored, from either pool.
• Settings panel — one control per global setting (the global knobs table above), plus a toggle per capability flag, each with the effective value, the shipped default, a dirty marker and a reset. It is reachable with no role selected, so you can tune the globals first. activateDefaultOnEmptySession and errorReporting are labelled no visual impact — they change a session behaviour and a side channel, not the ranking or the preview. An edit re-ranks the table and re-runs the preview; Export writes the global half and the role half in one validated, backed-up, atomic write.
• Role tabs — every role the plugin knows: the nine omp built-in roles, the shipped opt-in designer, and any role present only in the lock file. Each tab is tinted by provenance — default (omp built-in), plugin (shipped but opt-in), user (lock-file only) — and a disabled role is muted and dashed. Selecting a disabled role still ranks its models.
• Rank table — every eligible model for the selected role, with value, q, $/M (effective, thinking-adjusted), $/task (what you actually pay per task under your own token mix — see below), tok/s, ctx, a ★ Pareto marker, and a Δ column showing the rank delta against the role's effective def (what the plugin does today).
• Explain panel — click a row for the full decomposition: each weighted metric's raw value → cardinal transform → renormalized weight → contribution, the cost block (λ, effective vs billed price, penalty = λ·price, value = q − penalty), the per-task price breakdown, "why not higher" (the value gap to the model above plus the per-metric target that would close it), the models that dominate it on (price, q), and the Price levers section (below).
• Weight editor — edit the role's description, weights, required set, filters.image, thinking level, and λ override live; the table re-ranks on every change. The five endpoint filters (filters.tools, filters.minContextTokens, filters.minOutputTokens, filters.maxPriceUsdPerM, filters.preferredMinThroughput) are first-class fields here, not only knobs of the capability bundle that recommends them, so one def key has exactly one input. Weights are edited freely (no implicit rescaling): the Σ readout turns red until |Σ − 1| ≤ 0.01, and Normalize rescales in one click.
• Preview panel — for every managed role, the decision the updater would make: the model (from → to), the reason (adopted / switched / switched-cost / kept-margin / kept-eligible / no-current), the value and the best value, the fallback chain, and the config side effects (a disabled role's modelRoles line removed, the task.disabledAgents sync). It is computed on the updater's own candidate set — the catalog/tier gate, your allowed-providers whitelist and the keyed-catalog prune — regardless of the pool toggle, and it is probe-optimistic: it never issues a request, so the written pin can be a later route or absent, and a probe-blocked candidate can leave the pool. Both are labelled, not simulated. A role it could not decide (locked, no eligible candidate, a pin that matches no route) is listed with its reason, so a blank row is not read as "no change". The preview is read-only — it never writes config.yml or the lock file — and it updates live as you edit.
• Enabled toggle — opts a role in or out. Exporting writes enabled into the lock file; the next updater run drops a disabled role from the resolved set and deletes its modelRoles.<role> line, and re-enabling restores it.
• Locked toggle — tells the plugin to leave the role alone: it is still ranked and shown, but the updater never rewrites its selector or fallback chain and never removes it. Enable/disable still applies.
• + new role — creates a role from a template (general/code/price/throughput, Σ 1.0) that you then tune and Export.
• Export — writes the edited roles and the edited global settings into the settings lock file atomically and with a .bak-<timestamp> sibling, touching only the roles and global keys you edited. It is enabled when a role or a global setting is dirty, and it validates both halves before any write (an out-of-range global setting is refused with the offending key). The change takes effect on the next /refresh-roles in any running session (the lock file is re-read from disk on every run). Copy JSON / Download JSON emit the same payload for manual use.
• Hover explanations — every column header, metric name, role tab, and control carries a tooltip explaining its semantics.

The per-task price ($/task). The $/M column is the reported price — the billed 3:1 input:output blend OpenRouter charges, scaled by the role's thinking factor. The $/task column is what you pay per task: your own input:output token mix applied to the role's priced route. The mix comes from the plugin's own recorded usage (the role's median per-turn counts, from the llm-role-usage.jsonl ledger the plugin appends to as you work), falling back to the tokenProfile setting when a role has no recorded usage. It is display-only — the ranking still sorts on $/M — and it reads — when no profile is available (no recorded usage and no tokenProfile set) or the model has no usable route price. A model that is cheaper per token but more verbose can cost more per task; this column is where that shows up.

The price levers. The explain panel's Price levers section makes the price posture visible. The leader flip is the price weight w* at which a cheaper challenger overtakes the current leader, with the challenger named and the distance from your current weight — so you can see whether your price posture binds (a small bump flips the pick) or is slack. It is absent when no cheaper challenger exists. The thinking level table shows which model wins at each level, with the level's price factor — the factor multiplies the price axis, so it penalizes expensive models more, which is why auto favours the cheap model. Both readouts are computed on the pool you are viewing. In cheapest mode weights.price no longer decides the primary choice — the bar does — but it still shapes the value ranking (this panel's value column and the history's value fields) and the fallback chain's ordering.

The UI never reimplements the ranking math: the numbers on screen are exactly the numbers the plugin would use, and the Preview panel reuses the updater's own decision core, so the decision it shows is the decision the run makes. The server binds [IP_ADDRESS] only (no auth) and serves the SPA with no build step and no external requests.

Availability

The plugin keeps only models the OpenRouter key can actually run:

• Tier/budget gate — paid keys with budget get billed variants, free or exhausted keys get :free variants, :batch never. Every candidate must resolve in omp's catalog.
• Keyed-catalog fast path — when the account's OpenRouter setting "Filter the model catalog for API keys" is on, the key-authenticated GET /api/v1/models is a per-key allowlist: allowed candidates skip the probe and key-blocked candidates are pruned from the walk. With the setting off (or the fetch failing) the plugin falls back to the probe walk below — no abort.
• Probe verification — the fallback: every candidate is verified with a one-token completion probe, because the account's OpenRouter allowed-providers privacy whitelist is invisible to the key endpoint and the unauthenticated catalog, and a real request's 404 is the only reliable signal. Blocked candidates are excluded from selection and chains; a role with no clean candidate is left untouched.
• Allowed-providers whitelist — the account's OpenRouter allowed-providers privacy whitelist (openrouter.ai/settings/privacy) is learned once per run (a canary request that bills no tokens) and every model's route pool is narrowed to allowed providers before ranking, so a pin or a fallback hop is never written on a provider the account cannot use. The run notes OpenRouter allowed-providers whitelist active — N providers; M routes pruned (or … whitelist: no whitelist). With no whitelist set — or when the harvest fails — nothing changes.
• Explorer key marking — the explorer overlays every ranked model with a usable / key-blocked / unknown badge derived from the key-authenticated GET /api/v1/models (the account setting "Filter the model catalog for API keys" makes it a per-key allowlist). A hide key-blocked toggle drops blocked rows, and when the setting is off every badge reads unknown with a hint naming the OpenRouter setting. The mark is read-only: it never edits roles, reorders ranks or changes an Export.

Caveats

• Sparse metrics. A missing weighted metric contributes 0 while still occupying its share of the denominator, so weight sparse metrics as differentiators, never as required gates. website, writing and long_context are capability-filled (0.195) instead, so absence is not a penalty for them — with one residual inversion: the fill sits above the bottom of the covered field, so a covered model in the bottom of the field can still score below a model with no data (see docs/dev/quirks.md).
• Provider whitelist. The account's allowed-providers whitelist blocks many models, so the report's #1 and the written selector can differ — judge weights on the reachable pool. The updater prunes routes to the whitelist (see Availability), but the report has no token and is not whitelist-aware.
• Provider pinning opts out of auto-Exacto. A providerPin — or the automatic pin the providerPinning capability writes — puts a trailing @<slug> on the selector, which omp applies as OpenRouter provider: { only: [slug] } routing. only is exclusive, so a pinned role's requests bypass OpenRouter's automatic Exacto tool routing (which routes tool calls by tool-call correctness) and run on the pinned route. Pin only where you know the endpoint you want; leave tool-heavy roles on the default routing.
• Model concentration. The cheapest pick concentrates on one vendor; the fallback chains mitigate it.

Licence

MIT — see LICENSE.

Documentation

Maintainer docs (architecture, data sources, scoring, writes/state, explorer internals, releasing, agent authoring, testing) live in docs/dev/README.md; release history is in CHANGELOG.md.