@flexapp/opencode-plugin-litellm

LiteLLM provider plugin for opencode.

• Model discovery — pulls every chat model your LiteLLM proxy serves and registers them as one litellm provider, including the models behind wildcard passthroughs like openai/*.
• Readable names — Anthropic · Claude Opus 5 instead of anthropic/claude-opus-5, grouped by serving vendor in the picker, for models your proxy gives a display_name. Others keep their raw id.
• Real context windows — publishes each model's window so opencode compacts at the right point instead of treating it as unknown.
• Pricing and capabilities — maps per-token cost, reasoning, tool_call and temperature through from LiteLLM, so session cost is no longer $0.00 and models that can't take tools aren't offered them.
• Prompt caching — sends a stable cache key, and makes Anthropic-backed routing groups eligible for cache breakpoints they'd otherwise miss.
• Auth with Keychain fallback — uses opencode auth login first, then LITELLM_API_KEY / LITELLM_PROXY_API_KEY, then the macOS Keychain entry from Flex litellm login if present. Each source works for both model discovery and chat requests.
• Session affinity — pins each conversation to one upstream deployment.
• Route visibility — records which upstream actually served each request (medium-tier → Claude Sonnet 5) and can show it in the TUI sidebar.

Install

// opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@flexapp/opencode-plugin-litellm"],
  "provider": {
    "litellm": {
      "options": {
        "baseURL": "https://your-litellm-host/v1"
      }
    }
  }
}

Then add your key:

opencode auth login -p litellm

Fallbacks if that isn't set:

export LITELLM_API_KEY=sk-...   # or LITELLM_PROXY_API_KEY
litellm login                   # macOS Keychain via litellm-helper

That's it — models appear on next start.

Configuration

| What | Where | | --- | --- | | Proxy URL | provider.litellm.options.baseURL, or the LITELLM_BASE_URL env var | | API key | opencode auth login -p litellm (primary); else LITELLM_API_KEY / LITELLM_PROXY_API_KEY; else Keychain flex.litellm.virtual-key (litellm login) |

There is no default URL — it's deployment-specific. Without one the plugin stays inert and logs how to set it, rather than guessing at an endpoint.

baseURL should point at the OpenAI-compatible root (ending in /v1). The admin endpoint used for discovery (/v1/model/info) is derived from it.

Discovery deliberately uses /v1/model/info rather than /v2. Only v1 expands wildcard passthroughs into the concrete models behind them; v2 reports one opaque openai/* row, which isn't callable — often the difference between a handful of aliases and hundreds of concrete models.

Upgrading

opencode pins the plugin version it resolved on first install and never re-checks it — this is true of a plain spec and of @latest, and opencode plugin … --force only rewrites config. To pick up a release:

rm -rf ~/.cache/opencode/packages/@flexapp/opencode-plugin-litellm*

and restart. Or pin a version in opencode.json / tui.json (@flexapp/[email protected]) and bump it deliberately.

Optional: route in the sidebar

The TUI half shows the current upstream in the sidebar. It's a separate entry point, listed in tui.json:

{
  "plugin": ["@flexapp/opencode-plugin-litellm"]
}

Use the bare package name — do not append /tui. opencode resolves the ./tui subpath export itself; writing it out makes npm/bun read /tui as a filesystem path and the auto-install fails.

Nothing extra to install. The TUI entry ships prebuilt with @opentui/solid and solid-js left external, so opencode supplies its own runtime for them.

How the model list is organized

A LiteLLM proxy can serve hundreds of models under one provider. Two things make that list navigable.

family groups entries in the picker, set to the serving vendor — OpenAI, Vertex, xAI, Anthropic, Bedrock, Fireworks, plus Router.

Vendor is the axis that matters most here, since the same model is often served by several upstreams.

name uses the display name returned by LiteLLM's model info endpoint (display_name / name), falling back to the model id if none is provided. Many proxies set no display name, in which case entries read as their raw id — anthropic/claude-opus-5 rather than Claude Opus 5. That is also the quickest way to tell a LiteLLM entry from a same-named model on a native provider.

Model ids are otherwise untouched — display names, grouping, and the metadata below don't affect them, so -m litellm/medium-tier and any model set in config keep working. The one exception is the Anthropic-backed routing groups described under Prompt caching, which are registered under an -anthropic suffix.

Context limits

opencode sizes its context window from limit.context and compacts when it fills. A model published without one falls back to opencode's unknown-model default of 0 and compacts almost immediately, so the plugin maps the window through from LiteLLM's max_input_tokens / max_output_tokens.

The field names line up with opencode's exactly — spot-checked against its bundled models.dev catalog, they reproduce limit.context / limit.output for gpt-5 (272000/128000), claude-sonnet-4-5 (200000/64000), gemini 1M (1048576/131072) and gpt-5-chat (128000/16384).

Two cases get special handling:

• Several deployments behind one alias, whether capacity replicas or a routing group fanning across vendors: limits are the floor across them, so the window is never larger than the smallest deployment a request could land on.
• max_output >= max_input, which cannot be true of a real model. LiteLLM's model_prices data defaults max_tokens to max_input_tokens where the real cap is unknown, and the *-tier routing groups are usually among them. Propagating that would have opencode request an output size the upstream rejects, so those keep their context and take a conservative 32k output.

Aliases with no max_input_tokens are published without a limit rather than given an invented one, leaving opencode on its own fallback.

To check what a deployment resolves to:

opencode models litellm --verbose

Pricing and capabilities

LiteLLM publishes per-token prices and capability flags on the same /v1/model/info response used for discovery. The plugin maps them to opencode's cost, reasoning, tool_call and temperature.

Cost needs a unit conversion: opencode prices per million tokens, LiteLLM per single token. Verified against opencode's bundled models.dev catalog — claude-opus-4-5 5e-6/2.5e-5 becomes 5/25, and gemini-2.5-pro 1.25e-6/1e-5 becomes 1.25/10, tiered pricing included. The multiply is rounded, since 4e-7 * 1e6 is 0.39999999999999997 in binary floating point and many LiteLLM price fields hit a case like it.

Where deployments behind one alias disagree, cost takes the highest — the mirror of the floor used for limits, since a request may land on any backend and understating cost under-reports spend. Multi-vendor routing groups often differ by about 10% between legs (e.g. Bedrock vs OpenAI).

Capability flags are tri-state — true, false, or absent. Absent means "not recorded", not "unsupported": many rows omit supports_function_calling entirely, including models that plainly do support tools. A flag is only published when LiteLLM states it explicitly, leaving opencode on its own default otherwise. Inferring false from silence would disable tool calling across a large share of the catalog.

This matters most for tool_call, which opencode defaults to true. Aliases that are explicitly not tool-capable (for example openai/gpt-5-chat and its -latest variants) look like ordinary flagship models in the picker, so without the flag they'd fail on first tool use.

Aliases with no pricing at all (video, legacy completions, etc.) are published without cost rather than with zeros, which would read as "free" instead of "unknown".

Non-chat models

mode is not a reliable filter: image, TTS, embedding and realtime rows often report mode: "chat", including dall-e-3, whisper-1 and text-embedding-3-small. Nor is supported_openai_params — the generic parameter list those share is simply OpenAI's default, and gpt-4o has it too.

What does separate them is a missing max_input_tokens. That isolates unusable entries such as sora-2* (video), davinci-002 / babbage-002 / gpt-3.5-turbo-instruct* (legacy /v1/completions, no chat endpoint) and container (not a model). Those are skipped with a warning naming them.

This doubles as a correctness fix — no window meant opencode's default of 0, so those entries compacted on the first turn anyway.

Prompt caching

Two separate mechanisms, neither of which reaches an openai-compatible provider by default.

Cache key. opencode sends promptCacheKey (the session id) only for providers it knows cache implicitly — @ai-sdk/openai, azure, xai, mistral — or when setCacheKey is set. @ai-sdk/openai-compatible is in neither group, so the plugin sets provider.options.setCacheKey = true. An explicit false in your own config still wins.

Cache breakpoints. Anthropic bills prompt caching only where the request carries explicit cache_control markers; OpenAI caches long prefixes implicitly. opencode does send them, but decides who needs them by name: it tests whether the model's id or config key contains claude or anthropic.

Aliases like anthropic/claude-opus-5 pass that test by luck of naming. Routing groups don't — a group such as reasoning-tier may serve Claude but say nothing about it in the id — so every turn rewrote the cache instead of reading it. Measured on a ~12k-token prefix against the same model and deployment:

| | cache write | cache read | | --- | --- | --- | | without breakpoints | ~12k | 0 | | with breakpoints | 0 | ~12k |

Cache reads bill at roughly a tenth of input, so missing breakpoints is a permanent overcharge on Anthropic-backed routing groups.

Such groups are therefore registered under an -anthropic suffix (reasoning-tier-anthropic), which satisfies opencode's test. The id field pins the wire model name back to the real alias, so the request still goes out as reasoning-tier. This is the one case where the opencode-facing model id differs from the LiteLLM alias — -m litellm/reasoning-tier-anthropic.

Only the config key is safe to steer here: id becomes the wire model name and npm selects the SDK client, so changing either breaks the request outright.

Proxy-side caveats

Your proxy may set enable_anthropic_prompt_caching, which is meant to inject breakpoints server-side. In practice it may not fire even when enabled: the same warm prefix that reads from cache with inline cache_control can still read 0 via the proxy's own cache_control_injection_points. In litellm_logging.py the cache-control hook is only selected when no other CustomPromptManagement logger is registered first, which is a common cause.

Separately, the proxy hook stands down for any provider other than anthropic and bedrock, so a Vertex-served Claude would never be covered by it regardless.

Client-sent breakpoints are the path measured working, and LiteLLM defers to them when present, so the two don't double up.

Route recording

After each response the plugin writes the resolved route to session.metadata.litellmRoute, built from LiteLLM's response headers:

| Header | Meaning | | --- | --- | | x-litellm-model-group | what the client asked for (medium-tier) | | x-litellm-model-name | upstream that served it (anthropic/claude-sonnet-5) | | x-litellm-model-id | deployment row id, used as a fallback |

x-litellm-model-name is preferred since it names the upstream directly and works on the first request, before the deployment index is populated.

Session affinity works by sending opencode's stable session id as x-litellm-session-id, which lets LiteLLM's session_affinity engage instead of reshuffling deployments every turn. That has to be enabled proxy-side too (model_group_affinity_config).

Troubleshooting

An opaque UnknownError with a ref hash at startup, or "No api key passed in". Look earlier in the output for the real cause:

[litellm-plugin] no LiteLLM URL configured — set provider.litellm.options.baseURL …
[litellm-plugin] no stored key for provider "litellm" — run `opencode auth login -p litellm`

When either is missing, discovery bails, so no litellm/* model exists and opencode reports a generic error that hides the specific warning.

Sidebar shows no requests yet. The route is only recorded once a response comes back. If it persists past the first reply, look for [litellm-plugin] session metadata update failed.

The label changes between turns. Affinity isn't engaging proxy-side; check model_group_affinity_config.

Notes for maintainers

Six non-obvious things, each of which caused a silent failure:

• The config key becomes the model's opencode-facing id, while api.id — used as the wire model name in languageModel(api.id) — comes from the model's own id field, falling back to the key. That split is what lets a model be registered under one name and called by another. Renaming api.id or npm instead would break the request: the former sends a model the proxy doesn't have, the latter picks a different SDK client and wire format.

• tool_call defaults to true in opencode (C.tool_call ?? _?.capabilities.toolcall ?? !0), so a model that can't take tools is offered them unless the flag is explicitly published.

• chat.headers receives provider as a bare Provider (id at the top level) at runtime, even though the type says ProviderContext ({ source, info, options }). providerIdOf() accepts both. Reading only provider.info.id silently disables the plugin — the hook returns early on every request, so no session id is ever sent and no route is recorded.

• Use the injected client for session updates, not a client built from serverUrl. serverUrl advertises http://localhost:4096, but under opencode run the server is in-process and never binds that port, so requests there fail with "Unable to connect".

• client.session.update()'s generated body type omits metadata, but the server accepts and persists it. Hence the cast at the call site.

• The auth hook registers the litellm provider unconditionally, so the config hook must never return without leaving a well-formed provider entry behind (ensureProvider()). A litellm provider with no models object makes opencode's own getModel() throw undefined is not an object (evaluating '$.models') — for every provider, so a misconfigured LiteLLM takes unrelated models down with it.

Also: the auth file is read from $XDG_DATA_HOME/opencode/auth.json, falling back to ~/.local/share. Hardcoding the latter breaks any profile that sets XDG_DATA_HOME.

Packaging constraints

Five rules the plugin loader and packaging impose, none of them obvious:

• A default export shaped { server } or { tui } is required. A bare named export (export const LiteLLMPlugin) works for plugins loaded from a directory, because those are scanned for named exports — but once packaged, the loader throws must default export an object with server(). This is easy to miss precisely because local development works fine.

• exports must be keyed ./server and ./tui, not .. The loader looks up exports["./" + kind], and only falls back to main for the server entry. A package with just exports["."] resolves nothing for tui.

• The TUI entry must be importable on its own. Two ways to satisfy this, and mixing them silently breaks the plugin:

  • Ship a .tsx entry and declare @opentui/solid / solid-js as real dependencies. opencode transforms .tsx through @opentui/solid's babel loader, which has to resolve those from the installed package.
  • Ship a prebuilt .js entry with them left external (what this package does). The bare imports survive, and opencode's runtime plugin rewrites them to its own in-process modules — so nothing has to be installed.

  Shipping .tsx with those as optional peers satisfies neither: the transform has nothing to resolve, and the plugin never even appears in the Plugins dialog. That shipped as 0.3.1.

• Never compile the JSX with tsc's jsxFactory. Solid has no runtime JSX factory; createElement in @opentui/solid is the universal-renderer primitive taking a single tag name. React-shaped createElement(View, props, …) makes the reconciler throw Unknown component type inside the slot render, which opencode swallows per-slot — so the widget is blank while the Plugins dialog still reports it active. That shipped as 0.2.0/0.3.0. Build the node tree by hand instead.

• dist/ is gitignored but shipped, and exports["./tui"] resolves to it, so nothing in a normal edit-run loop regenerates it. A stale build keeps running silently: a debug line deleted from src/tui.ts stayed visible in the widget for days because the artifact predated the edit, and a fresh clone has no dist/ at all until someone builds by hand. prepare runs the build on a plain install and for git dependencies, which covers both. prepublishOnly is typecheck-only because prepare already builds during publish. The tradeoff is that a git-dependency install now needs bun on PATH; installs from the npm tarball are unaffected, since prepare doesn't run there and dist/ is already in files.

Note also that file:*.tgz plugin specs do not load — verified with a minimal plugin that loads correctly from a directory spec and from npm, but silently does nothing when referenced as a local tarball. Test packaging with a directory spec or a real registry install, not a tarball path.