@cardinal4/opencode-plugin-litellm

v2.0.0

Published

5 days ago

OpenCode plugin for LiteLLM proxy support with auto-detection and dynamic model discovery

0High
0Medium
0Low

cardinal4

opencode litellm lite-llm plugin llm-proxy openai-compatible

opencode-litellm

Drop-in LiteLLM provider for OpenCode with zero configuration.

Auto-detect a running LiteLLM proxy, pull every model from /v1/models, and register them in OpenCode. No model lists to hand-maintain. No restart loops. No surprises.

Quickstart · Configuration · How it works · FAQ · Contributing

npm package: @cardinal4/opencode-plugin-litellm · GitHub repo: cardin/opencode-litellm

✨ Why this plugin?

Maintaining a models block in opencode.json for every model your LiteLLM proxy exposes is a chore — every new entry in your model_list means a config edit, a restart, and a context-switch.

opencode-litellm removes that loop entirely. It uses OpenCode V2's provider transform API, queries your LiteLLM proxy at startup, and registers the discovered models in memory. The result: every model in litellm config.yaml shows up in OpenCode's picker automatically.

🚀 Quickstart

// 1. Add to opencode.json — OpenCode installs the plugin from npm automatically
{
  "$schema": "https://opencode.ai/config.json",
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "http://localhost:4000/v1"
      }
    }
  ]
}

# 2. Start LiteLLM (if it isn't already)
litellm --config config.yaml --port 4000

# 3. Run OpenCode — every model in your LiteLLM model_list is now available.
opencode

🎯 Features

| | | |---|---| | 🔍 Auto-detection | Probes localhost:4000, :8000, :8080 and adopts the first responsive proxy. | | 📡 Dynamic discovery | Queries /v1/models so your OpenCode model picker always reflects your live model_list. | | ⚡ Instant startup (SWR) | Discovered models are cached on disk and loaded synchronously — startup never blocks on the network. A background refresh on new sessions keeps the cache fresh; entries expire after 7 days. | | 🏷️ Smart formatting | Turns anthropic/claude-3-5-sonnet into Claude 3.5 Sonnet in the picker — handles versions, sizes, quantizations, and brand-cased names like gpt-4o. | | 🧠 Modality-aware | Enriches /v1/models entries with /v1/model/info (mode, token limits, capability flags) and hides embedding / image / audio models from the picker. | | 💵 Real pricing | Maps input_cost_per_token / output_cost_per_token (and cache read/write costs) from /v1/model/info into OpenCode's cost field, so the picker and /cost show what the proxy actually bills instead of $0.00. Models LiteLLM has no price for are left unpriced, not falsely marked free. | | 🧩 Reasoning-effort variants | When LiteLLM reports per-model effort support (supports_low_reasoning_effort, …), the plugin surfaces each level as a picker variant automatically. | | 🔐 Auth-aware | Honours LITELLM_API_KEY / LITELLM_MASTER_KEY env vars, plugin options.apiKey, or the active key connection from OpenCode's /connect. | | 🌐 Gateway-friendly | Supports plugin options.headers for proxies behind Cloudflare Access or other API gateways requiring extra HTTP headers. | | 🧩 Splittable catalog | includeModels / excludeModels (glob patterns) let one LiteLLM proxy be divided into several OpenCode providers — e.g. by naming prefix — without hand-maintaining a model list. | | 🎚️ Capability overrides | modelCapabilities forces or retracts per-model capability flags (supports_vision, supports_function_calling, …) when /v1/model/info is unavailable or disagrees with your deployment. | | ⏱️ Non-blocking startup | Health checks fail fast (3 s); discovery fetches are capped at 15 s (configurable via LITELLM_REQUEST_TIMEOUT_MS) for slow remote proxies. | | 📝 Service-side logging | Plugin logs are captured by the OpenCode V2 service logger instead of being written into the terminal UI. | | 🤝 Non-destructive merge | Only adds models you don't already have configured. Hand-curated entries are preserved verbatim. | | 🪶 Zero runtime deps | Only depends on @opencode/plugin. No build step, no bundler. | | 🔒 TypeScript strict | Strict-mode compiled, fully typed public API. |

⚙️ Configuration

Minimal config (recommended)

Point at your LiteLLM proxy — the plugin discovers all models automatically:

{
  "$schema": "https://opencode.ai/config.json",
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "http://localhost:4000/v1"
      }
    }
  ]
}

Custom URL or authentication

You do not need to list any models — the plugin still discovers them from /v1/models automatically. Use this form only when you need to point at a non-default URL or pass an API key:

{
  "$schema": "https://opencode.ai/config.json",
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "http://litellm.internal.example.com/v1",
        "apiKey": "{env:LITELLM_API_KEY}"
      }
    }
  ]
}

That's the whole config — every model in your LiteLLM model_list will appear in the picker.

Example: governed upstream route with Tuning Engines

If your team routes model traffic through Tuning Engines for policy, traces, approvals, and usage visibility, add it as an OpenAI-compatible upstream in your LiteLLM config. The plugin will discover the alias from LiteLLM just like any other model_list entry:

model_list:
  - model_name: te-gpt-5.4-mini
    litellm_params:
      model: openai/gpt-5.4-mini
      api_key: os.environ/TUNING_ENGINES_API_KEY
      api_base: https://api.tuningengines.com/v1

Then expose the key to LiteLLM and keep your OpenCode config pointed at the same LiteLLM proxy:

export TUNING_ENGINES_API_KEY=sk-te-...
litellm --config config.yaml --port 4000
opencode

OpenCode and this plugin still own model discovery and picker wiring. Tuning Engines sits on the upstream model route as the governed control plane.

Overriding or curating individual models (optional)

If you want to rename a model in the picker or otherwise hand-curate metadata, add it under models. The plugin preserves your entries verbatim and only injects discovered models whose key isn't already defined:

{
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "http://litellm.internal.example.com/v1",
        "apiKey": "{env:LITELLM_API_KEY}"
      }
    }
  ],
  "providers": {
    "litellm": {
      "models": {
        "openai/gpt-4o": {
          "name": "GPT-4o (curated)"
        }
      }
    }
  }
}

Here, openai/gpt-4o keeps your custom name; every other model from the proxy is still discovered and added automatically.

Reasoning models and effort variants

All discovered models — reasoning-tier included — register under your single LiteLLM provider and are invoked through /v1/chat/completions.

If LiteLLM reports per-model reasoning-effort support (e.g. supports_low_reasoning_effort, supports_medium_reasoning_effort, supports_high_reasoning_effort in model_info), the plugin automatically surfaces those as OpenCode variants under the discovered model. Each variant sets reasoningEffort to the reported level, so you can switch between effort levels from the model picker without hand-curating every entry.

Note: OpenAI's reasoning-tier models (gpt-5, o1, o3, o4) reject requests that combine reasoning_effort with function tools on /v1/chat/completions. If you hit that error, fix it on the LiteLLM side — e.g. enable the Responses API for that model (use_responses_api: true in its litellm_params) — or leave reasoningEffort unset for that model in OpenCode. See the FAQ entry below.

Authentication

If your LiteLLM proxy requires a master key, expose it via either approach:

| Method | Example | |---|---| | Env var | export LITELLM_API_KEY=sk-... | | Env var (alias) | export LITELLM_MASTER_KEY=sk-... | | Config | "options": { "apiKey": "{env:LITELLM_API_KEY}" } | | OpenCode /connect | Run /connect, search for your litellm provider entry, and paste the key |

The env var path lets you commit opencode.json without leaking secrets. When /connect exposes an active key connection for the provider, the plugin resolves it through OpenCode V2's integration API for discovery; OpenCode applies the same connection to model requests.

Slow proxies (`LITELLM_REQUEST_TIMEOUT_MS`)

Each discovery request (/v1/models, /v1/model/info) is capped at 15 s by default. Proxies with many database-defined models — or a gateway in front of LiteLLM — can legitimately take longer. Raise the budget with:

export LITELLM_REQUEST_TIMEOUT_MS=60000

The overall discovery cap scales with it (max of 20 s and the request timeout + 5 s), so a slow proxy never blocks startup indefinitely but isn't cut off mid-flight either. Invalid values fall back to the default.

Custom headers (Cloudflare Access, API gateways)

If your LiteLLM proxy is behind Cloudflare Access or another gateway that requires extra HTTP headers, use plugin options.headers:

{
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "https://litellm.internal.example.com/v1",
        "apiKey": "{env:LITELLM_API_KEY}",
        "headers": {
          "CF-Access-Client-Id": "{env:CF_ACCESS_CLIENT_ID}",
          "CF-Access-Client-Secret": "{env:CF_ACCESS_CLIENT_SECRET}"
        }
      }
    }
  ]
}

These headers are included in every request the plugin makes during model discovery (health check and /v1/models). To obtain a Cloudflare Access Service Token, follow the Cloudflare docs.

Splitting one proxy into multiple providers (`includeModels` / `excludeModels`)

If your LiteLLM catalog mixes naming conventions from different teams or environments (e.g. prod/* and staging/*), you can point two OpenCode providers at the same proxy and have each one surface only its slice:

{
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "providers": {
          "litellm": {
            "name": "Prod",
            "baseURL": "http://localhost:4000/v1",
            "includeModels": ["prod/*"]
          },
          "litellm-staging": {
            "name": "Staging",
            "baseURL": "http://localhost:4000/v1",
            "includeModels": ["staging/*"],
            "excludeModels": ["staging/*-canary"]
          }
        }
      }
    }
  ]
}

includeModels is evaluated first — only ids matching at least one pattern are kept. Omit it to keep everything.
excludeModels is evaluated after and always wins, even over includeModels.
Patterns support only * (any run of characters); everything else is matched literally, so dots in ids like gpt-4.1 need no escaping.
Filtering happens before the on-disk cache is written, so each provider's cached view respects its own filters.

Correcting capability flags (`modelCapabilities`)

Model classification (tool-call badge, attachments, reasoning, input modalities) leans on the capability flags LiteLLM reports via /v1/model/info. If your proxy doesn't expose that endpoint, or reports a flag that doesn't match your deployment, override flags per model id:

{
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "http://localhost:4000/v1",
        "modelCapabilities": {
          "te-gpt-5.4-mini": { "supports_function_calling": true },
          "openai/gpt-4o": { "supports_vision": false }
        }
      }
    }
  ]
}

Overrides apply on top of whatever the proxy reports, with an explicit false winning — a flag the proxy never reported can be forced on just the same.
Keys are exact model ids as they appear in /v1/models (not globs).
Overridden flags flow into the picker exactly like natively reported ones, and the adjusted view is what gets persisted to the model cache.
Changing modelCapabilities (or includeModels/excludeModels) starts a fresh discovery on the next start — the cache is scoped by that config — so the picker reflects the new flags immediately.

🔧 How it works

sequenceDiagram
    participant OC as OpenCode
    participant Plugin as opencode-litellm
    participant Cache as disk cache
    participant LL as LiteLLM proxy

    OC->>Plugin: setup(ctx)
    alt plugin options.baseURL configured
        Plugin->>Plugin: use configured baseURL
    else not configured
        Plugin->>LL: probe :4000, :8000, :8080 (GET /v1/models, 3 s fail-fast)
        LL-->>Plugin: 200 OK on one
        Plugin->>Plugin: auto-create provider entry
    end
    Plugin->>Cache: read SWR cache (no network yet)
    alt cache hit
        Cache-->>Plugin: cached models
        Plugin->>OC: register provider transform (instant startup)
    else cache miss
        Plugin->>LL: GET /v1/models (with auth if set)
        Plugin->>LL: GET /v1/model/info (best-effort)
        LL-->>Plugin: { data: [...models] } + per-model info
        Plugin->>Plugin: enrich models, hide non-chat (embedding/image/audio)
        Plugin->>Plugin: format names, infer modalities + limits + pricing
        Plugin->>OC: register provider transform
        Plugin->>Cache: persist for next startup
    end
    Note over Plugin,Cache: on session.created, revalidate cache in<br/>the background (throttled to 5 min)
    Plugin->>OC: provider.reload() after refresh
    OC->>OC: render model picker with all discovered models

On OpenCode startup the plugin's V2 setup(ctx) entrypoint runs.
If plugin options.baseURL is set, it is used. Otherwise common ports are probed — that probe is the only health check (3 s fail-fast per port).
With a configured baseURL the proxy is not contacted during startup unless the cache is cold; the discovery fetch itself fails fast if the proxy is unreachable.
Fast path: if a fresh on-disk cache exists (≤ 7 days old), its models are merged in synchronously — startup never waits on the network.
Cold path: /v1/models and /v1/model/info are fetched in parallel. Models are enriched with info metadata (mode, token limits, capability flags, per-token pricing — /v1/models omits these for database-defined models) and converted into OpenCode model entries with formatted name, inferred modalities, and cost (USD/1M tokens, converted from LiteLLM's USD/token). Non-chat models (embedding / image / audio) are excluded from the picker.
Discovered models are merged on top of any user-defined ones — never overwriting them — and persisted to the cache.
On every session.created event the cache is revalidated in the background (throttled to once per 5 minutes); provider.reload() makes successful refreshes visible without restarting OpenCode. The whole cold path is capped by a 20 s timeout so a slow proxy never blocks boot.

📋 Requirements

OpenCode 2.x with plugin support (@opencode/plugin ^2.0.10)

A running LiteLLM proxy:

pip install 'litellm[proxy]'
litellm --config config.yaml --port 4000

Node.js ≥ 20 (or Bun ≥ 1.0)

📦 Compatibility matrix

| LiteLLM version | OpenCode version | Status | |---|---|---| | ≥ 1.40 | 2.x | ✅ Supported | | 1.30 – 1.39 | 2.x | ⚠️ Should work (older /v1/models schema) | | < 1.30 | any | ❌ Unsupported |

❓ FAQ

Once LiteLLM exposes the model (restart or hot-reload LiteLLM if you edited its config.yaml), the plugin picks it up automatically: whenever you open a new session (and at most every 5 minutes), it re-queries the proxy, updates the on-disk cache, and reloads the provider catalog. The new model appears without restarting OpenCode. To force an immediate cold refetch, delete the cache directory (~/.cache/opencode-litellm/, or $XDG_CACHE_HOME/opencode-litellm/).

Yes. Set plugin options.baseURL to your remote URL and optionally set options.apiKey. Auto-detection only probes localhost, but explicit configuration works against any URL.

OpenCode starts normally either way. With a warm on-disk cache, the previously discovered models are served from the cache — no network call — so the picker works as usual; only the background refresh is skipped until the proxy is back. With a cold cache (first run, or after the 7-day expiry), the plugin logs a warning and you won't see LiteLLM-discovered models until the proxy is reachable again.

No. The merge is additive: anything you've already defined under providers.litellm.models is preserved exactly as-is. Discovered models are only added if their key isn't already present.

Yes — anything in your LiteLLM model_list shows up, including Ollama, Bedrock, Azure, OpenAI, Anthropic, Google, etc. That's the whole point of LiteLLM.

Cloudflare Access intercepts requests before they reach LiteLLM, so a plain Authorization: Bearer header isn't enough. Create a Cloudflare Access Service Token and pass the credentials via plugin options.headers:

{
  "plugins": [
    {
      "package": "@cardinal4/opencode-plugin-litellm@latest",
      "options": {
        "baseURL": "https://litellm.your-company.com/v1",
        "headers": {
          "CF-Access-Client-Id": "{env:CF_ACCESS_CLIENT_ID}",
          "CF-Access-Client-Secret": "{env:CF_ACCESS_CLIENT_SECRET}"
        }
      }
    }
  ]
}

The options.headers map works for any gateway that requires extra HTTP headers — not just Cloudflare.

This error comes from OpenAI: their reasoning-tier models (gpt-5, o1, o3, o4) refuse function-tool calls on /v1/chat/completions when reasoning_effort is set. The plugin registers every discovered model through the chat-completions path, so fix this on the LiteLLM side:

Enable the Responses API for that model in your LiteLLM config (e.g. use_responses_api: true in its litellm_params), or
Leave the model's reasoningEffort unset in OpenCode (don't pick a reasoning-effort variant for it).

If your model id doesn't look like a reasoning model to LiteLLM (e.g. you renamed it), also check that its model_info in litellm config.yaml carries the right supports_* flags.

🛠️ Development

git clone https://github.com/cardin/opencode-litellm.git
cd opencode-litellm
npm install
npm run typecheck
npm test

The project is intentionally tiny:

src/
├── index.ts                    # Public exports
├── types/index.ts              # LiteLLM API types
├── utils/
│   ├── litellm-api.ts          # health check, discovery (/v1/models + /v1/model/info), auto-detect
│   ├── format-model-name.ts    # name formatting, categorization
│   ├── model-cache.ts          # stale-while-revalidate on-disk model cache
│   ├── model-filter.ts         # includeModels/excludeModels glob filtering
│   └── model-capabilities.ts   # per-model capability flag overrides
└── plugin/
    └── index.ts                # V2 provider transform, enrichment, filtering, refresh

test/                           # vitest suite for the pure logic

See CONTRIBUTING.md for the full contributor workflow.

🗺️ Roadmap

[ ] Optional cost/latency overlay using LiteLLM's /spend and /health endpoints
[ ] V2 session-context hook for injecting LiteLLM routing tags / fallbacks

Have an idea? Open an issue.

🙏 Acknowledgements

Inspired by opencode-lmstudio by @agustif — the architectural blueprint for OpenCode model-discovery plugins.

Built on top of LiteLLM by the BerriAI team and OpenCode by the OpenCode contributors.

📄 License

If this project saved you time, consider giving it a ⭐ on GitHub.