@sidshaytay/pi-llama-swap

Choose models from your llama-swap server in pi, without maintaining a model catalog.

Quick start

Requires an installed pi and a reachable llama-swap server exposing /v1/models. This repository tests against pi 1.2.0; current pi requires Node.js 22.19 or later.

1. Install the extension:

   pi install npm:@sidshaytay/pi-llama-swap

2. If your server is elsewhere than http://localhost:8080, merge this into ~/.pi/agent/settings.json:

   {
     "llama-swap": { "url": "http://your-server:8080" }
   }

   Replace your-server with the hostname or IP address of your llama-swap server. Use its base URL, without /v1.

3. Start pi, or run /reload in an existing session, then open /model. Your chat models appear under the llama-swap provider.

Discovery requests only /v1/models, so it does not load sleeping models. Selecting a model and sending a prompt can cause llama-swap to load it. If the server is offline, pi still starts; reopen /model after the server is available to refresh discovery.

Check the model catalog

pi --list-models

Look for the llama-swap provider and the model IDs returned by your server. To try the extension from the repository root for one invocation:

pi -e ./packages/pi-llama-swap/extensions/llama-swap.ts

Configure your server connection

Put settings under "llama-swap" in pi's personal settings.json. Its default location is ~/.pi/agent/settings.json; PI_CODING_AGENT_DIR selects a different agent directory. Environment variables take precedence over that file.

| Setting | Environment variable | Default | Purpose | |---|---|---|---| | url | LLAMA_SWAP_URL | http://localhost:8080 | Server base URL. | | provider | LLAMA_SWAP_PROVIDER | llama-swap | Provider ID in pi. | | apiKey | LLAMA_SWAP_API_KEY | llama-swap-local | API key sent to the server. | | contextWindow | LLAMA_SWAP_CONTEXT_WINDOW | 128000 | Context window when discovery supplies none. | | maxTokens | LLAMA_SWAP_MAX_TOKENS | 32768 | Maximum output tokens per model. | | exclude | LLAMA_SWAP_EXCLUDE | Built-in case-insensitive regex | Model IDs to hide. |

The default exclude regex matches image/diffusion models, TTS, transcription, and common embedding families: image, diffusion, sdxl, flux, krea, lowvram, tts, embedding, embed, whisper, asr, bge, nomic, mxbai, e5, and clip. A custom regex replaces it. Reload after changing connection settings.

Adjust one model

Merge overrides into ~/.pi/agent/models.json. Use the actual model ID from your server:

{
  "providers": {
    "llama-swap": {
      "modelOverrides": {
        "qwen3.5-9b": { "reasoning": false },
        "gemma-4-e4b": { "contextWindow": 131072 }
      }
    }
  }
}

If you changed provider, use that ID in place of llama-swap. Pi reapplies model overrides when the catalog refreshes.

Reasoning defaults to enabled for every discovered model. IDs containing a separated no-thinking/nothinking marker or ending in -uncensored/_uncensored disable it. Override this when the model needs different behavior.

Vision is enabled when /v1/models reports image in architecture.input_modalities. A positive context_length overrides the fallback window. Declare capabilities in your llama-swap configuration so unloaded models can report them too; inside a model entry:

capabilities:
  in: [text, image]
  context: 131072

Update or remove

Run pi update --extensions to update unpinned installations, then /reload. To stop loading this extension:

pi remove npm:@sidshaytay/pi-llama-swap

Run /reload afterward. Connection settings and entries in models.json remain available if you reinstall.

Troubleshooting

The provider is empty. Check the server from the same machine running pi:

curl http://your-server:8080/v1/models

Replace the hostname with yours. Fix connection or authentication errors, then reopen /model. The discovery request does not attach the configured API key; if your server requires authentication on /v1/models, discovery will fail even when chat credentials are configured.

A model is missing. Check whether its ID matches the exclude regex. Set exclude to a regex that permits it, then reload.

Context size, vision, or reasoning is wrong. Check the metadata returned by /v1/models. Add server capabilities or a per-model override, then reload.

Links

Development and tests · MIT license