pi-llama-swap

Pi coding agent extension that registers a llama-swap provider and discovers models from a running llama-swap instance.

What it does

• Injects provider llama-swap with models from GET /v1/models
• Resolves per-model context from llama-swap APIs (/v1/models, /running) with 256K default — see Context window
• Uses OpenAI Chat Completions API (openai-completions) for streaming
• Reads optional config from ~/.pi/agent/pi-llama-swap.json to override defaults

Requirements

• pi coding agent (@earendil-works/pi-coding-agent)
• llama-swap running and reachable
• Node.js 18+ (for fetch)

Quick start

# Terminal 1: start llama-swap (example)
llama-swap --config ~/llama-swap/config.yaml --listen 127.0.0.1:8080

# Terminal 2: load extension
cd /path/to/pi-llama-swap
pi -e .

In pi: /model → pick llama-swap/your-model-id.

Commands

| Command | Args | Description | |---------|------|-------------| | /llama-swap-set-context-length | <number> or auto | Override context window for the current model (auto removes override and uses auto-detection) |

Verify from CLI:

pi -e . --list-models | grep llama-swap
curl http://127.0.0.1:8080/v1/models

Configuration

Defaults

| Setting | Default | |---------|---------| | Origin | http://127.0.0.1 | | Port | 8080 | | Base URL | http://127.0.0.1:8080/v1 |

No config file needed when llama-swap runs on the defaults above.

Config file

Create ~/.pi/agent/pi-llama-swap.json to override defaults:

{
  "origin": "http://127.0.0.1",
  "port": 8080,
  "apiKey": "optional-key",
  "contextOverrides": {
    "my-model": 32768
  }
}

| Field | Description | |-------|-------------| | origin | Scheme + host (e.g. http://192.168.1.10) | | port | TCP port (1–65535) | | basePath | API path prefix (default /v1; normalized to end with /v1) | | apiKey | Bearer token when llama-swap uses apiKeys | | contextOverrides | Per-model context overrides (model id → tokens). Use /llama-swap-set-context-length instead of editing manually. |

Load order: defaults → ~/.pi/agent/pi-llama-swap.json → environment variables.

Context window (per model)

Context size is auto-detected from llama-swap's /v1/models and /running endpoints at startup, with a default of 256K when nothing reports a value. User overrides set via /llama-swap-set-context-length take precedence over all auto-detected values.

# Restrict permissions when storing API keys
chmod 600 ~/.pi/agent/pi-llama-swap.json

Environment variables

Optional runtime overrides (highest precedence):

| Variable | Purpose | |----------|---------| | LLAMA_SWAP_URL | Origin URL (scheme + host, optional port/path) | | LLAMA_SWAP_PORT | Port override | | LLAMA_SWAP_API_KEY | API key override |

Auth

If llama-swap uses apiKeys in its config, set "apiKey" in pi-llama-swap.json or LLAMA_SWAP_API_KEY.

Without a key, extension uses placeholder local-no-auth so models appear in /model. Pi may send Authorization: Bearer local-no-auth; most unsecured local installs ignore it.

Troubleshooting

| Symptom | What to try | |---------|-------------| | Cannot reach llama-swap | Start llama-swap; check origin/port in config file | | HTTP 401 | Set apiKey in config or LLAMA_SWAP_API_KEY | | 0 models | Ensure models in llama-swap config; curl http://127.0.0.1:8080/v1/models | | Extension loads but chat fails | Confirm model id; first request may load model (slow) | | Config ignored | File must be ~/.pi/agent/pi-llama-swap.json; restart pi after edits |

Project layout

pi-llama-swap/
├── index.ts          # Extension entry
├── lib/
│   ├── config.ts     # Load ~/.pi/agent/pi-llama-swap.json
│   ├── url.ts        # URL building
│   ├── client.ts     # GET /v1/models
│   ├── context.ts    # Context window from llama-swap APIs
│   ├── provider.ts   # registerProvider
│   └── types.ts
├── package.json
└── README.md

License

MIT (see repository if published).