Note: This is how I use pi.dev + llama.cpp on my local machine. I created a plugin so that I can update my setup quickly.

pi-llama-server

Pi extension that integrates a running llama-server instance with the Pi Coding Agent. Provides live model listing and ability to load/unload via the llama-server API.

Prerequisites

• A running llama-server instance (from llama.cpp) in router-mode (the default if you don't mention -m)
• Pi Coding Agent installed (@mariozechner/pi-coding-agent)

Install

pi install npm:pi-llama-server

Or from git:

pi install git:github.com/user/pi-llama-server

Pi auto-discovers the extension via pi.extensions in package.json. No additional setup needed.

Configuration

The llama-server URL is resolved in this order:

1. Per-project config — create .pi/llama-server.json in your project root:

   { "url": "http://10.0.0.5:9090" }

2. Environment variable — set globally:

   export LLAMA_SERVER_URL=http://10.0.0.5:9090

3. Default — falls back to http://127.0.0.1:8080

Usage

Browse and manage models

Run the /models slash command inside Pi to see all models on the llama-server with live status:

| Status | Meaning | |--------|---------| | 🟢 loaded | Model is loaded and ready | | 🟡 loading | Model is being loaded | | 🔴 failed | Model failed to load | | ⚪ other | Unknown state |

Select a model to load, unload, or switch to it.

Switch models

Use Ctrl+P (or /model) in Pi to select any llama-server model for inference. The extension will automatically tell llama-server to load the chosen model.

How it works

When Pi starts, the extension:

1. Resolves the llama-server URL from config/env/default
2. Queries GET /models to discover available GGUF models
3. Registers each model as an OpenAI-compatible provider under {url}/v1
4. Listens for model switch events and calls POST /models/load on the server
5. Provides the /models interactive command for managing models

llama-server endpoints used

| Endpoint | Method | Purpose | |----------|--------|---------| | /models | GET | List all models | | /models/load | POST | Load a model | | /models/unload | POST | Unload a model | | /v1/... | POST | OpenAI-compatible completions (via Pi provider) |