piqwy

At-will Qwythos coding agent. Run piqwy in any directory and you get the pi.dev agent wired to Qwythos-9B running on a cheap on-demand RunPod GPU that spins up when you need it and 86's itself when idle. One file, zero dependencies, Node ≥18.

piqwy                 # ensure the model is serving, then launch the pi.dev agent in this dir
piqwy -p "add a test for utils.ts"   # headless one-shot
piqwy claude          # launch Claude Code on the model instead of pi.dev
piqwy models          # list the model catalog (active one marked)
piqwy model devstral  # switch models (persists); optional tier: piqwy model devstral smart
piqwy status          # daemon + live pods + stats (reuses / creates / est cost)
piqwy down            # terminate pods this daemon created
piqwy down --all      # terminate every piqwy pod on the account
piqwy stop            # stop the local daemon

Qwythos is the default; piqwy models lists the rest and piqwy model <name> (or -m <name> on any run) switches. See Models.

Flags

• --json — machine-readable JSON instead of the formatted output (status, down). Everything else prints for humans; pass this when you're scripting.

• --all — with down, also stop piqwy pods this daemon didn't create. Default down only touches pods this daemon started; if it sees other piqwy pods it leaves them and tells you.

• -- <args...> — forward everything after -- verbatim to the underlying harness (pi.dev or Claude Code). Use it for harness flags piqwy doesn't know about:

  piqwy -- --add-dir /some/repo                     # pass --add-dir to pi.dev
  piqwy claude -- --dangerously-skip-permissions    # pass a flag to Claude Code

  For claude, flags typed right after claude also pass through, so the -- is optional there.

Two harnesses

The daemon exposes both APIs on the same local port, so you can drive Qwythos with either agent:

• pi.dev (piqwy) — OpenAI-native; the daemon proxies /v1/chat/completions straight through.
• Claude Code (piqwy claude) — Claude Code speaks the Anthropic Messages API, so the daemon translates /v1/messages ⇄ OpenAI on the fly (deterministic mapping, tool-use IDs preserved, real SSE streaming). piqwy claude sets ANTHROPIC_BASE_URL at the daemon and launches claude here. Heads up: Qwythos-9B is a small model — Claude Code leans hard on strong tool-use, so expect a rougher ride than a frontier Claude model.

Models

One model serves at a time. Each catalog entry is a self-contained recipe — GGUF repo, quant ladder, native context, model-card sampling — pinned to the GPU order it runs on best, so switching a model re-points the whole pipeline (and cycles the pod, since the pod identity includes the repo/quant/ctx).

| name | what it is | best GPU | ctx | |---|---|---|---| | qwythos (default) | 9B hybrid-attention reasoner, cheapest | RTX 4090 (24GB) | 256k | | ornith-9b | Ornith-1.0-9B, agentic-coding specialist (SWE-bench 69%) | RTX 4090 (24GB) | 128k | | gemma-4-12b | Gemma 4 12B dense (Google), QAT + MTP 3× decode | RTX 4090 (24GB) | 256k | | qwen3-coder | Qwen3-Coder-30B-A3B MoE, fast agentic coder | L40S (48GB) | 32k | | devstral | Devstral-Small-2507, 24B dense SWE/tool-use specialist | L40S (48GB) | 128k | | gemma-4-26b | Gemma 4 26B-A4B MoE (Google), fast agentic decode | RTX 5090 (32GB) | 256k | | qwen36-27b | Qwen3.6-27B dense, flagship coding + MTP 2× decode | L40S (48GB) | 128k | | qwen36-35b | Qwen3.6-35B-A3B MoE, fast agentic coder (~140 t/s) | L40S / 5090 | 64k | | ornith-35b | Ornith-1.0-35B MoE, top open agentic coder (SWE-bench 76%) | L40S (48GB) | 128k |

Each runs on a single GPU sized to the model (24GB for the 9-12B, 48GB for the 24-35B), and every one is measured green — see MODELS.md for the red/green snake test and serving numbers (decode/prefill/cached) on real RunPod hardware. Contexts use a q8_0 KV cache (half the memory of f16) so most models reach 128k-256k on a single card; Gemma's sliding-window attention fits 256k on a 32GB 5090. The exceptions are qwen3-coder and devstral, whose architectures crash on quantized KV and so need f16 (capping qwen3-coder at 32k).

Best for agentic coding: the two ornith-* models are purpose-built for it (tool-calling + <think> reasoning). ornith-35b is the strongest open agentic coder here (SWE-bench Verified ~76%, ~160 t/s on an L40S); ornith-9b is the cheapest good coder (fits a 24GB 4090). Several models use MTP (multi-token-prediction) speculative decoding for a 2-3× decode speedup with no quality loss: gemma-4-12b (QAT+MTP) and qwen36-27b.

piqwy models                 # list them, active one marked, with GPU order + context
piqwy model qwen3-coder      # select (persists to ~/.piqwy/config.json)
piqwy model qwen3-coder smart # ...at the smart tier (Q8) instead of the default fast (Q4)
piqwy -m devstral            # use devstral for this run (also persists the choice)

Tiers trade quality for cost/speed: fast (~Q4), mid (~Q6), smart (~Q8). ctx is set to fit the first GPU in each model's preference list; bump it per-model in ~/.piqwy/config.json for a bigger card. Full-attention models (devstral, qwen25-coder, qwen3-coder) serve with an f16 KV cache — quantized KV + flash-attention crashes them on the first request. If the preferred GPU is dry or too small, piqwy falls through the rest of the list (community → secure) and tears down anything that places but can't serve. Add your own model by dropping an entry into models in the config — same shape as the built-ins.

How it works

piqwy (CLI) ──http──> piqwyd (daemon, local OpenAI PROXY) ──REST──> RunPod ──> model pod (llama.cpp)
   starts daemon if down        routes every request to a pod                  self-destructs when idle
   installs + wires pi.dev      (reuse live / create + warm if down)           proxy URL = OpenAI endpoint
   launches pi in cwd           tracks pods, records stats, prunes idle-killed

The daemon is the data plane: everything goes through http://127.0.0.1:8787/v1, and it routes each request to a Qwythos pod (strategy live today — the one running pod; pluggable later). When a pod self-destructs on idle, the next request transparently spins up a fresh one — clients never re-wire. Missing key / no capacity come back as OpenAI-shaped errors with a fix hint.

piqwy status shows an estimated cost, accrued by the daemon while it's polling, at each pod's actual RunPod rate (adjustedCostPerHr, after Savings Plans). It only counts pods this daemon tracked, and because it accrues on a poll interval it can be off by a few billing minutes — treat it as a ballpark, not an invoice.

Setup

export RUNPOD_API_KEY=...        # or put RUNPOD_API_KEY=... in ~/.piqwy/.env

Get a key at https://www.runpod.io/console/user/settings. No GPU quota forms. First run downloads the model (~1-3 min); later runs in the idle window reuse the warm pod instantly.

Run it

npx @baublet/piqwy               # once published (scoped, public)
# or, from a checkout:
node piqwy.mjs                   # single file, run anywhere

Config

Built-in defaults cover everything; override any of them in ~/.piqwy/config.json (deep-merged):

• model_key — the active model (default qwythos); models — the catalog (see Models). Each entry carries its own repo, quant ladder, ctx, native_ctx, gpu_prefs, and sampling.
• runpod — image, Community/Secure, disk, pod label, and the fallback gpu_prefs/gpu_types map.
• serve — ctx, np (continuous-batch slots), kv cache quant, idle_seconds (per-model ctx and sampling override these).
• proxy — host/port, routing strategy.
• harness — pi.dev provider name + install URL.

See MODELS.md for each model's red/green smoke test and measured serving performance on RunPod (regenerate it with npm run smoke -- --md).

Pod scripts & build

The pod-side shell is the source of truth in scripts/ (onstart.sh serves llama.cpp; idle_watchdog.sh self-destructs the pod on idle). They're base64-embedded into piqwy.mjs so the tool stays a single self-contained file. After editing either, regenerate the embed:

npm run build      # node build.mjs — re-injects scripts/*.sh into piqwy.mjs (also runs on publish)

Code style

Biome with stock defaults — no custom rules, nothing to argue about.

npm run format     # biome check --write — format + safe lint fixes
npm run check      # biome check — verify only (for CI)

build.mjs writes the base64 embeds in Biome's canonical shape, so the formatter and the build step don't fight over those lines.

Publishing

Not published yet. To publish (review first):

npm pack            # inspect the tarball contents
npm publish         # ship it (bump version in package.json first)