diffusionpi

Watch a diffusion LLM denoise its answer live in the terminal. diffusionpi runs Pi against a local vLLM serving DiffusionGemma and renders the model's real intermediate canvas above the editor: instead of text appearing in silent bursts, you see accepted tokens and renoise tokens converge into the final text on every denoising step.

No forked Pi involved: this is a declarative pi-factory app bundle plus ordinary Pi extensions.

bin/diffusionpi                   launcher (pi-factory wrapper)
app/                              pi-factory app bundle
  pi-factory.toml                 interactive session
  demo.pi-factory.toml            self-driving demo (no tools, auto prompts)
  extensions/diffusion-canvas.ts  the live canvas widget
  extensions/smooth-scroll.ts     gradual viewport scroll (Pi internals hack)
  prompts/                        demo prompts
docs/diffusion-canvas-repro.md    full reproduction guide

The self-driving demo driver and minimal demo chrome come from the shared pi-demo-mode extension, an npm dependency installed automatically.

Install

npm install -g github:osolmaz/diffusionpi

That puts diffusionpi on your PATH. Alternatively clone the repo and run bin/diffusionpi directly; the launcher fetches its npm dependency on first run.

Requirements

• Node 20+ and npm (Pi and pi-factory are fetched via npx).

• A vLLM server with the diffusion canvas side channel. The changes are pure Python, so they overlay the official precompiled kernels:

  VLLM_USE_PRECOMPILED=1 \
  VLLM_PRECOMPILED_WHEEL_COMMIT=4e5ca89cfe98121642d76b40e32a006f4d0fbf3b \
  pip install git+https://github.com/osolmaz/[email protected]

Run

Serve DiffusionGemma with canvas streaming:

vllm serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 \
  --host 127.0.0.1 --port 8000 \
  --max-model-len 32768 --max-num-seqs 16 --max-num-batched-tokens 8192 \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice --tool-call-parser gemma4 \
  --diffusion-stream-canvas

Then:

diffusionpi         # interactive session with the canvas
diffusionpi demo    # self-driving story demo (for recordings)
diffusionpi plan    # print the launch plan without running

The launcher turns the bundle in app/ into a Pi launch: provider vllm at http://127.0.0.1:8000/v1, model nvidia/diffusiongemma-26B-A4B-it-NVFP4, and the extensions. Edit app/pi-factory.toml if your server or model id differ.

The canvas widget needs no configuration: it derives the events and metrics URLs from the active model's baseUrl, with PI_DIFFUSION_CANVAS_EVENTS_URL and PI_DIFFUSION_CANVAS_METRICS_URL as overrides. Against a server without the side channel it falls back to a clearly labeled simulation paced by the real commit bursts.

Demo grid and recording

A wall of concurrent diffusionpi sessions (and an mp4 of it) is a job for demowall, a generic tool that runs N copies of any command in a tiled tmux grid and records tmux sessions in a themed Ghostty window:

# 2x2 wall of self-driving demo sessions:
demowall grid --concurrency 4 --start -- diffusionpi demo

# Record the wall to an mp4 in a Catppuccin-themed Ghostty window:
demowall record --session demowall-<timestamp> --out demo.mp4 --seconds 60

Concurrency above 4 needs --allow-high-concurrency and should match the vLLM server's --max-num-seqs.

Smooth scroll

Stock Pi appends a whole diffusion commit (~15+ lines) to the chat in one frame, which makes the viewport jump. The bundled smooth-scroll extension fixes this with a deliberate hack: it grabs the live TUI instance and wraps the chat container's render() so appended lines are revealed at a bounded rate (default 40 lines/s) instead of all at once. Real content, paced only at the render layer.

Because it reaches into Pi internals, it is version-sensitive: it checks that the component tree looks like Pi 0.8x and silently no-ops otherwise, so a Pi upgrade degrades to the stock jumpy behavior rather than breaking.

• DIFFUSIONPI_SMOOTH_SCROLL=0 disables it
• DIFFUSIONPI_SCROLL_SPEED=<lines/s> changes the reveal rate
• DIFFUSIONPI_SCROLL_DEBUG=<path> logs per-frame pacing decisions

Independently, you can also make the commits themselves smaller and more frequent server-side:

vllm serve ... --diffusion-config '{"canvas_length": 64}'

Smaller canvases trade some throughput for gentler commits.

The canvas widget also works in any plain Pi setup, no bundle needed: it is a single self-contained extension file, so copy app/extensions/diffusion-canvas.ts into your Pi extensions directory (e.g. ~/.pi/agent/extensions/) or pass it with Pi's --extension flag.

See docs/diffusion-canvas-repro.md for the full reproduction guide, including how the truthful streaming path works and how the vLLM fork is maintained.

License

MIT