pi-omni

push-to-talk voice extension for pi.dev: wires a local OpenAI-compatible STT/LLM/TTS stack (e.g. llama-swap serving whisper.cpp + llama.cpp + a TTS model) into a pi agent session, plus an optional browser UI.

getting started

prerequisites:

• node.js 20+
• a working pi installation
• arecord (alsa-utils) and a wav-on-stdin speaker (aplay, paplay, ffplay -nodisp -autoexit -, …)
• an OpenAI-compatible endpoint exposing STT, LLM, and TTS

install as a pi extension:

pi install npm:@khimaros/pi-omni

control voice mode from the pi tui:

> /omni              # push-to-talk: tap to record, VAD or re-tap to stop
> /omni-live         # continuous conversation: record → STT → LLM → TTS loop
> /omni-cancel       # cancel any active recording / TTS / chat loop
> /omni-setup        # configure endpoint, models, mic, speaker (re-run anytime)
> /omni-test [text]  # TTS round-trip diagnostic

control the web UI from the pi tui:

> /omni-web start    # start the web server
> /omni-web status   # view server status
> /omni-web open     # open the web UI in browser
> /omni-web stop     # stop the web server

or auto-start when pi launches (terminated when pi exits):

pi --omni-live       # continuous voice on launch
pi --omni-web        # web server on launch

run the web server standalone (no pi tui):

PI_VOICE_LLM_MODEL=qwen3-32b npx @khimaros/pi-omni

or install globally:

npm install -g @khimaros/pi-omni
PI_VOICE_LLM_MODEL=qwen3-32b pi-omni-web

then open http://127.0.0.1:4962.

Once loaded, the Web UI automatically tracks active sessions. You can use the premium glassmorphic sessions menu in the top-right corner to view a list of recent sessions, switch between them, or start a new session instantly. Reconnecting after a WebSocket disconnect or reloading the page automatically resumes the active session based on the URL hash.

pwa installation

the web UI is a Progressive Web App (PWA). you can "install" it to your home screen or desktop for a native-like experience:

1. open the URL in a supported browser (Chrome, Safari, Edge).
2. look for the "Install" icon in the address bar or select "Add to Home Screen" from the browser menu.
3. the app will appear on your device with a premium waveform icon.

from a source checkout

make            # install deps + build
make test       # run tests
make wasm       # rebuild wasm/apm (after touching wasm/apm/src/)

configuration

first run of /omni triggers /omni-setup automatically — it walks through endpoint, models, mic, speaker, and an end-to-end round-trip test. saved to ~/.pi/extensions/omni.json. re-run /omni-setup anytime to reconfigure.

env vars override the saved file:

| variable | default | purpose | | --- | --- | --- | | PI_VOICE_BASE_URL | http://localhost:8080/v1 | OpenAI-compatible endpoint | | PI_VOICE_API_KEY | sk-no-key | llama-swap usually ignores it | | PI_VOICE_STT_MODEL | whisper-1 | as exposed by your server | | PI_VOICE_TTS_MODEL | tts-1 | | | PI_VOICE_TTS_VOICE | alloy | | | PI_VOICE_LLM_MODEL | (none) | required for standalone pi-omni-web | | PI_VOICE_MIC_DEVICE | (default ALSA) | passed to arecord -D | | PI_VOICE_SPEAKER_CMD | aplay -q ... | reads WAV from stdin | | PI_VOICE_AEC_ENABLED | false | acoustic echo cancellation (WebRTC AEC3 WASM) | | PI_VOICE_AEC_DELAY_MS | 200 | expected speaker→mic round-trip | | PI_VOICE_BARGE_IN | false | keep mic open during TTS, cut in on speech | | PI_VOICE_BARGE_IN_MIN_MS | 300 | minimum speech duration to count as barge-in | | PI_VOICE_WEB_HOST | 127.0.0.1 | http bind address for the web server | | PI_VOICE_WEB_PORT | 4962 | http port for the web server |

cli flags for the pi extension:

| flag | env var | config key | effect | | --- | --- | --- | --- | | --omni-live | PI_OMNI_AUTO_LIVE=true | autoStartLive | start continuous voice on launch | | --omni-web | PI_OMNI_AUTO_WEB=true | autoStartWeb | start web server on launch |

cli flags for standalone pi-omni-web:

| flag | purpose | | --- | --- | | --listen <host:port> | http bind address; takes precedence over env vars | | -h, --help | usage |

echo cancellation & barge-in

set aecEnabled: true and bargeInEnabled: true (via /omni-setup or env) to keep the mic open during TTS so you can interrupt by speaking. without AEC, only enable barge-in on headphones — speaker output will feed back into the mic and the bot will interrupt itself.

the AEC is a Rust port of WebRTC AEC3 compiled to WASM, depended on as a file: package at wasm/apm/pkg/. rebuild after touching wasm/apm/src/:

make wasm
# or directly:
cd wasm/apm && wasm-pack build --target nodejs --release

build deps: rustup (e.g. via mise use -g rust@latest) with the wasm32-unknown-unknown target, plus wasm-pack.

roadmap

see ROADMAP.md for implemented and planned features.

architecture

src/
  extension/   pi extension entry (commands, shortcuts, event handlers)
  server/      HTTP + WS server hosting the browser client
  bin/         standalone executables (pi-omni-web)
  audio/       mic, STT, TTS, VAD, AEC, sentence chunker, sanitizer
  config.ts    shared config + env-var overrides
public/        browser client (no build step)
wasm/apm/      WebRTC AEC3 → WASM (rust)
test/          node --test files

development

make            # install deps + build (tsc)
make test       # build then run node --test
make lint       # type-check (tsc --noEmit)
make precommit  # lint + test
make install    # install globally from this checkout
make update     # npm update
make wasm       # rebuild wasm/apm
make pack       # npm pack into build/
make publish    # npm publish --access public
make clean      # rm -rf dist build

known limits

• sentence chunking is naive (split on .!?\n); abbreviations like "e.g." will split early.
• manual barge-in via /omni re-tap works without AEC; automatic barge-in needs AEC enabled or headphones.
• if the pi extension bus doesn't forward message_update, TTS waits for turn_end — still works, just less interactive.
• barge-in cuts off TTS instantly but the LLM keeps generating in the background until it finishes; its output is discarded.
• standalone pi-omni-web requires PI_VOICE_LLM_MODEL; the pi extension path doesn't (pi owns the LLM).