pi-voice-input

Local Chinese/English voice dictation for Pi. Press F2 to start recording, press again to stop — transcribed text fills into the input box. Fully offline: no cloud API keys, no network after the one-time model download.

Features

• Toggle dictation: F2 starts recording; press again to stop and fill the input box
• Real-time waveform: scrolling mic-level meter in the status bar while recording
• Offline first: local SenseVoice model via sherpa-onnx WASM (~45–220 MB one-time download)
• Chinese + English: mixed-language recognition out of the box (also Japanese, Korean, Yue)
• Cross-platform: Windows, macOS, Linux
• Editor-safe: inserts transcript at cursor, never auto-sends
• Configurable caveat prefix: optionally prefix transcripts with an ASR-disclaimer
• Smart audio device detection: auto-selects the system default recording device on Windows (via PowerShell COM), with manual override support

Install

pi install git:github.com/mistgc/pi-voice-input

This clones the repo and installs dependencies automatically. Restart Pi or run /reload.

Note: This project is not yet published to npm. Please use the GitHub install method above.

Quick Start

1. Install the extension
2. Restart Pi — the extension checks for the speech model
3. If the model isn't downloaded yet, Pi shows a warning — run /voice model download
4. Press F2 to start recording (a green waveform appears in the status bar)
5. Speak in Chinese, English, or both
6. Press F2 again to stop — transcribed text fills into the input box
7. Review the text, then press Enter to send

Commands

| Command | Description | |---------|-------------| | /voice start | Manually start recording | | /voice stop | Stop recording and transcribe | | /voice cancel | Discard current recording | | /voice status | Show model, transcriber, recording state, and detected recorder | | /voice config | Show current configuration (JSON) | | /voice model | Show model status | | /voice model download | Download/re-download the speech model | | /voice devices | List available DirectShow audio input devices (Windows only) |

Requirements

• Pi (peer dependency on @earendil-works/pi-coding-agent)
• Node.js ≥ 18
• A recording tool: ffmpeg (recommended, all platforms), or sox / arecord / pw-record (Linux)

# Install ffmpeg
brew install ffmpeg        # macOS
winget install ffmpeg      # Windows
apt install ffmpeg          # Linux

Configuration

~/.pi/agent/voice-input.json (auto-created on first run):

{
  "hotkey": "f2",
  "modelPath": "~/.pi/agent/voice-input-models/sense-voice-small",
  "modelUrl": "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2",
  "capture": {
    "tool": "auto",
    "ffmpegPath": "ffmpeg",
    "device": "",
    "sampleRate": 16000,
    "channels": 1,
    "maxSeconds": 120
  },
  "output": {
    "caveatPrefix": "",
    "appendTrailingSpace": true
  }
}

Key configuration options

| Option | Default | Description | |--------|---------|-------------| | hotkey | "f2" | Toggle recording keybinding (any valid Pi keybinding) | | capture.tool | "auto" | Recording backend: auto, ffmpeg, sox, arecord, or pw-record | | capture.device | "" | Audio input device: leave empty for auto-detect, or set a specific name. Use "default" on Windows to force the system default recording device. On macOS use ":0" (built-in mic) or ":1" (external) | | capture.sampleRate | 16000 | Audio sample rate in Hz (must be 16000 for the SenseVoice model) | | capture.channels | 1 | Audio channels (mono required) | | capture.maxSeconds | 120 | Maximum recording duration before timeout | | output.caveatPrefix | "" | Text prepended to each transcript (e.g. "[语音输入] "). Won't duplicate if already present in editor | | output.appendTrailingSpace | true | Add a space after the transcript for immediate typing |

Run /reload in Pi after editing the config file.

Audio device troubleshooting

If you get "No speech detected" or "ffmpeg exited with code N":

Windows — the default DirectShow device may not exist. List your devices:

ffmpeg -list_devices true -f dshow -i dummy

Or use /voice devices inside Pi. Then set the device name in ~/.pi/agent/voice-input.json:

{ "capture": { "device": "Microphone (Realtek High Definition Audio)" } }

You can also use "device": "default" to force the system default recording device (detected via PowerShell COM).

macOS — use ":0" (built-in mic) or ":1" (external). List with:

ffmpeg -f avfoundation -list_devices true -i ""

Linux — ensure pulseaudio or pipewire-pulse is running. Try pw-record (PipeWire native) or arecord (ALSA) if ffmpeg has issues.

Architecture

extensions/voice-input/
├── index.ts             # Entry point — registers shortcut, commands, events
├── recorder.ts          # Cross-platform audio capture via ffmpeg/sox/arecord/pw-record
├── transcriber.ts       # sherpa-onnx WASM wrapper (non-streaming offline recognizer)
├── model-manager.ts     # One-time model download + extraction + verification
├── config.ts            # ~/.pi/agent/voice-input.json read/merge/write
├── formatter.ts         # Transcript formatting (caveat prefix dedup, trailing space)
├── audio-visualizer.ts  # Real-time mic-level waveform in the status bar
├── sherpa-onnx.d.ts     # Type declarations for sherpa-onnx module
└── utils.ts             # Temp file helpers, recording state machine

Data flow

F2 → Recorder.start() → spawn ffmpeg/sox → WAV in OS temp dir
    → AudioVisualizer reads trailing PCM → waveform in status bar
    → F2 again → Recorder.stop() → graceful 'q' → finalized WAV
    → Transcriber.transcribe() → raw text
    → formatTranscript() → pi.ui.pasteToEditor() → inserted at cursor
    → temp WAV deleted

State machine

IDLE ──→ RECORDING ──→ TRANSCRIBING ──→ IDLE
  ↑                       (error)        │
  └──────────── cancel ──────────────────┘

Privacy

• All processing is local — audio never leaves your machine
• Temporary WAV files are deleted immediately after transcription
• Model is downloaded once from GitHub Releases

Building & Testing

npm install                     # Install all deps
npx vitest run                  # Run unit tests
npx vitest                      # Watch mode
npx tsc --noEmit                # Type-check only

Tests live in test/ and use vitest. There is no build step — Pi loads the .ts entry point directly.

License

MIT