voice-connect

A standalone voice service (STT + TTS) for a team of AI agents.

voice-connect gives agents and runtimes one voice layer with two halves — text-to-speech and speech-to-text — over a single multi-provider core. Each request runs a cascade with fallback inside the tool: cloud quality first, a local engine as the floor, so synthesis and transcription still work with no API key and no network. It runs on its own (configured from a file) or inside iapeer (configured from the peer-profile).

How it works

Two facades sit on one core. Agents reach it as an MCP server; runtimes reach it over an OpenAI-compatible HTTP service. Both call the same cascade — no synthesis or routing logic is duplicated between them.

   agents (MCP)                 runtimes (HTTP)
   tts / stt                    POST /v1/audio/speech
                                POST /v1/audio/transcriptions
        \                              /
         \                            /
          ▼                          ▼
   ┌──────────────────────────────────────┐
   │   core: router + provider cascade     │
   ├──────────────────────────────────────┤
   │ TTS:  Gemini → gpt-audio → F5 → Super │
   │ STT:  speaches → mlx-whisper          │
   └──────────────────────────────────────┘

The router advances to the next engine only on a known "can't-serve" error (quota, no key, unreachable); any other error propagates, so a real bug surfaces instead of being masked by a fallback. The last rung in each cascade is a local engine — Supertonic 3 for TTS, mlx-whisper for STT — with no fallback of its own: it is the floor, so a result is always produced when an engine is reachable at all.

Delivery is never the service's job. The MCP tts tool hands back a file path; you attach it yourself with send_to_peer(personality, attachments=[path]). The HTTP facade returns the audio bytes in the response.

Short text is synthesized synchronously — the path comes back at once. Text over ~2000 chars goes async: the tts tool returns a job_id immediately and a detached worker notifies you over IAP — sent from the system peer voice-notifier, a dedicated, always-stopped identity provisioned on init/update — when the file is ready. On hosts without iapeer no IAP message can reach you; poll the MCP tool job_status(job_id) instead until it reports done or failed. The HTTP /v1/audio/speech route is always synchronous (a runtime holds the connection and wants bytes back).

Requirements

• Node.js ≥ 18
• ffmpeg / ffprobe on PATH — audio encode and probe. On macOS: brew install ffmpeg.
• python3 — only for the first provision of the local Supertonic TTS floor (a managed venv is created on demand).
• mlx_whisper on PATH — only for the local STT floor (Apple-silicon Whisper, installed as a uv tool). Without it, STT needs a configured speaches endpoint.
• API keys — read from the environment, a shell rc file, or the standalone config file:
  • GEMINI_API_KEY — the primary TTS engine (Google Gemini TTS).
  • OPENROUTER_API_KEY — the gpt-audio TTS fallback rung (OpenRouter).
  • Neither key is required: with no key, TTS routing falls through to the local F5 / Supertonic engines.

Install

As a plugin (Claude Code / Codex), via the agfpd marketplace:

/plugin marketplace add agfpd/agfpd-marketplace
/plugin install voice-connect@agfpd

Or run the MCP server directly from npm:

npx -y @agfpd/voice-connect@latest

The package name is @agfpd/voice-connect; the MCP entry point is the voice-connect-mcp bin. Wire it into an MCP client by pointing the client at that command, e.g.:

{
  "mcpServers": {
    "voice-connect": {
      "command": "npx",
      "args": ["-y", "@agfpd/voice-connect@latest"]
    }
  }
}

Always-on HTTP service

The HTTP facade is meant to run as a long-lived local service so runtimes can hit it without spawning a process per request. It is self-managed: it owns its own launchd agent (label com.voice-connect.http), in the package's own namespace, and writes a discovery slot (~/.iapeer/voice-provider.json) consumers read for the endpoint.

node scripts/launchd-http.mjs render      # print the plist (no writes)
node scripts/launchd-http.mjs install     # write plist + slot, (re)bootstrap
node scripts/launchd-http.mjs status      # launchctl print + slot
node scripts/launchd-http.mjs uninstall   # bootout + remove plist + slot

It binds 127.0.0.1:8127 by default (local-only); override with PEER_VOICE_HTTP_HOST / PEER_VOICE_HTTP_PORT. GET /health reports liveness.

What it does

MCP tools

| Tool | In → out | Notes | |------|----------|-------| | tts | text → .ogg/opus file | Short → { path, ... } sync; long (>~2000 chars) → { job_id } + an IAP "done" message later. | | stt | audio file path → text | Returns { text, engine, fallback_from? }. | | job_status | job_id → status | running | done | failed | unknown — pull-floor for hosts without iapeer, or when you suspect a voice-notifier message was lost. |

HTTP routes (OpenAI-compatible)

| Route | Method | Body | Returns | |-------|--------|------|---------| | /v1/audio/speech | POST | JSON (input/text, voice?, model?/engine?, lang?, style?) | Ogg/Opus bytes; engine/fallback in X-Voice-* headers. | | /v1/audio/transcriptions | POST | multipart (file, language?, prompt?, engine?, response_format?) | { text }, or plain text with response_format=text. | | /health | GET | — | { status, service, version }. |

Engines

TTS — first applicable engine wins; the cascade advances on a known can't-serve error:

| Engine | Tier | Model | Notes | |--------|------|-------|-------| | Gemini | cloud primary | gemini-3.1-flash-tts-preview | ru+en in one pass; honors style; needs GEMINI_API_KEY. | | gpt-audio | cloud second | openai/gpt-audio (OpenRouter) | Multilingual one pass; honors style; needs OPENROUTER_API_KEY. | | F5-TTS | local | f5-tts | Live-prosody Russian rung; per-peer voice cloning; applies on the ru route only. | | Supertonic 3 | local floor | supertonic-3 | Offline, one pass in the routed language; the floor. |

STT — speaches when an endpoint is configured, otherwise the local floor:

| Engine | Tier | Notes | |--------|------|-------| | speaches | primary | OpenAI-compatible /v1/audio/transcriptions; set PEER_VOICE_STT_ENDPOINT. Skipped when unset. | | mlx-whisper | local floor | Offline Apple-silicon Whisper via the mlx_whisper CLI. |

tts parameters

tts(text, voice?, lang?, style?, note?, engine?, out_path?)

| Param | Type | Description | |-------|------|-------------| | text | string (required) | Text to speak. Mixed ru+en is fine — read in one pass. | | voice | string | Gemini prebuilt voice for the primary engine. Default Aoede. | | lang | ru | en | na | Language hint for the fallback engines (Gemini reads any language itself). Omit to auto-detect by character share. | | style | string | Delivery directive (tone / emotion / tempo) for the cloud engines; ignored by the local fallbacks. | | note | string | Reminder echoed back in the async "done" message (e.g. who to deliver to). Ignored for short (sync) text. | | engine | auto | gemini | gpt-audio | supertonic | Force an engine. Default auto (the full cascade). | | out_path | string | Absolute output .ogg path. Default: a unique file under ~/.iapeer/cache/peer-voice/out/. |

Returns — short text (sync): { path, engine, voice, lang?, probe, fallback_from? }. Long text (async): { job_id, status: "started" }, followed by an IAP message from the system peer voice-notifier: voice job <id> done path=<path> note=<note> when synthesis finishes (or failed reason=<...>). No iapeer on the host → no message; poll job_status(job_id) instead.

stt parameters

stt(audio_path, lang?, prompt?, engine?)

| Param | Type | Description | |-------|------|-------------| | audio_path | string (required) | Absolute path to the audio file to transcribe (e.g. a received voice .ogg). | | lang | string | Language hint (ISO-639-1: en, ru, …). Omit to auto-detect. | | prompt | string | Decoder-priming prompt — biases spelling/casing of terms (e.g. keep Claude Code in Latin). Not part of the output. | | engine | auto | speaches | mlx-whisper | Force an engine. Default auto. |

Returns { text, engine, fallback_from? }.

Standalone and iapeer modes

The mode is decided by one signal: whether the caller has an iapeer identity. Both modes share the same core and the same key ladder (env → shell rc → config file).

• Standalone — no iapeer identity. Voice and keys come from one config file: $PEER_VOICE_CONFIG, else <PEER_VOICE_HOME>/config.json:

  {
    "voice": { "gemini-3.1-flash-tts-preview": "Aoede", "supertonic-3": "F3" },
    "keys":  { "GEMINI_API_KEY": "...", "OPENROUTER_API_KEY": "..." }
  }

  voice-connect only reads this file; it never writes config itself.

• iapeer — the caller has an iapeer identity (PEER_PERSONALITY or a cwd peer-profile). Voice comes from the peer-profile; keys from the host env/rc. iapeer owns the configuration.

License

Apache-2.0. Platform: macOS.