voice-connect

A standalone voice service (STT + TTS) for a team of AI agents.

voice-connect gives agents and runtimes one voice layer with two halves — text-to-speech and speech-to-text — over a single multi-provider core. Each request runs a cascade with fallback inside the tool: cloud quality first, a local engine as the floor, so synthesis and transcription still work with no API key and no network. It runs on its own (configured from a file) or inside iapeer (configured from the peer-profile).

How it works

Two facades sit on one core. Agents reach it as an MCP server; runtimes reach it over an OpenAI-compatible HTTP service. Both call the same cascade — no synthesis or routing logic is duplicated between them.

   agents (MCP)                 runtimes (HTTP)
   tts / stt                    POST /v1/audio/speech
                                POST /v1/audio/transcriptions
        \                              /
         \                            /
          ▼                          ▼
   ┌──────────────────────────────────────┐
   │   core: router + provider cascade     │
   ├──────────────────────────────────────┤
   │ TTS:  Gemini → gpt-audio → F5 → Super │  ← default order,
   │ STT:  speaches → mlx-whisper          │     operator-configurable
   └──────────────────────────────────────┘

The router advances to the next engine only on a known "can't-serve" error (quota, no key, unreachable); any other error propagates, so a real bug surfaces instead of being masked by a fallback. The last rung in each cascade is a local engine — Supertonic 3 for TTS, mlx-whisper for STT — with no fallback of its own: it is the floor, so a result is always produced when an engine is reachable at all.

Delivery is never the service's job. The MCP tts tool hands back a file path; you attach it yourself with send_to_peer(personality, attachments=[path]). The HTTP facade returns the audio bytes in the response.

Short text is synthesized synchronously — the path comes back at once. Text over ~2000 chars goes async: the tts tool returns a job_id immediately and a detached worker notifies you over IAP — sent from the system peer voice-notifier, a dedicated, always-stopped identity provisioned on init/update — when the file is ready. On hosts without iapeer no IAP message can reach you; poll the MCP tool job_status(job_id) instead until it reports done or failed. The HTTP /v1/audio/speech route is always synchronous (a runtime holds the connection and wants bytes back).

Requirements

• Node.js ≥ 18
• ffmpeg / ffprobe on PATH — audio encode and probe. On macOS: brew install ffmpeg.
• python3 — only for the first provision of the local Supertonic TTS floor (a managed venv is created on demand).
• mlx_whisper on PATH — only for the local STT floor (Apple-silicon Whisper, installed as a uv tool). Without it, STT needs a configured speaches endpoint.
• API keys — read from the environment, a shell rc file, or the standalone config file:
  • GEMINI_API_KEY — the primary TTS engine (Google Gemini TTS).
  • OPENROUTER_API_KEY — the gpt-audio TTS fallback rung (OpenRouter).
  • Neither key is required: with no key, TTS routing falls through to the local F5 / Supertonic engines.

Install

As a plugin (Claude Code / Codex), via the agfpd marketplace:

/plugin marketplace add agfpd/agfpd-marketplace
/plugin install voice-connect@agfpd

Or run the MCP server directly from npm:

npx -y @agfpd/voice-connect@latest

The package name is @agfpd/voice-connect; the MCP entry point is the voice-connect-mcp bin. Wire it into an MCP client by pointing the client at that command, e.g.:

{
  "mcpServers": {
    "voice-connect": {
      "command": "npx",
      "args": ["-y", "@agfpd/voice-connect@latest"]
    }
  }
}

Always-on HTTP service

The HTTP facade is meant to run as a long-lived local service so runtimes can hit it without spawning a process per request. It is self-managed: it owns its own launchd agent (label com.voice-connect.http), in the package's own namespace, and writes a discovery slot (~/.iapeer/voice-provider.json) consumers read for the endpoint.

node scripts/launchd-http.mjs render      # print the plist (no writes)
node scripts/launchd-http.mjs install     # write plist + slot, (re)bootstrap
node scripts/launchd-http.mjs status      # launchctl print + slot
node scripts/launchd-http.mjs uninstall   # bootout + remove plist + slot

It binds 127.0.0.1:8127 by default (local-only); override with PEER_VOICE_HTTP_HOST / PEER_VOICE_HTTP_PORT. GET /health reports liveness.

What it does

MCP tools

| Tool | In → out | Notes | |------|----------|-------| | tts | text → .ogg/opus file | Short → { path, ... } sync; long (>~2000 chars) → { job_id } + an IAP "done" message later. | | stt | audio file path → text | Returns { text, engine, fallback_from? }. | | job_status | job_id → status | running | done | failed | unknown — pull-floor for hosts without iapeer, or when you suspect a voice-notifier message was lost. |

HTTP routes (OpenAI-compatible)

| Route | Method | Body | Returns | |-------|--------|------|---------| | /v1/audio/speech | POST | JSON (input/text, peer?, voice?, model?/engine?, lang?, style?) | Ogg/Opus bytes; engine/fallback/peer in X-Voice-* headers. | | /v1/audio/transcriptions | POST | multipart (file, language?, prompt?, engine?, response_format?) | { text }, or plain text with response_format=text. | | /health | GET | — | { status, service, version }. |

Speaking as a peer (peer). A runtime that voices messages on behalf of agents should send peer — the peer's identity — and not a voice name:

POST /v1/audio/speech
{ "input": "Проверка связи.", "peer": "boris" }
// → 200 audio/ogg
//    X-Voice-Engine: gpt-audio   X-Voice-Voice: onyx   X-Voice-Peer: boris
//    X-Voice-Fallback-From: gemini:GeminiQuotaError

A voice NAME belongs to one engine's namespace (onyx is gpt-audio's, Charon is Gemini's), so it can only be applied to the engine tried first; if the cascade falls through, that name means nothing on the rung that actually serves, which silently yields a default voice — a peer sounding like a stranger. peer carries identity instead, and the voice is looked up in that peer's interfaces.voice after the serving engine is known, so every rung speaks in their voice. It also selects that peer's F5 reference on the F5 rung, so the identity never splits between rungs.

Unknown peer, or a peer with no entry for the serving engine → the engine's built-in default (no invention, no error). voice still works and still overrides the primary engine only; if you send both, peer covers the fallback rungs. MCP callers need neither — the tool runs inside the calling peer's own process and resolves the identity itself.

Engines

TTS — first applicable engine wins; the cascade advances on a known can't-serve error:

| Engine | Kind | Model | Notes | |--------|------|-------|-------| | Gemini | cloud | gemini-3.1-flash-tts-preview | ru+en in one pass; honors style; needs GEMINI_API_KEY. | | gpt-audio | cloud | openai/gpt-audio (OpenRouter) | Multilingual one pass; honors style; needs OPENROUTER_API_KEY. | | F5-TTS | local GPU | f5-tts | Live-prosody Russian rung; per-peer voice cloning; applies on the ru route only. Needs an F5 service (PEER_VOICE_F5_URL). | | Supertonic 3 | local floor | supertonic-3 | Offline, one pass in the routed language; the floor. |

Cascade order (operator config)

Which engine is tried first is a property of the deployment — what hardware and paid accounts the host has — not of the package. Default: gemini,gpt-audio,f5,supertonic (the only order that works with zero local infrastructure). Override, highest priority first:

1. PEER_VOICE_TTS_ORDER=f5,gemini,gpt-audio,supertonic — env var; for the always-on service put it in ~/.iapeer/voice-provider.env, which is baked into the launchd plist and survives a reinstall.
2. "ttsOrder": ["f5", "supertonic"] in the config file (standalone installs).
3. Otherwise the package default.

A host with its own GPU box should put f5 first: it is free per call, where the cloud rungs are billed per response. Notes:

• Unknown names are dropped with a warning, never fatal — a typo in host config degrades to a working ladder instead of taking voice down.
• The offline floor is always present, appended if omitted, so a request can always land somewhere. Its position is left alone if you place it deliberately (supertonic,f5 = offline-only host).
• Omitted engines are genuinely absent — f5,supertonic never calls the cloud at all.
• F5 only serves Russian. On the en/na route it is skipped, not failed, so English still goes to the next rung. Putting f5 first saves money on Russian traffic only.
• Per-peer voices need refs. F5 speaks as a peer only from a cached reference sample, and refs are minted on a cloud synthesis. With f5 first the cloud rungs are rarely reached, so peers without a ref keep F5's default voice — and all sound alike. Backfill once per peer (engine: "gpt-audio" with peer), then F5 carries the identity for free.

STT — speaches when an endpoint is configured, otherwise the local floor:

| Engine | Tier | Notes | |--------|------|-------| | speaches | primary | OpenAI-compatible /v1/audio/transcriptions; set PEER_VOICE_STT_ENDPOINT. Skipped when unset. | | mlx-whisper | local floor | Offline Apple-silicon Whisper via the mlx_whisper CLI. |

tts parameters

tts(text, voice?, lang?, style?, note?, engine?, out_path?)

| Param | Type | Description | |-------|------|-------------| | text | string (required) | Text to speak. Mixed ru+en is fine — read in one pass. | | voice | string | Gemini prebuilt voice for the primary engine. Default Aoede. | | lang | ru | en | na | Language hint for the fallback engines (Gemini reads any language itself). Omit to auto-detect by character share. | | style | string | Delivery directive (tone / emotion / tempo) for the cloud engines; ignored by the local fallbacks. | | note | string | Reminder echoed back in the async "done" message (e.g. who to deliver to). Ignored for short (sync) text. | | engine | auto | gemini | gpt-audio | f5 | supertonic | Force one engine, overriding this host's configured order. Default auto (the cascade). A forced engine has no fallback — its failure propagates. | | out_path | string | Absolute output .ogg path. Default: a unique file under ~/.iapeer/cache/peer-voice/out/. |

Returns — short text (sync): { path, engine, voice, lang?, probe, fallback_from? }. Long text (async): { job_id, status: "started" }, followed by an IAP message from the system peer voice-notifier: voice job <id> done path=<path> note=<note> when synthesis finishes (or failed reason=<...>). No iapeer on the host → no message; poll job_status(job_id) instead.

stt parameters

stt(audio_path, lang?, prompt?, engine?)

| Param | Type | Description | |-------|------|-------------| | audio_path | string (required) | Absolute path to the audio file to transcribe (e.g. a received voice .ogg). | | lang | string | Language hint (ISO-639-1: en, ru, …). Omit to auto-detect. | | prompt | string | Decoder-priming prompt — biases spelling/casing of terms (e.g. keep Claude Code in Latin). Not part of the output. | | engine | auto | speaches | mlx-whisper | Force an engine. Default auto. |

Returns { text, engine, fallback_from? }.

Standalone and iapeer modes

The mode is decided by one signal: whether the caller has an iapeer identity. Both modes share the same core and the same key ladder (env → shell rc → config file).

• Standalone — no iapeer identity. Voice and keys come from one config file: $PEER_VOICE_CONFIG, else <PEER_VOICE_HOME>/config.json:

  {
    "voice": { "gemini-3.1-flash-tts-preview": "Aoede", "supertonic-3": "F3" },
    "keys":  { "GEMINI_API_KEY": "...", "OPENROUTER_API_KEY": "..." }
  }

  voice-connect only reads this file; it never writes config itself.

• iapeer — the caller has an iapeer identity (PEER_PERSONALITY or a cwd peer-profile). Voice comes from the peer-profile; keys from the host env/rc. iapeer owns the configuration.

License

Apache-2.0. Platform: macOS.