voicetalk

A live-voice channel for a team of AI agents — the voice-presence component of iapeer.

voicetalk gives a person presence in an iapeer agent team through live voice: the owner says a wake-word, speaks, and an agent answers out loud from the host's speaker. To the agent, the person is an ordinary peer it reaches with send_to_peer — voicetalk turns speech into an inbound message and the agent's reply into spoken audio. It is the third runtime of iapeer after telegram-runtime and notifier-runtime, and the same single identity an agent already has across Claude, Codex, and Telegram.

Built for iapeer. It isn't a standalone voice assistant — it runs only inside iapeer, alongside telegram-runtime and notifier-runtime. It's an iapeer runtime whose messages travel over iapeer's own routing; the speech-to-text and text-to-speech themselves live in a separate voice service (voice-connect), which voicetalk consumes.

How it works

   person speaking aloud                        agent (Claude/Codex)
        │  "Boris, …"  (wake-word + speech)         │  send_to_peer(arthur, "…")
        ▼                                            ▼
   ┌──────────────────────────────────────────────────────────┐
   │  voicetalk — the voice channel                             │
   │  inbound:  microphone → wake-word → STT → recipient peer   │
   │  outbound: peer → TTS → host speaker                        │
   │  open-mic follow-up · endpointing · earcons                │
   └──────────────────────────────────────────────────────────┘
              speech ↔ text via voice-connect (STT / TTS)

Quick start

Status: live voice (Фаза 3). The microphone, wake words, recognition and spoken replies all run; latency tuning is Фаза 4 — see the roadmap below.

You will need a working iapeer install, the voice-connect voice service running locally, and ffmpeg/ffplay on PATH.

npx @agfpd/voicetalk-runtime                      # install: binary on PATH + runtime manifest
iapeer create <person> --runtime voicetalk        # give a person a voice endpoint (operator-add)
voicetalk-runtime doctor                          # check the runtime + this peer's binding

Once a peer is created, the always-on voicetalk-runtime run router pumps IAP envelopes both directions: the microphone listens, and the owner speaks a peer's wake word to address it.

voicetalk-runtime devices                                            # list audio inputs
voicetalk-runtime interface --wake-words "дарвин=darwin,борис=boris" \
  --default-target darwin --device reSpeaker                         # set the binding
# (run is launched by the foundation under the always-on supervisor)
voicetalk-runtime say --peer linus "what's the build status"         # inject an utterance without the mic (debug)

Say «Борис, привет» and the rest of the sentence is delivered to boris; the reply is spoken on the host speaker, and the mic stays open for a follow-up. VOICETALK_AUDIO=0 runs the same router text-only, with no microphone at all. Configuration details: docs/.

What makes it different

• Transparent channel. Agents get no microphone or speaker tools — to an agent, the person is just a named peer. The same send_to_peer reaches a human by voice, by Telegram, or by an AI agent alike.
• One identity, another runtime. Voice is not a bolt-on assistant; it is one more runtime of the peer's single identity (from-runtime=voicetalk), exactly as Telegram is.
• Modality framing for a naive agent. An agent has no senses — only envelope text. voicetalk marks the inbound envelope so a peer with no knowledge of the system answers in the right modality ("spoken, live — reply by voice, briefly").
• Voice stays out of the core. Speech-to-text and text-to-speech live in the shared voice-connect service (multi-provider, one place to configure), not inside this runtime and not inside the iapeer foundation.
• Conversational turn-taking. A wake-word only opens a conversation; after a reply, an open-mic window lets the next utterance continue without re-triggering. Endpointing and earcons are owned here, inside the audio session — the agent only sees the next message.
• Wake evidence fails closed. Configured names are never injected into Whisper's decoder prompt: only an exact leading name heard in the unprimed transcript can open a dialogue, and repeated-name decoder loops are discarded before IAP delivery.

What it does not do (yet)

• No barge-in: the owner cannot interrupt a spoken answer by talking over it (that needs acoustic echo cancellation plus a neural VAD). While the runtime speaks, it does not listen.
• Wake detection runs on an unprimed transcript, not on an acoustic wake-word model — so a wake word costs one line of config and no training, at the price of transcribing every voiced segment. A bounded private ring retains wake-candidate segments for replayable diagnostics; an acoustic pre-gate remains the stronger future upgrade.
• Single host, single owner, local microphone and speaker. Multi-device and multi-owner are out of scope for the MVP.

Roadmap

• Фаза 0 — Scaffold. Package skeleton, CLI stub, contract constants. (done)
• Фаза 1 — Foundation integration (iapeer). voicetalk launch-adapter, runtime classification, iapeer create --runtime voicetalk. (adapter code-complete; deploy coordinated with Фаза 2)
• Фаза 2 — Runtime skeleton. runtime.json manifest + selfConfig hook; interfaces.voicetalk schema; bidirectional router loop with a text-only IAP round-trip (audio stubbed). (done)
• Фаза 3 — Voice pipeline. Always-on microphone → endpointing → STT → wake map (inbound), TTS → speaker (outbound) via voice-connect; turn-taking, follow-up window, earcons. (done)
• Фаза 4 — Latency and live acceptance. Tune the chain, conversational round-trip live.

Documentation

docs/ — what it is and how it will work (English; Russian in docs/ru/). This repository is the implementation.

License

Apache-2.0. Platform: macOS. voicetalk is the voice-presence runtime for the iapeer ecosystem — a component of iapeer, not a standalone system.