echo-voice

Client-side voice for Echo VoiceFlow: the caller owns the turn. Silero VAD (with an automatic energy fallback), push-to-talk / auto / hybrid turn control, local barge-in, reply waveform, and mute — a small package your integrators just install. Vanilla core + a React hook.

Built for the GrayPorter/Porter feedback: "nothing but the shopper should decide when their turn is over." This package makes the client end the turn (record- and-send or client VAD), and the Echo server buffers audio and waits for its commit — see endpointing below.

Install

npm install echo-voice          # onnxruntime-web + react are optional peers
# or, no build step:
<script src="https://cdn.jsdelivr.net/npm/echo-voice/dist/echo-voice.iife.js"></script>

Quick start (vanilla)

import { EchoVoice } from "echo-voice";

const voice = EchoVoice.connect({
  workflow: "ai-guru-voice",
  apiKey: "vfk_…",              // or ticket: "…"
  turn: "hybrid",               // "auto" | "manual" | "hybrid"
  endpointing: "client",        // this package owns the turn (default)
  vad: { engine: "auto", silenceMs: 700, minSpeechMs: 250 },
  metadata: { user_id: "u_42" },
});

voice.on("transcript", ({ text, final }) => …);
voice.on("reply",      ({ text, delta, final }) => …);
voice.on("waveform",   ({ bars, rms }) => drawEqualizer(bars));  // AI reply audio
voice.on("listening",  ({ on }) => …);   // local VAD: user speaking
voice.on("speaking",   ({ on }) => …);   // AI speaking

await voice.start();            // opens mic + loads the VAD

// push-to-talk (manual/hybrid):
btn.onmousedown = () => voice.startTalking();
btn.onmouseup   = () => voice.stopTalking();   // ends + commits the turn
voice.send();                   // hybrid: force-send now
voice.mute(); voice.unmute();

React

import { useEchoVoice } from "echo-voice/react";

function Mic() {
  const v = useEchoVoice({ workflow: "ai-guru-voice", ticket, turn: "hybrid" });
  return (
    <>
      <button onPointerDown={v.startTalking} onPointerUp={v.stopTalking}>Talk</button>
      <p>{v.transcript}</p><p>{v.reply}</p>
      <Equalizer bars={v.waveform} muted={v.muted} onMute={() => v.setMuted(!v.muted)} />
    </>
  );
}

Turn modes

• manual — push-to-talk / record-and-send. startTalking() … stopTalking(). Nothing but the user ends the turn. VAD is used only for barge-in.
• auto — client VAD ends the turn after silenceMs of silence. Utterances shorter than minSpeechMs are dropped as noise (no empty turns).
• hybrid — auto VAD plus a manual send() override, so a mid-sentence pause never fires but the user can force-send.

Local barge-in (default on): the moment the user starts talking over the AI, the package cancels the turn + ducks playback locally — faster than a server hop.

VAD

vad.engine: "auto" (default) loads Silero (ONNX via onnxruntime-web, from a CDN) and automatically falls back to a tiny energy gate if the model/wasm can't load. Force one with "silero" / "energy". Tune: silenceMs, minSpeechMs, preRollMs, positiveThreshold/negativeThreshold (Silero) or energyThreshold (energy), sileroModelUrl, ortBaseUrl.

For the smallest bundle, keep onnxruntime-web out of your app and let the CDN build load it lazily — the core stays tiny; Silero downloads on first start().

Endpointing (how it works with the server)

With endpointing: "client" the package connects with ?endpointing=client. The Echo server then buffers incoming audio and never auto-commits on silence — it responds only when the package sends {"type":"commit"} (at push-to-talk release, or client-VAD end). Barge-in sends {"type":"cancel"}. Set endpointing: "server" to revert to Echo's own server-side VAD (the default for everyone not using this package — nothing existing changes).

Events

status · session · resumed · transcript · reply · audio · turn · node · ended · context · output · speaking · listening · vad · waveform · level · muted · committed · error · close. voice.onAny((ev, data) => …) for all of them.

Resume + sessions

Pass session: priorSessionId to resume (rehydrates history; resumed event). voice.sessionId is the id to store against your user. See the platform's Integrate tab / docs/voiceflow-sessions.md for listing + per-user history.

Build / dev

npm i && npm run build     # → dist/ (esm, cjs, iife/CDN, .d.ts)
open demo/index.html       # live demo (paste a ticket)

Architecture

Mic (16kHz PCM) ─┬─► Transport ──WS──► Echo (buffers until commit)
                 └─► VAD (Silero|energy) ─► TurnController ─► commit / cancel / barge-in
Echo ──tts.chunk──► Player (gapless) ─► gain(mute) ─► analyser(waveform) ─► speakers

Files: core.ts (glue) · transport.ts (WS protocol) · audio/mic.ts · audio/player.ts · vad/{energy,silero,index}.ts · turn.ts · react/.