@fractallambda/voicebox-client

Thin TypeScript client for the voicebox TTS/STT service. Wraps the batch and streaming APIs, handles the WebSocket framing + seq ordering for you, and includes a Web Audio player for low-latency browser playback. Zero runtime dependencies — uses the platform fetch and WebSocket.

See the USAGE guide for the underlying HTTP/WS contract.

Install

npm install @fractallambda/voicebox-client

Requires global fetch + WebSocket: browsers and Node ≥ 22. On older Node, pass a WebSocket implementation:

import WebSocket from "ws";
new VoiceboxClient({ baseUrl, webSocketImpl: WebSocket });

Usage

import { VoiceboxClient } from "@fractallambda/voicebox-client";

const client = new VoiceboxClient({ baseUrl: "http://localhost:8080" });

await client.health();                          // { status: "ok", tts, stt }
const { voices } = await client.voices();        // available voice ids

// Batch synthesis -> a complete WAV clip
const { audio, sampleRate } = await client.synthesize({ text: "Welcome." });

// Batch transcription (Blob | ArrayBuffer | Uint8Array)
const { text } = await client.transcribe(audio, { filename: "clip.wav" });

// Custom terms: corrected in the transcript (e.g. "mithaler" -> "Mythallar")
await client.transcribe(audio, { vocabulary: ["Mythallar", "Neverwinter"] });

// Large corpora: register a named lexicon once, then reference it by name
await client.registerLexicon("campaign", ["Mythallar", "Neverwinter", /* …1000s… */]);
await client.transcribe(audio, { lexicon: "campaign" });
// also: listLexicons(), getLexicon(name), deleteLexicon(name)

Streaming (low time-to-first-audio)

stream() yields each clause's audio as soon as it's synthesized:

for await (const chunk of client.stream({ text: "First clause. Second clause." })) {
  // chunk: { seq, sampleRate, end, text, pcm: Int16Array }  — already in order
  myAudioSink.write(chunk.pcm, chunk.sampleRate);
}

Incremental — push text as your dialogue model produces it (tokens), and get audio clause-by-clause without waiting for the full reply:

async function* tokens() {
  for await (const t of myLLMStream) yield t; // any (async) iterable of text fragments
}
for await (const chunk of client.streamText(tokens(), { voice: "af_heart" })) {
  myAudioSink.write(chunk.pcm, chunk.sampleRate);
}

Cancel mid-stream (barge-in) with an AbortSignal:

const ac = new AbortController();
const it = client.stream({ text: longReply }, { signal: ac.signal });
// ...later: ac.abort();  // closes the socket, stops synthesis server-side

Codec (bandwidth)

By default (codec: "auto") the SDK requests Opus when it can decode it — i.e. in browsers via WebCodecs AudioDecoder — which cuts streaming bandwidth ~16× vs raw PCM, and falls back to PCM otherwise. stream() always yields decoded PCM AudioChunks, so your code doesn't change.

new VoiceboxClient({ baseUrl, codec: "opus" }); // force opus
client.stream({ text }, { codec: "pcm" });       // force pcm for one call

On Node there's no WebCodecs, so auto stays PCM. To use Opus on Node, inject a decoder:

new VoiceboxClient({
  baseUrl,
  codec: "opus",
  opusDecoderFactory: ({ sampleRate, channels }) => myOpusDecoder(sampleRate, channels),
});
// the factory returns { decodePackets(packets: Uint8Array[]): Int16Array | Promise<Int16Array> }

Browser playback

speak() streams and plays gaplessly via Web Audio:

const audioContext = new AudioContext();
await client.speak(
  { text: "Welcome, traveler. Sit by the fire." },
  { audioContext, onChunk: (c) => console.log("clause:", c.text) },
);

Or reassemble a stream into one downloadable WAV:

import { pcmToWav } from "@fractallambda/voicebox-client";

const chunks = [];
for await (const c of client.stream({ text })) chunks.push(c.pcm);
const blob = new Blob([pcmToWav(chunks, 24000)], { type: "audio/wav" });

API

| Method | Returns | Notes | |--------|---------|-------| | health() | Health | status is "ok" once both backends load | | voices() | Voices | voice ids for the active TTS backend | | synthesize(req, signal?) | SynthesisResult | batch; whole WAV + sampleRate | | transcribe(audio, opts?) | Transcript | audio: Blob/ArrayBuffer/Uint8Array; opts.vocabulary, opts.lexicon | | stream(req, opts?) | AsyncGenerator<AudioChunk> | WebSocket; ordered clause chunks | | streamText(source, opts?) | AsyncGenerator<AudioChunk> | incremental: push text fragments (LLM tokens) | | speak(req, { audioContext, ... }) | Promise<void> | browser; streams → Web Audio | | registerLexicon(name, terms) | LexiconInfo | register/replace a named lexicon | | listLexicons() | LexiconInfo[] | list registered lexicons | | getLexicon(name) | LexiconInfo | get one lexicon's info | | deleteLexicon(name) | {deleted} | remove a lexicon |

Helpers: pcmToWav, int16ToFloat32, pcmDurationSeconds, WebAudioPlayer, streamTts.

Develop

npm install
npm run build         # tsc -> dist/
npm run typecheck

# Integration smoke test against a running instance (mock backend is fine):
VOICEBOX_URL=http://localhost:8080 npm test