vocule

private speech to text that runs entirely in the browser. vocule is a custom inference engine built around one model, parakeet redux, running on webgpu and webassembly in a worker. audio never leaves the device: there is no speech server, cloud fallback or framework dependency, and one typed api covers files, live speech and recording.

website · npm · issues

model at a glance

| detail | value | | --- | --- | | model | parakeet redux by moondream, built from nvidia parakeet tdt 0.6b v3 | | task | speech to text | | weights | 177,774,490 bytes from vocule's cdn, or hugging face if that fails; sha-256 verified, cached by the browser | | revision | exact pin | | audio | 16 khz mono; other rates and channels are converted | | languages | 25 listed upstream; tested in english, spanish and french | | engine | webgpu + wasm, in a dedicated worker off the ui thread | | speed | 78.4×: 78 s of audio per second on long talks | | preparation | 0.30 s with the model cached (macbook air m4) | | photon | moondream's native runtime: 57.3× at its fastest (mps) on the same talks; vocule 1.37× faster | | word error | 4.832% in the browser; 4.817% photon cpu | | exact text | the same text as photon on every short test clip | | license | npm package mit; source private; model cc by 4.0 |

what 78× means

78× means about 78 seconds of recorded audio processed in one second, so one hour takes about 46 seconds after model preparation. on the same eleven talks and the same macbook air m4, unmodified photon, moondream's native runtime for this model, ran at 37.0× on the cpu by default, 49.6× on the cpu with 10 threads and 57.3× on mps, its default device on apple silicon; vocule took 27% less time than photon's fastest setting. photon is faster on short clips: warm, vocule takes 16% less time than photon's cpu default but 1.15× photon's mps time, and after the machine idles it takes 1.05–1.30× photon's time. the benchmarks below have the full table.

install

bun install @karanganesan/vocule
pnpm install @karanganesan/vocule
npm install @karanganesan/vocule

quick start

import { createSpeech } from "@karanganesan/vocule";

const speech = createSpeech();
try {
  await speech.prepare();
  const transcript = await speech.transcribe(file);
  console.log(transcript.text, transcript.metrics.realtimeFactor);
} finally {
  speech.dispose();
}

createSpeech() makes no request and asks for no permission. prepare() downloads and verifies the model, caches it for later visits, and warms the engine. it is optional: transcribe(), listen() and record() each prepare the model when it is not ready, and calls that overlap share one preparation. speech.ready says whether this instance is prepared. reuse one instance for sequential work and create another for concurrent inference.

benchmarks

measured on a macbook air m4 (10-core gpu, 32 gb) in chrome 153, against unmodified photon, moondream's native runtime for the same model, in each of its settings. every engine ran interleaved on the same machine with cooldowns between calls, on a prepared model. bold marks the best result in each row.

| workload | photon cpu, default | photon cpu, 10 threads | photon mps | vocule | | --- | ---: | ---: | ---: | ---: | | 11 long ted-lium talks, 2.5 hours of audio | 37.0× real time | 49.6× | 57.3× | 78.4× | | word errors on those talks (26,801 words) | 1,291 | 1,286 | 1,294 | 1,295 | | 8 short clips of 3–11 seconds, warm | 1,257 ms | 1,008 ms | 920 ms | 1,062 ms | | the same clips after 2 seconds idle | 1,265 ms | 1,025 ms | 1,223 ms | 1,327 ms |

• long recordings: vocule is 1.37× as fast as photon's fastest setting and 2.1× as fast as its default, with one more word error than photon's least accurate setting (1,295 against 1,286–1,294).
• short clips: photon's fastest settings are still ahead on single short clips; vocule beats photon's default cpu setting when the machine is warm.
• exact text: every short-clip transcript matched photon's text exactly, in chrome and in safari 26.6.
• live: the final text arrives 202 ms after the last audio chunk of an 11-second clip.

these numbers cover one laptop and one browser build. word error uses a simple local normalizer, so it is not comparable with published leaderboards. check your own devices before promising a speed.

features

• complete files: a File, Blob, url, Request, Response, encoded bytes, AudioBuffer, webcodecs AudioData, a file handle, pcm or a finite stream of any of them. long recordings are split at natural pauses automatically, with no duration cap.
• live speech: revisable drafts every 100 ms while someone speaks, settled segments that never change, and backpressure that fails clearly instead of falling behind.
• recording: pause and resume, uncompressed audio, playback, a wav export, and transcription of the finished clip.
• capture helpers: microphone, camera and screen-share audio, without lossy re-encoding.
• verified model: the weights and metadata are pinned by hash and checked on every load; a mirror can serve them, never a different model.
• small imports: model metadata alone is about 1 kb; the engine loads when inference starts.
• typed: every option, result and error type is exported.

transcribe

const result = await speech.transcribe(file, {
  signal: controller.signal,
  timestamps: "none",
});
console.log(result.text, result.metrics.audioSeconds, result.warnings);

await speech.transcribe({ samples: monoFloat32, sampleRate: 48_000 });
await speech.transcribe(float32Samples, { sampleRate: 16_000 });

uncompressed wav is read directly; other formats use the browser's decoder. multichannel audio is mixed to mono and other sample rates are converted to 16 khz. a MediaStream has no natural end, so it belongs in listen() or record().

live speech

const live = await speech.listen({
  onUpdate: ({ kind, text, settledText, draftText }) => render(text),
  onError: (error) => console.error(error),
});

const finalText = await live.stop();

listen() asks for the microphone when called, or uses { stream } if you pass one; a supplied stream stays yours. speech.realtime() takes pcm you push yourself and asks for no permission:

const live = speech.realtime({ onUpdate: (update) => render(update.text) });
live.push({ samples: chunk, sampleRate: 16_000 });
const finalText = await live.stop();

each update carries the whole view: text is the settled text followed by the current draft, and kind is "draft", "settled" or "final". stop() drains pending work and returns the settled text; cancel() discards it.

a live session starts preparing the model as soon as it opens, and audio that arrives first waits for it. if the model cannot load, the session ends with that error through onError and stop(). to start with a ready model, await speech.prepare() first.

record

const session = await speech.record();
session.pause();
session.resume();

const clip = await session.stop();
const player = await clip.play();
const transcript = await clip.transcribe();
console.log(transcript.text, clip.durationSeconds, clip.wav);
player.pause();

while you record, the model prepares in the background, so the clip is ready to transcribe sooner. capture never waits for the model, and if that preparation fails, clip.transcribe() prepares again and reports any error. a clip is also { samples, sampleRate }, so speech.transcribe(clip) works too. paused audio is left out of the clip, and session.cancel() releases capture and discards it.

configuration

createSpeech(options)

| option | type | default | effect | | --- | --- | --- | --- | | onProgress | (event: Progress) => void | none | reports download, verify, load, features, encode and decode progress | | cache | boolean | true | keeps the verified model in browser cache storage | | modelBaseURL | string | vocule's cdn, then hugging face | a directory serving the pinned revision's config.json, ternary.json, tokenizer.json and model.safetensors, used instead of the defaults with no fallback; hashes still apply | | artifactHashes | Record<"config.json" \| "ternary.json" \| "tokenizer.json", string> | pinned hashes | only for a mirror that intentionally serves different metadata bytes | | assetBaseURL | string \| URL | inline worker and worklet | a same-origin directory holding a copy of dist/, for strict content security policies | | workerFactory | () => Worker | none | builds the worker yourself; takes priority over assetBaseURL |

speech.transcribe(audio, options)

| option | type | default | effect | | --- | --- | --- | --- | | signal | AbortSignal | none | cancels the call | | sampleRate | number | none | required for a bare Float32Array | | timestamps | "none" \| "segment" \| "word" | "none" | only "none" is supported; the others raise CAPABILITY | | profileGpu | boolean | false | adds a gpu timing trace to metrics.gpuProfile where the browser allows it |

speech.realtime(options) and speech.listen(options)

| option | type | default | effect | | --- | --- | --- | --- | | onUpdate | (update: RealtimeUpdate) => void | none | receives every draft, settled and final view | | onError | (error: unknown) => void | none | reports a background failure; stop() also rejects | | signal | AbortSignal | none | cancels the session | | draftIntervalMs | number | 100 | time between draft attempts while speech is active; 0 turns drafts off | | vad | boolean | true | false settles fixed-length segments without detecting speech | | speechThreshold | number | 0.006 | rms level that opens an utterance | | silenceMs | number | 700 | trailing silence that settles an utterance | | minSpeechMs | number | 120 | continuous speech needed to open an utterance | | preRollMs | number | 240 | audio kept from before speech started | | maxUtteranceMs | number | 18000 | hard segment boundary; no audio is dropped | | maxQueuedSeconds | number | 45 | buffered audio allowed before BACKPRESSURE |

listen() also accepts stream, audio (MediaTrackConstraints, used only without a stream), workletURL, captureSampleRate (default 16000) and onLevel.

speech.record(options)

signal, stream, audio, workletURL, captureSampleRate, onLevel and onError, with the same meaning as for listen().

the result

transcribe() resolves to a Transcript: text; model (id, revision, sha256); metrics with audioSeconds, wallMs, inferenceMs, realtimeFactor and, for long recordings, the windows it used; warnings; segments, which is empty because timestamps are not produced; and verified, which stays false until a broad accuracy evaluation exists.

errors

every failure is a SpeechError with a stable code:

| code | meaning | | --- | --- | | ABORTED | the call or session was cancelled | | AUDIO_INVALID | the input could not be read or decoded | | AUDIO_TOO_LONG | reserved; complete files have no duration cap, so no current call raises it | | BACKEND_UNAVAILABLE | no usable webgpu adapter, or the worker could not start | | CAPABILITY | an unsupported option (such as timestamps) or a missing browser capability (such as an audio track or capture api) | | NETWORK | the model could not be downloaded | | INTEGRITY | downloaded bytes did not match the pinned hashes | | MODEL_FORMAT | the model files were not what the engine expects | | BUSY | another transcribe() or live session is running on this instance, or a session is not in the right state for the call | | BACKPRESSURE | live audio arrived faster than it could be transcribed | | DISPOSED | the instance was disposed | | GPU_LOST, GPU_ERROR | the gpu device was lost or failed; the next call prepares again | | DECODE_LIMIT | a safety limit stopped decoding; no truncated transcript is returned |

types

every public type is exported from the package root, for example:

import type {
  AudioInput,
  ListenOptions,
  RealtimeUpdate,
  Recording,
  RecordingSession,
  SpeechOptions,
  Transcript,
} from "@karanganesan/vocule";

more entry points

| import | use | | --- | --- | | @karanganesan/vocule/model | MODEL metadata (id, revision, hash, size, license) and the model's source urls (MODEL_SOURCES), without the engine | | @karanganesan/vocule/microphone | captureMicrophone() for a short clip (up to 18 seconds) and streamMicrophoneAudio() for raw pcm blocks | | @karanganesan/vocule/video | captureVideo({ source: "camera" \| "screen" }) returns the video and its raw audio | | @karanganesan/vocule/diagnostics | runDiagnostics(), a quick self-test of this browser's webgpu and webassembly support | | @karanganesan/vocule/worker | the worker module, for workerFactory or your own hosting |

strict content security policies

by default the worker starts from a blob: url, loads its webassembly from data: urls, and the capture worklet starts from a data: url, so no extra files need hosting. a policy therefore needs blob: in worker-src and data: in connect-src; without the latter, prepare() fails with BACKEND_UNAVAILABLE. if your policy allows only same-origin workers and scripts, copy the package's dist directory to your site and point vocule at it:

mkdir -p public/vocule
cp -R node_modules/@karanganesan/vocule/dist/. public/vocule/

const speech = createSpeech({ assetBaseURL: "/vocule/" });

the directory must be on the page's origin, with its files at their relative paths. vocule then needs no blob: or data: source: worker-src 'self' and script-src 'self' 'wasm-unsafe-eval' are enough, plus a connect-src that allows the model's sources: https://cdn.karanganesan.com, and for the fallback https://huggingface.co with its download hosts https://*.hf.co, or just your modelBaseURL.

requirements and limits

• webgpu, webassembly and a secure context (https or localhost); capture also needs browser audio apis and permission.
• tested in chrome 153 and safari 26.6 on apple silicon; test other browsers and devices before listing them as supported.
• the first prepare() downloads about 178 mb; later visits load it from the browser cache and prepare in about 0.3 s on a macbook air m4.
• word or segment timestamps are not available yet.
• live drafts are re-transcribed windows of recent audio, not streaming tokens.

credits and licenses

• model: parakeet redux by moondream, licensed cc by 4.0, built from nvidia parakeet tdt 0.6b v3. vocule downloads these weights from its cdn copy of the pinned revision, which carries the model card, or from hugging face; keep the attribution if you redistribute them.
• reference runtime: photon, moondream's native runtime built on kestrel, is the reference vocule is measured against and whose handling of long recordings it follows.
• inspiration: husky showed the model-specific engine approach.
• vocule: the name is a rare word for a small speech sound.

license

the @karanganesan/vocule package on npm is free to use under the mit license, exactly as published; see license in the package. vocule's source code is not open source and is not distributed: the package contains built, minified code only. the parakeet redux weights, which vocule downloads at runtime, are licensed separately under cc by 4.0.

issues

bugs, questions and feature requests go to karanganesan/vocule-public, vocule's public issue tracker.

let's talk

for anything that doesn't fit an issue, such as questions, ideas or something you are building with vocule, write to [email protected].