@codefusion-cc/audio-capture

Microphone to streaming PCM frames in the browser. It knows nothing about what the audio is for: a voice conversation, dictation or a level meter all start here.

Every frame has one format, whatever the microphone runs at: mono, PCM16 little-endian, 16 kHz, 20 to 80 ms long (default 20), numbered from 0 with the sample it starts at, so a gap (a frame dropped for a slow consumer) is visible.

import { startCapture, pcm16ToBytes } from '@codefusion-cc/audio-capture/browser'

// Call it from a click: the browser starts the audio context only after a gesture.
const capture = startCapture({ onLevel: ({ rms }) => drawMeter(rms) })

for await (const frame of capture.frames) {
  socket.send(pcm16ToBytes(frame.pcm)) // frame.sequence, frame.startSample
}
// When capture.frames ends, the capture has too: check capture.failure.
// To end it yourself: await capture.stop() (the rest of the audio arrives as a last, shorter frame).
console.log(capture.state, capture.failure, capture.droppedFrames)

Entry points

| Entry | For | Has | | --- | --- | --- | | . | Workers, servers, the page | The contract and the pure helpers: createResampler, float32ToPcm16, createFramer, measureLevel / createLevelMeter, createCapturePipeline (all four in one), pcm16ToBytes, the types | | ./browser | The page | startCapture, WORKLET_SOURCE, and everything from . | | ./react | React | useAudioCapture() | | ./testing | Apps' tests | createFakeCapture, generateTone, and everything from . |

startCapture(options)

Returns a Capture at once; the permission prompt may still be showing. States: requesting, recording, stopped, failed. It reads the microphone's real sample rate from the track (capture.inputSampleRate) and runs an AudioWorklet at that rate; the resampler converts to 16 kHz on the page.

• Failures are results. permission-denied, no-device, device-unavailable, device-lost (mid-recording; the frames until then are delivered), insecure-context, unsupported, worklet-unsupported (no AudioWorklet, or the module could not load), audio-blocked (the audio context would not start: no click) end the capture as failed with capture.failure = { code, message }, and frames ends. Only a programming mistake (frameMs outside 20 to 80) throws.
• A microphone lost mid-recording is opened again, once. Bluetooth headphones that hand over to another device or change profile, a default input that moves, a headset that goes to sleep: the browser ends the track although a microphone is still there. The capture then opens the default microphone (not deviceId: that device is the one that went away), builds a new audio graph at that microphone's own rate (a headset's 16 or 24 kHz, a built-in 48 kHz) and goes on in the same capture: the state stays recording, the frames keep their numbers, positions and length, at 16 kHz. The audio before and after is joined; onRecovered({ gapMs, atSample, loss, inputSampleRate, previousInputSampleRate }) says how much real time is missing and where (also capture.recovery and summary.recovery). A track that stays muted for three seconds on a visible page is treated the same (loss.cause: 'muted'); a hidden page's muted track is left alone and gets its three seconds when the page is back. A microphone that is muted again after the reopen is a person's mute (a key, the system's switch), not a lost device: the capture goes on, recording silence, and fails with reason: 'muted' only after twenty seconds, so a capture never records silence for good without the caller knowing. Bounded: one reopen per capture, five seconds for the browser to give the microphone, and none after stop(), on a hidden page, with the permission denied or with no audio input listed. Then the capture fails as device-lost with failure.reason (no-input, permission-withdrawn, page-hidden, lost-again, reopen-failed, muted) and failure.loss (cause, the track's label, whether devicechange fired in the two seconds before, audio inputs before and after, visibility), after delivering what was heard.
• Slow consumers. Frames wait in a queue of maxQueuedFrames (default 250, 5 s at 20 ms). When it is full the oldest frame is dropped and counted: capture.droppedFrames, onDropped(total), and the gap in sequence. With onFrame there is no queue: the callback is the consumer.
• Release. stop(), a failure, leaving the frames loop early, and (with stopOnHidden: true) a hidden page stop the microphone's tracks, disconnect the worklet and close the audio context. A page that is left or goes into the back/forward cache (pagehide) always releases it. stop() is safe at any time and twice, and never waits more than half a second for the worklet's last samples or a second for the context to close; during requesting it cancels, and a microphone the browser grants afterwards is released at once.
• Options. frameMs, echoCancellation (true), noiseSuppression (true), autoGainControl, deviceId, levelIntervalMs, onFrame, onLevel, onState, onDropped, onRecovered, workletUrl.

The worklet needs nothing from your build, unless your Content-Security-Policy says no

The capture module is a string in the package (WORKLET_SOURCE) that startCapture loads from a Blob URL, so there is no file to copy and no bundler setting (Vite, webpack and esbuild all just bundle ./browser).

Worklets are governed by script-src, not worker-src (measured in Chromium and Firefox). A page that sends script-src 'self' without blob: (HeyHubs, Peerly, dieselpaul and bigusson all do) ends the capture as worklet-unsupported. Either allow the Blob:

Content-Security-Policy: script-src 'self' blob:

or, without loosening the policy, serve the module as a file of the app's own origin and pass its address. Write it once before the build (Vite serves public/ as it is):

node --input-type=module -e "import { writeFileSync } from 'node:fs'; import { WORKLET_SOURCE } from '@codefusion-cc/audio-capture/browser'; writeFileSync('public/capture.worklet.js', WORKLET_SOURCE)"

and call startCapture({ workletUrl: '/capture.worklet.js' }) (or useAudioCapture({ workletUrl })). Write it again when the package is updated: the file is the package's own code.

React

const { state, level, failure, start, stop } = useAudioCapture({ onFrame: frame => send(frame) })
// <button onClick={start}>, level.rms for a meter

start does nothing while a capture is running (double clicks, StrictMode), and the capture stops on unmount. recovery is the lost microphone the running capture opened again, until the next start.

Tests

import { createFakeCapture } from '@codefusion-cc/audio-capture/testing'

const start = createFakeCapture({ tone: { frequency: 440, seconds: 1 }, inputSampleRate: 44100 })
const capture = start() // or useAudioCapture({ start })

The fake runs the real resampler, PCM16 conversion, framer and meter on a tone or your samples, with failure: { code, message, at: 'start' | <input sample> }, recovery: { at, gapMs, inputSampleRate, cause } (a lost microphone opened again at that input sample), stopWhenDone and realtime.