audiolab-live (sidecar)

Watches an audio stream on your own machine with the same stream detectors as the live monitor on the site, and sends out events: a name, a time, some numbers and a reason in plain words. Triggers can react to them (log, webhook, exit).

The audio never leaves the machine. It is read, measured and thrown away. The only thing that can leave is an event, and only if you set up a webhook.

Status: first version (WP-54, phase 2 of the live plan). Private, not published anywhere.

First event in ten minutes

You need Node 22 or newer and this repository (the sidecar uses the engine in node_modules/@audiolab/engine, which is not a public package; see "Limits").

1. Make a test stream, a 16 s phone-style line with a 200 ms gap at 5 s and clipping from 10 s to 12 s:

   node sidecar/examples/make-test-wav.mjs line.wav

2. Play it as a live stream (4 times faster than real time; leave out --speed for real time):

   node sidecar/bin/audiolab-live.mjs line.wav --speed 4

   The first event arrives 20 ms after the gap begins:

       5.02s  RAISED  dropout        began 5.00s. The audio has been digitally silent for 20 ms or more: the samples are exactly zero, as when packets are lost.
       5.22s  CLOSED  dropout        lasted 0.20 s, 5.00s to 5.20s. The audio was digitally silent for 200 ms: ...
      10.04s  RAISED  clipping       began 10.02s. Samples are stuck at the peak: 2 runs of 3 or more in a row within 0.5 s. The signal is clipping.

   No file at hand? node sidecar/bin/audiolab-live.mjs --demo does steps 1 and 2 in one go.

3. See what a rule would do, without sending anything (--dry-run):

   node sidecar/bin/audiolab-live.mjs line.wav --speed max --config sidecar/examples/phone-line.json --dry-run

   It prints lines such as [dry-run] rule "packet-loss" would POST to https://example.com/hooks/audiolab-live (headers authorization: ***): {...}. Header values are never printed. Nothing is sent and an exit action is not carried out.

4. Point a rule at your own receiver (edit the url, put the token in the environment, not in the file) and leave out --dry-run:

   AUDIOLAB_HOOK_TOKEN=... node sidecar/bin/audiolab-live.mjs line.wav --config my-rules.json

5. Your own audio. The sample files in public/samples/voice/ are MP3, so convert one first (ffmpeg), or pipe it:

   ffmpeg -i public/samples/voice/speech-female-plain.mp3 -ac 1 -ar 16000 plain.wav
   node sidecar/bin/audiolab-live.mjs plain.wav --speed max
   ffmpeg -i in.mp3 -f mulaw -ar 8000 -ac 1 - | node sidecar/bin/audiolab-live.mjs --stdin --rate 8000 --format mulaw

   (That sample does end in 7.5 s of exact digital zeros, and the monitor says so. That is the monitor working, not a fault in it.)

Ctrl+C stops and prints a summary: stream time, events and episodes per type, triggers fired, and what the webhooks delivered and lost. --json prints events and the summary as JSON lines on stdout; messages and warnings always go to stderr.

Inputs

| Input | How | |---|---| | WAV file as a stream | audiolab-live file.wav [--speed 1\|4\|max]. PCM 8/16/24/32 bit, float, mu-law; any channel count (mixed down to mono, as the site does). | | Raw PCM on a pipe | --stdin --rate 8000 --format s16le\|f32le\|mulaw | | WebSocket media stream | --ws-listen 8765 --ws-format mulaw-json (JSON messages start, media, stop, payload base64 mu-law, 8 kHz, as phone platforms send them) or --ws-format pcm16 (binary frames, little-endian, --rate, default 8000). One connection is one stream with its own monitor and summary. |

The WebSocket server listens on 127.0.0.1 only. It has no login and no TLS: bind wider (--ws-host) only behind your own proxy or firewall. Connections that carry an Origin header (a web page in a browser) are refused. With a two-track stream only the inbound track is read (--ws-track outbound|any to change); if the sequence numbers show media messages that never arrived, the sidecar says so on stderr (it does not invent an event for them).

Sample rates: the engine takes 4 kHz and up, so 8 kHz phone audio is used as it is. Below 4 kHz the stream is upsampled (linearly) first.

Events

Each event is one JSON object ("schema": "audiolab-live/1"). It is sent when the monitor raises it ("update": "raised") and again when it ends ("update": "closed").

| Field | Meaning | |---|---| | type | dropout, clipping, level_drop, dead_air, hum (bandwidth_drop exists in the engine but is off) | | update | raised or closed | | t | stream time (s) at which the monitor raised the event | | at | stream time (s) of this update (the moment a closed was known) | | started_at, ended_at | when the condition began and ended; ended_at is null while it lasts. Rough for hum, see below | | duration_s | known when closed, null before | | status | always "signals" | | reason | why, in plain words, from the engine | | evidence | the numbers behind it | | stream_id, seq, id | a random neutral code per stream (or the one you give with --stream-id), a counter, the engine's event number |

Stream time is counted in samples. For a file it is the position in the file; for a live source it is the audio you have received, not the wall clock.

What each detector means, and its status

The status of every detector is signals: none is validated on live streams yet, so an event is a signal, not a diagnosis, and nothing here is "detected" in the validated sense. (Wording as on the live page.)

| Type | Rule | How soon | |---|---|---| | dropout | a run of exact zeros of 20 ms or more, as lost packets leave | a fraction of a second | | clipping | runs of samples stuck at the loudest level, twice within half a second | a fraction of a second | | level_drop | the loud parts of the speech 12 dB or more under their usual level, for 2 s | a few seconds | | dead_air | nothing above -55 dBFS for 10 s | after the 10 s | | hum | a 50 or 60 Hz comb that stays when the speech goes | a few seconds; the times are rough (a hum is seen in the pauses, so when it began and ended can be a few seconds off) |

Other limits stated on the live page: a level drop is measured on the loud parts of the speech against their usual level, and can still fire when a television or a second voice fills the pauses; one channel is read. One more from our own test bench, not fixed yet: a very low voice can raise a hum event. (A steady background that stopped in a pause used to read as a level drop; since the engine of 19 October 2026 the level rule asks for the rises of speech, not a single step.) No hit rates or false alarm rates are claimed here; they are not published until they have been measured on material the detectors were not tuned on.

To tune the engine, put its options in the config under "monitor" (for example {"deadAir": {"thresholdDb": -50}}); they are passed on as they are. The defaults are the engine's and are proposals, not tuned values.

Triggers (config file)

{
  "monitor":  { },
  "delivery": { "queue_max": 1000, "timeout_ms": 5000, "max_attempts": 6, "backoff_ms": 500, "backoff_max_ms": 30000 },
  "rules": [
    { "name": "packet-loss", "when": "dropout", "cooldown_s": 10,
      "actions": [ { "type": "log" }, { "type": "webhook", "url": "https://...", "headers": { "authorization": "Bearer ${TOKEN}" } } ] }
  ]
}

| Key | | |---|---| | when | an event type, or a list of them | | update | raised (default) or closed | | min_duration_s | only events that lasted at least this long; needs the event to be over, so it implies closed | | min_evidence | {"runs": 3}: only when that number in the evidence is at least the value | | cooldown_s | after it fires for a stream and a type, stay quiet for this many seconds of stream time (default 0). A flapping line makes one event per gap; use a cooldown on any rule with a webhook | | actions | log (a line on stderr), webhook (url, headers), exit (code, default 1: stops the stream, prints the summary, exits with that code) |

${NAME} in a url or header is replaced from the environment, so a token need not be in the file. Mistakes in the file stop the start with a message that says what to change. examples/phone-line.json (packet loss, long gaps, clipping, dead air) and examples/tts-monitor.json (clipping, level drop, hum on TTS output) are starting points, not tuned settings.

Webhooks

A POST of JSON: {"schema", "kind": "trigger", "rule", "event": {...}}. The event is the object above and nothing else: numbers and text from the engine, never audio.

• Sending never waits for the network. A slow or dead receiver cannot slow the monitoring.
• Events wait in a bounded in-memory queue (queue_max, default 1000). Each is tried up to max_attempts times, with exponential back-off (backoff_ms doubling, never more than backoff_max_ms) on network errors, timeouts, 5xx, 408 and 429. Other 4xx answers are not retried (the message tells you to check the token or the url).
• If the queue is full the new event is dropped and counted. Nothing is lost silently: drops are logged (the first, then every 100th), and the summary has queue full, gave up, refused and not delivered at shutdown, added up as LOST.
• At shutdown it waits up to --drain-s seconds (default 5) for the queue.
• The queue lives in memory: events not delivered when the process dies are gone.

What leaves the machine, and what does not

| Stays on the machine | Can leave (only with a webhook you set up) | |---|---| | the audio, in any form: samples, frames, spectra, a recording | event type, stream time, reason, evidence numbers, status, stream_id, rule name |

The default stream_id is a random code. A call id or a phone number can be personal data, so --stream-id should be a code of your own that means nothing outside your systems. The reason texts come from the engine's measurements and contain no content of the speech. This README is not legal advice; the privacy text for this route still has to be read by a lawyer before it goes on the site.

Limits

• The engine (@audiolab/engine) is a private package. The sidecar runs inside this repository; to ship it on its own the engine's live entry has to be bundled or published first. That is a decision, not done.
• Not run: the 24 hour soak. What is run: 10 minutes of stream time (16 kHz, with a flapping line, about 4900 events) with the held memory measured before and after the last 8 minutes (test/soak.test.mjs); it did not grow.
• Clock drift and a stalled input are not handled: the monitor counts the samples it receives. If a source stops sending, no event is raised for the silence of the pipe; if its clock runs fast or slow against the wall clock, stream time follows the samples. Sequence gaps in a WebSocket stream are reported on stderr only.
• Only gaps that are zeros in the audio are dropouts. A platform that skips frames instead of sending zeros shows up as the sequence warning, not as an event.
• Mono only; stereo files are averaged. Linear upsampling below 4 kHz is crude.
• The WebSocket server is minimal (RFC 6455 subset: text, binary, ping, close, fragmentation; no compression, no TLS, no login).
• Webhook queue is in memory only.

Tests

cd sidecar && npm test        # node --test, no extra dependencies

They cover: a known 200 ms gap and a clipping run found on time, mu-law decoding on known values, a trigger with cooldown firing once, --dry-run sending nothing (a spy on fetch, with the same rules shown to post when not dry), a receiver that answers 500 or never answers (monitoring not slowed, every event accounted for), the WebSocket server with a phone-style stream, and the short soak.