meljs

Pure JavaScript mel spectrogram, FFT, and audio feature extraction for ML/AI.

Zero dependencies. Runs in browsers (including Web Workers, WebGPU pipelines) and Node.js. NeMo-compatible output validated against ONNX reference models.

Built for developers working with speech recognition, audio ML, and ONNX Runtime Web who need stateful, cacheable audio feature extraction, something ONNX preprocessor models can't provide.

Why meljs?

ONNX Runtime Web provides neural network inference but not audio preprocessing. Most ASR pipelines ship an ONNX preprocessor model (e.g. nemo128.onnx) to compute mel spectrograms, but ONNX models are stateless black boxes, every call recomputes everything from scratch. This creates two problems:

1. No caching for streaming. In real-time ASR, you process overlapping windows (e.g. 5s window, 1s hop). With an ONNX preprocessor, you recompute the mel spectrogram for the full 5s every time, even though 4s of audio is identical to the previous call.

2. No pipeline parallelism. The ONNX preprocessor runs on the same thread/session as inference, so encoding can't start until preprocessing finishes.

meljs solves both:

• Stateful & cacheable, IncrementalMelSpectrogram caches prefix frames across calls. In a 70% overlap streaming scenario, ~70% of frames are reused from cache, computing only new audio.
• Background pipeline, Because it's pure JS, meljs can run in a dedicated Web Worker, continuously producing mel frames as audio arrives. When the encoder needs features, they're already computed, 0.0ms preprocessing latency in the inference path.
• Pure JavaScript, no WASM, no native bindings, no model download, no build step
• NeMo-compatible, validated against NVIDIA NeMo's ONNX preprocessor (max error < 3.6e-4)
• Precision, Float64 STFT matching ONNX's double-precision pipeline
• Configurable, works with any mel bin count, FFT size, hop length, sample rate

Install

npm install meljs

Quick Start

Basic Usage

import { MelSpectrogram } from 'meljs';

// Create processor (NeMo-compatible defaults)
const mel = new MelSpectrogram({ nMels: 128 });

// Process audio (Float32Array, mono, 16kHz)
const { features, length } = mel.process(audioFloat32Array);
// features: Float32Array [128 × length], normalized
// length: number of valid frames

Streaming with Incremental Caching

For real-time applications with overlapping windows, IncrementalMelSpectrogram caches the prefix frames and only computes new ones:

import { IncrementalMelSpectrogram } from 'meljs';

const mel = new IncrementalMelSpectrogram({ nMels: 128 });

// First chunk: full computation
const r1 = mel.process(audioWindow1, 0);

// Second chunk: 70% overlap → reuses ~70% of frames
const overlapSamples = Math.floor(audioWindow2.length * 0.7);
const r2 = mel.process(audioWindow2, overlapSamples);
console.log(r2.cachedFrames, r2.newFrames);
// e.g., 347 cached, 153 computed, ~2x speedup

// Reset for new recording
mel.reset();

Low-Level Building Blocks

Use individual functions for custom pipelines:

import {
  hzToMel, melToHz,           // Slaney mel scale conversion
  createMelFilterbank,         // Triangular filterbank matrix
  createPaddedHannWindow,      // Symmetric Hann window
  precomputeTwiddles, fft,     // Radix-2 Cooley-Tukey FFT
  MEL_CONSTANTS,               // NeMo-compatible constants
} from 'meljs';

// Create a custom mel filterbank for 22050 Hz, 1024-point FFT
const fb = createMelFilterbank(80, 22050, 1024);

// Run FFT on a frame
const N = 512;
const tw = precomputeTwiddles(N);
const re = new Float64Array(N);
const im = new Float64Array(N);
// ... fill re with windowed audio frame ...
fft(re, im, N, tw);

Web Worker, Continuous Mel Producer

The most powerful pattern: run meljs in a dedicated Web Worker that continuously converts audio chunks into mel frames. The inference thread never waits for preprocessing, features are always ready.

// mel.worker.js, runs in background, fed audio chunks continuously
import { MelSpectrogram, MEL_CONSTANTS } from 'meljs';

const mel = new MelSpectrogram({ nMels: 128 });
let audioBuffer = new Float32Array(0);
let melBuffer = null;  // accumulates raw mel frames
let totalFrames = 0;

self.onmessage = ({ data }) => {
  if (data.type === 'push_audio') {
    // Append new audio, compute new mel frames incrementally
    // ~0.5ms per 80ms audio chunk
    const chunk = new Float32Array(data.audio);
    // ... append to buffer, compute new frames, store in melBuffer ...
  }

  if (data.type === 'get_features') {
    // Inference thread requests features, already computed!
    // Just slice from buffer and normalize. ~1-3ms.
    const { startFrame, endFrame } = data;
    const features = mel.normalize(melBuffer, totalFrames, endFrame - startFrame);
    self.postMessage({ features }, [features.buffer]);
  }
};

// main thread, inference sees 0.0ms preprocessing
const melWorker = new Worker(new URL('./mel.worker.js', import.meta.url));

// Feed audio continuously as it arrives from microphone
audioEngine.onChunk(chunk => melWorker.postMessage({ type: 'push_audio', audio: chunk }));

// When inference needs features, they're already computed
const features = await requestFeaturesFromWorker(startFrame, endFrame);
model.transcribe(null, { precomputedFeatures: features }); // Preprocess: 0.0ms

API Reference

MelSpectrogram

Main class for computing log-mel spectrogram features.

| Option | Default | Description | |--------|---------|-------------| | nMels | 128 | Number of mel bins (80 or 128 typical) | | sampleRate | 16000 | Audio sample rate in Hz | | nFft | 512 | FFT size | | winLength | 400 | STFT window length in samples | | hopLength | 160 | STFT hop length in samples | | preemph | 0.97 | Pre-emphasis coefficient (0 to disable) | | logZeroGuard | 2^-24 | Guard value for log computation |

Methods:

• process(audio), Full pipeline: audio → normalized log-mel features
• computeRawMel(audio), Compute un-normalized log-mel (for custom normalization)
• normalize(rawMel, nFrames, featuresLen), Apply Bessel-corrected normalization
• samplesToFrames(samples), Convert sample count to frame count
• framesToSamples(frames), Convert frame count to sample count

IncrementalMelSpectrogram

Extends MelSpectrogram with frame-level caching for streaming.

| Option | Default | Description | |--------|---------|-------------| | boundaryFrames | 3 | Extra frames to recompute at cache boundary | | (plus all MelSpectrogram options) | | |

Methods:

• process(audio, prefixSamples), Returns {features, length, cached, cachedFrames, newFrames}
• reset() / clear(), Clear the frame cache

Standalone Functions

| Function | Description | |----------|-------------| | hzToMel(freq) | Convert Hz to Slaney mel scale | | melToHz(mel) | Convert Slaney mel to Hz | | createMelFilterbank(nMels?, sampleRate?, nFft?) | Create triangular mel filterbank | | createPaddedHannWindow(winLength?, nFft?) | Create zero-padded Hann window | | precomputeTwiddles(N) | Precompute FFT twiddle factors | | fft(re, im, N, tw) | In-place radix-2 Cooley-Tukey FFT |

Constants

import { MEL_CONSTANTS } from 'meljs';
// { SAMPLE_RATE, N_FFT, WIN_LENGTH, HOP_LENGTH, PREEMPH, LOG_ZERO_GUARD, N_FREQ_BINS, DEFAULT_N_MELS }

Accuracy

Validated against NVIDIA NeMo's ONNX preprocessor (nemo128.onnx):

| Metric | Value | |--------|-------| | Mel filterbank max error vs ONNX | 2.645e-7 | | Full pipeline max error | 3.6e-4 | | Full pipeline mean error | 1.1e-5 | | Test signals validated | Sine, chirp, white noise, speech |

Performance

Node.js Benchmarks

Benchmarks on a modern desktop (Node.js, V8 JIT fully warmed):

| Duration | Processing Time | Realtime Factor | |----------|----------------|-----------------| | 0.5s | ~3ms | ~160x | | 1s | ~7ms | ~140x | | 5s | ~37ms | ~135x | | 10s | ~70ms | ~140x |

Browser Benchmarks (Chrome, WebGPU-capable desktop)

In-browser performance is slightly slower than Node.js due to V8's browser sandbox and JIT warm-up. First 2-3 calls are ~2x slower; steady-state numbers shown below:

| Audio Duration | Frames | meljs (JS) | nemo128.onnx (WASM+SIMD) | |----------------|--------|------------|--------------------------| | ~1s | 86–122 | 5–7 ms | 4–5 ms | | ~3s | 275–350 | 17–22 ms | 9–15 ms | | ~5s | 484–574 | 31–37 ms | 24–35 ms | | ~8s | 720–875 | 47–50 ms | 31–44 ms | | ~10s | 1028 | 63 ms | 42 ms | | ~15s | 1529 | 101 ms | 56 ms | | ~21s | 2139 | 144 ms |, |

Key observation: In single-file (non-streaming) browser usage, the ONNX preprocessor (nemo128.onnx) is ~1.5–2x faster for the preprocessing step itself, it runs on WASM with SIMD, which is highly optimized for tight numerical loops. However, preprocessing is only 5–15% of total transcription time; total end-to-end transcription time is virtually identical between both backends.

Where meljs wins

The performance advantage of meljs is not in raw single-pass speed, it's in architecture:

1. Incremental Caching for Streaming (~2x speedup)

In streaming ASR, you typically process overlapping windows (e.g. 5s window every 1s). With a stateless ONNX preprocessor, you recompute the full window every time. meljs caches prefix frames and only computes new audio:

| Mode | Time (5s window) | Speedup | |------|-----------------|---------| | Full recompute (or ONNX) | 71.7ms |, | | Incremental (70% cached) | 36.6ms | ~2x |

2. Background Worker Pipeline (zero-latency preprocessing)

The biggest impact comes from running meljs in a dedicated Web Worker as a continuous mel producer. Audio chunks are processed into mel frames as they arrive from the microphone, before inference even starts. When the encoder needs a feature window, it's already sitting in a buffer:

| Architecture | Preprocessing Latency in Inference Path | |-------------|----------------------------------------| | ONNX preprocessor (synchronous) | 30–140 ms (depends on window size) | | meljs in background worker | 0.0 ms (pre-computed, just a buffer read) |

This was measured in production with boncukjs, a real-time transcription app. The mel worker continuously ingests audio chunks (~0.5ms per 80ms chunk), and when the encoder needs a 5s feature window, it's retrieved from the buffer in ~1-3ms.

3. No Model Download (~5 MB saved)

meljs eliminates the need to download nemo128.onnx (~5 MB) from HuggingFace Hub and create an additional ONNX Runtime session, reducing initial load time.

Summary: When to use which

| Use Case | Recommendation | |----------|----------------| | Single-file transcription (batch) | Either works; ONNX is slightly faster per-call | | Streaming / real-time ASR | meljs, caching + background worker = zero-latency preprocessing | | Minimize download size | meljs, no ONNX preprocessor model needed | | Web Worker pipeline | meljs, pure JS, no WASM/ONNX constraints |

Pipeline Details

The mel spectrogram pipeline exactly matches NeMo's FilterbankFeatures:

1. Pre-emphasis, x[n] = x[n] - 0.97 * x[n-1] (Float32)
2. Zero-pad, N_FFT/2 = 256 samples each side
3. STFT, Cast to Float64, symmetric Hann window (400→512), 512-point FFT
4. Power spectrum, |real|² + |imag|² → Float32
5. Mel filterbank, Slaney-normalized triangular filterbank matmul
6. Log, log(mel + 2^-24)
7. Normalize, Per-feature mean/variance, Bessel-corrected (N-1 denominator)

NeMo's dither, narrowband augmentation, frame splicing, and pad-to-multiple are all disabled at inference and correctly omitted.

Testing

npm test          # Run all tests
npm run test:watch  # Watch mode

The test suite includes:

• Constants validation
• Mel scale roundtrip tests
• Filterbank shape and triangle continuity
• Hann window symmetry and padding
• FFT correctness (DC, sinusoid, Parseval's theorem, large sizes)
• Full pipeline correctness and determinism
• Incremental caching accuracy
• ONNX reference cross-validation (mel_reference.json)
• Performance benchmarks

Compatibility

meljs is used in production by:

• parakeet.js, Browser-based ASR with WebGPU
• boncukjs, Real-time transcription app

It replaces the ONNX preprocessor model (nemo128.onnx) in both projects, eliminating a ~5 MB model download and ONNX session overhead.

License

MIT