zstd-wasm-decoder

Tiny & performant decoder-only implementation of Zstandard.

| | | |----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Lightweight | 13.19kb / 17.17kb (zipped) for size or perf. optimized build | | Dictionary Support | Multiple and up to 2MB each | | Performant | ~1.6x throughput vs Node.js zlib (V8), ~0.96x vs Bun (JSC) | | Compatibility | • DecompressionStream API ponyfill• >94% worldwide browsers• Node 20-24, Cloudflare Workers, Vite, Bun• Can be loaded as pre-compressed inline base64 or as separate .wasm for CSP compliance | | Tested | Validated against vectors from the zstd reference implementation. | | Zero deps | No runtime dependencies (excluding build); compiled from source using latest clang & binaryen |

Implementation notes:

• Given the limitations of wasm memory management and to achieve appropriate code size & performance, memory is allocated to a fixed-size ring buffer, avoiding heap growth entirely. The buffer is sufficiently sized to handle the maximum memory required by level 19 compressed data.
• For use in browsers, the module is asynchronously compiled & cached at page load.

Usage - (Client Side)

import { decompress, ZstdDecompressionStream, decompressStream, createDecoder } 
from 'zstd-wasm-decoder'; // Default (Node/browser - automatically inferred)

import { ... } // For strict CSP policies (no unsafe-eval for WASM)
from 'zstd-wasm-decoder/external'; // .wasm fetched from same-origin

import { ... } // If you need the extra perf. (+30%) for +4kb in the browser
from 'zstd-wasm-decoder/perf' // or perf/external
                              // non-browser env uses perf. by default

import { ... }                
from 'zstd-wasm-decoder/cloudflare'; // for cloudflare workers

// 1. Simple decompression (with optional dictionary)
const data: Uint8Array = await decompress(compressedData, { 
  dictionary: await (await fetch('/dict.bin')).arrayBuffer()
});

Note: In development mode, the inlined version is served for /external to avoid bundler issues (e.g., in Vite).

// 2. Streaming API - fetch response
const stream: ReadableStream<string> = (await fetch('/file.zst')).body!
  .pipeThrough(new ZstdDecompressionStream())
  .pipeThrough(new TextDecoderStream());

// 3. Streaming API - with dictionary
const ds = new ZstdDecompressionStream({ 
  dictionary: await (await fetch('/dict.bin')).arrayBuffer()
});

// Alternatively
import { setupZstdDecoder } from 'zstd-wasm-decoder';
await setupZstdDecoder({ 
  dictionaries: ['/dict.bin'] // Accepts URLs
});
const ds = new ZstdDecompressionStream(); // Auto-detects dict from frame header

const ds: ReadableStream<Uint8Array> = blob.stream().pipeThrough(ds);

// 4. Manual streaming (for chunked data)
const { buf, in_offset }: { buf: Uint8Array, in_offset: number } = await decompressStream(chunk, reset);

// 5. Reusable decoder instance
const decoder = await createDecoder();
const result1: Uint8Array = decoder.decompressSync(data1);
const result2: Uint8Array = decoder.decompressSync(data2);

Important Considerations

• The default export is pre-minified and mangled. All builds tested against the full suite.
• Legacy ZSTD format is not supported, and the presence of magic bytes is expected; some libraries have this disabled by default.
• Consult the reference to interpret error codes, should any occur.
• Do not use the wasm module standalone (without js).
• Do not send any (compressed) sensitive data over continous, long-running streams. Side-channel attacks  |  CRIME  |  BREACH  |  Lucky Thirteen

Contributing

Prerequisites

macOS:

brew install llvm binaryen pnpm zopfli

Linux:

sudo apt-get install clang lld binaryen zopfli

Note: macOS's default /usr/bin/clang is a symlink to Apple Clang 17, which lacks the linker required for WebAssembly builds. You must install the full LLVM toolchain from Homebrew, build from source, or download the binaries (as done by the CI runner)

Setup

1. Clone and install dependencies:

git clone --recursive https://github.com/tadpole-labs/zstd-codec-lib.git
cd zstd-codec-lib
pnpm install

2. Configure LLVM path (if not auto-detected):

# macOS with Homebrew:
export LLVM_DIR=/opt/homebrew/opt/llvm

# Linux:
export LLVM_DIR=/usr

3. Verify toolchain:

cd packages/zstd-wasm-decoder
make check-tools

Development Workflow

# Full build (WASM + TypeScript)
pnpm run build:all

# Clean build
pnpm run clean:decoder
pnpm run build:all

# Run tests
pnpm test                    # All runtimes (Node + browsers + Bun)
pnpm run test:node           # Node.js only
pnpm run test:browsers       # Browser tests only
pnpm run test:bun            # Bun only

# Run benchmarks
pnpm run bench:full

License

This package is dual-licensed under Apache-2.0 OR MIT

The underlying zstd implementation is licensed under BSD-3-Clause.