@mizchi/jsimd

v0.3.0

Published

25 days ago

Small WebAssembly SIMD kernels, resident data structures, and dynamic Float32 fusion for JavaScript

0High
0Medium
0Low

mizchi

simd typescript wasm webassembly

@mizchi/jsimd

Small WebAssembly SIMD kernels, Wasm-resident data structures, and an opt-in dynamic Float32 fusion compiler for data-parallel JavaScript hot paths.

Goal

jsimd provides small, tree-shakeable data structures and bulk operations powered by hand-written WebAssembly SIMD kernels. It targets JavaScript hot paths where explicit SIMD can outperform the equivalent built-in implementation end to end.

Each feature ships as an independent package subpath with documented performance, boundary-cost, and bundle-size trade-offs.

Install

pnpm add @mizchi/jsimd

Runtime requirements

Node.js 24.5 or later
Deno 2.6 or later
Vite 8 or later
WebAssembly SIMD
WebAssembly ESM Integration for direct .wasm imports

The optional shared-buffer entrypoint additionally requires WebAssembly threads and shared memory. The ultra-log-log-parallel entrypoint requires SharedArrayBuffer and Workers but performs no atomic Wasm access. Browsers normally expose shared buffers through cross-origin isolation. Node uses node:worker_threads; Deno and browsers use Web Workers. Their factories are asynchronous because each Worker must initialize its module.

All supported environments use the same entrypoints. Existing single-threaded entrypoints expose synchronous operations after ESM Integration instantiates their Wasm modules. shared-buffer is the exception: its asynchronous factories compile the same kernel in each Worker against caller-owned shared memory. Consumers must also enable explicit resource management (using / Symbol.dispose).

f32-fusion is another explicit exception: it ships no static Wasm asset and asynchronously emits and compiles a restricted SIMD expression at runtime. It requires CSP permission for WebAssembly.compile(); use supportsF32Fusion() when deployment policy is unknown.

Usage

import { indexOf } from "@mizchi/jsimd/bytes";
import { WaveletMatrixUint32 } from "@mizchi/jsimd/wavelet-matrix-uint32";

const bytes = new TextEncoder().encode("simd:simd");
indexOf(bytes, 0x3a); // 4
indexOf(bytes, new TextEncoder().encode("simd")); // 0

// Wasm-resident data structures own memory, so release them with `using`.
using values = WaveletMatrixUint32.from([3, 1, 4, 1, 5, 9]);
values.rank(1, values.length); // 2
values.rangeFreq(0, values.length, 1, 5); // 4
values.quantile(0, values.length, 3); // 4

The single-threaded entrypoints use direct Wasm ES module imports supported by current Vite and Deno. Module loading performs initialization, after which their exported operations are synchronous.

Implementation guide

Each link below opens the feature README with its complete API, algorithm and literature sources, benchmark setup, and exact results. This table is only a rounded selection guide.

Owning structures keep data in Wasm linear memory and should be declared with using. They are a good fit when several bulk operations reuse resident data. Native arrays and collections usually remain the better choice for small inputs, one-shot work, point access, or workloads that repeatedly materialize complete results back into JavaScript.

Disposal returns allocator blocks to an internal reuse pool; it does not shrink WebAssembly.Memory. A stable reservedBytes or memoryBytes value after a workload is therefore expected. Stateful entrypoints expose allocatorStats() so tests can require liveAllocations and liveBytes to return to their baseline after each using scope.

Data structures

The canonical bitmap subpaths are bitmap, rank-select-bit-vector, and roaring-bitmap. bit-vector is intentionally reserved for a future immutable packed-bit sequence without a rank/select index; it is not currently exported. Pre-announcement compatibility aliases were removed so each structure has one public name.

Stateless and copy-inclusive kernels

Dynamic kernels

How to read the numbers

Performance results were recorded on Apple M5 with Node 24 or Deno 2.6 as documented by each linked feature README. They are workload samples, not cross-feature scores. Resident benchmarks usually exclude construction and final materialization; copy-inclusive kernels explicitly include boundary copies. Rerun the linked benchmark on the target engine and data distribution before choosing a representation.

Admission is based on the documented primary workload, not on every convenience method. A row that reports a slower JS case is retained only when a separate, representative bulk workload wins; that slower operation is outside the performance contract.

Build sizes come from isolated Vite 8.2 production fixtures. Each cell reports the gzip sizes of the minified JavaScript output and its independently emitted, stripped Wasm asset. f32-fusion reports zero Wasm because it generates the module at runtime. Importing a subpath does not pull in another feature's Wasm. A real application may share wrapper code or add other runtime code, so these figures are marginal fixtures rather than a prediction of total bundle size. Raw sizes remain available in each feature README.

The package is distributed as one npm package with subpath exports. npm releases contain compiled JavaScript and adjacent declarations; consumers do not need TypeScript runtime transformation for files under node_modules. Each .wasm import is typed by an adjacent kernels.d.wasm.ts, following Vite's allowArbitraryExtensions convention, without a generated environment-specific loader.

Examples

multithread-vector-search shards an exact Float32 index across Web Workers, uses shared SPSC task notification and result slots, and merges only workerCount × k candidates on the coordinator.

Storage package

@mizchi/jsimd-columnar builds typed schemas, ZoneMap/projection pushdown, resident SIMD page caching, and interchangeable Memory, IndexedDB, and Node filesystem backends. It is versioned separately as an experimental 0.x package: warm queries win the recorded selective workload, while cold storage restore remains slower than already-resident page-aware JavaScript.

Development

Implementation priorities, benchmark gates, and deferred candidates are maintained in TODO.md. A kernel is retained only when a documented workload justifies its Wasm boundary and bundle cost against the best relevant JavaScript builtin or scalar reference.

pnpm install
just build
just test
just bench
just memory-profile

just build compiles 30 Zig 0.16 src/<name>/kernels.zig sources and the deliberately hand-written bytes/kernels.wat source with SIMD128 and wasm-opt -Oz. Generated Wasm files are checked for their public exports, expected SIMD instructions, and per-module size limits, then validated with wasm-tools. just build-package emits the npm payload into packages/jsimd/dist/: compiled JavaScript, declarations, feature documentation, kernel sources, shared Zig source modules, and the corresponding optimized Wasm binaries. Development requires Zig 0.16, wasm-opt, and wasm-tools on PATH.

just memory-profile runs every owning data structure in an isolated Node process with explicit GC. It fails if live Wasm allocations do not return to baseline, allocator capacity keeps growing after warmup, or post-GC heap/external/ArrayBuffer memory does not plateau over the final rounds. Peak RSS is reported as an informational high-water mark because V8/Wasm tier-up and native allocators can retain code or arena pages after owned storage has been released.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

@mizchi/jsimd

Goal

Install

Runtime requirements

Usage

Implementation guide

Data structures

Stateless and copy-inclusive kernels

Dynamic kernels

How to read the numbers

Examples

Storage package

Development