RhyDB WASM

This directory builds a browser-usable version of RhyDB using WebAssembly (WASM). It uses Emscripten to compile the C++ sources to a .wasm binary and generates a small JavaScript loader that lets browser code call selected C++ functions.

CMakeLists.txt collects RhyDB core sources, excludes the CLI, HTTP API, tests, and benchmark code, and links the dependencies needed for preprocessing and SaneQL query execution. src/rhydb_wasm.cpp is the boundary between JavaScript and C++.

What It Exposes

The generated JavaScript module exports these embind functions:

• preprocess(preprocessingConfigPath): reads a RhyDB preprocessing config from Emscripten's virtual filesystem and returns a database handle.
• save(handle, outputDirectory): writes the in-memory database as RhyDB's normal processed-state files into the virtual filesystem.
• load(stateDirectory): loads a saved RhyDB processed state from the virtual filesystem and returns a database handle.
• query(handle, saneqlQuery): runs SaneQL and returns the result as NDJSON.
• info(handle): returns database information as JSON.
• dispose(handle): releases a database handle.

The module also exposes Emscripten's FS runtime API. Browser code uses FS to write uploaded files into the virtual filesystem before calling preprocess or load, and to read files back after calling save.

The WASM build supports raw and .zst preprocessing inputs. The native .xz input path is not included in the WASM target.

Browser queries use a smaller materialization cutoff than native RhyDB to keep intermediate batches within browser memory limits, especially when projecting full nucleotide sequences.

Artifacts

Building the WASM target creates these files relative to the RhyDB repository root:

• wasm/dist/rhydb_wasm.js: ES module loader generated by Emscripten. Import this from browser code.
• wasm/dist/rhydb_wasm.d.ts: Autogenerated type declarations.
• wasm/dist/rhydb_wasm.wasm: compiled RhyDB C++ code and linked dependencies.

All files must be served by the web app. The JavaScript loader fetches the .wasm file at runtime.

Install from NPM

The WASM build is also published as an NPM package.

npm install @rhydb/rhydb-wasm

The package is an ES module. Import the loader as a default import:

import createRhydbModule from "@rhydb/rhydb-wasm";

const rhydb = await createRhydbModule();

The .wasm file ships alongside rhydb_wasm.js and must be resolvable by your bundler so the loader can fetch it at runtime. The package version matches the RhyDB release version.

The current build uses pthreads, so the page serving the module must be cross-origin isolated. Pthreads are the POSIX thread API that Emscripten maps to Web Workers, and RhyDB uses them because its dependencies and query execution are built around threaded native code. Serve it over HTTP with these headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

For a local example server, run this from the RhyDB repository root:

python3 - <<'PY'
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer

class Handler(SimpleHTTPRequestHandler):
    def end_headers(self):
        self.send_header("Cross-Origin-Opener-Policy", "same-origin")
        self.send_header("Cross-Origin-Embedder-Policy", "require-corp")
        super().end_headers()

ThreadingHTTPServer(("127.0.0.1", 8088), Handler).serve_forever()
PY

Build Reference

Prerequisites:

• Emscripten SDK activated so emcmake, emcc, and em++ are on PATH. Install it via emsdk:

  git clone https://github.com/emscripten-core/emsdk.git
  cd emsdk && ./emsdk install latest && ./emsdk activate latest
  source ./emsdk_env.sh

• CMake

• Ninja

• uv

• a native C++ compiler for Conan build tools

From the RhyDB repository root, build the WASM artifacts:

make wasm

The compatibility wrapper ./wasm/scripts/build-wasm.sh also calls make wasm. The Makefile writes the Emscripten Conan profile to build/wasm/conanprofile-emscripten, installs dependencies into build/wasm, configures CMake with emcmake, builds rhydb_wasm, and copies the artifacts to wasm/dist.

Tests

./test contains a Node.js smoke test (node:test) that loads the built module, preprocesses the testBaseData/unitTestDummyDataset fixture, runs a SaneQL query, and round-trips a saved state through save/load. Run it with:

make wasm-test

This builds rhydb_wasm first if necessary, then runs node --test wasm/test/*.test.mjs.

Example App

An example app is available in ./example. Any static web server can be used as long as it sets the headers. Opening the index.html directly as a local file is not enough for the pthread-enabled build.

Usage Reference

Import and initialize the module:

import createRhydbModule from "./rhydb_wasm.js";

const rhydb = await createRhydbModule();

This assumes rhydb_wasm.js and rhydb_wasm.wasm are served from the same URL directory. If a web app serves them from different locations, pass Emscripten's locateFile option to tell the loader where to fetch rhydb_wasm.wasm.

Write uploaded files into the virtual filesystem and preprocess:

function mkdirp(path) {
  let current = "";
  for (const part of path.split("/").filter(Boolean)) {
    current += `/${part}`;
    if (!rhydb.FS.analyzePath(current).exists) {
      rhydb.FS.mkdir(current);
    }
  }
}

mkdirp("/input");
rhydb.FS.writeFile("/input/preprocessing_config.yaml", preprocessingConfigBytes);
rhydb.FS.writeFile("/input/database_config.yaml", databaseConfigBytes);
rhydb.FS.writeFile("/input/reference_genomes.json", referenceGenomeBytes);
rhydb.FS.writeFile("/input/sequences.ndjson.zst", ndjsonBytes);

rhydb.FS.chdir("/input");
const handle = rhydb.preprocess("preprocessing_config.yaml");

Query an in-memory database:

const ndjson = rhydb.query(handle, "default.groupBy({count:=count()})");
console.log(ndjson);

Save a processed state and read the files back:

mkdirp("/state");
rhydb.save(handle, "/state");

for (const entry of rhydb.FS.readdir("/state")) {
  if (entry !== "." && entry !== "..") {
    console.log(entry);
  }
}

Load an existing processed state:

mkdirp("/loaded-state");
// Write the saved RhyDB state files under /loaded-state first.
const loadedHandle = rhydb.load("/loaded-state");
console.log(rhydb.info(loadedHandle));

Release memory when a database is no longer needed:

rhydb.dispose(handle);
rhydb.dispose(loadedHandle);

For a complete browser example that uploads input files, downloads a saved state, uploads that state again, and runs a query, see example/.