wopfs

File-backed I/O for the wasmedia Emscripten ports. A tool run with callMain reads and writes OPFS files (in a dedicated worker), host files (under Node) or MEMFS (anywhere else) through one mount call. The input and output files are not staged in memory. There are no threads, no SharedArrayBuffer and no COOP/COEP headers.

The design brief is briefs/FILE_BACKED_IO.md. The implementation plan is docs/superpowers/plans/2026-10-03-wopfs-file-backed-io.md.

Install

npm install @wasmedia/wopfs

wopfs is published to npmjs.org with provenance, like the rest of the wasmedia family, so no .npmrc is needed. Versions 0.1.0–0.1.2 were published to this project's GitLab npm registry only; drop any @wasmedia:registry=https://gitlab.com/api/v4/packages/npm/ line when upgrading.

To release: bump version in package.json, update CHANGELOG.md, merge, then push a tag v<version> (Maintainers only: v* tags are protected). The tag pipeline runs every test job and then publish, which refuses a tag that doesn't match package.json.

Bundling

No bundler config is needed, for the browser or for Node. wopfs contains no static node: import anywhere: its Node backend is reached by a dynamic import, and it loads node:fs/node:path itself at runtime through an import() that bundlers are told to ignore. A browser never runs that code; under Node, bundled or not, the built-ins always resolve.

tests/bundle.test.ts checks this with real builds against dist/:

• browser builds with Vite, webpack 5 (no externals), and esbuild with and without --splitting (no --external);
• an esbuild Node bundle that runs a real mount and commit.

(0.2.0 and earlier imported node:fs statically, which needed --splitting --external:node:* with esbuild and externals: [/^node:/] with webpack. Those settings are now unnecessary but harmless.)

Porting a wasmedia tool to wopfs? Read docs/adopting.md first.

Quick start

import { mountFiles } from "@wasmedia/wopfs";

// In a dedicated worker. `module` is an Emscripten module built with -sEXPORTED_RUNTIME_METHODS=FS,callMain.
const mount = await mountFiles(module, "/work", {
  inputs: { "a.tif": fileHandleA, "b.tif": fileHandleB }, // read-only, opened up front
  outputDir: dirHandle,                                    // where committed outputs land
  scratch: 16,                                             // files the tool may create
  onExhausted: "spill",                                    // or "strict"
});
try {
  const code = module.callMain(["-o", "/work/out.tif", "/work/a.tif", "/work/b.tif"]);
  if (code === 0) {
    const written = await mount.commit(); // [{ path: "out.tif", size, handle }]
  }
} finally {
  await mount.release(); // closes every sync handle, unmounts, removes scratch files
}

A tool running under callMain is synchronous and cannot await, but opening an OPFS file is async. So mountFiles opens everything the run can touch before it starts: every input, plus a pool of scratch files for the files the tool creates. commit() moves the scratch files the tool used to their final names.

Backends

backend: "auto" (the default) picks one from the environment. You can also pass a backend name to force it.

| Environment | Backend | Inputs read in place | Also accepted | commit() returns | |---|---|---|---|---| | Dedicated worker with OPFS | opfs | FileSystemFileHandle | Uint8Array, Blob | { path, size, handle } | | Node | node | host path string | Uint8Array, Blob | { path, size, hostPath } | | Anything else (main thread) | memfs | none (all copied in) | Uint8Array, Blob, FileSystemFileHandle | { path, size, bytes } |

• opfs and node share the same filesystem code (HandleFS) and differ only in their backing files.
• memfs is Emscripten's own MEMFS, staged the way the ports stage files today. The conformance suite uses it as the oracle.
• If a browser's sync-handle methods turn out to be asynchronous (old Safari), backend: "opfs" throws WopfsUnsupportedError. "auto" warns and falls back to memfs instead.
• The Node backend is wopfs's own rather than Emscripten's NODEFS. None of the ports links -lnodefs.js, and adding it would mean relinking them.

Options

| Option | Default | Meaning | |---|---|---| | inputs | {} | Read-only inputs, keyed by path relative to the mountpoint ("a.tif", "sub/b.tif"). Parent directories are created. | | outputDir | none | opfs: a FileSystemDirectoryHandle. node: a host directory path. Ignored by memfs. Without it, opfs and node mount input-only: no scratch directory, nothing written anywhere, creating a file fails with EROFS, and commit() resolves to []. | | commitAs | none | Commit some outputs under another name: { "path/in/mount": "path/under/outputDir" }. For a tool that rewrites an input in place when the result belongs elsewhere: { "shot.NEF": "shot.tif" } commits the rewritten NEF as shot.tif and leaves the original. Keys the tool never created are ignored; two outputs landing on one name fail the commit before anything moves. Needs outputDir on opfs/node. | | scratch | 16 (0 without outputDir) | How many files the tool may create (opfs, node). Unlinked files return their slot. Setting it without outputDir is a WopfsInputError. | | onExhausted | "spill" | When the pool runs out. "spill": hold further new files in JS memory until commit(), with one warning. "strict": the tool's open fails with ENOSPC. | | backend | "auto" | "opfs", "node", "memfs" or "auto". | | onWarning | console.warn | Receives spill and fallback warnings. |

Semantics

• Inputs are read-only. They are never written, truncated, moved or deleted on disk. A write-open fails with EACCES, even if the tool chmods the file first. A tool that renames or unlinks an input changes only the mount's view of it.
• commit() replays final state. Every file the tool created that is still reachable when commit() runs lands in outputDir at its final relative path. rename and unlink are reflected because they changed the tree. Some consequences:
  • exiftool's in-place edit (write a temp file, rename the original to _original, rename the temp file into place) commits the new a.jpg and leaves the input untouched.
  • An output that already exists in outputDir is replaced.
  • Empty directories the tool created are not committed.
• commit() runs once. A second call throws WopfsStateError. If it fails partway it throws WopfsCommitError, whose .committed lists the files already in outputDir. A destination is only ever replaced by a complete file: spilled files are copied into the scratch directory first, then moved into place.
• An output may replace an input that lives in outputDir. Inputs are unlocked when commit() starts.
• After commit(), the run is over. Backing files are closed, so further access through the mount fails with EBADF (on every backend that holds host or OPFS files; memfs keeps working). Call release() next.
• Output names must be distinct on the host. On a case- or normalization-insensitive disk (macOS, Windows), two outputs that map to one host file (Out.txt, out.txt) fail the node commit with a WopfsError rather than one silently replacing the other.
• Input paths may not conflict. "a" and "a/b" together throw WopfsInputError.
• release() belongs in finally. It closes streams the tool left open and every sync handle (so the files unlock), unmounts, and removes the scratch directory. Without commit(), nothing is added to outputDir. It is safe to call more than once: calls made while one is in flight share it, and after any failure the next call retries.
  • If the module's cwd is inside the mount (a tool resolving relative paths), it is first moved to the nearest directory that survives cleanup: the mountpoint itself if it existed before mountFiles, otherwise the parent of the directories wopfs created. A cwd outside the mount is left alone, so a module that chdirs into a pre-existing /work once keeps that cwd across mounts.
  • It only unmounts its own mount. If that is already gone (an enclosing mount was released first) or another mount now sits at the same path, it leaves the path alone.
  • Parent directories wopfs created for the mountpoint are removed when empty and left otherwise (a sibling mount may share them).
  • If unmounting fails, it throws WopfsStateError after unlocking every file.
• During a run, the scratch directory .wopfs-scratch-<id> exists inside outputDir.
• Unlinked scratch files are recycled once their last stream closes, so temp-file churn does not exhaust the pool.
• Blob inputs in a worker are read on demand with FileReaderSync, as Emscripten's WORKERFS does, rather than whole into memory at mount. Elsewhere (Node, the main thread) a Blob is still read whole.
• Flushing: created files are flushed once, when committed (and on an explicit fsync), not on every close().
• Buffered stdio: data a tool leaves in an unflushed FILE* (no fclose, EXIT_RUNTIME=0) never reaches the filesystem. The same is true under MEMFS.
• Not a general VFS: no symlinks, and no guarantees for concurrent runs over the same files.

mmap

mmap is served by copying the mapped range into a heap block, which libc's munmap frees with emscripten_builtin_free. This costs a heap copy of the mapped range. With MAP_SHARED + PROT_WRITE, the data is written back on msync and munmap. A failed allocation is ENOMEM, as under MEMFS.

It needs the module to export HEAPU8 and an allocator. Like MEMFS, wopfs prefers _emscripten_builtin_memalign (64 KiB-aligned, the allocator munmap pairs with), and falls back to _malloc, which is the same allocator unless a port overrides malloc. Without them, mmap fails with ENODEV, and tools that fall back to read (libtiff does) still work. The conformance suite runs both variants and requires identical output. Exports are read by property descriptor, so an -sASSERTIONS build's abort-on-read stubs for unexported symbols are never triggered.

Memory: what wopfs saves

The brief assumed that MEMFS files live in the wasm heap. They don't. Emscripten's MEMFS keeps file contents in JS Uint8Arrays outside wasm linear memory, so staged files never counted against the 4 GiB wasm32 ceiling. They count against the tab's or process's memory, which is still the binding limit in browsers (and tight on iOS).

wopfs removes those staging copies:

• the encoded inputs;
• intermediate and final outputs;
• the caller's copies of each.

Two tests pin this down:

• tests/memory.test.ts (Node): reading a 256 MiB file through the node backend grows arrayBuffers by under 16 MiB, versus more than 256 MiB under MEMFS.
• The Playwright memory test: reading 256 MiB through OPFS grows the wasm heap by under 8 MiB.

What wopfs does not change is what a tool decodes into its own buffers. Enblend still needs about 18 B per input pixel. Tools that slurp whole files still slurp them.

Requirements on the module

• -sEXPORTED_RUNTIME_METHODS=FS (plus callMain to run it). No C/C++ changes, no relink beyond that.
• Optionally HEAPU8 and _malloc (-sEXPORTED_FUNCTIONS=...,_malloc), so mmap works.
• Built without --closure, which would rename the FS internals wopfs uses.

wopfs depends on Emscripten's legacy FS internals (createNode, node_ops/stream_ops, and so on), which are not a documented public API. mountFiles checks the module's FS first and throws WopfsRuntimeError naming anything that is missing.

The family's pinned toolchains are emsdk 5.0.5 (wexiftool), 5.0.6 (most ports) and 6.0.6 (wimagemagick, whose prebuilt ImageMagick dependency bundle is built with it). Every test runs on all three. To add a toolchain:

1. Add its version to tests/shared/fixtures.mjs, SUPPORTED_EMSDK in src/runtime.ts, native/Makefile and the fixtures job's EMSDK matrix in .gitlab-ci.yml. tests/emsdk-list.test.ts checks they agree.
2. Run npm run build:fixtures, npm test and npm run test:browser.

tests/errno.test.ts also fails if a toolchain renumbers the errno or flag values.

Browser support

Tested with Playwright 1.63: Chromium 153, Firefox 155, WebKit 26.6. The floor is: FileSystemSyncAccessHandle methods are synchronous, and FileSystemFileHandle.move() exists (commit() uses it to replace outputs safely). Both are detected at mount time.

Engine notes:

• WebKit / Safari has no OPFS in ephemeral sessions (Private Browsing; Playwright's default context): navigator.storage.getDirectory() rejects. In those sessions the caller has no handles to pass, and the main-thread memfs path still works.
• WebKit reports a file held by another sync handle as InvalidStateError; wopfs maps it to WopfsLockedError.
• File-picker and drag-and-drop handles are not OPFS files, so they can't get sync access handles. wopfs reports them as WopfsUnsupportedError (not "locked"), and backend: "auto" falls back to memfs, which reads them with getFile(). Copy such files into OPFS first to get file-backed I/O for them.
• Locks: a sync access handle locks its file exclusively. If another tab or worker holds an input, mountFiles rejects with WopfsLockedError (.path names the input) and releases everything it had opened. readwrite-unsafe mode is not used.

Brief open questions, answered

1. FS internals stability: pinned to emsdk 5.0.5, 5.0.6 and 6.0.6. Guarded by checkRuntime() and by running the whole suite on each.
2. JSPI: the API is promise-based at mount, commit and release, and exposes the pool only as a size hint. A JSPI backend could drop the pool without an API change.
3. Safari: supported where sync-handle methods are synchronous (detected at mount). Tested on WebKit 26.6.
4. Cross-tab locks: surfaced as WopfsLockedError. No readwrite-unsafe in v1.
5. License: MIT.

Development

npm install
npm run build:fixtures   # builds native/fsprobe.c with each pinned emsdk (needs Docker)
npm test                 # vitest: unit, fake-OPFS, Node, MEMFS, cross-backend conformance, memory
npm run test:browser     # builds dist/, then Playwright in Chromium, Firefox and WebKit
npm run typecheck

CI (.gitlab-ci.yml) builds the fixtures in one emscripten/emsdk:<version> job per pinned toolchain, using make -C native all LOCAL_EMCC=1 EMSDK_VERSIONS=<version>. It then runs typecheck + vitest on node:22 and the Playwright suite on the Playwright image matching @playwright/test.

native/fsprobe.c is a small C program with one command per file-operation pattern:

• copy;
• seek, sparse write, truncate, append and O_TRUNC;
• exiftool-style renames, rmdir and unlink;
• creating many files;
• mmap, read and write;
• unlink-while-open churn;
• a checksumming read;
• writing to an input.

The conformance suites run each command on every backend and compare exit codes, stdout and output bytes against MEMFS.

License

MIT. See LICENSE. It can be linked from both the GPL ports (whugin, wenblend) and the Apache/MIT ones (wvips, wexiftool).