@birchill/nice-sqlite-wasm

This is a custom build of SQLite that only supports the "opfs-sahpool" VFS. It's "nice" because:

• It removes the "opfs" VFS and worker parts of the JS bindings making for a smaller bundle size.
• It allows overriding the locateFile function so that you can provide a custom path for the WASM module (e.g. in order to support cache-busting filenames) or even a Response object (e.g. so you can abort the download).
• It fixes some warnings that otherwise might occur at build or run-time (e.g. the COOP/COEP header warning which is only relevant to the "opfs" VFS and a warning about dependencies based on expressions).

In general, it should be nicer for apps using bundlers that only need the "opfs-sahpool" VFS.

As an extra bonus, the whole build (including building the WASM module) is done in CI and published to npm with provenance data so you get full traceability.

[!NOTE] The JS/WASM part of SQLite is under heavy development and is expected to change a lot in future (e.g. using WASI instead of Emscripten). As a result this project may no longer be necessary or may become impractical to update.

For the official SQLite WASM package see sqlite-wasm.

Usage

Install:

npm install @birchill/nice-sqlite-wasm

You could probably just do an import sqlite3InitModule from '@birchill/nice-sqlite-wasm' and be done with it but the whole point of this module is to allow you to set it up for better caching and pre-loading. How you do that will depend on your bundler.

Following is an example for rspack.

rspack

First, we set up a way to generated the name of the WASM module so we can pass it to sqlite3InitModule.

In your rspack.config.js:

export default defineConfig((env) => {
  // ...
  module: {
    rules: [
      { resourceQuery: /url$/, type: 'asset/resource' },
    ],
  },
});

Second, we set up rspack to produce the desired output filename for the asset, otherwise you'll get something like [hash].wasm which isn't very helpful.

Furthermore, we want to drop the query string we enabled above.

export default defineConfig((env) => {
  // ...
  output: {
    assetModuleFilename: (fileInfo) => {
      // Generate a cacheable but readable filename for WASM files
      if (
        fileInfo.filename.endsWith('.wasm') ||
        fileInfo.filename.endsWith('.wasm?url')
      ) {
        return '[name].[hash].wasm';
      }
      return '[hash][ext][query]';
    },
  },
});

Then in your worker code:

import wasmUrl from '@birchill/nice-sqlite-wasm/sqlite3.wasm?url';
import sqlite3InitModule from '@birchill/nice-sqlite-wasm';

Then when you initialize SQLite:

const sqlite = await sqlite3InitModule({
  // Override SQLite's locateFile implementation which wants to resolve
  // the SQLite WASM binary relative to the source directory instead of
  // the asset name assigned by rspack.
  locateFile: (file) => {
    if (file === 'sqlite3.wasm') {
      // Since we strip the query string in our `assetModuleFilename`
      // option in rspack.config.js we don't need to worry about dropping
      // it here.
      //
      // If we were to stop doing that, however, we'd need to do
      // something like:
      //
      //   return new URL(wasmUrl, self.location.href).pathname;
      //
      // instead.
      return fetch(wasmUrl, {
        credentials: 'same-origin',
        // If you want to make the fetch abortable...
        signal: abortController.signal,
      });
    } else {
      throw new Error(`Unknown file: ${file}`);
    }
  },
});

You can also just return wasmUrl from locateFile if don't need to control the fetch yourself.

vite

I'm not sure how to configure vite, but if you're only using it for testing (i.e. using vitest) then you can just disable optimization there as explained in the official WASM module docs:

// vitest.config.js
import { defineConfig } from 'vite';

export default defineConfig({
  optimizeDeps: { exclude: ['@birchill/nice-sqlite-wasm'] },
});

Developing

Installing prerequisites

pnpm install

This will install some tools we use like oxfmt.

Fetching the SQLite source code

Ensure sqlite-revision.txt contains the desired SQLite version (e.g. version-3.51.1 or any tag from the SQLite git repository).

Then run:

pnpm run fetch

Applying patches

pnpm run patch

Creating new patches

First fetch the source then apply any existing patches.

Then go to work/sqlite-src, make edits and run git diff > ../../patches/patch-name.patch.

Something like:

pnpm run fetch
pnpm run patch
cd work/sqlite-src
git commit -am "Existing patches"
vim ext/wasm/api/pre-js.c-pp.js # For example
git diff > ../../patches/000x-patch-description.patch
git stash
cd ../..
git add patches/000x-patch-description.patch

Then update the table below.

Current patches

| Patch name | Purpose | | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | 0001-locatefile-nullish-coalesce.patch | Allow a user-provided locateFile function to be used (rather than clobbered). | | 0002-hardcode-locatefile-path.patch | Hardcodes the path used in the default locateFile implementation so that bundlers don't complain about dependencies based on expressions. | | 0003-locatefile-with-response.patch | Allows a user-provided locateFile function to return a Response or a Promise<Response>. |

Building the WASM module

Getting Emscripten

First you need to install Emscripten.

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh # Or source ./emsdk_env.fish etc.

Note that you need to run the emsdk_env.* script every time so you might want to add something to your shell's startup script.

e.g. for fish

if test -e ~/emsdk/emsdk_env.fish
    EMSDK_QUIET=1 source ~/emsdk/emsdk_env.fish
end

Getting wasm-strip (from wabt)

Then you need to install wabt for to make wasm-strip available.

Critically, this must be version 1.0.36 or higher or else wasm-strip will fail on the 64-bit build with "tables may not be 64-bit". We don't actually need the 64-bit build but there's no way to turn it off easily.

As of this writing, using sudo apt install wabt gives you version 1.0.34 which is not high enough.

Instead you probably want to download the latest release from https://github.com/WebAssembly/wabt/releases:

wget https://github.com/WebAssembly/wabt/releases/download/1.0.39/wabt-1.0.39-linux-x64.tar.gz
tar -xzf wabt-1.0.39-linux-x64.tar.gz
sudo mv ~/wabt-1.0.39 /usr/local/wabt-1.0.39
sudo ln -s /usr/local/wabt-1.0.39 /usr/local/wabt
sudo ln -s /usr/local/wabt/bin/* /usr/local/bin/

(or brew install wabt on macOS)

Actually building

Then run:

pnpm build

Making changes

After making changes, to update the changelog run:

pnpm changeset

Don't forget to add the changeset file created in /.changeset to the commit.