@swiftbrowser/wasm-compiler

The Swift compiler and linker built as WebAssembly, and what runs them: Milestone 1 of the in-browser compiler plan. Not published yet.

• src/driver.ts plans the jobs swiftc would run for wasm32-unknown-wasip1: one swift-frontend job per source, then wasm-ld. Pure TypeScript, so the browser can use it.
• src/wasm.ts reads the autolink entries out of the objects, in place of swift-autolink-extract.
• src/compile.ts runs those jobs on a host. src/node has two hosts: the native toolchain, and the WebAssembly tools under Node's WASI.
• src/cli.ts is swiftbrowser-wasmc.
• toolchain/ downloads the inputs and builds the WebAssembly tools.

The CLI

npm run build --workspace packages/wasm-compiler
node packages/wasm-compiler/bin/swiftbrowser-wasmc.js \
  --sdk <swift-sdk>/wasm32-unknown-wasip1 --tools $WASMC_WORK/build/tools \
  --module-cache ~/.cache/swiftbrowser-wasmc \
  --module-name Hello -o hello.wasm --run main.swift

toolchain/install-tools.sh puts swift-frontend.wasm and wasm-ld.wasm in $WASMC_WORK/build/tools. The first compile builds Clang's modules for SwiftShims and wasi-libc into the module cache; later ones reuse them.

--native-toolchain <dir> runs the same jobs with a native Swift toolchain instead, which is how the driver is checked on a machine with Swift.

Where it stands

Milestone 1 works: the CLI compiles Sources/HelloWasm/main.swift with swift-frontend.wasm, links it with wasm-ld.wasm, both under Node's WASI, and the result prints its line under Node and wasmtime. Multi-file modules and diagnostics (shown with host paths) work too.

| | Time | |---|---| | swift-frontend.wasm -c, HelloWasm, empty module cache | 1.55 s | | the same, module cache warm | 1.35 s | | wasm-ld.wasm | 0.61 s | | the whole CLI run | 2.0–2.3 s |

The same host, native tools: 0.07 s and 0.19 s (below). Most of the WebAssembly time is V8 compiling the modules lazily as functions are first called; wasmtime compiles the whole frontend up front, about 45 s per run.

What came out of the check against native (steps 1.3 and 1.4): the frontend's object, linked by the native wasm-ld, runs; its bytes match the native frontend's but for 16 bytes the native one randomizes too. wasm-ld.wasm and the native wasm-ld write byte-identical executables.

The tools' sizes (brotli at quality 11 with a 16 MB window):

| | Raw | gzip | brotli | |---|---|---|---| | swift-frontend.wasm, assertions, not stripped | 168.3 MB | 40.7 MB | 25.0 MB | | swift-frontend.wasm, no assertions, stripped | 118.8 MB | 35.0 MB | 21.5 MB | | wasm-ld.wasm, not stripped | 54.4 MB | 13.8 MB | 8.7 MB | | wasm-ld.wasm, stripped | 39.9 MB | 12.4 MB | 7.8 MB |

Stripping (STRIP=ON toolchain/install-tools.sh) does most of it: the name section is 43 MB of the frontend. Turning off assertions saves 7 MB raw. wasm-opt -Os on the stripped linker saves 8 MB raw but almost nothing compressed (7.8 → 7.7 MB brotli), so it is not used. The no-assertions frontend compiles hello world in 1.15–1.3 s under Node.

The in-memory host (memoryHost in src/memory-host.ts, over @bjorn3/browser_wasi_shim, which is what the browser will use) runs the same compile, link and program in about 1.3 s, building Clang's modules in memory as it goes. toolchain/record-sdk-files.mjs drives it and logs which SDK files the tools open: for hello world, 53 of the SDK's 3324 files, 46.6 MB, listed in sdk-manifests/hello.json. Most of that is Swift.swiftmodule (17 MB), libswiftCore.a (12 MB) and Swift.swiftdoc (4.5 MB, opened alongside the module; whether a compile can do without it is for Milestone 3's download work).

Things a 32-bit host ran into, all fixed in toolchain/patches:

• Types that assume pointers have three spare low bits (PointerIntPair with 3 bits, custom PointerLikeTypeTraits) get explicit alignas or a plain struct on 32-bit.
• BridgedSwiftObject's refcount field is 32 bits on a 32-bit host, as the Swift object header is.
• Clang's ZeroFilledMemoryBuffer uses calloc instead of mmap, and IRGen frees the SIL module on the same thread, since there are none.

Node 22's V8 crashes in the garbage collector with fast API calls when a module this size runs under WASI; src/node/wasi-host.ts turns them off (--no-turbo-fast-api-calls).

Apps (Milestone 2)

SWIFTBROWSER_COMPILER=wasm (or auto; packages/cli/src/compiler.ts) makes the CLI (packages/cli) build an app with these tools instead of swift build, for build and dev alike, and type-check it for check (packages/cli/src/wasmCompiler.ts), with no Swift install: the CLI lays out the Swift SDK for WebAssembly itself, from its npm packages (packages/cli/src/sdkPackages.ts), and uses the released kit, whose macros come built as WebAssembly. It reproduces SwiftPM's release build of the staged package: one whole-module -O frontend job against the kit's modules, Foundation's trimmed ICU data as an object written in TypeScript (dataObject in src/object.ts, in place of clang assembling .incbin), and a wasm-ld link with the app's exports and the kit's archives, linked whole. Package dependencies, copied for the shim or plain, are compiled module by module first (packages/cli/src/wasmPackages.ts), only the targets the app's products need. Without SwiftPM their manifests run on these tools: each is compiled against PackageDescription built for WebAssembly (toolchain/build-manifest-api.sh, shipped as manifest-api/ in @swiftbrowser/wasm-tools), linked with Foundation, and run under WASI the way SwiftPM runs it (packages/cli/src/manifest.ts). C targets compile with clang built as WebAssembly (clang.wasm, from the same LLVM build as wasm-ld; compileC in src/compile.ts). Macro targets in packages are linked as WebAssembly executables, like the kit's macros, and handed to the host as they are built (WasiHost.addPlugin).

Every example, built this way and run headless, gives the same op stream as its native build (scripts/snapshot-test.sh's check). With tools release swift-6.4.0-r3:

| Example | Macros | Frontend (-O, whole module) | wasm-ld | Snapshot | |---|---|---|---|---| | Counter, 3 files | | 3.7 s, 130 MB | 2.8 s, 409 MB | 111 ops, same | | Filters | | 5.4 s, 137 MB | 2.8 s, 409 MB | 145 ops, same | | Gallery | | 9.0 s, 144 MB | 3.0 s, 410 MB | 360 ops, same | | PhotoPicker | | 4.8 s, 137 MB | 2.8 s, 413 MB | 61 ops, same | | FoodTruck, 57 files | | 36.8 s, 227 MB | 2.7 s, 415 MB | 4064 ops, same | | Settings | @Observable | 7.2 s, 143 MB | 2.8 s, 410 MB | 154 ops, same | | Todos | @Model, @Query | 7.3 s, 142 MB | 2.7 s, 408 MB | 198 ops, same | | Journal | SwiftData | 8.0 s, 145 MB | 2.7 s, 409 MB | 85 ops, same | | Bookshelf | SwiftData, #Predicate | 9.0 s, 149 MB | 2.8 s, 415 MB | 341 ops, same | | CupcakeCorner | @Observable | 6.9 s, 144 MB | 2.6 s, 409 MB | 104 ops, same | | HackingWithSwift, 16 files | SwiftData, @Observable | 15.3 s, 172 MB | 2.7 s, 410 MB | 1182 ops, same | | Fidelity | @Observable, @Entry | 17.9 s, 168 MB | 2.7 s, 412 MB | 635 ops, same | | HotProspects + CodeScanner | SwiftData | 7.8 s + 4.2 s, 149 MB | 3.3 s, 424 MB | 167 ops, same |

Time is wall time under Node on the 4-core container (module cache warm); memory is each tool's linear memory when it exited, which is its peak. The native frontend compiles Counter in 1.1 s (below), so the WebAssembly one runs at about a third of native speed. Linking the kit's archives whole costs about 3 s and 410 MB whatever the app, which is cheap enough that static linking stays (Milestone 4's dynamic linking is not needed for time). In the dev server, Counter builds in 6.3 s and rebuilds on save.

Macros

A native compiler runs a macro plugin as a process. Here the plugins are WebAssembly too (toolchain/build-plugins.sh: the shim's SwiftDataMacros, the toolchain's ObservationMacros and SwiftMacros, the standard library's, each given a @main, and swift-testing's TestingMacros), built with swift-syntax for WASI, and the host runs them in the compiler's own thread (src/plugins.ts): the compiler, patched to ask the host instead of spawning (PluginRegistry.cpp), imports plugin_spawn, plugin_write, plugin_read and plugin_close from a swiftbrowser module; the host calls the plugin's swift_wasm_macro_v1_pump export once per message, with the message on its stdin and the reply taken from its stdout. Everything is synchronous, so it works the same under Node and in a browser worker; the e2e tests expand @Observable on both hosts, and CI does on a runner without Swift. A compiler is named a plugin with -load-plugin-executable /plugins/ObservationMacros.wasm#ObservationMacros; the CLIs find the plugins in plugins/ next to the tools.

Expanding macros needs the compiler built with swift-syntax and ASTGen (SWIFT_SYNTAX=ON toolchain/build-swift.sh): swift-syntax as static libraries, the regex parser from swift-experimental-string-processing, and no plugin server or shared-library macros on WASI. The frontend grows from 168 to 187 MB (27.3 MB brotli); each plugin is about 41 MB (7.2 MB brotli).

The module cache

The first compile that imports Foundation builds Clang modules for SwiftShims, the WASI libc and Foundation's C parts: 13 MB of PCMs. On disk (the CLIs) they stay in the module cache between builds. In memory (the browser) a session starts empty, so memoryHost({ moduleCache }) starts from a prebuilt cache and moduleCacheFiles() returns the one a session ends with. Every file in memory has the same timestamp, so a cache built for the same files and flags validates in any session and nothing is rebuilt; the e2e tests check that only the pruning timestamp changes. toolchain/prebuild-module-cache.mjs builds one by type-checking a source that imports Foundation and Observation (the cache directory is a hash of the flags, and a type-check and a compile share it): 8 modules, 13.0 MB, 3.3 MB brotli, for the playground to ship with the SDK, built with the flags it compiles with.

On the in-memory host, a program importing Foundation runs its frontend in 2.8 s with an empty cache and 1.0 s with the prebuilt one. Before writeAt in src/memory-host.ts, the empty cache took 12.6 s: browser_wasi_shim grows a file to exactly the size of each write, so writing a 10 MB PCM in small writes copied it quadratically; writes now go into a buffer that doubles.

What the examples found

• FoodTruck: Swift sized debug_value and alloc_stack instructions without the padding before their 8-byte-aligned trailing debug-expression elements, which SROA then wrote past on a 32-bit host (fixed from tools release swift-6.4.0-r2).
• HotProspects, built twice: LLVM maps a file read-only with MAP_NORESERVE to compare an output with the one it replaces, which wasi-libc's emulated mmap rejects, so rewriting an unchanged .swiftmodule failed (fixed in toolchain/patches/llvm-project).
• The in-memory host: browser_wasi_shim reports readlink as unsupported rather than "not a link", and grants directories no rights, both of which the compiler's checks on a plugin's path trip over; src/memory-host.ts answers as a real WASI host does.

Building the tools

The scripts in toolchain/ keep everything under $WASMC_WORK (about 30 GB), in this order:

export WASMC_WORK=~/wasmc
toolchain/fetch.sh               # Swift 6.4.0, its Swift SDK for Wasm, wasmtime; the sources, patched
toolchain/build-native-tools.sh  # llvm-tblgen and clang-tblgen for the build machine
toolchain/build-sysroot.sh       # the Swift SDK's sysroot, with a libc++ that has threads
toolchain/build-resources.sh     # the Swift SDK's resources, with a libc module map C++ interop can use
toolchain/build-llvm.sh          # LLVM, Clang and LLD for a wasm32-wasip1 host
toolchain/check-llvm.sh          # clang.wasm and lld.wasm compile and link a C program that runs
toolchain/build-cmark.sh         # swift-cmark for the same host
toolchain/build-swift.sh         # swift-frontend for the same host, and the LLVM libraries it links
toolchain/build-plugins.sh       # the macro plugins, as WebAssembly
toolchain/build-manifest-api.sh  # PackageDescription and CompilerPluginSupport for WebAssembly, for manifests
toolchain/install-tools.sh       # swift-frontend.wasm, wasm-ld.wasm, the plugins and manifest-api into $WASMC_WORK/build/tools

toolchain/publish-tools.sh pack, then upload, compresses the tools with brotli and puts them in Cloudflare R2 (bucket swiftbrowser-wasm-tools, under the release versions.sh names), and writes packages/wasm-tools/tools.json, which pins them. @swiftbrowser/wasm-tools (packages/wasm-tools) is how everything else gets them: its fetch.mjs downloads them from R2 into the package and checks them, the npm release publishes the package with them, and the WebAssembly compiler workflow (.github/workflows/wasm-compiler.yml) uses them to compile HelloWasm on a runner without Swift. The CLI takes them from that package when neither --tools nor SWIFTBROWSER_WASM_TOOLS is set. To use the published tools instead of building them:

CLOUDFLARE_API_TOKEN=... CLOUDFLARE_ACCOUNT_ID=... node packages/wasm-tools/fetch.mjs

Then, to record the SDK files a compile opens, on the in-memory host:

npm run build --workspace packages/wasm-compiler
ulimit -s unlimited
node --stack-size=65500 packages/wasm-compiler/toolchain/record-sdk-files.mjs \
  --sdk <swift-sdk>/wasm32-unknown-wasip1 --tools $WASMC_WORK/build/tools \
  --source Sources/HelloWasm/main.swift --out hello.json

The native toolchain's clang compiles everything against the Swift SDK's WASI sysroot, so the compiler's C++ and its Swift parts share one libc and one libc++. The compiler's Swift parts (SwiftCompilerSources, mandatory since Swift 6) are cross-compiled by the native swiftc with C++ interop. toolchain/patches holds the changes to LLVM, Swift and swift-syntax; after editing the sources under $WASMC_WORK/src, toolchain/save-patches.sh writes them back, each ending its description with the base it applies to. That base is pinned in toolchain/versions.sh: SWIFT_TAG and, per source tree, the commit the tag names, which fetch.sh checks every clone against. Moving to another Swift release means changing both, then rebasing the patches.

Tests

npm run test:unit checks the driver against the jobs the native driver and clang run, recorded in test/fixtures by toolchain/record-invocations.mjs. With WASMC_WORK set, test/e2e.test.ts also compiles, links and runs hello world with the native tools, and with the WebAssembly tools once $WASMC_WORK/build/tools holds them.

Native baseline

Step 1.1 measured the native Swift 6.4.0 frontend and wasm-ld for wasm32-unknown-wasip1 (toolchain/measure.py: wall time and peak resident memory), on a 4-core container that was also building LLVM, so the times are on the slow side. The examples are staged by the CLI and compiled against its kit, with the toolchain's macro plugins (@Observable needs ObservationMacros).

| Input | Step | Time | Peak memory | |---|---|---|---| | HelloWasm | -typecheck | 0.06 s | 101 MB | | HelloWasm | -c -Onone | 0.07 s | 114 MB | | HelloWasm | -c -O | 0.14 s | 129 MB | | HelloWasm | wasm-ld | 0.19 s | 119 MB | | Counter, 3 files | -typecheck, whole module | 0.13 s | 138 MB | | Counter | -c -Onone, one primary file | 0.20 s | 153 MB | | Counter | -c -O -wmo | 1.11 s | 228 MB | | Settings, 3 files | -typecheck, whole module | 0.30 s | 166 MB | | Settings | -c -O -wmo | 1.61 s | 200 MB | | HackingWithSwift, 16 files | -typecheck, whole module | 0.87 s | 186 MB | | HackingWithSwift | -c -O -wmo | 5.19 s | 245 MB |

A compile stays far below the 4 GB a 32-bit WebAssembly memory allows, so the compiler is built for wasm32; Memory64 is not needed.