@brainchop/mindgrab

Browser MeshNet brain segmentation on WebGPU, with WebGL2 and threaded-CPU fallbacks: MindGrab skull stripping, 18-class parcellation at two widths (16chan18cls, mindmap), and a 104-class Desikan-Killiany parcellation (mindsnap), over a fixed 256³ grid. A NIfTI ArrayBuffer goes in and a NIfTI ArrayBuffer comes out.

The whole pipeline — conform, normalise, the 13 or 25 MeshNet layers, classify, largest-component, reslice — runs inside a wasm module compiled from brainchop-c. This package owns only what that module deliberately does not: gzip, WebGPU device and WebGL2 context acquisition, the cross-origin-isolation probe, and the decision to refuse.

import { segment } from '@brainchop/mindgrab'

const input = await file.arrayBuffer()          // .nii or .nii.gz
const { image, elapsedMs } = await segment(input, { model: 'mindgrab' })
// image is gzipped if the input was

MindMap now filters selected tissue classes and refills suppressed speckles from surviving neighboring labels by default. Use { model: 'mindmap', legacyCleanup: true } for the original whole-brain-only cleanup. This requires wasm assets rebuilt from the same revision as the wrapper.

Categorical native-space output uses the browser's eight-sample majority vote, including background votes outside the source and first-sample tie breaking. Masks still use nearest neighbor; continuous tissue maps use linear resampling. Invalid, empty, more than 95% filled, or excessively fragmented categorical results fail with inference-failed before cleanup can disguise the failure.

Continuous tissue maps

import { segmentTissues } from '@brainchop/mindgrab'

const { tissues, backend } = await segmentTissues(input, {
  model: 'mindmap', worker: true,
})
// tissues.gm, tissues.wm, tissues.csf are NIfTI ArrayBuffers.
// Native geometry and input gzip encoding are preserved by default.

segmentTissues supports mindmap and 16chan18cls on WebGPU, WebGL2 and CPU. The GPU paths compute grouped priors, blur and support gating on the device; the shared C CAT-lite fitter runs on CPU. Fitting uses the MindMap-calibrated constants for both models. Model16 coverage checks implementation agreement, not tissue-model calibration. saveConform and gzipOutput work as for segment; mask, border and legacy categorical cleanup options are refused. The WebGL2 module reserves a 768 MiB wasm heap for the host-side tissue fit.

Backends

auto (the default) tries them in this order and reports which one ran in result.backend:

MindGrab on an M4 Pro, as time inside the wasm module and as what a segment() call actually costs. The difference is this package's own work — fetching and instantiating the module, gzip and gunzip, and copying the volume into and out of the module's filesystem — not the segmentation, which is wholly inside the first number:

| backend | needs | in the module | per call | | --- | --- | --- | --- | | 'webgpu' | WebGPU with shader-f16, 512 MiB buffers, a secure context | 1.6 s | 2.4 s | | 'webgl2' | WebGL2 with EXT_color_buffer_float | 3.4 s | 4.1 s, and it BLOCKS the calling thread — use worker: true | | 'cpu' | a cross-origin isolated page (see below) and ~2.15 GB of growable shared memory | ~10 s on 14 cores | ~12.6 s |

The CPU row's 2.6 s of wrapper time is about 3× the GPU rows' 0.7–0.8 s, which is not an arithmetic slip: it is a separate and larger module, and instantiating it spins up a worker per core.

WebGL2 is what Linux Firefox and Chrome take today, since neither enables WebGPU by default; those stacks are known to run correctly but have never been timed, so the figures above are macOS. Naming a backend explicitly makes an unsupported choice an error rather than a silent downgrade. The CPU module is the same fp32 C the command-line executables run and is bit-exact with them; the two GPU paths are fp16 and carry the voxel budgets under Accuracy.

The CPU fallback needs two headers

It is a pthreads build, so it imports shared memory, and a page that is not cross-origin isolated cannot instantiate it — the module hangs rather than failing, which is why this is checked before it is fetched. Serve the top-level document with:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Putting them on the .wasm response instead does nothing: isolation is a property of the document. Note the cost, which is the usual reason a deployment turns isolation back off again: under require-corp every cross-origin subresource the page loads — images, fonts, analytics, embeds — must itself send Cross-Origin-Resource-Policy or proper CORS, or it will be blocked. Then:

import { checkCpuSupport } from '@brainchop/mindgrab'

const cpu = checkCpuSupport()
if (!cpu.supported) console.warn(cpu.reasons.join('\n'))

Playwright's patched Firefox Nightly runs this module about 20× slower than Chromium, but installed Firefox 153.0.1 ran the same bit-exact model16 case in 7.6 s. The Nightly result is retained as a harness/compiler anomaly, not a warning about Firefox users generally.

Requirements

For the WebGPU path: WebGPU with the shader-f16 feature, maxStorageBufferBindingSize and maxBufferSize of at least 512 MiB (one f16 activation buffer), and a secure context — https or localhost. about:blank and file:// are not secure contexts, and there navigator.gpu is undefined, which looks exactly like a browser with no WebGPU at all.

Ask before you offer the user a button:

import { checkSupport } from '@brainchop/mindgrab'

const report = await checkSupport()
if (!report.supported) console.warn(report.reasons.join('\n'))

Models

| model | does | output | | --- | --- | --- | | 'mindgrab' | skull stripping in any modality | the input image with non-brain voxels floored | | '16chan18cls' | 18-class brain segmentation | label volume | | 'mindmap' | 18-class brain segmentation (24-channel) | label volume | | 'mindsnap' | 104-class Desikan-Killiany parcellation | label volume |

Each is a separate wasm module with its weights compiled in, fetched only when you ask for it. Options are per-model and the mismatched ones are refused before anything is downloaded, because an executable never advertises what it would decline:

await segment(buf, { model: 'mindgrab', saveConform: true })
// BrainchopError: mindgrab does not support `saveConform`: its output is the
// input image with non-brain voxels floored, so it is inherently in the input space

Options

segment(input: ArrayBuffer | ArrayBufferView, options: SegmentOptions)

| option | models | meaning | | --- | --- | --- | | model | all four | required; there is no sensible default among them | | ct | all four | input is CT; convert Hounsfield to Cormack units | | comply | all four | insert the compliance transform before conform | | saveConform | 16chan18cls, mindmap, mindsnap | return the conformed 256³ volume instead of reslicing to the input grid | | mask | mindgrab | also return a binary brain mask in result.mask | | borderMm | mindgrab | grow the mask border, in mm; implies mask | | gzipOutput | all four | override the default, which matches the input | | worker | all four | run the whole segmentation in a Web Worker. Recommended, and the only way to keep a page responsive during a webgl2 run, which is synchronous | | device | all four | reuse a GPUDevice you already own | | glContext | all four | reuse a WebGL2RenderingContext you already own; implies webgl2 | | assetPath | all four | base URL the model's .js/.wasm are served from — required under a bundler, see below | | timeoutMs | all four | inference limit (default 120000; 900000 for cpu, and for auto under worker: true, where the backend is chosen after the timer is armed). Module loading/instantiation has its own limit: the lesser of timeoutMs and two minutes, so a stalled pthread startup cannot masquerade as a slow CPU run. A lost GPU device otherwise never settles; only worker: true cancels for real | | onLog | all four | receives the module's output lines |

Output is in the input image's own space by default, for every model. This inverts brainchop-cli, where the conformed grid is the default.

--crop and --export-classes, which the native executables offer, are deliberately absent. Both force the CPU engine inside the GPU modules, which is compiled without threads there and takes 141–266 s per volume; accepting them would turn a two-second call into a four-minute one with no warning. The cpu backend is a different module and does not change this.

Bundlers

The emscripten glue is loaded by URL at run time, and it locates its own .wasm through its own import.meta.url — so the two files must stay adjacent and unhashed. Vite and Rollup will neither rewrite that URL nor emit the assets. Copy the model's .js and .wasm out of this package's dist/ into somewhere served, and point assetPath at it:

await segment(buf, { model: '16chan18cls', assetPath: '/brainchop/' })

assetPath must be same-origin; it feeds a dynamic import().

Errors

Every package error throws a BrainchopError with a code you can branch on: no-webgpu, no-adapter, device-too-small, no-f16, unsupported-option, bad-input, initialization-failed, inference-failed. initialization-failed means the module never started, so retrying can be meaningful; inference-failed means it started and did not finish. There is no fallback behind any of them, by decision — a silently downgraded segmentation is worse than a refusal. auto choosing between backends is not an exception: it happens before anything is fetched and only when you expressed no preference.

no-webgpu is also what a cpu refusal reports — "this page is not cross-origin isolated" arrives under that code rather than a fourth one, so read the message, or call checkCpuSupport() first.

Accuracy

The cpu backend is held to zero differing voxels against frozen CPU references. The model16 native-space majority fixture has a separate wasm reference: one pre-existing native/wasm conformed label difference is sampled by the new vote. It was reproduced from unmodified merged main and the new fixture independently verified against the browser voting function. See reference provenance. The two GPU paths are fp16, held to the same per-voxel budgets as the project's Metal backend rather than to bit-exactness. Historical nearest-neighbor results on the project's fixtures, as voxels of 16.7M (2 mm case, 1.6M):

| case | budget | webgpu | webgl2 | | --- | --- | --- | --- | | MindGrab | 64 | 37 | 33 | | 18-class (16-channel) | 1024 | 682 | 710 | | 18-class (16-channel), native 2 mm | 256 | 88 | 94 | | mindmap (24-channel, 18-class) | 1024 | 564 | 610 | | mindmap, native 2 mm | 256 | 88 | 86 | | mindsnap (104-way argmax) | 2048 | 1280 | 1219 | | mindsnap, native 2 mm | 512 | 198 | 184 |

The current majority-vote checks retain these budgets. Tissue maps retain the existing 2/255 RMSE gate over the nonzero union and exact NIfTI geometry.

Building

npm install
npm run build     # build all three backends' modules, stage into dist/, types, bundle
npm test          # drives the built package in a real browser

The test runs headed — headless Chromium on macOS has no GPU — and needs a checkout with playwright installed (BC_PLAYWRIGHT_ROOT).