@katana-video/web-bg-removal

Background removal that runs entirely in the browser. Hand-written WebGPU shaders (WebGL2 fallback), no inference runtime, no server, no upload. ImageBitmap in, ImageBitmap out. VideoFrame in, VideoFrame out. The frame never leaves the GPU: resample and normalize, the network, the matte upsample and the compositing are all shaders; the result comes back as an ImageBitmap/VideoFrame off the model's own canvas. Works from the main thread or a Worker (no DOM).

Two model families, both MIT:

| model | best for | runs at | download (f16) | speed (1024² image) | |---|---|---|---|---| | birefnet_lite | any subject, fine edges and hair, still images | 1024×1024 | 90 MB | M4 0.3 s · Intel Gen12 laptop 1.3 s · Gen9 Chromebook 18 s | | longpipe-xl | people, real time | 1280×768 | 14.5 MB | real time on Apple Silicon / dGPU | | longpipe-large | people, real time | 640×400 | 6.7 MB | real time on mid-range laptops | | longpipe-small | people, real time | 384×224 | 2 MB | real time on Chromebooks |

Try it in the browser → katana.video/remove-background (a thin page over this package; source is here).

Status: early. APIs may change between minor versions.

Install

npm install @katana-video/web-bg-removal

Image

import { BackgroundRemover } from '@katana-video/web-bg-removal'

const remover = await BackgroundRemover.load('birefnet_lite', { onProgress: (f) => console.log(`${(f * 100) | 0}%`) })
const bitmap  = await createImageBitmap(fileOrBlobOrImageElement)
const cutout  = await remover.removeBackground(bitmap)                 // ImageBitmap, RGBA, transparent background, same size as the input
// options: { output: 'alpha' } (grayscale matte) · { output: { background: '#ffffff' } } · { output: { background: someImageBitmap } }

Draw cutout to a canvas and canvas.toBlob('image/png') for a transparent PNG.

Video

Feed frames in order; the model keeps whatever temporal state it needs. Same timestamp comes back out. The longpipe tiers run longpipe's optical-flow stabilizer: a small flow net rides the matting encoder's cached activations, the previous matte is warped forward and blended with the fresh one where nothing moved, so static edges stop flickering while motion stays sharp. temporal: false at load turns it off (per-frame matting only).

const remover = await BackgroundRemover.load('longpipe-large')
const processor = new MediaStreamTrackProcessor({ track })            // or WebCodecs VideoDecoder output, etc.
for await (const frame of processor.readable) {
  const out = await remover.process(frame, { output: { background: '#00ff00' } })   // VideoFrame
  frame.close()
  // … encode / display / write to a MediaStreamTrackGenerator
  out.close()
}
remover.reset()                                                        // scene cut / seek: clears the temporal state

remover.alpha(src) returns the raw matte (Float32Array at model resolution) if you want to composite yourself (this is the one call that reads back from the GPU).

Workers

Everything is OffscreenCanvas/WebGPU/WebGL2 — no document, no window. Load the package in a module Worker and post ImageBitmaps/VideoFrames across (both are transferable) to keep the main thread free.

Weights

Weights load once from https://cdn.katana.video/web-bg-removal/v/<weights version>/ and are cached with the browser Cache API. The weights version only moves when the .bin files change (0.1.x all use v/0.1.0/). Directory listing with sizes and SHA-256: <base>/index.html, machine-readable <base>/manifest.json.

• Self-host: copy that directory to your own origin and pass weightsBaseUrl: 'https://your.cdn/path/'.
• Bring your own bytes: load(name, { weights: arrayBufferOrBlob }) (BiRefNet also takes { bin, manifest }), e.g. from IndexedDB.
• dtype: 'f16' | 'f32' | 'auto' — f16 is half the download and faster; auto picks f16 where the GPU supports it.
• backend: 'webgpu' | 'webgl' | 'auto' — WebGL2 is the fallback for browsers without WebGPU (~8–10× slower).
• clearWeightsCache() drops the cached files.

Demo

demo/index.html (single image) and demo/video.html (a clip through process() with fps and per-frame timing) run against the built dist/ — serve the repo root and open them.

Correctness

Every model is checked against its PyTorch reference: BiRefNet_lite's WebGPU and WebGL pipelines match to a matte MSE of ~1e-10 (f32) / ~2e-9 (f16) on a fixed test image; each kernel has a CPU-reference test on both backends; the longpipe tiers run longpipe's own PyTorch fixtures. npm test (vitest, headless Chromium).

How it works

• src/model/ — inference only. backends/webgpu (WGSL compute: register-tiled conv/GEMM, fused window attention, deformable conv as im2col+GEMM, …), backends/webgl (the same kernels as fragment shaders), networks/birefnet_lite.ts (the network as code). Per-device kernel presets are picked by a short sweep at load.
• src/longpipe/ — longpipe's model layer, vendored (MIT), for the person-matting tiers.
• src/model/backends/*/io.ts and src/longpipe/.../composite_fullres.ts — the GPU in/out path: frame texture -> normalized model input; logits/alpha -> sigmoid, bilinear upsample, composite (cutout / alpha / solid / image) -> the backend's OffscreenCanvas -> transferToImageBitmap() / new VideoFrame(canvas).
• src/pipeline/ — the public API: ImageBitmap/VideoFrame in and out, weight fetching and caching, output modes. Knows nothing about GPUs or which model is running.

Design notes and measurements live in the repo's docs/ (op inventory, kernel findings, API shape).

License

MIT for the SDK. Weights: BiRefNet (MIT, Zheng Peng et al.) and longpipe (MIT) — see WEIGHTS_LICENSE.