Longpipe 🐉

Fast, high-quality video effects in the browser — virtual backgrounds, face touch-up, auto-reframe, and mic noise removal in one open-source SDK.

Try the live demo → longpipe.dev/demo

Warning — This project is very new and still under active development. Expect API changes between versions and bugs.

Quick start

npm install longpipe

import { EffectsPipeline } from 'longpipe'

const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true })

const pipeline = new EffectsPipeline(stream, {
  background: 'blur',   // or an image, video, or solid color
  touchup: true,        // skin smoothing
  reframe: true,        // auto-frame the subject
  audio: 'denoise',     // mic noise removal
})

videoEl.srcObject = pipeline.stream   // available immediately
await pipeline.ready                  // optional — resolves once the effect is live

pipeline.stream is wired synchronously and emits the unprocessed input until the model is ready (~1–3 s on cold start), so users see live video the whole time. Model weights stream from cdn.longpipe.dev by default — no extra setup, and self-hostable.

Why Longpipe

• One model, many effects. Backgrounds, touch-up, and auto-reframe all run off a single shared encoder pass — enabling another effect doesn't cost another inference. Audio denoise runs in a parallel AudioWorklet, off the GPU entirely. (Architecture)
• More accurate and faster than the open-source alternatives. Custom-trained models running as pure WebGPU/WebGL shaders in a zero-copy, fully-GPU pipeline — no general-purpose runtime, no CPU↔GPU round trips.
• Works everywhere. Chromium, Firefox, and Safari (desktop + iOS). WebGPU when available, WebGL2 fallback, per-browser frame transport handled internally.
• Adapts to the device. Five model presets (xs–xl); autotune benchmarks the actual device at init and an adaptive controller swaps presets at runtime to hold 30 fps — on 10-year-old netbooks and the latest MacBook Pro alike.
• Production pedigree. Built by the founder of Vectorly, a commercial effects SDK acquired in 2021 — a ground-up redesign for the WebGPU era.

Best models on the web

Longpipe's hydranet architecture, custom model-specific shaders, zero-memory-copy pipeline, developer friendly interface and 100% MIT open source license mean that 🐲 Longpipe pareto dominates on performance, model quality and ease of use.

Documentation

Full docs live at longpipe.dev/docs:

• Getting started
• API reference — all EffectsPipeline options and methods
• Architecture — the hydranet: one encoder, many heads
• Backgrounds · Touch-up · Auto-reframe · Audio denoise
• Presets & autotune
• Model layer — drive the model directly, without the pipeline
• Self-hosting weights

Machine-readable for LLMs and agents: llms.txt · llms-full.txt

Roadmap

• [x] Background segmentation / virtual backgrounds
• [x] Background noise removal (audio, separate pipeline)
• [x] Face landmarks + touch-up
• [x] Multi-face support
• [x] Auto-reframe
• [ ] AR effects
• [ ] Lighting correction

License

SDK source code and pre-trained model weights are MIT-licensed — see LICENSE and WEIGHTS_LICENSE. You're free to self-host the weights inside other open-source or commercial projects under MIT-compatible terms.

The weights were trained on permissively-licensed public datasets (P3M-10k, AISegment, COCO, OpenImages), Z-Image synthetics, and a custom webcam dataset collected with participants' explicit, informed consent. The full dataset list and provenance details: longpipe.dev/docs/licensing.