Jervis MediaPipe

Production-oriented MediaPipe runtime and camera interaction SDK for the Web.

Jervis MediaPipe keeps MediaPipe inference in classic Web Workers, normalizes results into package-owned types, and provides an optional interaction layer for camera-driven pointer, scroll, drag, zoom, navigation, and virtual-keyboard experiences.

Documentation · Getting started · API reference · Gestures · Changelog

Install

npm install jervis-mediapipe

Quick start

The SDK loads models, WASM, and a worker over HTTP at runtime. In a Vite app the bundled plugin serves and emits all of them:

// vite.config.ts
import { defineConfig } from 'vite'
import { jervisMediaPipeVitePlugin } from 'jervis-mediapipe/vite'

export default defineConfig({
  plugins: [jervisMediaPipeVitePlugin()],
})

<video id="camera" autoplay playsinline muted></video>

import { createCameraControls } from 'jervis-mediapipe'

const video = document.querySelector<HTMLVideoElement>('#camera')!
const controls = createCameraControls({ video })

await controls.start()
// on teardown
await controls.destroy()

Move your hand to steer the cursor, pinch to click.

Not using Vite? See Serving the assets for the files to host and the one call that points the SDK at them. Skipping that step is the most common reason start() fails.

Highlights

• Hand, Face Detector, and Face Landmarker controllers.
• Worker-first inference with one frame in flight and dropped-frame metrics.
• Cursor smoothing, pinch lifecycle, swipe detection, and normalized results.
• Pointer, Scroll, and Keyboard interaction modes.
• Click, double click, context menu, drag, scroll, swipe navigation, and two-hand zoom.
• A camera-controlled virtual keyboard for inputs, textareas, and contenteditable elements.
• Gesture stabilization, confidence, hold progress, and custom command bindings.
• Optional DOM execution and visual overlay adapters.
• Framework-agnostic TypeScript with full type declarations.
• No runtime dependencies.

Built-in gestures

| Gesture | Action | | --- | --- | | Move tracked hand | Cursor | | Quick pinch | Click | | Two quick pinches | Double click | | Pinch hold | Context menu | | Pinch hold + move | Drag | | Open palm hold | Enter Scroll mode | | Hand movement in Scroll mode | Vertical / horizontal scroll | | Fist hold in Scroll mode | Return to Pointer | | Victory hold | Open virtual keyboard | | Point + pinch on a key | Type / activate key | | Open palm hold in Keyboard mode | Close keyboard | | Swipe left / right | Navigation event | | Two open palms | Zoom event |

Two-hand zoom needs hand: { numHands: 2 }; the default is 1.

Requirements

• A Chromium or WebKit browser with getUserMedia, Web Workers, and WebAssembly.
• A secure context — HTTPS in production, localhost in development.
• ESM only. There is no CommonJS build.
• Node 20.19+ for the Vite plugin.

Chrome and Safari are the verified engines. Others are expected to work but are not claimed as supported.

Package size

The tarball is around 16.5 MB, and 36 MB unpacked. Almost all of it is the MediaPipe model bundles and WASM binaries the SDK serves at runtime, so the package is self-contained and works offline with no CDN.

Only a subset is fetched by the browser: each controller loads its own model and worker, and a browser with WebAssembly SIMD never touches the nosimd fallback. See Serving the assets for the per-file breakdown and how to trim what you host.

Browser scope

The DOM adapter controls the web application that embeds Jervis MediaPipe. Browser JavaScript cannot globally control unrelated native desktop applications. Device-wide control should forward typed interaction actions to a trusted native desktop or accessibility bridge with explicit operating-system permissions.

License

MIT © Kerolos Zakaria. The full text ships in the package as LICENSE.

This package redistributes MediaPipe WASM binaries and pre-trained models from Google LLC, licensed under Apache-2.0. Attribution and the list of redistributed files ship in the package as NOTICE.