micro-handpose

WebGPU hand tracking for the browser. Multi-hand detection with ROI tracking, 21 landmarks per hand. No WASM, no ONNX Runtime — just 15 compute shaders. 74KB JS + model weights downloaded at runtime.

Live Demo | npm

Quick Start

npm install @svenflow/micro-handpose

import { createHandpose } from '@svenflow/micro-handpose'

const handpose = await createHandpose()
const hands = await handpose.detect(videoElement)

for (const hand of hands) {
  console.log(hand.handedness)       // 'left' | 'right'
  console.log(hand.keypoints.wrist)  // { x, y, z }
}

Create once, detect per frame. Weights download on first call from CDN and are cached by the browser. Full TypeScript types included.

Benchmarks

Mac Mini M4 Pro — Chrome 134

| | Median | p99 | Backend | |---|---|---|---| | micro-handpose | 2.2ms | 3.1ms | WebGPU | | MediaPipe | 4.0ms | 6.5ms | WebGPU | | MediaPipe | 4.5ms | 8.2ms | WASM |

~2x faster than MediaPipe on the same hardware. With ROI tracking, most frames skip palm detection entirely — only landmark inference runs (~1.5ms).

Features

• 74KB minified JS (17KB gzipped) + 7.7MB weights (served via CDN)
• ~2x faster than MediaPipe on the same hardware
• Multi-hand tracking — detects up to 3 hands simultaneously
• ROI tracking — uses previous landmarks to track between frames (same approach as MediaPipe), skipping palm detection for smoother, faster results
• 21 landmarks per hand following MediaPipe ordering
• Named keypoints — hand.keypoints.thumb_tip, hand.keypoints.wrist, etc.
• Zero dependencies — pure WebGPU compute shaders, no WASM or ONNX Runtime

API

createHandpose(options?)

Creates and initializes the detector. Downloads weights and compiles WebGPU pipelines.

| Option | Type | Default | Description | |--------|------|---------|-------------| | weightsUrl | string | jsdelivr CDN | Base URL for weight files | | scoreThreshold | number | 0.5 | Minimum hand confidence (0-1) | | palmScoreThreshold | number | 0.5 | Minimum palm detection score (0-1) | | maxHands | number | 3 | Maximum hands to detect |

handpose.detect(source)

Detect hands in an image or video frame. Returns HandposeResult[] (empty if no hands found).

Accepts: HTMLVideoElement, HTMLCanvasElement, OffscreenCanvas, ImageBitmap, HTMLImageElement, ImageData

interface HandposeResult {
  score: number                // Confidence (0-1)
  handedness: 'left' | 'right'
  landmarks: Landmark[]        // 21 points, normalized [0,1]
  keypoints: Keypoints         // Named access: .wrist, .thumb_tip, etc.
}

handpose.reset()

Reset tracking state. Call when switching between unrelated images to force palm re-detection.

handpose.dispose()

Release GPU resources.

How It Works

Video frame → Palm Detection (192×192, 15 compute shaders)
           → ROI crop (affine transform on GPU)
           → Landmark model (224×224 EfficientNet-B0, 42 compute shaders)
           → 21 landmarks + hand score
           → ROI tracking (landmarks → next frame's crop region)

On the first frame, palm detection finds hand bounding boxes. On subsequent frames, landmarks from the previous frame compute the crop region directly — palm detection is skipped entirely. This matches MediaPipe's tracking approach: smoother results and ~40% less compute per frame.

Self-Hosting Weights

const handpose = await createHandpose({
  weightsUrl: '/models/handpose'
})

Copy the weights/ directory from the npm package to your server.

Browser Support

| Browser | Status | |---------|--------| | Chrome 113+ | ✅ | | Edge 113+ | ✅ | | Safari 18+ (macOS/iOS) | ✅ | | Firefox Nightly | Experimental |

Error Handling

Check for WebGPU support before initializing:

if (!navigator.gpu) {
  console.error('WebGPU is not supported in this browser')
  // Fall back to a non-WebGPU solution or show a message
}

Wrap initialization in a try/catch to handle GPU adapter or device failures:

try {
  const handpose = await createHandpose()
  const hands = await handpose.detect(videoElement)
} catch (err) {
  console.error('Failed to initialize hand tracking:', err)
}

SSR / Server-Side Rendering

micro-handpose requires WebGPU and browser APIs (navigator.gpu, OffscreenCanvas, etc.) that are not available in server environments. If you use a framework with server-side rendering (Next.js, Nuxt, SvelteKit, etc.), make sure to only import and initialize it on the client:

// Next.js example (app router)
'use client'

import { useEffect, useState } from 'react'
import type { Handpose } from '@svenflow/micro-handpose'

export default function HandTracker() {
  const [handpose, setHandpose] = useState<Handpose | null>(null)

  useEffect(() => {
    import('@svenflow/micro-handpose').then(({ createHandpose }) => {
      createHandpose().then(setHandpose)
    })
  }, [])

  // ...
}

FAQ

Does it work on mobile? Yes. WebGPU is supported in Chrome on Android and Safari on iOS 18+. Performance varies by device.

How many hands can it track? Up to 3 by default. Set maxHands in the options to change this.

Does it work offline? Model weights are downloaded on first use and cached by the browser. After that, it works offline. You can also self-host the weights (see Self-Hosting Weights).

What license is the model under? The model architecture and weights are derived from MediaPipe's hand landmark model, which is published under the Apache 2.0 license.

Development

git clone https://github.com/svenflow/micro-handpose.git
cd micro-handpose
npm install
npm run dev    # Watch mode with hot reload
npm run build  # Production build

Credits

• Hand landmark model architecture and weights adapted from MediaPipe Hands (Apache 2.0 license).
• ROI tracking approach follows MediaPipe's pipeline design (palm detection + landmark tracking with re-detection on loss).

License

MIT