@map-gesture-controls/core

Turn any web map into a hands-free experience. This is the map-agnostic gesture detection engine behind map-gesture-controls. It uses MediaPipe hand-tracking WASM to detect hand gestures from a webcam feed, classify them in real time, and expose a clean event-driven API. All processing runs locally in the browser. No video data ever leaves the device.

Building with OpenLayers? Use @map-gesture-controls/ol instead. It wraps this package and adds map integration out of the box.

What it does

• Detects hands and classifies gestures at 30+ fps using MediaPipe Hand Landmarker
• Recognizes fist (pan), open palm (zoom), and idle states
• Manages gesture transitions with dwell timers and grace periods to avoid flickering
• Provides a configurable webcam overlay with corner/full/hidden display modes
• Ships fully typed TypeScript declarations

Install

npm install @map-gesture-controls/core

Quick start

import { GestureController } from '@map-gesture-controls/core';
import '@map-gesture-controls/core/style.css';

const controller = new GestureController({
  onGestureFrame(frame) {
    // frame.hands contains detected hands with landmarks and gesture type
    console.log(frame);
  },
});

// Must be called from a user interaction (button click) for webcam permission
await controller.start();

Exports

| Export | Type | Description | | --- | --- | --- | | GestureController | Class | Opens the webcam, runs MediaPipe detection, and emits gesture frames | | GestureStateMachine | Class | Manages gesture state transitions with dwell and grace timers | | WebcamOverlay | Class | Renders a configurable camera preview overlay | | classifyGesture | Function | Classifies a set of hand landmarks into fist, openPalm, or none | | getHandSize | Function | Computes the bounding size of a hand from its landmarks | | getTwoHandDistance | Function | Measures the distance between two detected hands | | DEFAULT_WEBCAM_CONFIG | Constant | Default webcam overlay settings | | DEFAULT_TUNING_CONFIG | Constant | Default tuning parameters |

Full TypeScript types are exported for GestureMode, GestureFrame, DetectedHand, WebcamConfig, TuningConfig, and more.

Gesture recognition

| Gesture | Detection rule | Use case | | --- | --- | --- | | Fist | One hand, 3+ fingers curled | Pan / drag | | Open palm | Two hands, all fingers extended and spread | Zoom in/out | | Idle | Anything else | No action |

Gestures are confirmed after a configurable dwell period (default 80 ms) and held through a grace period (default 150 ms) to prevent flickering when tracking briefly drops.

Use cases

• Kiosk and exhibit displays where touch screens get dirty or break down
• Accessibility for users who cannot use a mouse or touchscreen
• Touchless interfaces in medical, industrial, or public environments
• Custom map integrations beyond OpenLayers (build your own adapter using this core engine)

Browser support

Requires WebGL, getUserMedia (webcam), and WASM support. Works in Chrome 111+, Edge 111+, Firefox 115+, and Safari 17+.

Documentation

Full docs, live demos, and configuration reference at sanderdesnaijer.github.io/map-gesture-controls

Privacy

MediaPipe WASM and the hand landmarker model are loaded from public CDNs. No video frames are sent to any server. All gesture processing happens locally in the browser.

License

MIT