hand-gesture-control

Hand gesture detection and action mapping from MediaPipe landmarks. TypeScript types included.

Install

npm install hand-gesture-control @mediapipe/tasks-vision

Usage

import { HandStateManager, HAND_STATES, GESTURE_ACTIONS } from 'hand-gesture-control';
import { createHandLandmarker } from 'hand-gesture-control/mediapipe';

const manager = new HandStateManager({
  waitingTimeout: 10000,
  startingDelay: 3000,
  swipeSource: 'index' // 'index' | 'wrist'
});

const landmarker = await createHandLandmarker();
const results = await landmarker.detectForVideo(videoCanvas, Date.now());
const { state, primaryAction, actions, rotation, palmDirection } =
  manager.processLandmarks(results.landmarks?.[0]);

Default actions

| Gesture | Action | |---------|--------| | Swipe left / right | swipe_left / swipe_right | | Open palm / closed fist | zoom_out / zoom_in | | Palm left / right / up / down | rotate_left / rotate_right / rotate_up / rotate_down | | Thumb up | ok |

Override via actionMap. Set mapActions: false to get states only.

Config

new HandStateManager({
  swipeSource: 'index',
  swipeMinRange: 0.03,
  swipeCooldownMs: 2000,
  palmDirectionThreshold: 0.025,
  rotationActionThreshold: 0.5,
  enableContinuousRotation: true,
  mapActions: true,
  actionMap: { /* partial override */ }
});

Invalid config throws ConfigValidationError.

Exports

HandStateManager, HandGestureDetector, ActionMapper, mapGestureToAction, validateHandConfig, mergeHandConfig, HAND_STATES, GESTURE_ACTIONS, DEFAULT_CONFIG, DEFAULT_ACTION_MAP, createHandLandmarker

Types: HandConfig, GestureResult, HandState, GestureAction, Landmark, ActionMap, …