🖐️ react-hand-gesture-control

Control your React app with hand gestures — scroll, move a cursor, click, and more — all through your webcam. Powered by MediaPipe Hands.

🎬 What It Does

Wrap your app with a single component, tell it which routes to enable, and your users can:

• ☝️ Scroll pages and containers with one finger
• 🖐️ Move a virtual cursor by showing an open hand
• 🤏 Click on elements by pinching thumb + index finger
• ✊ Pause/resume gesture control with a fist

Works with any React router (or none at all). Zero dependencies on React Router.

📦 Installation

npm install react-hand-gesture-control

Peer dependencies: react >= 17, react-dom >= 17

🚀 Quick Start

import { HandGestureProvider } from "react-hand-gesture-control";

function App() {
  return (
    <HandGestureProvider routes="*" showCamera>
      <YourApp />
    </HandGestureProvider>
  );
}

That's it. Your app now responds to hand gestures.

🛣️ Route-Based Activation

Enable gestures everywhere or only on specific routes:

// All routes
<HandGestureProvider routes="*">

// Specific routes only
<HandGestureProvider routes={["/dashboard", "/gallery"]}>

// Wildcard support
<HandGestureProvider routes={["/admin/*"]}>

No react-router dependency — it reads window.location.pathname and intercepts pushState / replaceState internally.

🎮 Gesture Reference

The system has two modes: Normal and Cursor.

Normal Mode (default)

| Gesture | How | Action | | ------------------ | ------------------------------------------------------- | ------------------------- | | ☝️ Scroll Up | Point index finger, move down / hold in bottom zone | Page scrolls up | | ☝️ Scroll Down | Point index finger, move up / hold in top zone | Page scrolls down | | ☝️ Scroll Left | Point index finger, move left | Page scrolls left | | ☝️ Scroll Right | Point index finger, move right | Page scrolls right | | 🖐️ Activate Cursor | Show open hand (4+ fingers) | Switches to Cursor Mode | | ✊ Pause / Resume | Make a fist | Toggles gesture detection |

Cursor Mode

| Gesture | How | Action | | ------------------- | --------------------------------------- | ------------------------------------ | | 🖱️ Move Cursor | Move hand around (any pose) | Red cursor dot follows index finger | | 🤏 Click | Pinch thumb + index finger together | Dispatches a real click event | | ✊ Exit Cursor Mode | Make a fist | Hides cursor, returns to Normal Mode |

Key insight: Cursor mode is sticky. Once activated with an open palm, it stays on until you make a fist. No flickering.

📖 API Reference

<HandGestureProvider>

The main component. Wrap your app with it.

<HandGestureProvider
  routes="*"
  showCamera
  cameraPosition="bottom-right"
  scrollOptions={{ scrollSpeed: 20, scrollTarget: "auto" }}
  cursorOptions={{ showCursor: true, cursorSize: 24, cursorColor: "#ff0000" }}
  onGesture={(gesture) => console.log(gesture)}
  gestureCallbacks={{
    onScrollUp: () => {},
    onScrollDown: () => {},
    onScrollLeft: () => {},
    onScrollRight: () => {},
    onFist: () => {},
    onCursorMove: (x, y) => {},
    onCursorClick: (x, y) => {},
  }}
>
  <App />
</HandGestureProvider>

| Prop | Type | Default | Description | | ------------------ | -------------------------------------------------------------- | ---------------- | -------------------------------- | | routes | "*" \| string[] | required | Routes where gestures are active | | enabled | boolean | true | Master on/off switch | | showCamera | boolean | false | Show webcam preview overlay | | cameraPosition | "top-left" \| "top-right" \| "bottom-left" \| "bottom-right" | "bottom-right" | Preview position | | cameraWidth | number | 160 | Preview width (px) | | cameraHeight | number | 120 | Preview height (px) | | onGesture | (gesture) => void | — | Fires on any gesture | | gestureCallbacks | GestureCallbackMap | — | Per-gesture callbacks | | trackingOptions | HandTrackingOptions | — | MediaPipe config | | scrollOptions | ScrollOptions | — | Scroll behaviour | | cursorOptions | CursorOptions | — | Virtual cursor styling |

useHandGesture() Hook

Read gesture state from anywhere inside the provider:

import { useHandGesture } from "react-hand-gesture-control";

function GestureStatus() {
  const { gesture, isActive, isPaused, cursorPosition } = useHandGesture();

  return (
    <div>
      <p>Gesture: {gesture}</p>
      <p>Active: {isActive ? "Yes" : "No"}</p>
      {cursorPosition && (
        <p>
          Cursor: ({Math.round(cursorPosition.x)},{" "}
          {Math.round(cursorPosition.y)})
        </p>
      )}
    </div>
  );
}

| Field | Type | Description | | ---------------- | ------------------ | ---------------------------- | | gesture | GestureType | Current detected gesture | | landmarks | HandLandmarks | 21-point hand landmarks | | isActive | boolean | Whether tracking is running | | isPaused | boolean | Whether paused via fist | | cursorPosition | { x, y } \| null | Virtual cursor screen coords |

Option Types

ScrollOptions

{
  scrollSpeed?: number;       // Pixels per frame (default: 20)
  sensitivity?: number;       // 0–1, lower = more sensitive (default: 0.5)
  scrollTarget?: "auto" | "window";  // "auto" detects scrollable containers
  targetRef?: RefObject<HTMLElement>; // Explicit scroll container
}

CursorOptions

{
  showCursor?: boolean;       // Show cursor dot (default: true)
  cursorSize?: number;        // Dot size in px (default: 24)
  cursorColor?: string;       // Dot color (default: "#ff0000")
  cursorSmoothing?: number;   // 0–1, lower = smoother (default: 0.35)
}

HandTrackingOptions

{
  maxHands?: number;              // default: 1
  detectionConfidence?: number;   // default: 0.6
  trackingConfidence?: number;    // default: 0.4
}

GestureCallbackMap

{
  onScrollUp?: () => void;
  onScrollDown?: () => void;
  onScrollLeft?: () => void;
  onScrollRight?: () => void;
  onFist?: () => void;
  onCursorMove?: (x: number, y: number) => void;
  onCursorClick?: (x: number, y: number) => void;
}

🔧 Advanced Hooks

For full low-level control:

import {
  useHandTracking, // MediaPipe + camera lifecycle
  useGestureDetection, // Landmark → gesture classification
  useRouteMatch, // Route matching
} from "react-hand-gesture-control";

⚡ Performance

• 60fps gesture loop via requestAnimationFrame
• Zero-lag landmark reads — ref-based, no React state batching
• Throttled React updates — UI re-renders at ~20fps, detection runs at full speed
• Auto scroll detection — finds the correct scrollable container (divs, tables, modals)
• Camera stops when route doesn't match — saves CPU
• EMA smoothing reduces jitter without adding lag

🏗️ Build from Source

git clone <repo-url>
cd react-hand-gesture-control
npm install
npm run build

Output (dist/):

• index.cjs.js — CommonJS
• index.esm.js — ES Module
• index.d.ts — TypeScript declarations

🤝 Browser Support

Works in all modern browsers that support:

• getUserMedia (webcam access)
• WebGL (MediaPipe Hands)

Chrome, Edge, Firefox, Safari 15+.

📄 License

MIT

Built with ❤️ using React + MediaPipe Hands