handle-and-face 🦾👤

Improved accessibility and precise tracking for modern web applications.

handle-and-face is a React library that provides highly calibrated hooks for Hand Tracking (index finger pointer + pinch click) and Head Tracking (nose-driven pointer + blink click). Built on top of MediaPipe, it simplifies the integration of advanced computer vision for accessibility and interactive web experiences.

🇪🇸 Documentación en Español

Instalación

npm install handle-and-face

Hooks Disponibles

1. useHandTracking

Ideal para navegar usando el dedo índice como puntero.

• Gesto: Junta el pulgar y el índice para hacer click.
• Implementación:

const { videoRef, canvasRef, pointer, isClicking } = useHandTracking({
  sensitivityX: 2.5,
  onPinch: (x, y) => console.log("Click en", x, y),
});

2. useHeadTracking

Perfecto para control manos-libres y accesibilidad.

• Gesto: Parpadea una vez para click, dos veces rápido para doble click.
• Implementación:

const { videoRef, canvasRef, pointer, isClicking } = useHeadTracking({
  sensitivityX: 10.0,
  onClick: (x, y) => console.log("Click"),
  onDoubleClick: (x, y) => console.log("Doble Click"),
});

🇺🇸 Documentation in English

Installation

npm install handle-and-face

Available Hooks

1. useHandTracking

Use your index finger as a virtual mouse.

• Gesture: Pinch thumb and index to click.
• Usage:

const { videoRef, canvasRef, pointer, isClicking } = useHandTracking({
  sensitivityX: 2.0,
  onPinch: (x, y) => console.log("Pinch click at", x, y),
});

2. useHeadTracking

Touch-free navigation using head movement.

• Gesture: Blink to click, rapid double-blink for double-click.
• Usage:

const { videoRef, canvasRef, pointer, isClicking } = useHeadTracking({
  sensitivityX: 10.0,
  onDoubleClick: (x, y) => console.log("Double blink detected"),
});

🛡️ Stability & Precision

The library includes custom-built algorithms for:

• Dynamic Smoothing: Reduces lag during fast movements and adds weight for precision when aiming at small targets.
• Blink-Freeze: Temporarily anchors the pointer during blinks/clicks to prevent unintended cursor drift.
• Auto-Calibration: Pre-tuned sensitivity values for standard laptop cameras.

⚙️ Configuration Parameters

| Parameter | Type | Default | Description | | :-------------------- | :--------- | :----------- | :------------------------------------------------- | | sensitivityX/Y | Number | 2.0 - 10.0 | Pointer speed multiplier. | | jitterThreshold | Number | 1.5 | Minimum movement pixels to ignore (reduces noise). | | onPinch / onClick | Function | null | Single interaction callback. | | onDoubleClick | Function | null | Double interaction callback (Head Tracking only). |

📄 License

MIT © Starla.io