GestureV Controller

A sophisticated hand and face gesture recognition system that allows you to control your computer using natural body movements.

Features

• 85+ Gesture Controls: Hand gestures, facial expressions, and head movements
• Real-time Visual Feedback: On-screen display showing detection status, FPS, and active gestures
• Interactive Help System: Press H for quick reference, G for detailed gesture guide
• Dual-hand Support: Complex gestures like clapping, triangles, and double gestures
• Mouse Control: Open palm to enable, fist to disable mouse navigation
• Application Launchers: Gestures to open Gmail, Steam, Roblox, and more
• Media Controls: Volume, play/pause, track navigation
• Accessibility Features: Designed for users with limited mobility
• 100% Private Processing: All analysis happens locally on your device

Getting Started

Install (Fresh Computer)

Windows 10/11

1. Download or clone this repository.
2. Double-click Install.bat (or run powershell -ExecutionPolicy Bypass -File .\install.ps1).
3. The installer will install missing dependencies (Git + Python), create .venv, install packages, and start GestureV.

macOS

1. Open Terminal and cd into this project folder.
2. Run bash ./install.sh (or double-click GestureV-Mac.command).
3. The installer will install Homebrew (if needed), Git, Python, virtualenv, dependencies, and then start GestureV.

Linux

1. Open Terminal and cd into this project folder.
2. Run bash ./install.sh (or bash ./GestureV-Linux.sh).
3. The installer will install Git + Python + venv tools (if needed), create .venv, install dependencies, and start GestureV.

System Requirements

• Windows 10/11, macOS, or Linux
• Webcam (built-in or USB)
• Python 3.10+ (installed automatically by the installer scripts)
• Required dependencies: OpenCV, MediaPipe, NumPy, PyAutoGUI, psutil

Node.js-Only Quick Start (Recommended)

Use this when the computer is new and only has Node.js installed.

Before You Run

1. Install Node.js 18+.
2. Connect a webcam.
3. Make sure you have permission to install system packages.
   • Windows: winget must be available.
   • macOS: Homebrew install must be allowed.
   • Linux: sudo + supported package manager (apt, dnf, yum, pacman, zypper, or apk).

Command (All Platforms)

Run this exact command in your terminal:

npx gesturev@latest --user-id=YOUR_ID --cloud-url=https://www.gesturev.com

Examples:

• Windows (PowerShell or CMD): npx gesturev@latest --user-id=YOUR_ID --cloud-url=https://www.gesturev.com
• macOS (Terminal): npx gesturev@latest --user-id=YOUR_ID --cloud-url=https://www.gesturev.com
• Linux (Terminal): npx gesturev@latest --user-id=YOUR_ID --cloud-url=https://www.gesturev.com

What Happens on First Run

1. GestureV installs itself under ~/.gesturev.
2. Python is installed automatically if missing.
3. A virtual environment is created.
4. Required Python packages are installed.
5. The controller starts and calibration begins.

First run can take a few minutes depending on internet speed.

If It Fails

• winget not found on Windows: install App Installer from Microsoft Store, then run again.
• Permission errors on macOS/Linux: rerun with a user that can install dependencies.
• Camera read failures: close Zoom/Teams/browser camera tabs and run again.

First-Time Setup

1. Launch the application
2. Look straight at your camera for automatic calibration (takes ~3-5 seconds)
3. Follow the on-screen instructions during calibration
4. Once calibrated, you'll see the main controller interface

Controls During Operation

• Q: Quit the application
• H: Toggle help overlay (quick gesture reference)
• G: Toggle gesture status panel (detailed detection info)
• Look at camera: Ensures optimal face detection for eyebrow/wink/smile gestures

Gesture Reference

Mouse & Pointer Controls

• Open Palm: Enable mouse cursor control
• Fist: Disable mouse cursor control
• Index Finger Pointing: Move cursor
• Pinch (Thumb + Index): Click and drag
• Pinch Release: Mouse click

Navigation & Actions

• 👍 Thumbs Up: Press Enter
• 👎 Thumbs Down: Press Escape
• ✌️ Peace Sign: Take screenshot
• 👉 Wink Left: Left Arrow key
• 👈 Wink Right: Right Arrow key
• 😊 Smile: Space bar
• 👂 Ear Wiggle (Left/Right): Adjust volume (if configured)

Head Movements

• 👂 Head Shake (Left/Right): Escape
• 👆 Head Nod Down: Scroll down
• 👆 Head Nod Up: Scroll up
• ↔️ Head Tilt Left/Right: Custom actions (if configured)

Scrolling

• ✌️ Two Fingers (Vertical): Scroll up/down
• 👆 Head Nod: Alternative scroll control

Special Gestures (Two-Hand Required)

• ⭕⭕ Double Circle: Open Gmail
• 🔺🔺 Triangle: Open Roblox
• 👍👍 Double Thumbs Up: Open Steam
• ✌️✌️ Double Peace: Mute/unmute audio
• 👉👉 Double Point: Custom action
• 🤟🤟 Double Pinky: Custom action
• ✊✊ Double Fist: Custom action
• 🖐️🖐️ Double Open Palm: Custom action

Accessibility Gestures

• 👆 Index Finger Point: Mouse movement
• ✌️ Peace + Thumb: Special function
• 🤟 Love Sign: Custom action
• 🤘 Rock On: Play/pause media
• 👌 OK Sign: Custom action
• 🤞 Crossed Fingers: Custom action

Advanced Finger Gestures

• 👆👇 Index Down: Point down action
• 👍✊ Thumbs Up Fist: Hitchhiker's thumb
• 🖐️➡️ Palm Forward: Palm push forward
• ✌️👍 Peace + Thumb: Peace thumb combo
• 👌✌️ Index Middle Touch: Precision grip
• 👌🤟 Ring Pinky Up: Precision grip variant
• ✌️👌 Index Ring: Precision variation
• 👍👆 Thumb Middle Touch: Precision grip
• ✌️👈 Bent Index: Hook gesture
• ✌️👉 Bent Middle: Reverse hook
• 🖐️🤌 Fingers Spread: Spread fingers
• ✊🤌 Fingers Closed: Fist variation
• 👆👆👆 Three Fingers: Three-finger salute
• 👆👆👆👆 Four Fingers: Four-finger salute
• 👍🤟 Thumb Pinky Touch: Pinch variant
• 👍🤌 Thumb Ring Touch: Ring touch
• 👆➡️ Thumb Forward: Thrust gesture
• 👆👇👈 Index Middle Ring: Three-finger cluster
• 👆👉 Thumb Index Spread: Wide spread
• 👈↔️ Index V Spread: V-shape spread
• 👆👇 Thumb Index Touch: Precision touch
• 👆👈 Index Middle Touch: Middle touch
• 🤟👌 Ring Pinky Up: Pinky lift
• 👇 Point Down: Downward point
• 👍✊ Thumbs Up Fist: Hitchhiker
• 🖐️➡️ Palm Forward: Forward palm
• 👆👇👈 Index Ring Pinky: Skip finger
• ✌️✂️ Thumb Cross Index: Cross fingers
• 🦞 Claw Hand: Claw shape
• 🖐️⬆️ Flat Hand Up: Flat palm up
• ✌️👍 Peace Thumb: Peace thumb combo

Help System

Quick Help (H Key)

Press H during operation to toggle a compact overlay showing:

• Most commonly used gestures
• Their primary functions
• Memory aids for learning

Detailed Help (G Key)

Press G during operation to toggle a comprehensive panel showing:

• Complete gesture library organized by category
• Visual descriptions where applicable
• Current system status (FPS, detection states)
• Last triggered gesture for feedback

On-Screen Display (Always Visible)

• Top-left corner: FPS counter and detection status indicators
• Bottom center: Gesture toast notifications (shows when a gesture is recognized)
• Bottom-left: Real-time gesture detection status (when enabled with G key)

Calibration Tips

For best results:

1. Ensure good, even lighting on your face
2. Position yourself so your face is clearly visible in the camera feed
3. Look straight at the camera during calibration
4. Avoid wearing hats or sunglasses that obscure facial features
5. Keep background relatively uncluttered

The system automatically calibrates eyebrow raise sensitivity on startup. If gestures feel inconsistent, simply look away and back at the camera to recalibrate.

Troubleshooting

No Gestures Detected

• Ensure webcam is properly connected and not used by another application
• Check lighting conditions - avoid backlighting
• Make sure your face is well-lit and clearly visible
• Try pressing H to see if help appears (confirms program is running)

Mouse Not Working

• Make sure you've shown an open palm to enable mouse control
• Show a fist to disable if you want to use keyboard normally
• Check that your webcam isn't being blocked by privacy settings

False Positives

• Adjust lighting to reduce shadows on face
• Ensure no moving objects in background that might be mistaken for gestures
• Recalibrate by looking away briefly then back at camera

Performance Issues

• Close other applications using the camera
• Ensure your system meets minimum requirements
• The FPS counter in top-left shows real-time performance

Customization

Gesture actions can be customized by editing the gesture bindings file:

• Location: ~/.gesturev/standalone/gesture_bindings.json (Linux/Mac)
• Location: %USERPROFILE%\.gesturev\standalone\gesture_bindings.json (Windows)
• Or check the settings UI within the application

Each gesture can be mapped to:

• URL opening
• Key presses
• Hotkey combinations
• Text typing
• Application launching

Privacy & Security

✅ All processing happens locally - No video or data leaves your computer ✅ No internet connection required for core functionality ✅ No accounts, no tracking, no data collection

About

GestureV is designed to make computer interaction more intuitive, accessible, and fun. Whether you're looking to reduce strain from traditional input methods, enhance accessibility, or just try a novel way to control your PC, gesture control offers an exciting alternative.

Built with MediaPipe, OpenCV, and Python for robust, real-time performance.

Tip: Spend 5-10 minutes practicing the basic gestures (open/closed fist, thumbs up/down, peace sign, wink) to build muscle memory before trying advanced combinations.