eyeshot

A lightweight MCP server that gives Claude Code eyes and hands on native desktop apps — the one thing browser-automation tools can't do.

Chrome-devtools MCP, Playwright MCP, and Browser MCP already own the web: they read the DOM and drive pages. But nothing lets an agent operate a native desktop application — Notepad, a Slack client, an installer dialog, a settings window, an IDE's chrome. eyeshot fills exactly that gap with a see → act loop:

• See — screenshot (with a labeled coordinate grid), scan_screen (OCR any window's text → coordinates)
• Act — click, click_ocr_text, type_text, press_key, drag, scroll, focus_window

It runs alongside the browser tools, not instead of them.

How to click things

click_ocr_text — when the target has visible text (buttons, links, labels). Finds the text via OCR and clicks it. Accent-insensitive, no coordinate math.

screenshot(coord_grid=True) + click(x, y) — for targets without text (icons, image regions, close buttons). Each grid dot is labeled with its absolute screen coordinate — read the nearest label and pass it straight to click().

get_cursor_pos — most reliable of all: ask the user to hover over the target, then read the coordinates and click them.

Multi-monitor & mixed-DPI

eyeshot is Per-Monitor-DPI-Aware v2 and virtual-desktop aware, so it captures and clicks correctly on any display — including extended monitors running a different scale (e.g. a 200% laptop panel next to a 125% external), even those positioned above/left of the primary (negative coordinates).

• screenshot() — primary monitor (default)
• screenshot(monitor=0) — the whole virtual desktop (all monitors in one image)
• screenshot(monitor=2) — a specific extended display
• Coordinate-grid labels stay in absolute screen coordinates on every monitor, so a label read off a second screen works directly with click().

Tools

Seeing

| Tool | Description | |------|-------------| | screenshot | Capture the desktop, a monitor, or a window by title; coord_grid=True overlays a labeled coordinate grid; monitor= selects a display | | scan_screen | OCR a window (or the topmost non-Claude window) and cache every text → coordinate mapping | | get_window_bounds | Get {x, y, width, height} of a window by partial title | | list_windows | List every real top-level window (title, bounds, monitor, FG/MIN flags); ghost/off-screen windows filtered — disambiguate a title | | focus_window | Bring a window to the foreground by title; reports its monitor; to_primary=True restores + moves + maximizes it onto the primary monitor | | get_cursor_pos | Return the current cursor position as x,y |

Acting — mouse

| Tool | Description | |------|-------------| | click | Click at absolute coords, or window-local coords with title= (brings the window to front first); returns a screenshot of the monitor/window clicked. SendInput — all monitors, Chrome-safe | | click_ocr_text | Find visible text via OCR and click it; dx/dy to click an icon offset from a text label — primary tool for text targets | | click_text | Invoke a native UI element by text via the accessibility tree — works even when the window is occluded/background, no foreground (not Chrome web content) | | drag | Click-and-drag between two coordinates | | move_mouse | Move the cursor without clicking (hover effects) | | scroll | Scroll a window by title and return a screenshot |

Acting — keyboard & clipboard

| Tool | Description | |------|-------------| | type_text | Type text via clipboard+paste (reliable Unicode, incl. Vietnamese/emoji) | | press_key | Press a key or combo (enter, ctrl+s, win+r, …) | | set_clipboard / get_clipboard | Set / read clipboard text | | paste_to | Focus a window by title and paste text into it |

Coordinate helpers

| Tool | Description | |------|-------------| | show_coord_grid / hide_coord_grid | Persistent always-on-top, click-through coordinate grid across the whole virtual desktop | | show_mouse_coords / cancel_mouse_coords | Floating live (x, y) label that follows the cursor |

Claude Code shows at most 22 MCP tools per server.

Note: eyeshot deliberately focuses on native desktop control. For browser automation, use a dedicated browser MCP (chrome-devtools, Playwright, Browser MCP) directly.

Installation

Requirements: uv (runs the Python server with its inline dependencies) and Node.js 18+ (for the eyeshot CLI).

npm install -g @misterhuydo/eyeshot-mcp
eyeshot install

eyeshot install registers the MCP server in ~/.claude.json. Restart Claude Code once afterward for the server to load.

Upgrade:

npm install -g @misterhuydo/eyeshot-mcp

The config carries over between versions — no need to re-run eyeshot install.

Local development

cd eyeshot
npm install
node bin/eyeshot.js install    # registers `uv run index.py`

Restart Claude Code to pick up code changes — the Python server loads index.py at startup.

Architecture

eyeshot MCP (index.py — Python, run via uv)
  ├── screenshot / scan_screen   → mss capture + Pillow grid + rapidocr OCR
  ├── click / drag / move_mouse   → Win32 SendInput (virtual-desktop, DPI-v2 aware)
  ├── type_text / press_key       → clipboard-paste + key injection
  └── focus_window / *_bounds     → EnumWindows + GetWindowRect

Browser MCPs (separate) → drive web content; eyeshot drives the native desktop

Tech stack

• Runtime: Python 3.11+ via uv (inline script dependencies)
• MCP SDK: mcp[fastmcp]
• Screenshot: mss + Pillow
• OCR: rapidocr-onnxruntime (pure Python, no Tesseract; model pre-warmed at startup)
• Win32 API: ctypes (SendInput, EnumWindows, per-monitor DPI, …)
• CLI: Node.js (ESM) — eyeshot install