@funly/macpilot

MCP (Model Context Protocol) server for MacPilot — gives AI agents mouse, keyboard, screenshot, and accessibility control of macOS desktops.

MacPilot is a local HTTP API that runs as a menu bar app on macOS. This MCP server wraps that API so any MCP-compatible AI agent (Claude, etc.) can control the desktop through standard MCP tool calls instead of raw HTTP.

Quick Start

Install

npm install -g @funly/macpilot

Configure in Claude Code / Claude Desktop

Add to your MCP config:

{
  "mcpServers": {
    "macpilot": {
      "command": "npx",
      "args": ["-y", "@funly/macpilot"],
      "env": {
        "MACPILOT_HOST": "127.0.0.1",
        "MACPILOT_PORT": "49901",
        "MACPILOT_API_KEY": ""
      }
    }
  }
}

Environment Variables

| Variable | Default | Description | |---|---|---| | MACPILOT_HOST | 127.0.0.1 | MacPilot server host | | MACPILOT_PORT | 49901 | MacPilot server port | | MACPILOT_API_KEY | (empty) | API key if MacPilot has auth enabled |

Available Tools

Session Management

• session_status — Check current status (available/active/user_control/user_finished/paused)
• session_start — Start a session (required before any actions)
• session_end — End session (always call when done)

Screenshots

• capture_window — Screenshot a window (returns image + dimensions)
• capture_region — Screenshot a screen region

Mouse

• mouse_click — Click using normalized coords (nx/ny 0.0–1.0)
• mouse_double_click — Double-click (select word)
• mouse_scroll — Scroll at position
• mouse_move — Move cursor
• mouse_drag — Drag between positions
• mouse_position — Get current cursor position

Keyboard

• click_and_type — Click field + type text (most reliable form filling)
• keyboard_type — Type text via clipboard paste
• keyboard_key — Press key with modifiers
• keyboard_shortcut — Named shortcuts (copy, paste, etc.)

Apps & Windows

• list_apps — All running apps with windowIds
• focus_app — Bring app to front
• launch_app — Launch an app
• focused_app — Get frontmost app

Accessibility

• accessibility_elements — List UI elements (buttons, fields, labels)

Batch & Debug

• batch — Multiple actions in one call
• diagnostics — Recent API call log

Workflow

1. session_start with your agent name
2. list_apps to find target window
3. capture_window to see the screen
4. Calculate nx = pixelX / imageWidth, ny = pixelY / imageHeight
5. mouse_click or click_and_type at those coordinates
6. capture_window again to verify
7. session_end when done

License

MIT