🤖 ComputerMate

"Your AI's hands and eyes on any machine."
(The forbidden MCP of the AI era...)

ComputerMate is a powerful Model Context Protocol (MCP) server that grants AI models the ability to interact with your computer just like a human would. From taking screenshots to clicking buttons and typing text, ComputerMate exposes your local machine (or a sandboxed Docker container) as a set of tools for LLMs.

✨ Features & Highlights

• 🖥️ Cross-Platform: Unified support for Linux, macOS, and Windows via a single native mode.
• 🐳 Docker-First: Run in a fully isolated Linux desktop environment with VNC access and state persistence.
• 🖱️ Virtual Cursor: Enable a visual mouse cursor in Playwright by setting VIRTUAL_CURSOR=true.
• 🌐 Web Automation: Integrated Playwright support for high-performance, browser-only computer use.
• 📐 Smart Scaling: Automatically scale screenshots and coordinates to fit LLM context limits via MAX_SCALING_DIMENSION.

🛠️ Available Tools

ComputerMate exposes the following tools to the LLM:

| Tool | AI-Friendly Description | | :------------------ | :--------------------------------------------------------------------- | | screenshot | Take a full screenshot of the current screen or browser viewport. | | screenshot_region | Capture a specific rectangular area by providing two diagonal points. | | click | Move pointer and click (left, middle, right supported). | | double_click | Rapidly click twice at the given coordinates. | | scroll | Scroll the window content at (x, y) by given amount. | | type | Send keyboard text input to the active window. | | keypress | Send key combinations (e.g. ["ctrl", "c"], ["alt", "tab"]). | | move | Move the mouse pointer without clicking. | | drag | Drag the mouse from start point along a path of coordinates. | | wait | Pause execution for a set number of milliseconds. | | get_dimensions | Retrieve the screen or viewport width and height. | | get_environment | Returns the current platform (linux, macos, windows, browser). | | goto | (Playwright only) Navigate to a specific URL. | | back | (Playwright only) Go back in history. | | forward | (Playwright only) Go forward in history. | | get_current_url | (Playwright only) Retrieve the current active page URL. |

🚀 Quick Start

📦 Using npx (Local)

Run the server directly without installing:

# For local OS interaction (macOS, Windows, Linux)
npx @one710/computermate native

# For Playwright (Browser only)
npx @one710/computermate playwright

🐳 Using Docker (Safe & Persistent)

The recommended way to use ComputerMate is via Docker. This provides a sandboxed environment and VNC access.

# Clone the repository
git clone https://github.com/one710/computermate.git
cd computermate

# Start the server with persistence
docker-compose up --build

• MCP Endpoint: http://localhost:3000
• VNC View: localhost:5900 (Password: one710)

🛠️ Prerequisites & Installation

🐧 Linux (Ubuntu / Debian Desktop)

sudo apt-get update
sudo apt-get install -y libxtst-dev libpng-dev xvfb xdotool

🍎 macOS

Ensure you have granted Accessibility permissions to your terminal or IDE (e.g., Cursor, VS Code, iTerm2) in System Settings > Privacy & Security > Accessibility.

🪟 Windows

No external binaries are required!

⚙️ Configuration

Environment Variables

| Variable | Purpose | Default | | :---------------------- | :---------------------------------------------------------------------------------- | :------- | | COMPUTER_TYPE | (Docker / HTTP) Sets the computer backend: native or playwright. | native | | MAX_SCALING_DIMENSION | Caps the max width or height of screenshots (e.g., 1024x768). Scales coordinates. | None | | VIRTUAL_CURSOR | (Playwright) Shows a visual red dot where the "mouse" is. | false | | HEADLESS | (Playwright) Runs the browser in headless mode. | false |

🏗️ Local Development

To run from source:

# 1. Install dependencies
npm install

# 2. Build the project
npm run build

# 3. Run the server
node dist/index.js native

⚠️ Important Warnings

[!CAUTION] This tool grants significant control over your system. An LLM with access to ComputerMate can read your files, access your browser, and execute commands as the current user.

• Use with Caution: Do not run this on a machine with sensitive data or production access.
• Sandbox Recommended: Use the Docker environment for maximum safety.
• Monitor closely: Always keep an eye on what the AI is doing.

🛠️ MCP Configuration

Add this to your claude_desktop_config.json (or equivalent MCP client config):

{
  "mcpServers": {
    "computermate": {
      "command": "npx",
      "args": ["-y", "@one710/computermate", "native"],
      "env": {
        "MAX_SCALING_DIMENSION": "1024x768"
      }
    }
  }
}

🌉 Stdio-to-Docker Bridge (mcp-remote)

If your MCP client (like Cursor or older versions of Claude Desktop) only supports stdio transports, you can bridge it to the ComputerMate Docker container:

{
  "mcpServers": {
    "computermate": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote@latest",
        "http://localhost:3000/mcp",
        "--transport",
        "http-only"
      ]
    }
  }
}

This command starts a local stdio server that transparently forwards all requests to the ComputerMate container running at http://localhost:3000.

📄 License

This project is licensed under the MIT License. See the LICENSE file for details.