agent-device-remote

Control iOS simulators and Android emulators on a Mac from anywhere -- your laptop, a Docker container, or an AI coding agent like Claude Code.

This tool lets you remotely tap buttons, type text, take screenshots, and read the screen of an iOS simulator or Android emulator running on a Mac. It's designed so that AI agents can build and test mobile apps without needing direct access to the Mac.

How it works

Your machine (laptop / Docker / CI)            Mac with Xcode
┌──────────────────────────────┐              ┌──────────────────────────────┐
│                              │              │                              │
│  Claude Code / Cursor        │              │  ┌────────────────────────┐  │
│       │ stdio                │              │  │  ad-proxy  (port 9124) │  │
│       ▼                      │              │  │                        │  │
│  ┌──────────────┐            │   HTTPS      │  │  ┌──────────────────┐ │  │
│  │ ad-remote-mcp│ ───────────│─────────────►│  │  │  agent-device    │ │  │
│  │  (MCP server)│            │  Cloudflare  │  │  │  daemon (Node.js)│ │  │
│  └──────────────┘            │  Tunnel      │  │  └────────┬─────────┘ │  │
│                              │              │  │           │            │  │
│   ── or ──                   │              │  │  ┌────────▼─────────┐ │  │
│                              │              │  │  │  xcrun simctl /  │ │  │
│  ┌──────────────┐            │              │  │  │  XCTest          │ │  │
│  │ ad-remote    │ ───────────│─────────────►│  │  └────────┬─────────┘ │  │
│  │  (CLI)       │            │              │  │           │            │  │
│  └──────────────┘            │              │  └───────────┼────────────┘  │
│                              │              │              │               │
└──────────────────────────────┘              │  ┌───────────▼────────────┐  │
                                              │  │                        │  │
                                              │  │    iOS Simulator       │  │
                                              │  │    ┌────────────────┐  │  │
                                              │  │    │  📱 Your App   │  │  │
                                              │  │    │                │  │  │
                                              │  │    │  [Button] [Txt]│  │  │
                                              │  │    └────────────────┘  │  │
                                              │  └────────────────────────┘  │
                                              └──────────────────────────────┘

What runs where:

| Component | Runs on | What it does | |-----------|---------|-------------| | ad-proxy | Mac | Receives commands over HTTPS, talks to the simulator | | agent-device | Mac | Node.js daemon that actually controls the simulator (XCTest, simctl) | | ad-remote-mcp | Your machine | MCP server -- AI agents see simulator tools natively | | ad-remote | Your machine | CLI -- for scripts, CI, or manual use | | Cloudflare Tunnel | Both | Encrypted connection between your machine and the Mac over the internet |

How a command flows (e.g. "tap the Login button"):

1. Claude Code calls the `press` tool with target "@e5"
       │
2.     ▼  ad-remote-mcp sends POST /api/command to proxy
       │
3.     ▼  ad-proxy validates auth token + lease, checks command allowlist
       │
4.     ▼  ad-proxy invokes: npx agent-device press @e5 --state-dir /tmp/ad-lease-{id}/ --json
       │
5.     ▼  agent-device uses XCTest to tap the element on the simulator
       │
6.     ▼  result flows back through the chain to Claude

Key concepts

• Lease -- A reservation for a simulator. You allocate one before doing anything, and release it when done. Each lease gets its own isolated agent-device instance so multiple agents can use different simulators simultaneously without interference.
• Device isolation -- Each lease creates a separate --state-dir, which spawns its own agent-device daemon. The first lease picks the first booted simulator, the second lease picks the next one, etc. No two leases share a device.
• Snapshot -- A text representation of everything on screen (buttons, labels, text fields) with IDs like @e5. You read the snapshot to know what to tap.
• Screenshot -- An actual image (PNG) of the simulator screen.
• MCP -- Model Context Protocol. A standard that lets AI agents (Claude Code, Cursor, etc.) discover and use tools. The MCP server exposes simulator controls as native tools the agent can call.

Prerequisites

Install these before starting:

On the Mac (Step 1 machine)

| Requirement | How to install | Check | |-------------|---------------|-------| | macOS | -- | uname should say Darwin | | Node.js (18+) | nodejs.org or brew install node | node --version | | For iOS: Xcode | Mac App Store or xcode-select --install | xcodebuild -version | | For iOS: Simulators | Xcode > Settings > Platforms > iOS | xcrun simctl list devices | | For Android: Android Studio | developer.android.com | adb version | | For Android: Emulators | Android Studio > Device Manager > Create Device | emulator -list-avds |

On the client machine (Step 2 machine)

| Requirement | How to install | Check | |-------------|---------------|-------| | Node.js (18+) | nodejs.org | node --version |

Rust is not needed on the client -- the npm package includes pre-built binaries. If both machines are the same Mac, you only need the Mac prerequisites.

Step 1: Set up the Mac (the machine with simulators)

1.1 Install

npm install -g agent-device-remote

That's it. One command installs everything -- the proxy, CLI, MCP server, and agent-device (auto-included as a dependency).

Optionally install cloudflared for tunnel access:

brew install cloudflared

git clone https://github.com/KarthickSelvam/agent-device-remote.git
cd agent-device-remote
./setup.sh

1.2 Boot a device

iOS simulators:

xcrun simctl list devices available
xcrun simctl boot "iPhone 16"

If no simulators listed, open Xcode > Settings > Platforms and download an iOS runtime.

Android emulators:

# Make sure Android SDK tools are in PATH
export ANDROID_HOME=~/Library/Android/sdk
export PATH="$ANDROID_HOME/emulator:$ANDROID_HOME/platform-tools:$PATH"

emulator -list-avds
emulator -avd Pixel_9_Pro_XL -no-window &

If no AVDs listed, open Android Studio > Device Manager > Create Device.

Multi-agent: boot multiple devices -- each agent gets its own:

xcrun simctl boot "iPhone 16"
xcrun simctl boot "iPad Pro 13-inch (M5)"
emulator -avd Pixel_9_Pro_XL -no-window &

1.3 Start the proxy

ad-proxy

You'll see:

=== ad-proxy ===
  listen: 127.0.0.1:9124
  tunnel: https://random-words.trycloudflare.com
  token:  a7b3...e5f1

  Full token written to stderr (visible with RUST_LOG=info).

You need two values from this output:

• Tunnel URL -- the https://...trycloudflare.com address
• Auth token -- run with RUST_LOG=info to see the full token in stderr, or pass your own with --token

# To see the full token:
RUST_LOG=info ./target/release/ad-proxy 2>&1 | grep "auth token"

The proxy binds to localhost (127.0.0.1) by default. Traffic reaches it through the Cloudflare tunnel. If you need LAN access, use --bind 0.0.0.0.

Keep this terminal running. The proxy must stay active.

Step 2: Set up the client (your dev machine or Docker agent)

This is where you (or your AI agent) will control the simulator from.

Option A: MCP Server (for Claude Code, Cursor, etc.)

This is the recommended approach for AI agents. The agent will see simulator tools (tap, type, screenshot, etc.) as native capabilities.

2A.1 Install:

npm install -g agent-device-remote

That's it. No Rust, no cloning, no building. The package includes pre-built binaries for macOS and Linux.

git clone https://github.com/KarthickSelvam/agent-device-remote.git
cd agent-device-remote
cargo build --release
# Binary at: ./target/release/ad-remote-mcp

2A.2 Add MCP config for Claude Code.

Create .mcp.json in your project folder (or ~/.claude.json for global):

{
  "mcpServers": {
    "agent-device-remote": {
      "command": "ad-remote-mcp",
      "env": {
        "AD_REMOTE_URL": "https://random-words.trycloudflare.com",
        "AD_REMOTE_TOKEN": "paste-your-full-token-here",
        "AD_REMOTE_LEASE": "auto"
      }
    }
  }
}

Replace:

• AD_REMOTE_URL -- with the tunnel URL from Step 1.3
• AD_REMOTE_TOKEN -- with the full auth token from Step 1.3

If you installed via npm, just use "command": "ad-remote-mcp" (it's in PATH). If you built from source, use the full path: "command": "/path/to/target/release/ad-remote-mcp".

2A.3 Restart Claude Code. You should now see tools prefixed with agent-device-remote__ when Claude lists its available tools.

Available tools:

| Tool | What it does | |------|-------------| | Lease | | | lease_allocate | Reserve a simulator | | lease_release | Release when done | | lease_list | See your reservations | | Device info | | | devices | List simulators | | boot | Boot a device/simulator | | ensure_simulator | Create/boot a simulator | | apps | List installed apps | | appstate | Current app state | | session_list | Active daemon sessions | | App lifecycle | | | open_app | Launch app (with relaunch option) | | close_app | Close app | | upload_and_install | Upload local .app/.ipa/.apk to Mac and install | | install_app | Install from Mac-local path | | reinstall_app | Fresh reinstall | | Navigation | | | home | Home screen | | back | Back button | | app_switcher | Recent apps | | Taps | | | press | Tap element | | long_press | Tap and hold | | focus | Move focus | | Text | | | fill | Clear + type into field | | type_text | Type with keyboard | | Gestures | | | swipe | Swipe direction | | scroll | Scroll direction | | scroll_into_view | Scroll until visible | | pinch | Pinch in/out | | Inspection | | | snapshot | UI element tree with IDs | | diff_snapshot | Changes since last snapshot | | find | Search UI by text | | get | Get element property | | is_element | Check element state | | screenshot | Screen image (inline PNG) | | Waiting | | | wait | Wait for element/condition | | alert | Handle system dialog | | Clipboard | | | clipboard_read | Read clipboard | | clipboard_write | Write to clipboard | | Settings | | | settings | wifi/appearance/faceid/permissions | | Logs | | | logs_path / logs_start / logs_stop / logs_clear / logs_mark / logs_doctor | App log management | | Network | | | network_dump / network_log | Captured HTTP requests / live stream | | Performance | | | perf | Metrics (startup timing, memory) | | trace_start / trace_stop | Performance traces | | Files / Events | | | push_file | Push file to device | | trigger_app_event | Custom app events | | batch | Run batch commands | | replay | Replay recorded sessions (.ad) |

Typical workflow -- testing a built app:

1. lease_allocate -- get a simulator
2. upload_and_install -- send your .app/.ipa from local machine to the Mac and install it
3. open_app -- launch the installed app
4. snapshot -- read what's on screen (gets element IDs like @e3, @e5)
5. press / fill / type_text -- interact with elements
6. screenshot -- visually verify the result
7. lease_release -- free the simulator

Typical workflow -- testing an existing app (e.g. Settings):

1. lease_allocate -- get a simulator
2. open_app -- launch the app (e.g. com.apple.Preferences)
3. snapshot / press / screenshot -- interact and verify
4. lease_release -- free the simulator

Option B: CLI (for scripts, CI, or manual testing)

2B.1 Install and set connection details:

npm install -g agent-device-remote

export AD_REMOTE_URL=https://random-words.trycloudflare.com
export AD_REMOTE_TOKEN=paste-your-full-token-here

2B.2 Use the CLI:

# Allocate a simulator
ad-remote lease allocate
# Output will show a lease_id -- save it:
export AD_REMOTE_LEASE=<paste-lease-id-here>

# List simulators
ad-remote devices

# Upload and install your app (transfers file to Mac + installs)
ad-remote upload ./build/MyApp.app

# Open your app (or a system app)
ad-remote open com.example.myapp

# See what's on screen (text tree of UI elements)
ad-remote snapshot

# Tap a button (using element ID from snapshot)
ad-remote press "@e5"

# Take a screenshot
ad-remote screenshot --out screen.png

# Type some text
ad-remote type "hello world"

# Swipe up
ad-remote swipe up

# Release when done
ad-remote lease release

Troubleshooting

"daemon unreachable" / proxy won't start

The proxy checks that agent-device CLI is available at startup. If it fails:

# Check if agent-device is installed:
npx agent-device --version

# If not installed:
npm install -g agent-device

# Test it works:
npx agent-device devices --json

"No simulators found" / devices returns empty

# List all available simulators:
xcrun simctl list devices available

# If empty, install a runtime:
# Xcode > Settings > Platforms > + > iOS 18

# Boot one:
xcrun simctl boot "iPhone 16"

"unauthorized" errors

• Make sure the token in your client config matches the one from ad-proxy
• Run the proxy with RUST_LOG=info to see the full token in the logs
• Check there are no extra spaces or newlines in your token

Tunnel not connecting

# Check if cloudflared is installed:
cloudflared --version

# For LAN-only use (no tunnel):
./target/release/ad-proxy --no-tunnel --bind 0.0.0.0

# Then use the Mac's IP directly:
export AD_REMOTE_URL=http://192.168.1.100:9124

MCP server not showing up in Claude Code

• Make sure the path in .mcp.json is absolute (starts with /)
• Check the binary exists: ls -la /your/path/to/ad-remote-mcp
• Restart Claude Code after editing .mcp.json
• Check stderr for errors: run ad-remote-mcp manually to see if it starts

Configuration Reference

Proxy (ad-proxy)

| Flag | Env Var | Default | Description | |------|---------|---------|-------------| | --port | AD_PROXY_PORT | 9124 | Listen port | | --bind | AD_PROXY_BIND | 127.0.0.1 | Bind address (0.0.0.0 for all interfaces) | | --token | AD_PROXY_TOKEN | auto-generated | Bearer auth token | | --no-tunnel | AD_PROXY_NO_TUNNEL | false | Disable Cloudflare tunnel | | --tunnel-mode | AD_PROXY_TUNNEL_MODE | quick | quick (anonymous) or named (account) | | --tunnel-name | AD_PROXY_TUNNEL_NAME | -- | Tunnel name (for named mode) | | --tunnel-token | AD_PROXY_TUNNEL_TOKEN | -- | Connector token (for named mode) | | --tunnel-credentials | AD_PROXY_TUNNEL_CREDENTIALS | -- | Credentials JSON file path | | --tunnel-hostname | AD_PROXY_TUNNEL_HOSTNAME | -- | Custom hostname (e.g. sim.example.com) | | --max-leases | AD_PROXY_MAX_LEASES | 4 | Max concurrent leases (simulators in use) | | --max-leases-per-agent | AD_PROXY_MAX_LEASES_PER_AGENT | 2 | Max leases per agent | | --max-lease-lifetime | AD_PROXY_MAX_LEASE_LIFETIME | 7200 | Absolute max lease lifetime (seconds) | | --lease-duration-secs | AD_PROXY_LEASE_DURATION | 1800 | Lease duration before expiry (seconds) | | --heartbeat-grace-secs | AD_PROXY_HEARTBEAT_GRACE | 90 | Grace period after missed heartbeat | | --max-upload-bytes | AD_PROXY_MAX_UPLOAD_BYTES | 524288000 | Max upload file size (bytes, default 500MB) |

Client (ad-remote / ad-remote-mcp)

| Env Var | Description | |---------|-------------| | AD_REMOTE_URL | Proxy URL (from Step 1.3) | | AD_REMOTE_TOKEN | Auth token (from Step 1.3) | | AD_REMOTE_LEASE | Lease ID, or auto to allocate automatically |

Multi-Agent / Multi-Device

Multiple agents can use the proxy simultaneously, each on their own simulator:

Agent A (lease-1) ──► ad-proxy ──► agent-device --state-dir /tmp/ad-lease-1/ ──► iPhone 17 Pro
Agent B (lease-2) ──► ad-proxy ──► agent-device --state-dir /tmp/ad-lease-2/ ──► iPad Pro
Agent C (lease-3) ──► ad-proxy ──► agent-device --state-dir /tmp/ad-lease-3/ ──► iPhone 16e

How it works:

• Each lease creates an isolated state directory (/tmp/ad-lease-{uuid}/)
• Each state directory spawns its own agent-device daemon instance
• Each daemon instance automatically picks the next available booted simulator
• No two leases share a device -- enforced by agent-device's DEVICE_IN_USE locking

Setup for multi-agent:

1. Boot as many simulators as you need concurrent agents:

   xcrun simctl boot "iPhone 17 Pro"
   xcrun simctl boot "iPad Pro 13-inch (M5)"
   xcrun simctl boot "iPhone 16e"

2. Set --max-leases to match (default 4):

   ad-proxy --max-leases 3

3. Each agent allocates its own lease and gets a dedicated simulator automatically.

Isolation guarantees:

• Agents can only see/manage their own leases (X-Agent-Id enforcement)
• Each lease's commands run in an isolated daemon with its own device
• State directories are cleaned up on lease release or expiry

Cloudflare Tunnel Setup

By default, the proxy creates a quick tunnel -- a temporary, anonymous URL that changes on every restart. This is fine for development.

For production (stable URL, always-on Mac minis), use a named tunnel.

Quick tunnel (default)

ad-proxy
# URL: https://random-words.trycloudflare.com (changes each restart)

No Cloudflare account needed.

Named tunnel with token (recommended for production)

One-time setup:

1. Create a free Cloudflare account
2. Go to Zero Trust > Networks > Tunnels > Create a tunnel
3. Name it (e.g. mac-sim-01)
4. Under Install connector, copy the token
5. Add a Public Hostname (e.g. sim.example.com -> http://localhost:9124)

Run:

ad-proxy \
  --tunnel-mode named \
  --tunnel-token eyJhIjoiNGY... \
  --tunnel-hostname sim.example.com

The URL https://sim.example.com is now stable across restarts.

Named tunnel with credentials file

# One-time setup
cloudflared tunnel login
cloudflared tunnel create mac-sim-01
cloudflared tunnel route dns mac-sim-01 sim.example.com

# Run
ad-proxy \
  --tunnel-mode named \
  --tunnel-name mac-sim-01 \
  --tunnel-hostname sim.example.com

No tunnel (LAN / VPN)

ad-proxy --no-tunnel --bind 0.0.0.0
# Agents connect to http://<mac-ip>:9124

Architecture

Four Rust crates in one workspace:

| Crate | Binary | Runs on | What it does | |-------|--------|---------|-------------| | ad-remote-protocol | -- | -- | Shared types (commands, leases, errors) | | ad-proxy | ad-proxy | Mac | Proxy server with auth, leases, tunnel | | ad-remote | ad-remote | Client | CLI for scripts and manual use | | ad-remote-mcp | ad-remote-mcp | Client | MCP server for AI agents |

API Reference

All /api/* endpoints require Authorization: Bearer <token> header. Device commands also require X-Lease-Id and X-Agent-Id headers.

GET  /health                         No auth required. Returns proxy/daemon status.

POST /api/lease                      Allocate a lease.
GET  /api/lease                      List your leases (filtered by X-Agent-Id).
POST /api/lease/{id}/heartbeat       Extend a lease (requires X-Agent-Id).
DELETE /api/lease/{id}               Release a lease (requires X-Agent-Id).

POST /api/command                    Send a command (requires X-Lease-Id).
     Body: {"command": "press", "positionals": ["@e5"], "flags": {}}
GET  /api/devices                    List available simulators.
POST /api/screenshot                 Take screenshot (returns image/png).
POST /api/upload                     Upload and install an app (multipart).

Allowed commands: boot, open, close, install, reinstall, launch, terminate, home, back, app-switcher, press, click, tap, long-press, focus, type, fill, swipe, scroll, scrollintoview, pinch, snapshot, diff, find, get, is, screenshot, wait, alert, batch, devices, appstate, apps, session, ensure-simulator, clipboard, settings, logs, network, perf, metrics, trace, push, trigger-app-event, keyboard, replay. All other commands are rejected.

License

MIT -- see LICENSE.