@assemblyline-agents/computer-use

Official Assembly Line connection plugin for controlling a paired Mac from a remotely hosted agent. It is intended for Railway, Fly, Docker, and other deployments that cannot run Peekaboo on the same host.

The Mac runs Assembly Line Computer Host (installed by Assembly Line Builder) and makes an outbound authenticated WebSocket connection to an Assembly Line-compatible relay. Agent requests and host responses are application-layer encrypted; the relay routes opaque envelopes and cannot read tool arguments or results.

Setup

1. Install Assembly Line Builder on a Mac running macOS 15 or newer.
2. Enable Computer Host, grant Accessibility and Screen Recording, and pair the intended deployment.
3. Store the generated binding as ASSEMBLY_LINE_COMPUTER_USE_BINDING on the agent host. Optionally set ASSEMBLY_LINE_COMPUTER_USE_RELAY_URL for a self-hosted relay.
4. Add the plugin:

assembly-line add computer-use agent

The selected plugin enables its reviewed read and write tools without an Assembly Line approval. Local Computer Host policy may still prompt or deny them. Gate individual actions in agent.md:

capabilities:
  computer-use:
    connections:
      computer-use:
        approval:
          click: always
          type: always
          hotkey: always

Use disable under the same connection to remove actions entirely. Actions not listed retain the reviewed plugin default. There is no generic connection access field in agent.md.

The binding carries a raw P-256 agent private scalar and the Mac's uncompressed X9.63 P-256 public key, encoded inside an opaque base64url JSON value. It must be treated as a secret and is never sent to the relay as part of a request.

The first release is accessibility-first. It excludes screenshots/vision, recording, shell, clipboard/paste, browser-specific tools, and nested agents. Use @assemblyline-agents/peekaboo instead when the Assembly Line runtime itself is running on the Mac being controlled.