@elizaos/plugin-computeruse-root
v2.0.0-alpha.1
Published
Computer automation plugin for ElizaOS - enables AI agents to control the local machine (when supported) or a remote ComputerUse MCP server
Downloads
47
Readme
@elizaos/plugin-computeruse
Gives an elizaOS agent the ability to control a computer UI:
- Local mode: uses
@elizaos/computerusebindings directly via native accessibility APIs - MCP mode: uses a configured ComputerUse MCP server (local or remote), so the agent can control a different machine
Safety note: computer control is powerful. Run with least privilege, and only enable in trusted environments.
Platform Support
| Platform | Local Mode | MCP Mode | API | |----------|:----------:|:--------:|-----| | Windows | ✅ | ✅ | UI Automation | | macOS | ✅ | ✅ | Accessibility API (AX) | | Linux | ✅ | ✅ | AT-SPI2 |
Requirements:
- Windows: Works out of the box
- macOS: Requires Accessibility permissions (System Preferences → Privacy & Security → Accessibility)
- Linux: Requires AT-SPI2 (default on GNOME/KDE),
wmctrlandxdotoolfor X11
Configuration
COMPUTERUSE_ENABLED(default:false)COMPUTERUSE_MODE(default:auto) —auto | local | mcpCOMPUTERUSE_MCP_SERVER(default:computeruse) — name of the MCP server in your runtime settings.
Actions
COMPUTERUSE_OPEN_APPLICATIONCOMPUTERUSE_CLICKCOMPUTERUSE_TYPECOMPUTERUSE_GET_WINDOW_TREECOMPUTERUSE_GET_APPLICATIONS
Selector scoping (MCP mode)
When using a ComputerUse MCP server, selector-based actions must be scoped to a running process.
You can do that either by:
- Passing
processalongsideselector, or - Prefixing the selector with
process:<name> >> ...(e.g.process:notepad >> role:Button|name:Save)
Example
See examples/computer-use/* for TypeScript / Python / Rust runnable examples.
