@openclaw/linux-desktop-control

Linux desktop automation client for OpenClaw. Provides accessibility tree snapshots, input simulation, and screen capture via a local Unix socket daemon.

Installation

npm install @openclaw/linux-desktop-control

Prerequisite: The linux-desktop-control daemon must be running on the Linux system. See daemon setup.

Usage

import { DesktopControlClient } from '@openclaw/linux-desktop-control';

const client = new DesktopControlClient();

// Check if daemon is running
if (await client.ping()) {
  // Get accessibility tree snapshot
  const snapshot = await client.snapshot({ depth: 3 });
  console.log(`Found ${snapshot.meta.app_count} apps`);

  // Type text
  await client.type('Hello, World!');

  // Press key combination
  await client.press('ctrl+shift+t');

  // Mouse control
  await client.mouseMove(500, 300);
  await client.mouseClick('left');
}

API

new DesktopControlClient(socketPath?)

Create a client. Uses /run/user/$UID/linux-desktop-control.sock by default.

Methods

| Method | Description | |--------|-------------| | ping() | Check if daemon is running | | status() | Get daemon status (pid, uptime, version) | | snapshot(options?) | Get accessibility tree (app, depth options) | | findElement(id) | Find element by ID | | click(element) | Click an element by reference | | type(text) | Type text via keyboard | | press(keys) | Press key combination (e.g., ctrl+c) | | mouseMove(x, y) | Move mouse to coordinates | | mouseClick(button?) | Click mouse button (left, right, middle) |

Types

interface Snapshot {
  timestamp: string;
  focused_app?: string;
  tree: DesktopElement;
  meta: {
    app_count: number;
    element_count: number;
    duration_ms: number;
  };
}

interface DesktopElement {
  id: string;
  role: string;
  name?: string;
  description?: string;
  bounds?: { x: number; y: number; width: number; height: number };
  states?: string[];
  actions?: string[];
  children?: DesktopElement[];
}

interface StatusInfo {
  pid: number;
  uptime_secs: number;
  connections: number;
  version: string;
}

Daemon Setup

The daemon requires:

• AT-SPI2 accessibility enabled (GTK_MODULES=gail:atk-bridge)
• ydotool for input simulation
• GNOME/Wayland session

See the daemon repository for full setup instructions.

License

MIT