mooncat-computeruser

JavaScript desktop automation SDK backed by a small Rust process. The native process uses enigo for operating-system mouse and keyboard input; the JavaScript client communicates with it through newline-delimited JSON over stdin/stdout.

The 0.1.x MVP package targets Windows x64.

Build and test

npm run native:build
npm test
npm run test:e2e -- --dry-run
npm run test:e2e

The normal smoke test visibly moves the pointer along a small rectangle and restores its original position. It does not click unless --click is supplied.

JavaScript API

import { ComputerUserClient } from "mooncat-computeruser";

const computer = await ComputerUserClient.start();

try {
  await computer.mouse.moveTo(600, 400, { durationMs: 500 });
  await computer.mouse.click("left");
  await computer.mouse.scroll(3);
  await computer.keyboard.typeText("Hello from mooncat");
  await computer.keyboard.hotkey(["control", "a"]);
  await computer.keyboard.press("backspace");
  await computer.screen.capture({ path: "C:\\temp\\desktop.png" });
} finally {
  await computer.close();
}

This is OS-level injected input. It moves the visible pointer, but it is not USB HID hardware input and does not bypass operating-system security boundaries such as the Windows UAC secure desktop.

Windows foreground windows

The SDK exposes operating-system window facts without application-specific matching rules:

const windows = await computer.window.list();
const target = windows.find((window) => /notepad\.exe$/i.test(window.executablePath ?? ""));
if (!target) throw new Error("Notepad window not found");

const activated = await computer.window.activate({
  windowHandle: target.windowHandle,
  timeoutMs: 5_000,
});
await computer.window.maximize({ windowHandle: target.windowHandle });

window.list() reports top-level window handles, process IDs, titles, class names, executable paths when accessible, and visibility/minimized state. window.activate() accepts either a windowHandle or the legacy processId; handle-based activation is preferred when an application owns multiple processes or windows. Activation restores minimized/hidden windows and succeeds only after Windows reports the exact target handle as the foreground window. window.maximize() changes only the supplied top-level window and verifies that Windows reports it as maximized. screen.capture() writes a PNG for the primary display, or for an explicitly supplied pixel rectangle. Keyboard press, down, up, and hotkey methods inject normal operating-system input without application-specific shortcuts.

window.waitForAttention() waits for Windows to report that a target window or process requested taskbar attention, such as an application icon beginning to flash. It returns on the first matching event or when the timeout expires. A long-lived watcher should use a dedicated client because this call intentionally occupies its native process while waiting.

const event = await computer.window.waitForAttention({
  processId: target.processId,
  timeoutMs: 30_000,
});
if (event.attentionRequested) console.log("application requested attention", event);

OCR

mooncat-computeruser owns and ships its OCR runtime and pinned PP-OCRv5 mobile model assets. Consumers do not install OCR dependencies or download models at runtime.

const result = await computer.screen.recognizeTextOcr({
  x: 300,
  y: 35,
  width: 600,
  height: 45,
});
console.log(result.text, result.confidence);

Maintainers prepare and verify the model assets with npm run ocr:prepare and npm run ocr:check. Both model archives are integrity-pinned before publication.