agent-rover

A multi-platform TypeScript test driver for GUI applications

What Is This?

Few problems are as painful as automated testing for GUI applications on Windows and X11. agent-rover provides a driver interface for automated tests written in TypeScript in these environments.

WIP: X11 agent

flowchart LR
  subgraph TestProject["Test project (Node.js / TypeScript)"]
    TestCode["Test code, such as Vitest"]
    Driver["agent-rover API"]
    TestCode --> Driver
  end

  subgraph TargetSession["Target OS GUI session"]
    Agent["agent-rover-agent"]
    GuiSession["Windows / X11 GUI session"]
    Application["GUI application under test"]
    Agent -->|window enumeration / input / image capture / file operations| GuiSession
    GuiSession --> Application
  end

  Driver <-->|custom TCP protocol / authentication handshake| Agent

agent-rover provides small agent programs that can be used in Windows and X11 environments. By starting an agent on the target OS, you can remotely operate applications from a Node.js environment through a Jest-like testing interface. In other words, the operations that we humans perform against applications in a remote session over RDP can be done through an API.

Register the endpoint and its token in $HOME/.agent-rover first.

[default]
test-agent.example.com=<access-token>

With agent-rover, you can write test code like this:

import { writeFile } from 'node:fs/promises';

import { connectRemoteAgent } from 'agent-rover';

// Connect to the remote agent.
const agent = await connectRemoteAgent();

// Save a file in the remote environment.
await agent.files.writeFile(
  String.raw`C:\test.txt`,
  Buffer.from('test text file')
);

// Launch the application.
const process = await agent.processes.launchManaged({
  path: 'notepad.exe',
});

// Get the application window.
const notepadWindow = await process.waitForWindow({
  visible: true,
});

// Activate the application.
await notepadWindow.activate();

// Enter text by simulating keyboard typing.
await agent.keyboard.pasteText('Here is a remote message');

// Capture the window image and save it.
const screenshot = await notepadWindow.screenshot();
await writeFile('capture.png', screenshot.image);

await process.releaseAsync();

In addition to observing and operating GUI applications themselves, agent-rover also includes APIs for operating the target GUI session. As shown in the example above, you can send and receive files and launch applications.

Features

• Place the agent application on a remote machine and drive test operations remotely.
• Launch, discover, operate, and inspect the state of target GUI applications. File operations, such as sending and receiving files, are also supported.
• Capture the screen or an application window as a PNG image, or as an H.264 MP4 video on a supported Windows agent.
• Prebuilt agents are available for Windows (i686/amd64 on XP SP2 or later) and Linux X11 (i686/amd64/armv7l/arm64/riscv64).
• Agent communication uses a custom TCP protocol. Authentication uses a digest handshake, although the protocol messages themselves are not encrypted.
• Image recognition assertions for exact or close matches, and OCR-based assertions.

Preparation

Download the agent for the target platform from the releases page. The agent is very small and avoids runtime dependencies on other libraries as much as possible. After extracting the archive, you can run it as-is. No installation is required.

Start the Windows agent from your interactive desktop. It runs in the system tray.

C:\> agent-amd64.exe

Double-click the tray icon or choose “Show logs” to open the viewer. The version banner, token field and Copy button appear above the log list. Use Copy to register the token in $HOME/.agent-rover on the driver machine. When you supply a token with --unsafe-token, the token field and Copy button are hidden. Closing the viewer keeps the agent running; choose “Exit” from the tray to stop it.

Then install agent-rover in your npm project:

npm install -D agent-rover

You can use any test framework. For example, with Vitest, you can write the following:

import { describe, expect, it } from 'vitest';
import { connectRemoteAgent } from 'agent-rover';

describe('remote agent smoke test', () => {
  it('finds a running Notepad window', async () => {
    // Connect to the remote agent.
    const agent = await connectRemoteAgent();

    try {
      // Enumerate windows.
      const windows = await agent.windows();
      // Is notepad.exe running?
      const hasNotepad = windows.some((window) => {
        return (
          window.visible && window.process.name.toLowerCase() === 'notepad.exe'
        );
      });
      expect(hasNotepad).toBe(true);
    } finally {
      // Release the remote agent.
      agent.release();
    }
  });
});

Documents

For more information, please visit the repository.

License

Under MIT.