TUI Agent

Deterministic eyes and hands for OpenTUI applications. Renders a real application into OpenTUI's memory terminal, drives keyboard, paste, mouse, and resize events, and saves text, styled spans, SVG frames, focus state, layout diagnostics, and adapter-owned semantic state after every action.

It is an installable npm package and a Bun CLI. The harness is application-neutral; a small trusted adapter mounts your TUI and optionally exposes routes, dialogs, backend requests, or other state useful for assertions.

Requirements and installation

• Bun 1.2 or newer
• OpenTUI 0.3.4 or newer
• macOS, Linux, or Windows supported by OpenTUI's native packages

bun add --dev tui-agent @opentui/core
bunx tui-agent doctor

The tarball follows normal npm package conventions and can also be installed with npm install --save-dev. The executable requires bun because OpenTUI's test renderer and TypeScript adapter loading use Bun at runtime.

Quick start

Create test/tui/adapter.tsx:

/** @jsxImportSource @opentui/solid */
import { render } from "@opentui/solid"
import type { TuiAgentAdapter } from "tui-agent"

const adapter: TuiAgentAdapter = {
  name: "my-tui",
  async launch({ setup }) {
    await render(() => <App />, setup.renderer)
    return {
      inspect: () => ({ route: "home", state: { ready: true } }),
      dispose() {},
    }
  },
}

export default adapter

Create test/tui/smoke.json:

{
  "$schema": "node_modules/tui-agent/scenario.schema.json",
  "name": "smoke",
  "adapter": "./adapter.tsx",
  "terminal": { "width": 100, "height": 30 },
  "steps": [
    { "action": "waitForText", "text": "Describe the task" },
    { "action": "type", "text": "hello" },
    { "action": "assert", "visible": ["hello"] }
  ]
}

Run it:

bunx tui-agent run test/tui/smoke.json
bunx tui-agent run test/tui/smoke.json --json --artifacts /tmp/tui-smoke

CLI

tui-agent list [--json]
tui-agent run <scenario> [--adapter <module>] [--artifacts <dir>] [--keep-fixture] [--json]
tui-agent interact <scenario> [--adapter <module>] [--artifacts <dir>] [--keep-fixture]
tui-agent show <run-directory> [--json]
tui-agent doctor [--json]

Every command supports --help; failures are written to stderr and return a nonzero exit code.

Scenario model

Scenarios are plain JSON validated against the schema. Available actions:

| Action | Purpose | | --- | --- | | waitForText, waitForAbsent, waitForFocus, waitForIdle | Wait for visual, focus, or idle state | | key, type, paste | Keyboard input, modifiers, repetition, and bracketed paste | | click, doubleClick, clickText, move, drag, scroll | Mouse input | | resize | Trigger real terminal reflow | | wait | Pause | | capture | Name an evidence frame | | find | Search current frame + scrollback buffer for a regex pattern | | scrollback | Scroll the in-memory scrollback buffer | | assert | Check text, cursor, focus, request evidence, route, dialog, state, and highlights |

Each step captures: .txt (text), .svg (styled frame), .spans.json (spans), .tree.json (renderable hierarchy), .state.json (renderer metrics + adapter inspection).

Highlights

Every snapshot includes a highlights field listing background-highlighted spans on screen — the standard way TUI programs indicate selected items, active tabs, or focused buttons. Agents can assert on highlights directly:

{ "action": "assert", "highlights": [{ "text": "Settings" }] }

Adapter API

type TuiAgentAdapter = {
  name: string
  prepare?: (context: AdapterPrepareContext) => AdapterPreparation | Promise<AdapterPreparation>
  launch: (context: AdapterLaunchContext) => RunningTui | Promise<RunningTui>
}

• prepare() — optional setup before scenario runs. Create fixture files, set environment.
• launch() — mount the real TUI on context.setup.renderer. Return dispose + optional inspect.

The inspect() method returns semantic state (route, dialog, mode) that powers declarative assertions without parsing text.

Examples

This repo includes a Spinosa TUI adapter at adapters/spinosa.ts and several scenarios in scenarios/. These demonstrate fixture preparation, renderer factory interception, fake SDK transport, request capture, Effect cleanup, and JSON-safe semantic inspection:

cd packages/tui
bun tools/tui-agent/cli.ts run workspace-session
bun tools/tui-agent/cli.ts interact workspace-session

License

MIT