@relictombs/opentui-device

Run a real, headless Android Emulator inside an OpenTUI application. The emulator remains a native accelerated process; its authenticated loopback gRPC screenshot stream is projected into a Kitty-capable terminal. Touch is translated through the authenticated controller, while keyboard input uses bounded fixed-argument commands against that emulator's exact ADB serial.

headless Android Emulator
  -> authenticated loopback gRPC RGBA stream
  -> bounded latest-frame reconciliation
  -> owned top-down in-memory RGBA
  -> direct Kitty graphics data
  -> OpenTUI Device Surface

This is an Android-first early implementation. It intentionally does not add an Android backend to @relictombs/opentui-browser, silently download SDK components, or expose an unauthenticated gRPC port.

Current capabilities

• Own or borrow one Managed Device Runtime / Shared Device.
• Launch an existing AVD with -no-window and token-authenticated, dynamically allocated loopback gRPC.
• Wait for the emulator's controller status to report a completed boot.
• Stream high-density RGBA8888 frames while preserving the Emulator's top-down row order for Kitty.
• Bound frames to one being presented and the latest pending frame.
• Preserve phone aspect ratio with centered terminal letterboxing.
• Keep native Android touch coordinates independent from scaled capture and terminal geometry.
• Translate clicks, drags, releases, wheel gestures, typing, paste, Android Back, and Android Home.
• Attach one exact ADB emulator for bounded screenshots, semantic UI snapshots, and agent actions.
• Expose nine local MCP tools without an arbitrary shell escape hatch.
• Guarantee a pressure-zero touch release on pointer-out, hide, close, and renderer destruction.
• Gracefully request VM shutdown, then use bounded SIGTERM / SIGKILL escalation.

The first version is portrait-oriented. Rotation controls and rotation-aware touch transforms are deliberately deferred.

Requirements

• macOS on Apple Silicon for the provided setup target.
• Node.js 24 or newer. Bun is only needed when contributing from the Relictombs source checkout.
• @opentui/core 0.5.1.
• A local Kitty-graphics terminal. Ghostty is the primary target.
• OpenTUI's alternate-screen mode and a direct local renderer.
• Android SDK Emulator, platform tools, an arm64 system image, and an existing AVD.

Kitty always receives decoded pixels on the terminal host. Remote device execution uses Relictombs's separate encoded media transport instead of attempting to share a filesystem path.

The DeviceRenderable quick start below is the local library path: the AVD, Android SDK, OpenTUI, and Kitty terminal are on one machine, with no FFmpeg or network transport. For complete OpenCode setup, run npx --yes @relictombs/[email protected] and choose Android or both surfaces, then select local or remote execution. In the remote layout the client does not need the Android SDK; the wizard checks the media dependencies on both machines.

Install

npm install @relictombs/opentui-device @opentui/core

Kitty transport, Presentation coordination, terminal readiness, and MCP plumbing are private bundled implementation packages. They do not need to be installed or published separately.

Advanced standalone host setup

Most OpenCode users should use the interactive @relictombs/opencode wizard above. Library integrators can inspect the Android host without changing it:

npx --yes @relictombs/[email protected] doctor

Print the complete setup plan. This is also read-only:

npx --yes @relictombs/[email protected] setup

After reviewing the plan, explicitly install Homebrew OpenJDK, the Android command-line tools, platform tools, Emulator, API 36 Google APIs arm64 image, API 36 platform metadata, and create the OpenTUI_Pixel_8_API_36 Pixel 8 AVD:

npx --yes @relictombs/[email protected] setup --execute

The target system-image download is approximately 1.87 GB and the script requires at least 12 GiB free. It never uses --force and preserves an existing AVD with the target name. Homebrew must already be available, but OpenJDK does not need a separate prerequisite command.

These commands run on Node through the package executable, including when invoked through npx; Bun is not installed or required on the Android host. setup remains read-only until the explicit --execute flag is present. The Node-powered doctor and setup commands become available after @relictombs/[email protected] is published with this CLI.

Quick start

import { createCliRenderer } from "@opentui/core"
import { Effect } from "effect"
import { DeviceRenderable } from "@relictombs/opentui-device"

const renderer = await createCliRenderer({
  screenMode: "alternate-screen",
  exitOnCtrlC: true,
})

const device = new DeviceRenderable(renderer, {
  launch: { avdName: "OpenTUI_Pixel_8_API_36" },
  width: "100%",
  height: "100%",
  border: true,
})

renderer.root.add(device)
await Effect.runPromise(device.start())
device.focus()

Run the included example:

bun run example

Set OPENTUI_DEVICE_AVD to select another existing AVD.

Capture requests default to framePixelRatio: 2, which keeps the phone sharp in terminals whose reported pixel resolution is logical rather than Retina backing resolution. Width, height, and total pixels remain capped at 1080×1920 and about 2.1 megapixels. Set framePixelRatio: 1 on DeviceRenderable to trade image density for lower frame bandwidth.

Agents and MCP

The terminal user and an agent can inspect and operate the same running emulator. The Device Surface uses the authenticated gRPC pixel and touch path. Terminal typing, paste, and navigation use the same bounded exact-serial ADB adapter as agent automation because the API 36 Emulator accepts but ignores controller keyboard events. A semantic snapshot contains accessibility text, descriptions, resource IDs, relationships, bounds, and snapshot-scoped node IDs.

The local stdio MCP server exposes:

• status, snapshot, and screenshot
• tap, snapshot-scoped tap_node, and swipe
• type_text, press, and launch_app

There is no arbitrary ADB shell tool. Commands use fixed argv shapes, bounded time/output, and an exact emulator serial. tap_node re-reads the current accessibility hierarchy before acting; terminal input or another client changing the UI makes the old node reference fail as stale.

Build the package, start bun run example, then configure an MCP host to launch the built executable. For example:

{
  "mcpServers": {
    "android": {
      "command": "node",
      "args": ["/absolute/path/to/relictombs/packages/opentui/device/dist/agent-mcp.js"]
    }
  }
}

The server automatically selects the only running emulator. Set ANDROID_SERIAL=emulator-5554 in its environment when multiple emulators exist. bun run mcp is a source-checkout convenience command for hosts that can launch package scripts.

Runtime ownership

AndroidEmulator owns the emulator process and authenticated controller:

import { AndroidEmulator } from "@relictombs/opentui-device"
import { Effect } from "effect"

const program = Effect.gen(function* () {
  const runtime = yield* AndroidEmulator.launch({
    avdName: "OpenTUI_Pixel_8_API_36",
  })
  const status = yield* runtime.device.status()
  yield* Effect.logInfo("Android ready", status)
}).pipe(Effect.scoped)

await Effect.runPromise(program)

Ownership is explicit:

| Device Surface input | What the surface closes | | ---------------------- | ------------------------------------------- | | No source, or launch | Its Managed Device Runtime and Presentation | | runtime | Only its Presentation | | device | Only its Presentation |

Closing a surface never kills a borrowed runtime or Shared Device.

Effect services

The root package and curated @relictombs/opentui-device/effect entrypoint expose the same Effect-native operations and scoped services:

• ManagedDeviceRuntime owns one emulator process for one scope and exposes status, display, input, and snapshot operations as Effects.
• DeviceSurface owns renderer attachment, Presentation, and graphics cleanup while borrowing a supplied runtime; its handle exposes Effect-native settling and snapshots.
• AndroidAutomation attaches to one required emulator-<port> serial and maps every operation into a typed error channel. Mutating actions remain uninterruptible until the underlying bounded ADB command settles.

DeviceSurface.layer provides the shared @relictombs/kitty/effect service by default, so protocol decoding, terminal writes, and exact placement cleanup share one scoped Effect owner. Synchronous OpenTUI callbacks admit work into that same coordinator. Browser and device Presentation ownership is delegated to the shared @relictombs/presentation Effect core. Advanced hosts can inject a Kitty service through DeviceSurface.layerFromKitty.

import { Effect, Fiber } from "effect"
import { DeviceSurface, ManagedDeviceRuntime } from "@relictombs/opentui-device/effect"

const session = Effect.gen(function* () {
  const managed = yield* ManagedDeviceRuntime.Service
  const surfaces = yield* DeviceSurface.Service
  const handle = yield* surfaces.open(renderer, {
    runtime: managed.runtime,
    width: "100%",
    height: "100%",
  })
  const runtimeState = yield* managed.snapshot()
  const surfaceState = yield* handle.snapshot()
  yield* Effect.logInfo("device ready", { runtimeState, surfaceState })
  handle.renderable.focus()
  yield* Effect.never
}).pipe(
  Effect.provide(DeviceSurface.layer),
  Effect.provide(ManagedDeviceRuntime.layer({ avdName: "OpenTUI_Pixel_8_API_36" })),
  Effect.scoped,
)

const fiber = Effect.runFork(session)
// On host shutdown:
await Effect.runPromise(Fiber.interrupt(fiber))

Agent hosts can build AndroidAutomation.layer({ serial }) from managed.runtime.device.serial inside that same owning generation scope, so agent and human control cannot drift onto different emulators.

Verification

Run these contributor checks from packages/opentui/device in a repository checkout.

bun run check
bun run test:android
bun run test:agent
bun run test:packed

test:android is a live smoke test. It requires the SDK and target AVD, launches the built runtime, checks controller status, decodes one streamed RGBA frame through the Presentation source, and always closes the emulator.

test:agent launches the Managed Device Runtime, captures a PNG and semantic hierarchy through ADB, launches Settings, taps one snapshot-scoped node, restores Home, and closes every owned resource.

Protocol note

The Android Emulator controller API is currently marked experimental by AOSP. All protocol-specific field definitions, authentication, exact RGBA frame validation, and discovery decoding are isolated under src/android/ so future emulator changes have one adaptation seam.