browser-pilot

Automation-first browser control over Chrome DevTools Protocol for AI agents. Version 0.1.0 supports Node.js, Bun, and Cloudflare Workers with zero production dependencies.

Companion package: Flightplan provides simple, reusable, cheap automation on top of browser-pilot. It will be released immediately after browser-pilot. Start with bunx flightplan --help for the higher-level interface.

Install

bun add browser-pilot
# or
npm install browser-pilot

CLI quick start

On Chrome 144+, enable remote debugging at chrome://inspect/#remote-debugging, then connect:

bp connect --name dev
bp exec -s dev '{"action":"goto","url":"https://example.com"}'
bp snapshot -i -s dev
bp exec -s dev '[{"action":"click","selector":"ref:e4"}]'

Use bp --help to route a task by job and bp --version to check the CLI build.

| Job | Commands | | --- | --- | | Inspect page state | snapshot, page, forms, text, targets, diagnose | | Review structured state | review, delta | | Act in the browser | exec, run | | Capture a human workflow | record | | Analyze behavior over time | trace | | Exercise voice and media | audio | | Change browser conditions | env |

For multiple local Chrome profiles, use --channel or --user-data-dir. Use bp connect --new-tab --page-url <url> to start from a fresh tab. New tabs stay in the background unless --foreground is passed.

Known issue

Native select can emit a synthetic untrusted change event before the final event. Apps that reject untrusted events may observe an intermediate rejected event; the final selection is correct.

Library quick start

Hosted browser:

import { connect } from 'browser-pilot';

const browser = await connect({
  provider: 'browser-use',
  apiKey: process.env.BROWSER_USE_API_KEY,
});

const page = await browser.page();
await page.batch([
  { action: 'goto', url: 'https://example.com/login' },
  { action: 'fill', selector: ['#email', 'input[type=email]'], value: '[email protected]' },
  { action: 'submit', selector: 'form' },
  { action: 'assertUrl', expect: '/dashboard' },
]);

await browser.close();

Local Chrome from TypeScript:

import { connect, getBrowserWebSocketUrl } from 'browser-pilot';

const browser = await connect({
  provider: 'generic',
  wsUrl: await getBrowserWebSocketUrl(),
});

The 0.1.0 interface

• Target-safe control: each page keeps its own CDP session. Target selection, popups, workers, and cross-origin iframes are explicit and isolated.
• Inspect before acting: snapshots produce reusable ref: selectors. page, forms, text, review, delta, resolveAll, and diagnose expose progressively richer reads.
• Verified actions: every action can use readiness waits, assertions, outcome conditions, failure hints, bounded retries, and target provenance.
• Reusable evidence: record and trace share one version 2 artifact model. audio and env cover voice pipelines and browser-state failure modes.

Record and replay

Capture a human workflow with bp record:

bp connect --name demo
bp record -s demo --profile automation -f ./artifacts/demo.recording.json
# perform the flow manually, then stop with Ctrl+C
bp record summary ./artifacts/demo.recording.json
bp record inspect ./artifacts/demo.recording.json
bp record derive ./artifacts/demo.recording.json -o ./artifacts/demo.workflow.json
bp run ./artifacts/demo.workflow.json -s demo

record captures an existing named session. Start with record summary or record inspect, not the raw artifact. record derive produces browser-pilot workflow JSON for bp run; Flightplan is the companion interface for simple reusable automation.

For proof while replaying known steps:

bp connect --name validation --record
bp exec -s validation -f ./artifacts/demo.workflow.json

Iframes and tabs

The action interface uses switchFrame and switchToMain:

[
  {"action":"switchFrame","selector":"iframe#payment"},
  {"action":"fill","selector":"#card-number","value":"4242424242424242"},
  {"action":"switchToMain"}
]

The library method is page.switchToFrame(selector). Cross-origin frames support click, fill, type, focus, press, shortcut, text, waitFor, and evaluate. Unsupported actions fail with a clear error. Cross-origin support requires Chrome site isolation (--site-per-process).

Providers

Use Browser Use when local Chrome is unavailable:

BROWSER_USE_API_KEY=bu_... bp connect --provider browser-use

See Providers for BrowserBase, Browserless, local Chrome, and direct CDP connections.

Documentation

• Getting Started
• CLI Guide
• Automation workflows
• Action recording
• Trace workflows
• Page API
• Browser API
• Types
• LLM contract