browser-pilot

Automation-first browser control over Chrome DevTools Protocol for AI agents. Supports Node.js, Bun, and Cloudflare Workers with no mandatory production dependencies. Node authenticated WebSockets use the optional ws peer.

Companion package: Flightplan provides simple, reusable automation on top of browser-pilot. Start with bunx flightplan --help for the higher-level interface.

Install

bun add browser-pilot
# or
npm install browser-pilot

CLI quick start

On Chrome 144+, enable remote debugging at chrome://inspect/#remote-debugging, then connect:

bp connect --name dev
bp exec -s dev '{"action":"goto","url":"https://example.com"}'
bp snapshot -i -s dev
bp exec -s dev '[{"action":"click","selector":"ref:e4"}]'

Use bp --help to route a task by job and bp --version to check the CLI build.

For repository validation, run bun run check. It is quiet on success: each leg prints a short status block and a temporary log path; failures print the captured diagnostics.

| Job | Commands | | --- | --- | | Inspect page state | snapshot, page, forms, text, targets, diagnose | | Review structured state | review, delta | | Act in the browser | exec, run | | Capture a human workflow | record | | Analyze behavior over time | trace | | Inject protocol traffic | emit | | Exercise voice and media | audio | | Change browser conditions | env | | Discover or invoke page tools | webmcp status, webmcp list, webmcp call | | Authenticate behind Cloudflare Access | connect --cf-access, env auth | | Reuse a login across sessions | env auth save, env auth inspect, connect --auth |

For multiple local Chrome profiles, use --channel or --user-data-dir. Use bp connect --new-tab --page-url <url> to start from a fresh tab. New tabs stay in the background unless --foreground is passed.

The CLI keeps a browser-scoped daemon alive so subsequent commands reuse one remote-debugging connection (and one Chrome permission prompt). Use bp connect --no-daemon for an explicit direct connection; CI should set BROWSER_PILOT_NO_DAEMON=1 because detached daemons are not available there. Closing the final logical session keeps a local daemon available for reuse. Use bp daemon list and bp daemon stop --daemon-id <id> to inspect or stop it.

WebMCP is page-scoped and experimental. Use bp webmcp status to check the secure-context, origin-isolation, and tools Permissions Policy prerequisites, then bp webmcp list or bp webmcp call against the active session. Chrome's origin trial starts at version 149; local testing requires chrome://flags/#enable-webmcp-testing.

Known issue

Native select can emit a synthetic untrusted change event before the final event. Apps that reject untrusted events may observe an intermediate rejected event; the final selection is correct.

Library quick start

Cloudflare Chromium from Node (Node 18, 22 and 24):

npm install browser-pilot ws
# Set CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN in the trusted host.

import { connect } from 'browser-pilot';
const browser = await connect({ provider: 'cloudflare' });
try {
  const page = await browser.page();
  await page.goto('https://example.com');
  console.log(await page.title());
} finally {
  const cleanup = await browser.close();
  // cleanup_pending retains the allocation ID for an explicit retry.
}

cloudflare:chromium is explicit Chromium; cloudflare:kitesurf is opt-in and experimental. The latter is connection-bound: a lost owner returns SESSION_LOST, never a replacement browser. Imported Node use needs no Bun or daemon. The CLI keeps a daemon owner for named Cloudflare sessions. See the Workers guide and validation report for runtime and live gates.

Hosted browser:

import { connect } from 'browser-pilot';

const browser = await connect({
  provider: 'browser-use',
  apiKey: process.env.BROWSER_USE_API_KEY,
});

const page = await browser.page();
await page.batch([
  { action: 'goto', url: 'https://example.com/login' },
  { action: 'fill', selector: ['#email', 'input[type=email]'], value: '[email protected]' },
  { action: 'submit', selector: 'form' },
  { action: 'assertUrl', expect: '/dashboard' },
]);

await browser.close();

Local Chrome from TypeScript:

import { connect, getBrowserWebSocketUrl } from 'browser-pilot';

const browser = await connect({
  provider: 'generic',
  wsUrl: await getBrowserWebSocketUrl(),
});

Interface

• Target-safe control: each page keeps its own CDP session. Target selection, popups, workers, and cross-origin iframes are explicit and isolated.
• Inspect before acting: snapshots produce reusable ref: selectors. page, forms, text, review, delta, resolveAll, and diagnose expose progressively richer reads.
• Verified actions: every action can use readiness waits, assertions, outcome conditions, failure hints, bounded retries, and target provenance.
• Reusable evidence: record and trace share one version 2 artifact model. audio and env cover voice pipelines and browser-state failure modes.

Record and replay

Capture a human workflow with bp record:

bp connect --name demo
bp record -s demo --profile automation -f ./artifacts/demo.recording.json
# perform the flow manually, then stop with Ctrl+C
bp record summary ./artifacts/demo.recording.json
bp record inspect ./artifacts/demo.recording.json
bp record derive ./artifacts/demo.recording.json -o ./artifacts/demo.workflow.json
bp run ./artifacts/demo.workflow.json -s demo

record captures an existing named session. Start with record summary or record inspect, not the raw artifact. record derive produces browser-pilot workflow JSON for bp run; Flightplan is the companion interface for simple reusable automation.

For proof while replaying known steps:

bp connect --name validation --record
bp exec -s validation -f ./artifacts/demo.workflow.json

Drive the protocol with emit

When the UI cannot produce the message you need, send a frame on a WebSocket the page already owns with bp emit ws. It travels the app's real connection, so it carries real headers, cookies, and session tokens.

bp emit ws --list -s mysession
# OPEN       wss://worker-uat....workers.dev/session/...  [main]
bp emit ws '{"type":"client.response.text","content":"show me black shirts"}' \
  --match 'wss://worker-uat*' \
  --await-match '*search_results_surfaced*' --await-timeout 30000 -s mysession
# delivered: <socketUrl>; reply correlates to the injected turn, UI renders 12 products

Before injecting, capture one real client-to-server frame with bp trace/bp listen to confirm the message type the UI actually sends - a similarly named type can be a server-to-client echo that is silently ignored if you send it instead. Prefer --await on a specific field or a narrow --await-match glob over '*', which also matches heartbeat frames on chatty sockets.

Iframes and tabs

The action interface uses switchFrame and switchToMain:

[
  {"action":"switchFrame","selector":"iframe#payment"},
  {"action":"fill","selector":"#card-number","value":"4242424242424242"},
  {"action":"switchToMain"}
]

The library method is page.switchToFrame(selector). Cross-origin (out-of-process, OOPIF) frames support click, fill, type, focus, press, shortcut, text, waitFor, and evaluate. Unsupported actions (select, check, hover, scroll, snapshot, diagnose, etc.) fail with a clear error; call switchToMain() first. Cross-origin support requires Chrome site isolation (--site-per-process).

This covers common payment-field topologies (Adyen/Stripe-style secured/tokenized card fields):

• A cross-origin iframe directly on the page.
• A same-origin iframe nested INSIDE a cross-origin iframe (e.g. Stripe Elements' controller frame embedding a card-field frame).
• A same-origin wrapper iframe (e.g. a checkout modal container) ABOVE a cross-origin iframe.
• fill/type/focus tolerate a focusable input styled with opacity: 0 (a real bounding box is still required) - the technique secured card fields commonly use to keep native autofill/paste/IME behaviour on the real input while a styled proxy sits over it. click stays strict for these.

Not supported inside a cross-origin frame: snapshot, diagnose, select, check/uncheck, hover, scroll, forms, and similar read/interaction helpers - call switchToMain() first.

Authenticated targets (Cloudflare Access)

Connect straight through a Cloudflare-Access-protected origin using a service token, no manual login step:

bp connect --new-tab --page-url https://app.example.com --cf-access

This mints a CF_Authorization JWT out-of-band (cookie mode, the default) and applies it before the first navigation. From the library:

import { connect, mintCfAccessJwt } from 'browser-pilot';

const browser = await connect({ provider: 'generic' });
const page = await browser.page();
const { cookie } = await mintCfAccessJwt({
  url: 'https://app.example.com',
  clientId: process.env.CF_ACCESS_CLIENT_ID!,
  clientSecret: process.env.CF_ACCESS_CLIENT_SECRET!,
});
await page.setCookie(cookie);

For persisted auth that survives reattach/daemon restarts, use bp env auth set-cookie / set-headers (see CLI Guide). Full design and lifecycle semantics: CLI Guide § env.

Reuse a login (cookie snapshot auth)

Log in once by hand, save the cookies to a private file, and restore them into later sessions instead of repeating the login:

bp connect --name app-login --new-tab --foreground --page-url https://app.example.com
# ... log in manually ...
bp env auth save app -s app-login
bp connect --name app-work --auth app

BROWSER_PILOT_AUTH=<name-or-path> is an equivalent environment fallback for --auth, handy in CI. See the cookie snapshot auth guide for scope rules, file privacy, expiry/rotation, and the GitHub Actions pattern.

Package exports

| Export | Use | |---|---| | browser-pilot | Node/Bun library, including environment credentials and local browser discovery | | browser-pilot/core | Browser/Page APIs and providers with explicit connection options; no local discovery or environment fallback | | browser-pilot/browser, /cdp, /providers, /actions | Focused entries for the browser, CDP, provider, and action layers | | browser-pilot/adapters/node | Optional in-process session owner and filesystem artifact sink | | browser-pilot/adapters/memory | Test doubles and an in-memory artifact sink | | browser-pilot/shell | Shell-agnostic shell commands over the shared browser APIs | | browser-pilot/just-bash | Thin just-bash command binding over browser-pilot/shell | | browser-pilot/cli | Native bp command |

// Portable connection with explicit credentials or a pre-created session.
import { connectCore } from 'browser-pilot/core';

const browser = await connectCore({ provider: 'generic', wsUrl });
try {
  const page = await browser.page();
  await page.goto('https://example.com');
} finally {
  await browser.close();
}

Flightplan, the native CLI, and embedding applications use the same Browser, Page, and action receipt contracts. The shell adapter covers common operations; consumers can add commands using the public library. See Architecture for runtime boundaries and optional host adapters.

Providers

Use Browser Use when local Chrome is unavailable:

BROWSER_USE_API_KEY=bu_... bp connect --provider browser-use

See Providers for BrowserBase, Browserless, local Chrome, and direct CDP connections.

BrowserBase reads BROWSERBASE_API_KEY (and optional BROWSERBASE_PROJECT_ID, auto-resolved from the key when exactly one project exists) as an environment fallback for apiKey/projectId. Copy .env.example to .env to set these locally. The Node CLI loads .env without overriding existing variables; use --env-file <path> to select a file or BROWSER_PILOT_NO_DOTENV=1 to disable its loader. Bun has its own environment loader. See CLI configuration for supported dotenv syntax.

browser.close() returns a release status for hosted providers; retry on cleanup_pending (see Providers).

Documentation

• Getting Started
• CLI Guide
• Architecture
• Providers
• Shell / just-bash adapter
• Cloudflare Workers
• Automation workflows
• Action recording
• Trace workflows
• Page API
• Browser API
• Types
• Cookie snapshot auth
• LLM contract