pi-browser-harness

Full browser control for pi agents. Navigate, click, type, screenshot, and extract data from your real Chrome — all driven through natural language via CDP.

Quick Start

# 1. Install
pi install npm:pi-browser-harness

# 2. Enable Chrome remote debugging
#    Open chrome://inspect/#remote-debugging in Chrome,
#    tick "Discover network targets", click Allow.
#    Or launch Chrome with --remote-debugging-port=9222

# 3. Connect
/browser-setup

Requirements

• pi (latest)
• Node.js ≥ 22
• Chrome / Chromium / Edge

Tools

| Tool | Purpose | |------|---------| | browser_screenshot | Capture a PNG screenshot of the current page | | browser_click | Click at viewport coordinates | | browser_type | Type text into the focused element | | browser_press_key | Press a key (Enter, Tab, Escape, arrows, etc.) with optional modifiers | | browser_scroll | Scroll the page by delta pixels | | browser_navigate | Navigate to a URL | | browser_new_tab | Open a new tab, optionally navigate | | browser_open_urls | Open multiple URLs in parallel tabs | | browser_go_back / browser_go_forward / browser_reload | History navigation | | browser_page_info | Get URL, title, viewport, scroll position, or dialog state | | browser_list_tabs / browser_current_tab / browser_switch_tab | Tab discovery and switching | | browser_execute_js | Run JavaScript in the page and return the result | | browser_http_get | Direct HTTP GET outside the browser — much faster for APIs | | browser_wait / browser_wait_for_load | Sleep, or wait for readyState === 'complete' | | browser_handle_dialog | Accept or dismiss alert / confirm / prompt | | browser_run_script | Execute a temporary script with daemon and Node.js access |

Core Patterns

The golden rule: screenshot → act → screenshot → verify. Never assume — always look.

Navigation

browser_new_tab({ url: "https://example.com" })
browser_wait_for_load()
browser_screenshot()

Form filling

browser_screenshot()                   # see the form
browser_click({ x: 400, y: 200 })      # click the search input
browser_type({ text: "query" })        # type
browser_press_key({ key: "Enter" })    # submit
browser_wait_for_load()
browser_screenshot()                   # verify

Data extraction

browser_execute_js({ expression: "document.querySelector('.price').innerText" })

browser_http_get({ url: "https://api.github.com/repos/amankumarsingh77/pi-browser-harness" })

// Structured arrays
browser_execute_js({ expression: `Array.from(document.querySelectorAll('.result')).map(el => ({
  title: el.querySelector('h3')?.textContent,
  link:  el.querySelector('a')?.href,
}))` })

Research workflow

browser_navigate({ url: "https://google.com/search?q=..." })
// read search results
browser_open_urls({ urls: ["url1", "url2", "url3"] })
browser_list_tabs()
browser_switch_tab({ targetId: "..." })
browser_screenshot()
browser_execute_js({ expression: "document.querySelector('.content').innerText" })

Scrolling

browser_screenshot()
browser_scroll({ deltaY: -500 })
browser_screenshot()

Keyboard modifiers

| Key | Bit | |-----|-----| | Alt | 1 | | Ctrl | 2 | | Meta / Cmd | 4 | | Shift | 8 |

Combine with bitwise OR:

browser_press_key({ key: "c", modifiers: 2 })    // Ctrl+C
browser_press_key({ key: "v", modifiers: 4 })    // Cmd+V
browser_press_key({ key: "T", modifiers: 10 })   // Ctrl+Shift+T

SPAs (React, Vue, etc.)

browser_wait_for_load only checks readyState. Poll for elements after:

browser_execute_js({ expression: "!!document.querySelector('.loaded-content')" })

Dialogs

JS dialogs (alert / confirm / prompt) freeze the page. Check browser_page_info first — if it reports a dialog, handle it before anything else:

browser_handle_dialog({ action: "accept" })       // confirm
browser_handle_dialog({ action: "dismiss" })       // cancel
browser_handle_dialog({ action: "accept", promptText: "hello" })  // prompt

Temporary Scripts

When built-in tools aren't enough, write a script to disk and run it:

// 1. Write the script
write("/tmp/scrape-pages.js", `
  const results = [];
  for (const url of params.urls) {
    await daemon.cdp("Page.navigate", { url });
    await new Promise(r => setTimeout(r, 2000));
    const title = await daemon.evaluateJS("document.title");
    results.push({ url, title });
  }
  return { content: [{ type: "text", text: JSON.stringify(results, null, 2) }] };
`)

// 2. Execute it
browser_run_script({ path: "/tmp/scrape-pages.js", params: { urls: [...] } })

Script bindings: params, daemon, require, signal, onUpdate, ctx, console, fetch, JSON, Buffer, setTimeout, clearTimeout.

Scripts are written to disk — auditable and re-runnable.

Commands

| Command | Description | |---------|-------------| | /browser-setup | Connect pi to Chrome (run once) | | /browser-status | Show daemon health and current page | | /browser-reload-daemon | Restart the connection |

What NOT to do

• Don't launch your own browser — you're connected to the user's real Chrome
• Don't type credentials — if you hit an auth wall, stop and ask
• Don't assume a page action worked — screenshot to verify
• Don't write pixel coordinates into documentation — describe selectors and text instead
• Don't ignore dialogs — check browser_page_info first

Architecture

pi agent → pi-browser-harness (TypeScript)
               │ CDP WebSocket
               ▼
            Chrome

Temporary scripts run inside the harness process with full daemon and Node.js access.

Troubleshooting

| Symptom | Fix | |---------|-----| | DevToolsActivePort not found | Open chrome://inspect/#remote-debugging, tick the checkbox, click Allow | | Connection fails after Chrome restart | Run /browser-reload-daemon | | Page seems loaded but content is missing | SPA — poll with browser_execute_js for the element you need | | A JS dialog is blocking actions | browser_page_info will report it — use browser_handle_dialog | | Daemon not starting | Run /browser-setup to re-run guided setup |

Contributing

See CONTRIBUTING.md for setup, conventions, and PR process.

Security

This extension drives your real Chrome. The agent can see open tabs, read page content, submit forms, and act inside authenticated sessions. browser_run_script evaluates JavaScript in the pi process with full require access — review any temporary scripts before executing them.

License

MIT — see LICENSE.