stagehand-openai-sdk

Give GPT a browser through the OpenAI SDK. StagehandBrowserDriver implements the driver contract of OpenAI's browser toolset (BrowserToolset from openai/helpers/browser) on top of Stagehand: every member (navigation, page reading with refs, clicks and typing, forms, screenshots and zoom, tabs, waits, plus the opt-ins below), across iframes and open shadow roots, against a local Chrome, a Chromium you already run, or a Browserbase session. The OpenAI SDK's ResponsesToolRunner runs the model's tool loop; this package runs the browser.

Quickstarts for TypeScript and Python, with local Chrome and Browserbase variants: browserbase/openai-cua-toolset.

Install

npm install stagehand-openai-sdk openai

Node.js 22.18 or later. openai is a peer dependency (>=7.33.0 <8, the releases that ship the browser toolset helpers): install it yourself, and keep a single copy in your tree, since the driver reports refusals with the SDK's own error class. The Stagehand runtime this driver needs is bundled in the package (Stagehand 4.2.0-anthropic.3), so there is nothing else to install. The package also installs the published @browserbasehq/stagehand for code that imports Stagehand directly; the driver always uses the bundled copy. Local browsers must be Chromium-based (Chrome, Chromium, Edge).

Quick start

import OpenAI from "openai";
import { ResponsesToolRunner } from "openai/helpers/responses-tool-runner";
import { StagehandBrowserDriver, createBrowserToolset } from "stagehand-openai-sdk";

const driver = await StagehandBrowserDriver.launch({
  headless: false, // watch the browser work; set true to hide it
  allowedDomains: ["example.com", "iana.org"],
});
// OpenAI's BrowserToolset over the driver: the default members, a URL policy derived from the domain lists,
// and the default confirm (a liveness check, not human consent).
const browser = createBrowserToolset(driver);
try {
  const runner = new ResponsesToolRunner({ browser });
  const response = await runner.run(
    new OpenAI(),
    { model: "gpt-6-luna", store: false, input: "Open example.com and tell me the page heading." },
    { maxSteps: 20 },
  );
  console.log(response.output_text);
} finally {
  await driver.close(); // releases the browser; the toolset never closes the driver
}

Other browsers:

await StagehandBrowserDriver.browserbase({ allowedDomains: [...] }); // a Browserbase session (BROWSERBASE_API_KEY)
await StagehandBrowserDriver.connect({ cdpUrl: "ws://127.0.0.1:9222/devtools/browser/...", allowedDomains: [...] }); // a running Chromium
await StagehandBrowserDriver.attach(stagehand); // a Stagehand you already own; close() leaves it running

createBrowserToolset is a convenience. You can also build the toolset yourself: new BrowserToolset(driver, { confirm, urlPolicy }), with makeUrlPolicy(driver) for the domain lists.

Options

Driver options go to any factory; toolset options go to createBrowserToolset(driver, { ... }).

| Option | What it does | | --- | --- | | allowedDomains / blockedDomains | Hosts, IP literals, CIDR ranges or single-label hosts. A domain entry includes its subdomains and the block list wins. Enforced inside the browser on every HTTP(S) request (navigations, redirects, sub-resources, popups, workers) and WebSocket handshakes, and checked on navigate / new_tab before anything is sent. | | headless, chromePath, viewport | Local launch settings (viewport defaults to 1280x720). | | maskSelectors | Elements painted over in every screenshot. | | privacy.hideValues | Which field values read as [value hidden] (default: password, file, card and one-time-code fields). | | screenshot | format, quality, detail; upload returns a Files API file_id instead of a data URL. | | uploads / downloads | Grant resolvers for upload_file / download_file (see below). Off by default. | | tracer | An OpenTelemetry tracer: one browser.<member> span per call. | | timeouts, logging | Every runtime call is bounded; logs go through your sink with typed text struck out. | | members / additionalMembers | The exact member set, or opt-ins added to the defaults (toolset option). | | confirm, urlPolicy | Your consent hook, and a policy AND-ed with the one derived from the domain lists (toolset options). |

Enable optional tools

The default members cover reading, clicking, typing, forms, screenshots, tabs and waits. The rest are opt-ins:

| Member | What it adds | Needs your own confirm | | --- | --- | --- | | click_at, hover_at, drag_path, hold_keys | Pointer and keyboard by viewport coordinates, for canvases and custom widgets | no | | read_page_scoped | Lists page scopes (landmarks, forms, dialogs, tables, iframes) and reads one subtree | no | | set_checked | Sets a checkbox or radio to a state, idempotently | no | | run_script | Runs JavaScript in the page | yes | | upload_file / download_file | Moves files through single-use grants your code resolves | yes | | read_diagnostics | Redacted console and network lines | yes |

Add opt-ins with additionalMembers, or pass the exact set with members (not both). Selecting run_script, upload_file, download_file or read_diagnostics without a confirm throws. Your confirm replaces the default for every member, so approve the ones you consider safe and decide on the rest:

import { StagehandBrowserDriver, createBrowserToolset } from "stagehand-openai-sdk";

const driver = await StagehandBrowserDriver.launch({
  headless: false,
  allowedDomains: ["example.com"],
  uploads: {
    roots: ["/srv/agent-files"], // upload paths must sit inside these
    resolveGrant: async (grantId) => (grantId === "report" ? { path: "/srv/agent-files/report.pdf" } : null),
  },
  downloads: {
    resolveGrant: async (grantId) =>
      grantId === "invoice"
        ? { accept: (c) => c.name.endsWith(".pdf"), deliver: async (path, c) => { /* copy `path` somewhere */ } }
        : null,
  },
});

const browser = createBrowserToolset(driver, {
  additionalMembers: ["click_at", "read_page_scoped", "set_checked", "run_script", "upload_file", "download_file"],
  confirm: async (ctx) => {
    if (ctx.member === "run_script" || ctx.member === "upload_file" || ctx.member === "download_file") {
      return askAPerson(ctx.member, ctx.arguments); // your UI or policy
    }
    return true;
  },
});

Tell the model which grant ids it may use (in your instructions). It names one as grant_id; your resolver turns it into a file ({ path } or { payload }) or refuses it (null). Grants are single use. Downloads land in a per-driver quarantine directory and reach your disk only through deliver.

Limitations

• Domain lists cover the browser's HTTP(S) requests and WebSocket handshakes, not WebRTC or other non-HTTP traffic, so they are not complete network isolation. Run untrusted tasks in an isolated browser and treat page content as untrusted.
• JavaScript dialogs (alert, confirm, prompt, beforeunload) are answered automatically and reported to the model as a dialog change; the message is never shown. The OpenAI toolset has no member to accept a dialog.
• Closed shadow roots are not read.
• Local browsers must be Chromium-based. Firefox and WebKit are not supported.
• download_file is not available in Browserbase sessions.
• The default confirm checks only that the driver is alive and the tab exists. Pass your own for anything that needs a person or a policy decision.
• If the browser connection is lost, close the toolset and create a new driver.

Examples

See browserbase/openai-cua-toolset for runnable examples.

License

MIT. Includes Stagehand (MIT, Browserbase) under vendor/stagehand.