TestPilot

TestPilot runs natural-language browser tests through OpenAI computer use.

npm install --global @macevoorhees/testpilot
testpilot activate
testpilot init
testpilot studio

The public npm package is a small bootstrap. It contains no native executable and runs no install script. After activation, it securely downloads only the TestPilot runtime for this computer from the private TestPilot release service, verifies its exact version, platform, byte size, and SHA-256, then installs it atomically in the user's cache. Archives with unsafe paths, links, more than 4,096 entries, or more than 512 MiB of expanded file data are rejected before extraction.

Supported computers:

• macOS on Apple silicon or Intel
• glibc-based Linux on arm64 or x64
• Windows on x64

Node.js 22 or newer and the system tar command are required. No separate .NET installation is required. The native runtime includes .NET, Playwright, and the console renderer; TestPilot ensures its compatible Chromium browser is available when needed.

Alpine/musl Linux is not supported by this release and is rejected before a runtime download. Linux hosts must already include the system libraries needed by Chromium; for CI, use a Debian/Ubuntu-based image with Playwright-compatible browser dependencies.

Activation

Run testpilot activate and paste the license key into the hidden prompt. Never put the key in the command itself. Positional keys are rejected to keep them out of shell history and process arguments.

The raw license key is used once and is never saved. TestPilot persists only the returned activation token with user-only filesystem permissions. Run testpilot deactivate to remove the saved activation.

The native runtime captures the scoped activation values only for validation, then removes all license credentials from its process environment before it can launch Playwright or Chromium child processes.

testpilot help, testpilot help <command>, and testpilot version work before activation or a runtime download. After TestPilot has downloaded and verified a runtime, free local commands such as doctor, validate, merge, actions, environments, and credentials continue to use that safe cache without a current activation. init, studio, run, convert, and install-browser remain license-gated.

TestPilot Studio workspace

From a TestPilot suite, run:

testpilot studio

Studio opens the local visual authoring workspace. Create, save, rename, move, or recoverably delete tests and reusable actions; manage tags and folders; see exact reusable-action matches; run a selected test; follow the browser and Computer Use cursor beside the editor; and open the completed screenshot report. Use --project DIR for another suite, --port N for a fixed local port, or --no-open to print the loopback address without launching a browser. Press Ctrl+C in the terminal to close Studio.

For parallel CircleCI jobs, pre-provision these two project secrets:

TESTPILOT_ACTIVATION_TOKEN=tpa_...
TESTPILOT_INSTALLATION_ID=circleci-my-project-qe

Every job sharing the token must use the same stable installation ID. A raw TESTPILOT_LICENSE_KEY can bootstrap a single non-concurrent job, but it should not be used independently across parallel jobs because repeated activation rotates the installation token.

Step timeouts

Set execution.stepTimeoutSeconds in testpilot.config.json to control the hard deadline for each natural-language test step. The default is 120 seconds; valid values are 1 through 3600. This is distinct from browser.operationTimeoutSeconds, which limits each individual browser operation performed inside the step.

TestPilot retries only transient OpenAI API requests, not a failed step or whole failed test. By default it makes the initial request plus up to two retry attempts for HTTP 429, HTTP 5xx, network errors, or per-request timeouts.