🧑‍🚀 Testronaut

Testronaut is an autonomous testing framework powered by LLMs and Playwright.
It lets you define mission-based tests in plain English, then runs them through a real browser to validate UI workflows — all while generating human-readable reports.

🌌 Join the Mission Control Community

Got questions, ideas, or missions to share?
Join the Discord to connect with other Testronauts, get support, and help shape the framework’s future.

🚀 Features

• Write tests in plain English — no brittle selectors
• Runs real browser sessions via Playwright
• Works with multiple LLM providers (OpenAI, Google Gemini, and more coming)
• Modular tool and DOM-reasoning system
• Dynamic rate-limit and token-tracking logic
• Generates JSON + HTML reports automatically

📖 Documentation

Looking for deeper guides, API references, and examples?
Check the official docs:

Includes:

• Quickstart and setup
• Writing advanced missions
• Configuring providers and models
• CLI options
• Mission Control integration
• Troubleshooting and FAQs

📦 Installation

Global install (recommended — use testronaut directly):

npm install -g testronaut

Then initialize your project:

testronaut --init

Run the sample mission:

testronaut welcome.mission.js

One-off / no install (use npx to run without installing):

npx testronaut --init
npx testronaut welcome.mission.js

📁 Project Structure

missions/
├── login.mission.js
├── logout.mission.js
└── dashboard.mission.js

Each mission exports a string or function and calls runMissions.

✍️ Example Mission

import { runMissions } from 'testronaut';

export const loginMission = `
Visit ${process.env.URL}.
Fill the username field with ${process.env.USERNAME}.
Fill the password field with ${process.env.PASSWORD}.
Click the Login button.
Wait for the dashboard to appear.
Take a screenshot.
Report SUCCESS if the dashboard is loaded, otherwise FAILURE.
`;

export async function executeMission() {
  await runMissions({ mission: loginMission }, "Login Mission");
}

Create a .env file with your credentials and LLM API key (depending on your chosen provider):

# For OpenAI
OPENAI_API_KEY=sk-...

# Or for Gemini
GEMINI_API_KEY=AIza...

URL=https://example.com/login
[email protected]
PASSWORD=********

🧠 LLM Provider Support

Testronaut is provider-agnostic.
Choose your preferred LLM at init or via environment variables.

# During init
testronaut --init

# Or override anytime
TESTRONAUT_PROVIDER=gemini TESTRONAUT_MODEL=gemini-2.5-pro testronaut

Current supported providers:

| Provider | Example Models | |-----------|----------------| | OpenAI | gpt-4o, gpt-4.1, o3, gpt-5, gpt-5.1, etc. | | Google Gemini | gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-8b |

More providers coming soon (Anthropic, Mistral, etc.).

🏃 Running Missions

Run all missions:

testronaut

Run a specific mission:

testronaut login.mission.js

Chain missions together:

await runMissions({
  preMission: [loginMission],
  mission: fillContactFormMission,
  postMission: logoutMission,
}, "Contact Form Flow");

🧰 Developer Mode (Staging API)

Use the staging API base URL:

testronaut --dev

If the staging deployment is protected by Vercel, pass the bypass secret:

testronaut --dev --vercel-bypass=YOUR_SECRET login

You can also set the bypass secret via environment variables:

export VERCEL_AUTOMATION_BYPASS_SECRET=YOUR_SECRET
# or
export TESTRONAUT_VERCEL_BYPASS=YOUR_SECRET

🔐 Automated MFA Codes

Testronaut can retrieve a stored TOTP MFA code from the Testronaut API during a mission. This uses the sessionToken saved by testronaut login in the project root testronaut-config.json.

The agent will prefer the automated get_mfa_code tool when an MFA nickname is known, then fall back to the manual human input tool if the code is unavailable, the entry is missing, the feature is disabled, or the API session cannot access it.

If the nickname is missing or does not match exactly, the tool can call the MFA list endpoint to see available nicknames. It will retry simple case, spacing, or punctuation mismatches, such as rudy poo matching rudy-poo.

Add a default MFA nickname to testronaut-config.json:

{
  "sessionToken": "eyJ...",
  "mfaName": "github-test-mfa"
}

Or pass the nickname for a single run:

testronaut login.mission.js -o mfa=github-test-mfa

Mission text can also name the MFA entry:

export const loginMission = `
Log in to GitHub.
When prompted for MFA, use the MFA nickname github-test-mfa.
`;

Use staging API endpoints with the same developer flag:

testronaut --dev login.mission.js -o mfa=github-test-mfa

To inspect MFA API traffic during a run, enable debug logging:

testronaut --debug --dev login.mission.js -o mfa=github-test-mfa
# or
TESTRONAUT_API_DEBUG=1 testronaut --dev login.mission.js -o mfa=github-test-mfa

This writes sanitized request and response details to missions/mission_reports/api-debug.log, including the resolved endpoint URL, status, content type, response keys, body preview, parsed response shape, and list endpoint nicknames. Session tokens, bypass secrets, and MFA code values are redacted.

Notes:

• Run testronaut login first so sessionToken exists.
• The CLI calls the API host, not the app host.
• The MFA API feature flag must be enabled.
• Paid access is required by the API for paid-gated MFA operations. If the API returns a payment or access error, the mission can still ask for a manual code when human input is enabled.

📋 Reports

Testronaut generates both JSON and HTML reports automatically under:

missions/mission_reports/

Each includes:

• Steps executed
• Token usage
• Screenshots
• Pass/Fail summaries

🧪 Under the Hood

• Playwright for browser automation
• LLMs for reasoning, DOM parsing, and tool use
• Token throttling + adaptive cooldowns
• Extensible architecture for custom tools and workflows
• DOM trimming controls to cap list sizes (env TESTRONAUT_DOM_LIST_LIMIT or config dom.listItemLimit; use all cautiously—it can spike token use)
• Resource guard to ensure full list/table downloads (config resourceGuard or env TESTRONAUT_RESOURCE_*)

🧭 Mission Control

Mission Control lets you:

• View all reports in one dashboard
• Track mission history and success rates
• Compare results across environments
• Access screenshots and step details anytime

📄 License

MIT

☕ Support the Mission

🤖 Built with ❤️ by Shane Fast

If Testronaut saves you time, consider fueling the mission: