TestBro CLI

Command-line interface for AI-powered browser testing. Generate, run, and manage automated Playwright tests from your terminal.

Installation

Global Installation (Recommended)

npm install -g @test-bro/cli

Local Development

npm install -g @test-bro/cli

For local CLI validation workflows (unit tests, dist smoke tests, and npm pack artifact testing), see local-cli-testing.md.

Quick Start

1. Authenticate

# Get API token from https://testbro.dev/settings/tokens
testbro login --token tb_live_xxxxx

2. Create Project Configuration

# Initialize config in your project directory
testbro config init

# Edit test-bro.config.json
{
  "baseUrl": "https://your-app.com",
  "projectId": "your-project-id",
  "mode": "hybrid"
}

3. Run Tests

# Run from natural language description
testbro run --description "Test user login with valid credentials"

# Run from PRD/requirements file
testbro run --file ./requirements.md

# Run saved test case
testbro run --test-case tc_abc123

# Run all project test cases
testbro run --project proj_xyz789

Commands

Authentication

testbro init

Interactive setup wizard for first-time configuration. Recommended for new users.

# Interactive mode (recommended)
testbro init

# Non-interactive mode (CI/CD)
testbro init --token tb_live_xxxxx --non-interactive

# Non-interactive with project
testbro init --token tb_live_xxxxx --project-id proj_abc123 --non-interactive

# Skip project setup
testbro init --token tb_live_xxxxx --skip-project --non-interactive

Options:

• --token <token>: API token (skips interactive authentication prompt)
• --non-interactive: Skip all prompts, use defaults (for CI/CD)
• --project-id <id>: Set project ID in non-interactive mode
• --skip-project: Skip project setup entirely

What it does:

1. Global Configuration: Creates ~/.testbro/config.json if needed
2. Authentication: Prompts for API token and validates it
3. Project Setup: Offers to create new project, use existing, or skip
4. Verification: Runs health checks on API, token, and project access

Output (Interactive Mode):

Welcome to TestBro!

Step 1/4: Global configuration
Config directory: ~/.testbro/ (will be created)
Global config created at ~/.testbro/config.json

Step 2/4: Authentication
> Enter your API token: tb_live_***********
Validating token... done
Logged in as [email protected]

Step 3/4: Project setup
? Initialize project-level config here? (Y/n): Y
? Create a new project or use an existing one?
  > Create new project
    Use existing project
    Skip

? Project name: My App
? Base URL [http://localhost:3000]:
? Default test mode [hybrid]:

Creating project... done
Project "My App" created (ID: proj_abc123)
Created test-bro.config.json

Step 4/4: Verification
Checking API connection... healthy
Checking authentication... valid
Checking project access... ok

Setup complete!

Next steps:
  testbro run --url http://localhost:3000 -d "Test the login page"
  testbro generate --file requirements.md --save
  testbro config show

testbro login

Authenticate with TestBro API.

# Token-based authentication (recommended)
testbro login --token tb_live_xxxxx

# Custom API URL
testbro login --token tb_live_xxxxx --url https://testbro.dev

Options:

• -t, --token <token>: API token from your TestBro account
• -u, --url <url>: API URL (default: https://testbro.dev)

testbro logout

Remove stored credentials.

testbro logout

testbro whoami

Show current authenticated user and active project.

testbro whoami
# Output:
# Logged in as: [email protected]
# Active Project: My Test Project (proj_123)

Project Management

testbro projects

List all projects for the authenticated user.

testbro projects

Output:

Your Projects:
  • My Web App (proj_abc123)
    https://myapp.com
    Created: 2024-01-15

  • Mobile Site (proj_xyz789)
    https://mobile.example.com
    Created: 2024-01-20

testbro projects create

Create a new project interactively.

testbro projects create

# Interactive prompts:
# ? Project name: My New App
# ? Description (optional): E-commerce testing suite
# ? Base URL (optional): https://shop.example.com

testbro use-project

Set or clear the active project.

# Set active project
testbro use-project proj_abc123

# Clear active project
testbro use-project --clear

Options:

• -c, --clear: Clear the active project

Test Generation

testbro generate

Generate test cases from a file or description.

# Generate from PRD file
testbro generate --file ./prd.md

# Generate from text description
testbro generate --description "Test checkout flow with payment"

# Generate and save to project
testbro generate --file ./requirements.md --save

# Specify project
testbro generate --file ./ac.md --save --project proj_123

Options:

• -f, --file <path>: Path to requirements/PRD file
• -d, --description <text>: Text description
• -s, --save: Save generated test cases to project
• -p, --project <id>: Project ID (overrides active project)

Example Output:

Generating test cases...

Generated 3 test cases:

1. Test: User Login Flow
   Steps:
     1. Navigate to login page
     2. Fill in email field
     3. Fill in password field
     4. Click login button
     5. Verify redirect to dashboard

2. Test: Invalid Login Attempt
   Steps: ...

3. Test: Password Reset Flow
   Steps: ...

Save to project? [y/N] y
✓ Saved 3 test cases to project

Test Execution

testbro run

Run tests against a URL.

# Run from description
testbro run --url https://example.com --description "Test login flow"

# Run from file (with deduplication)
testbro run --file ./acceptance-criteria.md

# Run specific test case
testbro run --test-case tc_abc123

# Run all project test cases
testbro run --project proj_xyz789

# Use different execution modes
testbro run --mode ai --description "Test search functionality"
testbro run --mode hybrid --file ./test-plan.md
testbro run --mode selector --test-case tc_123

# Run in headed mode (visible browser)
testbro run --headed --description "Debug login issue"

# Learn selectors from AI actions
testbro run --mode ai --learn-selectors --yes

Options:

Target:

• -u, --url <url>: Target URL (or use baseUrl from config)
• --file <path>: Generate and run from PRD/AC file
• -d, --description <text>: Feature description to test
• -p, --project <id>: Run all test cases from project
• -t, --test-case <id>: Run specific test case

Execution:

• -m, --mode <mode>: Execution mode: selector (default), ai, or hybrid
• --headed: Run browser in headed mode (visible)
• --remote: Run on remote staging environment
• --timeout <ms>: Timeout per step (default: 30000ms)
• --sequential: Keep browser session between test cases (default: true)
• --no-sequential: Start fresh browser session for each test case

Output:

• -f, --format <format>: Output format: pretty or json (default: pretty)
• -v, --verbose: Show verbose output

Recording:

• --record-video: Record video of test execution (automatically uploads to server)
• --record-trace: Capture Playwright trace for debugging (automatically uploads to server)

Selector Learning:

• --learn-selectors: Learn selectors from AI actions
• --yes: Auto-confirm selector learning

Deduplication (when using --file):

• --no-dedup: Skip deduplication check
• --auto-merge: Auto-apply smart merge without prompting

Examples:

# Basic test run
testbro run --url https://myapp.com --description "Test user registration"

# Run with AI mode and selector learning
testbro run --mode ai --learn-selectors --description "Test complex form"

# Run from PRD with smart deduplication
testbro run --file ./prd.md --auto-merge

# Debug test in visible browser
testbro run --headed --verbose --test-case tc_123

# CI/CD integration (JSON output)
testbro run --format json --project proj_123 > results.json

Output (Pretty Format):

Running tests...

✓ Test: User Login Flow (2.3s)
  ✓ Navigate to login page (450ms)
  ✓ Fill in email field (320ms)
  ✓ Fill in password field (290ms)
  ✓ Click login button (180ms)
  ✓ Verify redirect to dashboard (1.1s)

Summary:
  Tests: 1 passed, 0 failed, 1 total
  Duration: 2.3s

Output (JSON Format):

{
  "summary": {
    "totalTestCases": 1,
    "totalPassed": 1,
    "totalFailed": 0,
    "totalDuration": 2340
  },
  "testCases": [
    {
      "id": "tc_1",
      "name": "User Login Flow",
      "status": "passed",
      "duration": 2340,
      "steps": [...]
    }
  ]
}

Configuration

testbro config

Show current configuration.

testbro config

Output:

TestBro Configuration

User Config (~/.testbro/config.json)
  Path: /Users/name/.testbro/config.json
  API URL: https://testbro.dev
  Authenticated: Yes
  Active Project: proj_abc123

Project Config (test-bro.config.json)
  Path: /path/to/project/test-bro.config.json
  Base URL: https://myapp.com
  Environment: local
  Mode: hybrid
  Timeout: 30000ms
  Project ID: proj_abc123

Authentication
  Token: tb_l...xyz (from user config)
  OpenRouter API Key: sk-o...abc (from OPENROUTER_API_KEY env)

testbro config show

Alias for testbro config.

testbro config init

Create test-bro.config.json template.

testbro config init

Note: For interactive setup with authentication and project selection, use testbro init instead.

Creates test-bro.config.json in the current directory:

{
  "baseUrl": "https://your-app.com",
  "environment": "local",
  "mode": "hybrid",
  "timeout": 30000,
  "headed": false,
  "projectId": "",
  "credentials": {
    "username": "${USERNAME}",
    "password": "${PASSWORD}"
  },
  "deduplication": {
    "enabled": true,
    "autoMerge": false
  }
}

Configuration Files

User Config (~/.testbro/config.json)

Stores user-level settings and authentication.

{
  "apiUrl": "https://testbro.dev",
  "token": "tb_live_xxxxx",
  "activeProjectId": "proj_abc123"
}

Never commit this file to git.

Project Config (test-bro.config.json)

Project-specific settings, checked into version control.

{
  "baseUrl": "https://staging.myapp.com",
  "environment": "staging",
  "mode": "hybrid",
  "timeout": 30000,
  "headed": false,
  "projectId": "proj_xyz789",
  "credentials": {
    "username": "${TEST_USERNAME}",
    "password": "${TEST_PASSWORD}"
  },
  "deduplication": {
    "enabled": true,
    "autoMerge": false
  }
}

Config Options:

• baseUrl (string): Default URL for tests
• environment ("local" | "staging"): Execution environment
• mode ("selector" | "ai" | "hybrid"): Default execution mode
• timeout (number): Default step timeout in milliseconds
• headed (boolean): Run browser in headed mode
• projectId (string): TestBro project ID
• credentials (object): Environment variable references for auth
• deduplication (object): Test case deduplication settings
  • enabled (boolean): Enable deduplication checks
  • autoMerge (boolean): Automatically apply smart merge

Can commit to git. Use environment variables for secrets.

Environment Variables

Required for AI Mode

OPENROUTER_API_KEY=sk-or-v1-xxxxx

Optional

# Authentication (alternative to testbro login)
TEST_BRO_TOKEN=tb_live_xxxxx

# Test credentials (referenced in config)
[email protected]
TEST_PASSWORD=SecureP@ssw0rd

# API URL override
TESTBRO_API_URL=https://testbro.dev

Session Continuity

Sequential Test Execution (Default)

By default, TestBro keeps the browser session alive between test cases when running multiple tests in a single command. This enables sequential flows where later test cases build on the state left by earlier ones.

Example:

testbro run --project proj_123
# Test 1: Navigate to /login, log in, verify dashboard
# Test 2: Continue from dashboard, click profile, verify profile page
# Test 3: Continue from profile, edit settings, verify changes

How it works:

• First test case includes a navigate step to the starting URL
• Subsequent test cases skip their leading navigate step and continue from the current page
• Browser context and cookies persist across all test cases in the run
• If using auto-login (storageState), authentication is injected before the test plan starts

Opt out:

testbro run --project proj_123 --no-sequential
# Each test case starts fresh with a new browser session

When to use --no-sequential:

• Tests should be fully independent
• Each test requires a clean browser state
• Testing logout/login flows that conflict with session persistence

Execution Modes

Selector Mode (Default)

Fast execution using learned selectors from previous AI runs.

testbro run --mode selector --test-case tc_123

Pros:

• Fast (no AI API calls)
• Reliable for stable UIs
• No API costs

Cons:

• Requires learned selectors
• May fail on UI changes

AI Mode

Uses GPT-4o-mini vision for every action.

testbro run --mode ai --description "Test complex workflow"

Pros:

• Handles dynamic UIs
• No selectors needed
• Adapts to changes

Cons:

• Slower execution
• API costs per run
• Requires OPENROUTER_API_KEY

Hybrid Mode (Recommended)

Tries selectors first, falls back to AI if needed.

testbro run --mode hybrid --file ./tests.md

Pros:

• Best of both worlds
• Fast when selectors work
• Resilient to UI changes
• Can learn new selectors

Cons:

• Requires OPENROUTER_API_KEY
• Slight overhead on fallback

Selector Learning

Automatically learn selectors from AI executions to speed up future runs.

# Enable selector learning
testbro run --mode ai --learn-selectors --description "Test flow"

# Auto-confirm learned selectors
testbro run --mode hybrid --learn-selectors --yes

Workflow:

1. AI executes actions and records element interactions
2. Selectors are generated from DOM analysis
3. Selectors are verified on the page
4. User confirms selectors to save (or --yes auto-confirms)
5. Selectors are stored in database
6. Future runs use learned selectors (selector or hybrid mode)

Deduplication

When generating tests from files, TestBro can detect and merge similar test cases.

# Enable deduplication (default)
testbro run --file ./requirements.md

# Skip deduplication
testbro run --file ./requirements.md --no-dedup

# Auto-merge similar tests
testbro run --file ./requirements.md --auto-merge

Deduplication Modes:

• Smart (default): Prompt user for duplicates

  • Skip test cases >90% similar
  • Update test cases 75-89% similar
  • Create new test cases <75% similar

• Force (--no-dedup): Create all test cases

• Auto-merge (--auto-merge): Apply smart mode without prompts

CI/CD Integration

GitHub Actions

name: TestBro Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18

      - name: Install TestBro CLI
        run: npm install -g @test-bro/cli

      - name: Run Tests
        env:
          TEST_BRO_TOKEN: ${{ secrets.TEST_BRO_TOKEN }}
          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
        run: |
          testbro run --project proj_123 --format json > results.json

      - name: Upload Results
        uses: actions/upload-artifact@v3
        with:
          name: test-results
          path: results.json

GitLab CI

test:
  image: node:18
  script:
    - npm install -g @test-bro/cli
    - testbro run --project $PROJECT_ID --format json > results.json
  artifacts:
    paths:
      - results.json
  variables:
    TEST_BRO_TOKEN: $TEST_BRO_TOKEN
    OPENROUTER_API_KEY: $OPENROUTER_API_KEY

Error Handling

Common Errors

Not authenticated:

Error: Not authenticated. Run 'testbro login --token <token>' first.

Solution: Run testbro login --token tb_live_xxxxx

Missing OPENROUTER_API_KEY:

Error: OPENROUTER_API_KEY environment variable is required for AI mode.

Solution: Set environment variable:

export OPENROUTER_API_KEY=sk-or-v1-xxxxx

No URL specified:

Error: No URL specified. Use --url or set baseUrl in test-bro.config.json

Solution: Add URL to config or use --url flag

Debug Mode

# Verbose output
testbro run --verbose --test-case tc_123

# Headed mode (see browser)
testbro run --headed --description "Debug test"

# Longer timeout
testbro run --timeout 60000 --test-case tc_123

Best Practices

1. Use Project Config for Shared Settings

{
  "baseUrl": "https://staging.myapp.com",
  "mode": "hybrid",
  "projectId": "proj_123",
  "deduplication": {
    "enabled": true,
    "autoMerge": false
  }
}

Commit to git for team consistency.

2. Store Secrets in Environment Variables

# .env (don't commit)
TEST_BRO_TOKEN=tb_live_xxxxx
OPENROUTER_API_KEY=sk-or-v1-xxxxx
[email protected]
TEST_PASSWORD=password123

Reference in config:

{
  "credentials": {
    "username": "${TEST_USERNAME}",
    "password": "${TEST_PASSWORD}"
  }
}

3. Use Hybrid Mode for Best Results

testbro run --mode hybrid --learn-selectors

Balances speed and flexibility.

4. Enable Selector Learning

testbro run --mode ai --learn-selectors --yes

First run uses AI, subsequent runs use selectors.

5. Use JSON Output in CI/CD

testbro run --format json --project proj_123 > results.json

Parse results programmatically.

Development

# Install dependencies
pnpm install

# Build
pnpm build

# Install official CLI globally
npm install -g @test-bro/cli

# Run locally
pnpm start -- run --help

# Type check
pnpm typecheck

# Lint
pnpm lint

Related Packages

• @testbro/core - Type definitions and schemas
• @testbro/runner - Playwright test executor
• @testbro/ai-agent - AI-powered testing
• apps/web - TestBro web API

License

MIT