TesterArmy CLI

Local CLI for running Tester Army QA tests with local browsers. More info: https://tester.army

Installation

npm install -g testerarmy

Or run without installation:

npx testerarmy --help

The CLI installs Playwright browser binaries automatically on first ta run when missing. No separate npx playwright install step needed.

Commands

Aliases

The CLI can be invoked using either testerarmy or ta:

ta --help
testerarmy --help

Authentication

Get your API key from the Tester Army Dashboard, then run:

ta auth

Or with the full name:

testerarmy auth --api-key YOUR_API_KEY

Status

Check if CLI is already authenticated and where the active API key comes from:

ta status

Machine-readable output:

ta status --json

status checks key sources in runtime priority order:

1. TESTERARMY_API_KEY env var
2. Stored config (~/.config/testerarmy/config.json)

If no key exists, authenticated is false.

Sign Out

Remove stored credentials:

ta signout

Or:

ta logout

Run Tests

Run a QA test against a website:

ta run "test if my website works correctly on https://example.com"

With options:

ta run "test login flow" --url https://example.com --headed
ta run "test login flow" --url https://example.com --browser firefox

Run from Markdown File

Execute tests defined in markdown files:

# Single test file
ta run tests/homepage.md

# Directory of tests (runs in parallel)
ta run tests/

# With custom concurrency
ta run tests/ --parallel 5

When running a directory, the CLI looks for TESTER.md in that directory and prepends its contents to each test file. This is useful for shared setup instructions (like authentication).

Options

• --url <url>: Target URL if not included in prompt
• --headed: Run browser in headed mode (visible)
• --browser <browser>: Browser to run (chrome, firefox, safari; safari requires macOS)
• --json: Output results as JSON
• --api-key <key>: Override API key for single run
• --base-url <url>: Override API base URL
• --timeout <ms>: Set timeout in milliseconds
• --parallel <n>: Number of parallel tests when running directory (default: 3)
• --output <file>: Write output to file
• --system-prompt-file <file>: Override the base system prompt with file contents

Environment Variables

For automation and CI/CD, you can use environment variables:

# API credentials
export TESTERARMY_API_KEY="your-api-key"
export TESTERARMY_BASE_URL="https://tester.army"

# Target URL for tests (alternative to --url)
export TESTERARMY_TARGET_URL="https://example.com"

# Then run without prompts
ta run "test login flow" --json

Debug Artifacts

For evals and offline review, use --debug:

ta run "verify pricing toggle works" --url http://127.0.0.1:3100 --debug

This writes a debug-run.json artifact inside the run directory with:

• streamed chat/tool chunks
• assistant transcript summary
• normalized tool-call timeline
• final QA result or error details

When --output <file> is provided, the CLI now writes a machine-readable run envelope on both success and failure so external runners can locate the artifacts reliably.

Config

The CLI stores config at:

~/.config/testerarmy/config.json

Config keys:

• apiKey: Tester Army dashboard API key
• baseUrl: API base URL (default: https://tester.army)

CI/CD Example

export TESTERARMY_API_KEY="${{ secrets.TESTERARMY_API_KEY }}"
export TESTERARMY_TARGET_URL="https://staging.example.com"

# Run all tests in directory
ta run tests/ --json > results.json

# Check exit code
if [ $? -ne 0 ]; then
  echo "Tests failed"
  exit 1
fi

Examples

# Interactive authentication
ta auth

# Check auth/key status
ta status --json

# Quick test
ta run "check homepage loads on https://mysite.com"

# Full command
testerarmy run "test checkout flow" --url https://shop.example.com --headed

# Run with Firefox
ta run "test checkout flow" --url https://shop.example.com --browser firefox

# Run from markdown file with env URL
ta run tests/10-create-api-key.md --headed

# Batch run all tests
ta run tests/ --parallel 4 --json

Exit Codes

• 0: All tests passed
• 1: One or more tests failed
• 2: Runtime/CLI error