TesterArmy CLI

Local CLI for running Tester Army QA tests with local browsers. More info: https://tester.army

Installation

npm install -g testerarmy

Or run without installation:

npx testerarmy --help

The CLI installs Playwright browser binaries automatically on first ta run when missing. No separate npx playwright install step needed.

Commands

Aliases

The CLI can be invoked using either testerarmy or ta:

ta --help
testerarmy --help

Agent Quickstart

This is the full project-to-run flow an agent can discover from ta --help:

ta auth
echo '{"name":"Example","url":"https://example.com","projectType":"web"}' | ta projects create --json
ta projects list --json
echo '{"category":"site_structure","title":"Auth route","content":"Login is at /login","importance":"high"}' | ta memories create --project <projectId> --json
echo '{"title":"Login flow","steps":[{"title":"Navigate to /login","type":"act"},{"title":"Dashboard loads","type":"assert"}]}' | ta tests create --project <projectId> --json
ta tests list --project <projectId> --json
ta tests run <testId> --remote --wait --json
ta runs wait <runId> --json

Authentication

Get your API key from the Tester Army Dashboard, then run:

ta auth

Or with the full name:

testerarmy auth --api-key YOUR_API_KEY

Status

Check if CLI is already authenticated and where the active API key comes from:

ta status

Machine-readable output:

ta status --json

status checks key sources in runtime priority order:

1. TESTERARMY_API_KEY env var
2. Stored config (~/.config/testerarmy/config.json)

If no key exists, authenticated is false.

Sign Out

Remove stored credentials:

ta signout

Or:

ta logout

Run Tests

Run a QA test against a website:

ta run "test if my website works correctly on https://example.com"

With options:

ta run "test login flow" --url https://example.com --headed
ta run "test login flow" --url https://example.com --browser firefox

Run Dashboard Tests

Fetch and run tests defined in the TesterArmy dashboard on local browsers:

# List your projects
ta projects list

# List tests and groups for a project
ta tests list --project <projectId>
ta groups list --project <projectId>

# Run a single dashboard test
ta tests run <testId>
ta tests run <testId> --headed --url http://localhost:3000

# Run all tests in a group
ta tests run --group <groupId> --project <projectId> --parallel 3

# Queue a remote run and wait for results
ta tests run <testId> --remote --wait --json

# Enable or disable a test without deleting it
ta tests disable <testId>
ta tests enable <testId>

The target URL defaults to the project URL from the dashboard. Override with --url or TESTERARMY_TARGET_URL.

Inspect Run History

List historical dashboard runs and filter them by project or test:

ta runs list --project <projectId> --json
ta runs list --project <projectId> --test <testId> --json

Use --all to follow every page of results.

Project Credentials

List masked credentials for a project:

ta projects credentials <projectId> --json

Create credentials from JSON stdin so passwords do not have to be passed as CLI arguments:

echo '{"kind":"login","label":"Admin","username":"[email protected]","password":"secret"}' | ta projects credentials-create <projectId> --json
echo '{"kind":"inbox","label":"Signup inbox"}' | ta projects credentials-create <projectId> --json

Project Memories

Project memories are durable agent context for a project. Add memories before creating or running tests when there are routes, credentials, or product facts the agent should remember.

List memories:

ta memories list --project <projectId> --json

Create a memory from JSON stdin:

echo '{"category":"site_structure","title":"Auth route","content":"Login is at /login","importance":"high"}' | ta memories create --project <projectId> --json

Delete a memory:

ta memories delete <memoryId> --project <projectId> --json

Fields:

• category: site_structure, test_insights, or user_preferences
• title: short label, max 200 characters
• content: memory content used by agents
• importance: high, medium, or low

Upload Mobile Apps

Upload an iOS Simulator app to a mobile project:

ta upload-app --app-path ios/build/Build/Products/Release-iphonesimulator/MyApp.app --project <projectId>

The command accepts raw .app directories and .app.zip, .zip, .app.tar.gz, .tar.gz, or .tgz archives. Raw .app directories are packed as .app.zip with the bundle at the archive root.

For temporary CI uploads, set an expiration in seconds:

ta upload-app --app-path MyApp.app --project <projectId> --remove-after 3600 --json

Use --output ./result.json to write the JSON payload to a specific file, or --output ./artifacts/ to write ta-app-upload-<date>.json inside a directory.

Android .apk and .apks artifacts are supported. .ipa, .aab, and .xapk files are rejected because TesterArmy needs iOS Simulator .app builds or Android APK/APK set artifacts.

Run CI Group Tests

Trigger a dashboard group in TesterArmy cloud and wait for all returned runs:

testerarmy ci \
  --group <groupId> \
  --project <projectId> \
  --app-id <appId> \
  --commit-sha <sha> \
  --pr-number <number> \
  --delete-app-after-run

Options:

• --group <groupId>: Test group to run
• --project <projectId>: Project that owns the group and uploaded app
• --app-id <appId>: Uploaded mobile app ID to use for iOS tests
• --target-url <url>: Target URL override for web tests
• --commit-sha <sha>: Commit SHA for server-owned GitHub check reporting
• --pr-number <number>: Pull request number for GitHub metadata
• --delete-app-after-run: Delete the explicit --app-id after terminal runs
• --timeout <ms>: Maximum wait time in milliseconds
• --poll-interval-seconds <seconds>: Poll interval for run status checks
• --json: Print { summary, cleanup, warnings, timedOut }
• --output <path>: Write the same JSON envelope to a file, or to ta-result-<date>.json inside a directory path such as ., ./, or ./artifacts/

Run Dynamic Mobile PR Tests

Trigger the dynamic PR agent against a mobile build and wait for the returned run:

testerarmy pr run-dynamic \
  --project <projectId> \
  --platform android \
  --app-id <appId> \
  --pr-number <number> \
  --pr-title "Improve onboarding" \
  --commit-sha <sha> \
  --json

Dynamic PR mode uses POST /api/v1/projects/{projectId}/pull-request-runs. Use --platform ios for iOS Simulator builds and --platform android for APK runs. Provide exactly one mobile selector: --app-id, --bundle-id, or --artifact-url with --artifact-filename. --delete-app-after-run only deletes an explicit --app-id. You can pass extra PR context with --pr-description, --base-branch, and --head-branch.

Options

• --url <url>: Target URL if not included in prompt
• --headed: Run browser in headed mode (visible)
• --browser <browser>: Browser to run (chrome, firefox, safari; safari requires macOS)
• --json: Output results as JSON
• --api-key <key>: Override API key for single run
• --base-url <url>: Override API base URL
• --app-path <path>: Required app bundle/archive path for upload-app
• --project <id>: Required project ID for project-scoped commands
• --remove-after <seconds>: Auto-delete an uploaded app after the given number of seconds
• --timeout <ms>: Set timeout in milliseconds
• --parallel <n>: Number of parallel tests for group runs (default: 3)
• --output <path>: Write output to a file or directory
• --system-prompt-file <file>: Override the base system prompt with file contents

Environment Variables

For automation and CI/CD, you can use environment variables:

# API credentials
export TESTERARMY_API_KEY="your-api-key"
export TESTERARMY_BASE_URL="https://tester.army"

# Target URL for tests (alternative to --url)
export TESTERARMY_TARGET_URL="https://example.com"

# Mobile CI test inputs
export TESTERARMY_PROJECT_ID="your-project-id"
export TESTERARMY_GROUP_ID="your-group-id"
export TESTERARMY_APP_PATH="./build/MyApp.app"

# Then run without prompts
ta run "test login flow" --json

Debug Artifacts

For evals and offline review, use --debug:

ta run "verify pricing toggle works" --url http://127.0.0.1:3100 --debug

This writes a debug-run.json artifact inside the run directory with:

• streamed chat/tool chunks
• assistant transcript summary
• normalized tool-call timeline
• final QA result or error details

When --output <file> is provided, the CLI now writes a machine-readable run envelope on both success and failure so external runners can locate the artifacts reliably.

Config

The CLI stores config at:

~/.config/testerarmy/config.json

Config keys:

• apiKey: Tester Army dashboard API key
• baseUrl: API base URL (default: https://tester.army)

CI/CD Example

Dashboard tests in GitHub Actions

- name: Run dashboard tests
  run: npx testerarmy tests run --group <groupId> --project <projectId> --json --parallel 3
  env:
    TESTERARMY_API_KEY: ${{ secrets.TESTERARMY_API_KEY }}

Examples

# Interactive authentication
ta auth

# Check auth/key status
ta status --json

# Quick test
ta run "check homepage loads on https://mysite.com"

# Full command
testerarmy run "test checkout flow" --url https://shop.example.com --headed

# Run with Firefox
ta run "test checkout flow" --url https://shop.example.com --browser firefox

# Add project memory
echo '{"category":"test_insights","title":"Checkout flake","content":"Wait for payment iframe before asserting success","importance":"medium"}' | ta memories create --project <projectId>

# Batch run dashboard group tests
ta tests run --group <groupId> --project <projectId> --parallel 4 --json

Exit Codes

• 0: All tests passed
• 1: One or more tests failed
• 2: Runtime/CLI error