flakewatch

CLI for Flakewatch -- AI-powered Playwright test failure triage.

Install

npm install -g flakewatch

Or use with npx (no install needed):

npx flakewatch analyze --results-dir ./test-results

Commands

flakewatch analyze

Analyze a Playwright test-results directory.

# Basic usage
npx flakewatch analyze --results-dir ./test-results

# With options
npx flakewatch analyze \
  --results-dir ./test-results \
  --config ./flakewatch.config.ts \
  --format markdown \
  --mode full \
  --base-url http://localhost:3000

Options:

| Flag | Short | Description | |---|---|---| | --results-dir | -r | Path to Playwright test-results directory (default: ./test-results) | | --config | -c | Path to config file | | --format | -f | Output format: markdown, json, github-comment | | --mode | -m | screenshot (default) or full (live browser) | | --base-url | -b | App URL (required for full mode) |

flakewatch investigate

Investigate a single test file interactively.

npx flakewatch investigate --test-file ./tests/login.spec.ts

Setup

Set your Anthropic API key:

export ANTHROPIC_API_KEY=sk-ant-...

Optionally create a flakewatch.config.ts for custom settings. See the main README for config options.

Generating JSON results

Flakewatch works best with Playwright's JSON reporter output. Add it to your test run:

PLAYWRIGHT_JSON_OUTPUT_NAME=test-results/results.json \
  npx playwright test --reporter=list,json

Or in playwright.config.ts:

reporter: [
  ['list'],
  ['json', { outputFile: 'test-results/results.json' }],
],

License

MIT