pingu

AI-powered test failure analysis. Feed it results from any test framework, get categorized diagnoses and historical pattern detection.

npx pingu-cli analyze results.xml --triage   # analyze JUnit XML with AI
npx pingu-cli analyze report.json --triage    # analyze JSON results with AI
npx pingu-cli history                         # view past runs and patterns

Why

Tests failed. You open CI logs, scroll through red, and spend 20 minutes figuring out which failures are real bugs, which are flaky, and which are your test data being wrong.

Pingu does that for you. Feed it test results from any framework (JUnit XML or JSON), and get a categorized diagnosis of every failure -- plus historical pattern detection across runs.

How It Works

test results  ->  analyze  ->  AI triage  ->  history patterns  ->  CI integration
 (JUnit/JSON)    (parse)     (categorize)    (flaky/trending)     (PR comment, issues)

1. Analyze results from any test framework -- JUnit XML, pingu JSON
2. Triage failures with Claude AI or offline heuristics (status code mapping)
3. Detect patterns across runs -- flaky tests (pass/fail alternation), trending failures (consecutive failures since commit X)
4. Act -- post a PR comment with results, file GitHub issues, write JUnit XML

Install

npm install -g pingu-cli

Requires Node.js 18+. For AI triage: ANTHROPIC_API_KEY or OPENAI_API_KEY.

Usage

Analyze test results

# From pytest
pytest --junitxml=results.xml
pingu analyze results.xml --triage

# From Jest
jest --reporters=jest-junit
pingu analyze junit.xml --triage

# From Go
go test -v ./... 2>&1 | go-junit-report > results.xml
pingu analyze results.xml --triage

# Full CI pipeline
pingu analyze results.xml --ci   # --triage --comment --remedy --junit

Failure categories

Each failure is categorized as:

| Category | What it means | Retryable? | |----------|-------------|-----------| | api_bug | Bug in the API (500, unexpected response) | No | | test_bug | Bug in the test (bad request, wrong assertion) | No | | config_issue | Auth/config problem (401, 403) | No | | infra_issue | Infrastructure (timeout, connection refused) | Yes | | transient | Temporary (502, 503, 504, 429) | Yes |

History and patterns

Every pingu analyze run is saved to .pingu/runs/. After a few runs, pingu detects:

• Flaky tests -- tests that alternate pass/fail across recent runs
• Trending failures -- tests that started failing N runs ago, correlated with commit SHA

pingu analyze results.xml --triage --sha abc1234  # tag run with commit
pingu history                                      # view past runs
pingu history --last 20                            # more history

CI integration

pingu analyze results.xml --ci

This enables: AI triage, PR comment with results, GitHub issue filing for non-retryable failures, and JUnit XML output.

Requires GITHUB_TOKEN and GITHUB_REPOSITORY environment variables (set automatically in GitHub Actions).

Options

analyze <file>        Path to JUnit XML or JSON report
  --format <fmt>      Force input format: junit, json (auto-detected)
  --url <url>         Base URL label for the report
  --triage            Analyze failures with AI
  --provider <name>   LLM provider: anthropic, openai (default: anthropic)
  --model <model>     Model name
  --json              Output as JSON
  --junit <path>      Write JUnit XML report
  --comment           Post results as PR comment
  --remedy            File GitHub issues for failures
  --sha <sha>         Git SHA to associate with this run
  --ci                Shorthand for --triage --comment --remedy --junit

history
  --last <n>          Number of recent runs (default: 10)

License

MIT