autoreport

An AI-powered HTML reporter for Playwright — failure analysis, locator auto-healing, flamecharts, historical trends, bug-tracker integration, and webhook alerts, in a single self-contained report.html.

Features

• Self-contained HTML — dark theme default, zero external CDN dependencies, works offline
• AI failure analysis — OpenAI + Anthropic, with automatic caching to cut duplicate-failure costs
• Locator auto-healing — detects brittle CSS/XPath and suggests getByRole / getByText / getByTestId alternatives
• Flamechart timelines — see which test steps are actually slow
• Historical sparklines — the last 5 runs per test at a glance (stored in SQLite)
• Bug tracker integration — auto-file Jira / GitHub issues for failures
• Slack / Teams webhooks — fire a formatted summary when the quality gate fails
• Quality gate — fail your CI build when pass-rate drops below a configurable threshold
• TypeScript-first with strict types and zero runtime dependencies

Requirements

• Node.js >= 22.5.0 (uses the built-in node:sqlite module — no native compilation needed)
• Playwright >= 1.40

Installation

npm install --save-dev @vigneshplre4/autoreport

Quick Start

1. Add the reporter to playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    ['@vigneshplre4/autoreport', {
      outputDir: './autoreport-output',
      qualityGatePassRate: 90,
      failOnQualityGate: true,
    }],
  ],
});

See examples/playwright.config.ts for a more complete example.

2. Set environment variables

Copy .env.example to .env and fill in:

AI_PROVIDER=openai
OPENAI_API_KEY=sk-...

3. Run your tests

npx playwright test

Your HTML report appears at autoreport-output/report.html.

Reporter Options

| Option | Default | Description | |---|---|---| | outputDir | ./autoreport-output | Directory for report.html and the SQLite history DB | | reportFileName | report.html | Name of the HTML file | | enableAI | auto | AI analysis. Auto-enabled when OPENAI_API_KEY or ANTHROPIC_API_KEY is set | | qualityGatePassRate | 90 | Fail CI when pass-rate (excluding skipped) drops below this % | | failOnQualityGate | true | Set process.exitCode = 1 on gate failure | | sparklineRunCount | 5 | History slots per test | | maxConcurrentAIRequests | 3 | Cap on parallel AI calls to avoid rate limits | | logLevel | info | debug | info | warn | error | silent |

Environment Variables

See .env.example for the complete list. Highlights:

AI

• AI_PROVIDER — openai (default) or anthropic
• OPENAI_API_KEY, OPENAI_MODEL (default gpt-4o-mini)
• ANTHROPIC_API_KEY, ANTHROPIC_MODEL (default claude-sonnet-4-6)

Bug tracker (optional)

• BUG_TRACKER=jira + JIRA_HOST, JIRA_EMAIL, JIRA_API_TOKEN, JIRA_PROJECT_KEY
• BUG_TRACKER=github + GITHUB_TOKEN, GITHUB_REPO (owner/repo)

Notifications (optional)

• NOTIFIER=slack + SLACK_WEBHOOK_URL
• NOTIFIER=teams + TEAMS_WEBHOOK_URL

Architecture

src/
├── core/          Registry, ConfigLoader, Cache, RetryPolicy, Logger
├── interfaces/    IAIProvider, IDatabaseProvider, IBugTracker, INotification
├── factories/     One factory per provider family
├── providers/
│   ├── ai/        BaseAIProvider + OpenAI + Anthropic
│   ├── database/  SqliteProvider (node:sqlite)
│   ├── bug-tracker/  Jira, GitHub
│   └── notification/ Slack, Teams
├── analysis/      LocatorHealer, PromptBuilder, FailureAnalyzer, QualityGate
├── html/          HtmlGenerator + inline CSS/JS + components
└── reporter.ts    The Playwright Reporter entry point

Adding a new provider (say, Gemini for AI):

1. Create src/providers/ai/GeminiProvider.ts extending BaseAIProvider
2. Wire it into src/factories/AIProviderFactory.ts
3. Done — all shared logic (caching, retries, JSON parsing) already works

Development

npm install
npm run typecheck
npm test
npm run build

License

MIT