@releaseqa/playwright

Playwright reporter plugin for ReleaseQA. Automatically reports your Playwright test results to ReleaseQA for tracking, analysis, and automatic flaky test detection.

Installation

npm install --save-dev @releaseqa/playwright

Quick Start

Add the reporter to your Playwright configuration:

// playwright.config.ts
import { defineConfig } from '@playwright/test'

export default defineConfig({
  reporter: [
    ['html'],
    ['@releaseqa/playwright', {
      apiKey: process.env.RELEASEQA_API_KEY,
    }]
  ]
})

That's it! Your test results will now be reported to ReleaseQA.

Configuration

| Option | Type | Default | Description | |--------|------|---------|-------------| | apiKey | string | process.env.RELEASEQA_API_KEY | Your ReleaseQA API key | | project | string | Auto-detected | Project name (defaults to package.json name) | | tags | string[] | [] | Custom tags for filtering | | baseUrl | string | https://releaseqa.com/api/v1 | API URL (for self-hosted) | | debug | boolean | false | Enable debug logging |

Environment Variables

You can also configure the reporter via environment variables:

| Variable | Description | |----------|-------------| | RELEASEQA_API_KEY | Your ReleaseQA API key | | RELEASEQA_PROJECT | Override project name | | RELEASEQA_BASE_URL | Custom API URL |

Features

🔄 Automatic Flaky Test Detection

The reporter automatically detects flaky tests by analyzing Playwright's retry mechanism:

• If a test fails on the first attempt but passes on retry → marked as flaky
• Flaky test patterns are tracked and reported in your dashboard
• Get notifications when tests become flaky

📸 Artifact Tracking

The reporter captures paths to:

• Screenshots (on failure)
• Videos (when enabled)
• Traces (when enabled)

🏷️ Rich Metadata

Each test result includes:

• File path and line number
• Browser and project information
• Retry count and duration
• Full error messages and stack traces

How It Works

The reporter hooks into Playwright's reporter lifecycle:

1. onBegin — Creates a new test run in ReleaseQA
2. onTestEnd — Collects results, detects flaky tests
3. onEnd — Flushes remaining results and marks the run as complete

Example Configuration with All Options

// playwright.config.ts
import { defineConfig } from '@playwright/test'

export default defineConfig({
  retries: 2, // Enable retries for flaky test detection
  reporter: [
    ['html'],
    ['@releaseqa/playwright', {
      apiKey: process.env.RELEASEQA_API_KEY,
      project: 'web-app',
      tags: ['e2e', 'smoke'],
      debug: true,
    }]
  ]
})

Example Output

[ReleaseQA] Test run created: run_abc123
[ReleaseQA] Detected 2 flaky tests
[ReleaseQA] Test run completed: https://releaseqa.com/dashboard/runs/run_abc123

License

MIT