@checkly/playwright-reporter

Official Playwright reporter for Checkly. Automatically upload test results, screenshots, videos, and traces to gain visibility into your end-to-end tests.

Fully compatible with Playwright's native JSON reporter - use it as a drop-in replacement that adds Checkly integration.

Installation

npm install --save-dev @checkly/playwright-reporter

Requirements:

• Node.js >= 18.0.0
• Playwright >= 1.40.0
• A Checkly account

Quick Start

1. Get Your Credentials

1. Go to Account Settings > General to find your Account ID
2. Go to User Settings > API Keys to create an API key

2. Configure Playwright

Add the reporter to your playwright.config.ts:

import { defineConfig } from '@playwright/test';
import { createChecklyReporter } from '@checkly/playwright-reporter';

export default defineConfig({
  reporter: [
    ['list'],
    createChecklyReporter(),
  ],
});

Tip: Using createChecklyReporter() provides full intellisense for configuration options. Alternatively, use the array syntax: ['@checkly/playwright-reporter', { ... }]

3. Set Environment Variables

export CHECKLY_API_KEY=your_api_key
export CHECKLY_ACCOUNT_ID=your_account_id

Or pass them inline:

CHECKLY_API_KEY=... CHECKLY_ACCOUNT_ID=... npx playwright test

4. Run Tests

npx playwright test

You'll see a session URL in the output:

View test results: https://app.checklyhq.com/test-sessions/abc123

[...test execution...]

✓ Check the results: https://app.checklyhq.com/test-sessions/abc123

Configuration Options

import { createChecklyReporter } from '@checkly/playwright-reporter';

createChecklyReporter({
  outputDir: 'test-results',
  sessionName: 'My Test Suite',
  dryRun: false,
  verbose: false,
  showProgress: true,
  showSummaryTable: true,
})

| Option | Type | Default | Description | |--------|------|---------|-------------| | apiKey | string | CHECKLY_API_KEY env var | Checkly API key | | accountId | string | CHECKLY_ACCOUNT_ID env var | Checkly account ID | | outputDir | string | Playwright's outputDir | Directory for assets, JSON, and ZIP | | sessionName | string \| function | Auto-generated | Custom session name | | dryRun | boolean | false | Create ZIP without uploading | | verbose | boolean | false | Enable debug logging | | showProgress | boolean | true | Show real-time test progress in the terminal | | printSteps | boolean | false | Print individual test steps (requires showProgress) | | showSummaryTable | boolean | true | Show per-project breakdown table after the run | | scrubbing | object \| false | See Secret Scrubbing | Configure secret scrubbing for reports and traces | | testCommand | string | Auto-detected | Override the test command shown in the Checkly UI |

Output files (written to outputDir):

• checkly-report.json - JSON test report
• checkly-report.zip - ZIP archive with report and assets

Environment Variables

| Variable | Description | |----------|-------------| | CHECKLY_API_KEY | Your Checkly API key | | CHECKLY_ACCOUNT_ID | Your Checkly account ID | | CHECKLY_REPORTER_VERBOSE | Set to true for detailed debug output | | CHECKLY_REPORTER_DRY_RUN | Set to true to skip uploading | | CHECKLY_REPORTER_OUTPUT_DIR | Override the output directory | | CHECKLY_REPORTER_SHOW_SUMMARY_TABLE | Set to false to hide the summary table | | CHECKLY_TEST_COMMAND | Override the test command shown in the Checkly UI |

Environment variables take precedence over options passed in the config.

What Gets Uploaded

• Test results and status (passed/failed/flaky)
• Execution duration and timing
• Screenshots (on failure or explicit capture)
• Video recordings
• Playwright traces
• Console logs and network requests (extracted from traces)

Secret Scrubbing

The reporter automatically scrubs sensitive values from JSON reports, trace files, and logs before upload.

By default, any environment variable whose name starts with CHECKLY_SECRET_ is scrubbed. You can configure additional scrubbing strategies:

createChecklyReporter({
  scrubbing: {
    envVars: ['DB_PASSWORD', 'STRIPE_KEY'],
    autoDetect: true,
    prefix: 'CHECKLY_SECRET_',
    replacement: '[REDACTED]',
  },
})

| Option | Type | Default | Description | |--------|------|---------|-------------| | envVars | string[] | [] | Environment variable names whose runtime values should be scrubbed | | autoDetect | boolean | false | Scrub env vars matching common secret patterns (SECRET, KEY, TOKEN, PASSWORD, CREDENTIAL, AUTH, PRIVATE, API) | | prefix | string \| false \| null | 'CHECKLY_SECRET_' | Scrub all env vars starting with this prefix. Set to false or '' to disable | | replacement | string | '*********' | Custom replacement string for scrubbed values |

Set scrubbing: false to disable scrubbing entirely.

Flaky Test Detection

The reporter automatically detects flaky tests:

• Flaky test: A test that failed initially but passed after retry
• Degraded status: Set when there are flaky tests but no complete failures
• Overall status: passed if all tests eventually pass, failed otherwise

Running Locally (No Upload)

For local development without uploading to Checkly:

# Simply don't set CHECKLY_API_KEY or CHECKLY_ACCOUNT_ID
npx playwright test

What happens:

• Reporter creates checkly-report.zip locally
• Upload is automatically skipped
• Tests run normally
• You can inspect the ZIP file for debugging

Dry Run Mode

Create local JSON and ZIP files without uploading:

createChecklyReporter({
  dryRun: true,
})

Conditional dry run (skip uploads when no credentials):

createChecklyReporter({
  dryRun: !process.env.CHECKLY_API_KEY,
})

CI/CD Integration

GitHub Actions

name: Playwright Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        env:
          CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
          CHECKLY_ACCOUNT_ID: ${{ secrets.CHECKLY_ACCOUNT_ID }}
        run: npx playwright test

GitLab CI

test:
  image: mcr.microsoft.com/playwright:v1.57.0-jammy
  stage: test
  script:
    - npm ci
    - npx playwright test
  variables:
    CHECKLY_API_KEY: $CHECKLY_API_KEY
    CHECKLY_ACCOUNT_ID: $CHECKLY_ACCOUNT_ID

Setting up variables:

1. Go to your GitLab project
2. Navigate to Settings > CI/CD > Variables
3. Add CHECKLY_API_KEY (check "Mask variable")
4. Add CHECKLY_ACCOUNT_ID (check "Mask variable")

Multiple Reporters

Combine with other Playwright reporters:

import { createChecklyReporter } from '@checkly/playwright-reporter';

export default defineConfig({
  reporter: [
    createChecklyReporter(),
    ['html', { outputFolder: 'playwright-report' }],
    ['list'],
    ['junit', { outputFile: 'test-results/junit.xml' }],
  ],
});

Understanding Test Sessions

The reporter creates suite-level test sessions, not individual test file results.

When you run npx playwright test:

1. One test session is created, named after your directory
2. One test result for the entire run
3. All assets uploaded together in a single ZIP file

Benefits:

• Consolidated view of your entire test suite
• Efficient storage
• Track overall pass/fail rates over time

Security

Always use environment variables for credentials. Never commit API keys to version control.

// DON'T DO THIS
createChecklyReporter({
  apiKey: 'chk_api_...',  // Hardcoded credentials!
})

// DO THIS
createChecklyReporter()  // Reads from environment

Use the Secret Scrubbing feature to ensure sensitive values don't leak into uploaded reports and traces.

Documentation

For detailed documentation, visit checklyhq.com/docs/detect/testing/playwright-reporter.

License

Apache 2.0