playwright-smart-retry-plugin

A Playwright reporter that dynamically disables retries for tests with a persistent failure history. Instead of exhausting all configured retries on a test that is known to always fail, the plugin skips them and lets the test fail immediately — saving time in CI and surfacing real signal faster.

No changes to your test files are required. Configuration is done entirely in playwright.config.ts.

How it works

1. On every run: the reporter records the final outcome (passed or failed after all retries) of each test in a local JSON file (.smart-retry-history.json by default).
2. At the start of the next run: the reporter reads the history and, for any test identified as a persistent failure, sets test.retries = 0 before execution begins.
3. Persistent failure is determined by a configurable window (last N runs, or a time-based window) and a failure threshold.

Installation

npm install --save-dev playwright-smart-retry-plugin

Basic setup

Add the reporter to your playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: 2,
  reporter: [
    ['list'],
    ['playwright-smart-retry-plugin'],
  ],
});

That's all. The plugin uses sensible defaults out of the box.

Configuration

All options are optional. Pass them as the second element of the reporter tuple.

reporter: [
  ['playwright-smart-retry-plugin', {
    historyFile: '.smart-retry-history.json',
    windowSize: 1,
    historyTtl: '30d',
    failureThreshold: 1.0,
    minimumRuns: 1,
  }],
]

Options reference

| Option | Type | Default | Description | |---|---|---|---| | historyFile | string | '.smart-retry-history.json' | Path to the JSON history file, relative to the working directory. | | windowSize | number | 1 | Number of most-recent runs to analyse. Ignored when windowDuration is set. | | windowDuration | string | — | Time-based window (e.g. '2h', '7d', '30m'). When set, takes precedence over windowSize. | | historyTtl | string | '30d' | Maximum age of runs kept in the history file. Older runs are pruned automatically. Accepts the same format as windowDuration. | | failureThreshold | number | 1.0 | Fraction of runs (0–1) that must be failures for a test to be considered a persistent failure. 1.0 means all runs must have failed. | | minimumRuns | number | 1 | Minimum number of runs in the selected window before the plugin makes any decision. Useful to avoid acting on a single data point. |

Duration format

windowDuration and historyTtl accept a number followed by a unit:

| Unit | Meaning | |---|---| | ms | Milliseconds | | s | Seconds | | m | Minutes | | h | Hours | | d | Days |

Examples: '500ms', '30s', '5m', '2h', '7d'.

Examples

Default behaviour

If a test failed in its last run, retries are disabled on the next run.

// playwright.config.ts
reporter: [['playwright-smart-retry-plugin']]

This is equivalent to:

reporter: [['playwright-smart-retry-plugin', {
  windowSize: 1,
  failureThreshold: 1.0,
  minimumRuns: 1,
  historyTtl: '30d',
}]]

Window by number of runs

Disable retries only if the test failed in all of the last 3 runs:

reporter: [
  ['playwright-smart-retry-plugin', {
    windowSize: 3,
    failureThreshold: 1.0,
    minimumRuns: 3,
  }],
]

Window by time

Disable retries if ≥ 80% of runs in the last 24 hours failed (and there are at least 2 data points):

reporter: [
  ['playwright-smart-retry-plugin', {
    windowDuration: '24h',
    failureThreshold: 0.8,
    minimumRuns: 2,
  }],
]

Custom TTL and permissive threshold

Keep history for 7 days; disable retries if at least 2 of the last 3 runs failed:

reporter: [
  ['playwright-smart-retry-plugin', {
    historyTtl: '7d',
    windowSize: 3,
    failureThreshold: 0.67,
    minimumRuns: 3,
  }],
]

Shared history file across projects

Point multiple Playwright projects to the same history file:

export default defineConfig({
  reporter: [
    ['playwright-smart-retry-plugin', { historyFile: '.playwright-history.json' }],
  ],
  projects: [
    { name: 'chromium', use: { browserName: 'chromium' } },
    { name: 'firefox',  use: { browserName: 'firefox' } },
  ],
});

Bypassing the history

Set the SMART_RETRY_IGNORE_HISTORY environment variable to skip the history check entirely. Useful in CI when you want to force the standard Playwright retry behaviour (e.g. when debugging a specific failure):

SMART_RETRY_IGNORE_HISTORY=1 npx playwright test

History file

The history file is a plain JSON file. Each entry uses a composite key of the relative file path and the full test title path.

Do not commit the history file to version control. Its content changes on every run and would generate noise in your git history. Instead, add it to .gitignore and persist it between CI runs using a cache.

Add to .gitignore:

.smart-retry-history.json

Persisting history in GitHub Actions

Use actions/cache to restore and save the history file around each workflow run. The cache key rotates on every run (github.run_id) so the latest history is always saved, while restore-keys ensures the most recent available cache is restored even when the exact key doesn't match.

# .github/workflows/test.yml
name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm

      - run: npm ci

      - name: Restore Playwright history
        uses: actions/cache/restore@v4
        with:
          path: .smart-retry-history.json
          key: playwright-history-${{ github.run_id }}
          restore-keys: |
            playwright-history-

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - run: npx playwright test

      - name: Save Playwright history
        if: always()
        uses: actions/cache/save@v4
        with:
          path: .smart-retry-history.json
          key: playwright-history-${{ github.run_id }}

The if: always() on the save step ensures the history is updated even when tests fail, so the next run has accurate data.

History file format

{
  "tests/login.spec.ts::Auth > should redirect after login": {
    "title": "should redirect after login",
    "titlePath": ["Auth", "should redirect after login"],
    "file": "tests/login.spec.ts",
    "runs": [
      { "date": "2026-03-25T10:00:00.000Z", "status": "failed", "duration": 4200 },
      { "date": "2026-03-26T09:00:00.000Z", "status": "failed", "duration": 3900 }
    ]
  }
}

License

MIT