@testperch/playwright

Playwright Test integration for TestPerch.

Use this package to:

• report Playwright runs, tests, steps, output, and attachments to TestPerch
• exclude remotely quarantined tests from CI runs
• run only previously failed or flaky tests during CI build retries
• pre-filter attempt-1 pull request runs with project-configured AI E2E optimization

Installation

pnpm add -D @testperch/playwright @playwright/test

@playwright/test is a peer dependency. If Playwright is already installed in your project, install only @testperch/playwright.

Credentials

Create a TestPerch API key for your project and expose these values to Playwright:

• TESTPERCH_API_KEY
• TESTPERCH_URL

The reporter does not load .env files. Load environment variables in your Playwright config or a shared config module before creating the reporter options.

// testperch.config.ts
import type { TestPerchConnectionOptions } from "@testperch/playwright";

function requiredEnv(name: string) {
	const value = process.env[name];

	if (!value) {
		throw new Error(`${name} is required.`);
	}

	return value;
}

export const testPerchConfig = {
	apiKey: requiredEnv("TESTPERCH_API_KEY"),
	url: requiredEnv("TESTPERCH_URL"),
} satisfies TestPerchConnectionOptions;

Report Playwright Results

Add the TestPerch reporter to playwright.config.ts.

import { defineConfig } from "@playwright/test";
import { testPerchReporter } from "@testperch/playwright";

import { testPerchConfig } from "./testperch.config";

export default defineConfig({
	reporter: [["html", { open: "never" }], testPerchReporter(testPerchConfig)],
});

Run Playwright normally. Run events and output that is not associated with a test are buffered for up to 250 ms and sent in sequential batches of at most 100 events or 180 KB. Test-associated events are grouped by the run-specific test ID and retry number, then held in memory until the attempt finishes. The reporter waits for that attempt's attachment uploads to finish or fail before sending the entire completed attempt group in one request; the normal event-count and byte limits do not split these groups. Each request is retried once for transient network failures, request timeouts, 408, 429, and 5xx responses. Pending completed events are flushed before the reporter exits.

When Playwright is interrupted, the reporter discards unfinished test-attempt groups, aborts pending attachments, prioritizes the terminal run event, and bounds shutdown reporting to five seconds. Interrupted CI runs are reported as cancelled; interrupted local runs remain interrupted. Partial test-attempt data is intentionally never flushed during this shutdown path. A hard process kill or lost runner cannot be observed by the reporter and is eventually classified as unreported by the service.

pnpm exec playwright test

Remote Test Exclusions

The reporter automatically applies TestPerch quarantine and failed-only retry behavior before Playwright shards or runs the discovered suite. Excluded tests do not execute and do not appear in Playwright reports or TestPerch ingestion.

CI logs include the number of failed/flaky matches, quarantine exclusions and branch exceptions, and the final number of test cases that remain after filtering.

Tests continue to import test and expect directly from @playwright/test.

Local runs always execute quarantined tests. In CI, quarantined tests are excluded unless the quarantine allows the detected branch. Branch matching is exact and case-sensitive.

A quarantine matches tests by project-relative file path and Playwright title path, so it applies across Playwright projects and browsers.

Before a CI run starts, TestPerch resolves one execution plan. For attempts after the first, it looks up completed runs with the same CI build ID and runs only tests with a failed, timed-out, interrupted, or unreported attempt, including tests that later recovered and became flaky. If no completed source run exists, the run is treated as the first E2E execution: AI selection is used when configured, otherwise the complete suite runs. If completed source runs had no failures, the retry run excludes every test and completes successfully.

If a remote quarantine or execution-plan lookup fails, the reporter logs a warning and lets tests run.

When AI E2E optimization is enabled for the matched project environment, the reporter sends the unique repository-relative spec-file list for a pull request's first E2E execution, then polls every five seconds for up to three minutes. Concurrent shards with the same CI build ID join one shared selection, and reruns reuse decided or pending work when the earlier attempt never completed E2E. A decision may select all specs, no specs, or an exact non-empty subset. Unknown paths, duplicate paths, missing or malformed decision summaries, API errors, and timeouts all fail open and run the complete suite. Successful decisions include a concise rationale for the GitHub pull request comment. Pull requests changing more than 100 files are rejected by the reporter before provider dispatch.

Disable failed-only retry filtering with:

reporter: [
	testPerchReporter({
		...testPerchConfig,
		onlyRunFailedTestsForCiBuildRetries: false,
	}),
];

CI Metadata

GitHub Actions metadata is detected automatically from native GitHub environment variables:

• CI status from GITHUB_ACTIONS
• attempt from GITHUB_RUN_ATTEMPT
• branch from GITHUB_HEAD_REF or GITHUB_REF_NAME
• CI build ID from GITHUB_RUN_ID
• git SHA from GITHUB_SHA
• pull request number from GITHUB_EVENT_PATH or GITHUB_REF
• pull request changed-file count from pull_request.changed_files in GITHUB_EVENT_PATH
• pull request head SHA from pull_request.head.sha in GITHUB_EVENT_PATH
• pull request target branch from pull_request.base.ref in GITHUB_EVENT_PATH or GITHUB_BASE_REF
• immutable GitHub repository ID from repository.id in GITHUB_EVENT_PATH
• repository URL from GITHUB_SERVER_URL and GITHUB_REPOSITORY

Run attempts are one-based. GitHub's first GITHUB_RUN_ATTEMPT value is 1, and the reporter passes it through unchanged. Explicit attempt values lower than 1 are ignored.

For non-GitHub CI providers or custom environment variable names, pass explicit metadata:

testPerchReporter({
	...testPerchConfig,
	attempt: process.env.CI_ATTEMPT ? Number(process.env.CI_ATTEMPT) : undefined,
	branch: process.env.GIT_BRANCH,
	ciBuildId: process.env.CI_BUILD_ID,
	ciProvider: process.env.CI_PROVIDER,
	githubRepositoryId: process.env.GITHUB_REPOSITORY_ID,
	gitSha: process.env.GIT_SHA,
	pullRequestChangedFileCount: process.env.PULL_REQUEST_CHANGED_FILE_COUNT
		? Number(process.env.PULL_REQUEST_CHANGED_FILE_COUNT)
		: undefined,
	pullRequestHeadSha: process.env.PULL_REQUEST_HEAD_SHA,
	pullRequestTargetBranch: process.env.PULL_REQUEST_TARGET_BRANCH,
	repositoryUrl: process.env.REPOSITORY_URL,
});

Options

| Option | Description | | ------------------------------------- | ---------------------------------------------------------------------------------- | | apiKey | Required TestPerch project API key. | | url | Required TestPerch app URL. | | attempt | One-based CI attempt number. Defaults to 1 when no CI value is detected. | | branch | Branch name for run metadata and quarantine branch matching. | | ci | Whether the run is executing in CI. Defaults to detected CI environment variables. | | ciBuildId | CI build identifier used for run grouping and failed-only retries. | | ciProvider | CI provider name, such as github-actions. | | gitAuthor | Explicit author metadata for the run. | | githubRepositoryId | Immutable GitHub repository ID used for GitHub App publication. | | gitSha | Commit SHA for the run. | | pullRequestNumber | Pull request number for the run. | | pullRequestChangedFileCount | Changed-file count used to skip AI selection above 100 files. | | pullRequestHeadSha | Pull request head SHA used to identify a GitHub comment publication. | | pullRequestTargetBranch | Target branch used to match pull-request environments. | | repositoryUrl | HTTP(S) repository URL used to link source metadata. | | runId | UUID v7 run ID. Generated automatically when omitted. | | shardIndex / shardTotal | One-based positive shard metadata when reporting a sharded Playwright run. | | requestTimeoutMs | Timeout for TestPerch API requests. Defaults to 20000. | | attachmentUploadTimeoutMs | Timeout for attachment uploads. Defaults to 30000. | | onlyRunFailedTestsForCiBuildRetries | Set to false to disable failed-only retry behavior. |

Advanced Contract Exports

Most Playwright projects should use the main package exports shown above. Advanced integrations that need to validate TestPerch Playwright payloads can import schemas and types from @testperch/playwright/contract.