@apica-io/asm-playwright-runner

A lightweight CLI wrapper around Playwright that executes browser automation from multiple input formats, captures a rich execution trace, and emits a structured, machine-readable result model.

It is designed to run a single "collection" — a Playwright script, a Playwright Test file, a Python pytest file, or a declarative JSON flow — and turn its execution into consistent artifacts (trace model, screenshots, sources, and a tar bundle) that downstream tooling can consume.

Features

• Four input formats — declarative JSON flows, plain Playwright scripts, Playwright Test (@playwright/test) files, and Python pytest Playwright tests.
• Automatic script-type detection — the correct execution strategy is chosen from the file extension and contents; no flag required.
• Multi-browser — Chromium, Firefox, and WebKit.
• Headless or headed execution.
• Structured trace model — every run produces a trace-model.json describing actions, events, console/stdio, network resources, timings, and errors.
• Artifact generation — screenshots, source files, and a combined artifacts.tar.
• Custom Chromium/Chrome executable path.
• SSL client-certificate authentication (mutual TLS).
• Extra HTTP headers injection.
• Configurable timeouts and log levels.
• Pass-through of arbitrary Playwright CLI arguments via advanced options.
• Credential redaction — secrets are stripped from the published result model.

Requirements

• Node.js 18+
• Playwright browsers installed (npx playwright install)
• For Python pytest collections only:
  • Python 3.8+
  • pytest-playwright available on PATH (pip install pytest-playwright && playwright install)

Installation

Global

npm install -g @apica-io/asm-playwright-runner

As a project dependency

npm install @apica-io/asm-playwright-runner

Then invoke it via npx asm-playwright-runner … or an npm script.

Usage

asm-playwright-runner <collection> [options]

<collection> is the path to the file to execute. Its type is detected automatically:

| File pattern | Detected type | Execution strategy | | ------------------------------------ | ------------------------ | ------------------------------------- | | *.json | JSON flow | Built-in action runner | | *.py | Python Playwright | pytest | | contains test( | Playwright Test | npx playwright test | | contains export default | Plain Playwright script | Direct invocation of the default export |

Options

| Option | Description | Default | | ------------------------------------ | --------------------------------------------------------------- | ---------- | | -b, --browser <browser> | Browser type (chromium, firefox, webkit) | chromium | | -v, --verbose | Accepted but currently has no effect — use --logLevel debug | false | | --chromiumPath <path> | Custom Chromium/Chrome executable path | — | | -e, --config <path> | Playwright Test config file (Playwright Test collections only) | — | | -dd, --dataDir <dir> | Directory holding certificates and test data (a file resolves to its directory) | — | | -dk, --decryptKey <decryptKey> | Decryption key for --config | — | | -ev, --envVars <envVars...> | Environment variables for the collection, as name=value pairs | — | | --headless | Run browser headless | true | | --no-headless | Run browser with a visible UI | — | | -r, --resultDir <dir> | Directory for result artifacts | result | | -l, --logLevel <logLevel> | Log level (info, debug, error; any log4js level accepted) | info | | --extraHTTPHeaders <headers> | Extra HTTP headers as a JSON string | — | | --testTimeout <timeout> | Timeout in milliseconds for each test (the whole flow/script for JSON and plain scripts) | — | | --checkAttemptTimeout <timeout> | Timeout in milliseconds for the whole run, all tests together | — | | --actionTimeout <timeout> | Timeout in milliseconds for each action (click, fill...) | — | | --navigationTimeout <timeout> | Timeout in milliseconds for each navigation (goto, reload, waitForURL...) | — | | -op, --advancedOptions <args...> | Additional raw Playwright/pytest CLI arguments (Test/pytest only) | — | | -rv, --returnResult | Print the full structured result object to stdout | false | | -rs, --includeSource | Include source files in the extracted results | false | | -sc, --screenshot <mode> | Image mode (always, never, downtime), as the check runner names it | downtime | | -bl, --blur <level> | Blur every captured screenshot, 1 (light) to 4 (heavy) | — | | --sslClientCert <path> | Client certificate (PEM) | — | | --sslClientKey <path> | Client certificate private key | — | | --sslClientCertOrigin <origin> | Exact origin the certificate is presented to (https://host[:port]); required with --sslClientCert | — | | --sslClientPassphrase <passphrase> | Client certificate passphrase | — | | --proxyServer <server> | Proxy every browser request through this server (http://host:port, socks5://host:port) | — | | --proxyBypass <hosts> | Comma-separated hosts that skip the proxy (localhost,.example.com) | — | | --proxyUsername <username> | Proxy username, for a proxy that requires authentication | — | | --proxyPassword <password> | Proxy password | — |

Option applicability by collection type

The runner manages the browser directly for JSON flows and plain Playwright scripts, but delegates Playwright Test and pytest collections to their native CLIs. As a result, some options only apply to certain types:

| Option | JSON flow | Plain script | Playwright Test | pytest | | --------------------- | :-------: | :----------: | :-------------: | :----: | | --browser | ✅ | ✅ | ✅ | ✅ | | --headless / --no-headless | ✅ | ✅ | ✅ (--headed) | — | | --chromiumPath | ✅ | ✅ | — | — | | --config | — | — | ✅ | — | | --decryptKey | — | — | ✅ (--config) | — | | --testTimeout | ✅ | ✅ | ✅ | ✅ | | --checkAttemptTimeout | ✅ | ✅ | ✅ | ✅ | | --actionTimeout | ✅ | ✅ | ✅ | ✅ | | --navigationTimeout | ✅ | ✅ | ✅ | ✅ | | --extraHTTPHeaders | ✅ | ✅ | — | — | | --sslClient* | ✅ | ✅ | — | — | | --proxy* | ✅ | ✅ | ✅ | ✅ | | --dataDir | ✅ | ✅ | ✅ (ASM_DATA_DIR only) | ✅ (ASM_DATA_DIR only) | | --envVars | ✅ | ✅ | ✅ | ✅ | | --advancedOptions | — | — | ✅ | ✅ | | --resultDir, --logLevel, --verbose, --returnResult, --includeSource | ✅ | ✅ | ✅ | ✅ | | --screenshot, --blur | ✅ | ✅ | ✅ | ✅ |

Examples

Run a plain Playwright script

asm-playwright-runner ./samples/playwright-script.spec.ts

Run a Playwright Test file

asm-playwright-runner ./samples/playwright-test-script.spec.ts --browser chromium --headless

Values are checked before anything is launched, so a bad one fails immediately with its own name rather than part-way through a run: --browser must be chromium, firefox or webkit (case and surrounding space are ignored), and --testTimeout, --checkAttemptTimeout, --actionTimeout and --navigationTimeout must be positive numbers of milliseconds. Both exit 1 with the reason on stderr. A --logLevel log4js does not recognise falls back to info instead of aborting the run — logging at the wrong verbosity is not worth losing a result over.

--screenshot must be always, never or downtime - the check runner's image mode, spelled the same way here so a check carries one value across both runners - and --blur an integer from 1 to 4. Omitting --screenshot gives downtime, but a value that is not one of the three is an error rather than a silent fall back to it - a mistyped mode is a caller that believes it asked for something else. Playwright's own on / off / only-on-failure are rejected on the same grounds.

--dataDir accepts a file as well as a directory. A caller that names a check's resources individually — a certificate, a CSV of test data — can pass whichever one it has to hand, and the directory holding it is used, with a warning saying so. A path that does not exist is still an error.

Run a Playwright Test file with its own config

asm-playwright-runner ./tests/login.spec.ts --config ./tests/playwright.config.ts

--config is also spelled -e, for callers that name this file as the check's environment file.

--config is forwarded to playwright test --config. Without it, Playwright resolves playwright.config.ts from the runner's working directory — not from the collection's directory — so a config sitting next to the spec is silently ignored and settings such as use.baseURL never apply.

--browser and projects are mutually exclusive. Playwright Test fails with "Cannot use --browser option when configuration file defines projects". The runner therefore forwards --browser only when you pass it explicitly, so a config with projects works out of the box. If you pass both, expect that error — set browserName inside each project instead.

Settings the runner owns and always forwards cannot be overridden by a config: --trace on and --output are required for the trace model to be produced at all. A config setting trace: 'off' has no effect.

Run a Python pytest file

asm-playwright-runner ./samples/new-script-python.py --browser chromium -r resultpy -l debug

Run a JSON flow

asm-playwright-runner ./samples/example.json --browser chromium -r results

Run with Firefox

asm-playwright-runner ./scripts/test.spec.ts --browser firefox

Run with a visible browser

asm-playwright-runner ./scripts/test.spec.ts --no-headless

Store results in a specific directory

asm-playwright-runner ./scripts/test.spec.ts --resultDir ./results

Use a custom Chromium binary

macOS:

asm-playwright-runner ./scripts/test.spec.ts \
  --chromiumPath "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

Windows:

asm-playwright-runner ./scripts/test.spec.ts \
  --chromiumPath "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"

Inject extra HTTP headers

asm-playwright-runner ./scripts/test.spec.ts \
  --extraHTTPHeaders '{"Authorization":"Bearer token","X-Test":"true"}'

Configure a timeout

Four timeouts, each a positive number of milliseconds:

| Flag | Limits | JSON flow | Plain script | Playwright Test | pytest | | --------------------- | ---------------------------------------------------------- | :-------: | :----------: | :-------------: | :----: | | --testTimeout | each test as a whole | ✅ (whole flow) | ✅ (whole script) | ✅ (--timeout) | ✅ (each test body) | | --checkAttemptTimeout | the whole run, all tests together | ✅ (whole flow) | ✅ (whole script) | ✅ (whole run) | ✅ (whole session) | | --actionTimeout | each action (click, fill, waits...) | ✅ | ✅ | ✅ (use.actionTimeout) | ✅ | | --navigationTimeout | each navigation (goto, reload, waitForURL...) | ✅ | ✅ | ✅ (use.navigationTimeout) | ✅ |

# Each test 60s, the whole file 5 minutes, each action 10s, each navigation 30s.
asm-playwright-runner ./samples/playwright-test-script.spec.ts \
  --testTimeout 60000 --checkAttemptTimeout 300000 --actionTimeout 10000 --navigationTimeout 30000

• --testTimeout - a JSON flow or plain script counts as one test, so it bounds the whole flow or script. When it runs out the run fails with Test timeout of <n>ms exceeded., the remaining steps are skipped, and the trace up to that point is still saved. For pytest it bounds each test's body (fixture setup excluded), applied by the runner's asm_timeout plugin; for Playwright Test it is passed through as --timeout, which covers the test's hooks and fixtures too.
• --checkAttemptTimeout - when it runs out, the test in progress fails with Check timeout of <n>ms exceeded. and the remaining tests are not run; the tests that did run keep their traces, the one cut off included. For Playwright Test the runner's bootstrap enforces it, with --global-timeout set 5s later as a safety net. pytest has no counterpart, so asm_timeout keeps the budget for the session: each test gets whichever is shorter, --testTimeout or what is left of the session's, and the session stops at the test it runs out in. A JSON flow or plain script is the run's only test, so it is cut off exactly as --testTimeout would, by whichever of the two is shorter.
• --actionTimeout / --navigationTimeout - for JSON, plain and pytest they are set as the browser context's defaults, so popups and pages the script opens itself get them too; the navigation one outranks the action one for goto and the like. Playwright Test has no CLI flags for these, so the runner's bootstrap applies them.

A lower value of the script's or config's own wins - --testTimeout and --checkAttemptTimeout are each the lower of the flag and what the collection sets itself:

| Collection | Test timeout from | Check attempt timeout from | | --- | --- | --- | | Playwright Test | the Playwright config's timeout (top level and per project), test.setTimeout() and the like | the Playwright config's globalTimeout | | pytest | pytest-timeout's @pytest.mark.timeout, or its timeout option / ini value (seconds) | pytest-timeout's session_timeout option / ini value (seconds) |

# lower-timeouts.config.ts sets timeout: 1500, so each test gets 1.5s, not 60s.
asm-playwright-runner ./samples/timeouts/pwtest-config-lower.spec.ts \
  --config ./samples/timeouts/lower-timeouts.config.ts --testTimeout 60000

For Playwright Test this works through a wrapper config the runner writes next to the config (so testDir and relative imports resolve as before) and deletes when the run ends; the flags are not passed as --timeout / --global-timeout, which would replace a lower config value. Without a config the flags are passed as those. A flag left out keeps the config's value. JSON flows and plain scripts have no config, so the flags apply as given.

The script's or config's own action and navigation timeouts come first - --actionTimeout and --navigationTimeout only fill in where the collection sets none:

| Collection | Used before the flag (first wins) | | --- | --- | | Playwright Test | test.use({actionTimeout}), a project's use, then the Playwright config's use | | Plain script | the script's own page.setDefaultTimeout() / context.setDefaultTimeout() (and the navigation ones) | | pytest | the test's own set_default_timeout() / set_default_navigation_timeout(), in a fixture or the test |

An explicit 0 (no limit) in the config counts as set. A JSON flow has no timeouts of its own, so the flags apply as given.

--testTimeout is a hard ceiling - nothing in the script or its config can raise it.

• Playwright Test - the runner's bootstrap caps every timeout Playwright runs at --testTimeout: the test's own, each beforeAll/afterAll hook's, and each fixture declared with its own {timeout}. So test.setTimeout(...), test.setTimeout(0), testInfo.setTimeout(...), test.slow() (in the body, at describe/file level, or as a slow annotation), test.describe.configure({timeout}) and a config or project timeout all stop at the ceiling. This relies on Playwright internals; if an upgrade moves them, the run logs a warning and only each test's starting budget is capped.
• pytest - the budgets are read once as the asm_timeout plugin loads, before any test module is imported, so a test cannot raise them by rewriting os.environ.
• JSON flow, plain script - the runner owns the timer; the script has no way to reach it.

--checkAttemptTimeout cannot be raised from a script either: Playwright only reads it from the config, which can only lower it. A spec's test.use({actionTimeout}) still wins over --actionTimeout, but an action can never outlast the test it is in.

SSL client-certificate (mutual TLS) authentication

asm-playwright-runner ./scripts/test.spec.ts \
  --sslClientCert ./client-cert.pem \
  --sslClientKey ./client-key.pem \
  --sslClientPassphrase mypassword \
  --sslClientCertOrigin https://api.example.com

--sslClientCertOrigin names the origin the certificate is offered to, and it is required whenever --sslClientCert is given. Playwright matches it against each request's origin exactly - scheme, host and port, with no wildcard - and presents nothing at all when it does not match. The server then reports a handshake it dropped, or answers with its own "no certificate" error, neither of which points back at the certificate. So the run fails up front instead, before a browser is launched, if the origin is missing, unparseable or not https.

The port is only needed when it is not the default: https://api.example.com covers 443, and https://api.example.com:8443 is a different origin. Anything past the origin is discarded, so a full URL may be passed.

A live example runs against BadSSL, who publish a client certificate for exactly this purpose:

bash samples/certs/badssl/fetch-certs.sh   # downloads the certificate and splits it into cert + key
npm run test:pw-mtls-badssl

samples/badssl-mtls.spec.ts drives https://client.badssl.com/, which serves its page only over a client-authenticated handshake - so asserting on the page title is really an assertion about the handshake. The server distinguishes the two ways it can go wrong, which makes it useful for testing the negative cases as well:

| --dataDir | Certificate offered | Page title | | ---------------------------------- | -------------------------------------- | ------------------------------------------ | | samples/certs/badssl | BadSSL's, trusted | client.badssl.com | | samples/certs/badssl/wrong-cert | self-signed, not issued by BadSSL's CA | 400 The SSL certificate error | | either, with the wrong --sslClientCertOrigin, or no --sslClientCert | none | 400 No required SSL certificate was sent |

Both directories hold a client-cert.pem and a client-key.pem, so the negative case is the same command with --dataDir pointed one level deeper - samples/certs/badssl/wrong-cert/make-wrong-cert.sh generates that pair. Note the third row: a mismatched origin looks exactly like passing no certificate at all, because Playwright silently offers nothing when the origin does not match.

Route browser traffic through a proxy

asm-playwright-runner ./scripts/test.spec.ts --proxyServer http://proxy.example.com:3128 --proxyUsername proxyuser --proxyPassword proxypass --proxyBypass "localhost,.internal.example.com"

Only --proxyServer is required. It accepts http, https, socks4 and socks5; a bare host:port is an HTTP proxy. Credentials go in --proxyUsername / --proxyPassword, as plain text - not in the server url, which Playwright ignores. Chromium does not support authentication on a SOCKS proxy.

The proxy is set on the browser at launch, for every collection type, so contexts a script creates itself go through it too. A proxy the collection sets itself wins: launchOptions.proxy or the proxy option in a Playwright Test config or test.use(), browser_type_launch_args / browser_context_args in pytest, or a proxy passed to launch() / newContext() directly.

The test:proxy/* npm scripts run the samples in samples/proxy through a local proxy, samples/proxy/proxy-server.js, which speaks HTTP (user asm, password asm-proxy-pass), SOCKS4 and SOCKS5 on one port. It answers http://127.0.0.1:18099/ itself, where nothing else listens, so a passing run proves the traffic went through it. test:proxy/socks4-* and test:proxy/socks5-* cover each collection type over SOCKS; test:proxy/bypass and test:proxy/bad-auth are expected to fail.

Pass environment variables to the collection

--envVars takes name=value pairs and publishes them into the environment the collection runs under, so a script reads them the way it always would - process.env.USER_PASS in JavaScript or TypeScript, os.environ["USER_PASS"] in Python:

asm-playwright-runner ./scripts/login.spec.ts   --envVars [email protected] USER_PASS='s3cr3t'

This works for every collection type. JSON flows and plain scripts run in this process; Playwright Test and pytest are spawned as children and inherit the same environment, which also means a playwright.config.ts that reads process.env while it loads sees the values.

Details worth knowing:

• The value is split on the first = only, so a value may contain more of them (--envVars 'TOKEN=abc=def==').
• A bare name with no = defines the variable empty, as export NAME= does in a shell.
• A repeated name takes its last value.
• --envVars is variadic: it consumes every following argument until the next option. Put <collection> before it, or the collection path is swallowed as a pair.
• Values whose name looks like a credential (pass, pwd, secret, token, apikey, auth, and similar) are redacted out of the published trace model. Names that do not - USER_EMAIL, BASE_URL - are left in the results, where they are useful for diagnosis.

Point the run at a data directory

--dataDir names the directory holding a check's certificates and test data:

asm-playwright-runner ./scripts/test.spec.ts   --dataDir ./check-data   --sslClientCert client-cert.pem   --sslClientKey client-key.pem   --sslClientPassphrase mypassword   --sslClientCertOrigin https://api.example.com

It does two things:

1. --sslClientCert and --sslClientKey are resolved against it, so a certificate bundle can ship alongside the collection and be referred to by bare filename. An absolute path is left untouched.
2. The absolute directory is exported as ASM_DATA_DIR, which is how a collection finds its test data - path.join(process.env.ASM_DATA_DIR, 'users.csv'). Read it from the environment rather than writing a relative path: a spawned Playwright Test or pytest run does not necessarily share the runner's working directory.

The run fails before the browser starts if the directory does not exist.

Run a check with encrypted files

Encryption is based on the cryptify npm module, the same as in asm-pm-runner, and --decryptKey applies to exactly one file:

| Decrypted | What it is | | ------------------- | ---------------------------------------------------------------- | | --config / -e | the Playwright Test config a check ships as its environment file |

npm install -g cryptify
cryptify encrypt ./scripts/pw.config.ts   --password 'Secret123!'

asm-playwright-runner ./scripts/test.spec.ts   --config ./scripts/pw.config.ts   --decryptKey 'Secret123!'

Nothing else is decrypted. The collection is read exactly as given, and so are --sslClientCert and --sslClientKey — a check that ships those encrypted has to decrypt them before handing them over, or the TLS handshake fails without naming a key as the cause. Since --config applies to Playwright Test collections only, --decryptKey has no effect on a JSON flow or a plain script.

The password must satisfy cryptify's own complexity rules (at least 8 characters, with an uppercase letter, a lowercase letter, a number and a special character), or the run fails before the browser starts.

The runner never rewrites the encrypted file. It is decrypted to a copy named <pid>_<filename> beside the original; the run uses that copy and deletes it on exit. Beside the original rather than in the system temp directory because Playwright resolves testDir and a config's own relative imports against the directory that config sits in. Two consequences worth knowing:

• The directory holding the encrypted config has to be writable.
• The plaintext copy exists on disk for the length of the run, and a SIGKILL leaves it behind. It is named <pid>_*, so it is easy to spot and to add to .gitignore.

Pass raw arguments to the Playwright/pytest CLI

Two spellings, both ending up as the same argument list. Use a bare -- and write the flags exactly as the native CLI expects them:

asm-playwright-runner ./tests/e2e.spec.ts -- --workers=2 --retries=1

Or pass them to --advancedOptions as a single quoted string:

asm-playwright-runner ./tests/e2e.spec.ts --advancedOptions "--workers=2 --retries=1"

Quotes inside the string are honoured, so a value containing spaces survives intact:

asm-playwright-runner ./tests/e2e.spec.ts --advancedOptions "--grep 'my test name'"

Several unquoted dash-prefixed flags after --advancedOptions will not work (--advancedOptions --workers=2 --retries=1 fails with unknown option '--retries=1'). Commander claims each dash-prefixed word as an option of its own, so quote them or use --.

Print the structured result to stdout

asm-playwright-runner ./scripts/test.spec.ts --returnResult

Input formats

1. Plain Playwright script

A module that default-exports an async function receiving a Playwright Page. The runner launches the browser, creates a page, and invokes the function.

import { expect, Page } from "playwright/test";

export default async function example(page: Page) {
    await page.goto("https://playwright.dev/");
    await expect(page).toHaveTitle(/Playwright/);

    await page.getByRole("link", { name: "Get started" }).click();
    await expect(page.getByRole("heading", { name: "Installation" })).toBeVisible();
}

.ts files are transpiled on the fly via tsx (falling back to ts-node if present).

2. Playwright Test file

A standard @playwright/test file using test(...). Executed through npx playwright test with tracing enabled.

import { test, expect } from '@playwright/test';

test('has title', async ({ page }) => {
    await page.goto('https://apica.io/');
    await expect(page).toHaveTitle(/Apica/);
});

3. Python pytest file

A standard pytest-playwright test. Executed through pytest with --tracing on.

import re
from playwright.sync_api import Page, expect

def test_has_title(page: Page):
    page.goto("https://playwright.dev/")
    expect(page).to_have_title(re.compile("Playwright"))

4. JSON flow

A declarative collection of steps run by the built-in action runner — no code required.

{
  "name": "example",
  "steps": [
    { "action": "goto", "url": "https://playwright.dev/" },
    { "action": "assertTitle", "expected": "Playwright" },
    { "action": "click", "selector": "role=link[name='Get started']" },
    { "action": "waitForSelector", "selector": "role=heading[name='Installation']" },
    { "action": "assertVisible", "selector": "role=heading[name='Installation']" }
  ]
}

Supported actions

| Action | Required fields | Description | | ----------------- | ------------------------ | --------------------------------------------- | | goto | url | Navigate to a URL | | click | selector | Click an element | | dblclick | selector | Double-click an element | | hover | selector | Hover over an element | | type | selector, value | Fill an input with a value | | press | selector, key | Press a key while focused on an element | | keyboardType | text | Type text via the keyboard | | keyboardPress | key | Press a key via the keyboard | | check | selector | Check a checkbox/radio | | uncheck | selector | Uncheck a checkbox | | selectOption | selector, value | Select an option in a <select> | | uploadFile | selector, files | Set input files for an upload control | | screenshot | path | Capture a screenshot to path | | reload | — | Reload the current page | | waitForSelector | selector | Wait for an element to appear | | waitForTimeout | ms | Wait for a fixed duration (ms) | | waitForResponse | urlPattern | Wait for a matching network response | | assertText | selector, expected | Assert element text contains expected | | assertTitle | expected | Assert page title contains expected | | assertVisible | selector | Assert element is visible |

Credential redaction

Playwright's trace records call arguments, headers, cookies and request bodies exactly as they were, so without intervention a login flow publishes its password. The runner redacts secrets at the publish boundary, replacing them with ***** in trace-model.json, in artifacts.tar and in the --returnResult payload alike.

What is redacted:

| Where | Rule | | ----- | ---- | | Typed values | Values filled into fields whose selector names a secret — #password, [name=pwd], otp, cvv, card, credential, ssn, pin | | Echoes of those values | Step titles, action logs and stdio carrying the same string | | Headers | Request and response headers matched on exact name: authorization, proxy-authorization, cookie, set-cookie, x-api-key, x-auth-token, x-csrf-token, and similar | | Cookies | Every cookie value, on both sides | | Query parameters | Credential-looking names in queryString[], in request.url, and in any URL appearing in free text | | Request bodies | postData.text and postData.params — urlencoded pairs and JSON string fields | | Context options | extraHTTPHeaders, httpCredentials.password, proxy.password, client-certificate passphrases | | Runner-supplied secrets | --sslClientPassphrase, --decryptKey, --proxyPassword and --extraHTTPHeaders values, plus --envVars values whose name reads as a credential (USER_PASS, API_TOKEN, ...), matched exactly, model-wide |

Selectors, header and cookie names, URL paths and every non-secret value are preserved — the model stays useful for diagnosis. A username is not treated as a secret:

{ "selector": "#email",    "value": "[email protected]"  }
{ "selector": "#password", "value": "*****"  }

What redaction does not cover

• trace.zip is Playwright's own artifact, written by Playwright, and is not redacted. It is not included in artifacts.tar, but it does sit in the result directory — do not ship it.
• Raw dumps at --logLevel debug (trace-raw-data*.json, test-trace-raw-data.json) are the unparsed trace and are not redacted. The runner warns when debug logging is enabled.
• --includeSource extracts source files verbatim, so a credential hardcoded in a script is published as written. Pass secrets with --envVars instead of hardcoding them.
• --envVars values under a name that does not read as a credential are treated as ordinary data and left in the model. A secret passed as SETTING_7=... is only removed if one of the rules above catches it where it was used — name it SETTING_7_TOKEN, or anything the credential patterns match, and it is scrubbed model-wide.
• Console output is free text. A secret an application logs itself is only removed if it matches a value redaction already identified.
• Field matching is by name. A password field whose selector suggests nothing sensitive — say #field-3 — is not recognised, so give secret inputs recognisable selectors.

Output artifacts

When a run completes, the following are written under --resultDir (default result):

• trace-model.json — the structured execution model (actions, events, stdio, network resources, timings, errors). For Playwright Test / pytest runs, a combined model keyed by run name is written at the top level, with per-run models in each subdirectory.
• trace.zip — the raw Playwright trace (JSON-flow and plain-script runs).
• screenshots/ — screenshots/screencast frames extracted from the trace, subject to --screenshot and --blur (see below). Each frame is published under a generated guid, which is the sha1 its screencast_frames entry carries; the trace's own name for it - page id and swap time - is not published.
• source/ — source files and error-context.md (only when --includeSource is set).
• artifacts.tar — an uncompressed tar bundle of the screenshots and source directories.

Screenshots

--screenshot decides how much of the screencast survives into the artifacts. The three modes are the check runner's image mode (ALWAYS, NEVER, DOWNTIME), so a check's stored setting is passed through unchanged:

• always — every frame is extracted and listed in screencast_frames.
• never — no frames are extracted and screencast_frames is empty. For JSON flows and plain scripts the screencast is not recorded at all; Playwright Test and pytest record one into their own trace, from which nothing is then published. The pages themselves stay in the model, with their navigation timings.
• downtime (default) — the screencast is recorded, then kept only for traces that failed, which is the check being down. The verdict is per trace, so in a Playwright Test or pytest run the failing test keeps its frames while the tests that passed beside it publish none. A failure counts whether it was recorded in the trace, in the test hook trace (where a failed expect() lands), or seen by the runner itself.

--blur <1-4> blurs every extracted frame, for checks whose pages show customer data: 1 still leaves the layout readable, 4 leaves shape and colour only. A frame that cannot be blurred is dropped rather than published unblurred, with a warning naming it — the blurred copy is the only one written, and the original never leaves the trace.

When --logLevel debug is set, raw trace dumps (trace-raw-data.json, test-trace-raw-data.json) are also written next to each trace for debugging.

With --returnResult, the combined model is additionally printed to stdout as JSON so a calling process can capture it directly.

Exit codes

• 0 — run completed successfully.
• 1 — the collection file was missing, script-type detection failed, a step/test failed, or a trace could not be processed.

Further documentation

| Document | Covers | | -------- | ------ | | Architecture | Execution pipeline, script-type detection, the four execution strategies, trace processing, result layout | | Trace model reference | Complete field reference for trace-model.json | | Navigation timing | How DOM lifecycle timings are captured across the three injection paths | | Troubleshooting | Symptoms, causes and fixes | | Changelog | Every published version, including breaking changes to the result contract |

Development

# Install dependencies
npm install

# Build (TypeScript -> dist/)
npm run build

# Type-check tests
npm run typecheck

# Run the test suite
npm test
npm run test:watch
npm run test:coverage

Convenience scripts for running the bundled samples with ts-node are defined in package.json (e.g. npm run test:playwright-script, npm run test:playwright-json, npm run test:playwright-python).

npm run test:pw-env-encrypted is the one that exercises --decryptKey:

npm run test:pw-env-encrypted

It first runs samples/encrypt-config.js, which copies samples/pw-env.config.ts to samples/pw-env-encrypted.config.ts and encrypts the copy with Secret123! — cryptify works in place, so the sample itself stays readable. The runner is then pointed at the ciphertext with -e samples/pw-env-encrypted.config.ts -dk Secret123!.

Decryption is load-bearing in that run rather than decorative: the config is what sets USER_EMAIL / USER_PASS, baseURL and the webServer block, so if the copy did not decrypt the test could not reach /login at all. The plaintext <pid>_ copy is deleted on exit; the encrypted one stays behind for the next run and is git-ignored.

License

ISC © Apica