@apica-io/asm-playwright-runner

A lightweight CLI wrapper around Playwright that executes browser automation from multiple input formats, captures a rich execution trace, and emits a structured, machine-readable result model.

It is designed to run a single "collection" — a Playwright script, a Playwright Test file, a Python pytest file, or a declarative JSON flow — and turn its execution into consistent artifacts (trace model, screenshots, sources, and a zipped bundle) that downstream tooling can consume.

Features

• Four input formats — declarative JSON flows, plain Playwright scripts, Playwright Test (@playwright/test) files, and Python pytest Playwright tests.
• Automatic script-type detection — the correct execution strategy is chosen from the file extension and contents; no flag required.
• Multi-browser — Chromium, Firefox, and WebKit.
• Headless or headed execution.
• Structured trace model — every run produces a trace-model.json describing actions, events, console/stdio, network resources, timings, and errors.
• Artifact generation — screenshots, source files, and a combined artifacts.zip.
• Custom Chromium/Chrome executable path.
• SSL client-certificate authentication (mutual TLS).
• Extra HTTP headers injection.
• Configurable timeouts and log levels.
• Pass-through of arbitrary Playwright CLI arguments via advanced options.
• Credential redaction — secrets are stripped from the published result model.

Requirements

• Node.js 18+
• Playwright browsers installed (npx playwright install)
• For Python pytest collections only:
  • Python 3.8+
  • pytest-playwright available on PATH (pip install pytest-playwright && playwright install)

Installation

Global

npm install -g @apica-io/asm-playwright-runner

As a project dependency

npm install @apica-io/asm-playwright-runner

Then invoke it via npx asm-playwright-runner … or an npm script.

Usage

asm-playwright-runner <collection> [options]

<collection> is the path to the file to execute. Its type is detected automatically:

| File pattern | Detected type | Execution strategy | | ------------------------------------ | ------------------------ | ------------------------------------- | | *.json | JSON flow | Built-in action runner | | *.py | Python Playwright | pytest | | contains test( | Playwright Test | npx playwright test | | contains export default | Plain Playwright script | Direct invocation of the default export |

Options

| Option | Description | Default | | ------------------------------------ | --------------------------------------------------------------- | ---------- | | -b, --browser <browser> | Browser type (chromium, firefox, webkit) | chromium | | -v, --verbose | Accepted but currently has no effect — use --logLevel debug | false | | --chromiumPath <path> | Custom Chromium/Chrome executable path | — | | -e, --config <path> | Playwright Test config file (Playwright Test collections only) | — | | -dd, --dataDir <dir> | Directory holding certificates and test data (a file resolves to its directory) | — | | -dk, --decryptKey <decryptKey> | Decryption key for --config and the client certificate | — | | -ev, --envVars <envVars...> | Environment variables for the collection, as name=value pairs | — | | --headless | Run browser headless | true | | --no-headless | Run browser with a visible UI | — | | -r, --resultDir <dir> | Directory for result artifacts | result | | -l, --logLevel <logLevel> | Log level (info, debug, error; any log4js level accepted) | info | | --extraHTTPHeaders <headers> | Extra HTTP headers as a JSON string | — | | --timeout <timeout> | Test timeout in milliseconds (must be a positive number) | — | | -op, --advancedOptions <args...> | Additional raw Playwright/pytest CLI arguments (Test/pytest only) | — | | -rv, --returnResult | Print the full structured result object to stdout | false | | -rs, --includeSource | Include source files in the extracted results | false | | --sslClientCert <path> | Client certificate (PEM) | — | | --sslClientKey <path> | Client certificate private key | — | | --sslClientPassphrase <passphrase> | Client certificate passphrase | — |

Option applicability by collection type

The runner manages the browser directly for JSON flows and plain Playwright scripts, but delegates Playwright Test and pytest collections to their native CLIs. As a result, some options only apply to certain types:

| Option | JSON flow | Plain script | Playwright Test | pytest | | --------------------- | :-------: | :----------: | :-------------: | :----: | | --browser | ✅ | ✅ | ✅ | ✅ | | --headless / --no-headless | ✅ | ✅ | ✅ (--headed) | — | | --chromiumPath | ✅ | ✅ | — | — | | --config | — | — | ✅ | — | | --decryptKey | ✅ (--sslClient*) | ✅ (--sslClient*) | ✅ (--config) | — | | --timeout | ✅ | ✅ | ✅ | — | | --extraHTTPHeaders | ✅ | ✅ | — | — | | --sslClient* | ✅ | ✅ | — | — | | --dataDir | ✅ | ✅ | ✅ (ASM_DATA_DIR only) | ✅ (ASM_DATA_DIR only) | | --envVars | ✅ | ✅ | ✅ | ✅ | | --advancedOptions | — | — | ✅ | ✅ | | --resultDir, --logLevel, --verbose, --returnResult, --includeSource | ✅ | ✅ | ✅ | ✅ |

Examples

Run a plain Playwright script

asm-playwright-runner ./samples/playwright-script.spec.ts

Run a Playwright Test file

asm-playwright-runner ./samples/playwright-test-script.spec.ts --browser chromium --headless

Values are checked before anything is launched, so a bad one fails immediately with its own name rather than part-way through a run: --browser must be chromium, firefox or webkit (case and surrounding space are ignored), and --timeout must be a positive number of milliseconds. Both exit 1 with the reason on stderr. A --logLevel log4js does not recognise falls back to info instead of aborting the run — logging at the wrong verbosity is not worth losing a result over.

--dataDir accepts a file as well as a directory. A caller that names a check's resources individually — a certificate, a CSV of test data — can pass whichever one it has to hand, and the directory holding it is used, with a warning saying so. A path that does not exist is still an error.

Run a Playwright Test file with its own config

asm-playwright-runner ./tests/login.spec.ts --config ./tests/playwright.config.ts

--config is also spelled -e, for callers that name this file as the check's environment file.

--config is forwarded to playwright test --config. Without it, Playwright resolves playwright.config.ts from the runner's working directory — not from the collection's directory — so a config sitting next to the spec is silently ignored and settings such as use.baseURL never apply.

--browser and projects are mutually exclusive. Playwright Test fails with "Cannot use --browser option when configuration file defines projects". The runner therefore forwards --browser only when you pass it explicitly, so a config with projects works out of the box. If you pass both, expect that error — set browserName inside each project instead.

Settings the runner owns and always forwards cannot be overridden by a config: --trace on and --output are required for the trace model to be produced at all. A config setting trace: 'off' has no effect.

Run a Python pytest file

asm-playwright-runner ./samples/new-script-python.py --browser chromium -r resultpy -l debug

Run a JSON flow

asm-playwright-runner ./samples/example.json --browser chromium -r results

Run with Firefox

asm-playwright-runner ./scripts/test.spec.ts --browser firefox

Run with a visible browser

asm-playwright-runner ./scripts/test.spec.ts --no-headless

Store results in a specific directory

asm-playwright-runner ./scripts/test.spec.ts --resultDir ./results

Use a custom Chromium binary

macOS:

asm-playwright-runner ./scripts/test.spec.ts \
  --chromiumPath "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"

Windows:

asm-playwright-runner ./scripts/test.spec.ts \
  --chromiumPath "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"

Inject extra HTTP headers

asm-playwright-runner ./scripts/test.spec.ts \
  --extraHTTPHeaders '{"Authorization":"Bearer token","X-Test":"true"}'

Configure a timeout

asm-playwright-runner ./scripts/test.spec.ts --timeout 120000

SSL client-certificate (mutual TLS) authentication

asm-playwright-runner ./scripts/test.spec.ts \
  --sslClientCert ./client-cert.pem \
  --sslClientKey ./client-key.pem \
  --sslClientPassphrase mypassword

Pass environment variables to the collection

--envVars takes name=value pairs and publishes them into the environment the collection runs under, so a script reads them the way it always would - process.env.USER_PASS in JavaScript or TypeScript, os.environ["USER_PASS"] in Python:

asm-playwright-runner ./scripts/login.spec.ts   --envVars [email protected] USER_PASS='s3cr3t'

This works for every collection type. JSON flows and plain scripts run in this process; Playwright Test and pytest are spawned as children and inherit the same environment, which also means a playwright.config.ts that reads process.env while it loads sees the values.

Details worth knowing:

• The value is split on the first = only, so a value may contain more of them (--envVars 'TOKEN=abc=def==').
• A bare name with no = defines the variable empty, as export NAME= does in a shell.
• A repeated name takes its last value.
• --envVars is variadic: it consumes every following argument until the next option. Put <collection> before it, or the collection path is swallowed as a pair.
• Values whose name looks like a credential (pass, pwd, secret, token, apikey, auth, and similar) are redacted out of the published trace model. Names that do not - USER_EMAIL, BASE_URL - are left in the results, where they are useful for diagnosis.

Point the run at a data directory

--dataDir names the directory holding a check's certificates and test data:

asm-playwright-runner ./scripts/test.spec.ts   --dataDir ./check-data   --sslClientCert client-cert.pem   --sslClientKey client-key.pem   --sslClientPassphrase mypassword

It does two things:

1. --sslClientCert and --sslClientKey are resolved against it, so a certificate bundle can ship alongside the collection and be referred to by bare filename. An absolute path is left untouched.
2. The absolute directory is exported as ASM_DATA_DIR, which is how a collection finds its test data - path.join(process.env.ASM_DATA_DIR, 'users.csv'). Read it from the environment rather than writing a relative path: a spawned Playwright Test or pytest run does not necessarily share the runner's working directory.

The run fails before the browser starts if the directory does not exist.

Run a check with encrypted files

Encryption is based on the cryptify npm module, the same as in asm-pm-runner, and one key covers the set:

| Decrypted | What it is | | ------------------- | ---------------------------------------------------------------- | | --config / -e | the Playwright Test config a check ships as its environment file | | --sslClientCert | the client certificate, after it is resolved against --dataDir | | --sslClientKey | its private key, likewise |

The collection itself is not decrypted — it is read exactly as given.

npm install -g cryptify
cryptify encrypt ./scripts/pw.config.ts ./certs/client.pem ./certs/client.key   --password 'Secret123!'

asm-playwright-runner ./scripts/test.spec.ts   --config ./scripts/pw.config.ts   --dataDir ./certs --sslClientCert client.pem --sslClientKey client.key   --decryptKey 'Secret123!'

Encrypt all of them, or none. --decryptKey is applied to every file in that table that the run was given, so a plaintext file among encrypted ones fails the run with Failed to decrypt <path> rather than being passed through. This matches asm-pm-runner, which decrypts its environment file and SSL material the same way.

The password must satisfy cryptify's own complexity rules (at least 8 characters, with an uppercase letter, a lowercase letter, a number and a special character), or the run fails before the browser starts.

The runner never rewrites an encrypted file. Each is decrypted to a copy named <pid>_<filename> beside the original; the run uses those copies and deletes them on exit. Beside the original rather than in the system temp directory because Playwright resolves testDir and a config's own relative imports against the directory that config sits in. Two consequences worth knowing:

• The directory holding each encrypted file has to be writable.
• The plaintext copy exists on disk for the length of the run, and a SIGKILL leaves it behind. It is named <pid>_*, so it is easy to spot and to add to .gitignore.

Pass raw arguments to the Playwright/pytest CLI

Two spellings, both ending up as the same argument list. Use a bare -- and write the flags exactly as the native CLI expects them:

asm-playwright-runner ./tests/e2e.spec.ts -- --workers=2 --retries=1

Or pass them to --advancedOptions as a single quoted string:

asm-playwright-runner ./tests/e2e.spec.ts --advancedOptions "--workers=2 --retries=1"

Quotes inside the string are honoured, so a value containing spaces survives intact:

asm-playwright-runner ./tests/e2e.spec.ts --advancedOptions "--grep 'my test name'"

Several unquoted dash-prefixed flags after --advancedOptions will not work (--advancedOptions --workers=2 --retries=1 fails with unknown option '--retries=1'). Commander claims each dash-prefixed word as an option of its own, so quote them or use --.

Print the structured result to stdout

asm-playwright-runner ./scripts/test.spec.ts --returnResult

Input formats

1. Plain Playwright script

A module that default-exports an async function receiving a Playwright Page. The runner launches the browser, creates a page, and invokes the function.

import { expect, Page } from "playwright/test";

export default async function example(page: Page) {
    await page.goto("https://playwright.dev/");
    await expect(page).toHaveTitle(/Playwright/);

    await page.getByRole("link", { name: "Get started" }).click();
    await expect(page.getByRole("heading", { name: "Installation" })).toBeVisible();
}

.ts files are transpiled on the fly via tsx (falling back to ts-node if present).

2. Playwright Test file

A standard @playwright/test file using test(...). Executed through npx playwright test with tracing enabled.

import { test, expect } from '@playwright/test';

test('has title', async ({ page }) => {
    await page.goto('https://apica.io/');
    await expect(page).toHaveTitle(/Apica/);
});

3. Python pytest file

A standard pytest-playwright test. Executed through pytest with --tracing on.

import re
from playwright.sync_api import Page, expect

def test_has_title(page: Page):
    page.goto("https://playwright.dev/")
    expect(page).to_have_title(re.compile("Playwright"))

4. JSON flow

A declarative collection of steps run by the built-in action runner — no code required.

{
  "name": "example",
  "steps": [
    { "action": "goto", "url": "https://playwright.dev/" },
    { "action": "assertTitle", "expected": "Playwright" },
    { "action": "click", "selector": "role=link[name='Get started']" },
    { "action": "waitForSelector", "selector": "role=heading[name='Installation']" },
    { "action": "assertVisible", "selector": "role=heading[name='Installation']" }
  ]
}

Supported actions

| Action | Required fields | Description | | ----------------- | ------------------------ | --------------------------------------------- | | goto | url | Navigate to a URL | | click | selector | Click an element | | dblclick | selector | Double-click an element | | hover | selector | Hover over an element | | type | selector, value | Fill an input with a value | | press | selector, key | Press a key while focused on an element | | keyboardType | text | Type text via the keyboard | | keyboardPress | key | Press a key via the keyboard | | check | selector | Check a checkbox/radio | | uncheck | selector | Uncheck a checkbox | | selectOption | selector, value | Select an option in a <select> | | uploadFile | selector, files | Set input files for an upload control | | screenshot | path | Capture a screenshot to path | | reload | — | Reload the current page | | waitForSelector | selector | Wait for an element to appear | | waitForTimeout | ms | Wait for a fixed duration (ms) | | waitForResponse | urlPattern | Wait for a matching network response | | assertText | selector, expected | Assert element text contains expected | | assertTitle | expected | Assert page title contains expected | | assertVisible | selector | Assert element is visible |

Credential redaction

Playwright's trace records call arguments, headers, cookies and request bodies exactly as they were, so without intervention a login flow publishes its password. The runner redacts secrets at the publish boundary, replacing them with ***** in trace-model.json, in artifacts.zip and in the --returnResult payload alike.

What is redacted:

| Where | Rule | | ----- | ---- | | Typed values | Values filled into fields whose selector names a secret — #password, [name=pwd], otp, cvv, card, credential, ssn, pin | | Echoes of those values | Step titles, action logs and stdio carrying the same string | | Headers | Request and response headers matched on exact name: authorization, proxy-authorization, cookie, set-cookie, x-api-key, x-auth-token, x-csrf-token, and similar | | Cookies | Every cookie value, on both sides | | Query parameters | Credential-looking names in queryString[], in request.url, and in any URL appearing in free text | | Request bodies | postData.text and postData.params — urlencoded pairs and JSON string fields | | Context options | extraHTTPHeaders, httpCredentials.password, client-certificate passphrases | | Runner-supplied secrets | --sslClientPassphrase, --decryptKey and --extraHTTPHeaders values, plus --envVars values whose name reads as a credential (USER_PASS, API_TOKEN, ...), matched exactly, model-wide |

Selectors, header and cookie names, URL paths and every non-secret value are preserved — the model stays useful for diagnosis. A username is not treated as a secret:

{ "selector": "#email",    "value": "[email protected]"  }
{ "selector": "#password", "value": "*****"  }

What redaction does not cover

• trace.zip is Playwright's own artifact, written by Playwright, and is not redacted. It is not included in artifacts.zip, but it does sit in the result directory — do not ship it.
• Raw dumps at --logLevel debug (trace-raw-data*.json, test-trace-raw-data.json) are the unparsed trace and are not redacted. The runner warns when debug logging is enabled.
• --includeSource extracts source files verbatim, so a credential hardcoded in a script is published as written. Pass secrets with --envVars instead of hardcoding them.
• --envVars values under a name that does not read as a credential are treated as ordinary data and left in the model. A secret passed as SETTING_7=... is only removed if one of the rules above catches it where it was used — name it SETTING_7_TOKEN, or anything the credential patterns match, and it is scrubbed model-wide.
• Console output is free text. A secret an application logs itself is only removed if it matches a value redaction already identified.
• Field matching is by name. A password field whose selector suggests nothing sensitive — say #field-3 — is not recognised, so give secret inputs recognisable selectors.

Output artifacts

When a run completes, the following are written under --resultDir (default result):

• trace-model.json — the structured execution model (actions, events, stdio, network resources, timings, errors). For Playwright Test / pytest runs, a combined model keyed by run name is written at the top level, with per-run models in each subdirectory.
• trace.zip — the raw Playwright trace (JSON-flow and plain-script runs).
• screenshots/ — screenshots/screencast frames extracted from the trace.
• source/ — source files and error-context.md (only when --includeSource is set).
• artifacts.zip — a bundle of the screenshots and source directories.

When --logLevel debug is set, raw trace dumps (trace-raw-data.json, test-trace-raw-data.json) are also written next to each trace for debugging.

With --returnResult, the combined model is additionally printed to stdout as JSON so a calling process can capture it directly.

Exit codes

• 0 — run completed successfully.
• 1 — the collection file was missing, script-type detection failed, a step/test failed, or a trace could not be processed.

Further documentation

| Document | Covers | | -------- | ------ | | Architecture | Execution pipeline, script-type detection, the four execution strategies, trace processing, result layout | | Trace model reference | Complete field reference for trace-model.json | | Navigation timing | How DOM lifecycle timings are captured across the three injection paths | | Troubleshooting | Symptoms, causes and fixes | | Changelog | Every published version, including breaking changes to the result contract |

Development

# Install dependencies
npm install

# Build (TypeScript -> dist/)
npm run build

# Type-check tests
npm run typecheck

# Run the test suite
npm test
npm run test:watch
npm run test:coverage

Convenience scripts for running the bundled samples with ts-node are defined in package.json (e.g. npm run test:playwright-script, npm run test:playwright-json, npm run test:playwright-python).

License

ISC © Apica