@lockweb/qa-workflow

Playwright-powered QA workflow tooling for recording tests, running targeted test sets, capturing visual baselines, comparing snapshots, and generating shareable reports.

This package is built around a reusable CLI-first workflow exposed through qaw.

Author: Chris Lock Owner: Lock Web Development LLC

Status

This package is public, but it is not open source at this time. It currently uses the UNLICENSED license while the public API and reuse terms are still being finalized.

Goals

• Make Playwright setup easier to adopt.
• Make test recording and publishing easier to manage.
• Support targeted test execution across suites and targets.
• Provide visual regression capture and compare workflows.
• Generate review-friendly reports that are easier to share with stakeholders.

CLI

The primary CLI entrypoints are:

qaw
qaw init
qaw validate

qaw opens the interactive menu by default.

Direct commands are also available:

qaw record
qaw run
qaw publish
qaw clean
qaw auth
qaw tests
qaw targets
qaw visual
qaw visual capture
qaw visual compare
qaw update
qaw status

All commands also accept:

--config ./path/to/qa-workflow.config.json

When a command supports --ci, the behavior is:

• passed values are used directly
• missing optional values fall back to the normal default
• missing required values fail instead of prompting
• invalid passed values fail instead of prompting

Boolean flags accept true|false, yes|no, 1|0, on|off.

Command Reference

qaw

Interactive main menu.

qaw --config ./qa-workflow.config.json

qaw init

Initial project scaffolding.

qaw init [--root-dir qa-workflow] [--sample-csv true|false] [--config true|false] [--playwright-config true|false] [--script true|false] [--script-name qaw] [--overwrite] [--ci]

Flags:

• --root-dir Workflow root folder. Default: qa-workflow
• --sample-csv Create the sample visual regression CSV
• --config Create qa-workflow.config.json
• --playwright-config Create playwright.config.js
• --script Add a local npm script to package.json
• --script-name npm script name to add. Default: qaw
• --overwrite Overwrite existing generated files when supported
• --ci Run without prompts

qaw validate

Config validation.

qaw validate [--config ./qa-workflow.config.json]

qaw record

Interactive or scripted raw recording flow.

qaw record [--suite <suite>] [--target <target>] [--auth continue|open|cancel] [--host <hostKey>] [--name <test-name>] [--subfolder folder/path] [--path /relative/path] [--ci]

Flags:

• --suite Suite name
• --target Record-enabled target for that suite
• --auth Auth preflight action: continue, open, or cancel
• --host Start host key. Alias: --start-host
• --name Recording name. Default: new-test
• --subfolder Optional subfolder under test/recorded/<suite>/
• --path Relative path to open on the selected host
• --ci Run without prompts

qaw record warns when the generated raw spec contains no Playwright assertions and prevents silent overwrite of existing recorded files.

qaw publish

Promotes a recorded spec into a runnable published spec.

qaw publish [--file recorded/spec.raw.spec.ts] [--name <test-name>] [--subfolder folder/path] [--smoke true|false] [--regression true|false] [--writes-content true|false] [--tags foo,bar] [--personas admin:pass,editor:fail] [--remove-recorded true|false] [--ci]

Flags:

• --file Recorded spec path relative to test/recorded
• --name Final published test name
• --subfolder Optional subfolder under test/published/<suite>/
• --smoke Add or omit @smoke
• --regression Add or omit @regression
• --writes-content Mark the generated test as content-writing. Alias: --writesContent
• --tags Extra comma-separated tags
• --personas Comma-separated persona declarations in user[:pass|fail] form
• --remove-recorded Remove the raw recorded spec after publish. Default: true
• --ci Run without prompts

qaw publish derives the suite and target from metadata written into the raw recording by qaw record. It does not prompt for or override the recorded target.

For interactive publish, the selected recorded file is the source of defaults for name, subfolder, smoke, regression, writes-content, and extra tags.

qaw publish prevents silent overwrite of existing published test files.

qaw clean

Removes recorded specs under the configured recorded directory.

qaw clean [--yes]

Flags:

• --yes Skip the confirmation prompt

qaw run

Runs published tests for a suite folder, with optional tag filtering.

qaw run [--suite <suite>] [--target <target>] [--run-type all|tags|path] [--mode headless|headed|ui|watch] [--tags smoke,regression] [--path folder/or/spec.spec.ts] [--auth continue|open|cancel] [--ci]

Flags:

• --suite Suite folder under test/published
• --target Run-enabled target for that suite
• --run-type all, tags, or path
• --mode headless, headed, ui, or watch
• --tags Tag filter used when --run-type tags is selected
• --path File or folder path relative to the selected suite root. Used when --run-type path is selected
• --auth Auth preflight action: continue, open, or cancel
• --ci Run without prompts

watch mode runs tests headed with one worker and sets PLAYWRIGHT_WATCH=true for the generated Playwright config template. That template disables the overall test timeout in watch mode and applies launchOptions.slowMo using PLAYWRIGHT_SLOW_MO, which defaults to 250.

qaw run writes structured suite run data under reports/runs and renders a suite dashboard under reports/html when a suite report renderer is available. The default renderer is @lockweb/qa-report, and consumers can override it with reporting.suite.renderer. If no renderer is installed, qaw still saves the report data bundle and prints a message telling you how to install a renderer.

qaw auth

Host-level auth management for a suite and target.

qaw auth [--suite <suite>] [--target <target>] [--host <hostKey>] [--action create|replace|clear|keep|skip] [--ci]

Flags:

• --suite Suite name
• --target Target name
• --host Host key to operate on directly
• --action create, replace, clear, keep, or skip
• --ci Run without prompts. In --ci mode, --host and --action are required

qaw tests

Lists published tests for a suite, grouped by folder and annotated with tags.

qaw tests [--suite <suite>] [--folder folder/path] [--tags smoke,perdiem] [--sort folder|name|tags] [--ci]

Flags:

• --suite Suite folder under test/published
• --folder Optional subfolder filter under the suite
• --tags Optional comma-separated tag filter. All listed tags must be present
• --sort Sort within the suite by folder, name, or tags
• --ci Run without prompts

qaw targets

Lists configured targets.

qaw targets [--suite <suite>] [--target <target>]

Flags:

• --suite Restrict output to one suite
• --target Restrict output to one target

qaw visual

Interactive visual regression menu.

qaw visual
qaw visual [--action capture|compare|delete|cancel] [--suite <suite>] [--target <target>] [--csv path/to.csv] [--prefix run-name] [--before snapshot] [--after snapshot] [--zip true|false] [--yes] [--ci]

Flags:

• --action capture, compare, delete, or cancel
• --suite Suite name
• --target Visual-enabled target for capture
• --csv CSV file path
• --prefix Optional capture name prefix
• --before Before snapshot name for compare
• --after After snapshot name for compare
• --zip Include the share zip when comparing
• --yes Required to allow delete in --ci mode
• --ci Run without prompts

qaw visual capture

Direct visual capture entrypoint.

qaw visual capture --target <target> --suite <suite> --csv path/to.csv [--snapshot snapshot-name]

Older positional form is still supported:

qaw visual capture [snapshot] <target> <suite> <csvPath>

qaw visual compare

Direct visual compare entrypoint.

qaw visual compare --before <snapshot> --after <snapshot> --suite <suite> [--zip true]

Older positional form is still supported:

qaw visual compare <beforeSnapshot> <afterSnapshot> <suite> [--zip]

Compare always saves the report data bundle. If a visual renderer is installed, qaw also renders HTML and opens the report automatically. If no renderer is available, qaw prints a message telling you to install @lockweb/qa-report or configure another visual report package.

qaw update

Updates @lockweb/qa-workflow in the consuming project and refreshes Playwright browser binaries for the installed version.

qaw update [--version 0.2.2]

Flags:

• --version Install a specific version instead of latest

qaw status

Shows package and Playwright runtime state.

qaw status [--json]

Flags:

• --json Emit JSON instead of plain text

Project Setup

For a one-off setup command:

npx qaw init

qaw init can:

• ask where workflow files should live and set paths.rootDir
• create qa-workflow.config.json
• create a sample visual regression CSV at <rootDir>/visual-regression/sample.csv
• create playwright.config.js
• add a local npm script such as npm run qaw

playwright.config.js is required for running tests through qa-workflow. It scopes Playwright to the package-managed test directory so Playwright does not auto-discover unrelated *.spec.* files elsewhere in the repo.

It should also set outputDir inside the workflow root so Playwright writes .last-run.json and other run artifacts under the workflow workspace instead of the repo root.

After the package is installed in a project, the same commands can be run through the local binary or a repo script.

The package also includes a config validation command:

npx qaw validate --config ./qa-workflow.config.json

qaw validate checks:

• config file presence and JSON parsing
• consumer playwright.config.js presence
• suites structure
• target capability fields and host definitions
• referenced CSV file paths
• persona support module paths when personas are configured
• selector and boolean field types
• basic path configuration sanity

Configuration

The tool is structured around project-level configuration rather than hardcoded project logic.

Example:

{
  "paths": {
    "rootDir": "."
  },
  "suites": {
    "public-suite": {
      "defaultRequiresLogin": false,
      "personas": {
        "admin": {
          "label": "Administrator",
          "username": "alice",
          "password": "example-password"
        },
        "editor": {
          "label": "Editor",
          "accountId": "editor-123"
        }
      },
      "personaSupport": {
        "module": "qa-workflow/support/persona-switch.cjs"
      },
      "full": true,
      "selectors": {
        "header": "header",
        "main": "main",
        "footer": "footer"
      },
      "csvSets": {
        "main-menu": "visual-regression/public-suite/main-menu.csv",
        "homepage": "visual-regression/public-suite/homepage.csv"
      },
      "targets": {
        "prod": {
          "run": true,
          "record": false,
          "visual": true,
          "writesContent": false,
          "hosts": {
            "site.cmp": {
              "baseUrl": "https://cmp.example.gov",
              "requiresLogin": true,
              "default": true
            },
            "site.preview": {
              "baseUrl": "https://preview.example.gov",
              "requiresLogin": true
            },
            "site.public": {
              "baseUrl": "https://www.example.gov",
              "requiresLogin": false
            }
          }
        }
      }
    }
  }
}

Config Reference

Configuration fields:

• paths.rootDir Base workflow folder inside the consuming repo. Standard subpaths are derived from this automatically.
• paths.testsDir Optional override for the tests root. Default: <rootDir>/test
• paths.recordedDir Optional override for raw recorded specs. Default: <testsDir>/recorded
• paths.specsDir Optional override for published specs. Default: <testsDir>/published
• paths.supportDir Optional override for consumer-owned support files such as persona switch modules. Default: <testsDir>/support
• paths.authDir Optional override for saved auth state. Default: <rootDir>/auth/.auth
• paths.visualDir Optional override for visual regression assets. Default: <rootDir>/visual-regression
• paths.visualRunsDir Optional override for captured visual runs. Default: <visualDir>/runs
• paths.visualReportsDir Optional override for generated visual reports. Default: <visualDir>/reports
• reporting.visual.renderer Optional report renderer package or path. Default: @lockweb/qa-report
• suites Ordered collection of logical test buckets. The first suite listed is treated as the default suite.
• suites.<suite>.defaultRequiresLogin Default login requirement for that suite. Tests can still override this at the test level.
• suites.<suite>.personas Optional ordered map of persona keys to persona metadata. Extra fields are allowed and are passed through to the consumer persona support module unchanged.
• suites.<suite>.personaSupport.module Optional project-relative path to the consumer-provided persona switch module. This module should export switchPersona(page, personaKey, persona, context) and may export resetPersona(page, context).
• suites.<suite>.full Optional boolean controlling full-page visual capture and compare. Default: true
• suites.<suite>.selectors Optional ordered map of region names to selectors used for visual regression capture and compare. Each configured key becomes its own report section and can be combined with full-page capture.
• suites.<suite>.csvSets Ordered map of visual regression CSV names to project-relative file paths. The first CSV listed is treated as the default for that suite.
• suites.<suite>.targets Ordered map of runnable environments for that suite. The first target listed is treated as the default target for that suite.
• suites.<suite>.targets.<target>.run Optional boolean. Default: true. Controls whether the target appears in run flows.
• suites.<suite>.targets.<target>.record Optional boolean. Default: true. Controls whether the target appears in record flows.
• suites.<suite>.targets.<target>.visual Optional boolean. Default: true. Controls whether the target appears in visual capture flows.
• suites.<suite>.targets.<target>.writesContent Optional boolean. Default: true. Controls whether tests marked as content-writing are allowed to run on that target.
• suites.<suite>.targets.<target>.hosts Required non-empty object of host definitions for that target.
• suites.<suite>.targets.<target>.hosts.<hostKey>.baseUrl Required base URL for that host.
• suites.<suite>.targets.<target>.hosts.<hostKey>.aliases Optional array of additional absolute origins that should normalize to the same host key during publish. Use this for canonical-domain pairs such as https://example.test and https://www.example.test.
• suites.<suite>.targets.<target>.hosts.<hostKey>.requiresLogin Required login mode for that host. Use true for login required, "optional" for login available but not required, or false for no login handling.
• suites.<suite>.targets.<target>.hosts.<hostKey>.default Required on exactly one host per target. Used as the default start host and Playwright base URL for that target.

Default behavior:

• first suite listed = default suite
• first target listed in that suite = default target
• first CSV listed in that suite = default visual regression CSV
• full-page capture is enabled unless suites.<suite>.full is set to false
• target capability flags default to true unless explicitly set to false

This lets a project keep the config minimal while still allowing any path or suite-specific behavior to be overridden explicitly.

Persona behavior:

• if a suite does not define personas and persona support, publish does not ask about personas
• if a suite defines personas and a persona support module, publish offers those personas
• if a test references a persona that no longer exists in suite config, runtime fails with a clear message
• if a test does not set a persona:
  • no auth required = anonymous user
  • auth required = base logged-in user

Persona test API

Published specs can use createPersonaTest from @lockweb/qa-workflow/support/persona-test.

Persona-generated tests preserve the original spec callsite in Playwright UI and reports, so shared helpers do not shift those tests under the package's own node_modules path.

Shared-loop pattern:

import { test } from '@playwright/test';
import { createPersonaTest } from '@lockweb/qa-workflow/support/persona-test';

const personaTest = createPersonaTest(test, [
  { user: 'group_creator_osc' },
  { user: 'group_publisher_osc' },
]);

personaTest('shared flow', async ({ page, persona }) => {
  if (persona.user === 'group_creator_osc') {
    // creator branch
  }

  if (persona.user === 'group_publisher_osc') {
    // publisher branch
  }
});

Single-persona pattern:

import { test } from '@playwright/test';
import { createPersonaTest } from '@lockweb/qa-workflow/support/persona-test';

createPersonaTest(test, [{ user: 'group_creator_osc' }])(
  'creator-only flow',
  async ({ page }) => {
    // creator-only steps
  }
);

When personas are active, the callback fixtures include persona with:

• persona.key Persona key used for lookup in suite config
• persona.user Persona user value from the test declaration
• persona.record Full persona record from suite config
• persona.label Resolved display label used in the generated test title
• persona.expectation Optional pass or fail expectation from the test declaration

Auth host behavior:

• hosts with requiresLogin: true appear in Auth and are treated as login required
• hosts with requiresLogin: "optional" appear in Auth and are treated as login optional
• hosts with requiresLogin: false are excluded from Auth and other auth status surfaces

Repo Boundary

qa-workflow is the shared engine. Consuming repositories own their local setup and assets.

Package-owned:

• CLI entrypoints
• config loading and default resolution
• auth orchestration
• run / record / publish logic
• visual capture, compare, and reporting
• generic persona orchestration
• sanitized scaffolding in templates/

Consumer-owned:

• qa-workflow.config.json
• playwright.config.js
• suite definitions and target URLs
• visual regression CSV files
• published specs under test/published
• recorded specs under test/recorded
• saved auth state
• persona switch modules and other project-specific support code

If playwright.config.js is missing, qaw run may cause Playwright to scan unrelated tests in the consuming repo. The simplest fix is to run npx qaw init and create it.

This package should not contain real consumer target URLs, real consumer CSVs, published specs, recorded specs, auth state, or project-specific persona switch logic.

Maintainer Notes

The sections below are for package maintenance, not for consuming projects.

Package Layout

Current package structure:

• bin/ Thin CLI entrypoints such as qaw
• src/cli/ Command parsing, menu flow, setup, validation, and maintenance commands
• src/config/ Config loading, defaults, and target resolution
• src/auth/ Auth state and login helpers
• src/run/ Record, publish, run, and listing flows
• src/visual/ Visual capture, compare, report generation, and visual menu flow
• src/support/ Package-owned support helpers such as persona orchestration
• templates/ Sanitized files used by qaw init

Local Development

Common local development commands in this repo:

npm run qaw
npm run smoke:cli
npm test
npm run record
npm run run:interactive
npm run visual:interactive

The short CLI name exposed by the package is:

qaw

The package repo also includes a smoke check for CLI and config loading:

npm run smoke:cli

Maintainer-facing package tests run with:

npm test

Published Package Contents

Published package contents are intentionally limited to:

• bin/
• src/
• templates/
• README.md
• LICENSE