@afixt/usecase-runner

Convert structured accessibility use case definitions (YAML) into executable Playwright test scripts.

@afixt/usecase-runner bridges the gap between accessibility test planning and test automation. Author use cases in a human-readable YAML format using a domain-specific step syntax, and the tool generates Playwright tests that target elements exclusively through their ARIA roles and accessible names. If an element isn't properly exposed in the accessibility tree, the test fails — by design.

Why This Exists

Manual accessibility use case testing is thorough but slow. Existing test automation frameworks don't enforce accessibility-first element targeting. This tool brings those two worlds together:

• QA testers write use cases in plain YAML — no JavaScript required
• Generated tests use getByRole() and getByLabel() exclusively — never CSS selectors or XPath
• Every test failure is an accessibility finding: a missing label, an incorrect role, a broken keyboard interaction
• Reports match the same tabular format used in manual use case test deliverables

Installation

npm install @afixt/usecase-runner

Playwright is a peer dependency. Install it separately if you haven't already:

npm install @playwright/test
npx playwright install

Requirements: Node.js >= 22.0.0

Quick Start

1. Initialize a sample use case

npx usecase-runner init

This creates sample-login.uc.yaml and usecase-runner.config.yaml in the current directory.

2. Write a use case

# login-success.uc.yaml

id: login-success
title: 'Log In To The System'
type: positive
description: 'Verify that a user with valid credentials can log in and reach the dashboard.'

preconditions:
  - 'User has a valid username and password'
  - 'User is not already logged in'

start_location: 'https://www.example.com'

expected_result: 'The user will be logged in and redirected to the dashboard'

data:
  username: '[email protected]'
  password: 'xyzpdq123!)'
  dashboard_url: 'https://www.example.com/dashboard'

steps:
  - locate: link "Client Sign In"
  - focus: link "Client Sign In"
  - activate: link "Client Sign In"
  - verify: url "https://www.example.com/login.php"
  - locate: field "username"
  - focus: field "username"
  - enter: field "username" value "{{ username }}"
  - locate: field "password"
  - focus: field "password"
  - enter: field "password" value "{{ password }}"
  - locate: button "Login"
  - focus: button "Login"
  - activate: button "Login"
  - verify: url "{{ dashboard_url }}"

3. Validate the use case

npx usecase-runner validate login-success.uc.yaml

4. Generate Playwright tests

npx usecase-runner generate login-success.uc.yaml --outdir ./tests/generated

5. Run the generated tests

npx playwright test ./tests/generated

Or generate and run in one step:

npx usecase-runner generate login-success.uc.yaml --outdir ./tests/generated --run

Step DSL Reference

Steps follow the pattern: keyword: role_or_type "accessible_name" [modifiers]

Core Keywords

| Keyword | Purpose | Example | |---------|---------|---------| | locate | Assert element is visible in the accessibility tree | locate: button "Submit" | | focus | Move keyboard focus and verify it was received | focus: field "Email" | | enter | Type a value into a text input | enter: field "Email" value "[email protected]" | | select | Choose from checkbox, radio, or select controls | select: checkbox "I agree" | | activate | Click or keyboard-activate a button or link | activate: button "Submit" | | verify | Assert a condition is true | verify: url "/dashboard" |

Supplementary Keywords

| Keyword | Purpose | Example | |---------|---------|---------| | wait | Explicit wait (ms) | wait: 2000 | | wait_for | Wait for a condition | wait_for: text "Loading complete" | | navigate | Direct URL navigation | navigate: 'https://example.com/page' | | screenshot | Capture a screenshot | screenshot: 'error-state' | | keyboard | Raw keyboard input | keyboard: 'Tab' | | scroll | Scroll element into view | scroll: to text "Footer" | | note | Annotation (no action) | note: 'Check visual layout' |

Role Tokens

| Token | Playwright Locator | Notes | |-------|-------------------|-------| | link | getByRole('link', { name }) | | | button | getByRole('button', { name }) | | | field | getByLabel(name) | Inputs, textareas | | checkbox | getByRole('checkbox', { name }) | | | radio | getByRole('radio', { name }) | | | select | getByRole('combobox', { name }) | Dropdown / <select> | | heading | getByRole('heading', { name }) | Optional level: N | | tab | getByRole('tab', { name }) | | | dialog | getByRole('dialog', { name }) | | | menu | getByRole('menu', { name }) | | | menuitem | getByRole('menuitem', { name }) | | | region | getByRole('region', { name }) | Landmarks | | navigation | getByRole('navigation', { name }) | | | text | getByText(name) | Visible text | | image | getByRole('img', { name }) | | | table | getByRole('table', { name }) | | | row | getByRole('row', { name }) | | | cell | getByRole('cell', { name }) | |

Custom ARIA roles: locate: role "treegrid" name "File Browser"

Verify Sub-types

| Sub-type | Example | What It Checks | |----------|---------|----------------| | url | verify: url "/dashboard" | Page URL | | title | verify: title "Dashboard" | Page title | | text | verify: text "Welcome" | Visible text | | heading | verify: heading "Welcome" | Heading role | | alert | verify: alert "Form has errors" | role="alert" content | | field_error | verify: field_error "password" | aria-invalid + aria-describedby | | field_value | verify: field "Email" has_value "[email protected]" | Input value | | visible | verify: visible button "Submit" | Element visibility | | hidden | verify: hidden dialog "Confirm" | Element hidden | | enabled | verify: enabled button "Submit" | Not disabled | | disabled | verify: disabled button "Submit" | Disabled state | | checked | verify: checked checkbox "Remember Me" | Checked state | | focus | verify: focus field "Email" | Has focus | | count | verify: count link "Remove" is 3 | Element count | | attribute | verify: field "Email" attribute "aria-required" is "true" | Attribute value | | live_region | verify: live_region "3 results found" | aria-live region content | | download | verify: download "report.pdf" | File download |

Extension / Alternate Cases

Extension cases reference a parent and override steps from a given point, useful for testing error paths:

id: login-error-short-password
title: 'Log In — Alt 1 (Error State)'
extends: login-success
type: negative
description: 'Trigger error handling by providing an invalid password'

preconditions: []

data:
  password: 'abc'  # overrides parent value

steps_override:
  from_step: 11    # after entering the password
  steps:
    - 'activate: button "Login"'
    - 'verify: alert "Errors are preventing successful submission of this form."'
    - 'verify: field_error "password"'

Template Variables

Steps support {{ variable }} syntax for data-driven testing. Variables are resolved from the use case data block or external data files:

data:
  username: '[email protected]'

steps:
  - enter: field "Username" value "{{ username }}"

External shared data files can be configured in usecase-runner.config.yaml:

data_files:
  - ./data/test-accounts.yaml
  - ./data/urls.yaml

Two Execution Modes

Code Generation (recommended for CI/CD)

Generates standalone .spec.ts files that can be committed to version control and run with npx playwright test:

npx usecase-runner generate ./usecases/ --outdir ./tests/generated/

Direct Execution

Runs use cases in-process without generating intermediate files, for ad-hoc testing:

npx usecase-runner run ./usecases/login-success.uc.yaml --browser chromium --headed

Programmatic API

import {
  parseUseCaseFile,
  generatePlaywrightTest,
  runUseCase
} from '@afixt/usecase-runner';

// Parse a use case file
const useCase = await parseUseCaseFile('./usecases/login-success.uc.yaml');

// Generate a Playwright test file
const testCode = generatePlaywrightTest(useCase);

// Or run directly
const result = await runUseCase(useCase, {
  browser: 'chromium',
  headed: false,
  screenshotOnFailure: true,
});

console.log(result.score); // 'pass' | 'pass_with_conditions' | 'fail'

CLI Reference

usecase-runner <command> [options]

Commands:
  generate <path>    Generate Playwright .spec.ts files from use case YAML
  run <path>         Execute use cases directly and produce reports
  validate <path>    Validate use case YAML files without running them
  init               Create sample use case file and config in current directory

Options:
  --config, -c       Path to config file (default: ./usecase-runner.config.yaml)
  --outdir, -o       Output directory for generated tests (generate mode)
  --browser, -b      Browser to use: chromium, firefox, webkit
  --headed           Run in headed mode
  --run              Generate and immediately run (generate mode)
  --set key=value    Override test data variables
  --report, -r       Report format: json, html (comma-separated for multiple)
  --verbose, -v      Verbose output
  --help, -h         Show help

Configuration

Create usecase-runner.config.yaml in your project root (or run usecase-runner init):

browser: chromium
headed: false
slow_mo: 0
timeout: 30000
viewport:
  width: 1280
  height: 720

continue_on_failure: true
screenshot_on_failure: true
screenshot_dir: ./screenshots

report_formats:
  - json
  - html
report_dir: ./reports

data_files:
  - ./data/test-accounts.yaml

Scoring

Each use case run produces a score:

| Score | Meaning | |-------|---------| | Pass | All steps completed successfully | | Pass w/ Conditions | All steps completed but with retries, timeout adjustments, or warnings | | Fail | One or more steps could not be completed |

With continue_on_failure: true (the default), execution continues past failures to gather maximum diagnostic information.

Architecture

YAML Files (.uc.yaml)
        │
        ▼
   ┌─────────────┐
   │   Parser     │  YAML ingestion, Zod validation, step parsing,
   │ + Validator  │  {{ variable }} interpolation
   └──────┬──────┘
          │
          ▼
   ┌─────────────┐
   │ UseCase AST │  In-memory representation
   └──┬──────┬───┘
      │      │
      ▼      ▼
 CodeGen   Executor     Two execution paths
 (.spec.ts) (direct)
      │      │
      ▼      ▼
   ┌─────────────┐
   │  Reporter   │  JSON / HTML output
   └─────────────┘

Module Structure

| Module | Purpose | |--------|---------| | src/parser/ | YAML ingestion, Zod schema validation, step string parsing, template interpolation | | src/codegen/ | Transforms UseCase AST into Playwright .spec.ts files via Handlebars templates | | src/runner/ | Direct execution against a live Playwright page, plus reporting | | src/types/ | Core TypeScript interfaces and constants | | src/cli.ts | CLI entry point |

Design Philosophy

This tool is intentionally opinionated about accessibility:

1. Role-based locators only. Elements must be discoverable via getByRole() or getByLabel(). No CSS selector or XPath fallback exists in the DSL.

2. locate tests the accessibility tree, not the DOM. A visually hidden element that's also hidden from assistive technology will fail.

3. focus tests keyboard access. If an interactive element can't receive keyboard focus, it's a finding.

4. verify: alert requires role="alert". Error messages must be announced via live regions, not just visually displayed.

5. verify: field_error checks the full pattern: aria-invalid="true" on the field AND an associated error message via aria-describedby or aria-errormessage.

6. No escape hatch. If you can't target something by its accessible name or role, that's a finding, not a limitation.

Development

npm run build          # Compile TypeScript
npm run dev            # Watch mode
npm test               # Run tests
npm run test:coverage  # Tests with coverage
npm run lint           # ESLint
npm run format         # Prettier

License

MIT