Framestate

A config-driven Playwright test runner built around visual-regression (screenshot baseline diffing) testing, with an interactive terminal-based failure/debug flow. It knows nothing about any particular project — you point it at your project with a config file and a few small plugin modules.

Requirements

Framestate only targets Linux, running Wayland with sway, and assumes its dependencies are already installed — it does not attempt to be cross-platform or auto-install anything:

• Node.js 18+
• A Chromium build (installed via npx playwright install chromium)
• The desktop tools used by the interactive failure/debug flow, all resolved via $PATH and swappable in config (tools.*): a terminal (default foot), an editor (default nvim), an image viewer (default imv), jq, tidy, and (optionally) a firefox-developer-edition build
• dbus-monitor, if any of your tests use match_notification to assert on desktop notifications
• adb, only if you re-enable the (currently disabled) Android/waydroid test target — see src/browser.mjs

Install

npm install
npx playwright install chromium

Configure

Copy framestate.config.example.mjs to framestate.config.mjs in your project (or anywhere, and pass --config <path>), and fill in your project's paths, server origins, and browser defaults. See that file for every available field.

Framestate discovers tests itself: it recursively scans testDir (relative to projectRoot) for files named *.test.mjs, each of which must default-export an object keyed by test id. See src/testCollector.mjs.

Plugins

Three pieces of project-specific logic aren't part of framestate itself — they're supplied as plugin modules referenced from plugins.* in your config, resolved relative to the config file:

• resolver (optional) — export generateResolverValues(test, options), called on each test before it runs to resolve any dynamic values in its definition. Defaults to the identity function.
• testData (optional) — default-export an object with a current_time field, used to seed test_runtime.current_time. Defaults to the current time.
• webhooks (optional) — export primeWebhooks(test_id), used to back test_runtime.webhooks. If omitted, using test_runtime.webhooks in a test throws a clear error explaining the plugin is missing.

Run

npx framestate                    # run all tests
npx framestate my-test-id         # filter to a specific test
npx framestate --headed           # run non-headless
npx framestate --config ./path/to/framestate.config.mjs

See src/cli.mjs for the full flag list (--skip, --slow-mo, --trace, --start-from, --poke-around, --only-test-on-browser, --no-timeout, --view-har, --har-filter, --reset).

--reset deletes the skip-cache file (assets_used.json) outright before the run starts, so every test has no last_passed_on_commit to skip on and actually runs. Use this after changing something the skip-cache can't see on its own — most notably an untracked/gitignored config file like .env.local, since the skip-cache is git-diff based and only ever notices changes to committed files.

On a screenshot/match failure, framestate drops into an interactive prompt in the terminal it was run from (y to accept the new baseline, n to rerun, d/hd/har/p/f/fd to open various diff/HAR/HTML/screenshot viewers, sql for an ad-hoc SQL REPL against your backend, reimport to hot-reload the test file, o to open the file the test is defined in, u to print the current URL, nh/poke to rerun headed/interactively). Type anything else to see this list again.

Architecture

src/
  index.mjs        entrypoint — loads config, launches the browser, runs tests
  cli.mjs          CLI flag parsing
  config.mjs        config loading, validation, plugin resolution
  state.mjs          shared runtime state (mutexes, browser/context handles, caches)
  browser.mjs         browser/context lifecycle, page instrumentation
  runner.mjs           test discovery + the per-test orchestration loop
  testCollector.mjs     recursively finds and imports "*.test.mjs" files under testDir
  lifecycle.mjs         pass/fail handling, the interactive failure REPL, cleanup
  capture.mjs            screenshot/HTML capture, baseline diffing (pixelmatch)
  viewers.mjs              desktop-tool launchers for the failure REPL
  logging.mjs               structured logging (Roarr), browser console capture
  notifications.mjs          dbus-monitor based desktop notification matching
  backend.mjs                 HTTP calls to your app's test-support backend
  gitCache.mjs                  git-diff based "skip unchanged tests" caching, gated on framestate's own version
  version.mjs                    reads framestate's own package.json version
  metadata.mjs                   assets_used.json / captured-requests I/O

Known gap

Rerunning a failed test with n from the interactive failure prompt deadlocks if that test was using the shared default browser context (the common case) — see the note in src/lifecycle.mjs next to default_context_lock_release. This was already broken in the original tool (it called a method that was never defined) and hasn't been fixed here, since fixing it means designing the missing synchronization rather than just relocating code. Reproduce with a failing default-context test and press n.