fmg-fvp-automation

Playwright (TypeScript) against the applications built on form.io: Form Manager, Form View Pro and PDF, plus USWDS, Open Source Serverless and EFBM.

Setup, the shared stack, the gate command, running against your own PR, the ratchet and the conventions all live one level up: ../README.md.

pnpm -F fmg-fvp-automation pw:component    # the gate — 214 tests, ~5 min

A PR review environment deploys formio-server, the portal and pdf-server — not Form Manager or Form View Pro. So this suite has nothing of its own to drive there, and unlike the formio and formio-js suites it isn't wired for a pr env. To test a server fix that a PDF spec here would exercise, run the formio suite's PDF specs against the PR instead:

PR_NUMBER=<your PR> pnpm -F formioautomation pw:pr:api   # includes the api/*pdf* specs

See Run it against your own PR for the two PR shapes and what each does differently.

What the gate covers

playwright/tests/component/{fmg,fvp,pdf} — copies of the fastest hermetic local-only specs from the three main applications: no third-party calls, nothing skipped, nothing flaky. Unlike the e2e suites they are hermetic and order-independent, and the local project ignores the folder so the copies never double-run.

214 tests · 214 passed · 0 failed        ~5 min

There is no expected-skip list here: the gate is meant to be entirely green.

What CI runs, and you do not

Seven numbered suites under playwright/tests/e2e/, each split into numbered subfolders that run in order — testIsolation is effectively off inside a suite, so specs depend on earlier state:

| Module | Folder | Runs on | | --- | --- | --- | | FMG | 1-form-managerr | local :3000 (+ the OAuth spec on a deployed env) | | FVP | 2-form-viewpro | local :3000 (+ the OAuth spec on a deployed env) | | PDF | 3-pdf | local :3000 (+ 12-remote-env-tests on a deployed env) | | PDF-USWDS | 4-pdf-uswds | local :3000 (+ 08/02 designer-access on remote-uswds, VPN) | | PDF-performance | 5-pdf-performance-testing | local :3000 | | OSS | 6-open-source-serverless | the open-source app on :3001 | | EFBM | 7-efbm-and-demo | local :3000, new headless | | hosted | 8-hosted | portal.test-form.io, VPN — the only hosted coverage |

pnpm run pw:fmg · pw:fvp · pw:pdf · pw:pdf-uswds · pw:efbm
pnpm run pw:pdf-performance-100 · pw:pdf-performance-300    # headed, workers=1
pnpm run oss-up && pnpm run pw:oss && pnpm run oss-down     # the separate :3001 stack

The USWDS suites need the current uswds-viewer image instead of pdf-server — the shared stack takes that as a swap: PDF_IMAGE=formio/uswds-viewer PDF_TAG=<tag>.

The deployed-env halves in that table are yours to run, not CI's. pw-manual passes deployed_env through to the suite, but it runs only the local project — the additive --project=remote / remote-uswds specs are not wired into it.

Layout

playwright.config.ts defines 11 projects. The ones with a reason worth knowing:

| Project | Why it exists | | --- | --- | | component | the pre-PR gate; local ignores its folder so the copies never double-run | | efbm | same local backend as local, but channel: "chromium" — the demo app's client-side licence check needs crypto.subtle, which the default headless shell does not expose (it fails "Invalid key provided" only in headless) | | hosted-setup → hosted | logs in as the static account to capture a storageState, then runs 8-hosted | | remote / remote-uswds | the OAuth and deployed-PDF specs; remote-uswds needs VPN | | oss-setup → oss → oss-cleanup | the :3001 app, made hermetic by a per-run disposable admin |

Suites are selected by path, e.g. pnpm exec playwright test playwright/tests/e2e/3-pdf/01-pdf-smoke-test.

Worth knowing before you edit a spec

• Mount pattern. Reach a form's render by navigating straight to #/project/<id>/form/<id>/ (@utils/mount.mountForm) instead of walking the product (portal home → project search → card → Forms nav → form row → "Use" tab). @support/mount-scope.setupMountScope wraps the whole per-spec setup. New or converted specs should use it; the portal_walk ratchet metric may only go down.

• The .env file is a personal override, not a default. It is gitignored, so a fresh clone has none and everything points at localhost:3000. If you inherit one, check it — PW_BASE_URL=http://127.0.0.1:3100 belongs to a remapped stack, and against the ordinary stack that address answers ECONNREFUSED, the setup project fails, and every spec reports "did not run".

• The hosted folder is a curated smoke suite, deliberately non-flaky: 01-fmg (including an SMTP email action), 02-fvp, 03-pdf and 04-sso-oauth (including a live Auth0 round-trip). Its PDF API and usage legs are parked on FIO-12071.

• CI is commit-message driven — the subject must start with a version and name suite aliases (fmg, fvp, pdf, pdf-uswds, perf, oss, efbm), split on space / & / comma but never -, which appears inside the version itself. A standalone hosted token runs only 8-hosted and overrides the aliases:

  git commit -m "9.8.4-rc.1 fmg fvp efbm"             # local suites
  git commit -m "9.9.0-rc.1 fmg&fvp&efbm remote-dev"  # …with OAuth on remote-dev
  git commit -m "9.9.1-rc.0 hosted"                   # ONLY the 8-hosted folder
  git commit -m "9.8.4-rc.1"                          # no alias → skipped

• Per-suite agent notes and results tables live in doc/agent/ (local-only).