Wave Readiness Validation — Functional & Technical Reference

Start here if you found this on npm. This package installs a GitHub Actions pipeline that copies your Business Central production environment to a sandbox, upgrades that sandbox to the next wave, and replays your recorded business processes against it — so you find out what the update breaks before your users do.

The full setup guide, WAVE-READINESS-SETUP.md, ships inside the package. To read it before installing anything, run this from any folder:

npx @inmindtechnologies/bc-wave-readiness guide

It writes WAVE-READINESS-SETUP.md there and nothing else. Start at its phase 0; the guide tells you when to come back and install. Running it in the repository you will install into is fine — the installer recognises the file as its own and does not treat it as a collision. The source repository is private, so links from this page into it will not open for you — everything the guide needs is in the package.

1. Overview

This repository implements Wave Readiness Validation for Microsoft Dynamics 365 Business Central. It contains a fully automated CI/CD pipeline that replays recorded UI business scenarios (page scripts) against a sandbox environment — optionally upgraded to the next BC wave — to detect regressions before updates reach production.

The core tool is @microsoft/bc-replay, which replays .yml page-script recordings through a headless Playwright browser against a live BC web client.

High-level flow

 ┌───────────────────────────────────────────────────────────────┐
 │                  workflow_dispatch (manual)                    │
 │  Inputs: targetVersion, scriptArea, refreshSandbox             │
 └────────────────────────────┬──────────────────────────────────┘
                              │
                 ┌────────────▼────────────┐
                 │   setup  (ubuntu)        │  Scan PageScriptLibrary/
                 │   Build area matrix      │  for .yml scripts
                 └────────────┬────────────┘
                              │
              ┌───────────────▼───────────────┐
              │  copy-and-upgrade (windows)     │
              │  1. Delete sandbox              │
              │  2. Copy prod → sandbox         │
              │  3. Schedule platform upgrade   │
              │  4. Poll until upgrade complete  │
              └───────────────┬───────────────┘
                              │
              ┌───────────────▼───────────────┐
              │  master-data (windows)          │  Phase 1
              │  Run MasterData - 1/ scripts   │  (vendors, items, setup)
              └───────────────┬───────────────┘
                              │
              ┌───────────────▼───────────────┐
              │  replay (windows)               │  Phase 2
              │  Sequential per-area replay    │  (Purchase, Invoice,
              │  sorted by folder name suffix   │   Return, Payment)
              └───────────────┬───────────────┘
                              │
              ┌───────────────▼───────────────┐
              │  deploy-report (ubuntu)         │
              │  1. Build per-run HTML report   │
              │  2. Persist to history branch   │
              │  3. Deploy to Azure App Service │
              │  4. Open/close GitHub issues    │
              └───────────────┬───────────────┘
                              │
              ┌───────────────▼───────────────┐
              │  notify (ubuntu)                │
              │  Send email via MS Graph        │
              └───────────────────────────────┘

2. Repository Structure

bc-wave-readiness/
├── .github/
│   ├── actions/
│   │   ├── acquire-entra-token/action.yml    # Composite action: OAuth token
│   │   └── setup-bc-replay/
│   │       ├── action.yml                    # Composite action: install bc-replay
│   │       └── strip-steps-reporter.js       # Playwright reporter patch
│   ├── scripts/
│   │   ├── build-report-site.sh              # Generate aggregated HTML report
│   │   ├── Get-EntraToken.ps1                # Client-credentials token, masked
│   │   ├── Get-ReplayFailureCause.ps1        # One-line cause for a failed recording
│   │   ├── Invoke-ReplayArea.ps1             # Run all scripts in one area
│   │   ├── manage-failure-issues.sh          # Auto-manage GitHub Issues
│   │   └── Resolve-BcUpgrade.ps1             # Resolve & schedule BC upgrade
│   ├── workflows/
│   │   ├── WaveReadinessValidation.yaml      # Main CI workflow
│   │   ├── UpdateWaveReadiness.yaml          # Weekly self-update → PR (needs UPDATE_TOKEN)
│   │   └── tests.yaml                        # Source-repo test suite (not installed)
│   └── dependabot.yml                        # Weekly bumps for the pinned actions
├── PageScriptLibrary/                        # Page script recordings (ships empty — record your own)
│   ├── README.md                             # Library usage guide
│   └── MasterData - 1/                       # Phase 1 (reserved): seed-data scripts run first
├── infra/
│   └── report-site.bicep                     # Azure App Service for the report site
├── tests/                                     # Source-repo test suite (not installed)
│   ├── run-tests.ps1                         # Runner: pwsh ./tests/run-tests.ps1
│   ├── mutation-check.ps1                    # Proves the suite can actually fail
│   ├── Invariants.Tests.ps1                  # Static facts about workflow/installers/docs
│   └── Installer.Tests.ps1                   # setup-pipeline.ps1 against throwaway repos
├── fleet/                                     # Fleet-operator layer (optional for forks)
│   ├── onboard-customer.ps1                  # One-command per-customer onboarding
│   ├── ONBOARDING.md                         # Fleet onboarding guide
│   └── docs/                                  # Internal design specs & plans
├── bc-wave-readiness.code-workspace          # VS Code workspace config
├── setup-pipeline.ps1                        # Installer — run from the target repo
├── LICENSE                                    # MIT
├── README.md                                 # This file (architectural reference)
└── WAVE-READINESS-SETUP.md                                  # Step-by-step provisioning guide for new tenants

Pipeline vs fleet layers. Everything above fleet/ is the generic pipeline that setup-pipeline.ps1 installs into an adopter's repo. fleet/ holds inMind's multi-customer onboarding automation and internal design docs; it is never installed into a target repo.

The library ships empty — every project records its own scenarios against the customer's data. Scripts are discovered by .yml extension and replayed in natural file-name order, where a number inside the name sorts by value (Script 2 runs before Script 10). See PageScriptLibrary/README.md for recording instructions and WAVE-READINESS-SETUP.md for the full adoption checklist.

3. Workflow: WaveReadinessValidation.yaml

Location: .github/workflows/WaveReadinessValidation.yaml

3.1 Trigger

Manual only (workflow_dispatch) with four inputs:

| Input | Type | Default | Description | |---|---|---|---| | targetVersion | string | latest | BC version to upgrade sandbox to. Accepts latest, bare major (28), or exact N.M (28.3). Ignored when skipUpgrade is checked. | | skipUpgrade | boolean | false | Skip the platform upgrade entirely and replay at the current sandbox version. Overrides targetVersion. The report, failure issues and email then show the version the sandbox is actually on. | | scriptArea | string | '' | Restrict replay to one area folder (e.g. Purchase - 2). Blank = all. | | refreshSandbox | boolean | true | Delete + copy production → sandbox before replay. Uncheck to reuse existing sandbox. |

3.2 Permissions & Concurrency

• Permissions: contents: read (workflow-level). The deploy-report job overrides with contents: write (push history branch), issues: write (manage failure issues), and id-token: write (OIDC federated login to Azure).
• Concurrency group: wave-readiness — only one run at a time, does not cancel in-progress.

3.3 Environment Variables

Defined at the workflow level (top of WaveReadinessValidation.yaml). Values can be overridden per-environment via GitHub repository variables — see WAVE-READINESS-SETUP.md for the full list.

| Variable | Default | Override | Purpose | |---|---|---|---| | SOURCE_ENV | PRODUCTION | vars.BC_SOURCE_ENV | Production BC environment name | | TARGET_ENV | SANDBOX-Waves | vars.BC_TARGET_ENV | Sandbox BC environment name | | ADMIN_API_BASE | https://api.businesscentral.dynamics.com/admin/v2.28/applications/BusinessCentral | vars.BC_ADMIN_API_BASE | BC Admin Center API base URL | | MASTER_DATA_AREA | MasterData - 1 | (none) | Reserved folder for seed data | | REPORT_SITE_URL | — | vars.REPORT_SITE_URL | Azure App Service base URL (trailing slash required) | | NOTIFY_FROM | (unset) | vars.NOTIFY_FROM | From mailbox for Graph sendMail (must exist in the tenant). Unset ⇒ email step skipped. | | NOTIFY_TO | (unset) | vars.NOTIFY_TO | To recipient for the report email. Unset ⇒ email step skipped. |

3.4 Required Secrets

| Secret | Purpose | |---|---| | BC_ADMIN_TENANT_ID | Entra tenant ID | | BC_ADMIN_CLIENT_ID | Entra app (client) ID with AdminCenter.ReadWrite.All | | BC_ADMIN_CLIENT_SECRET | Entra app client secret | | BC_REPLAY_USER | BC user with PAGESCRIPTING-PLAY permission set | | BC_REPLAY_PASSWORD | BC user password | | BC_REPLAY_TOTP_SECRET | (Optional) TOTP secret for MFA | | AZURE_DEPLOY_CLIENT_ID | Federated identity client ID for azure/login (OIDC) | | AZURE_DEPLOY_TENANT_ID | Federated identity tenant ID for azure/login (OIDC) | | AZURE_DEPLOY_SUBSCRIPTION_ID | Subscription that owns the App Service |

Repository variables (Settings → Variables → Actions):

| Variable | Purpose | |---|---| | REPORT_SITE_URL | Public URL of the Azure App Service (e.g. https://wave-readiness-reports.azurewebsites.net/). Trailing slash required. | | AZURE_WEBAPP_NAME | App Service resource name used by azure/webapps-deploy. | | BC_SOURCE_ENV / BC_TARGET_ENV | (Optional) Override the prod / sandbox environment names defined in section 3.3. | | BC_ADMIN_API_BASE | (Optional) Override the BC Admin API base URL — needed if Microsoft bumps the API version. | | NOTIFY_FROM / NOTIFY_TO | (Optional) Override the notification email From / To. |

3.5 Jobs

Job 1: setup

• Runner: ubuntu-latest
• Purpose: Discover script areas under PageScriptLibrary/ and build a JSON matrix.
• Logic: Scans for subdirectories containing .yml files. Separates the master-data area from the remaining replay areas. Respects the scriptArea input filter.
• Outputs: areas (JSON array), hasScripts, hasAreas, masterDataArea.

Job 2: copy-and-upgrade

• Runner: windows-latest | Timeout: 300 min
• Depends on: setup
• Purpose: Prepare the sandbox environment.
• Steps:
  1. Acquire Entra token — client-credentials OAuth flow, via Get-EntraToken.ps1.
  2. Delete existing sandbox — DELETE /environments/{name}, poll until 404.
  3. Copy prod → sandbox — POST /environments/{source}/copy, then poll the operation until Succeeded. The poll mints its own token and renews it every 45 minutes.
  4. Re-acquire token — copy can outlast the 60-min token lifetime, and the schedule step needs a live one.
  5. Schedule upgrade — calls Resolve-BcUpgrade.ps1 to resolve the target version and PATCH it.
  6. Poll upgrade — monitors the Update operation for up to four hours, renewing its token every 45 minutes; warns after 1h of queued state.
  7. Warm-up wait — 90s sleep for sandbox to warm up after copy/upgrade.
• Outputs: resolvedVersion, resolvedType, upgradeSkipped, currentVersion, skipReason.

Job 3: master-data

• Runner: windows-latest | Timeout: 60 min
• Depends on: setup, copy-and-upgrade
• Condition: masterDataArea is non-empty.
• Purpose: Run all scripts under MasterData - 1/ to seed vendors, items, G/L setup.
• Failure behavior: Throws and blocks all downstream phases (the data they need won't exist).
• Artifact: replay-results-MasterData - 1 (Playwright reports + JUnit XML).

Job 4: replay

• Runner: windows-latest | Timeout: 180 min
• Depends on: setup, copy-and-upgrade, master-data
• Condition: Areas exist AND master-data succeeded or was skipped.
• Purpose: Run remaining areas sequentially in folder-name sort order.
• Failure behavior: Logs errors per area but continues to the next. Throws after all areas finish if any failed.
• Artifact: replay-results (all areas merged).

Job 5: deploy-report

• Runner: ubuntu-latest
• Depends on: all previous jobs | Condition: always() + scripts were discovered.
• Purpose: Build and publish the HTML report site.
• Steps:
  1. Restore the wave-readiness-history branch as a working tree under history/ and note its tip.
  2. Download all per-area artifacts.
  3. Run build-report-site.sh to generate per-run and top-level index pages.
  4. Write the tree as a single parentless commit and force-push it to the history branch, with a lease on the tip noted in step 1. The branch is a snapshot of the current site, never a log of past commits (see 7.1).
  5. Package the site (plus a tiny zero-dep Node static server that resolves /foo/ → /foo/index.html) into a zip.
  6. Login to Azure via OIDC federated identity (azure/login@v2) and deploy the zip with azure/webapps-deploy@v3 to the configured App Service. Entra Easy Auth on the App Service handles end-user authentication.
  7. Run manage-failure-issues.sh to open/comment/close GitHub Issues.
  8. Write a rich step summary with PASS/FAIL headline, version info, and links.
• Outputs: pageUrl (App Service base URL), runUrl (this run's per-report URL), total, failed.

Job 6: notify

• Runner: ubuntu-latest
• Depends on: all previous jobs | Condition: always() + scripts were discovered.
• Purpose: Send the notification email.
• Channel: Microsoft Graph sendMail API. Sends an HTML email with pass/fail summary, version info, and report links.

4. Composite Actions

4.1 acquire-entra-token

Location: .github/actions/acquire-entra-token/action.yml

Purpose: Perform a standard OAuth 2.0 client-credentials flow against Microsoft Entra ID to obtain a bearer token.

| Input | Required | Description | |---|---|---| | tenant-id | Yes | Entra tenant ID | | client-id | Yes | Entra app (client) ID | | client-secret | Yes | Entra app client secret | | scope | Yes | OAuth scope (e.g. https://api.businesscentral.dynamics.com/.default) |

Output: token — the bearer access token, masked with ::add-mask:: so it never appears in logs.

Security: The token is immediately masked. Even if a downstream step accidentally echoes it, GitHub Actions will redact it from the log output.

4.2 setup-bc-replay

Location: .github/actions/setup-bc-replay/action.yml

Purpose: Install @microsoft/bc-replay and patch its Playwright HTML reporter to strip sensitive step data.

Steps:

1. Install Node.js (default v24).
2. Create a minimal package.json in $RUNNER_TEMP and npm install @microsoft/bc-replay.
3. Copy strip-steps-reporter.js into bc-replay's dist/ directory.
4. Patch playwright.config.js to use the custom reporter instead of the default html reporter.

Why the patch? bc-replay's default HTML report records every page.fill() argument as a Playwright step — including the replay user's password. The custom reporter strips all steps arrays from test results before the report is written. The patch also disables Playwright tracing entirely (trace: 'off'), since trace zips serialize the same page.fill() arguments in their internal action log and would leak the password via the published Pages site. Video and screenshots are kept — screenshots are stored as attachments (not steps) so strip-steps-reporter.js leaves them intact, and BC's login form masks the password input so neither video nor screenshots expose credentials.

5. Scripts

5.1 Invoke-ReplayArea.ps1

Location: .github/scripts/Invoke-ReplayArea.ps1
Language: PowerShell
Called by: master-data and replay jobs.

Purpose: Replay every .yml page script within a single PageScriptLibrary/ area, sequentially.

Parameters:

| Parameter | Required | Description | |---|---|---| | AreaName | Yes | Folder name under PageScriptLibrary/ (e.g. Purchase - 2) | | SandboxUrl | Yes | Full BC web client URL (e.g. https://businesscentral.dynamics.com/{tenant}/{env}/) | | WorkspaceRoot | Yes | Repository checkout root |

Behavior:

1. Lists all .yml files in the area folder in natural name order: numbers inside a name compare by value, so Script 2 runs before Script 10.
2. For each script, invokes npx --no-install replay from $RUNNER_TEMP (where setup-bc-replay installed it).
3. Passes authentication via environment variable names (BC_USER, BC_PASS, BC_TOTP) — never sees raw credentials.
4. Writes results to replay-results/{AreaName}/{ScriptName}/ (Playwright report + JUnit XML).
5. For each failed script, runs Get-ReplayFailureCause.ps1 and writes its one-line answer to failure-cause.txt next to results.xml, prints it in the log, and emits a ::error annotation titled with the script name.
6. Returns a [pscustomobject] with Area, Total, Failed, Causes — does not throw on failures. The caller decides the error strategy.

5.1a Get-ReplayFailureCause.ps1

Location: .github/scripts/Get-ReplayFailureCause.ps1
Language: PowerShell
Called by: Invoke-ReplayArea.ps1, once per failed script.

Purpose: Turn what bc-replay leaves in a script's result directory into one line that names the actual cause. bc-replay's own console output for a failure is only "One or more test recordings failed."

Parameter: ScriptResultDir — the directory bc-replay was given as -ResultDir.

Sources, in order of precision:

| Source | What it gives | When present | |---|---|---| | bc-replay's replay log attachment | The failed step's number, description and message, e.g. Step 7 "Validate Balance ($) is 0" failed: Was expecting '0' but got '32538'. | bc-replay caught the error itself (validate step, missing control) | | results.xml | Playwright's failure text per attempt: timeouts, locator errors | Always, once Playwright ran | | error-context.md | What was on screen at the moment of failure: an unanswered dialog (quoted), or the Microsoft sign-in / MFA page | Timeouts |

Attempts: Playwright retries a failed recording once. The final attempt decides the outcome, so its cause is reported; when an earlier attempt failed differently, that is appended in brackets so a flaky sign-in is not mistaken for the real problem (or vice versa).

Output: exactly one line, never empty. The script never throws: a parser error degrades to a generic pointer at the report so it cannot mask the original failure.

5.2 Resolve-BcUpgrade.ps1

Location: .github/scripts/Resolve-BcUpgrade.ps1
Language: PowerShell
Called by: copy-and-upgrade job.

Purpose: Resolve a user-supplied version alias against the BC Admin API's /updates endpoint, then PATCH the selected version to schedule the sandbox upgrade.

Parameters:

| Parameter | Required | Description | |---|---|---| | Token | Yes | Bearer token with admin API access | | AdminApiBase | Yes | BC Admin API base URL | | EnvironmentName | Yes | Sandbox environment name | | RequestedVersion | Yes | latest, bare major (28), or exact N.M (28.3) |

Version resolution logic:

| Input | Behavior | |---|---| | latest | Newest released GA or Preview version (excludes EarlyAccessPreview) | | Bare major (e.g. 28) | Newest released minor within that major wave | | Exact N.M (e.g. 28.3) | Must be released and offered; pending versions are rejected |

Edge cases handled:

• Sandbox already on the requested version → skip with warning, return Skipped=$true.
• No released versions available → skip with informational message.
• Requested major has no released minors but has pending → throw with expected availability.
• EarlyAccessPreview excluded from alias resolution (requires Partner Sandbox licensing).

Output: [pscustomobject] with CurrentVersion, Resolved, ResolvedType, Skipped, SkipReason.

5.3 build-report-site.sh

Location: .github/scripts/build-report-site.sh
Language: Bash
Called by: deploy-report job.

Purpose: Build the static HTML report site, retaining a rolling history of the last N runs plus a small permanent summary of every run.

Inputs (environment variables):

| Variable | Default | Description | |---|---|---| | HISTORY_DIR | site | Persistent site root (history branch working tree) | | RUN_ID | $GITHUB_RUN_ID | Unique run identifier | | RUN_NUMBER | $GITHUB_RUN_NUMBER | Human-friendly run number | | RUN_TIMESTAMP | now (UTC) | ISO-8601 timestamp | | TARGET_VERSION | <no upgrade> | Version shown in the report header | | MAX_RUNS | 7 | Number of runs to retain before pruning | | KEEP_SUMMARY | true | false disables summary/ and deletes it if present. Set from vars.REPORT_KEEP_SUMMARY | | FAILURES_MD | unset | Optional path. When set, a Markdown table of failed scripts and their causes is written there for the step summary (empty file when nothing failed) |

Process:

1. Flatten artifacts — copies downloaded replay artifacts into runs/{run_id}/{area}/{script}/ structure.
2. Generate per-run index.html — for each area, lists each script with:
   • PASS/FAIL status derived from results.xml (failures=0 → PASS).
   • Test count / failure count.
   • The one-line cause from failure-cause.txt (written by Invoke-ReplayArea.ps1) for a failed script.
   • Link to the Playwright HTML report.
3. Write metadata.json — sidecar with run_id, timestamp, target_version, total, failed counts. Unless KEEP_SUMMARY is false, a copy goes to summary/{run_id}.json, which is never pruned.
4. Prune old runs — keeps only the MAX_RUNS most recent (sorted by run ID descending). summary/ is untouched.
5. Generate top-level index.html — history table listing every retained run with status and links.
6. Export counts — writes total and failed to $GITHUB_ENV and $GITHUB_OUTPUT.

5.4 manage-failure-issues.sh

Location: .github/scripts/manage-failure-issues.sh
Language: Bash
Called by: deploy-report job.

Purpose: Automatically manage GitHub Issues to track script failures across runs.

Behavior per script:

| Script Status | Existing Open Issue? | Action | |---|---|---| | Failed | No | Create issue titled [Wave Readiness] {area} / {script} | | Failed | Yes | Comment on the existing issue ("Still failing on run #N") | | Passed | Yes | Close the issue with "Passing again" comment | | Passed | No | No action |

Key design decisions:

• Stable issue titles prevent duplicate issues for flaky scripts.
• Uses the wave-readiness label (auto-created, red #B60205).
• Fetches all open issues with the label upfront (one API call) to avoid per-script search hits.
• Missing results.xml is treated as a failure (matches the report's "NO RESULT" handling).
• Each issue body and "still failing" comment includes the one-line cause from failure-cause.txt, plus a link to the Playwright report on GitHub Pages.
• If the repository has Issues disabled, the script logs a warning and exits successfully; the report site and history branch still record the failures.

5.5 strip-steps-reporter.js

Location: .github/actions/setup-bc-replay/strip-steps-reporter.js
Language: JavaScript (Node.js)
Used by: setup-bc-replay action.

Purpose: Playwright HTML reporter subclass that removes all steps data from test results before writing the report.

Technical details:

• Extends Playwright's built-in HtmlReporter class.
• Recursively clears results[].steps across all test suites in the onEnd() hook.
• This prevents bc-replay's page.fill() arguments (including passwords) from appearing in the published report.

5.6 Get-EntraToken.ps1

Location: .github/scripts/Get-EntraToken.ps1
Language: PowerShell
Called by: the acquire-entra-token composite action, and the copy and upgrade polls in copy-and-upgrade from inside their loops.

Purpose: The one place the pipeline exchanges a client secret for a bearer token (client-credentials flow against login.microsoftonline.com). It emits ::add-mask:: before returning, so every token the pipeline holds is masked in the run log no matter which caller minted it.

Parameters:

| Parameter | Required | Description | |---|---|---| | -TenantId | Yes | Entra tenant ID | | -ClientId | Yes | Entra app (client) ID | | -ClientSecret | Yes | Entra app client secret | | -Scope | Yes | OAuth scope, e.g. https://api.businesscentral.dynamics.com/.default or https://graph.microsoft.com/.default |

Output: the access token as a string.

Why the polls call it directly: an access token lives about an hour. A sandbox copy can take that long and a major-version upgrade regularly takes longer, so a poll that holds the token minted before the operation started gets a 401 with the operation still running. Each poll therefore mints a token on its first iteration and again whenever the current one is 45 minutes old.

6. Page Script Library

6.1 What are Page Scripts?

Page scripts are .yml files recorded using Business Central's built-in Page Scripting recorder. Each file captures a complete business scenario as a sequence of UI interactions:

• navigate — navigate to a page via the role center.
• page-shown / page-closed — assert page lifecycle.
• input — enter a value into a field.
• focus — move focus to a specific field.
• invoke — trigger an action (New, Post, AssistEdit, LookupOk, etc.).
• copy-value — capture a field value to a clipboard variable.
• scope + condition — conditional execution (if field equals value, run sub-steps).

Scripts reference pages, fields, and actions by their AL captions and runtime IDs, making them sensitive to UI changes across BC wave updates — which is exactly the point.

6.2 Library Conventions

The library ships empty: each adopting project records scenarios against its own customer data using BC's built-in Page Scripting recorder (see PageScriptLibrary/README.md for the recording guide). The layout rules the workflow enforces:

• Area folders sit directly under PageScriptLibrary/ and are named <Name> - <N>; the numeric suffix controls the order areas replay in (e.g. Purchase - 2 before Invoice - 3).
• MasterData - 1 is reserved. Its scripts run first, sequentially, on a dedicated job, and a failure there blocks everything downstream — put the seed data every other scenario depends on here (vendors, items, posting setup, the My Settings role-centre script).
• Within an area, scripts replay in natural file-name order (Script 2 runs before Script 10); prefix filenames with a number when order matters.
• New area folders are discovered dynamically — drop a .yml in and the next run picks it up.

7. Report & Notification System

7.0 Output Storage Summary

| Output | Where stored | Retention | |---|---|---| | Per-script Playwright report + JUnit XML | replay-results/{AreaName}/{ScriptName}/ on the CI runner | Job lifetime only (ephemeral) | | Same results (archived) | GitHub Actions artifacts: replay-results-{area} (master-data job) and replay-results (replay job) | 7 days | | Persistent run history (source of truth) | wave-readiness-history branch → runs/{run_id}/. The branch is a single parentless commit, rewritten each run | Last 7 runs (older runs pruned by build-report-site.sh; nothing of them remains in git) | | Per-run pass/fail summary | wave-readiness-history branch → summary/{run_id}.json | Every run ever published (a few hundred bytes each). Off when vars.REPORT_KEEP_SUMMARY is false | | Live report website | Azure App Service (Entra Easy Auth) — mirrors the history branch | Mirrors history branch | | GitHub Issues | Repository Issues tab, label wave-readiness | Until closed (auto-closed on next pass) | | Email notification | Recipient inbox via Microsoft Graph sendMail | Email server retention | | Run step summary | GitHub Actions run page ($GITHUB_STEP_SUMMARY) | GitHub Actions log retention |

7.1 Report Site (Azure App Service)

The workflow publishes a static HTML site to an Azure App Service with a rolling history of runs. The App Service is configured with Entra Easy Auth so the site is restricted to your organisation's users — no GitHub Pages, no anonymous access:

https://{your-app-service}.azurewebsites.net/
├── index.html                      # History: table of last 7 runs
├── server.js                       # Zero-dep Node server (directory → index.html)
├── package.json                    # Tells App Service to start server.js
├── summary/
│   └── {run_id}.json               # One per run ever published; never pruned (absent when REPORT_KEEP_SUMMARY=false)
└── runs/
    └── {run_id}/
        ├── index.html              # Per-run: areas + scripts with pass/fail
        ├── metadata.json           # Machine-readable run summary
        └── {area}/
            └── {script}/
                ├── results.xml     # JUnit XML results
                └── playwright-report/
                    └── index.html  # Playwright HTML report (video + screenshots; trace and steps stripped)

The site content lives on a dedicated branch, wave-readiness-history, so run data survives across workflow executions. Each deploy-report run regenerates the zip from that branch and pushes it to App Service, so the site always mirrors the branch.

The branch is a snapshot, not a log. Every run writes the whole tree as one parentless commit and force-pushes it, with a lease on the tip it fetched, so at any moment the branch holds exactly one commit and no ancestry. A pruned run therefore leaves nothing behind in git, the branch stays the size of one site (the videos dominate; a run over a few dozen recordings is tens of megabytes), and a clone never drags old screenshots along. The long-term record is summary/, which is kept in the tree for as long as the branch exists, at a few hundred bytes per run. Set the repository variable REPORT_KEEP_SUMMARY to false to keep nothing beyond the retained runs; the next run then removes the folder.

What that means for the people managing the repository:

• It is owned by the workflow. Nothing else should commit to it. Hand-made changes are overwritten on the next run.
• It is safe to delete. The next run recreates it from that run's results, and the site is rebuilt from it. Deleting it loses the retained reports and the summary/ record, nothing else.
• It shares no ancestor with any other branch. git merge refuses it as unrelated history, so it cannot be merged into your code by accident.
• Rulesets must leave it alone. A rule that requires pull requests, signed commits, linear history, or forbids force-pushes on every branch will reject the workflow's push and fail the deploy-report job. Scope such rules to your working branches or exclude wave-readiness-history.
• Clone without it when you do not need it. git clone --single-branch fetches only the default branch. A plain clone also downloads the history branch, which is a few hundred megabytes.

7.2 GitHub Issues

Each failing script automatically gets a GitHub Issue titled [Wave Readiness] {area} / {script}. Issues are:

• Created on first failure.
• Commented on subsequent failures ("Still failing on run #N").
• Closed when the script passes again.

All managed issues carry the wave-readiness label.

This needs Issues enabled on the repository. When they are disabled, the deploy-report job logs a warning and skips this step rather than failing.

7.3 Email Notification

Sent via Microsoft Graph sendMail API (using the same Entra app). Contains an HTML table with:

• Pass/fail headline
• Target version, sandbox info
• Script total/failed counts
• Phase statuses
• Links to the report and run history

7.4 Step Summary

Every run writes a rich markdown summary to $GITHUB_STEP_SUMMARY visible on the Actions run page, with the same version/status/link information.

8. Security Considerations

| Concern | Mitigation | |---|---| | Credentials in logs | Entra tokens masked via ::add-mask::. BC credentials passed as env var names, never values. | | Credentials in reports | strip-steps-reporter.js removes all Playwright step data (including page.fill() arguments) from published HTML reports. Screenshots are kept as attachments; BC's login form masks the password input so they are safe to publish. | | Token lifetime | Tokens re-acquired between long operations, and renewed every 45 minutes inside the copy and upgrade polls (a major-version upgrade can run well past the 60-min lifetime). | | HTML injection in email | All values sourced from workflow inputs or API responses are HTML-escaped before insertion into the email body. | | Concurrency | cancel-in-progress: false prevents a second dispatch from killing an in-flight sandbox copy. | | Permissions | Workflow-level permissions are minimal (contents: read); broader permissions only on the deploy-report job that needs them. | | Screenshots and recordings of business data | The published reports contain screenshots and videos of the sandbox, which is a copy of production. They exist in two places: the Easy Auth report site and the wave-readiness-history branch, which everyone with read access to the repository can clone. The branch is a single-commit snapshot, so a pruned run leaves no trace in git. Keep the repository private and treat read access as access to the last seven runs' media. |

9. How to Use

This repository is not a template to fork. It is the source for an installer that adds the pipeline to a repository you already have. Any GitHub repo will do — the pipeline neither reads nor assumes anything about the rest of its contents.

Install into your repo

Full procedure: WAVE-READINESS-SETUP.md is the end-to-end implementation guide — prerequisites, the install, the Entra apps, the BC replay user, the Azure report site, secrets, first run, and troubleshooting by symptom. What follows is just the install command.

From the root of the target repository (requires git + an authenticated GitHub CLI):

npx @inmindtechnologies/bc-wave-readiness guide     # write WAVE-READINESS-SETUP.md here, install nothing
npx @inmindtechnologies/bc-wave-readiness doctor    # check prerequisites
npx @inmindtechnologies/bc-wave-readiness           # install
npx @inmindtechnologies/bc-wave-readiness provision # create the Azure report site

The install adds the workflow, composite actions, helper scripts, WAVE-READINESS-SETUP.md, and an empty PageScriptLibrary/ skeleton (no demo recordings), then offers to provision Azure and to run the secrets wizard.

provision builds the whole report-site stack — resource group, App Service, Entra Easy Auth, the OIDC deploy identity with a correctly-shaped federated credential, and the role assignment — and writes the AZURE_* secrets and variables it produced. It is the same code path the fleet onboarding script uses (lib/Azure.ps1), so both routes build identical infrastructure. It needs the Azure CLI and az login; everything else still works without it.

Afterwards, provision the Entra app for BC and the replay user per WAVE-READINESS-SETUP.md — those live in the BC tenant and are not automated.

Later:

• npx @inmindtechnologies/bc-wave-readiness update — re-sync the pipeline files from this repo (review with git diff).
• npx @inmindtechnologies/bc-wave-readiness configure — re-run the secrets wizard.
• npx @inmindtechnologies/bc-wave-readiness provision — re-run Azure provisioning; it is idempotent and repairs a drifted federated credential.

The install also drops in .github/workflows/UpdateWaveReadiness.yaml, which runs the update weekly and opens a pull request with the diff. It stays inert until an UPDATE_TOKEN secret is added, because GitHub forbids the built-in GITHUB_TOKEN from modifying files under .github/workflows/ and the pipeline's own workflow is one of the files an update rewrites. Without the secret it writes a job summary explaining itself rather than failing. It never pushes to a default branch. See WAVE-READINESS-SETUP.md section 12.

The npm package is a thin front end over setup-pipeline.ps1, which it ships. That is deliberate: the script is not one of the pipeline's managed files, so a copy committed into an adopter's repo would be frozen at install time and never receive a fix. npx resolves the current published version on every run. The script still works standalone for anyone who would rather not have Node: the package tarball (linked from the npm package page under Versions) contains setup-pipeline.ps1 alongside the pipeline files it installs, so extract it and run pwsh ./setup-pipeline.ps1 from the root of the target repository.

Where to install it

The recommended home is a dedicated repository per Business Central environment, separate from any AL source. The pipeline tests an environment, production copied to a sandbox and upgraded, not an extension: a customer with no custom code needs it, and a customer with five extensions needs one, not five. A repository of its own keeps three things separate that do not mix well with a code repository:

• Audience. Page scripts are recorded by the people who run the business processes, and the reports contain screenshots and recordings of their data. Neither group is the same as the people who read AL source.
• Permissions. The report job needs write access to contents and issues and an OIDC identity trusted by Azure. Granting that to a repository holding only recordings and reports is a small decision.
• Side effects. Failure issues land in this repository's tracker, the history branch grows here, and rulesets written for code review never collide with the workflow's push.

Installing into an existing repository works, and the installer does not distinguish the two. The section below lists what it touches.

Coexisting with what is already there

The pipeline is designed to be a guest in a repo it does not own:

• It never fires on its own. The workflow is workflow_dispatch only, so it cannot race another workflow on push or PR. Its concurrency group is wave-readiness, separate from anything else in the repo.
• It adds one GitHub environment, report-site, used only as the deploy target for the report. Existing environments are untouched.
• It claims a named, minimal set of files — the workflow, two composite actions, four scripts under .github/scripts/, and WAVE-READINESS-SETUP.md. Individual files, not the .github/scripts directory: -Update will never delete anything else you keep alongside them.
• A first install refuses to overwrite. If any of those paths already exists, the installer stops and lists them rather than clobbering your repo. Use -Force only once you have looked at what would be lost.
• PageScriptLibrary/ is yours. It is created empty on first install and never touched by -Update.
• It owns one branch, wave-readiness-history, created on the first run and force-pushed as a single-commit snapshot on every run. Exclude that branch from any ruleset that requires pull requests or signed commits or that blocks force-pushes; see 7.1.

The only files added at the repo root are WAVE-READINESS-SETUP.md, .pipeline-version, the PageScriptLibrary/ folder, and setup-pipeline.ps1 itself.

Requirement: the repo's plan must support deployment environments — GitHub Team, Pro, or Enterprise if the repo is private. setup-pipeline.ps1 creates the report-site environment during install and warns if it cannot.

Onboard a fleet of customer repos

fleet/onboard-customer.ps1 provisions a customer end-to-end — Azure report-site, GitHub secrets/variables, and the pipeline itself — in one command. It installs into an existing customer repo on a branch and opens a pull request for review (greenfield/empty repos are seeded directly). The BC replay user and admin consent stay manual. See fleet/ONBOARDING.md.

Run against the next wave preview

1. Go to Actions → Wave Readiness Validation → Run workflow.
2. Set targetVersion to latest (or a specific version like 28 or 28.3).
3. Leave refreshSandbox checked.
4. Click Run workflow.

Iterate on a specific area without refreshing

1. Set scriptArea to the area name (e.g. Purchase - 2).
2. Uncheck refreshSandbox.

Add a new business scenario

1. Open BC → search "Page Scripting" → + New → name the script → Start Recording.
2. Perform the business flow → Stop Recording → Save → Download.
3. Place the .yml in the appropriate area folder (or create a new area folder).
4. Open a PR. The workflow discovers areas dynamically.

Run locally

npm install @microsoft/bc-replay
$env:BC_USER = '[email protected]'
$env:BC_PASS = '...'
npx replay "PageScriptLibrary/Purchase - 2/MyScenario.yml" `
  -StartAddress 'https://businesscentral.dynamics.com/<tenantId>/<sandbox>/' `
  -Authentication AAD -UserNameKey BC_USER -PasswordKey BC_PASS `
  -ResultDir ./local-results

10. Dependencies & Prerequisites

| Dependency | Purpose | |---|---| | @microsoft/bc-replay | Headless Playwright-based BC page script player | | Playwright (bundled) | Browser automation engine | | Node.js 24 | Runtime for bc-replay | | Entra app registration (BC + Graph) | OAuth client with AdminCenter.ReadWrite.All (BC) and Mail.Send (Graph) | | Entra app registration (Azure deploy) | Federated identity used by azure/login to deploy the App Service zip | | BC user account | With PAGESCRIPTING-PLAY permission set in the production environment | | Azure App Service | Report hosting (Linux, Node runtime, Easy Auth enabled) | | GitHub environment report-site | Deploy target; the Azure federated credential's subject is derived from it. Needs GitHub Team/Pro/Enterprise on a private repo — Free supports environments on public repos only |

Third-party GitHub Actions

Every uses: in the workflow and the composite actions is pinned to a commit SHA with a trailing # vX.Y.Z comment — both for supply-chain safety (a mutable tag can be repointed at new code) and because .github/dependabot.yml reads that comment to know which version is current. Dependabot opens one grouped PR per week bumping the SHA and the comment together; without the pin it has nothing to track and a floating @v5 rots silently until a runner deprecation breaks a run.

Its directories glob covers .github/actions/* as well as the workflow directory, so composite actions added later are watched without editing the config.

All jobs run on GitHub-hosted runners. Actions on the node24 runtime need Actions Runner ≥ 2.327.1, so a fork that moves any job to a self-hosted runner must keep that runner current.

For step-by-step provisioning instructions — from creating Entra apps to wiring up the GitHub secrets and variables — see WAVE-READINESS-SETUP.md.

11. Licence

MIT, © 2026 inMind Technologies. See LICENSE. Every script, workflow, action and template the installer places in your repository carries a two-line copyright and SPDX header so the attribution travels with the code.

This is a helper library, not a managed service. It is provided as is, without warranty of any kind, and you use it at your own risk: it runs in your tenant, against a copy of your data, under your configuration. The licence covers the tooling only; the page-script recordings you create with it are yours.