@elliemae/encw-leak-runner

Playwright-based memory-leak detection runner for Encompass Web microapps. Wraps @elliemae/encw-heap-doctor and @elliemae/smoked-suite to capture before/after heap snapshots around scripted user flows, compare retained sizes, and emit machine- and human-readable reports.

Internal package. The Tier 1 / Tier 2 API split below documents the surface for in-monorepo consumers and Jenkins. There is no backwards-compatibility guarantee until external consumers exist.

Installation & build

# from repo root
pnpm install
pnpm --filter @elliemae/encw-leak-runner build

Quick start

# list registered scenarios
pnpm --filter @elliemae/encw-leak-runner exec leak-runner list

# run all scenarios against a target environment (locally)
# select a named env from leak-runner.config.json (e.g. Q3, UAT1, PROD)…
export ENCW_INSTANCE_ID=BE11226875
export ENCW_USER_ID=admin
export ENCW_PASSWORD='your-password'
pnpm --filter @elliemae/encw-leak-runner exec leak-runner run --all --env Q3

# …or provide a base URL via env var fallback
export BASE_URL=https://q3.elliemae.io
pnpm --filter @elliemae/encw-leak-runner exec leak-runner run --all

# run a single scenario by key
pnpm --filter @elliemae/encw-leak-runner exec leak-runner run one-admin/export-navigation

Configuration

Two layers:

1. leak-runner.config.json (non-secret runner defaults)

Lives at the package root. Holds runner options, AI settings, and the envs URL map. Never holds credentials or other secrets.

{
  "$schema": "./leak-runner.schema.json",
  "runner": {
    "headless": true,
    "outputDir": "./leak-reports/",
    "topN": 5
  },
  "ai": {
    "enabled": false,
    "model": "Claude3.7",
    "temperature": 0.3
  },
  "envs": {
    "LOCALHOST": "http://localhost:3000",
    "Q3": "https://q3.elliemae.io",
    "Q4": "https://q4.elliemae.io",
    "UAT1": "https://uat1.elliemae.io",
    "PROD": "https://encompass.elliemae.io"
  }
}

The envs map is just a convenience: each key is a name you can pass to --env <name> (or the ENCW_ENV env var) and the runner resolves the matching URL. Add, remove, or override entries as you like.

2. Environment parameters (per-run, required)

| Parameter | CLI flag | Env var | Required | | ----------- | -------------------- | ------------------ | ---------------------- | | Base URL | (no flag) | BASE_URL | yes (or use --env) | | Env name | --env <name> | ENCW_ENV | no (resolves base URL) | | Instance ID | --instance-id <id> | ENCW_INSTANCE_ID | yes | | User ID | --user-id <id> | ENCW_USER_ID | yes | | Password | (no flag) | ENCW_PASSWORD | yes |

--env <name> (or ENCW_ENV) looks up the URL in the envs map from leak-runner.config.json. Precedence for base URL: BASE_URL > --env / ENCW_ENV map lookup. An unknown env name fails with exit code 2 and lists the defined envs. If no source resolves a base URL at all, the runner exits with code 2 and prints every missing field by name with the CLI flag and env-var name to use.

Precedence

| Layer | Runner options (headless, outputDir, topN) | Environment params (baseUrl, instanceId, userId) | | ---------------- | ------------------------------------------------ | ------------------------------------------------------------------ | | CLI flag | Highest | Highest | | process.env | Mid | Mixed (BASE_URL highest for base URL; id/user from CLI beat env) | | Config file | Low | Indirect — via envs map looked up by --env / ENCW_ENV | | Built-in default | Lowest | No defaults — required |

Secrets

ENCW_PASSWORD and the optional GENICE_API_KEY (when ai.enabled) are read only from process.env. Neither has a CLI flag, so secrets cannot leak into shell history, process listings, or CI logs.

Adding a scenario

Scenarios live under lib/scenarios/<microapp>/. Each microapp has an index.ts barrel that exports its scenarios; lib/scenarios/index.ts aggregates them into the global ScenarioRegistry.

lib/scenarios/
├── one-admin/
│   ├── export-navigation.scenario.ts
│   ├── index.ts                          // exports oneAdminScenarios
│   └── page-models/
│       ├── ExportPageModel.ts
│       ├── SelectSettingsPageModel.ts
│       └── index.ts
└── index.ts                              // builds the global ScenarioRegistry

A scenario implements MicroappLeakScenario:

import type { MicroappLeakScenario } from '../../types/scenario.js';

export const myScenario: MicroappLeakScenario = {
  id: 'my-scenario', // kebab-case; combines with microapp into the registry key
  name: 'My Scenario', // human-readable display name
  description: 'What this scenario exercises',
  tags: ['critical'],
  microappSelector: 'iframe#my-microapp',
  url: () => '/path/to/microapp', // relative; ScenarioRunner sets baseURL on the context
  async action(page, frame) {
    // Drive the microapp through the user flow you want to leak-check.
  },
  async back(page) {
    // Optional: undo / navigate back so the next iteration starts clean.
  },
  repeat: () => 3, // default 3
  thresholds: {
    maxRetainedSizeDeltaBytes: 10 * 1024 * 1024,
    maxNewLeakGroups: 0,
  },
};

Add it to the microapp barrel and aggregate it in lib/scenarios/index.ts:

// lib/scenarios/my-microapp/index.ts
import type { MicroappLeakScenario } from '../../types/scenario.js';
import { myScenario } from './my-scenario.scenario.js';

export const myMicroappScenarios: readonly MicroappLeakScenario[] = [
  myScenario,
];

// lib/scenarios/index.ts
export const scenarioRegistry: ScenarioRegistry = new ScenarioRegistry()
  .register({ microapp: 'one-admin', scenarios: oneAdminScenarios })
  .register({ microapp: 'my-microapp', scenarios: myMicroappScenarios });

Running in Jenkins

pipeline {
  agent { label 'node20-playwright' }
  parameters {
    choice(name: 'ENV', choices: ['Q3', 'Q4', 'UAT1', 'PROD'],
           description: 'Looks up base URL from leak-runner.config.json envs')
    string(name: 'INSTANCE_ID', defaultValue: 'BE11226875')
    string(name: 'USER_ID',     defaultValue: 'admin')
    string(name: 'CREDENTIAL_ID', defaultValue: 'encw-q3-password',
           description: 'Jenkins credential ID for the password')
    choice(name: 'MODE', choices: ['all', 'by-tag', 'by-name'])
    string(name: 'NAME_OR_TAG', defaultValue: '')
  }
  environment {
    ENCW_PASSWORD  = credentials("${params.CREDENTIAL_ID}")
    GENICE_API_KEY = credentials('genice-api-key')   // optional
  }
  stages {
    stage('Install') { steps { sh 'pnpm install --frozen-lockfile' } }
    stage('Build')   { steps { sh 'pnpm --filter @elliemae/encw-leak-runner build' } }
    stage('Run leaks') {
      steps {
        script {
          def args = "--env ${params.ENV} --instance-id ${params.INSTANCE_ID} --user-id ${params.USER_ID}"
          if (params.MODE == 'all')     args += " --all"
          if (params.MODE == 'by-tag')  args += " --tag ${params.NAME_OR_TAG}"
          if (params.MODE == 'by-name') args += " ${params.NAME_OR_TAG}"
          sh "pnpm --filter @elliemae/encw-leak-runner exec leak-runner run ${args}"
        }
      }
    }
  }
  post {
    always {
      junit 'libs/encw-leak-runner/leak-reports/junit.xml'
      archiveArtifacts artifacts: 'libs/encw-leak-runner/leak-reports/**/*', fingerprint: true
    }
  }
}

Exit codes:

| Code | Meaning | | ---- | ------------------------------------------------ | | 0 | All scenarios passed | | 1 | At least one scenario failed thresholds | | 2 | Configuration error (missing required parameter) |

This split lets Jenkins distinguish a flaky test from misconfiguration.

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                          CLI / Jenkins                           │
│                bin/leak-runner.ts (Commander)                    │
└───────────────┬───────────────────────────────────┬──────────────┘
                │                                   │
                ▼                                   ▼
    ┌────────────────────┐               ┌────────────────────┐
    │ RunnerConfigLoader │               │  ScenarioRegistry  │
    │  (file+env+CLI)    │               │   (Composite)      │
    └─────────┬──────────┘               └─────────┬──────────┘
              │ RunnerConfig                       │ scenarios
              ▼                                    ▼
           ┌────────────────────────────────────────────┐
           │              BatchRunner                   │
           │  iterates scenarios → ScenarioRunner       │
           └───────────────┬───────────────┬────────────┘
                           │               │
                           ▼               ▼
     ┌─────────────────────────┐   ┌────────────────────────┐
     │      ScenarioRunner     │   │  Reporters (Console,   │
     │ ┌─────────────────────┐ │   │   JUnit) — pluggable   │
     │ │  AuthManager        │ │   └────────────────────────┘
     │ │  PageSetup          │ │
     │ │  (smoked-suite)     │ │
     │ │  IframeHeapProfiler │ │
     │ │   ↳ extends         │ │
     │ │     HeapMemoryProf. │ │
     │ │  ThresholdEvaluator │ │
     │ │  AiEnhancer (opt.)  │ │
     │ │   (heap-doctor)     │ │
     │ └─────────────────────┘ │
     └─────────────────────────┘
              │ Playwright
              ▼
       ┌─────────────┐
       │   Browser   │
       │  (microapp  │
       │   iframe)   │
       └─────────────┘

Patterns in play:

• Composite + Builder — ScenarioRegistry.register() composes microapp ScenarioGroups into a flat key→scenario map.
• Strategy — ConfigSource (file / env / CLI) lets each input format own its parsing in isolation; RunnerConfigLoader only knows merging.
• Command — each CLI subcommand is a CliCommand class registered with Commander via a uniform register(program, deps) contract.
• Dependency injection — every BatchRunner, ScenarioRunner, and CliCommand takes its collaborators via the constructor / register call, keeping unit tests honest.

Heap-doctor / smoked-suite integration

| Library | Provides | Where wired | | ---------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | @elliemae/encw-heap-doctor | ComparisonReport type, AiEnhancer, renderComparisonMarkdown | lib/runner/scenarioRunner.ts, lib/runner/aiEnhancementStep.ts, lib/analysis/thresholdEvaluator.ts | | @elliemae/smoked-suite | AuthManager, PageSetup, HeapMemoryProfiler (base class) | lib/runner/scenarioRunner.ts, lib/browser/iframeHeapProfiler.ts |

Upgrading either dependency is a matter of bumping the workspace version and re-running tests; the integration surface is small and concentrated in the files above.

How heap-doctor fits into a scenario run

When the runner executes a scenario, heap-doctor is the engine that turns two raw heap snapshots into a structured leak report. Here is the full journey from browser to pass/fail result:

sequenceDiagram
    participant SR as ScenarioRunner
    participant IHP as IframeHeapProfiler
    participant BR as Browser (CDP)
    participant HD as HeapDoctor
    participant TE as ThresholdEvaluator
    participant AI as AiEnhancer

    SR->>IHP: captureSnapshot('before')
    IHP->>BR: CDP takeHeapSnapshot
    BR-->>IHP: before.heapsnapshot

    loop N times (default 3)
        SR->>BR: scenario.action(page, frame)
        SR->>BR: scenario.back(page)
    end

    SR->>BR: forceGarbageCollection()

    SR->>IHP: captureSnapshot('after')
    IHP->>BR: CDP takeHeapSnapshot
    BR-->>IHP: after.heapsnapshot

    SR->>IHP: compare('before', 'after', topN)
    IHP->>HD: new HeapDoctor({ topN }).compare(before, after)
    Note over HD: parses both snapshots<br/>diffs object graphs<br/>groups survivors by retainer chain
    HD-->>IHP: ComparisonReport<br/>(delta, newLeakGroups[], leakResults[])
    IHP-->>SR: ComparisonReport

    SR->>TE: evaluate(report, thresholds)
    Note over TE: retainedSizeDelta > maxRetainedSizeDeltaBytes?<br/>newLeakGroups.length > maxNewLeakGroups?
    TE-->>SR: ThresholdResult { passed, reason }

    opt ai.enabled = true
        SR->>AI: enhance(leakResults[])
        Note over AI: sends leaks to GenICE<br/>parses action-item suggestions
        AI-->>SR: leakResults[] with aiSuggestions[] populated
        SR->>SR: renderComparisonMarkdown(report)
    end

    SR->>SR: write <scenario-id>.md + junit.xml entry

Plain-English summary of each step:

| Step | What happens | Who does it | | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | 1 — Before snapshot | Chromium's CDP protocol dumps the iframe's V8 heap to a .heapsnapshot file. This is the baseline — every object alive before the user flow. | IframeHeapProfiler (wraps smoked-suite's HeapMemoryProfiler) | | 2 — Run scenario | The scenario's action() function drives the microapp through a user flow (e.g. open a modal, save, close). It repeats N times so transient noise averages out. | Your scenario code | | 3 — Force GC | Chrome is told to run garbage collection. This evicts objects that were already unreachable — we only want to see objects that should have been freed but weren't. | HeapMemoryProfiler via CDP | | 4 — After snapshot | A second heap dump is taken. Any object present here but not in the baseline, and still reachable, is a candidate for a leak. | IframeHeapProfiler | | 5 — Compare (heap-doctor) | HeapDoctor.compare() parses both snapshot files, diffs the object graphs, groups surviving objects by their retainer chains, and returns a ComparisonReport with the retained-size delta and a list of suspected leak groups. | @elliemae/encw-heap-doctor | | 6 — Threshold check | ThresholdEvaluator reads delta.retainedSizeDelta and delta.newLeakGroups.length from the report and compares them against the scenario's configured limits (defaults: 10 MB / 0 new leak groups). | ThresholdEvaluator | | 7 — AI enhancement | If ai.enabled is true, AiEnhancer sends the leak results to GenICE and writes specific action-item suggestions back into the report before rendering the final markdown. | @elliemae/encw-heap-doctor (AiEnhancer, renderComparisonMarkdown) | | 8 — Write report | The markdown (with or without AI suggestions) is written to <outputDir>/<scenario-id>.md. A JUnit entry is appended for CI. | ScenarioRunner |

Key insight: heap-doctor only touches steps 5 and 7. Steps 1–4 capture the raw snapshots using the browser's built-in profiling API; step 6 is a simple numeric comparison. This means you can change thresholds or disable AI without touching the snapshot logic, and vice versa.

Output artefacts

For each scenario run, in <outputDir>:

• <scenario-id>.md — markdown report with retained-size delta, top leak groups, and (when ai.enabled) an "AI Suggested Fix" section.
• snapshots/ — temporary heap snapshots, deleted after each scenario.

For each batch run:

• junit.xml — JUnit XML aggregating per-scenario pass/fail for Jenkins.

Troubleshooting

• Configuration error: Missing required parameter(s): ... — set the listed CLI flag(s) or env var(s).
• Configuration error: Unknown env "...". Known envs in leak-runner.config.json: ... — the --env <name> (or ENCW_ENV) does not match an entry in the config's envs map. Add the entry or use one of the listed names.
• Iframe not found for selector: ... — the microapp container changed; update microappSelector in the scenario.
• Could not get content frame for selector: ... — the iframe is OOPIF. IframeHeapProfiler handles this case via CDP — verify your Chromium build is up-to-date.
• Playwright reports a missing browser — run pnpm --filter @elliemae/encw-leak-runner exec playwright install chromium once on the agent.

Internal API reference

Tier 1 — Scenario authoring

import type {
  MicroappLeakScenario,
  EnvironmentParams,
  RunnerOptions,
  AiConfig,
  RunnerConfig,
  ScenarioGroup,
  ScenarioEntry,
} from '@elliemae/encw-leak-runner';

Tier 2 — Embedding (unstable)

import {
  ScenarioRegistry,
  BatchRunner,
  ScenarioRunner,
  RunnerConfigLoader,
  FileConfigSource,
  EnvVarConfigSource,
  CliOverrideConfigSource,
  RequiredEnvParamsResolver,
  MissingRequiredParamError,
  UnknownEnvError,
  ConsoleReporter,
  JunitReporter,
  IframeHeapProfiler,
  ThresholdEvaluator,
  buildProgram,
  defaultDeps,
} from '@elliemae/encw-leak-runner';

Tier 2 has no backwards-compatibility guarantee until at least one external consumer exists. Coordinate with the owners before depending on it.