Failure-Hub: Forensic Observability Tool

A high-performance, forensic observability framework for Playwright that streams Gzipped crash data to a real-time dashboard.

Dashboard Preview

Quick Start for Playwright Users

npx failure-hub-playwright@latest init
npx playwright test

Run the initializer from any folder that should become or already is a Playwright project. The command is downloaded by npm and always installs files into the current folder. If that folder does not yet contain a package.json, the initializer creates one automatically.

The repository-development command below is intentionally different:

node bin/failure-hub-playwright.js init

That form only works inside a checkout of this package because bin/... is a relative file path. End users should use the npx failure-hub-playwright@latest init command.

Architecture

graph TD
    A[Playwright Test] -->|Intercepts Error| B[Failure Collector]
    B -->|Captures Context| C[Forensic Translator]
    C -->|Redacts PII| D[Sanitization Pipeline]
    D -->|Shrinks up to 80%| E[Gzip Compression]
    E -->|Streams via node:fetch| F[Failure Hub Dashboard]

1. System Overview (The "Why")

Traditional test reporters often rely on slow, bulky file uploads (like full traces or videos) that can easily overwhelm orchestration layers and networks during high-concurrency failure events. We engineered a better way.

The Forensic Architecture We moved away from sluggish multipart form uploads to a high-speed, native Gzipped JSON forensic stream. Instead of capturing bloated traces, the Failure-Hub intelligently strips away the noise and captures precisely what you need—the "crime scene" data—and transmits it instantly.

The 50 MB Gzip Safety Valve When running large parallel suites, simultaneous failures can generate over 1 GB of raw artifact data. The Failure-Hub employs proactive DOM sanitization and Gzip compression at the source to reduce this dramatically. As a hard backstop, the uploader enforces a strict 50 MB Gzip limit per payload — if compression still cannot bring the payload below this threshold, domHtml is automatically truncated and a warning is logged. This guarantees your orchestration layer never receives a "data bomb", even when 100+ tests fail simultaneously.

2. The "Shared Bundle" (What to Share)

To integrate this forensic architecture into any project, you simply need to copy the core bundle into your repository. The bundle consists of three lightweight files:

• failure-hub/core/uploader.ts (The Transport): A framework-agnostic fetch layer handling the strict 10s timeouts and native zlib compression.
• failure-hub/playwright/fixture.ts (The Collector): An { auto: true } Playwright fixture that actively listens to console logs, network errors, and captures the exact DOM state upon failure.
• failure-hub/playwright/reporter.ts (The Translator): The forensic engine that cleans, sanitizes, redacts, and encodes the raw data into a pristine JSON payload before dispatch.

3. Step-by-Step Installation

One-command setup

From the root of an existing Playwright project, run:

npx failure-hub-playwright@latest init

The initializer:

• copies the Failure Hub fixture, reporter, feature flag, and uploader into failure-hub/;
• creates empty local JSONL and text log files without replacing existing logs;
• adds the @failure-hub/* TypeScript/JavaScript path alias;
• registers the reporter when the Playwright config does not already define one; and
• runs npm install so all dependencies declared by the Playwright project are downloaded, and adds @playwright/test and @types/node when they are not already declared.

It is safe to run the command again. Locally modified Failure Hub files are preserved unless you explicitly pass --force. Use --dry-run --skip-install to preview the setup without changing the project. Rerunning init also adds the apiKey reporter option to projects that were initialized by an older package version.

After initialization, test files use the collector fixture:

import { test, expect } from "@failure-hub/playwright/fixture";

Then run the normal Playwright command:

npx playwright test

Enable or disable failure logging

The generated failure-hub/playwright/report-loggen.ts file contains the logging switch:

export const REPORT_LOGGEN = true;

• true: failed and timed-out tests are appended to failure-hub/api_failure_logs.jsonl and failure-hub/api_failure_logs.txt.
• false: the Failure Hub fixture does not collect evidence, the Failure Hub reporter is not registered, screenshots are disabled, and no Failure Hub log entries are added.

Passing tests never add Failure Hub log entries. The initializer creates the two log files as empty files; entries are appended only after a test failure while logging is enabled.

If the project already has a custom reporter, the initializer does not replace it. It prints an instruction to add ./failure-hub/playwright/reporter.ts to the existing reporter list.

Prerequisites

• Node.js: v20.0 or higher
• Playwright: v1.40 or higher

Installation

Simply ensure you have the standard Playwright dependencies installed in your project:

npm install -D @playwright/test
npm install -D @types/node

(No external HTTP clients like Axios or Request are required. The system leverages native Node APIs.)

The "Mock Hub" Setup (Local Dashboard)

To visually confirm your payloads locally before connecting to a real orchestration layer, run the provided Mock Server:

node mock-server.js

The Forensic Dashboard will immediately be available at http://localhost:3000.

4. Integration Guide (What to Change)

Config Change

Update your playwright.config.ts to register the Forensic Reporter and tune the artifact collection.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    ['./failure-hub/playwright/reporter.ts', { 
      apiKey: process.env.FAILURE_HUB_API_KEY || 'test-api-key-12345',
      endpoint: process.env.FAILURE_HUB_ENDPOINT || 'http://localhost:3000/api/ingest' 
    }]
  ],
  use: {
    // Disable heavy traces; the Forensic Reporter provides the needed context
    trace: 'off', 
    // Ensure media is captured on failure for Base64 encoding
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

Test Script Change: The "Single-Line" Fix

To leverage the Forensic Collector (fixture.ts), simply change the import statement in your test files:

Before:

import { test, expect } from '@playwright/test';

After:

import { test, expect } from '../failure-hub/playwright/fixture';

5. How it Works (The Forensic Pipeline)

Before any data leaves the runner, the reporter.ts passes the raw failure through a 4-Stage Sanitization Pipeline:

1. ANSI Stripping: Automatically strips complex terminal color codes (\u001b[...m) from error messages and stack traces, ensuring cleanly readable text on the dashboard.
2. DOM Slimming: Strips out all heavy <svg>, <style>, and data:image/... elements from the captured HTML. This critical step preserves the structural DOM context while eliminating the massive byte-weight of embedded visuals.
3. PII Redaction: Actively scrubs sensitive strings. Any occurrences of "password", "token", or "authorization" within the DOM or logs are permanently replaced with "***REDACTED***".
4. Base64 Encoding: Reads the physical screenshot and video attachments using node:fs and converts them into pure Base64 strings embedded directly within the JSON payload.

The Transport Layer Once the JSON is sanitized, uploader.ts compresses it using node:zlib (gzipSync). It then broadcasts the binary stream using the native Node fetch API. To guarantee the test suite is never blocked by a slow network, the fetch request is wrapped in a strict 10-second Promise.race timeout. If the server doesn't respond, the system gracefully logs the error and moves on.

6. Data Reference (What we collect)

Every forensic payload dispatched to your orchestrator adheres to a strict JSON structure containing the following vital fields:

| Field | Type | Description | | :--- | :--- | :--- | | apiKey | String | The API key supplied through the Failure Hub reporter options in playwright.config.ts. | | testName | String | The exact title of the failing test case. | | sourceCode | String | The full content of the test file, dynamically read from the disk. | | line / column | Number | The precise location coordinates where the error was thrown. | | error | String | The clean, ANSI-stripped error assertion message. | | stack | String | The ANSI-stripped stack trace for deep debugging. | | domHtml | String | The sanitized, slimmed, and redacted HTML structural snapshot. | | logs | String | A concatenated stream of all console events and network errors. | | screenshotBase64| String | The Base64 encoded screenshot captured exactly at the moment of failure. | | metadata | Object | High-level context including timestamp, browserName, viewport, workerIndex, and retry count. |

7. Developer Experience

When testing locally, the Forensic Dashboard (http://localhost:3000) provides an unparalleled debugging experience.

• The Evidence: Inspect the fully syntax-highlighted sourceCode block, automatically scrolled and emphasized on the exact failing line.
• The Timeline: Review the browserLogs through an integrated, fully searchable text console.
• The Structure: Dive into the domHtml via an expandable, safe iframe window, allowing you to visually inspect the exact layout constraints that caused the failure.

8. Extension Guide (Framework Agnostic)

While the provided Collector and Translator are written specifically for Playwright, the Core Transport (failure-hub/core/uploader.ts) is completely Framework Agnostic.

If your team uses Jest, Cypress, or Selenium, you only need to write a custom Translator (Reporter) to gather the test data and map it to our JSON structure. Simply pass the resulting object into the uploadFailureReport() function, and the Core Transport will handle the Gzip compression, headers, and 10s timeout orchestration for you automatically.

JavaScript Compatibility

Yes, this works seamlessly for pure JavaScript projects!

• Playwright JS Projects: Playwright automatically transpiles TypeScript under the hood. You can copy the .ts files provided here directly into your plain JavaScript project and import them without any extra configuration.
• Elegant Imports for Pure JS Teams: If you are using plain JavaScript and do not have a tsconfig.json file, you do not need to use ugly relative paths (e.g., ../../failure-hub). Simply create a jsconfig.json file in your project root with the following configuration:

  {
    "compilerOptions": {
      "baseUrl": ".",
      "paths": {
        "@failure-hub/*": ["failure-hub/*"]
      }
    }
  }

  Playwright will instantly recognize this, allowing you to use elegant imports: import { test } from '@failure-hub/playwright/fixture';
• Other JS Environments: If using pure Node.js (e.g., Jest without TS), simply strip the TypeScript types from uploader.ts and use the resulting pure JavaScript file. The underlying logic uses native Node APIs (fetch, node:zlib) and works everywhere.

9. API Schema & Manual Testing

If you want to understand exactly what data is pushed to the orchestrator or if you want to test the pipeline manually, two reference files are included in the root directory:

1. sample-payload.json: This file contains the exact raw JSON schema that the Failure Hub generates. It shows all the forensic data points (test details, DOM snapshot, Base64 screenshot, redacted browser logs, environment context) before it is Gzip compressed.
2. post-sample.js: A tiny Node script that demonstrates exactly how to compress and POST the sample-payload.json data to the Mock Server endpoint.

You can run node post-sample.js from the root of the project to manually fire a fake failure payload and watch it appear instantly on the Dashboard (http://localhost:3000). This is the perfect way to familiarize yourself with the API contract!