@sachin9210/ultimate-playwright-performance

A comprehensive performance testing utility for Playwright that captures Web Vitals, action timings, resource metrics, and API calls with configurable SLA thresholds.

Installation

Local Package (Workspace)

If using in the same monorepo/workspace:

# In the Playwright project root
npm install ./packages/playwright-performance-helper

As Published Package

npm install @sachin9210/ultimate-playwright-performance

Quick Start

Option 1: Direct Usage with Default Thresholds (Simplest)

import { PerformanceHelper } from '@sachin9210/ultimate-playwright-performance';
import { test, expect } from '@playwright/test';

test.describe('Performance Tests', () => {
    let perf;

    test.beforeEach(async ({ page }) => {
        // Uses default thresholds - no config needed!
        perf = new PerformanceHelper(page);
    });

    test('measure page performance', async ({ page }) => {
        // Start capture BEFORE navigation
        await perf.startCapture();

        // Navigate and perform actions
        await page.goto('https://example.com');

        // Track specific actions
        perf.startAction('Login');
        // ... perform login steps
        perf.endAction('Login');

        // Generate reports in afterEach
    });

    test.afterEach(async ({ page }) => {
        await perf.getMetrics();
        perf.printActionTimings();
        perf.printApiSummary();
        await perf.printSummary();

        const timestamp = new Date().toISOString().split('T')[0];
        await perf.generateReport(`test-${timestamp}.json`);
        await perf.generateReport(`test-${timestamp}.html`, 'html');
    });
});

Option 2: Using the Fixture Pattern (No Config Required)

import { test as base, expect } from '@playwright/test';
import { createPerformanceFixture } from '@sachin9210/ultimate-playwright-performance';

// Uses default thresholds - works out of the box!
const test = base.extend({
    perf: createPerformanceFixture(),
});

test.describe('Performance Tests', () => {
    test('measure page performance', async ({ page, perf }) => {
        // Performance helper is automatically available and initialized
        // Auto-capture starts when page loads
        // Reports are automatically generated after test completes!
        
        await page.goto('https://example.com');

        // Track specific actions
        perf.startAction('Login');
        // ... perform login steps
        perf.endAction('Login');

        // No afterEach needed - reports are generated automatically!
    });
});

Option 3: Using the Default Fixture (No Config, Zero Setup!)

import { test, expect } from '@sachin9210/ultimate-playwright-performance/fixture';

test.describe('Performance Tests', () => {
    test('measure page performance', async ({ page, performance }) => {
        // Performance helper with default thresholds
        // Auto-capture and auto-reporting enabled by default!
        
        await page.goto('https://example.com');
        performance.startAction('Login');
        // ... perform login steps
        performance.endAction('Login');
        
        // Reports (JSON, HTML, CSV) are automatically generated after test!
    });
});

Automatic Report Generation

Good news: Reports are automatically generated after each test completes! No afterEach hook needed! 🎉

By default, the fixture automatically:

• ✅ Captures all performance metrics
• ✅ Prints summaries to console (action timings, API summary, full summary)
• ✅ Generates reports in all formats (JSON, HTML, CSV)
• ✅ Names files based on test title and timestamp

Example:

const test = base.extend({
    perf: createPerformanceFixture(),
});

test('my performance test', async ({ perf }) => {
    perf.startAction('Login');
    // ... perform actions
    perf.endAction('Login');
    // Reports are automatically generated after test completes!
    // No afterEach hook needed!
});

Disable Automatic Reporting

If you want to generate reports manually (e.g., in a custom afterEach hook):

const test = base.extend({
    perf: createPerformanceFixture({
        autoReport: false  // Disable automatic reporting
    }),
});

test('my test', async ({ perf }) => {
    // ... test code
});

test.afterEach(async ({ perf }, testInfo) => {
    // Generate reports manually
    await perf.getMetrics();
    perf.setTestName(testInfo.title);
    await perf.generateReport('custom-report.json', ['json', 'html', 'csv']);
});

Customize Report Formats

Choose which formats to generate automatically:

const test = base.extend({
    perf: createPerformanceFixture({
        reporters: ['csv', 'html']  // Only generate CSV and HTML (skip JSON)
    }),
});

Report Formats

CSV Reports

Generate CSV reports in playwright-performance format with test names, action names, start times, end times, and durations:

// Set test name (required for CSV reporting)
perf.setTestName('My Test Name');

// Generate CSV report
await perf.generateReport('performance-results.csv', 'csv');

// Or generate multiple formats at once
await perf.generateReport('test-results.json', ['csv', 'html', 'json']);

CSV Format:

Test Name,Action Name,Start Time,End Time,Duration (ms)
My Test Name,Login,2024-01-15T10:00:00.000Z,2024-01-15T10:00:02.500Z,2500
My Test Name,Search,2024-01-15T10:00:03.000Z,2024-01-15T10:00:03.800Z,800

Automatic CSV Reporting:

// CSV reports are automatically generated when using the fixture
// Test name is automatically set from testInfo.title
const test = base.extend({
    perf: createPerformanceFixture(), // CSV included by default
});

Manual CSV Reporting (if autoReport is disabled):

test.afterEach(async ({ perf }, testInfo) => {
    await perf.getMetrics();
    
    // Set test name from testInfo
    perf.setTestName(testInfo.title);
    
    const timestamp = new Date().toISOString().split('T')[0];
    const testName = testInfo.title.replace(/[^a-zA-Z0-9]/g, '-').toLowerCase();
    
    // Generate CSV, HTML, and JSON reports
    await perf.generateReport(`${testName}-${timestamp}.json`, ['csv', 'html', 'json']);
});

HTML Reports

Generate beautiful HTML reports with detailed metrics, charts, and visual indicators:

await perf.generateReport('report.html', 'html');

JSON Reports

Generate machine-readable JSON reports for CI/CD integration:

await perf.generateReport('report.json', 'json');

Multiple Formats

Generate multiple report formats in a single call:

// Generate all formats
await perf.generateReport('test-results.json', ['json', 'html', 'csv']);

// Or specify individual formats
const [jsonPath, htmlPath, csvPath] = await perf.generateReport('results.json', ['json', 'html', 'csv']);

Consolidated Reports

Generate consolidated performance reports from multiple test runs:

# Via npm script (if configured in your package.json)
npm run report:consolidated

# Direct execution
node node_modules/@sachin9210/ultimate-playwright-performance/bin/generate-consolidated-report.js

# With custom pattern
node node_modules/@sachin9210/ultimate-playwright-performance/bin/generate-consolidated-report.js --pattern "*-2024-12-*.json"

# With custom output
node node_modules/@sachin9210/ultimate-playwright-performance/bin/generate-consolidated-report.js --pattern "district-coordinator-*.json" --output "district-coordinator-summary.html"

Or use the CLI command (if installed globally or via npx):

npx @sachin9210/ultimate-playwright-performance generate-consolidated-report

The script will:

1. Look for all JSON performance reports in .artifacts/performance-reports/
2. Aggregate metrics across multiple test runs
3. Generate a single HTML report with side-by-side comparisons
4. Show trends, averages, and identify performance regressions

Configuration

⚡ Default Thresholds (Works Out of the Box!)

Good news: The package works immediately with zero configuration! Default thresholds are built-in and based on Core Web Vitals recommendations:

• LCP (Largest Contentful Paint): Good < 2.5s, Needs Improvement < 4.0s
• FCP (First Contentful Paint): Good < 1.8s, Needs Improvement < 3.0s
• FID (First Input Delay): Good < 100ms, Needs Improvement < 300ms
• CLS (Cumulative Layout Shift): Good < 0.1, Needs Improvement < 0.25
• TTFB (Time to First Byte): Good < 800ms, Needs Improvement < 1.8s
• INP (Interaction to Next Paint): Good < 200ms, Needs Improvement < 500ms
• Action Timing: Good < 1s, Needs Improvement < 3s

You can start using it right away:

// Direct usage - defaults work!
const perf = new PerformanceHelper(page);

// Fixture pattern - defaults work!
const test = base.extend({
    perf: createPerformanceFixture(),
});

Customizing Thresholds

If you need different thresholds, you have several options:

Option 1: Inline Custom Thresholds

Pass thresholds directly when creating the helper:

// Direct usage with custom thresholds
const perf = new PerformanceHelper(page, {
    LCP: { good: 2000, needsImprovement: 3500 },
    FCP: { good: 1500, needsImprovement: 2800 },
    actionTiming: { good: 800, needsImprovement: 2500 },
});

// Fixture with custom thresholds
const test = base.extend({
    perf: createPerformanceFixture({
        thresholds: {
            LCP: { good: 2000, needsImprovement: 3500 },
            actionTiming: { good: 800, needsImprovement: 2500 },
        },
    }),
});

Option 2: Using Package Default Thresholds

Import and use the default thresholds from the package:

import { defaultThresholds } from '@sachin9210/ultimate-playwright-performance';

// Use defaults directly
const perf = new PerformanceHelper(page, defaultThresholds);

// Or override specific values
const perf = new PerformanceHelper(page, {
    ...defaultThresholds,
    actionTiming: { good: 800, needsImprovement: 2500 },
});

Option 3: Environment-Based Config File (Recommended for Teams)

Create a config file in your project (e.g., tests/config/performance-thresholds.js):

// tests/config/performance-thresholds.js
export const performanceThresholds = {
    default: {
        LCP: { good: 2500, needsImprovement: 4000 },
        FCP: { good: 1800, needsImprovement: 3000 },
        FID: { good: 100, needsImprovement: 300 },
        CLS: { good: 0.1, needsImprovement: 0.25 },
        TTFB: { good: 800, needsImprovement: 1800 },
        INP: { good: 200, needsImprovement: 500 },
        actionTiming: { good: 1000, needsImprovement: 3000 },
    },
    production: {
        // Stricter thresholds for production
        LCP: { good: 2000, needsImprovement: 3500 },
        // ... other metrics
    },
    staging: {
        // Relaxed thresholds for staging
        LCP: { good: 3000, needsImprovement: 5000 },
        // ... other metrics
    },
    development: {
        // Most relaxed for local dev
        LCP: { good: 4000, needsImprovement: 6000 },
        // ... other metrics
    },
};

export function getThresholdsForEnvironment(env = null) {
    const environment = env || process.env.environment || process.env.TEST_ENV || 'default';
    return performanceThresholds[environment] || performanceThresholds.default;
}

Copy the example config template:

# Copy the example config from the package
cp node_modules/@sachin9210/ultimate-playwright-performance/src/config/example-thresholds.js tests/config/performance-thresholds.js

Use it in your tests:

import { createPerformanceFixture } from '@sachin9210/ultimate-playwright-performance';
import { getThresholdsForEnvironment } from './config/performance-thresholds.js';

// Get thresholds based on environment variable
const environment = process.env.environment || 'test';
const thresholds = getThresholdsForEnvironment(environment);

const test = base.extend({
    perf: createPerformanceFixture({ thresholds }),
});

// Or set via environment variable automatically:
// TEST_ENV=production npm test
// environment=staging npm test

Config File Structure

Your config file should export an object with this structure:

export const performanceThresholds = {
    // Environment name
    environmentName: {
        // Core Web Vitals
        LCP: { good: 2500, needsImprovement: 4000 },        // milliseconds
        FCP: { good: 1800, needsImprovement: 3000 },        // milliseconds
        FID: { good: 100, needsImprovement: 300 },          // milliseconds
        CLS: { good: 0.1, needsImprovement: 0.25 },         // score (0-1)
        TTFB: { good: 800, needsImprovement: 1800 },        // milliseconds
        INP: { good: 200, needsImprovement: 500 },          // milliseconds
        
        // Custom action timings
        actionTiming: { good: 1000, needsImprovement: 3000 }, // milliseconds
    },
};

Notes:

• good: Threshold for "good" performance (green status)
• needsImprovement: Threshold for "needs improvement" (yellow status)
• Values above needsImprovement are considered "poor" (red status)
• You can override only the metrics you need; others will use defaults

Where to Put Your Config File

You can put your config file anywhere in your project. Common locations:

• tests/config/performance-thresholds.js (recommended)
• config/performance-thresholds.js
• playwright.config/performance-thresholds.js

Just import it where you need it in your test files.

Features

• ✅ Core Web Vitals: LCP, FID, CLS, FCP, TTFB, INP
• ✅ Action Timing: Track custom user actions
• ✅ Resource Metrics: Track all page resources
• ✅ API Monitoring: Automatic API request/response tracking
• ✅ HTML Reports: Beautiful, detailed performance reports
• ✅ JSON Reports: Machine-readable reports for CI/CD
• ✅ CSV Reports: playwright-performance compatible format with test names, actions, and timings
• ✅ Configurable Thresholds: Custom SLA thresholds per metric
• ✅ Configurable Reporters: Choose which report formats to generate (CSV, HTML, JSON, or all)
• ✅ Consolidated Reports: Combine multiple test runs with CLI tool

API Reference

See the full documentation in Readme/README_PERFORMANCE_HELPER.md

License

ISC