@testivai/witness-playwright

Status: ✅ Production Ready | Last Updated: January 11, 2026 Version: 0.1.12

Project: @testivai/witness-playwright (MVP - 1 Month Plan)

Role: The "Sensor" or The "Witness"

Goal: To provide a lightweight Playwright integration that (1) captures snapshots, DOM, and layout/bounding-box data, and (2) a custom reporter that uploads this data, along with viewport/browser context, in a batch-oriented way. The goal is to observe the application under test and collect evidence (snapshots, DOM, layout data) without burdening the developer with complex configuration.

Architecture

This package provides two main exports:

Configuration

To use the reporter, you need to configure it in your playwright.config.ts file. You must also provide your API URL and Key via environment variables (TESTIVAI_API_URL and TESTIVAI_API_KEY).

Get Your API Key:

1. Go to your TestivAI dashboard
2. Create a new project
3. Copy your API key (format: tstvai-xxxxxxxxxxxx)

Environment Setup:

# Create .env file
echo "TESTIVAI_API_KEY=tstvai-your-key-here" > .env

Note: The SDK automatically uses the production API URL. You only need to set TESTIVAI_API_KEY.

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  // ... other config

  reporter: [
    ['list'], // You can use other reporters alongside it
    ['@testivai/witness-playwright/reporter']
  ],
});

The reporter will automatically detect if it is running in a CI environment (such as GitHub Actions, GitLab, Jenkins, etc.) and will tag the evidence batch with a unique run ID. This ensures that snapshots from parallel jobs are correctly grouped together into a single test run.

Usage

Here is a basic example of how to use the snapshot function within a Playwright test file:

import { test } from '@playwright/test';
import { testivai } from '@testivai/witness-playwright';

test('my example test', async ({ page }, testInfo) => {
  await page.goto('https://example.com');

  // Capture a snapshot with a custom name (uses CDP by default)
  await testivai.witness(page, testInfo, 'example-home');

  // Or capture a snapshot without a name (one will be generated from the URL)
  await testivai.witness(page, testInfo);
});

Features

• 📸 Full-page screenshots with CDP or scroll-and-stitch methods
• 🌳 DOM capture for structural analysis
• 🔍 Smart DOM analysis with fingerprinting to reduce false positives
• 📐 Layout analysis with configurable tolerance
• 🤖 AI-powered visual analysis using Vertex AI
• 📊 Performance metrics capture (optional)
• 🔍 Lighthouse audits (optional)
• ⚙️ Flexible configuration per-project or per-test
• 🚀 CI/CD integration with automatic batch processing

📸 Screenshot Capture Methods

The SDK supports two full-page screenshot capture methods:

1. Chrome DevTools Protocol (CDP) - Default

• ✅ Clean, single capture without repeated navigation elements
• ✅ More realistic page representation
• ✅ Better for SPA applications with fixed layouts
• ✅ Automatic CSS height override for full page capture

2. Scroll-and-Stitch - Backup Method

• Available for compatibility or specific use cases
• Captures multiple viewport screenshots and stitches them together
• Use by setting useCDP: false

Configuration

// Default (CDP approach)
await testivai.witness(page, testInfo, 'test-name');

// Explicitly use CDP
await testivai.witness(page, testInfo, 'test-name', { 
  useCDP: true 
});

// Use scroll-and-stitch backup method
await testivai.witness(page, testInfo, 'test-name', { 
  useCDP: false 
});

// Set CDP as default in testivai.config.ts
export default {
  useCDP: true  // Use CDP for all tests
};

For detailed comparison, see FULL_PAGE_SCREENSHOT_COMPARISON.md.

1. testivai.witness(): A utility function to be called by the user within their Playwright tests to capture evidence.
2. TestivAIPlaywrightReporter: A custom Playwright reporter, configured in playwright.config.ts, that runs after all tests are complete. It is responsible for collecting all the evidence captured during the test run, batching it together with context (like Git and browser information), and uploading it to the Testivai service.

📦 Installation

npm install @testivai/witness-playwright

Performance Metrics

NEW: Automatically capture page performance metrics and Core Web Vitals!

Basic Performance Capture (Enabled by Default)

Every snapshot automatically captures:

• Core Web Vitals: LCP, FCP, CLS
• Page Load Metrics: DOM Content Loaded, Load Complete
• Navigation Timing: Navigation start and timing data

No configuration needed - it just works!

Optional Lighthouse Integration

For comprehensive performance audits, enable Lighthouse:

// testivai.config.ts
export default {
  performance: {
    captureTimings: true,      // Basic timing (default: ON)
    enableLighthouse: false,   // Lighthouse audit (default: OFF)
    lighthouseThresholds: {
      performance: 80,
      accessibility: 90,
      bestPractices: 80,
      seo: 80
    }
  }
};

Per-Test Configuration

Enable Lighthouse for specific tests:

test('performance critical page', async ({ page }, testInfo) => {
  await page.goto('https://example.com');
  
  await testivai.witness(page, testInfo, 'homepage', {
    performance: {
      enableLighthouse: true,
      lighthouseThresholds: {
        performance: 90
      }
    }
  });
});

What Gets Captured

Basic Timing (Always Captured):

• First Contentful Paint (FCP)
• Largest Contentful Paint (LCP)
• Cumulative Layout Shift (CLS)
• DOM Content Loaded
• Load Complete

Lighthouse (When Enabled):

• Performance Score (0-100)
• Accessibility Score (0-100)
• Best Practices Score (0-100)
• SEO Score (0-100)
• Detailed Core Web Vitals

� DOM Analysis

The SDK can perform intelligent DOM analysis to detect structural changes while avoiding false positives from dynamic content.

Enabling DOM Analysis

import { testivai } from '@testivai/witness-playwright';

test('with DOM analysis', async ({ page }, testInfo) => {
  await page.goto('https://example.com');
  
  await testivai.witness(page, testInfo, 'homepage', {
    dom: {
      enableFingerprint: true,  // Generate unique DOM hash
      enableStructure: true,    // Count elements, depth, etc.
      enableSemantic: true,     // Analyze headings, landmarks
      ignoreAttributes: ['data-testid', 'id', 'class'], // Skip dynamic attrs
      ignoreContentPatterns: [/\d{4}-\d{2}-\d{2}/, /\d+/], // Skip dates/numbers
    }
  });
});

What DOM Analysis Provides

1. Fingerprint: A unique hash of the normalized DOM structure

   • Ignores dynamic attributes and content
   • Fast way to detect if anything changed
   • Reduces false positives from timestamps, IDs, etc.

2. Structure Analysis: Counts and categorizes elements

   • Total element count
   • Element type distribution
   • Maximum DOM depth
   • Interactive element counts (buttons, forms, etc.)

3. Semantic Analysis: Understands page structure

   • Heading hierarchy (h1-h6 counts)
   • Landmark elements (header, nav, main, footer)
   • List and table counts
   • Image count

4. Component Detection: Identifies UI components

   • Based on data-testid, data-component, or class names
   • Groups related elements
   • Useful for component-aware testing

Reducing False Positives

DOM analysis includes smart defaults to avoid false positives:

• Ignores: Dynamic IDs, timestamps, random numbers, CSS classes
• Skips: Script/style tags, meta tags, inline styles
• Normalizes: Whitespace, dynamic content patterns

Example Configuration

// Strict configuration - catches only major changes
const strictConfig = {
  dom: {
    enableFingerprint: true,
    enableStructure: false,  // Skip element counts
    enableSemantic: true,    // Focus on semantic changes
    ignoreAttributes: ['data-testid', 'class', 'id', 'style'],
    ignoreContentPatterns: [/\d+/g, /uuid-/i],
  }
};

// Component-focused configuration
const componentConfig = {
  dom: {
    enableFingerprint: false,
    enableStructure: false,
    enableSemantic: false,
    // Focus only on component detection
  }
};

�📊 Progress

See progress.md for detailed development progress.