@compare-ui/runner

Execution helpers for design-comparison runs.

This package takes a comparison case, runs it through @compare-ui/core, writes the produced artifacts and report, and returns a structured result that can be used by tests, Storybook, or CLI commands.

It is the shared execution and persistence layer. Browser and Storybook readiness remain adapter concerns, while image preprocessing remains owned by @compare-ui/core.

Installation

pnpm add @compare-ui/runner @compare-ui/core

@compare-ui/core should be installed in the consuming app because @compare-ui/runner is designed to use the app's shared core instance.

What it exports

Main functions:

• runDesignComparisonCase(...)
• createFailureMessage(...)

Main types:

• DesignComparisonExecutionCase
• DesignComparisonOutputConfig
• DesignComparisonRunResult

Published entrypoint:

• @compare-ui/runner

That entrypoint is intended to resolve from the published build output, and the workspace verifies it through a tarball smoke test.

Execution case

The runner works with one execution case at a time.

Example shape:

import type {
  ArtifactInstruction,
  CompareConfig,
  CropOffsets,
  ImageResource,
  ImageSize,
  ReportingConfig,
} from '@compare-ui/core';

type ViewportSize = ImageSize;

type DesignComparisonExecutionCase<TArgs = never> = {
  name: string;
  viewport?: ViewportSize;
  reference: ImageResource;
  actual: ImageResource;
  compare?: CompareConfig;
  artifacts?: ArtifactInstruction[];
  reporting?: ReportingConfig;
  args?: Partial<TArgs>;
};

Typical inputs:

• name
• reference image resource
• actual image resource
• compare config
• artifact config
• reporting config

The runner uses @compare-ui/core to prepare the input resources before comparison.

That means workflows can still rely on core-owned image preparation behavior such as:

• crop handling
• explicit size normalization
• explicit background flattening

The image-processing rules themselves belong to @compare-ui/core. The runner's job is to execute the case, persist outputs, and shape the final run result.

Output config

The runner also needs an output location for the run.

type DesignComparisonOutputConfig = {
  outputFolder: string;
  runId: string;
};

Outputs are written to:

{outputFolder}/{name}/{runId}/{viewportX}x{viewportY}/

Typical contents:

• reference.png
• actual.png
• diff.png
• generated review artifacts such as overlay.png or grid-row.png
• comparison-report.json

Run result

runDesignComparisonCase(...) returns a structured result.

import type { AcceptanceResult, ComparisonMetrics, CropOffsets, ImageSize } from '@compare-ui/core';

type SourceReport = {
  originalSize: ImageSize;
  crop: Required<CropOffsets>;
  finalSize: ImageSize;
};

type DesignComparisonRunResult = {
  name: string;
  status: 'passed' | 'failed' | 'error';
  metrics?: ComparisonMetrics;
  acceptance?: AcceptanceResult;
  artifactPaths: string[];
  errors: string[];
  reportPath: string;
  sources: {
    reference: SourceReport;
    actual: SourceReport;
  };
};

This makes it easy to:

• print artifact paths in a CLI
• fail a Playwright test with useful output
• inspect a JSON report later

Basic example

import { gridRowArtifact, overlayArtifact } from '@compare-ui/core';
import { createFailureMessage, runDesignComparisonCase } from '@compare-ui/runner';

const result = await runDesignComparisonCase({
  executionCase: {
    name: 'example-screen',
    viewport: { x: 320, y: 640 },
    reference: {
      type: 'fs',
      path: './tests/fixtures/example-screen-reference-padded.png',
      crop: {
        bottom: 20,
      },
    },
    actual: {
      type: 'buffer',
      buffer: screenshotBuffer,
    },
    compare: {
      threshold: 0.1,
      includeAntiAliasedPixels: false,
      writeDiffArtifact: true,
      acceptance: {
        maxDiffPercent: 0.5,
      },
    },
    artifacts: [
      overlayArtifact(),
      gridRowArtifact({
        items: ['overlay', 'actual', 'reference'],
        size: 16,
        lineWidth: 2,
      }),
    ],
    reporting: {
      writeJsonReport: true,
      printArtifactPaths: true,
    },
  },
  output: {
    outputFolder: './design-tests',
    runId: crypto.randomUUID(),
  },
});

if (result.status !== 'passed') {
  throw new Error(createFailureMessage({ result }));
}

Minimal compare-only example

If you only need a compare run plus the persisted report and default compare artifacts, you can omit extra artifact instructions.

import { createFailureMessage, runDesignComparisonCase } from '@compare-ui/runner';

const result = await runDesignComparisonCase({
  executionCase: {
    name: 'example-screen',
    reference: {
      type: 'fs',
      path: './tests/fixtures/example-screen-reference.png',
    },
    actual: {
      type: 'buffer',
      buffer: screenshotBuffer,
    },
    compare: {
      threshold: 0.1,
      writeDiffArtifact: true,
      acceptance: {
        maxDiffPercent: 0.5,
      },
    },
  },
  output: {
    outputFolder: './design-tests',
    runId: crypto.randomUUID(),
  },
});

if (result.status !== 'passed') {
  throw new Error(createFailureMessage({ result }));
}

Failure messages

createFailureMessage(...) formats a run result into a readable message.

The message is intended to include:

• run name
• viewport
• metric summary
• report path
• artifact paths
• any collected errors

This is useful for:

• Playwright test failures
• Storybook-driven checks
• CLI output

Effective viewport

The runner uses the prepared reference image size for the final output segment.

Example:

const executionCase = {
  name: 'home-screen',
  viewport: { x: 320, y: 640 },
  reference: {
    type: 'fs',
    path: './tests/fixtures/home-screen-reference.png',
    crop: {
      bottom: 20,
    },
  },
  actual: {
    type: 'buffer',
    buffer: screenshotBuffer,
  },
};

If the reference image is cropped from 320x640 to 320x620, the run output is written under the 320x620 viewport segment.

Typical usage

The runner is intended to be used by higher-level packages that already know how to produce the actual screenshot:

• CLI commands that compare two existing images
• Playwright component-test fixtures
• Storybook-based screenshot tests

It is the package that turns a single comparison case into:

• generated PNG artifacts
• a persisted JSON report
• a structured run result

When a workflow needs size normalization or transparent-background alignment, use the core-owned preprocessing model and then execute the prepared case through the runner.