Voxel Render Tests

Reusable visual regression test harness for the Voxel application.

This package owns the render-test definitions, Voxel Puppeteer helper DSL, baseline image diffing, and render-test result uploads. Worker processes such as folio_assets_builder should own scheduling, browser lifetime, and machine deployment, then call this package as a library.

Library API

import { runRenderTestBatch } from "canaray-render-tests";

const summary = await runRenderTestBatch({
  browser,
  renderTestId,
  testCases,
  voxelUrl,
  caseId,
  stateVersion,
  viewport,
  queryParams,
});

browser is a Puppeteer Browser supplied by the worker host. testCases should include at least file_path and may include name, screenshot_name, viewport, or query_params. file_path can point to one visual test file or to a folder containing visual tests. Mixed folder/file lists are supported and run in the order supplied; folder contents are expanded recursively in sorted path order at that point in the list.

The runner imports each requested .visual.js file in register-only mode, executes the registered visual test against Voxel, downloads baselines, diffs screenshots, and uploads render-test results through the existing Canaray render-test API.

Lifecycle API

The old prepare.mjs and after.mjs scripts are exposed as importable functions:

import {
  afterRenderTest,
  prepareForRenderTest,
  runRenderTestRun,
} from "canaray-render-tests";

Use runRenderTestRun() when one worker job contains the complete render-test run:

const summary = await runRenderTestRun({
  browser,
  testCases,
  voxelUrl,
  caseId,
  stateVersion,
});

console.log(summary.renderTestId);

If renderTestId is omitted, runRenderTestRun() creates the render-test record before running the before lifecycle step. Pass renderTestId only when the render-test record has already been created by the caller.

For distributed or batched execution, run lifecycle steps once around all batches:

await prepareForRenderTest({
  testId: renderTestId,
  testType: "all/FDA_compliance",
  totalNumberOfTests: totalTestsAcrossAllBatches,
});

for (const batch of batches) {
  await runRenderTestBatch({ browser, renderTestId, testCases: batch });
}

await afterRenderTest({ testId: renderTestId });

Do not call afterRenderTest() after every batch. It groups results, generates evidence, adjusts missing result counts, and ends the render-test run.

Runtime Environment

The runner uses these environment variables when present:

• TEST_ID: render-test record ID used for result uploads.
• TEST_TYPE: category/path prefix used in result metadata.
• VOXEL_URL: Voxel URL to load. Defaults to local render-test URL.
• VOXEL_CASE_ID: case ID override.
• VOXEL_STATE_VERSION: state version override.
• CANARAY_USERNAME / CANARAY_PASSWORD: render-test API credentials.
• FORCE_HEADLESS=true: force headless mode for the legacy AVA launcher.

runRenderTestBatch() sets the relevant render-test environment variables temporarily for the duration of the batch and restores previous values after it finishes.

CLI

npx voxel-render-tests help
npx voxel-render-tests build-manifest --voxel-commit <sha>
npx voxel-render-tests enqueue --render-test-id <id> --voxel-commit <sha>
npx voxel-render-tests test:serial
npx voxel-render-tests after

Existing npm scripts wrap this CLI for compatibility.

Extracting To Its Own Repo

To preserve history when creating the new repository, use a history-filtering tool from the current slicaray repository root, for example:

git filter-repo --path rendertest/ --path-rename rendertest/:

Run that in a clone made specifically for extraction, then push the resulting repository to the private package repo. Do not run history filtering in the working application clone.

After publishing or private Git installing the package, folio_assets_builder can depend on it and import the library API directly.