Voxel Render Tests

Reusable visual regression test harness for the Voxel application.

This package owns the render-test definitions, Voxel Puppeteer helper DSL, baseline image diffing, and render-test result uploads. Worker processes such as folio_assets_builder should own scheduling, browser lifetime, and machine deployment, then call this package as a library.

Library API

import { runRenderTestBatch } from "canaray-render-tests";

const summary = await runRenderTestBatch({
  browser,
  renderTestId,
  testCases,
  voxelUrl,
  caseId,
  stateVersion,
  viewport,
  queryParams,
});

browser is a Puppeteer Browser supplied by the worker host. testCases should include at least file_path and may include name, screenshot_name, viewport, or query_params. file_path can point to one visual test file or to a folder containing visual tests. Mixed folder/file lists are supported and run in the order supplied; folder contents are expanded recursively in sorted path order at that point in the list.

The runner imports each requested .visual.js file in register-only mode, executes the registered visual test against Voxel, downloads baselines, diffs screenshots, and uploads render-test results through the existing Canaray render-test API.

voxelUrl overrides the default http://localhost:8080 Voxel location for that run. It is read at execution time, so long-running workers can supply a different URL for each job without restarting the worker process.

Lifecycle API

The old prepare.mjs and after.mjs scripts are exposed as importable functions:

import {
  afterRenderTest,
  buildDistributedFinalizerJob,
  finalizeDistributedRenderTest,
  prepareForRenderTest,
  runRenderTestRun,
} from "canaray-render-tests";

Use runRenderTestRun() when one worker job contains the complete render-test run:

const summary = await runRenderTestRun({
  browser,
  testCases,
  voxelUrl,
  caseId,
  stateVersion,
});

console.log(summary.renderTestId);

If renderTestId is omitted, runRenderTestRun() creates the render-test record before running the before lifecycle step. Pass renderTestId only when the render-test record has already been created by the caller.

For distributed execution, prepare once and run every batch without per-job lifecycle hooks. Exactly one batch is designated as the completion coordinator:

await prepareForRenderTest({
  testId: renderTestId,
  testType: "all/FDA_compliance",
  totalNumberOfTests: totalTestsAcrossAllBatches,
});

// In ordinary worker processes:
await runRenderTestBatch({
  browser,
  renderTestId,
  testCases: assignedBatch,
});

// The designated Folio job uses the complete lifecycle wrapper:
await runRenderTestRun({
  browser,
  renderTestId,
  testCases: assignedBatch,
  totalNumberOfTests: totalTestsAcrossAllBatches,
  before: false,
  after: false,
  coordinateCompletion: true,
});

After its own tests finish, the coordinator polls shared server-side result counters every 15 seconds. At 100% it runs strict finalization, which requires a complete, non-duplicated set of server-side test records and does not read worker-local screenshots, diffs, or error files.

If more than 6/7 of expected results are present and the count does not change for five minutes, the coordinator closes the run as incomplete and records the missing count as failed. It does not publish partial Jira/Ketryx compliance evidence. Runs below that ratio have a two-hour absolute coordination timeout so a Folio Maker cannot wait forever. These defaults can be overridden with coordinatorOptions.

finalizeDistributedRenderTest() and a standalone finalizer job remain available for manual recovery:

const finalizerJob = buildDistributedFinalizerJob({
  renderTestId,
  expectedTotal: totalTestsAcrossAllBatches,
  repository: "canaraydotcom/WebGL2-DICOM-Viewer",
  prNumber,
  headSha,
  suiteVersion,
  dispatchKey,
});

The resulting render_test_finalize job contains no test cases and does not open Voxel.

The legacy afterRenderTest() entry point retains local-result fallback behavior for standalone CLI execution.

Jira / Ketryx Publishing

afterRenderTest() always writes the aggregate Ketryx artifacts under test-results/. When JIRA_PUBLISH_EXECUTIONS=true is set, it also creates one Jira Test Execution per mapped VT/Jira test item and attaches VT-scoped JUnit XML plus JSON evidence files. Set JIRA_PUBLISH_REQUIRED=true to fail the run when Jira publishing fails.

Required Jira configuration can come from environment variables or the excluded services/jiraCredentials.json file:

• JIRA_API_TOKEN: Basic auth token value.
• JIRA_DOMAIN: Jira host, for example https://example.atlassian.net.
• JIRA_REST_API_PATH: REST API path, usually rest/api/3.
• JIRA_TEST_EXECUTION_ISSUE_TYPE: defaults to Test Execution.
• JIRA_EXECUTES_LINK_TYPE: defaults to Executes.
• JIRA_TEST_EXECUTION_FIELDS: optional JSON object merged into created issue fields.
• JIRA_COPIED_FIELD_IDS: optional comma-separated source issue fields to copy.

Runtime Environment

The runner uses these environment variables when present:

• TEST_ID: render-test record ID used for result uploads.
• TEST_TYPE: category/path prefix used in result metadata.
• VOXEL_URL: Voxel URL to load. Defaults to local render-test URL.
• VOXEL_CASE_ID: case ID override.
• VOXEL_STATE_VERSION: state version override.
• CANARAY_USERNAME / CANARAY_PASSWORD: render-test API credentials.
• FORCE_HEADLESS=true: force headless mode for the legacy AVA launcher.

runRenderTestBatch() sets the relevant render-test environment variables temporarily for the duration of the batch and restores previous values after it finishes.

CLI

npx voxel-render-tests help
npx voxel-render-tests build-manifest --voxel-commit <sha>
npx voxel-render-tests enqueue \
  --render-test-id <id> \
  --voxel-commit <sha> \
  --repository canaraydotcom/WebGL2-DICOM-Viewer \
  --pr-number <number> \
  --dispatch-key <key>
npx voxel-render-tests test:serial
npx voxel-render-tests after

enqueue deterministically expands the packaged manifest into exactly seven balanced Folio Maker jobs and posts { "jobs": [...] } to https://queue.canaray.com/api/v1/bullmq/folio_maker_queue/job/bulk. It obtains a Bearer JWT using CANARAY_USERNAME and CANARAY_PASSWORD. Override the endpoint with QUEUE_CANARAY_URL, pass an existing JWT with --auth, or use --dry-run to inspect the complete payload. Jobs contain non-overlapping folder paths rather than hundreds of individual test records. The jobs are ordered by estimated test count, and the largest first slice carries coordinate_completion: true; the other six slices finish immediately after uploading their assigned results.

Existing npm scripts wrap this CLI for compatibility.

Extracting To Its Own Repo

To preserve history when creating the new repository, use a history-filtering tool from the current slicaray repository root, for example:

git filter-repo --path rendertest/ --path-rename rendertest/:

Run that in a clone made specifically for extraction, then push the resulting repository to the private package repo. Do not run history filtering in the working application clone.

After publishing or private Git installing the package, folio_assets_builder can depend on it and import the library API directly.