visual-regression-testing

Wrapper utility for performing BackstopJS visual regression tests and storing artifacts (reference sets and reports) in S3.

Usage

AWS Credentials

The reference sets are stored in AWS S3, so you'll need to set your AWS credentials on your environment prior to running this utility.

Running with NPX

# Run scenarios and collect a baseline
npx --yes --package @quantum-sec/visual-regression-testing --call 'vreg-test [args]'

# Approve the baseline to create a reference set
npx --yes --package @quantum-sec/visual-regression-testing --call 'vreg-approve [args]'

• The vreg-test command performs a visual regression test against the current reference set.
• The vreg-approve command promotes the most recent test result set as the new reference set.

Arguments

• --config – optionally specify a path to the .vregrc.json file. (default: $PWD)

Configuration

You should create a .vregrc.json file with the following configuration parameters:

{
  "project": "Example",
  "scenariosPath": "./vreg/scenarios/",
  "baseUrl": "https://quantum-sec-dev.refined.site",
  "viewports": [
    "sm",
    "lg",
    { "label": "custom", "width": 1024, "height": 768 },
  ],
  "s3BucketName": "vreg.dev.platform.quantum.security",
  "s3BucketRegion": "ap-southeast-1",
  "storageKey": "example"
}

• project – the name of the project or test suite (used for display purposes only)
• scenariosPath – the starting path to search for scenario (.vreg.js) files. (default: ./vreg/scenarios/)
• baseUrl - the URL of the application being tested; note that if you use localhost and run this on a build server the possibility of port binding conflict exists.
• viewports – a list of viewport sizes to test; this can be either a list of preset breakpoints (i.e. xs, sm, md, lg, xl, xxl, or max) or an object containing label, width, and height properties.
• s3BucketName – the name of the S3 bucket in which reference sets and reports are stored
• s3BucketRegion – the AWS region in which the S3 bucket resides
• storageKey – the folder within the S3 bucket in which reference sets and reports are stored

Defining Visual Regression Tests

The utility will search your project for .vreg.js files and include them in the test suite. These files should be CommonJS modules that export an object with the options provided by BackstopJS. See https://github.com/garris/BackstopJS#using-backstopjs

module.exports = [{
  label: 'example',
  url: '/',
  readySelector: 'h1.site-title',
  delay: 2000, // Allow time for images to load
}];

Note that the url property should be relative unless you're testing something outside of the baseUrl you defined in your .vregrc.json file (uncommon).