Kilotest

An ensemble testing service with a focus on accessibility

Features

This application uses an ensemble of 10 rule engines to test public web pages for front-end quality (i.e. accessibility, usability, and conformity to standards).

The testing paradigm employed by Kilotest is discussed in these papers:

• How to run a thousand accessibility tests
• Testaro: Efficient Ensemble Testing for Web Accessibility
• Accessibility Metatesting: Comparing Nine Testing Tools

Kilotest acts as a server with human and agent users and with one or more testing agents that obtain jobs from Kilotest and do the actual testing. Those agents are instances of the Testaro package.

An active production instance of Kilotest may require multiple testing agents to handle the load, because testing one web page typically takes about 3 minutes and agents test only one page at a time.

Getting started locally with 1 testing agent

Installation

In the steps below, hosts T and K may be the same host or two different hosts. Host T can be a Debian stable, Ubuntu LTS, Windows, or macOS host. Host K can be any server host that can run the latest LTS version of Node.js. If hosts T and K differ, then they must be open to https traffic and host K must permit https requests from host T.

1. Clone the Testaro project into a new directory on host T.
2. In that directory, install the Testaro dependencies: npm install.
3. Update the Testaro dependencies and rebuild: npm run deps.
4. Clone the Kilotest repository into a new directory on host K.
5. In that directory, install the Kilotest dependencies: npm install.
6. Copy the env.testaro file from the kilotest directory to .env in the testaro directory and replace the __placeholder__ values in .env with actual values.
7. Copy the env.example file in the kilotest directory to a new .env file in the same directory and replace the __placeholder__ values in .env with actual values.

Usage

1. In the testaro directory, make Testaro start listening for jobs: node call netWatch true nn true, where nn is the number of seconds to wait between checks for new jobs.
2. In the kilotest directory, start the Kilotest service: node index.

Contributing

Contributions are welcome! You can use GitHub issues to initiate discussions and propose changes. If you want to contribute code, please fork the repository and create a pull request.

Making Kilotest a service

See the SERVICE.md file for instructions on how to make Kilotest a service.

Using Kilotest as an AI tool

See the AI-TOOL.md file for information about the configuration of Kilotest as an AI tool.