Crawler for web accessibility

This is a web accessibility crawler written in Node.js

Getting Started

To get started with the crawler, follow these steps:

1. Install dependencies with npm install
2. Run the crawler for a domain with DOMAIN=<domain> npm run crawl. For example, DOMAIN=deranged.dk npm run crawl runs for deranged.dk and www.deranged.dk. DOMAIN defaults to digst.dk.

• You can also set a env var LIMIT to limit the number of pages crawled in the crawl. E.g. DOMAIN=kbhbilleder.dk LIMIT=10000 npm run crawl to limit to ten thousand pages that finish evaluation (skipped or failed pages do not count towards the total).

Running on Ubuntu

When running the crawler locally on ubuntu, you can get an error because of AppArmor which restricts puppeteer.

You need to run the following script, where CHROMIUM_BUILD_PATH is the path to puppeteer's chrome. Example: /home/user/.cache/puppeteer/chrome/linux-140.0.7339.80/chrome-linux64/chrome. The correct path can be found in the AppArmor error message.

export CHROMIUM_BUILD_PATH=/@{HOME}/chromium/src/out/**/chrome
cat | sudo tee /etc/apparmor.d/chrome-dev-builds <<EOF
abi <abi/4.0>,
include <tunables/global>

profile chrome $CHROMIUM_BUILD_PATH flags=(unconfined) {
  userns,

  # Site-specific additions and overrides. See local/README for details.
  include if exists <local/chrome>
}
EOF
sudo service apparmor reload  # reload AppArmor profiles to include the new one