agentic-testing

Agentic testing for web applications

Use Zerocheck's QA agent to test your web application's user journeys. Turn a release checklist into browser checks with explicit expected outcomes, review the results, and save the tests for repeatable runs.

agentic-testing is an official command alias for Zerocheck, with the same commands and configuration. Examples below use agentic-testing; the main documentation uses zerocheck.

New to Zerocheck? Request pilot access. We provision a customer instance and help you configure your first handful of tests during hands-on onboarding.

Already onboarded? Run your saved tests, or open the instance URL supplied during onboarding to use the web app.

Before you start

Test execution requires a Zerocheck account, a provisioned customer instance and a project in that instance. There is no default public service.

For the CLI, have these ready:

• Node.js 20.19 or newer.
• Your instance URL and account access, supplied during onboarding.
• Your project files: zerocheck.yaml with the correct project, environment and app URL, plus at least one saved YAML test in its configured test directory. We set up these files with you during onboarding.
• A running app reachable from your machine, with the test accounts, fixtures and credential variables your checks need.

Local execution uses Zerocheck's AI service and uploads selected test definitions, page context and run evidence. It requires network access and authentication even though the browser runs on your machine.

Run your saved tests

Run from the app repository containing the project files above. Replace the instance URL with the one you received during onboarding.

npm install --save-dev --save-exact [email protected]
export ZEROCHECK_API="https://your-instance.example.com"
npx agentic-testing login
npx agentic-testing install
npx agentic-testing validate
npx agentic-testing run --json .zerocheck/latest-run.json

login prints a URL and code for you to approve in your browser. install downloads Chromium for local runs. validate checks the saved YAML without running a browser.

run executes the saved tests in the configured test directory, using default_environment from zerocheck.yaml. Confirm that environment points to the app you intend to test. When a results link is printed, open it to inspect outcomes, failed steps and screenshots after signing in. The JSON file also retains the run details. If publication fails, follow the upload-retry command printed by the CLI.

To run just one test, use its actual repository path:

npx agentic-testing run zerocheck/tests/your-test.yaml

Starting from the web app

Open your provisioned instance, choose a saved test and run it against the intended environment. You can use the web app without installing this package.

To bring an individual web test into your repository, use Download YAML and put that file in the configured test directory. This creates an independent copy. Web tests are edited in the web app; repository tests are edited in their files. The CLI does not automatically download your saved web tests.

For a reviewed checklist import, the CLI can save its drafted tests with import --draft IMPORT_ID, as shown below. An import ID is different from an individual saved-test ID or run ID.

After installing and logging in, replace the project ID and app URL, then initialize a new configuration:

npx agentic-testing init --project "YOUR_PROJECT_ID" --env dev --url "http://localhost:3000"

init creates zerocheck.yaml and an empty test directory. If the configuration already exists, edit that file to change its project or environments.

Add your onboarding-provided test YAML, download an agreed web test, or draft tests from a real release checklist:

npx agentic-testing install
npx agentic-testing import release-checklist.md --env dev

Import drafts and runs checks immediately. Review the displayed YAML and browser results before saving. Resolve missing inputs and inspect failed or incomplete checks. Replace IMPORT_ID with the import identifier printed by the command:

npx agentic-testing import --draft IMPORT_ID
npx agentic-testing validate
npx agentic-testing run --env dev

Saving a draft does not change a failed result into a pass. Keep the expected outcome when investigating a failure.

What a test looks like

Each repository test is a readable YAML file. Adapt this example to a route and outcome that exist in your app:

version: zerocheck/v1
name: Release notes are available
blocks_merge: true
steps:
  - Navigate to /releases
  - Verify the Release notes heading is visible

Coding agents and other runners

After completing project setup, a compatible coding agent can use the same CLI through its local stdio MCP server:

npx agentic-testing mcp --project-dir "/absolute/path/to/your/app"

Configure the process in your MCP client using the MCP guide. Device sign-in needs your approval. The MCP tools can inspect, draft, save and run tests, then read their results.

For hosted execution, add --runner hosted to run; Chromium installation on your machine is unnecessary for that runner. The hosted browser must be able to reach the configured app URL. It cannot automatically reach your laptop's localhost. Hosted CLI runs still use repository configuration and test files.

When a run needs attention

| Situation | Next step | | --- | --- | | No instance configured | Set ZEROCHECK_API to your customer instance before login. | | No configuration or tests found | Add the project configuration and saved YAML tests. init creates an empty test directory. | | Missing test credentials | Supply the variables referenced by your configured environment. | | A test fails or cannot complete | Open the result and inspect the failed step and evidence. | | Result publication fails | Follow the upload-retry command printed by the CLI. |

Exit codes: 0 means the run completed within the configured merge policy; 1 means a blocking failure; 2 means a setup, incomplete-execution, infrastructure or result-publication error.

Request pilot access · Developer resources · CLI guide · Web app guide · Results