Verity

Verity is an MCP server for QA automation. It connects your AI client directly to your test suite — probe a URL for testable UI modules, generate compilable test files, execute the suite, diagnose failures, and receive a prioritized improvement plan. All through natural language.

Use Cases

• Test generation from a live page: Point Verity at a URL. It opens headless Chromium, maps the DOM into typed modules (forms, navs, dialogs, CTAs), intercepts live API calls, and generates a runnable test file with pre-filled selectors and assertions — not a stub.
• Failure diagnosis: After a failing run, ask for failure details. Verity returns the error message, every Playwright API call extracted from the trace, and direct paths to the screenshot, video, and trace file.
• Self-improving suite health: After enough runs, ask for an optimization plan. Verity classifies each test as broken, flaky, slow-regressing, or stable — and surfaces coverage gaps from previously analyzed URLs.
• Domain-aware test generation: Add a qa-knowledge.md with your business rules and historical bugs. Generated tests embed that context directly as comments, escaping generic stub output.

Installation

npm install -g qa-mcp-verity

Or run without a global install:

npx qa-mcp-verity

Client Setup

Two environment variables are required for every client:

| Variable | Description | |---|---| | TEST_RUNNER | Test framework to use. Default: playwright | | QA_PROJECT_ROOT | Absolute path to the project under test |

Cursor

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (per-project):

{
  "mcpServers": {
    "verity": {
      "command": "npx",
      "args": ["qa-mcp-verity"],
      "env": {
        "TEST_RUNNER": "playwright",
        "QA_PROJECT_ROOT": "/absolute/path/to/your-project"
      }
    }
  }
}

Open Cursor Settings → MCP and confirm Verity appears as connected.

Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "verity": {
      "command": "npx",
      "args": ["qa-mcp-verity"],
      "env": {
        "TEST_RUNNER": "playwright",
        "QA_PROJECT_ROOT": "/absolute/path/to/your-project"
      }
    }
  }
}

Restart Claude Desktop after saving.

Claude Code

claude mcp add verity npx qa-mcp-verity \
  -e TEST_RUNNER=playwright \
  -e QA_PROJECT_ROOT=/absolute/path/to/your-project

Verify the server is registered:

claude mcp list

Gemini CLI

Edit ~/.gemini/settings.json:

{
  "mcpServers": {
    "verity": {
      "command": "npx",
      "args": ["qa-mcp-verity"],
      "env": {
        "TEST_RUNNER": "playwright",
        "QA_PROJECT_ROOT": "/absolute/path/to/your-project"
      }
    }
  }
}

Codex CLI

Edit ~/.codex/config.toml:

[mcp_servers.verity]
command = "npx"
args    = ["verity"]

[mcp_servers.verity.env]
TEST_RUNNER     = "playwright"
QA_PROJECT_ROOT = "/absolute/path/to/your-project"

Supported Runners

| TEST_RUNNER value | Framework | Generated test format | |---|---|---| | playwright / playwright-ts | @playwright/test | .spec.ts | | pytest / pytest-playwright | pytest-playwright | .py | | jest | Jest | .test.ts | | cypress | Cypress | .cy.ts | | go / go-test | go test | _test.go |

Tools

Discovery

| Tool | Description | |---|---| | get_runner_info | Returns the active runner and all available options. Call this first in every session. | | list_tests | Native test collection from the active runner. |

Execution

| Tool | Description | |---|---| | run_tests | Run the full suite. Accepts filter, browser (chromium/firefox/webkit), and headed. | | run_failed | Re-run only the tests that failed in the last run. | | codegen | Open an interactive Playwright recording session. |

Reporting

| Tool | Description | |---|---| | get_test_report | Cached summary from the last run (total, passed, failed, skipped, duration). | | get_failure_details | Full failure packet: error, trace steps, screenshot/video/trace paths. Optionally filter by test_id. | | get_test_history | Per-run pass rates over the last N runs, oldest-first for trend analysis. | | generate_html_report | Self-contained HTML report with screenshots inlined. No external dependencies. |

Generation

| Tool | Description | |---|---| | analyze_url | Probe a URL with headless Chromium. Returns typed modules with selectors and candidate test cases, plus live API calls and layout overflow warnings. | | generate_test | Generate a runnable test file from a description. Pass a module from analyze_url to get pre-filled selectors instead of a stub. | | auto_generate_tests | analyze_url + generate_test in one call — one test per detected module. |

Optimization

| Tool | Description | |---|---| | get_optimization_plan | Three-layer analysis: broken/flaky/slow tests, MCP usage patterns, AI test adoption rate. Written to optimization-plan.md in the project root. |

Domain Knowledge

| Tool | Description | |---|---| | get_qa_context | Read qa-knowledge.md — business rules that produce domain-aware tests. Accepts an optional section name. | | init_qa_knowledge | Scaffold a starter qa-knowledge.md in the project root. |

Resources

| URI | Description | |---|---| | verity://report/html | Latest test report as HTML | | verity://report/json | Latest report.json | | verity://report/optimization | Latest optimization-plan.md |

Configuration

| Variable | Default | Description | |---|---|---| | TEST_RUNNER | playwright | Active test framework | | QA_PROJECT_ROOT | ./tests_project | Root directory for all file I/O | | QA_TIMEOUT_SECONDS | 600 | Subprocess timeout in seconds | | QA_KNOWLEDGE_FILE | <root>/qa-knowledge.md | Override path for the domain knowledge file |

Domain Knowledge

Generated tests use surface behavior by default — they fill forms, click buttons, and check for visible elements. To make them reflect your business rules:

# In your AI client:
init_qa_knowledge

Edit the scaffolded qa-knowledge.md to add business rules, historical bug regression points, exact UI error strings, and user journeys. On the next generate_test call, that context is embedded directly into the generated file.

See docs/qa-knowledge.example.md for a worked example across e-commerce, SaaS, content, and financial domains.

Development

npm install
npm run build      # compile src/ → dist/
npm run typecheck  # type-check without emitting
npm run dev        # watch mode

See docs/framework.md for the architecture reference: data flows, internal report format, security model, and how to add a new runner.

License

MIT