QAA - QA Automation Agent

Multi-agent QA pipeline for Claude Code. Analyzes any codebase, generates a complete test suite following industry standards, validates everything, and delivers the result as a draft pull request.

scan → map → research → analyze → plan → generate → validate → deliver

The Problem

• Starting from zero is painful — a new project with no tests means weeks of setup
• Coverage gaps are invisible — without analysis, teams don't know what's missing until production breaks
• Standards drift — different team members write tests differently: inconsistent locators, vague assertions, mixed naming
• QA is always behind dev — features ship faster than tests get written

The Solution

QAA runs a pipeline of 12 specialized AI agents, each responsible for one stage:

| Stage | What happens | Output | |-------|-------------|--------| | Scan | Detects framework, language, testable surfaces | SCAN_MANIFEST.md | | Research | Investigates testing ecosystem via Context7 MCP and official docs | TESTING_STACK.md, FRAMEWORK_CAPABILITIES.md | | Map | Deep-scans codebase with 4 parallel agents (testability, risk, patterns, existing tests) | 8 codebase documents | | Analyze | Produces risk assessment, test inventory, testing pyramid | QA_ANALYSIS.md, TEST_INVENTORY.md | | Plan | Groups test cases by feature, assigns to files, resolves dependencies | GENERATION_PLAN.md | | Generate | Writes test files, POMs, fixtures, configs following project standards | Test suite on disk | | Validate | 4-layer validation (syntax, structure, dependencies, logic) with auto-fix | VALIDATION_REPORT.md | | Deliver | Creates branch, commits per stage, opens draft PR | Pull request URL |

Install

npx qaa-agent

The interactive installer:

1. Copies agents, commands, skills, templates, and workflows into your runtime directory
2. Registers two MCP servers in your user-scope config (~/.claude.json) so they're available in all projects:
   • Playwright MCP — live browser control for E2E tests and locator extraction
   • Context7 MCP — up-to-date library documentation on demand
3. Merges required permissions into settings.json

Supported runtimes: Claude Code, OpenCode

Install scope: Global (~/.claude/, available in all projects) or Local (./.claude/, this project only)

Requirements

• Node.js 18+
• Claude Code installed

Bundled MCP servers

Both MCP servers are registered automatically in ~/.claude.json when you run npx qaa-agent. No manual setup required — once installed, they're available in every Claude Code project on your machine.

Playwright MCP — live browser control

Uses @playwright/mcp to:

• Open a real browser and navigate your running app
• Extract actual locators (data-testid, ARIA roles, labels) from live pages
• Run E2E tests, capture failures, and auto-fix locator mismatches
• Build a persistent Locator Registry (.qa-output/locators/) that caches real locators across features

Context7 MCP — up-to-date library docs

Uses @upstash/context7-mcp to:

• Fetch the latest documentation for Playwright, Cypress, Jest, Vitest, pytest, and any other library the agent is working with
• Keep generated tests aligned with current framework APIs instead of outdated training data
• Free tier: ~60 requests/hour, ~3,300 tokens/query

Verifying the MCPs are connected

Open Claude Code in any project and type /mcp. You should see both playwright and context7 listed as connected.

Manual config (fallback)

If for any reason the automatic registration fails, you can add the servers manually to ~/.claude.json:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest"]
    },
    "context7": {
      "command": "npx",
      "args": ["-y", "@upstash/context7-mcp@latest"]
    }
  }
}

Quick Start

New project, no tests

/qa-start --dev-repo ./myproject --auto

Runs the full pipeline end-to-end: scan, map, analyze, plan, generate, validate, and deliver as a draft PR.

Mature project, new feature

/qa-map                                          # build the "brain" (once)
/qa-create-test "password reset"                 # generate tests using codebase knowledge
/qa-pr --ticket PROJ-123 "password reset tests"  # ship as draft PR

From a Jira ticket

/qa-from-ticket https://company.atlassian.net/browse/PROJ-456
/qa-pr --ticket PROJ-456 "login flow tests"

Run pipeline against a live app

/qa-start --dev-repo ./myproject --app-url https://staging.example.com --auto

The --app-url flag tells the pipeline that the app is running at that URL. The E2E runner, validator (Layer 5), and testid-injector use it to verify against the real DOM.

Generate tests directly from a URL (single-shot)

/qa-create-test https://practicetestautomation.com/practice-test-login/

The new from-aut mode in /qa-create-test navigates the URL with Playwright MCP, discovers interactive elements, and generates tests directly without running the full pipeline. Faster than /qa-start --app-url when you only need tests for one specific page.

Natural language input (any command)

/qa-start use playwright on my project at C:\Users\me\My Project test against https://staging.example.com

The intent detector extracts URLs, paths (with spaces), and framework hints from natural language. Flags always win over NL. The INPUT DETECTION banner shows the resolved values before starting.

Fix broken tests after a deploy

/qa-fix ./tests/e2e/checkout*
/qa-pr --ticket PROJ-789 "fix checkout tests"

Commands

| Command | Purpose | |---------|---------| | /qa-start | Full pipeline end-to-end (scan through PR) | | /qa-research | Research testing ecosystem via Context7 MCP | | /qa-map | Deep codebase analysis with 4 parallel agents | | /qa-create-test <feature> | Generate tests for a specific feature | | /qa-fix [path] | Diagnose and fix broken tests | | /qa-audit [path] | 6-dimension quality audit with scoring | | /qa-pr | Create a draft pull request from QA artifacts | | /qa-testid [path] | Inject data-testid attributes into components |

Additional Commands

| Command | Purpose | |---------|---------| | /qa-from-ticket <url> | Generate tests from a Jira/Linear/GitHub Issue | | /qa-analyze | Analyze a repo without generating tests | | /qa-validate [path] | Validate test files against standards | | /qa-gap | Find coverage gaps between dev and QA repos | | /qa-report | Generate a QA status report | | /qa-audit | Full quality audit with weighted scoring | | /qa-blueprint | Generate QA repo structure from scratch | | /qa-research | Research best testing stack for a project | | /qa-pom | Generate Page Object Models | | /update-test | Improve existing tests incrementally |

Run any command in Claude Code to see full usage and available flags.

Three Workflows

QAA adapts to the project's QA maturity:

Option 1: No QA repo yet — Full pipeline from scratch. Produces a complete test suite, repo blueprint, and draft PR.

/qa-start --dev-repo ./myproject

Option 2: Immature QA repo — Scans both repos, fixes broken tests, fills coverage gaps, standardizes existing tests.

/qa-start --dev-repo ./myproject --qa-repo ./tests

Option 3: Mature QA repo — Surgical additions only. Finds thin coverage areas and adds targeted tests without touching working code.

/qa-start --dev-repo ./myproject --qa-repo ./tests

The "Brain" — Codebase Map

Before generating anything, QAA maps the codebase with 4 parallel agents producing 8 documents:

| Focus | Documents | |-------|-----------| | Testability | TESTABILITY.md, TEST_SURFACE.md — what's testable, entry points, mock boundaries | | Risk | RISK_MAP.md, CRITICAL_PATHS.md — business-critical paths, security-sensitive areas | | Patterns | CODE_PATTERNS.md, API_CONTRACTS.md — naming conventions, API shapes, import style | | Existing tests | TEST_ASSESSMENT.md, COVERAGE_GAPS.md — current quality, frameworks, gaps |

Every downstream agent reads these documents. The result: generated tests feel native to the codebase, not generic boilerplate.

Standards Enforced

Every generated artifact follows strict rules:

Testing Pyramid

         /  E2E  \        3-5%   (critical path smoke only)
        /  API    \       20-25% (endpoints + contracts)
       / Integration\     10-15% (component interactions)
      /    Unit      \    60-70% (business logic, pure functions)

Locator Hierarchy

1. Tier 1 (Best): data-testid, ARIA roles with accessible names
2. Tier 2 (Good): Form labels, placeholders, visible text
3. Tier 3 (Acceptable): Alt text, title attributes
4. Tier 4 (Last Resort): CSS selectors, XPath — always with a // TODO comment

Page Object Model

• One class per page, no god objects
• No assertions in POMs — assertions belong in test specs
• Locators as readonly properties
• Every POM extends a shared BasePage

Assertion Quality

// Good — concrete values
expect(response.status).toBe(200);
expect(data.name).toBe('Test User');

// Bad — never do this
expect(response.status).toBeTruthy();
expect(data).toBeDefined();

Test Case IDs

Every test case has a unique ID following the pattern:

• UT-MODULE-001 — unit tests
• INT-MODULE-001 — integration tests
• API-RESOURCE-001 — API tests
• E2E-FLOW-001 — E2E tests

Validation

Generated tests pass through a 4-layer validation with auto-fix (up to 3 loops):

1. Syntax — does it parse? Are imports correct?
2. Structure — POM rules, file organization, naming conventions
3. Dependencies — all imports resolve, mocks set up correctly
4. Logic — assertions are concrete, locators follow tier hierarchy

If issues remain, the Bug Detective classifies each failure:

| Classification | Action | |----------------|--------| | APPLICATION BUG | Flagged for developer — not auto-fixed | | TEST CODE ERROR | Auto-fixed at HIGH confidence | | ENVIRONMENT ISSUE | Documented with setup instructions | | INCONCLUSIVE | Flagged with evidence for manual review |

Framework Support

QAA auto-detects the project's existing stack and matches it:

Languages: JavaScript/TypeScript, Python, Java, .NET/C#, Go, Ruby, PHP, Rust

Test Frameworks: Playwright, Cypress, Jest, Vitest, pytest, Selenium, Robot Framework, and more

Build Tools: Vite, Next.js, Nuxt, Angular, Vue, Webpack, SvelteKit

Git Platforms: GitHub, Azure DevOps, GitLab

Learning System

QAA remembers your preferences across sessions. When you correct it — "use Playwright, not Cypress" or "our branches start with feature/" — it saves the rule permanently to MY_PREFERENCES.md. Every agent reads your preferences before generating output.

Your team's conventions always win over defaults.

Architecture

qaa-agent/
  agents/          # 12 specialized QA agents
  commands/        # 9 slash commands (user-facing entry points)
  skills/          # 6 reusable skills
  templates/       # 10 artifact templates (output format contracts)
  workflows/       # 7 workflow orchestration specs
  frameworks/      # 7 framework registry entries (one JSON per framework)
  bin/             # Installer + intent-detector + context7-cache helpers
  bin/lib/         # intent-detector.cjs, context7-cache.cjs (shared modules)
  docs/            # User documentation
  CLAUDE.md        # QA standards (read by every agent)
  .mcp.json        # Playwright + Context7 MCP server config
  settings.json    # Claude Code permissions
  VERSION          # Current version (mirrors package.json version)

Agents

| Agent | Responsibility | |-------|---------------| | qa-scanner | Framework detection, file tree scanning | | qa-codebase-mapper | 4-parallel-agent deep analysis | | qa-analyzer | Risk assessment, test inventory, pyramid | | qa-planner | Test case grouping, file assignment | | qa-executor | Test file, POM, fixture generation | | qa-validator | 4-layer validation with auto-fix | | qa-e2e-runner | Browser-based test execution via Playwright MCP | | qa-bug-detective | Failure classification with evidence | | qa-testid-injector | data-testid attribute injection | | qa-project-researcher | Testing stack research | | qa-discovery | Project discovery | | qa-pipeline-orchestrator | Pipeline coordination |

Git Workflow

QAA follows strict git conventions:

• Branch: qa/auto-{project}-{date} (e.g., qa/auto-shopflow-2026-03-18)
• Commits: One per agent stage — qa(scanner): produce SCAN_MANIFEST.md for shopflow
• PR: Draft PR with analysis summary, test counts, coverage metrics, validation status

Documentation

All documentation is included in the installed package under docs/, templates/, and CLAUDE.md.

License

MIT