@qa-gentic/stlc-agents

AI-powered QA STLC automation — from Azure DevOps or Jira Cloud work item state change to self-healing Playwright TypeScript in a Helix-QA project.

Works with GitHub Copilot (VS Code Agent mode), Claude Code, Cursor, and Windsurf.
Also runs fully headless via the webhook bridge — no human in the loop required.

What It Does

Five Python MCP servers cover the full QA Software Test Life Cycle. A sixth server (Playwright MCP) drives a real browser during code generation.

| Agent | Server name | Input | Output | |---|---|---|---| | Agent 1 | qa-test-case-manager | ADO PBI / Bug / Feature | Manual test cases created & linked (TestedBy-Forward), deduped on re-trigger | | Agent 2 | qa-gherkin-generator | ADO Feature / PBI / Bug | .feature file validated and attached to work item | | Agent 3 | qa-playwright-generator | Gherkin + optional AX-tree context_map | locators.ts + page objects + step defs (cached, retrieved via get_generated_files) | | Agent 4 | qa-helix-writer | Generated .ts files + helix_root | Files written to Helix-QA directory layout, never overwrites | | Agent 5 | qa-jira-manager | Jira Story / Bug / Task | Test cases created & linked in Jira, .feature attached to issue, deduped on re-trigger |

End-to-End Flow

Webhook-triggered (headless)

A work item state change in ADO or Jira fires a webhook POST to the bridge. The bridge normalises the payload and routes it through the pipeline automatically.

ADO Service Hook / Jira Webhook / GitHub Issues Webhook
        │  POST  /webhook/ado | /webhook/jira | /webhook/github
        ▼
webhook_orchestrator/webhook_bridge.py  (FastAPI — qa-stlc-webhook)
        │
        ├─ _parse_*_webhook()         raw payload → WebhookEvent (state transitions only)
        ├─ orchestrator.classify_state  event → test_cases | gherkin_playwright | skip
        │
        └─ orchestrator.run_pipeline
              ├─ test_cases          → Agent 1 (ADO) or Agent 5 (Jira)
              └─ gherkin_playwright  → Agent 2 → Agent 3 → Agent 4

State → stage mapping

Routing is purely state-driven — the work item state in the webhook payload is the single source of truth. There are no env-var or payload overrides, and events that do not contain an actual state transition (title edits, comments, label noise) are dropped.

| Platform | Work item state | Stage | Agents called | |---|---|---|---| | ADO | Active / Approved / Committed / In Progress / In Review / Ready for Testing | test_cases | Agent 1 only | | ADO | Resolved / Done / Closed / Completed / Released | gherkin_playwright | Agents 2 → 3 → 4 | | Jira | In Progress / Active / In Development / In Review / Ready for QA / Approved | test_cases | Agent 5 only | | Jira | Resolved / Done / Closed / Completed / Released | gherkin_playwright | Agents 2 → 3 → 4 | | GitHub | workflow label added (in progress, ready for testing, …) | test_cases | pending qa-github-manager | | GitHub | issue closed | gherkin_playwright | pending qa-github-manager |

Any other state is dropped with an explicit {"accepted": false, "stage": "skip"} ack. State sets live in webhook_orchestrator/orchestrator.py. GitHub events are parsed and routed; the GitHub agent itself is not implemented yet (pipelines/github_issues.py reports blocked until a qa-github-manager MCP server exists).

STAGE_TEST_CASES — Agent 1 / Agent 5

fetch_work_item / fetch_jira_issue
        ↓
[LLM] generate_test_cases()        ← pipeline bridges fetch → LLM → create
        ↓
create_deduped_test_cases()        ← internally: get_linked → filter dupes → create_and_link

On re-trigger, titles already linked are skipped (case-insensitive, stop-word normalised). Only net-new test cases are created.

STAGE_FULL_PIPELINE — Agents 2 → 3 → 4

First error stops the chain and surfaces in the result dict. Subsequent agents are not called.

Agent 2 — Gherkin

ADO Feature : fetch_feature_hierarchy  →  [LLM] generate_gherkin
ADO PBI/Bug : fetch_work_item_for_gherkin  →  [LLM] generate_gherkin
Jira        : fetch_jira_issue (via Agent 5)  →  [LLM] generate_gherkin
        ↓
validate_gherkin_content()             ← structural check before attach
        ↓  (if invalid → pipeline stops, returns validation errors)
ADO Feature : attach_gherkin_to_feature()
ADO PBI/Bug : attach_gherkin_to_work_item()
Jira        : attach_gherkin_to_issue()    ← uploads via Jira attachment API

The generate_and_attach_gherkin composite tool wraps validate + attach for CI/headless callers.

Agent 3 — Playwright

generate_playwright_code(gherkin_content, page_class_name)
        ↓  returns cache_key (files held in-memory)
get_generated_files(cache_key)
        ↓  returns { "path/to/file.ts": "content", ... }

page_class_name is derived from event["title"] (ADO) or event["summary"] (Jira) — camel-cased, max 4 words. Without a context_map, locators are Gherkin-inferred (stability=0). Pass app_url to embed a snapshot hint comment in locators.ts.

Agent 4 — Helix writer

inspect_helix_project(helix_root)
        ↓  framework_state: absent | partial | present
write_helix_files(helix_root, files, mode)
        mode = "scaffold_and_tests"  if absent or partial
        mode = "tests_only"          if present

Agent 4 handles all deduplication and conflict renaming internally. No file is ever overwritten.

Install

# 1. Install the CLI + npm package globally
#    Prompted to choose integration: ado / jira / both
npm install -g @qa-gentic/stlc-agents

# 2. Bootstrap your project
qa-stlc init --vscode --integration ado    # GitHub Copilot / VS Code — ADO
qa-stlc init --vscode --integration jira   # GitHub Copilot / VS Code — Jira
qa-stlc init --vscode --integration both   # GitHub Copilot / VS Code — both
# or for Claude Code:
qa-stlc init --integration ado

# 3. Scaffold a new Playwright + Cucumber + TypeScript QA project
qa-stlc scaffold --name my-qa-project

# 4. Start the Playwright browser server (required for live-locator generation)
npx @playwright/mcp@latest --port 8931

Migrating an existing Playwright project

If you have an existing Playwright (with or without Cucumber) project and want to bring it onto the agent rails, use stlc-migrate:

# Preview without writing
stlc-migrate --source ./my-old-tests --helix ./helix-qa --dry-run

# Full migration — every agent skill, .mcp.json, healer scaffold, agent docs
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration both

# ADO-only or Jira-only
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration ado
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration jira

# Refresh scaffold/skills on an existing migrated tree.
# Also auto-removes stale .feature/.steps.ts/.page.ts/.locators.ts orphans
# from the role dirs (e.g. left over after switching --preserve-tree on/off).
stlc-migrate --source ./my-old-tests --helix ./helix-qa --conflict overwrite

# Preserve the source's subfolder grouping under features/steps/pages/locators
# (default flattens). Sub-dir names are kebab-cased.
stlc-migrate --source ./my-old-tests --helix ./helix-qa --preserve-tree

# Explicitly scaffold VRT (visual regression testing). Auto-enabled when the
# source already has vrt:* scripts or VRT_MODE in any .env file.
stlc-migrate --source ./my-old-tests --helix ./helix-qa --features=vrt

The tool emits a complete agent-ready tree:

| Layer | What lands | |---|---| | Code | Page objects + step files + locator files in {selector, intent, stability} format; every locator interaction routed through LocatorHealer.*WithHealing; non-BDD .spec.* files losslessly converted to one-scenario-per-test BDD | | Healing infra | 8-strategy LocatorHealer (attribute / type-hint / role / label / text / AI Vision) with intent-shape-aware attribute matching (:is(input, …)[id*="password"] instead of bare [id*="password"]), LocatorRepository with schema-versioned heal-only persist + per-env file suffix, HealingDashboard (read/write at 7890 + read-only mirror at HEALIX_REVIEW_PORT, Confirm/Revert clear the entry from view, Approve-all bulk action), TimingHealer, VisualIntentChecker | | VRT (opt-in) | src/helper/vrt/{vrtCapture,vrtReport}.util.ts, src/test/steps/vrt.steps.ts, vrt:baseline / vrt:test npm scripts, pngjs/pixelmatch/fs-extra runtime deps. Defaults are generic; project-specific masks via VRT_GLOBAL_MASK_SELECTORS env CSV. Enabled by --features=vrt or auto-detection. | | Agent assets | .mcp.json (all 6 MCP servers + Playwright MCP), .claude/skills/<name>/SKILL.md per integration, .github/copilot-instructions/<name>.md flat copies, AGENT-BEHAVIOR.md + ORCHESTRATION_RULES.md at root + .claude/ + .github/ | | Configs | cucumber.js with dotenv/config as first requireModule, package.json with dotenv + @types/* companions, tsconfig.json aliases, .env.example documenting every healing knob | | Report | MIGRATION-REPORT.md — counts, TODOs, removed orphans, VRT manual-wiring snippet (when source ships its own hooks) |

See skills/migrate-framework/SKILL.md for the full pipeline + environment variable reference.

Healing dashboard

Once a migrated project has run any test that triggers healing, inspect the results via the dashboard:

ENV=qa4 npm run healix:dashboard     # full UI at http://localhost:7890
ENV=qa4 npm run healix:review        # read-only mirror at http://localhost:7891

The dashboard shows each healed locator with its original selector, the specific selector that actually matched on the live page (introspected from the matched element — e.g. #login-username, not the generic search union), and Confirm / Revert buttons. Confirm rewrites the original locator .ts file's selector: field so the next run uses the verified selector as the primary probe — no healing chain required until the page changes again.

Cost Tracking Activation

npm install (npm install -g @qa-gentic/stlc-agents): Cost tracking is activated automatically after the Python servers are installed. No manual step required.

pip install (pip install qa-gentic-stlc-agents): Cost tracking is not activated automatically. You must run one of the following after pip install:

qa-stlc-apply-cost
# or
python -m stlc_agents.shared.install_hook

This patches all MCP servers to log tokens and cost for every tool call.

qa-stlc init does five things:

1. pip install qa-gentic-stlc-agents — installs all five Python MCP servers + rules files
2. Copies skill files to .github/copilot-instructions/ (and .claude/ if not --vscode)
3. Copies custom agent files to .github/agents/
4. Copies ORCHESTRATION_RULES.md to project root for reference during multi-step tasks
5. Writes .vscode/mcp.json (or .mcp.json) with all six servers configured

Requirements

| Requirement | Value | |---|---| | Node.js | >= 18 | | Python | >= 3.10, < 3.14 | | ADO PAT | Required for Agents 1–4 (ADO pipeline) | | Jira OAuth 2.0 (3LO) | Required for Agent 5 (Jira pipeline) | | Playwright MCP | npx @playwright/mcp@latest --port 8931 |

ADO .env vars:

ADO_ORGANIZATION_URL=https://dev.azure.com/your-org
ADO_PROJECT_NAME=YourProject
ADO_PAT=your-personal-access-token
APP_BASE_URL=https://your-app.example.com

# LLM — pick one provider:
AI_HEALING_PROVIDER=anthropic
AI_HEALING_API_KEY=sk-ant-...
# or: OPENAI_API_KEY / AZURE_OPENAI_API_KEY / GITHUB_TOKEN (Copilot)

Jira additional .env vars:

JIRA_CLIENT_ID=your-atlassian-oauth-client-id
JIRA_CLIENT_SECRET=your-atlassian-oauth-client-secret
JIRA_CLOUD_ID=your-atlassian-cloud-id

Webhook bridge additional .env vars:

WEBHOOK_SECRET=your-shared-secret
HELIX_PROJECT_ROOT=/path/to/helix-qa   # where Agent 4 writes files
PLAYWRIGHT_MCP_URL=http://localhost:8931/mcp   # leave blank to skip Agent 3

CLI Commands

| Command | Description | |---|---| | qa-stlc init [--vscode] [--integration ado\|jira\|both] | Full bootstrap: pip install + skills + agent files + MCP config | | qa-stlc scaffold [--name n] [--dir path] | Copy full Playwright + Cucumber + TypeScript boilerplate to a new project | | qa-stlc skills [--target claude\|vscode\|cursor\|windsurf] | Copy skill files to the correct AI coding agent directory | | qa-stlc mcp-config [--vscode] [--print] | Write .vscode/mcp.json or .mcp.json with all servers configured | | qa-stlc verify | Check that all MCP servers are reachable and auth is cached | | qa-stlc-webhook [--host] [--port] [--reload] | Start the webhook bridge (FastAPI — requires pip install qa-gentic-stlc-agents[webhook]) |

Orchestration Rules

When qa-stlc init is run, ORCHESTRATION_RULES.md is installed to your project root and placed in both npm (node_modules/@qa-gentic/stlc-agents/) and pip (site-packages/stlc_agents/) installations.

Refer to this file in multi-step QA workflows to ensure:

• Step breakdown: every task is decomposed into named steps with explicit inputs/outputs
• Pre-flight gates: output validation runs before generation, not after
• No skipped steps: intermediate data is structured and handed off explicitly, never inferred
• Countable verification: code generation steps state expected item counts before implementation
• Zero stubs: no partial outputs with TODOs or empty bodies passed downstream

Integration per AI coding agent:

• GitHub Copilot (VS Code): Add to .github/copilot-instructions.md or reference in your workspace prompt
• Claude Code: Reference in .claude/instructions.md or as a project document
• Cursor: Add to .cursor/rules/orchestration.mdc (scope: always)
• Windsurf: Reference in .windsurf/rules.md or global rules panel

See section 8 of ORCHESTRATION_RULES.md for agent-specific integration notes.

Tool Reference

Agent 1 — qa-test-case-manager (Azure DevOps)

| Tool | Description | |---|---| | fetch_work_item | Fetch a PBI, Bug, or Feature with acceptance criteria, coverage hints, and existing TC count. Returns epic_not_supported for Epics. | | get_linked_test_cases | List all test cases linked via TestedBy-Forward (used by dedup). | | create_and_link_test_cases | Create structured manual test cases and link to work item. Feature confirmation gate fires unless confirmed=true. | | create_deduped_test_cases | Headless/webhook tool. Internally calls get_linked_test_cases, filters duplicates (normalised title match), then calls create_and_link_test_cases on net-new only. Safe to call on every re-trigger. |

Agent 2 — qa-gherkin-generator (Azure DevOps)

| Tool | Description | |---|---| | fetch_feature_hierarchy | Fetch a Feature + all child PBIs/Bugs + coverage hints. | | fetch_work_item_for_gherkin | Fetch a single PBI or Bug with parent Feature context. | | validate_gherkin_content | Structural check: @smoke/@regression tags, scenario count (5–10 feature / 3–9 work_item), every scenario has When, no duplicate titles. | | attach_gherkin_to_feature | Validate + upload + link .feature to a Feature work item. | | attach_gherkin_to_work_item | Validate + upload + link .feature to a PBI or Bug. | | generate_and_attach_gherkin | Headless/webhook composite. Accepts pre-generated gherkin_content, validates, attaches. Returns status: validation_failed with errors if invalid — pipeline must re-generate. |

Agent 3 — qa-playwright-generator (ADO + Jira)

| Tool | Description | |---|---| | generate_playwright_code | Generate locators.ts, *Page.ts, *.steps.ts, cucumber-profile.js. Returns cache_key. Optional context_map for AX-tree-verified locators; optional app_url embeds snapshot hint. | | get_generated_files | Retrieve full file content by cache_key from Agent 3's in-memory cache. | | scaffold_locator_repository | Generate the five Helix-QA healing infrastructure files (LocatorHealer, TimingHealer, VisualIntentChecker, LocatorRepository, HealingDashboard). Call once per project. | | validate_gherkin_steps | Check for duplicate step strings and missing When steps. | | attach_code_to_work_item | Attach generated TypeScript delta files to an ADO work item. |

Agent 4 — qa-helix-writer (ADO + Jira)

| Tool | Description | |---|---| | inspect_helix_project | Returns framework_state: present / partial / absent. Drives mode selection in pipeline. | | write_helix_files | Write files to Helix-QA layout. mode=scaffold_and_tests for new/partial projects; mode=tests_only for existing. Never overwrites; deduplicates and conflict-renames. | | list_helix_tree | Full directory listing of a Helix-QA project. | | read_helix_file | Read an existing file for overlap detection. |

Agent 5 — qa-jira-manager (Jira Cloud)

| Tool | Description | |---|---| | fetch_jira_issue | Fetch a Story, Bug, or Task with acceptance criteria and coverage hints. Returns epic_use_hierarchy for Epics. | | get_linked_test_cases | List all issues linked via is tested by / Test link type (used by dedup). | | create_and_link_test_cases | Create test case issues in Jira (type Test, falls back to Task) and link. Steps stored as ADF table — no Xray required. Epic confirmation gate fires unless confirmed=true. | | create_deduped_test_cases | Headless/webhook tool. Internally calls get_linked_test_cases, filters duplicates (normalised summary match), then calls create_and_link_test_cases on net-new only. Safe on every re-trigger. | | attach_gherkin_to_issue | Upload a .feature file as an attachment to a Jira issue via the Jira attachment API. File is named {issue_key}_{summary_kebab}_regression.feature. |

Three-Layer Self-Healing Architecture

| Layer | Class | What It Heals | |---|---|---| | 1 — Locator | LocatorHealer | cached-healed → primary → attribute (intent-shape-aware) → type-hint → role → label → text → AI Vision | | 2 — Timing | TimingHealer | network-trace drift → auto-adjusted timeouts → HealingDashboard | | 3 — Visual | VisualIntentChecker | element screenshot diff at assertions → HealingDashboard |

Healed selectors persist in LocatorRepository. All AI suggestions require human approval at http://localhost:7890 before entering version control. The attribute strategy constrains its substring matches by intent shape ("password input" → :is(input, textarea, select, [contenteditable])) so it can't pick up unrelated elements such as <a id="forgot-password"> when the intent calls for an input.

Webhook / Auto-Trigger Setup

Local dev

# 1. Install webhook extras
pip install "qa-gentic-stlc-agents[webhook]"

# 2. Start the bridge
make serve

# 3. Expose via ngrok
ngrok http 8080

# 4. Register hooks
make register-ado-hooks  BRIDGE_URL=https://xxxx.ngrok.io
make register-jira-hooks BRIDGE_URL=https://xxxx.ngrok.io

Azure Functions deploy

make deploy-azure FUNC_APP=stlc-webhook-bridge
make register-ado-hooks  BRIDGE_URL=https://stlc-webhook-bridge.azurewebsites.net/api
make register-jira-hooks BRIDGE_URL=https://stlc-webhook-bridge.azurewebsites.net/api

Docker

make docker-build-webhook
make docker-run-webhook

LLM provider selection

| Provider | Env var(s) | Default model | |---|---|---| | anthropic | AI_HEALING_API_KEY or ANTHROPIC_API_KEY | claude-haiku-4-5-20251001 | | copilot | GITHUB_TOKEN | gpt-4o | | openai | OPENAI_API_KEY | gpt-4o-mini | | azure-openai | AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT | gpt-4o-mini | | ollama | OLLAMA_HOST (optional) | llama3.2 |

Override model at any time: LLM_MODEL=claude-sonnet-4-6

Token cost (headless, Haiku default)

| Stage | Tokens / work item | Approx cost | |---|---|---| | Test case generation (state → Approved) | ~8,000–15,000 | $0.01–$0.03 | | Gherkin generation (state → Done) | ~10,000–18,000 | $0.01–$0.04 | | Playwright TS generation (state → Done) | ~20,000–35,000 | $0.03–$0.07 |

ADO vs Jira — Side by Side

| Step | Azure DevOps | Jira Cloud | |---|---|---| | Trigger | Service Hook: workitem.updated | Webhook: jira:issue_updated | | Fetch issue | fetch_work_item | fetch_jira_issue | | Check duplicates | get_linked_test_cases | get_linked_test_cases | | Create test cases (interactive) | create_and_link_test_cases | create_and_link_test_cases | | Create test cases (headless) | create_deduped_test_cases | create_deduped_test_cases | | Gherkin attach | attach_gherkin_to_feature / attach_gherkin_to_work_item | attach_gherkin_to_issue | | Headless Gherkin composite | generate_and_attach_gherkin | generate_and_attach_gherkin + attach_gherkin_to_issue | | Playwright generation | generate_playwright_code (shared) | generate_playwright_code (shared) | | Write to disk | write_helix_files (shared) | write_helix_files (shared) | | Link relation | TestedBy-Forward | is tested by / Test | | Auth | MSAL silent + browser (~/.msal-cache/) | OAuth 2.0 3LO + browser (~/.jira-cache/) |

Run Tests

# Full suite
ENABLE_SELF_HEALING=true \
HEALING_DASHBOARD_PORT=7890 \
APP_BASE_URL=<your-app-base-url> \
APP_EMAIL=<email> \
APP_PASSWORD=<password> \
cucumber-js --config=config/cucumber.js -p <feature_profile>

# Smoke only
cucumber-js --config=config/cucumber.js -p <feature_profile> --tags "@smoke"

Documentation

• docs/ARCHITECTURE-ADO.md — Full technical architecture, ADO pipeline
• docs/ARCHITECTURE-JIRA.md — Full technical architecture, Jira pipeline
• docs/QUICKSTART-ADO.md — Quick start for Azure DevOps
• docs/QUICKSTART-JIRA.md — Quick start for Jira Cloud
• docs/WALKTHROUGH-ADO.md — End-to-end walkthrough, ADO pipeline
• docs/WALKTHROUGH-JIRA.md — End-to-end walkthrough, Jira pipeline
• docs/migration-guide.md — Full stlc-migrate migration guide (preserve-tree, VRT, healer wiring)
• docs/qa-stlc-commands.html — Command reference for the CLI and setup flows

License

MIT