Autotests Overkill

Autotests Overkill is a standalone AI-assisted testing platform for autotests2, caseplatform-web, optional fis_platform backend context, and CI artifacts. It is designed for Codex-style work: inspect existing coverage, extract frontend/backend contracts and runtime logic, generate draft specs, triage failures, and narrow rerun scope without silently mutating consumer repositories.

Runtime requirement:

• Node.js 22.5+ in both Windows and WSL environments
• No native database addon is required; the knowledge store runs on the built-in node:sqlite module so the same package install works from either environment

What v1.1 adds

• Legacy widget contract extraction for files such as dateRangeInput_control.js and dateRangeInput_singleton.js
• Stronger spec blueprints that reuse real autotests2 setup, commands, helpers, and selector patterns
• Acceptance coverage for both modern and legacy widgets
• Knowledge-store-backed retrieval for asset search, area context, impact analysis, and blueprint support
• Optional fis_platform backend knowledge through an evidence-driven ApiDiscoveryCatalog, api-interaction documents, and backend-endpoint contracts
• Optional ReportPortal and GitLab adapters with graceful artifacts-first fallback
• Codex MCP setup for npm package usage and local checkout development

Repository layout

• src/ implementation code
• mcp/ MCP entrypoint notes
• skills/ Codex skills for end-to-end workflows
• indexer/ indexing entrypoint notes
• connectors/ connector notes
• templates/ prompt and blueprint templates
• docs/ operator and integration documentation
• tests/ unit, integration, and acceptance checks

Install

Published npm package install:

1. Run npm install -D @mknrt/autotests-overkill.
2. Copy node_modules/@mknrt/autotests-overkill/overkill.config.example.json to overkill.config.json.
3. Set repos.autotests2, repos.caseplatformWeb, optional repos.fisPlatform, and ci.artifactsRoot.
4. Run npx autotests-overkill index.

Repository-local development:

1. Run npm install.
2. Copy overkill.config.example.json to overkill.config.json.
3. Set repos.autotests2, repos.caseplatformWeb, optional repos.fisPlatform, and ci.artifactsRoot.
4. Run npm run build.
5. Run npm run index.

Daily workflow

1. Refresh local knowledge with npm run index.
2. Start the MCP server through the consumer repository's .codex/config.toml, or run npx autotests-overkill mcp directly for manual checks.
3. Use find_frontend_contract, find_existing_test_assets, and generate_spec_blueprint for a new feature draft.
4. Use review_test_draft to critique a draft for reuse, selector stability, setup drift, and likely gap coverage before editing autotests2.
5. Use find_backend_test_context when a request flow needs backend validation, params, async completion, or contract-drift evidence.
6. Use summarize_ci_context, triage_failed_run, and analyze_diff_impact for failure or change triage.
7. Review generated output manually before editing autotests2.

CLI commands:

• npx autotests-overkill index
• npx autotests-overkill mcp
• npx autotests-overkill acceptance coverage caption
• npx autotests-overkill acceptance blueprint dateRangeField
• npx autotests-overkill acceptance review caption

Verification commands

• npm run verify
• npm run mcp:smoke
• npm run acceptance:coverage
• npm run acceptance:coverage:legacy
• npm run acceptance:blueprint
• npm run acceptance:review
• npm run acceptance:failure
• npm run acceptance:impact

Consumer config

The platform reads overkill.config.json with these keys:

• repos.autotests2
• repos.caseplatformWeb
• repos.fisPlatform optional path to ../fis_platform
• ci.artifactsRoot
• ci.reportPortal.endpoint
• ci.reportPortal.project
• ci.reportPortal.token
• ci.gitlab.baseUrl
• ci.gitlab.projectId
• ci.gitlab.token
• cacheDir

The default path is local and artifacts-first. Missing ReportPortal or GitLab credentials do not break tools; the platform keeps using local stats, screenshots, videos, and captured API payloads.

Troubleshooting

• If MCP or indexing fails with ERR_UNKNOWN_BUILTIN_MODULE for node:sqlite, your Node.js version is too old. Upgrade to Node.js 22.5 or newer in that environment.
• If startup fails with database is locked, another process is already using the same cache database. Stop the competing process or configure a different cacheDir.

Backend integration

Backend indexing is discovery-first and evidence-driven, not hardcoded. The first bridge is an ApiDiscoveryCatalog built from autotests2 support code such as request wrappers, endpoint constants, cy.request, fetch, XMLHttpRequest, and API capture helpers. Selected high-signal families become api-interaction documents with normalizedPaths, consumerSignalIds, evidencePaths, confidence, and whySelected.

When repos.fisPlatform points to ../fis_platform, refresh passes that catalog explicitly into backend discovery. fis_platform sources are selected by REST/JAX-RS signals such as @Path, @GET, @POST, @PUT, @DELETE, @QueryParam, @HeaderParam, @Consumes, and @Produces; noisy directories such as thirdparty, extjs, oryx, generated, node_modules, and vendor are excluded by rule. Matching sources become backend-endpoint documents with endpoint paths, params, rule hints, evidence paths, confidence, and whyMatched.

Only high-confidence links (confidence >= 0.7) drive blueprint, triage, and impact recommendations. Lower-confidence backend matches may be surfaced as exploratory evidence, but they are not treated as recommendations. Runtime API captures are read on demand by triage_failed_run in iteration 1 and are not persisted as raw knowledge-store documents.

MCP tools

• find_existing_test_assets
• review_test_draft
• find_backend_test_context
• find_test_area_context
• find_frontend_contract
• find_frontend_runtime_logic
• map_feature_to_existing_coverage
• analyze_test_gaps
• generate_spec_blueprint
• recommend_selector_strategy
• build_setup_teardown_plan
• analyze_diff_impact
• triage_failed_run
• build_rerun_scope
• summarize_ci_context

Every tool returns the same normalized shape:

• summary
• evidence
• recommended_actions
• repo_paths
• confidence
• warnings

Codex MCP setup

This repository can be consumed either from a Git checkout or as a published npm package:

• .mcp.json registers a local checkout MCP command only
• skills/ is auto-discovered as the bundled skill set

For npm package consumers such as autotests2, prefer a project-local Codex config:

[mcp_servers.autotests-overkill]
command = "node"
args = ["./node_modules/@mknrt/autotests-overkill/dist/src/cli/runMcpServer.js"]
enabled = true
startup_timeout_sec = 20
tool_timeout_sec = 120

See consumer-integration.md, operator-cookbook.md, and plugin-packaging.md for setup details.