@app-studio/qa-doctor

Proves that a repository's qa.bindings.json describes the repository it is actually in.

Why this exists

A mature QA setup was copied from one repository into another without its bindings being rewritten. In the new repository the guard hook's patterns matched no file, so it denied nothing while looking exactly like a guard with nothing to complain about. The documented ports were wrong, so every instruction built on them sent people to a door that was not there. The seeded accounts did not exist, so agents signing in with them reported an authentication bug. And the documentation that was accurate was referenced by no entrypoint, so nobody read it.

None of that failed loudly. All of it was greppable. This package is that observation turned into a check.

Two ways to run it, one implementation

On demand:

pnpm exec qa doctor            # human report
pnpm exec qa doctor --json     # for agents and CI
pnpm exec qa doctor --strict   # warnings become errors

Inside the repository's own suite, which is what makes it un-skippable:

// tests/qa-doctor.test.ts
import { doctorSuite } from '@app-studio/qa-doctor/vitest';

doctorSuite();

One it() per rule. qa doctor catches a wrong binding when somebody runs it; this catches it on every CI run, in the suite that already gates merges — so skipping the check means skipping the suite, which is a decision somebody has to make out loud. A self/* rule verifies this file exists and falls inside the globs the test command selects, because a test nothing runs is worse than no test: the repository looks checked.

What it checks

| Group | Proves | | --- | --- | | bindings/repo-name-matches | repo.name equals package.json's name — the copied-file detector, run before anything else can be misled. | | commands/scripts-resolve | Every declared command names a real script or an installed binary, and the files it reads exist. | | commands/layers-reference-commands | Layers point at commands the registry defines, and layer keys are unique. | | commands/gates-reference-commands | The gate matrix asks for evidence this repository can produce. | | commands/one-package-manager | Nothing tells anyone to run npm in a pnpm repository. | | commands/smoke (CLI only) | The cheap commands — typecheck, lint, format check — actually run. | | paths/exist | Every declared path is there, unless listed under deliberatelyAbsent. | | paths/guard-globs-live | Every non-empty guard pattern matches at least one real file. The inert-guard detector. | | paths/generated-exist | The trees declared generated are really there. | | ports/bound-by-agrees | Each port is the number the file that binds it actually uses, and no two surfaces claim one port. | | env/required-declared | Required variables are written down where a newcomer will find them. | | env/secrets-not-committed | No committed file carries a real value for a variable marked secret. | | seed/accounts-in-source | Every promised account appears in the seed sources. | | hooks/registered | Each enabled hook is wired under the right event, with a matcher that covers what it must. | | hooks/wrappers-resolve | The wrappers exist, are executable, and point at logic that is installed. | | hooks/deny-protects-generated | Generated trees cannot be hand-edited. | | self/doctor-test-wired, self/hooks-test-wired | The checking runs inside this repository's suite. | | docs/entrypoints-point-at-qa | The files an agent reads on arriving say how QA works here. |

Rule ids are stable: the CLI groups by them, tests assert on them, and hosts grep for them.

The report

Each finding carries three things, not one:

✖ ports/bound-by-agrees — each declared port is the number the file that binds it actually uses
   ports.front says 5173, but playwright.config.ts never mentions that number.
   in: playwright.config.ts
   what this means: Every instruction built on this port sends people to a door
     that is not there, and the resulting connection refused reads as a broken
     application rather than a wrong binding.
   fix: Open playwright.config.ts, read the port it binds, and put that number
     in ports.front.

A doctor that prints rule names and exit codes gets ignored within a week, and an ignored doctor is indistinguishable from no doctor at all.

Writing your own rules

import { runRules, createDoctorContext } from '@app-studio/qa-doctor';
import type { Rule } from '@app-studio/qa-core';

const noOrphanedFeatures: Rule = {
  id: 'features/registered',
  describes: 'every feature directory is in the composed registry',
  appliesTo: (context) => context.exists('src/features'),
  check(context) {
    const declared = context.read('src/generated/registry.ts') ?? '';
    return context
      .glob('src/features/*/manifest.json')
      .filter((path) => !declared.includes(path.split('/')[2]!))
      .map((path) => ({
        level: 'error' as const,
        rule: 'features/registered',
        message: `${path} is not in the composed registry.`,
        means: 'The feature exists on disk and is not part of the application, which is invisible until someone looks for it.',
        fix: 'Run the composer, or delete the directory.',
      }));
  },
};

Rules are pure functions of a DoctorContext, so they test against fixture repositories rather than against your real one — which matters more than usual here, because a doctor nobody has watched go red is exactly the inert machinery this package exists to prevent.

Requirements

Node.js 20 or newer. vitest is an optional peer, needed only for the /vitest entry point.