test-confidence

CLI tool that evaluates test quality for changed files in a PR and posts results as a GitHub comment.

How it works

1. Diffs changed TypeScript/TSX files against a base branch
2. Maps each component to its test files
3. Runs rules against the test ASTs
4. Produces a confidence score (high / medium / low) per component

Install

npm install -g test-confidence
# or
pnpm add -g test-confidence

Usage

# compare against origin/main (default)
test-confidence

# compare against a different base
test-confidence --diff origin/develop

# output JSON
test-confidence --json

# post result as a PR comment
GITHUB_TOKEN=<token> GITHUB_REPOSITORY=owner/repo test-confidence --pr 42

Exit code is 1 if any component scores low confidence, 0 otherwise — useful in CI.

Rules

| Rule | Severity | Impact | |------|----------|--------| | no-tests | high | −40 | | no-interaction | high | −25 | | get-by-test-id | medium | up to −30 | | weak-assertions | medium | −15 |

Scores start at 100 and impacts are cumulative. Thresholds: ≥80 → high, ≥50 → medium, <50 → low.

CI example

- name: Test confidence check
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    GITHUB_REPOSITORY: ${{ github.repository }}
  run: test-confidence --pr ${{ github.event.pull_request.number }}

Contributing

See CONTRIBUTING.md.

License

MIT