qualitylens

Test coverage map — automated + manual, by functional area. Know exactly what's tested, what isn't, and where the risk is — before every release.

The problem

Your CI is green. Your Playwright suite has 600 tests. But does /billing/refund have a test? Does /onboarding/import-data? When was /auth/sso last tested manually?

Nobody knows. Until production breaks.

qualitylens answers the question your coverage tool can't: "Which user-facing flows have no test coverage — automated or manual?"

Free vs Pro

| Feature | Free (qualitylens) | Pro (qualitylens + qualitylens-pro) | |---------|-----------------|--------------------------------| | Playwright test source | ✅ | ✅ | | YAML manual coverage | ✅ | ✅ | | Console reporter | ✅ | ✅ | | qualitylens init / validate | ✅ | ✅ | | CI --fail-under gate | ✅ | ✅ | | HTML reporter | — | ✅ | | JSON reporter | — | ✅ | | Markdown reporter | — | ✅ | | Azure DevOps Test Plans | — | ✅ |

Both tiers use the single qualitylens binary. Pro features activate automatically when qualitylens-pro is installed alongside.

Installation

Free:

npm install -g @democratize-quality/qualitylens

Pro:

npm install -g @democratize-quality/qualitylens @democratize-quality/qualitylens-pro

No config change needed — the CLI detects qualitylens-pro at runtime.

Quick start

npx @democratize-quality/qualitylens init   # creates qualitylens.yaml in your project root
npx @democratize-quality/qualitylens scan   # generates console report (+ HTML/JSON/MD with Pro)

Sample output:

qualitylens: 68% total coverage (41% auto, 27% manual, 32% none)

Authentication         ████████████████████ 92% ✅
Checkout & Payments    ████████████░░░░░░░░ 54% ⚠️
Onboarding             ████████████░░░░░░░░ 60% ⚠️
Admin & Settings       █████░░░░░░░░░░░░░░░ 28% ❌

No coverage found (8 routes):
  ❌ /checkout/promo-code
  ❌ /billing/refund
  ❌ /billing/invoice-download
  ❌ /admin/billing-settings
  ❌ /admin/permissions
  ❌ /admin/audit-log
  ❌ /onboarding/import-data
  ❌ /auth/mfa-setup

Stale manual coverage (>30 days, 2 routes):
  ⚠️  /onboarding/welcome     — last tested 34d ago (priya)
  ⚠️  /onboarding/profile-setup — last tested 34d ago (priya)

How it works

qualitylens combines test sources against your app routes:

| Source | What it reads | How | Tier | |--------|--------------|-----|------| | Playwright | *.spec.ts test titles | AST parsing | Free | | YAML | manualCoverage entries | YAML file | Free | | Azure DevOps | Test Plan test case names | REST API | Pro |

It fuzzy-matches test descriptions against your app's routes, groups them by functional area, and reports what's covered, what's manual-only, what's stale, and what has no coverage at all.

See docs/HOW_IT_WORKS.md for full details.

Configuration

Place a qualitylens.yaml in your project root:

projectName: my-saas-app
staleThresholdDays: 30

routes:
  type: nextjs
  path: ./pages

areas:
  - name: Checkout & Payments
    patterns: [/checkout, /billing]
  - name: Authentication
    patterns: [/auth, /login]

# Optional: Azure DevOps Test Plans
# ado:
#   orgUrl: https://dev.azure.com/myorg
#   project: MyProject
#   planId: 42
#   patEnvVar: ADO_PAT

# Manual coverage (from Excel, exploratory testing, etc.)
manualCoverage:
  - route: /auth/sso
    lastTested: 2025-03-01
    tester: james

# CI gates
thresholds:
  - area: Checkout & Payments
    minCoverage: 70

Full reference: docs/CONFIGURATION.md

CLI reference

qualitylens scan [options]
  --config <path>       path to qualitylens.yaml (default: ./qualitylens.yaml)
  --output <path>       output directory (default: ./)
  --format <formats>    console (free) | html,json,markdown (Pro) — comma-separated
  --since <branch>      only show routes changed since this git branch
  --fail-under <n>      exit 1 if total coverage < n (CI gate)

qualitylens init            interactive setup — creates qualitylens.yaml
qualitylens validate        checks config and ADO connection

Pro formats — if you run --format html without qualitylens-pro installed, qualitylens prints an install hint and continues. Nothing crashes.

CI integration

Free:

- name: Run qualitylens
  run: npx @democratize-quality/qualitylens scan --format console --fail-under 60

Pro (HTML + JSON report, ADO source):

- name: Run qualitylens Pro
  run: npx @democratize-quality/qualitylens scan --format html,json,console --fail-under 60
  env:
    ADO_PAT: ${{ secrets.ADO_PAT }}

Full examples: docs/CI_INTEGRATION.md

VS Code extension (Pro)

The qualitylens Pro VS Code extension shows a live coverage dashboard in your sidebar. The panel updates automatically when qualitylens.yaml changes. Requires qualitylens-pro.

Built by

Raj Uppadhyay — author of Scalable Test Automation with Playwright

Related tools:

• playwright-mcp-yaml
• ADO Testing Extension