cypress-trust-gate

v1.1.7

Published

a day ago

A unified Cypress quality intelligence tool that detects flaky tests, accessibility violations, slow tests, and locator fragility – generating a beautiful HTML + JSON dashboard report.

0High
0Medium
0Low

saran.mv

cypress testing flaky-tests accessibility axe quality dashboard sdet qa cli locator performance

🛡️✨ Cypress Trust Gate ✨🛡️

Unified Cypress quality intelligence – detect flaky tests 🎲, accessibility violations ♿, slow tests 🚀, and brittle locators 🔍. Generate a beautiful HTML + JSON dashboard report.

Why cypress-trust-gate?

QA teams invest heavily in Cypress test suites, but test quality erodes silently:

| Problem | Symptom | Cost | |---------|---------|------| | 🎲 Flaky tests | CI passes then fails randomly | Engineers ignore failures | | ♿ A11y blindspots | Axe violations pile up unreported | Legal & UX risk | | 🐌 Slow tests | Suite takes 40 minutes to run | CI bottleneck | | 🔍 Brittle locators | Tests break on UI refactors | Maintenance burden |

cypress-trust-gate surfaces all four dimensions in a single, actionable dashboard.

⚙️ How It Works

cypress-trust-gate is a multi-layered analysis engine that combines dynamic test execution with static code analysis:

🔄 Test Orchestration (Dynamic): It programmatically runs your Cypress suite multiple times. By parsing the generated JUnit XML reports, it tracks execution paths and uses Shannon Entropy to calculate the exact probability of a test being flaky.
🕵️‍♂️ Accessibility Auditing (Dynamic): It automatically injects axe-core into your application during the test run. It audits the DOM state and captures the exact HTML snippets and CSS selectors that violate WCAG guidelines.
📖 Static Code Analysis (Static): Without even running the browser, it reads your .cy.ts spec files. It uses regex and AST-style pattern matching to flag anti-patterns (e.g., hardcoded cy.wait(), deep CSS chains like :nth-of-type) before they cause issues.
🧠 Scoring Engine: It feeds all collected data into a weighted algorithm to generate a Unified Trust Score (0-100) and provides you with actionable remediation steps!

🌟 Key Benefits

🛑 Stop Merging Flaky Tests: Catch non-deterministic tests before they get into your main branch.
🧑‍🦯 Automated Accessibility: Ensure your application remains accessible without writing dedicated A11y tests.
⏱️ Faster CI Pipelines: Identify the top bottleneck tests and fix "anti-patterns" dragging down your execution time.
🔧 Zero-Maintenance Selectors: Prevent UI refactors from breaking tests by strictly enforcing data attributes (data-cy) over brittle CSS paths.
📊 Beautiful Reporting: Share a clean, score-based HTML dashboard with stakeholders, PMs, and developers.

💻 Compatibility

| Environment | Supported Versions | |-------------|--------------------| | Cypress | v10.0.0 – v15.x+ (Latest) | | Node.js | v16.x – v22.x+ (LTS) | | OS | Windows, macOS, Linux |

Core Features

⚡ Flaky Test Detection

Runs your suite N times and compares results
Shannon entropy scoring (0 = stable, 100 = chaos)
Retry-aware: flags tests that only pass after Cypress retries
Per-test risk level: low | medium | high | critical
Specific remediation recommendations

♿ Accessibility Violations

Integrates with axe-core via cypress-axe
Auto-scans after each test or on demand
Classifies violations: minor | moderate | serious | critical
Weighted scoring (critical violations penalise more)
Captures violation rule ID, description, help URL, and affected DOM nodes
✨ NEW: Interactive HTML dashboard element viewer – inspect the exact target selector and HTML snippet that caused the violation.

🚀 Slow Test & Anti-Pattern Detection

Flags tests exceeding your threshold (default: 8s)
Static analysis of spec files for:
- cy.wait(<number>) – hard-coded waits
- login() inside beforeEach – repeated auth overhead
- Excessive cy.visit() calls
Recommends cy.session(), cy.intercept(), and assertion-based waits
Shows top 10 slowest tests with p95 duration

🔍 Locator Fragility Analysis

Static scanner for brittle selector patterns in spec files
Detects:
- :nth-child / :nth-of-type selectors
- Deep CSS chains (3+ levels)
- Chained class selectors (.foo.bar.baz)
- Dynamic auto-generated IDs (#comp8432)
- Long cy.contains() text matches
Fragility levels: stable | warning | brittle
Suggests [data-cy="..."] / [data-testid="..."] replacements

Installation

# As a dev dependency in your Cypress project
npm install cypress-trust-gate --save-dev

# Install peer dependencies
npm install cypress-axe axe-core mocha-junit-reporter --save-dev

# Or install globally for the CLI
npm install -g cypress-trust-gate

🚀 Step-By-Step Integration Guide

Follow these 5 steps to get full quality intelligence for your Cypress suite.

1. Install Dependencies

Install the core package along with its required peer dependencies for reporting and accessibility scanning.

npm install cypress-trust-gate --save-dev
npm install cypress-axe axe-core mocha-junit-reporter --save-dev

2. Initialise trust-gate

Run the init command to scaffold the configuration and see example integration files.

npx cypress-trust-gate init

This creates:

trust-gate.config.ts: Central quality thresholds and scoring weights.
cypress/e2e/trust-gate-demo.cy.ts: A sample spec showing good and bad patterns.

3. Register the Plugin

In your cypress.config.ts, import the plugin and register it in setupNodeEvents. This enables the collection of accessibility and performance data.

import { defineConfig } from 'cypress';
import { trustGateCypressPlugin } from 'cypress-trust-gate/plugin';
import trustGateConfig from './trust-gate.config';

export default defineConfig({
  e2e: {
    // 1. Required: Use mocha-junit-reporter for results analysis
    reporter: 'mocha-junit-reporter',
    reporterOptions: {
      mochaFile: './trust-gate-report/results/run-0/results-[hash].xml',
    },
    // 2. Recommended: Enable retries for flakiness detection
    retries: { runMode: 2, openMode: 0 },

    setupNodeEvents(on, config) {
      // 3. Register the trust-gate plugin
      trustGateCypressPlugin(on, config, {
        outputDir: trustGateConfig.outputDir,
        accessibility: { enabled: trustGateConfig.accessibility.enabled },
      });
      return config;
    },
  },
});

4. Setup Support Hooks

Add the trust-gate commands to your global support file (cypress/support/e2e.ts) to automate data collection across all tests.

import 'cypress-trust-gate/commands';
import 'cypress-axe';

// Auto-inject axe-core before each test
beforeEach(() => {
  cy.injectAxeIfEnabled();
});

// Auto-scan for violations after each test
afterEach(function () {
  cy.runTrustGateScan(
    this.currentTest?.title ?? 'unknown',
    Cypress.spec.relative
  );
});

5. Execute & Analyze

Run your suite multiple times to detect flakiness and generate the final dashboard.

npx cypress-trust-gate run --runs 3 --open

Running Specific Tests

You can target a single file or a folder using the --spec flag:

# Single file
npx cypress-trust-gate run --spec "cypress/e2e/login.cy.ts" --runs 3 --open

```bash
# Specific folder
npx cypress-trust-gate run --spec "cypress/e2e/api/*.cy.ts" --runs 5

CLI Reference

`cypress-trust-gate init`

Scaffolds starter config and integration files.

cypress-trust-gate init [--dir <path>]

| Option | Description | Default | |--------|-------------|---------| | --dir | Target directory | . |

`cypress-trust-gate run`

Executes Cypress N times and generates the full quality report.

cypress-trust-gate run [options]

| Option | Description | Default | |--------|-------------|---------| | --runs <n> | Number of Cypress runs | 3 | | --spec <pattern> | Glob pattern for specs | cypress/e2e/**/*.cy.ts | | --config <path> | Path to config file | auto-discover | | --output <dir> | Report output directory | ./trust-gate-report | | --threshold <n> | Min acceptable overall score | 75 | | --open | Open HTML report after generation | false | | --fail-on-threshold | Exit 1 if score < threshold | false |

CI example:

cypress-trust-gate run \
  --runs 3 \
  --spec "cypress/e2e/**/*.cy.ts" \
  --threshold 80 \
  --fail-on-threshold

`cypress-trust-gate analyze`

Analyzes existing results without re-running Cypress.

cypress-trust-gate analyze [options]

| Option | Description | Default | |--------|-------------|---------| | --results <dir> | JUnit XML directory | ./trust-gate-report/results | | --spec <pattern> | Glob pattern for specs | cypress/e2e/**/*.cy.ts | | --output <dir> | Report output directory | ./trust-gate-report | | --open | Open report after generation | false |

`cypress-trust-gate report`

Regenerates the HTML dashboard from an existing JSON report.

cypress-trust-gate report [options]

| Option | Description | Default | |--------|-------------|---------| | --input <path> | JSON report path | ./trust-gate-report/trust-gate-report.json | | --output <dir> | HTML output directory | ./trust-gate-report | | --open | Open report in browser | false |

Cypress Plugin Integration

`cypress.config.ts`

import { defineConfig } from 'cypress';
import { trustGateCypressPlugin } from 'cypress-trust-gate';

export default defineConfig({
  e2e: {
    reporter: 'mocha-junit-reporter',
    reporterOptions: {
      mochaFile: './trust-gate-report/results/run-0/results-[hash].xml',
    },
    retries: { runMode: 2, openMode: 0 },
    setupNodeEvents(on, config) {
      trustGateCypressPlugin(on, config, {
        outputDir: './trust-gate-report',
        accessibility: { enabled: true },
      });
      return config;
    },
  },
});

`cypress/support/e2e.ts`

import 'cypress-trust-gate/commands';
import 'cypress-axe';

beforeEach(() => {
  cy.injectAxeIfEnabled();
});

afterEach(function () {
  cy.runTrustGateScan(this.currentTest?.title ?? '', Cypress.spec.relative);
});

Available Custom Commands

| Command | Description | |---------|-------------| | cy.injectAxeIfEnabled() | Injects axe-core if TRUST_GATE_A11Y=true | | cy.runTrustGateScan(title, specFile) | Runs axe scan and logs violations | | cy.trustGateLogA11y(violations) | Directly posts violations to the plugin task |

Configuration

Create trust-gate.config.ts in your project root:

import type { TrustGateConfig } from 'cypress-trust-gate';

const config: TrustGateConfig = {
  projectName: 'My App',
  runs: 3,
  outputDir: './trust-gate-report',

  accessibility: {
    enabled: true,
    threshold: 80,
    scanMode: 'afterEach', // 'afterEach' | 'onDemand'
    runOnly: ['wcag2a', 'wcag2aa'],
  },

  flakiness: {
    enabled: true,
    retriesAware: true,       // Count retry-passes as flaky events
    flakinessThreshold: 20,   // Entropy score threshold
  },

  performance: {
    enabled: true,
    slowTestThresholdMs: 8000,
    detectHardWaits: true,
  },

  locators: {
    enabled: true,
    preferredAttributes: ['data-cy', 'data-testid'],
  },

  scoring: {
    weights: {
      flakiness: 0.35,
      accessibility: 0.25,
      performance: 0.20,
      locatorFragility: 0.20,
    },
  },
};

export default config;

Trust Scores Explained

Every dimension produces a score from 0 to 100 (higher = better).

| Score | Trust Level | Meaning | |-------|-------------|---------| | 90–100 | 🟢 Excellent | Suite is highly trustworthy | | 75–89 | 🟡 Good | Minor issues to address | | 55–74 | 🟠 Fair | Noticeable quality debt | | 35–54 | 🔴 Poor | Significant problems | | 0–34 | ⛔ Critical | Suite cannot be trusted |

Flakiness Score

Derived from Shannon entropy averaged across all tests.

entropy = 0 → perfectly deterministic (green)
entropy = 100 → 50/50 pass/fail (red)

Accessibility Score

Penalises violations by impact weight:

| Impact | Weight | |--------|--------| | critical | 15 | | serious | 7 | | moderate | 3 | | minor | 1 |

Score uses exponential decay: 100 × e^(-penalty/150)

Performance Score

Starts at 100, deducted 8 points per slow test (max -80) and 3 points per anti-pattern (max -20).

Locator Fragility Score

Starts at 100, deducted 10 points per brittle locator (max -70) and 3 points per warning (max -30).

Report Dashboard

The generated index.html dashboard includes:

Score gauges – SVG ring gauges for each dimension and overall
Trust badge – colour-coded trust level label
Recommendations panel – prioritised fixes for your team
Chart.js charts – bar charts for entropy, performance, and score overview
Flaky test table – per-test entropy scores and recommendations
Accessibility table – violations with links to Deque documentation
Slow test table – top 10 slowest tests with duration
Anti-pattern table – detected bad practices with fix suggestions
Locator table – brittle/warning selectors with suggested replacements

🛠️ Troubleshooting & FAQs

1. "No JUnit results found in..." or "No tests detected"

Why it happens: Your test suite ran (or attempted to run), but the mocha-junit-reporter didn't generate .xml files where the tool expected them. How to fix:

Ensure you have installed the reporter: npm install mocha-junit-reporter --save-dev

Double-check your cypress.config.ts/js. You must include:

reporter: 'mocha-junit-reporter',
reporterOptions: {
  mochaFile: './trust-gate-report/results/run-0/results-[hash].xml'
}

Make sure Cypress isn't crashing before the tests run (e.g., due to an unreachable baseUrl or a syntax error).

2. "Cannot find module './trust-gate.config'"

Why it happens: You added the plugin to your cypress.config.js but you haven't generated the configuration file yet. How to fix:

Run the initialization command in your project root:
```
npx cypress-trust-gate init
```

3. NPM Peer Dependency Conflict (ERESOLVE)

Why it happens: Another Cypress plugin in your project (e.g. cypress-drag-drop) is enforcing a strict older Cypress version, causing a conflict when installing the reporter. How to fix:

Install using the --legacy-peer-deps flag:

npm install cypress-trust-gate mocha-junit-reporter --save-dev --legacy-peer-deps

4. Tests fail immediately due to `baseUrl`

Why it happens: Cypress pings baseUrl before running tests. If your app isn't running locally, Cypress crashes immediately. How to fix:

Either spin up your local environment (e.g., npm run start) in a separate terminal before running npx cypress-trust-gate run.
OR Temporarily remove the baseUrl from your Cypress config if you are testing external environments.

Programmatic API

import {
  loadConfig,
  analyzeFlakiness,
  analyzeAccessibility,
  analyzePerformance,
  analyzeLocators,
  buildReport,
  writeHtmlReport,
  writeJsonReport,
  runAnalysis,
} from 'cypress-trust-gate';

// Load config with defaults merged
const config = loadConfig('./trust-gate.config.ts');

// Or use the full orchestration pipeline
const report = await runAnalysis({
  config,
  specPattern: 'cypress/e2e/**/*.cy.ts',
  resultsDir: './results',
});

console.log('Overall score:', report.scores.overall);
console.log('Trust level:', report.trustLevel);

Project Structure

cypress-trust-gate/
├── src/
│   ├── types/          # Centralised TypeScript interfaces
│   ├── config/         # Config loader and defaults
│   ├── analyzers/
│   │   ├── flakiness.ts       # Shannon entropy scorer
│   │   ├── accessibility.ts   # axe violation aggregator
│   │   ├── performance.ts     # slow test + anti-pattern detector
│   │   └── locators.ts        # static selector fragility scanner
│   ├── parsers/
│   │   └── junitParser.ts     # JUnit XML → TestRunResult
│   ├── report/
│   │   ├── builder.ts         # TrustGateReport assembler
│   │   └── htmlGenerator.ts   # HTML dashboard generator
│   ├── orchestrator/   # Full analysis pipeline
│   ├── plugin/
│   │   ├── index.ts    # Cypress Node plugin
│   │   └── commands.ts # Browser-side custom commands
│   ├── runner/         # Cypress execution wrapper
│   ├── cli/
│   │   └── commands/   # init | run | analyze | report
│   └── index.ts        # Public API barrel export
├── bin/
│   └── cli.js          # CLI entry point
├── tests/unit/         # Vitest tests for all analyzers
├── example/            # Example Cypress integration
├── sample/             # Sample report JSON
├── trust-gate.config.ts
├── package.json
├── tsconfig.json
└── README.md

Contributing

Contributions are welcome! Please open an issue first for significant changes.

git clone https://github.com/mvsaran/Cypress-Trust-Gate.git
cd cypress-trust-gate
npm install
npm run build
npm test

👨‍💻 Author

Saran Kumar

GitHub: @mvsarankumar
LinkedIn: (www.linkedin.com/in/saran-kumar-a69775215)