Flakiness.io CucumberJS Formatter

A custom CucumberJS formatter that generates Flakiness Reports from your Cucumber test runs. The formatter automatically converts CucumberJS test results into the standardized Flakiness JSON format, preserving complete Gherkin structure, test outcomes, and environment information.

Supported Gherkin Features

• Scenarios & Scenario Outlines (with multiple Examples blocks)
• Rules
• Tags & tag inheritance (Feature → Rule → Scenario → Examples)
• Steps (Given/When/Then with keyword prefix)
• Data Tables
• Before & After Hooks (named and unnamed)
• Feature, Rule, and Scenario descriptions
• Attachments (this.attach() and this.log())
• Retries (--retry)
• Parallel execution (--parallel)
• All statuses: passed, failed, pending, undefined, ambiguous, skipped

Table of Contents

• Requirements
• Installation
• Quick Start
• Uploading Reports
• Viewing Reports
• Features
  • Environment Detection
  • CI Integration
• Configuration Options
  • flakinessProject?: string
  • title?: string
  • endpoint?: string
  • token?: string
  • outputFolder?: string
  • disableUpload?: boolean
• Example Configuration

Requirements

• @cucumber/cucumber 12.0 or higher
• Node.js project with a git repository (for commit information)

Installation

npm install -D @flakiness/cucumberjs

Quick Start

Add the formatter to your cucumber.mjs:

export default {
  paths: ['features/**/*.feature'],
  import: ['features/support/**/*.ts'],
  format: [
    'progress',
    ['@flakiness/cucumberjs', '.flakiness/cucumber-formatter.log'],
  ],
  formatOptions: {
    flakinessProject: 'my-org/my-project',
  },
};

Run your tests. The report will be automatically generated in the ./flakiness-report folder:

npx cucumber-js

View the interactive report:

npx flakiness show ./flakiness-report

Uploading Reports

Reports are automatically uploaded to Flakiness.io after test completion. Authentication can be done in two ways:

• Access token: Provide a token via the token format option or the FLAKINESS_ACCESS_TOKEN environment variable.
• GitHub OIDC: When running in GitHub Actions, the formatter can authenticate using GitHub's OIDC token — no access token needed. See GitHub Actions integration for setup instructions.

If upload fails, the report is still available locally in the output folder.

If you combine this formatter with a progress formatter, keep the progress formatter on stdout and redirect @flakiness/cucumberjs to a file as shown above. Upload status messages are printed to stderr, so the uploaded report URL stays visible in the terminal.

Viewing Reports

After test execution, you can view the report using:

npx flakiness show ./flakiness-report

Features

Environment Detection

Environment variables prefixed with FK_ENV_ are automatically included in the environment metadata. The prefix is stripped and the key is converted to lowercase.

Example:

export FK_ENV_DEPLOYMENT=staging
export FK_ENV_REGION=us-east-1

This will result in the environment containing:

{
  "metadata": {
    "deployment": "staging",
    "region": "us-east-1"
  }
}

Flakiness.io will create a dedicated history for tests executed in each unique environment. This means tests run with FK_ENV_DEPLOYMENT=staging will have a separate timeline from tests run with FK_ENV_DEPLOYMENT=production, allowing you to track flakiness patterns specific to each deployment environment.

CI Integration

The formatter automatically detects CI environments and includes:

• CI run URLs (GitHub Actions, Azure DevOps, Jenkins, GitLab CI)
• Git commit information
• System environment data

Configuration Options

All options are passed via CucumberJS's formatOptions in your configuration file.

flakinessProject?: string

The Flakiness.io project identifier in org/project format. Used for GitHub OIDC authentication — when set, and the Flakiness.io project is bound to the GitHub repository running the workflow, the formatter authenticates uploads via GitHub Actions OIDC token with no access token required.

formatOptions: {
  flakinessProject: 'my-org/my-project',
}

title?: string

Optional human-readable report title. Typically used to name a CI run, matrix shard, or other execution group.

Defaults to the FLAKINESS_TITLE environment variable, or empty otherwise.

formatOptions: {
  title: 'Shard 1/4 — Linux Chrome',
}

endpoint?: string

Custom Flakiness.io endpoint URL for uploading reports. Defaults to the FLAKINESS_ENDPOINT environment variable, or https://flakiness.io if not set.

Use this option to point to a custom or self-hosted Flakiness.io instance.

formatOptions: {
  endpoint: 'https://custom.flakiness.io',
}

token?: string

Access token for authenticating with Flakiness.io when uploading reports. Defaults to the FLAKINESS_ACCESS_TOKEN environment variable.

If no token is provided, the formatter will attempt to authenticate using GitHub OIDC.

formatOptions: {
  token: 'your-access-token',
}

outputFolder?: string

Directory path where the Flakiness report will be written. Defaults to flakiness-report in the current working directory, or the FLAKINESS_OUTPUT_DIR environment variable if set.

formatOptions: {
  outputFolder: './test-results/flakiness',
}

disableUpload?: boolean

When set to true, prevents uploading the report to Flakiness.io. The report is still generated locally. Can also be controlled via the FLAKINESS_DISABLE_UPLOAD environment variable.

formatOptions: {
  disableUpload: true,
}

Example Configuration

Here's a complete example with all options:

export default {
  paths: ['features/**/*.feature'],
  import: ['features/support/**/*.ts'],
  format: [
    'progress',
    ['@flakiness/cucumberjs', '.flakiness/cucumber-formatter.log'],
  ],
  formatOptions: {
    flakinessProject: 'my-org/my-project',
    title: 'My Test Run',
    endpoint: process.env.FLAKINESS_ENDPOINT,
    token: process.env.FLAKINESS_ACCESS_TOKEN,
    outputFolder: './flakiness-report',
    disableUpload: false,
  },
};