Cypress Azure Sync

A Cypress plugin to sync tests with Azure Test Plans.

Overview

This plugin automates the process of creating test cases in Azure Test Plans based on Cypress test specifications. It:

1. Extracts test titles, tags, and metadata (Test Plan ID, Test Suite ID, Area Path) from Cypress spec files
2. Identifies tests that already have Azure Test Plan IDs in their titles (format: "123456: should login successfully")
3. Creates test cases in Azure Test Plans via the REST API for tests that don't already have IDs
4. Sets the automation state field (configurable, default is 'Custom.AutomationState') to "Automated" for each test case
5. Updates spec files with test case IDs after creation, adding the ID to the test title
6. Generates a mapping document between test titles and Azure Test Plan IDs

Prerequisites

• Node.js (v14 or higher)
• Cypress (v10 or higher)
• Cypress tests with tags
• Azure DevOps account with access to Test Plans
• Personal Access Token (PAT) with appropriate permissions

Installation

npm install cypress-azure-sync --save-dev

Configuration

There are three ways to configure the plugin:

1. Configuration File (Recommended)

Create a configuration file in your project root:

// cypress-azure-sync.config.js
module.exports = {
  azure: {
    // Azure DevOps organization name
    organization: process.env.AZURE_DEVOPS_ORG || 'your-organization',

    // Azure DevOps project name
    project: process.env.AZURE_DEVOPS_PROJECT || 'your-project',

    // Personal access token with appropriate permissions
    token: process.env.AZURE_DEVOPS_TOKEN || 'your-token',

    // API version
    apiVersion: process.env.AZURE_DEVOPS_API_VERSION || '7.0',

    // Default Test Plan ID (optional if specified in spec files)
    testPlanId: process.env.AZURE_TEST_PLAN_ID || '12345',

    // Default Test Suite ID (optional if specified in spec files)
    testSuiteId: process.env.AZURE_TEST_SUITE_ID || '67890',

    // Field name for automation state (default is 'Custom.AutomationState')
    automationStateField: process.env.AZURE_AUTOMATION_STATE_FIELD || 'Custom.AutomationState'
  },

  cypress: {
    // Glob pattern to find spec files
    specPattern: process.env.CYPRESS_SPEC_PATTERN || 'cypress/e2e/**/*.{js,jsx,ts,tsx}'
  },

  output: {
    // Path to output mapping files
    mappingPath: 'test-mapping'
  }
};

You can generate this file automatically using the plugin's task:

// In your cypress.config.js or cypress.config.ts
cy.task('createAzureSyncConfig');

2. Environment Variables

You can use environment variables to configure the plugin:

# Azure DevOps Configuration (Required)
AZURE_DEVOPS_ORG=your-organization
AZURE_DEVOPS_PROJECT=your-project
AZURE_DEVOPS_TOKEN=your-personal-access-token
AZURE_DEVOPS_API_VERSION=7.0

# Test Plan Configuration (Optional if specified in spec files)
AZURE_TEST_PLAN_ID=your-test-plan-id
AZURE_TEST_SUITE_ID=your-test-suite-id
AZURE_AUTOMATION_STATE_FIELD=Custom.AutomationState

# Cypress Configuration (Required)
CYPRESS_SPEC_PATTERN=cypress/e2e/**/*.{js,jsx,ts,tsx}

3. Programmatic Configuration

You can pass configuration directly to the plugin:

// In your cypress.config.js or cypress.config.ts
const { defineConfig } = require('cypress');
const { initAzureSync } = require('cypress-azure-sync');

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      return initAzureSync({
        azure: {
          organization: 'your-organization',
          project: 'your-project',
          token: 'your-token',
          apiVersion: '7.0',
          testPlanId: '12345',
          testSuiteId: '67890',
          automationStateField: 'Custom.AutomationState'
        },
        cypress: {
          specPattern: 'cypress/e2e/**/*.{js,jsx,ts,tsx}'
        },
        output: {
          mappingPath: 'test-mapping'
        }
      })(on, config);
    },
  },
});

Usage

JavaScript

// cypress.config.js
const { defineConfig } = require('cypress');
const { initAzureSync } = require('cypress-azure-sync');

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      return initAzureSync()(on, config);
    },
  },
});

TypeScript

// cypress.config.ts
import { defineConfig } from 'cypress';
import { initAzureSync } from 'cypress-azure-sync';

export default defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      return initAzureSync()(on, config);
    },
  },
});

Running the Sync

Using Cypress Task

You can trigger the sync in your Cypress tests:

// In your Cypress test or support file
describe('Azure Sync', () => {
  it('syncs tests with Azure Test Plans', () => {
    cy.task('syncWithAzure');
  });
});

Or with options:

cy.task('syncWithAzure', {
  specPattern: 'cypress/e2e/specific-folder/**/*.spec.js'
});

Using NPX Command (CLI)

You can also run the sync directly from the command line using npx:

npx cypress-azure-sync

This will use the configuration from your cypress-azure-sync.config.js file or environment variables.

You can specify a custom configuration file:

npx cypress-azure-sync sync --config=./path/to/config.js

To create a sample configuration file:

npx cypress-azure-sync init

Or specify a custom output path for the configuration file:

npx cypress-azure-sync init --output=./path/to/config.js

For help and available commands:

npx cypress-azure-sync help

Cypress Test Format

For the plugin to work correctly, your Cypress tests should include tags in one of the following formats:

Spec File Metadata Comments (Optional)

Each spec file can include comments at the beginning of the file to specify the Test Plan ID, Test Suite ID, and Area Path for the tests in that file. You can use either regular comments or JSDoc comments:

Regular Comments

// TestPlanId = 170880
// TestSuiteId = 170921
// AreaPath = 'Your Project Name\Application Name\Feature'

JSDoc Comments

/**
 * TestPlanId = 170880
 * TestSuiteId = 170921
 * AreaPath = 'Your Project Name\Application Name\Feature'
 */

These values will be used when creating test cases in Azure Test Plans. If these comments are not present in a spec file, the plugin will fall back to using the values from the configuration.

Test Case ID Format (Optional)

If a test already has an Azure Test Plan ID, you can include it in the test title using the format <ID>: <title>:

describe('Test Suite', () => {
  it('123456: should do something', { tags: ['tag1', 'tag2'] }, () => {
    // Test implementation
  });
});

Tests with existing IDs will be skipped during the creation process. This is useful for tests that have already been synced with Azure Test Plans.

Array Format

describe('Test Suite', () => {
  it('should do something', { tags: ['tag1', 'tag2'] }, () => {
    // Test implementation
  });
});

Automatic Test Case ID Updates

After creating test cases in Azure Test Plans, the plugin automatically updates your spec files by adding the test case ID to the test title. For example:

Before:

it('should do something', { tags: ['tag1', 'tag2'] }, () => {
  // Test implementation
});

After:

it('123456: should do something', { tags: ['tag1', 'tag2'] }, () => {
  // Test implementation
});

This makes it easy to track which tests have already been synced with Azure Test Plans and prevents duplicate test cases from being created in subsequent runs.

Handling Duplicate Test Names

The plugin properly handles multiple tests with the same name in the same test file. Each test with the same name will be assigned its own unique test case ID in Azure DevOps, and the spec file will be updated accordingly. For example:

Before:

it('should do something', { tags: ['tag1', 'tag2'] }, () => {
  // First implementation
});

it('should do something', { tags: ['tag3', 'tag4'] }, () => {
  // Second implementation
});

After:

it('123456: should do something', { tags: ['tag1', 'tag2'] }, () => {
  // First implementation
});

it('789012: should do something', { tags: ['tag3', 'tag4'] }, () => {
  // Second implementation
});

This ensures that each test case in Azure DevOps is correctly mapped to its corresponding test in the spec file, even when multiple tests have the same name.

Output

The plugin generates two mapping documents:

• test-mapping.csv: CSV file with test case IDs, titles, tags, area paths, test plan IDs, and test suite IDs
• test-mapping.json: JSON file with the same information

Example CSV format:

Test Case ID,Test Title,Tags,Area Path,Test Plan ID,Test Suite ID
238226,"Validate get all levels response schema","@levels, @api","Your Project Name\Application Name\Feature",170880,170921

Azure DevOps Permissions

The Personal Access Token (PAT) needs the following permissions:

• Test Plan: Read & Write
• Work Items: Read & Write

Troubleshooting

If you encounter issues:

1. Check your configuration for correct values
2. Ensure your PAT has the necessary permissions
3. Verify that your Cypress tests include tags in the correct format
4. Check that your spec files have the correct metadata comments if you're using them
5. Ensure the Area Path exists in your Azure DevOps project
6. Verify that the field specified in automationStateField (default is 'Custom.AutomationState') exists in your Azure DevOps project
7. Check the console output for error messages and warnings

License

MIT