Playwright Reporter for QAStudio.dev

A comprehensive Playwright test reporter that automatically sends test results to the QAStudio.dev test management platform.

Features

• ✅ Automatic test result reporting to QAStudio.dev
• ✅ Auto-creation of test runs
• ✅ Map Playwright tests to QAStudio.dev test cases
• ✅ Upload screenshots and videos for failed tests
• ✅ Capture error context (code snippets, error location, test steps)
• ✅ Include console output for enhanced debugging
• ✅ Batch API requests for optimal performance
• ✅ Retry handling (only reports final results)
• ✅ Detailed error handling with fallback mode
• ✅ Rich metadata extraction from test annotations
• ✅ TypeScript support with full type definitions

Installation

npm install --save-dev @qastudio-dev/playwright

Or with yarn:

yarn add -D @qastudio-dev/playwright

Quick Start

1. Configure Environment Variables

Create a .env file in your project root:

QA_STUDIO_API_URL=https://qastudio.dev/api
QA_STUDIO_API_KEY=your-api-key-here
QA_STUDIO_PROJECT_ID=your-project-id-here

2. Update Playwright Configuration

Add the reporter to your playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'], // Keep default reporters
    [
      '@qastudio-dev/playwright',
      {
        apiUrl: process.env.QA_STUDIO_API_URL!,
        apiKey: process.env.QA_STUDIO_API_KEY!,
        projectId: process.env.QA_STUDIO_PROJECT_ID!,
        environment: process.env.CI ? 'CI' : 'local',
        createTestRun: true,
      },
    ],
  ],
});

3. Run Your Tests

npx playwright test

Test results will be automatically sent to QAStudio.dev!

Configuration Options

| Option | Type | Required | Default | Description | | ---------------------- | ------- | -------- | -------------- | --------------------------------------------------- | | apiUrl | string | ✅ | - | QAStudio.dev API base URL | | apiKey | string | ✅ | - | API key for authentication | | projectId | string | ✅ | - | QAStudio.dev project ID | | testRunId | string | ❌ | - | Existing test run ID (auto-created if not provided) | | environment | string | ❌ | 'default' | Environment name (e.g., 'CI', 'staging') | | createTestRun | boolean | ❌ | true | Auto-create test run if testRunId not provided | | testRunName | string | ❌ | Auto-generated | Name for new test runs | | testRunDescription | string | ❌ | - | Description for new test runs | | milestoneId | string | ❌ | - | Associate test run with milestone | | verbose | boolean | ❌ | false | Enable detailed logging | | batchSize | number | ❌ | 10 | Batch size for sending results | | uploadScreenshots | boolean | ❌ | true | Upload screenshots for failed tests | | uploadVideos | boolean | ❌ | true | Upload videos for failed tests | | includeErrorSnippet | boolean | ❌ | true | Include code snippet showing where error occurred | | includeErrorLocation | boolean | ❌ | true | Include precise error location (file, line, column) | | includeTestSteps | boolean | ❌ | true | Include test execution steps for failed tests | | includeConsoleOutput | boolean | ❌ | false | Include console output (stdout/stderr) | | maxRetries | number | ❌ | 3 | Max retry attempts for API requests | | timeout | number | ❌ | 30000 | API request timeout (ms) | | silent | boolean | ❌ | true | Don't fail tests if API is unavailable |

Linking Tests to QAStudio.dev Test Cases

Test case mapping is optional! The reporter will automatically send all test results to QAStudio.dev, even without explicit test case IDs. Tests are automatically mapped based on the hierarchy of test names and groups from your Playwright test structure (using describe blocks and test titles). This maintains the same organization in QAStudio.dev as in your test files.

However, if you want to link specific Playwright tests to existing QAStudio.dev test cases for better organization and traceability, there are two ways to do so:

Method 1: Using Annotations (Recommended)

import { test, expect } from '@playwright/test';

test('user can login', async ({ page }) => {
  test.info().annotations.push({
    type: 'testCaseId',
    description: 'QA-123', // QAStudio.dev test case ID
  });

  // Your test code...
});

Method 2: Include ID in Test Title

test('[QA-123] user can login', async ({ page }) => {
  // Your test code...
});

Without Test Case Mapping

test('user can login', async ({ page }) => {
  // Your test code...
  // This test will still be reported to QAStudio.dev automatically!
});

Adding Tags and Metadata

Enhance your test reports with custom metadata:

test('critical user flow', async ({ page }) => {
  test
    .info()
    .annotations.push(
      { type: 'testCaseId', description: 'QA-124' },
      { type: 'tag', description: 'critical' },
      { type: 'tag', description: 'smoke' },
      { type: 'priority', description: 'high' },
      { type: 'owner', description: 'qa-team' }
    );

  // Your test code...
});

All annotations are sent to QAStudio.dev as metadata.

Error Context and Debugging

By default, the reporter captures rich error context to help debug failed tests:

What's Captured by Default

• Error Code Snippet: The actual code where the error occurred, with the failing line highlighted
• Precise Error Location: File path, line number, and column where the error happened
• Test Execution Steps: Step-by-step trace of what Playwright was doing when the test failed
• Console Output: Standard output and errors (disabled by default, opt-in)

Customizing Error Context

You can control what error context is captured:

export default defineConfig({
  reporter: [
    [
      '@qastudio-dev/playwright',
      {
        // ... other options
        includeErrorSnippet: true, // Include code snippet (default: true)
        includeErrorLocation: true, // Include error location (default: true)
        includeTestSteps: true, // Include execution steps (default: true)
        includeConsoleOutput: false, // Include stdout/stderr (default: false)
      },
    ],
  ],
});

Disabling Error Context

To reduce data sent to the platform, you can disable specific features:

{
  includeErrorSnippet: false,     // Don't send code snippets
  includeErrorLocation: false,    // Don't send precise error location
  includeTestSteps: false,        // Don't send execution steps
  includeConsoleOutput: false,    // Don't send console output (already default)
}

Example: Error Context Data

When a test fails with error context enabled, the reporter sends:

{
  error: "Expected 'Login' to be visible",
  stackTrace: "Error: Expected 'Login' to be visible\n  at ...",
  errorSnippet: "  23 | await expect(page.getByRole('button', { name: 'Login' })).toBeVisible();\n      ^",
  errorLocation: {
    file: "/tests/auth.spec.ts",
    line: 23,
    column: 5
  },
  steps: [
    {
      title: "page.goto",
      category: "pw:api",
      duration: 1245,
      location: { file: "/tests/auth.spec.ts", line: 21, column: 3 }
    },
    {
      title: "expect.toBeVisible",
      category: "expect",
      duration: 5000,
      error: "Expected 'Login' to be visible",
      location: { file: "/tests/auth.spec.ts", line: 23, column: 5 }
    }
  ],
  consoleOutput: {
    stdout: "Test starting...\nNavigating to login page...",
    stderr: "Warning: Deprecated API usage"
  }
}

This rich context makes it much easier to understand and debug test failures directly in QAStudio.dev.

Advanced Configuration

Using with Existing Test Run

To add results to an existing test run instead of creating a new one:

export default defineConfig({
  reporter: [
    [
      '@qastudio-dev/playwright',
      {
        apiUrl: process.env.QA_STUDIO_API_URL!,
        apiKey: process.env.QA_STUDIO_API_KEY!,
        projectId: process.env.QA_STUDIO_PROJECT_ID!,
        testRunId: 'existing-run-id', // Use existing run
        createTestRun: false, // Don't create new run
      },
    ],
  ],
});

CI/CD Integration

Example GitHub Actions workflow:

name: E2E Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run tests
        env:
          QA_STUDIO_API_URL: ${{ secrets.QA_STUDIO_API_URL }}
          QA_STUDIO_API_KEY: ${{ secrets.QA_STUDIO_API_KEY }}
          QA_STUDIO_PROJECT_ID: ${{ secrets.QA_STUDIO_PROJECT_ID }}
        run: npx playwright test

Debugging

Enable verbose logging to troubleshoot issues:

export default defineConfig({
  reporter: [
    [
      '@qastudio-dev/playwright',
      {
        // ... other options
        verbose: true, // Enable detailed logs
        silent: false, // Throw errors instead of failing silently
      },
    ],
  ],
});

Error Handling

By default, the reporter operates in "silent" mode, meaning it won't fail your tests if the QAStudio.dev API is unavailable. Errors are logged to the console but don't affect test execution.

To make API failures throw errors:

{
  silent: false;
}

API Reference

Reporter Lifecycle

The reporter follows Playwright's reporter lifecycle:

1. onBegin - Creates or connects to test run
2. onTestBegin - Tracks test start time
3. onTestEnd - Collects test results and attachments
4. onEnd - Sends results in batches and completes test run

Test Result Data

Each test result includes:

• Test case ID (if linked)
• Title and full path
• Status (passed/failed/skipped/timedOut)
• Duration
• Error messages and stack traces
• Code snippet showing where error occurred (if available)
• Precise error location (file, line, column)
• Test execution steps with timing and errors
• Console output (stdout/stderr, optional)
• Attachments (screenshots, videos, traces)
• Browser/project information
• Custom metadata from annotations

Troubleshooting

Tests not appearing in QAStudio.dev

1. Check your API credentials are correct
2. Enable verbose mode to see detailed logs
3. Verify the project ID exists in QAStudio.dev
4. Check that tests are linked with valid test case IDs

API timeout errors

Increase the timeout value:

{
  timeout: 60000; // 60 seconds
}

Too many API requests

Increase batch size to reduce number of requests:

{
  batchSize: 50;
}

Screenshots/videos not uploading

Ensure Playwright is configured to capture them:

export default defineConfig({
  use: {
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

Examples

Check the examples/ directory for complete working examples:

• examples/playwright.config.ts - Full configuration example
• examples/example.spec.ts - Test examples with annotations
• examples/.env.example - Environment variables template

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Development Setup

# Clone the repository
git clone https://github.com/QAStudio-Dev/playwright.git
cd @qastudio-dev/playwright

# Install dependencies
npm install

# Build the project
npm run build

# Run tests
npm test

License

MIT License - see LICENSE file for details

Support

• 📚 Documentation
• 🐛 Issue Tracker
• 💬 Discussions

Changelog

See CHANGELOG.md for release history.

Made with ❤️ for the QA community