@open-xchange/playwright-hindsight

Playwright reporter that sends test results to Hindsight.

Installation

npm install @open-xchange/playwright-hindsight

Configuration

In your playwright.config.ts:

import { defineConfig } from '@playwright/test'

export default defineConfig({
  reporter: [
    ['list'],
    ['@open-xchange/playwright-hindsight', {
      url: 'https://hindsight.example.com',
      project: 'my-project',
      client: 'playwright',
    }],
  ],
})

Options

All options can also be set via environment variables — useful for CI pipelines.

| Option | Env var | Default | Description | |--------|---------|---------|-------------| | url | HINDSIGHT_URL | http://localhost:8080 | Hindsight API base URL | | branch | CI_COMMIT_REF_NAME | local | Git branch name | | project | CI_PROJECT_PATH | hindsight | Project identifier | | pipelineId | CI_PIPELINE_ID | — | CI pipeline ID | | client | — | playwright | Client identifier | | deployment | REPORT_PROJECT | — | Deployment identifier | | clientId | HINDSIGHT_CLIENT_ID | — | OIDC client ID | | clientSecret | HINDSIGHT_CLIENT_SECRET | — | OIDC client secret | | tokenEndpoint | HINDSIGHT_TOKEN_ENDPOINT | — | Keycloak token endpoint | | redact | — | [] | Values removed from failure evidence before it leaves CI | | artefactBaseUrl | — | GitLab job artefacts | Base URL attachments are published under |

Options passed in the config take precedence over environment variables.

Authentication

When clientId, clientSecret, and tokenEndpoint are provided (via options or env vars), the reporter authenticates using the OIDC client credentials flow. The token is cached and refreshed automatically.

When credentials are not configured, requests are sent without authentication — suitable for local development against an unprotected API.

What gets reported

After all tests complete, the reporter sends a single batch request to POST /api/v1/test-runs containing:

| Field | Source | |-------|--------| | feature | Test suite title (test.parent.title) | | scenario | Test title | | state | passed, failed, or skipped (timeouts → failed) | | duration_ms | Test duration | | time | Test start time (ISO 8601) | | branch | From options or CI_COMMIT_REF_NAME | | project | From options or CI_PROJECT_PATH | | pipeline_id | From options or CI_PIPELINE_ID | | client | From options (default: playwright) | | run_id | Generated per attempt | | failure | Error message, stack trace and screenshot or trace path, for failed and timed-out tests only |

Failure context

A failing or timed-out test reports what Playwright knows about the failure: error.message, error.stack, and the path of its screenshot or trace attachment. Passing and skipped tests report none, and a retry reports its own evidence under its own run_id. A timeout with no error still reports that it timed out.

Test output contains credentials, so evidence is cleaned before it is sent. The reporter always removes its own clientSecret, and redact takes any other values that must not leave CI:

['@open-xchange/playwright-hindsight', {
  url: 'https://hindsight.example.com',
  redact: [process.env.TEST_ACCOUNT_PASSWORD],
}],

Hindsight also removes recognisable secrets on arrival (authorization headers, tokens, credentials in URLs, private keys), but it can only match what secrets generally look like. Values specific to your project belong in redact.

Under GitLab CI the attachment is reported as a URL into the job's artefacts, built from CI_JOB_URL and the file's path within CI_PROJECT_DIR. That link works while the job's artefacts exist and 404s once they expire. Outside CI, and for files written outside the project, the local path is kept as-is.

Set artefactBaseUrl to publish somewhere else:

['@open-xchange/playwright-hindsight', {
  artefactBaseUrl: 'https://artefacts.example.com/run-7',
}],

The job must actually publish that directory (artifacts:paths) for the link to resolve. Evidence itself expires after 30 days by default, so this is for investigating recent failures rather than long-term history.

CI example (GitLab)

e2e tests:
  variables:
    HINDSIGHT_URL: https://hindsight.example.com
    HINDSIGHT_TOKEN_ENDPOINT: https://keycloak.example.com/realms/my-realm/protocol/openid-connect/token
    # HINDSIGHT_CLIENT_ID and HINDSIGHT_CLIENT_SECRET set as masked CI/CD variables
  script:
    - npx playwright test

Token provider utility

The package also exports createTokenProvider for use in custom setup scripts (e.g., seeding test data):

import { createTokenProvider } from '@open-xchange/playwright-hindsight'

const provider = createTokenProvider({
  clientId: process.env.HINDSIGHT_CLIENT_ID,
  clientSecret: process.env.HINDSIGHT_CLIENT_SECRET,
  tokenEndpoint: process.env.HINDSIGHT_TOKEN_ENDPOINT,
})

const token = await provider.getToken()

Error handling

API errors are logged as warnings — they never fail the test run.

License

AGPL-3.0-or-later