@bugmojo/playwright-reporter

Your Playwright run, kept as proof. Every test you already run becomes a recorded result in BugMojo — with the trace, the video, the screenshots and the console output attached, tied to the exact commit and the exact deployed URL.

A failing test that BugMojo has never seen before becomes a test case and a bug, automatically. That is how a bug an agent found at 2am is waiting for you in the morning instead of scrolling past in a CI log.

It will never fail your build. No API key, no network, an expired token, a 40 MB trace, a 500 from the server — the reporter writes one line to stderr and gets out of the way. Reporting is not allowed to be the reason npx playwright test goes red.

Install

npm install -D @bugmojo/playwright-reporter
# or: pnpm add -D @bugmojo/playwright-reporter

Configure (3 lines)

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['list'], ['@bugmojo/playwright-reporter']],          // 1
  use: {
    trace: 'retain-on-failure',                                     // 2
    video: 'retain-on-failure',                                     // 3
    screenshot: 'only-on-failure',
  },
});

Then two environment variables:

BUGMOJO_API_KEY=bm_key_...     # Settings -> API keys, with write permission
BUGMOJO_PROJECT=WEB            # your project key

That is the whole setup. npx @bugmojo/cli init writes both blocks for you if you would rather not edit the config by hand.

What you get

| After one npx playwright test | Where it lands | | --- | --- | | A run stamped with the commit SHA, deployed URL, environment and runner (LOCAL / CI) | the project's run history | | One result per test — status, duration, retries, the first line of the failure | pinned to the test case's exact revision | | The trace.zip, the video, the screenshots, stdout/stderr, and any JSON you attached | evidence on the failing result, replayable in the trace viewer | | A PROPOSED test case + a bug for a failure with no matching case | the review queue, with the failure already written up | | Flaky tests marked as flaky (failed, then passed on retry) | the flakiness report |

How a spec finds its test case

Two conventions, both supported. Use whichever fits.

1. Annotate the test — survives renaming the file:

test('checkout with a saved card', async ({ page }) => {
  test.info().annotations.push({ type: 'bugmojo', description: 'WEB-42' });
  // ...
});

2. Name the spec file — the convention bugmojo verify already uses, so repro specs exported from BugMojo work with no edits:

tests/bugmojo/WEB-42.spec.ts            -> WEB-42
tests/bugmojo/WEB-42-checkout.spec.ts   -> WEB-42   (prefix before the 2nd dash)

A [WEB-42] prefix in the test title and a @WEB-42 Playwright tag work too. An annotation can also carry a raw BugMojo test-case id.

No match, and the test failed? The reporter says so, and BugMojo creates the case as PROPOSED plus a bug with the spec path and the failure line. Turn it off with BUGMOJO_CREATE_MISSING_CASES=false or createMissingCases: false.

Configuration reference

Every option can be set in playwright.config.ts or via the environment. The config file wins.

| Option | Env var | Default | What it does | | --- | --- | --- | --- | | apiKey | BUGMOJO_API_KEY | — | Required. A key with write permission. | | project | BUGMOJO_PROJECT | — | Required. Project key (WEB) or id. | | apiUrl | BUGMOJO_API_URL | https://www.bugmojo.com | Self-hosted or staging instance. | | planVersionId | BUGMOJO_PLAN_VERSION | — | Bind the run to a frozen test-plan version. | | suiteName / suiteId | BUGMOJO_SUITE / BUGMOJO_SUITE_ID | Automated (Playwright) | Which suite the run hangs off. | | environment | BUGMOJO_ENVIRONMENT | — | staging, production, … | | deployUrl | BUGMOJO_DEPLOY_URL | PLAYWRIGHT_BASE_URL | The build under test. | | commitSha | BUGMOJO_COMMIT_SHA | CI vars, then git rev-parse HEAD | What was tested. | | runner | — | detected | LOCAL or CI. | | runId | BUGMOJO_RUN_ID | — | Append to an existing run. | | externalRunId | BUGMOJO_RUN_KEY | the CI run id | Shards of one job share it, so a sharded suite is ONE run. | | createMissingCases | BUGMOJO_CREATE_MISSING_CASES | true | Failing test with no case -> PROPOSED case + bug. | | uploadEvidence | BUGMOJO_UPLOAD_EVIDENCE | true | Upload artifacts at all. | | evidenceOnFailureOnly | BUGMOJO_EVIDENCE_ON_FAILURE_ONLY | true | Skip artifacts for clean passes. | | includeStdio | BUGMOJO_INCLUDE_STDIO | true | Attach captured stdout/stderr. | | timeoutMs | BUGMOJO_TIMEOUT_MS | 120000 | Whole-reporter budget; exceeded means give up. | | disabled | BUGMOJO_DISABLED | false | Turn reporting off entirely. | | debug | BUGMOJO_DEBUG | false | Print what was resolved and uploaded. |

reporter: [
  ['@bugmojo/playwright-reporter', {
    project: 'WEB',
    environment: 'staging',
    evidenceOnFailureOnly: false,   // keep traces for green runs too
  }],
],

In CI

- run: npx playwright test
  env:
    BUGMOJO_API_KEY: ${{ secrets.BUGMOJO_API_KEY }}
    BUGMOJO_PROJECT: WEB
    BUGMOJO_ENVIRONMENT: staging
    PLAYWRIGHT_BASE_URL: ${{ steps.deploy.outputs.url }}

GITHUB_SHA (or the pull request's head SHA), GITHUB_RUN_ID and CI are read automatically, so sharded jobs merge into one BugMojo run and a re-run of the same job is idempotent rather than a duplicate.

Sizes, and what gets skipped

Artifacts are uploaded straight to storage on a presigned URL — the bytes never pass through the BugMojo API. Per-artifact ceilings:

| Kind | Cap | | --- | --- | | Trace, video | 100 MB | | HAR, rrweb | 25 MB | | Screenshot, JSON report, other files | 10 MB | | Console output (stdout/stderr) | 5 MB, tail-truncated |

Plus 25 artifacts and 250 MB per test. Anything over is skipped with a warning — the result itself is always recorded.

Privacy

The reporter sends test titles, spec paths, failure messages, the artifacts Playwright produced, and the run's commit/URL metadata. It never reads or uploads your application source.

Troubleshooting

| You see | Why | | --- | --- | | reporting is off — no API key (set BUGMOJO_API_KEY) | Expected on a contributor's machine with no key. The run is unaffected. | | No BugMojo test case matched … | The spec carries no annotation, tag or recognised file name. Add an annotation, or let a failure create the case. | | evidence storage is not configured on this BugMojo instance | A self-hosted instance without S3. Results still land; artifacts do not. | | reporting exceeded 120000ms — giving up | The API was unreachable or very slow. Nothing else is affected. |

Set BUGMOJO_DEBUG=1 to see the resolved project, runner, CI provider and API URL before the run starts.

License

MIT · bugmojo.com