@orangebeard-io/vitest-orangebeard-reporter
v1.0.5
Published
Vitest reporter that sends real-time results to Orangebeard via @orangebeard-io/javascript-client
Readme
- Maps Vitest files, suites and tests to Orangebeard test runs, suites and tests.
- Forwards
console.log/console.errorand other console output as structured logs attached to the running test. - Reports BEFORE / AFTER style tests (e.g. setup/teardown) as
TestType.BEFORE/TestType.AFTER. - When coverage is enabled, creates a single AFTER test named "Coverage report" that logs a markdown coverage table by file.
Installation
Install the reporter:
npm install --save-dev vitest-orangebeard-reporterFor local development of this repo, run npm install in the project root instead.
Configuring Orangebeard
Create a new file named orangebeard.json in the project root folder, next to package.json. Add the following entry:
{
"endpoint": "https://app.orangebeard.io/[ORGANIZATION]",
"token": "[LISTENER TOKEN]",
"project": "example-project",
"testset": "My Vitest set",
"description": "A Vitest test run",
"attributes": [
{
"key": "Tool",
"value": "Vitest"
}
],
"referenceUrl": "https://docs.orangebeard.io/"
}
It's good practice to omit the token from the json file and get it from your env:
Windows cmd:
set orangebeard_token=[LISTENER TOKEN]Linux/Mac:
export orangebeard_token=[LISTENER TOKEN]Environment properties
Properties can also be set in the build, by passing them as environment variables.
It's important to mention that environment variables have precedence over the orangebeard.json definition.
$ export ORANGEBEARD_ENDPOINT=https://app.orangebeard.io/[ORGANIZATION]
$ export ORANGEBEARD_TOKEN=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
$ export ORANGEBEARD_PROJECT=example_project
$ export ORANGEBEARD_TESTSET=Jest testset
$ export ORANGEBEARD_DESCRIPTION=My awesome testrun
$ export ORANGEBEARD_ATTRIBUTES=key:value; value;Using the reporter in Vitest
Add the reporter to your vitest.config.ts:
import { defineConfig } from 'vitest/config';
import OrangebeardVitestReporter from 'vitest-orangebeard-reporter';
export default defineConfig({
test: {
reporters: [new OrangebeardVitestReporter()],
},
});If you are using a monorepo or custom project configuration, make sure the reporter is added to the Vitest project where you want results to be sent to Orangebeard.
Running with coverage
When running Vitest with coverage enabled (for example using the built-in v8 provider):
npx vitest run --coverageThe reporter will:
- Collect the aggregated coverage map from Vitest via
onCoverage. - At the end of the run, create a dedicated AFTER test called "Coverage report" in a top-level Coverage suite.
- Attach a markdown table log with one row per file, including line / statement / branch / function percentages.
This gives you a single place in Orangebeard where you can inspect per-file coverage for the run.
Mapping Vitest to Orangebeard
The reporter follows these mapping rules:
Test run
onTestRunStartcreates an Orangebeard test run using the configuredtestset,descriptionandattributesfrom the JavaScript client.onTestRunEndwaits for any pending async work (attachments) and then finishes the test run.
Suites
- Each test file becomes a root suite, using the path relative to the current working directory (e.g.
tests/sample.test.ts). - Nested
describeblocks are represented as nested Orangebeard suites beneath the file suite.
- Each test file becomes a root suite, using the path relative to the current working directory (e.g.
Tests
- Each Vitest
it/testbecomes an Orangebeard test. - The test name is taken from
test.name. - The description includes the source location when available, e.g.
sample.test.ts:10. - Status is mapped from Vitest state (
passed,failed,skipped,pending, ...) toFinishTest.Status.
- Each Vitest
BEFORE / AFTER classification
- Test names that contain
beforeAll,before all,beforeEach,before eachorsetupare reported asTestType.BEFORE. - Test names that contain
afterAll,after all,afterEach,after eachorteardownare reported asTestType.AFTER.
- Test names that contain
Console logs
onUserConsoleLogreceives Vitest console events and forwards them as Orangebeard logs.- Logs are attached to the running test when possible (using the Vitest task id → test UUID mapping).
- Logs emitted before a test has started are buffered and flushed as soon as the test is created.
Attachments
- When Vitest test artifacts are recorded (e.g. screenshots, traces written to disk),
onTestCaseArtifactRecordlogs a short summary entry and uploads the underlying file as an Orangebeard attachment.
- When Vitest test artifacts are recorded (e.g. screenshots, traces written to disk),
Running this package locally
- Install dependencies:
npm install- Build the TypeScript sources:
npm run build- Run the unit tests (including the reporter integration tests):
npx vitest run --coverageThis will also exercise the Orangebeard integration (using the mocked client in tests) and generate a coverage report for the reporter itself.
Sample project
The tests/ folder in this repository contains sample tests that exercise:
- Passing, failing, skipped and todo tests.
- Global
beforeAll/afterAllhooks. - Per-test
beforeEach/afterEachhooks. - Console logging during tests and hooks.
When you run the sample suite with the reporter configured, you should see in Orangebeard:
- A single test run with a suite named
tests/sample.test.tsand nested suitesample suite. - Tests for each case and hook, with appropriate status and type (BEFORE / TEST / AFTER).
- Console log entries attached to the relevant tests.
- A Coverage suite with a "Coverage report" test containing a coverage table per file.
