@c9up/helix

Framework-agnostic testing toolkit for the Ream ecosystem — a Vitest-compatible test runner CLI with fluent assertions, container overrides, and time-travel. Host-specific fakes (bus / HTTP / DB) live in their own integration packages, not here — helix stays dependency-light.

Sub-barrels

• @c9up/helix — assertions + expect, the test entry point
• @c9up/helix/runtime — Vitest-compatible DSL (test, describe, expect, vi, lifecycle hooks)
• @c9up/helix/runtime/worker — worker entry for the parallel runner
• @c9up/helix/container — useContainer / override / overrideOn / spy
• @c9up/helix/time — time-travel helpers

CLI

helix test [paths...|suites...]  # run paths, or suites named in helix.config
helix test --watch               # re-run on file changes
helix test --coverage            # V8 coverage + LCOV + thresholds
helix test --diff-cov            # diff coverage vs main branch
helix test --bail                # stop at the first failure
helix test --failed              # re-run what failed last time
helix test --list-pinned         # print the .pin()ed tests, run nothing

Suites

Declare named suites and helix test unit runs one, the AdonisJS way. With no positional, every suite runs, in order:

// helix.config.ts
export default {
  timeout: 2_000,          // AdonisJS `tests.timeout`
  forceExit: false,        // AdonisJS `tests.forceExit`
  suites: [
    { name: "unit", files: ["tests/unit/**/*.spec.(js|ts)"] },
    { name: "functional", files: ["tests/functional/**/*.spec.ts"], timeout: 30_000 },
  ],
}

A suite's name reaches the tests as ctx.test.options.meta.suite.name and rides on the suite:* events — meta.suite is the suite object, not the bare string. Without a config file — or when a positional is not a suite name — positionals stay paths, exactly as before.

A suite may carry configure(suite), which gets the same handle as the bootstrap's configureSuite and runs after it. It costs an import of the config module in EVERY worker, because a function cannot cross the process boundary: configureSuite in tests/bootstrap.ts does the same job for free, and is what AdonisJS itself uses. Nothing is imported when no suite declares one.

Suites run one after another, and the sequence is what the run-wide flags act on: --watch re-runs every selected suite on each change (one watcher for the whole sequence), --bail stops at the suite that failed, and the --failed cache holds every suite's failures.

files takes three forms — one pattern, an array of them, or a callback returning the URLs (files: () => [pathToFileURL(…)]), which runs in the CLI process and can therefore reach files no suffix-based discovery would find.

Pattern entries are plain paths (a directory is walked with helix's suffix discovery) or globs — *, **, ?, {a,b}, (a|b), [abc] and the extglob quantifiers @(a|b), ?(a|b), *(a|b), +(a|b), which covers AdonisJS's own defaults verbatim (tests/unit/**/*.spec.(js|ts)). An entry starting with ! subtracts from what the others selected ("!tests/unit/slow/**").

One form is refused rather than approximated: the negated extglob GROUP !(a|b). Its semantics have no faithful regex rendering, and a near-miss would silently select the wrong files — the one thing a file selector must not do. Write a !pattern entry instead.

Filters: --tests and --groups take exact titles, --files matches path segments (--files=user, --files=unit/*), --tags matches ANY of the given tags (--match-all, spelled --matchAll too, requires every one), and a ~@tag / !@tag entry excludes. --suite=<name> names the suite the files belong to ("default" otherwise — the implicit suite). --grep is a helix extra: a regex or substring over the full test name.

--bail stops at the first failure; --bail-layer=group|suite|runner says how far that reaches. Within a file the remaining tests are reported as SKIPPED. Files not yet started are dropped rather than skipped — a named deviation that follows from per-file process isolation: reporting them as skipped would mean spawning every remaining file just to collect names.

--list-pinned collects the files, prints the tests marked .pin() and runs nothing — the global setup hooks are skipped too, since nothing they would open gets used. It stays on the TypeScript pool: it prints a list rather than executing, so the native engine's fast path buys nothing.

--failed re-runs only what failed last time, from the cache each run writes to node_modules/.cache/helix/summary.json (a { tests } shape). --reporters=spec,json activates several reporters at once. --force-exit (or forceExit in the config) calls process.exit() as soon as the run ends; without it the process exits on its own once the event loop drains — so a resource a test left open surfaces as a diagnosable hang instead of being swallowed.

All of these work on BOTH orchestrators: the native (Rust) engine serializes the full per-test summary and implements bail and the reporter chain itself. Only coverage, diff coverage and a pluggable reporter instance keep a run on the TypeScript pool.

Bootstrap

tests/bootstrap.ts is the AdonisJS bootstrap module, with the same three exports — an Adonis one ports over unchanged:

// tests/bootstrap.ts
export const plugins = [apiClient({ baseUrl })]
export const runnerHooks = {
  setup: [(runner) => migrate()],   // may resolve to its own undo
  teardown: [() => closePool()],
}
export const configureSuite = (suite) => {
  if (["functional", "e2e"].includes(suite.name)) {
    return suite.setup(() => httpServer.start())
  }
}

It is picked up automatically (helix.config's bootstrap overrides the path) and imported by each worker before its test file, so a plugin's context macros exist by the time the first test declares itself.

A setup hook receives the runner and may RESOLVE TO ITS OWN UNDO — the AdonisJS idiom (setup: [() => testUtils.db().migrate()], where migrate() resolves to the rollback). Returned undos unwind first, then the declared teardowns, both in reverse order.

configureSuite receives the Suite surface: name, setup, teardown, bail, and the onTest / onGroup taps — each mapped onto the node the runtime actually reads, so suite.onTest(t => t.timeout(30_000)) really does change the timeout. What a callback cannot get is what only the owner of execution has (add, stack, exec, failed): helix builds the tree from the file's own describe/test and runs it itself.

Two more Config fields live here too, since helix has no bin/test.ts to put them in: filters ({ tests, groups, tags, matchAll } — the CLI flags still win over them) and importer, the hook that replaces the plain dynamic import of a test file. filters.files / filters.suites stay CLI-side (--files, a suite positional): helix settles the file list before any worker — and so any bootstrap — exists, and filtering there still avoids the spawn.

runnerHooks run ONCE for the whole run, in the process that spawns the workers. A migration in setup migrates once, not once per test file. Everything else in the bootstrap is per worker because it has to be: a context macro, a filter, an importer only mean anything in the process that loads the test file, which is why plugins are where an in-memory resource belongs.

Plugins

A plugin is a function run once at configure() time, handed the run's configuration object — plus two helix extras:

await configure({
  plugins: [
    ({ config, cliArgs, runner, emitter, context, cleanup }) => {
      context.macro("greeting", "hello")       // extend the test context
      emitter.on("test:end", (t) => { … })     // observe the run
      cleanup(async () => server.close())      // close resources afterwards
    },
  ],
})

• config — the run's BaseConfig, filled in with what this run actually is: cwd, timeout, retries, filters, configureSuite, reporters, plugins, importer, refiner, forceExit, setup, teardown. What a plugin can STEER is marked as such below; filters.files / filters.suites and reporters.activated report what the CLI decided, because the file list and the reporter chain are settled before a worker exists
• cliArgs — every flag the CLI forwarded to this worker (the set: tags, tests, groups, files, matchAll, timeout, retries, reporters, bail, bailLayer, failed, forceExit, suite)
• runner — getSummary(), failed, bail(), onSuite(), suites, and registerReporter(), which hands a reporter this worker's runner and emitter. It observes THIS FILE; run-wide output is the CLI's (--reporters, run({ reporterInstance })). add / start / exec / end throw a RunnerNotDrivableError explaining that the CLI owns discovery and execution — a sentence rather than the missing-property crash they would otherwise be
• emitter — runner:start / suite:* / group:* / test:*, with errors[].error the thrown Error itself
• context — macro / getter (also on the TestContext class)
• cleanup — a teardown run once the file's tests finish

config and cliArgs are handed over MUTABLE and read back once every plugin has run, so a plugin can raise config.timeout, push a setup hook, narrow cliArgs.tags, call config.refiner.add("tags", […]) or replace config.configureSuite and have the run follow. Plugins therefore run BEFORE both the run's setup hooks and configureSuite.

In package.json, call the helix bin directly — in npm scripts it resolves to node_modules/.bin/helix and bootstraps the TS loader itself, so the verbose node --import tsx node_modules/@c9up/helix/bin/helix.js … form is unnecessary:

{
  "scripts": {
    "test": "helix test app --threads=1",
    "test:coverage": "helix test app --threads=1 --coverage"
  }
}

Self-testing (Stage 2a — coexistence)

Helix is currently tested by two runners in parallel:

• pnpm test — vitest runs tests/integration/**. This is the established safety net: every commit must verdict pass here.
• pnpm test:self — helix runs tests/selftest/**. This is helix testing itself: the runtime DSL, lifecycle hooks, spies, fake timers, and a small set of parity proofs. Vitest is excluded from tests/selftest/** so it doesn't touch helix-DSL imports.

Both commands run independently in CI. Stage 2b will retire vitest once the helix self-test corpus reaches parity coverage with the vitest suite.

Vitest parity proofs

A handful of identical test bodies live in BOTH directories (tests/selftest/parity-*.test.ts and tests/integration/parity-*-mirror.test.ts). When both runners pass the same assertions, helix's matcher semantics are compatible with vitest's for the asserted surface area. The mirror pairs currently cover:

• equality matchers (toBe, toEqual, not.toBe)
• string / array toContain
• assertion failure shapes (AssertionError thrown, message contains both received and expected values)

What the VITEST parity proofs do NOT cover today (the runner semantics are covered by the self-test suite):

• Spy / fake-timer parity (vi.fn, vi.spyOn, vi.useFakeTimers).
• Failure-pipeline parity (i.e. that both runners REPORT a failed test the same way, at the runner level). The current parity-fail mirrors catch the failure in-process, so both runners verdict pass on those files — only the matcher message format is compared.

A divergence guard in tests/integration/parity-mirror-divergence.test.ts asserts the mirror bodies stay byte-identical (modulo the runner import line). Stage 2b's vitest retirement plan must include deleting tests/integration/parity-*-mirror.test.ts and the divergence guard itself, since they exist solely to bridge the two runners.

The Stage 2b cutover criterion is "every runtime DSL surface (describe, test, expect.*, lifecycle hooks, vi.*) has at least one selftest, and the runner-level failure pipeline is proven via a fixture + child-process pattern". When that bar is met, vitest can be removed; until then it stays as the safety net.