@gfdelarue/testkit

@gfdelarue/testkit is a test runner and CLI for coding agents and CI. It runs a repository's ordinary Vitest test files, such as src/sum.test.ts, as they are, and typed *.testkit.ts suites for Playwright and Node. It assigns tests to groups by directory, starts only the local services a group uses, provisions Docker-managed local Postgres databases when a group needs one, schedules the files, and collects results and artifacts. It starts no service for a test whose group uses none. Coding agents use Testkit to write tests and hunt bugs, and CI uses it to run those tests. It needs no account or server.

Testkit exists to replace false confidence with inspectable execution evidence. It makes claims that deserve trust: these tests genuinely executed; these were the runtimes, fixtures, and inputs supplied to the run; this is what passed, failed, was skipped, or did not run; and this is the evidence behind the result. Testkit retains provenance, logs, and verifiable artifacts so those claims can be audited rather than accepted on the strength of a green check mark.

Configure Testkit with an optional repo-root testkit.config.ts.

CI gate

Give the installed CLI one stable package script and use that script locally and in CI:

{
  "scripts": {
    "testkit": "testkit run"
  }
}

npm ci
npm run testkit

The command exits zero only when at least one test file completed and every required file passed. Test failures, infrastructure failures, interrupted or not-run files, a skipped file, and an empty selection all fail the gate. A file that a skip rule or its own skip metadata skips fails the gate even when every other file passes, and --allow-empty doesn't change that. npm run testkit -- --allow-empty is an explicit escape hatch for a job whose empty selection is intentional; it should not be the default CI command.

Testkit writes the immutable, verifiable run bundle under .testkit/results/runs/<run-id>/. Upload .testkit/results with if: always() when CI evidence should survive a failed job.

Report a run to GitHub or Gitea

testkit ci report reads the last run in the checkout and posts it in three places: one commit status, the job's step summary, and one pull request comment that it updates in place.

permissions:
  contents: read
  issues: read
  pull-requests: write
  statuses: write

steps:
  - run: npm ci
  - run: npm run testkit
  - run: npx testkit ci report
    if: always()
    env:
      GITHUB_TOKEN: ${{ github.token }}

• The testkit run step decides whether the job fails. ci report exits zero when it posts the report, and that includes the report of a failed run.
• GitHub Actions passes the job's token to a run step only when the step sets it, as the env lines above do. Gitea Actions accepts the same lines.
• Testkit reads the token from GITHUB_TOKEN or GH_TOKEN for GitHub, and from GITEA_TOKEN for Gitea. In a Gitea Actions job it also reads GITHUB_TOKEN. With no token set, it asks git credential fill. The token that actions/checkout keeps in the checkout does not answer that request.
• Testkit reaches GitHub Enterprise Server only from a GitHub Actions job on that server, so testkit bug file works there only in a job.
• GitHub gives a pull request from a fork a read-only token. ci report then writes the step summary and fails when it posts the status. Add continue-on-error: true to the step when the repository takes pull requests from forks.
• When the checkout holds no results for its commit, ci report posts a failure status with the description run failed: Testkit found no results for this commit.
• A job that runs Testkit more than once passes --context <name> to each report, so each run keeps its own status and its own comment.

File a bug as an issue

Testkit files a confirmed bug as an issue in the repository's GitHub or Gitea. The issue names the regression test that fails because of the bug, the commit that introduced the faulty lines, and the command that reproduces the failure. Bug records gives the issue format.

1. Write a Testkit test that fails because of the bug.
2. Write the bug record as a JSON file. testkit bug guide prints the fields and the cause, impact, and escape lists.
3. Run testkit bug record bug.json. It checks the record, stores a draft under .testkit/bugs/, and prints the draft id.
4. Commit the test and push the commit.
5. Run testkit bug file <id>.

testkit bug file runs the test at the checked-out commit and files the issue only when the test fails. It stops when tracked files have uncommitted changes, when no remote branch contains the commit, or when an open issue already covers the test. The token needs permission to create issues and labels.

testkit bug record saves the --vitest-config, --vitest-project, and --vitest-runner it was given in the draft, and asks for --vitest-project when two Vitest projects run the test's file. testkit bug file reruns with them, and the issue's reproduce command carries them. With TESTKIT_REQUIRE_COMPLETE_SUITE=1, it reruns every file the selection lists, then reads the recorded failure.

testkit bug file takes the same supplied values as testkit run and reruns the test on them. The issue's reproduce command carries those flags, so testkit bug file refuses inputs that would put a raw secret in it. It takes --env-from and --environment-file, and takes --base-url and --endpoint only for URLs without credentials, a query, or a fragment. It refuses --database and --env; pass those values through --env-from or the environment file. The command runs from the repository root, so a relative --environment-file needs testkit bug file run from there too.

Name the product's areas

Add areas to testkit.config.ts so that each issue names the part of the product that holds the fault:

export default defineConfig({
  areas: {
    checkout: { name: "Checkout", paths: ["src/checkout/**"] },
    billing: { name: "Billing", paths: ["src/billing/**", "src/invoices/**"] },
  },
});

testkit bug file puts an area:<id> label on the issue. If no area claims the fault's file, it stops and names the file. Without areas, Testkit files bugs with no area. See areas.

Pass a run whose failures are all filed

Every failing test fails the run by default. To keep CI passing while a filed bug waits for its fix, set bugs.knownFailures:

export default defineConfig({
  bugs: { knownFailures: "allow" },
});

A run then passes when an open issue with the testkit label covers each failure. A new failure still fails the run, and closing an issue makes its test count again. When Testkit cannot read the issues, every failure is new. Known failures gives the exact rule.

With "allow", the run reads the open issues. In CI, pass the job's token to the testkit run step as well:

steps:
  - run: npm run testkit
    env:
      GITHUB_TOKEN: ${{ github.token }}

The job also needs the issues: read permission.

Run a campaign

A campaign gives a coding agent a target and the instructions to file what it finds.

npx testkit campaign start --target "Refunds and partial refunds"
npx testkit campaign status
npx testkit campaign finish

campaign start prints the agent's brief: the target, the files that no area claims, how to work, and the bug record format. Give that output to the agent. campaign status lists the bugs recorded so far. campaign finish writes the summary for the pull request that adds the tests to .testkit/campaigns/<id>/summary.md and prints it.

Prepare and publish a report bundle

Prepare a new directory from your own template, recorded bug IDs, and selected run artifacts. All inputs are optional. Repeat --bug for related findings and --artifact for supporting files.

npx testkit artifacts --run <run-id> --json
npx testkit report prepare ./draft-report \
  --template ./report-template.md \
  --instructions ./reporting-instructions.md \
  --bug <bug-id> --run <run-id> --artifact <ref.path>

Preparation copies the template unchanged to report.md, or creates an empty file. It puts recorded facts in context.json and selected, verified run files under evidence/. Drafting instructions are printed for the author and are not copied into the bundle. Write and review the narrative yourself. Templates have no special syntax, and preparation never overwrites an existing directory.

Publish that directory, or any directory you prepared yourself, to an explicit repository, branch, and bundle path:

npx testkit report publish ./draft-report \
  --repository https://github.com/your-team/private-reports \
  --branch main --path reports/refund-race --json

Use a full Gitea repository URL to publish there. owner/name uses the host of the checkout's origin. Authentication uses the same environment variables and Git credential helper as testkit bug file; the token needs repository write access. Both commands support --dir to select the Testkit checkout and --json for structured output. Local input paths are relative to your current directory. Publication needs Git installed and works without Testkit config.

Publication replaces the chosen bundle directory, including removing files that are no longer in your local bundle. It preserves other repository paths and leaves your checkout, index, and branch untouched. The destination branch must exist unless the repository is empty. A concurrent branch update rejects the push; review the new branch state before trying again. Fetching requests a blob-free snapshot and monitors temporary Git storage against a 256 MiB budget, with possible brief overshoot between checks. Use a smaller reports repository if it exceeds that budget. An unchanged bundle returns its existing revision without creating a commit.

The result includes the exact revision and a provider url pinned to it. --message sets the commit message. Commits use the standard GIT_AUTHOR_* and GIT_COMMITTER_* identity variables when set, otherwise Testkit <testkit@localhost>. Keep the revision used for a submission. A failed push can have an uncertain result after a lost connection, so check the reported commit at the destination before retrying.

Nested text and binary files keep their bytes and names, without line-ending conversion or Git LFS filters. Publication accepts ordinary files up to 16 MiB each, 64 MiB total, and 1000 files. Directory depth is limited to 32, with at most 128 directories including the repository root and destination parents. Symlinks, Git metadata, traversal paths, and filenames containing backslashes or line breaks are rejected. Filenames must be UTF-8. Empty directories are not stored by Git. Review everything in the selected directory before publishing it, including hidden files.

Neither command needs Turnstone, a paid feature, or a fixed report format. To view the result in Turnstone, connect the destination repository and select the same branch and the parent folder, such as reports. The author submits the report to the chosen platform.

Execution engines

Testkit has two built-in engines:

• node-vitest runs plain Vitest test files with Node and Vitest.
• playwright runs ui files with Playwright and managed Chromium.

A plain Vitest test file needs no Testkit suffix, directory, metadata, or wrapper. It keeps its native vitest imports and its repository paths, has no test type, and runs under the engine's general node-vitest profile. Testkit does not evaluate testkit.config.ts in the test's process, and it installs the values of the services the test's group uses before the test's imports evaluate. See Plain Vitest test files for which files Testkit runs.

Every typed *.testkit.ts file must export defineFile({ engine: "..." }). The filename suffix supplies the public test type; Testkit validates that explicit pair and derives a runner profile. It never infers a default engine for a typed file or falls back to a different engine.

Consumers declare only @gfdelarue/testkit, which installs the exact versions of the tools its engines run. A product's own copies of Vitest, Playwright, or their tools don't change what Testkit runs.

Playwright browser installation is explicit:

npx @gfdelarue/testkit setup playwright
npx @gfdelarue/testkit setup playwright --with-deps # CI and fresh hosts

Normal runs never install or download a browser. Installation covers what an install contains, the supported platforms, and how to recover a broken one. Typechecking covers the supported TypeScript versions and tsconfig.playwright.json.

Testkit bounds the output it captures from each command it runs, and asks the command and its descendants to stop on cancellation or timeout.

Managed task environment is engine input. Playwright suites read it from readPlaywrightContext()?.env. Managed Node processes also expose each value as an immutable, non-enumerable process.env.KEY property so libraries that read process.env directly see it, but non-Node child processes do not receive consumer values unless the suite explicitly passes them.

Consumer environment names are checked case-insensitively. Every engine reserves APPDATA, ComSpec, HOME, LANG, LC_ALL, LOCALAPPDATA, PATH, PATHEXT, SystemRoot, TEMP, TMP, TMPDIR, USERPROFILE, XDG_CACHE_HOME, XDG_CONFIG_HOME, and XDG_DATA_HOME. In addition:

• Node/Vitest reserves BASE_URL, FORCE_COLOR, NODE_OPTIONS, NO_COLOR, UPDATE_SNAPSHOT, VITEST_POOL_ID, VITEST_WORKER_ID, and the TESTKIT_ keys it sets itself, listed in Environment; other TESTKIT_ and VITEST_ keys, such as TESTKIT_REQUIRE_COMPLETE_SUITE, pass through, though Vitest's own TEST, VITEST, VITEST_MODE, FORCE_TTY, MODE, DEV, PROD, and SSR replace a group's values;
• Playwright reserves the TESTKIT_, PLAYWRIGHT_, PWTEST_, and PW_TEST_ prefixes plus BASE_URL, FORCE_COLOR, NODE_OPTIONS, and NO_COLOR.

Case-variant duplicates and invalid names or NUL-containing values fail instead of being normalized. Node/Vitest accepts at most 512 entries and 4 MiB of UTF-8 key/value data. Playwright accepts at most 512 entries and 2 MiB of UTF-8 key/value data. Environment lists the complete limits, and Named endpoints and databases covers the environment API.

Usage

cd my-product

# Run every testkit-managed suite in batch mode
npx @gfdelarue/testkit run

# Inspect discovered tests without running them
npx @gfdelarue/testkit discover
npx @gfdelarue/testkit discover --output-mode verbose
npx @gfdelarue/testkit discover --json > .testkit/discovery.json
npx @gfdelarue/testkit discover --output .testkit/discovery.json

# Filter by type
npx @gfdelarue/testkit --type ui
npx @gfdelarue/testkit --type plain

# Run up to 8 test tasks at once
npx @gfdelarue/testkit --task-concurrency 8

# One file-level wall clock budget for every suite file
npx @gfdelarue/testkit --file-timeout-seconds 60

# Specific group / suite
npx @gfdelarue/testkit --group frontend --type ui -s navigation
npx @gfdelarue/testkit --group api -s health
npx @gfdelarue/testkit --group api --type plain
npx @gfdelarue/testkit --type ui -s ui:checkout

# Exact file
npx @gfdelarue/testkit --type ui --file __testkit__/checkout.ui.testkit.ts
npx @gfdelarue/testkit run --file src/sum.test.ts

# Temporarily ignore repo-declared skip rules
npx @gfdelarue/testkit --ignore-skip-rules --file src/billing.test.ts

# Lifecycle
npx @gfdelarue/testkit status
npx @gfdelarue/testkit lint
npx @gfdelarue/testkit destroy
npx @gfdelarue/testkit cleanup
npx @gfdelarue/testkit cleanup --resources --dry-run


# Diagnostics: refresh or verify the source schema cache
npx @gfdelarue/testkit db schema refresh --service api
npx @gfdelarue/testkit db schema verify --service api

--type ui and typed suite selectors such as ui:checkout select typed files only. --type plain selects plain Vitest test files only. A run without --type, or with --type all, runs both. --type narrows the other filters, so --group api --type plain runs the api group's plain files and none of its UI files.

run, discover, artifacts, workspace, and typecheck select test groups with --group. status, destroy, cleanup, and db schema take --service to name a service.

Database cleanup is scoped to the selected product. cleanup --resources --global is retired; use --dir <product> cleanup --resources for each product. After deleting a product directory, remove its orphaned containers and volumes explicitly through Docker.

Use execution.taskConcurrency in config and --task-concurrency on the CLI. A task is a typed file, a batch of typed files, or one Vitest invocation of plain files. Plain files in one invocation run with Vitest's file parallelism, which taskConcurrency doesn't limit. execution.workers, presets.nodeNext({ workers }) and --workers are deprecated aliases. When both names are provided in config or on the CLI, their values must match. The CLI value overrides config after those checks. See the execution settings.

Batch run output stays intentionally short: one line per completed file, concise failure blocks, and a final summary. Service logs, captured runtime output, and emitted artifacts are persisted under .testkit/results/.

Text output is coloured in a terminal and plain otherwise. As in Node, NO_COLOR turns colour off and FORCE_COLOR overrides it: 0 or false turns colour off and any other value turns it on.

Testkit seals an immutable local run bundle. testkit run prints Run ID: <id> after sealing in text output and returns runId in --json output, for passed and failed runs.

Testkit starts the files that took longest in earlier runs first, so slow files start while workers are free. The run document records the order under planning.

Run against a supplied environment

testkit run can take the values its tests need instead of provisioning them. Every selected group then runs on those values alone. Testkit starts none of the group's services, creates no database, and runs no .setup.* module beside a test. It also ignores the env values and env files those services declare, so supply every value the tests read. Groups, selection, skip rules, and other policy work as before.

npx @gfdelarue/testkit run --group checks \
  --base-url http://127.0.0.1:3000 \
  --endpoint api=http://127.0.0.1:3001 \
  --database store=postgres://app:[email protected]:5432/store \
  --env FEATURE_FLAG=on \
  --env-from API_TOKEN=CI_API_TOKEN

npx @gfdelarue/testkit run --environment-file supplied.json

• --base-url URL supplies the default endpoint, which tests receive as BASE_URL.
• --endpoint NAME=URL supplies a named endpoint, usually named after the service it stands in for.
• --database NAME=URL supplies a named PostgreSQL database.
• --env NAME=VALUE supplies an environment value.
• --env-from NAME=HOST_KEY supplies the value of host variable HOST_KEY under NAME.
• --environment-file PATH reads values, prepared setup data, and handle descriptors from a file.

Every flag except --base-url and --environment-file can be repeated. Tests read named values with endpoint(name) and database(name) from @gfdelarue/testkit/environment. An --env, --env-from, or file env entry named BASE_URL or DATABASE_URL sets the default endpoint or database, and an empty database URL means no database. Without an explicit default, a group's BASE_URL and DATABASE_URL are those of the first service in its uses that has a supplied endpoint or database. Either way the default is also available by the group's name, as in a configured run, so it may not differ from a named endpoint or database with that name.

Testkit reads host variables only through --env-from. A BASE_URL or DATABASE_URL already set in the shell does not switch a run to supplied values. Two inputs may define the same value only if they agree. Any other repeat, such as --base-url and --env BASE_URL with different URLs, or a missing --env-from host variable, fails the run before it starts. Before a test starts, each supplied endpoint must give a valid HTTP response. A 4xx or 5xx response counts as valid.

Each supplied database must also accept a connection before a test starts. Testkit connects, signs in with the URL's credentials, runs SELECT 1, and disconnects. It handles trust, password, md5, and SCRAM-SHA-256 authentication. TLS follows the URL's sslmode as in libpq. The default, prefer, tries TLS first and then a plain connection, and allow tries them the other way round. verify-ca and verify-full need sslrootcert, set to a CA file or to system, and sslrootcert=system alone means verify-full. host=/path connects through the Unix socket in that directory and ignores sslmode. The check reads only the URL. It ignores PG* variables, ~/.pgpass, and ~/.postgresql/root.crt, and it presents no client certificate. A database that is unreachable, fails TLS, rejects the credentials, does not exist, or is not PostgreSQL fails the run before any test starts. The message names the database, never its URL.

The environment file is JSON:

{
  "version": 1,
  "baseUrl": "http://127.0.0.1:3000",
  "databaseUrl": "postgres://app:[email protected]:5432/app",
  "endpoints": { "api": "http://127.0.0.1:3001" },
  "databases": { "store": "postgres://app:[email protected]:5432/store" },
  "env": { "FEATURE_FLAG": "on" },
  "ui": {},
  "setupData": { "actor": "Grace" },
  "handles": {
    "webhook": {
      "kind": "http-capture",
      "url": "http://127.0.0.1:41000/webhook",
      "requestsUrl": "http://127.0.0.1:41000/__testkit__/<capability>/requests",
      "resetUrl": "http://127.0.0.1:41000/__testkit__/<capability>/requests"
    }
  }
}

Only version is required, and this Testkit reads version 1. baseUrl, databaseUrl, and ui may be null, as an environment rendered by Testkit writes them, and null there counts as absent. Tests receive setupData as the data a setup module would have returned: {} when the file leaves it out, and null when the file sets it to null. handles entries must each name a kind, and tests reach them with handle(name). An http-capture entry holds the url, requestsUrl, and resetUrl that a capture sidecar reports, and gives the test the same handle a setup module's capture would. Other kinds are handed on, and handle() throws EnvironmentHandleError with "unsupported" for them. Any other field, such as service or database configuration, is rejected. Errors name the file and the field and do not repeat the file's contents.

A run records the values its tests received in this shape. Each acquisition in run.json's environment section has them under values, with secrets and capture capabilities redacted. Its setupData is the supplied setup data or what the file's setup module returned, and {} without either. Sealing redacts setup data under a key that names a secret or that redact lists, and records the rest as returned, so list any other sensitive key in redact. With "version": 1 added, an acquisition's values are an environment file that gives a second run the same values, unless redaction changed one of them.

Tooling Adapters

testkit also ships tool-specific config helpers so consumer repos do not need repo-local runtime policy files just to cooperate with managed runs.

// drizzle.config.ts
import { defineConfig } from "@gfdelarue/testkit/drizzle";

export default defineConfig({
  schema: "./src/db/schema/index.ts",
  out: "./src/db/migrations",
  dialect: "postgresql",
  dbCredentials: {
    url: process.env.DATABASE_URL!,
  },
});

// vitest.config.ts
import { defineConfig } from "@gfdelarue/testkit/tooling/vitest";

export default defineConfig({
  test: {
    include: ["src/**/*.test.ts"],
  },
});

// playwright.config.ts
import { defineTestkitConfig, devices } from "@gfdelarue/testkit/playwright";

export default defineTestkitConfig({
  testDir: "./__testkit__",
  projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
});

Managed UI suites use @gfdelarue/testkit/playwright for Testkit's Playwright helpers and re-exported authoring API. They may also import exact public exports from the managed @playwright/test, playwright, and playwright-core packages. Those imports resolve to Testkit's pinned versions at run time and when typechecking, whatever Playwright version the product installs.

For scripts and app-runtime code, @gfdelarue/testkit/env provides shared helpers for managed runtime detection, dotenv loading, and local-database safety:

import { assertLocalDatabaseUrl, loadDotenvFiles, shouldLoadDotenv } from "@gfdelarue/testkit/env";

if (shouldLoadDotenv()) {
  loadDotenvFiles({ files: [".env", ".env.local"] });
}

assertLocalDatabaseUrl(process.env, "seed.ts");

Setup

See the configuration schema for the complete, strict testkit.config.ts contract.

Create testkit.config.ts at repo root:

import { app, database, defineConfig, toolchain } from "@gfdelarue/testkit/config";

export default defineConfig({
  execution: {
    taskConcurrency: 8,
    fileTimeoutSeconds: 60,
  },
  lint: {
    rules: {
      unitTestBoundaries: true,
    },
  },
  fingerprints: {
    exclude: ["next-env.d.ts"],
  },
  toolchains: {
    frontendNode: toolchain.node({
      cwd: "frontend",
      detect: "auto",
      install: "download",
    }),
  },
  services: {
    api: app.node({
      cwd: ".",
      entry: "src/index.ts",
      envFiles: [".env.testkit"],
      database: database.postgres({
        runtimeConnections: 20,
        sourceSchema: database.schema.fromEnv("PRODUCTION_DATABASE_URL"),
        template: {
          inputs: ["db/schema.sql", "scripts/seed.ts"],
          migrate: [{ kind: "sql-file", path: "db/schema.sql" }],
          seed: [{ kind: "command", run: "npm run db:seed" }],
          verify: [{ kind: "module", target: "src/testkit/verify-seed.ts#verifySeed" }],
        },
      }),
      runtime: {
        instances: 1,
        maxConcurrentTasks: 4,
      },
    }),
    frontend: app.next({
      cwd: "frontend",
      mode: "start",
      dependsOn: ["api"],
      envFiles: ["frontend/.env.testkit"],
      env: {
        values: {
          NEXT_PUBLIC_API_URL: "{baseUrl:api}",
        },
      },
      runtime: {
        instances: 1,
        maxConcurrentTasks: 2,
        toolchain: "frontendNode",
      },
    }),
  },
});

Every service Testkit starts listens on a port the OS assigns for that run. Testkit passes it to the service as PORT and as {port} in start, env, baseUrl, and readyUrl, so two runs of the same repository on one host, such as two CI jobs on a runner or two agent worktrees, never compete for a port. A service that exits before it becomes ready gets a fresh port and up to three attempts in total. Each run's run.json lists every service's actual URL, port, and attempts under environment.services. A config that sets local.port, the port option of app.node or app.next, or a literal port in local.baseUrl or local.readyUrl fails to load; delete it and use {port}.

Every typed test file declares its execution engine and any file-local scheduling metadata next to the test. A plain Vitest test file declares neither:

import { defineFile } from "@gfdelarue/testkit";

export const testkit = defineFile({
  engine: "node-vitest",
  skip: "Billing is currently unavailable locally",
});

A file can also name locks. Give each test its own data says when to use one.

In a typed file, the engine value is a required literal. The supported public authoring path is @gfdelarue/testkit/playwright. Playwright suites may also import exact public APIs directly from Testkit's managed @playwright/test, playwright, and playwright-core packages. There are no legacy aliases or implicit-runner compatibility paths.

testkit.config.ts is optional. Without it, Testkit runs every test file with zero setup: it starts no server, provisions no database, and loads no .env file. A test that needs a server or a database gets one from a service that testkit.config.ts declares. The config is also the primary escape hatch for:

• worker count and per-file runtime budget
• per-file wall clock timeout budget
• multi-service graphs
• local runtime instance counts
• per-runtime concurrent task caps
• repo-managed Node toolchains for prepare/start commands
• one-time runtime preparation steps for stable shared servers
• local DB binding configuration
• source-backed schema verification
• template database migrate / seed / verify stages
• explicit per-file or per-suite locks
• named HTTP suite profiles
• repo-level fingerprint include/exclude policy
• repo-declared suite/file skip policies with explicit reasons

runtime.prepare is the generic build-once hook for shared runtimes. It runs once per runtime generation before local services start, and runs again when its declared inputs change. This is the right way to move expensive browser targets from next dev / watch mode to stable build-and-start flows.

Setup steps (runtime.prepare, database template, and other Testkit-run Node commands) inherit NODE_OPTIONS. TESTKIT_SETUP_NODE_OPTIONS appends extra flags to NODE_OPTIONS for those steps only, for example a larger heap for a heavy migration. Anything preloaded through it runs inside your product's setup processes, so set it deliberately.

Testkit-owned Docker resources are labelled at creation time so they can be found again even if product-local .testkit state is deleted. testkit destroy removes labelled resources for the current product after service state is destroyed. It leaves run results under .testkit/results and other files under .testkit, such as the compiled config and timings, in place. testkit cleanup --resources cleans stale resources for the current product. Both commands remove only the current product's resources and leave other products' alone, including their idle caches. Host-local Postgres data lives in a Testkit-labelled named Docker volume, so cleanup can find and remove it. Database changes for one product run one at a time; other products don't wait for them. Use cleanup --resources --dry-run to preview the selected product's cleanup.

database.template is the database-side equivalent for reusable template DB state. When database.sourceSchema is configured, Testkit treats the configured source database as the schema source of truth. A normal testkit run resolves a commit-aware source schema cache under .testkit/db/<service>/source-schemas/, applies that cached schema to the local template DB, runs local template setup, and verifies that the replayed local schema still matches the source dump. If local replay differs, Testkit refreshes from the source once for the current cache key and retries. If it still differs, the run fails with schema diagnostics under .testkit/results/schema.

Testkit keys the source schema cache on the repository's state. A clean worktree shares the entry for its commit. A dirty worktree or a directory outside Git gets its own entry, keyed on the content of its files.

The key changes when a file's bytes, type, or executable mode changes, including changes Git's own status can hide, such as sparse checkout or line-ending normalization, and when a submodule moves. Untracked files are included unless the repository's .gitignore files exclude them; global or per-clone Git excludes don't count. A path Testkit can't read, or a tree that changes while Testkit reads it, fails the run instead of producing a key.

Testkit-owned outputs are always excluded from fingerprints: .testkit/, and .next-testkit/. testkit.config.ts is normal repo configuration and is not excluded automatically. For user-managed generated files, add repo-level fingerprints.exclude patterns; fingerprints.include can opt specific paths back in below an excluded directory, including through wildcard directory patterns. Fingerprint patterns use / as the only separator; a single backslash is rejected when the config loads. A doubled backslash addresses a literal POSIX backslash in a filename (for example, "fixtures\\\\literal.txt" in testkit.config.ts source).

Branch names and worktree paths are recorded as metadata but do not affect clean commit cache keys, so branch renames and clean worktrees at the same commit reuse the same source schema. Dirty worktrees are isolated by content fingerprint so local experiments cannot overwrite a clean commit baseline.

Template setup executes in three explicit phases:

• migrate
• seed
• verify

Schema drift is checked after each successful migrate and seed step, and verify only runs once local replay matches the source schema. Source refreshes dump the schema only, in Docker, so seed and reference data never enter the baseline and no host pg_dump is needed. Keep schema-changing setup in its own step where possible; a single command that changes schema and then fails before exiting cannot be refreshed at the midpoint.

A source schema refresh uses one connection. A Neon pooled source URL is rewritten to the matching direct Neon endpoint. Other PgBouncer or pooler URLs fail; configure a direct source URL for those providers. Refreshes of the same cache entry run one at a time, even from separate Testkit processes, and Testkit prunes old inactive cache entries automatically.

For most repos, prefer declarative step objects directly inside database.postgres({ template: ... }) and runtime.prepare.steps. The supported shapes are:

• { kind: "command", run: "..." }
• { kind: "sql-file", path: "..." }
• { kind: "module", target: "file.ts#exportName" }

Local files and ordinary external packages remain valid module targets. An unsupported or private @gfdelarue/testkit target fails when the config loads, before preparation or database setup starts.

runtime.toolchain is the first-class way to make those prepare/start commands run under the correct Node toolchain instead of whatever node/npm happened to launch testkit. Node toolchains support:

• host verification mode: install: "require-host"
• cached repo-local provisioning mode: install: "download"
• auto-detection from:
  • package.json#volta.node
  • .nvmrc
  • .node-version
  • .tool-versions (nodejs)
  • package.json#engines.node
  • package.json#volta.npm
  • package.json#packageManager
  • package.json#engines.npm

install: "download" supports the Node versions and platforms Testkit lists, and fails before downloading for any other. Downloads are cached under .testkit/_toolchains/node, and a modified cache entry fails instead of running. runtime.toolchain.cwd is a product-relative path and cannot contain a parent-directory segment. The official Linux Node archives are glibc builds, so install: "download" fails on musl before it downloads anything.

Example:

toolchains: {
  frontendNode: toolchain.node({
    cwd: "frontend",
    detect: "auto",
    install: "download",
  }),
},
services: {
  frontend: app.next({
    cwd: "frontend",
    runtime: {
      toolchain: "frontendNode",
    },
  }),
}

Every terminal run is sealed under .testkit/results/runs/<run-id>/ with a canonical run.json, provenance.json, manifest.json, and the run's exact artifact/log/setup files. Payloads are SHA-256 addressed under .testkit/results/objects/sha256, so repeated bytes are deduplicated without removing their logical per-run paths. .testkit/results/latest.json remains a convenience projection; it is not the historical authority.

run.json is schema 20. groups holds each group's suites and files. environment is the run's environment record. For each acquisition, it holds:

• the source that prepared it: configured, static, supplied, or none;
• the tasks it served and the services it started;
• each file's setup;
• how it was released;
• the values its tests received.

It also holds each service's start attempts, the service and setup logs, the run's capacity and database backend, and the values tests asked for that the environment lacked. Runs that Testkit 0.4.x stored as schema 19 stay readable through testkit artifacts, ci report, known-failure checks, and the workspace API.

Sealing replaces secrets with <redacted> in the run record and in every text file of the bundle. A key names a secret when it holds PASSWORD, TOKEN, SECRET, PRIVATE_KEY, AUTH, COOKIE, SESSION, or API_KEY in any letter case. Sealing redacts:

• A KEY=value assignment under such a key, an Authorization, Proxy-Authorization, Cookie, or Set-Cookie header value, a JSON string value under such a key, even when the key is spelled with escapes, a single-quoted 'password': '…' value, and a URL's password, wherever they appear. A quoted value ends at its closing quote and keeps its quotes, and a value that starts with an auth scheme such as Bearer ends with its line. A secret in one JSON value leaves the record's other values as they are.
• The value of each such key, and of each key that redact names, in the environment Testkit runs in. When the value is a URL, its password counts too.
• The values the run's environment resolved: each value under such a key or a key that redact names, and the password of each URL. A PostgreSQL URL's password may also be a password query parameter, as libpq reads it. This covers config env, env files, local.env, dependency services, base, readiness, UI, and database URLs, --env, --env-from, and an --environment-file. A value under such a key that is a URL counts only for its password.
• The unguessable segment in each HTTP capture's operation URLs, from a setup module's capture or an --environment-file handle.

Name a secret's key in redact when it holds none of the words above:

export default defineConfig({
  redact: ["STRIPE_SK", "PGPASS"],
});

Testkit matches the names in any letter case, in a configured run and a supplied run alike. redact adds keys whose values sealing redacts; the forms in the first item still use the words above.

A value shorter than eight characters is not redacted on its own. Sealing also redacts a value's JSON-escaped forms, each line of a value that spans lines, and keys in the free-form data and metadata that tests attach. When output cuts a secret short and marks the cut with … or ..., sealing redacts the part that shows. testkit run --json and the run's progress output apply the same redaction.

Some secrets stay visible:

• What a setup module prints goes to the terminal as written.
• Sealing does not look inside compressed or binary files, such as a Playwright trace.zip. It refuses to seal a binary file that holds one of the values above in plain bytes.
• When a run's process is killed before it seals the run, testkit stop, testkit cleanup, or the next run seals it from another process. That process does not have the values the run's environment resolved or the redact list, so it redacts only the first item and its own environment's values under the words above. The bundle's archive.redaction.status is then source-targets-unavailable instead of applied.

Use the artifact archive config to opt into pruning or atomic publication to an external directory. Local retention is indefinite by default:

export default defineConfig({
  artifacts: {
    retention: {
      maxRuns: 50,
      maxAgeDays: 30,
      preserveSuites: ["ui:performance"],
    },
    publication: {
      directory: "/var/lib/testkit-run-archive",
      required: true,
    },
  },
  services: {
    // ...
  },
});

Retention decisions and publication receipts are recorded under .testkit/results/retention and .testkit/results/publications. A required publication failure fails the command but leaves the verified local bundle in place. Discover and verify retained evidence with:

testkit artifacts --list --json
testkit artifacts --run <run-id> --verify --json
testkit artifacts --run <run-id> path/to/test.testkit.ts --json

The default artifact list shows names, kinds, summaries, and paths without reading payload files. Select a file with --format text to read its contents, or use --json for structured output including the selected payloads.

Programmatic consumers can import listRunBundles, loadRunBundleArtifact, loadRunBundleManifest, and verifyRunBundle from @gfdelarue/testkit/workspace.

Authoring

Named endpoints and databases

@gfdelarue/testkit/environment gives a test the URL and database of each service its group uses, by the service's name:

import { database, endpoint } from "@gfdelarue/testkit/environment";

const response = await fetch(new URL("/health", endpoint("api")));
const reportsDatabase = database("reports");

Both functions return strings. A service's database is the one Testkit provisions for it, or the DATABASE_URL its env declares. The module works in Vitest and Playwright files and in the modules they import. Testkit installs the values before any of that code loads, so module-level code and native hooks can read them, such as a Playwright config and its globalSetup or a Vitest beforeAll.

A name with no value throws EnvironmentValueMissingError, with kind, valueName, and the available names. valueName is null when the missing value is the run's default URL. Outside a Testkit run every call throws it.

A test that fails on this error, or a file whose imports throw it, ends as an infrastructure failure in Vitest and Playwright. The run fails, and testkit bug file refuses to file it. A test that catches the error and carries on is unaffected. When the run gives a test no base URL, a Playwright page.goto() to a relative URL ends the same way.

BASE_URL and DATABASE_URL hold the values of the first service in uses that has each one. Playwright gives BASE_URL and baseURL in canonical form, so an origin ends with /. endpoint() returns the URL unchanged in every engine.

The group's default endpoint and database are also available by the group's name, whether the run starts its services or takes supplied values. For a group checks that uses api, endpoint("checks") and endpoint("api") return the same URL. A group whose default differs from a used service's value with the group's name, such as a group api with uses: ["web", "api"], gets no values: its files end as an infrastructure failure that names the group, and other groups still run.

Handles

handle(name) from @gfdelarue/testkit/environment gives a test an object for something its environment prepared. Today that is an HTTP capture, which a setup module starts with sidecars.httpCapture(). The handle is named after the capture's id:

// tests/webhook.test.setup.mjs
export async function setup({ sidecars }) {
  await sidecars.httpCapture({ id: "webhook", path: "/webhook" });
}

// tests/webhook.test.ts
import { handle } from "@gfdelarue/testkit/environment";

const webhook = handle("webhook");
await placeOrder({ callbackUrl: webhook.url });
const [delivery] = await webhook.requests();
await webhook.reset();

A capture handle has kind ("http-capture"), url (where the product sends its requests), requests(), which returns the captured requests oldest first, and reset(), which forgets them. Both methods return a Promise in Vitest and Playwright. Each call fails after 10 seconds if the capture hasn't answered in full.

A handle is a reference, not a connection. It can list or reset its own capture and nothing else. The capture's operation URLs hold an unguessable segment, so holding the handle is the authority. Testkit keeps that segment out of the run's results, artifacts, and events, and the capture refuses any other operation. When Testkit releases the file's environment, the capture closes and the handle expires. Calling a handle never starts a sidecar or a service.

• A name with no handle throws EnvironmentValueMissingError with kind "handle", and the test ends as an infrastructure failure, as for a missing endpoint.
• Every capture created without an id gets the id http-capture. If two do, handle("http-capture") throws EnvironmentHandleAmbiguousError with the count. Give each capture a distinct id.
• A call that the capture refuses, such as one from a stale or forged reference, throws EnvironmentHandleError with reason "refused". A call whose capture no longer answers throws it with "unavailable", and a descriptor this Testkit cannot use throws it with "invalid" or "unsupported". These end the test as ordinary failures.

Give each test its own data

Testkit runs test files at the same time. Files on one runtime share its services and its database. Write each test so that it passes beside any other test, in any order.

• Create the rows the test needs, and filter every query and update to them. Don't depend on rows that another test creates, and don't count, update, or delete across a whole table. Every test sees the rows from template.seed, so read them and leave them unchanged.
• When the test ends, leave its rows in place. Testkit resets the databases it provisions. By default, every run starts from a fresh clone of the template database. See database.reset.
• Clean up only what Testkit doesn't own, such as an account in a third-party sandbox. Register the cleanup as soon as the test creates the account. Give the account a unique name, so that a leftover from a cleanup that never ran can't collide with the next run.

import { randomUUID } from "node:crypto";
import { endpoint } from "@gfdelarue/testkit/environment";
import { expect, onTestFinished, test } from "vitest";

import { sandbox } from "../payments-sandbox.js";

test("a refund reaches the sandbox customer", async () => {
  const customer = await sandbox.createCustomer({ email: `refund-${randomUUID()}@example.test` });
  onTestFinished(() => sandbox.deleteCustomer(customer.id));

  const response = await fetch(new URL("/api/refunds", endpoint("api")), {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ customerId: customer.id, amountCents: 500 }),
  });
  expect(response.status).toBe(201);
});

A Node setup module returns its cleanup with its data. Testkit runs only the cleanup that setup returns. If a step in setup can throw after setup creates something, undo the creation before rethrowing.

A test that has to count or change a whole table needs a database that other tests don't write to. Give the test a group of its own, and have the group use a service with its own database. Below, the ledger group owns the test files under src/ledger, and its tests get the database of ledger-db, built from the same schema:

services: {
  api: app.node({
    // ...
  }),
  "ledger-db": database.fixture({
    template: { migrate: [{ kind: "sql-file", path: "db/schema.sql" }] },
  }),
},
groups: {
  api: { roots: ["src"], uses: ["api"] },
  ledger: { roots: ["src/ledger"], uses: ["ledger-db"] },
},

locks is an escape hatch for a resource that can't be duplicated, such as a payment provider's sandbox account with one webhook URL that some tests change. Files that name the same lock never run at the same time within one run. Testkit doesn't share locks between runs, such as two CI jobs. Two tests that need a lock because they share rows need their own rows instead.

export const testkit = defineFile({ engine: "node-vitest", locks: ["payments-sandbox-webhook"] });

Node setup beside a test

A test such as tests/webhook.test.ts can have a sibling tests/webhook.test.setup.mjs. Testkit runs its named setup export, or its default export, before starting the test engine:

// tests/webhook.test.setup.mjs
export async function setup({ sidecars }) {
  const capture = await sidecars.httpCapture({ path: "/webhook" });
  return {
    data: { webhookUrl: capture.url, requestsUrl: capture.requestsUrl },
  };
}

Setup is part of preparing the file's environment. Testkit runs it once, after it acquires the services the file's group uses and before the test engine starts. Testkit never batches a Playwright file that has setup with other Playwright files, and a plain Vitest file that has setup runs in its own Vitest invocation. Files without setup still batch.

Setup runs with its own JavaScript state. It can use Node APIs, prepare external resources, and start HTTP capture listeners. Tests reach a capture by its id through handle(). A capture's requestsUrl and resetUrl hold an unguessable segment that Testkit keeps out of the run's records, and its capture path may not start with /__testkit__/. The context contains deadlineMs, signal, env, leaseDir, productDir, serviceName, testFile, and sidecars. serviceName holds the name of the test's group. Setup has the same permissions as the Testkit process. It is not a sandbox for untrusted code.

A run on supplied values does not run setup modules. Its tests receive the environment file's setupData instead.

Return plain JSON, or { data, cleanup } with an optional cleanup function. Testkit passes the data to the engine as setupData and records it with the setup module path and sidecar handles. Tests read it with setupData() from @gfdelarue/testkit/environment, which returns {} when the file has no setup module and throws outside a Testkit run. Bare null or undefined, and a cleanup-only result, give empty data. { data: null } preserves null. Data must encode to at most 1 MiB of JSON. Invalid data reports its JSON path, such as $.items[1].

Setup shares the task's absolute deadline. On failure or cancellation, Testkit runs registered cleanup and closes its sidecars. Once setup succeeds, those resources remain available until the test and its native hooks finish and Testkit releases the file's environment. Cleanup callbacks run in reverse registration order. The runner uses the task timeout for cleanup, capped at 60 seconds. Testkit stops a setup that hangs past that limit, which closes its listeners.

Thrown setup errors, deadline expiry, and invalid data fail the task with the setup module path. Consumer cleanup errors fail the task; a failure in Testkit's own cleanup, or a setup that stops unexpectedly, counts as an infrastructure failure. Cleanup remains idempotent. HTTP capture retains at most 100 requests by default, configurable up to 1,000. Each body defaults to 1 MiB, configurable up to 8 MiB, and retained bodies total at most 8 MiB per listener. A setup can create up to 16 listeners, each with at most 256 connections.

Discovery tries sibling .setup.mjs, sibling .setup.js, then the matching .mjs and .js files in __setup__, in that order. For the example above, the fallback is tests/__setup__/webhook.test.mjs.

HTTP and persistence checks

Use plain Vitest files for ordinary HTTP, service, and database checks. Read configured URLs with endpoint(name) and database(name) from @gfdelarue/testkit/environment, then use fetch or your application's database client.

Record data-flow evidence

dataFlow(context) from @gfdelarue/testkit/node-vitest/data records what a plain Vitest test sees data do: metrics, counts, bytes, durations, money, ratios, sequences, records, actors, systems, transitions, events, findings, external calls, and cache, LLM, and RPC usage. Budgets check the recorded values, for example that a metric stays under a limit or that a sequence was recorded.

import { endpoint } from "@gfdelarue/testkit/environment";
import { dataFlow } from "@gfdelarue/testkit/node-vitest/data";
import { test } from "vitest";

test("lists one page of orders", async (context) => {
  const flow = dataFlow(context);
  const response = await fetch(`${endpoint("api")}/orders?page=1`);
  const body = await response.text();
  flow.recordHttpRequest({
    method: "GET",
    path: "/orders",
    status: response.status,
    responseBytes: Buffer.byteLength(body),
  });
  flow.expectMetric("http.response.bytes").atMost(64_000);
});

• dataFlow needs the context of a running test. Every call in one attempt returns the same recorder, and a call without that context throws.
• The library reads no URL, environment value, fixture file, or Testkit config. Read named URLs from @gfdelarue/testkit/environment, log in from test code, and read files with fs.
• A failed budget fails its test when the test finishes. The failure detail has the kind data-flow-budget and a key made of the test's full title and the budget.
• Testkit writes data-flow.json and data-flow.md for a file whose tests recorded something. Each such test is one scenario, from its final attempt. A test that records nothing adds nothing, and a file whose tests record nothing gets no data-flow artifacts.

Discovery

Testkit discovers plain Vitest test files and typed suites.

Plain Vitest test files

Testkit runs plain Vitest test files with the repository's own Vitest and its config, as vitest run does from the product directory. It loads the root vitest.config.* or vite.config.* the way Vitest does and lists the files that config's projects select, without running global setup. Nested configs and package scripts have no effect. Without a config, Vitest's defaults select files that match **/*.{test,spec}.?(c|m)[jt]s?(x), outside node_modules and .git.

discover, run, typecheck, bug record, and bug file take three flags that change the selection:

• --vitest-config <path> uses that config, relative to the product directory. A missing file, or one outside the product directory, fails before any test runs.
• --vitest-project <name> runs only that project. An unknown name fails and lists the projects.
• --vitest-runner managed runs plain files with Testkit's own Vitest and its default rules, without the repository's config.

Testkit supports Vitest 4.x. Another major, a broken installation, a root Vitest config that fails to load, or a Vitest that package.json declares but that is not installed fails before any test runs, naming the version and path. --vitest-runner managed avoids the repository's Vitest. A run that selects only typed files, with --type ui or with typed --file paths, bug record or bug file for a typed test, and the commands that select no tests (status, run status, doctor, destroy, cleanup, and db schema) don't load the repository's Vitest, so they still work. A product that neither installs nor declares Vitest has no plain test files.

Plain files run in one Vitest invocation per environment their groups give them. Groups that give their files the same values share one, such as root and the other groups that use no services. A group that uses a service Testkit starts gets an invocation of its own. Inside each invocation, root global setup, teardown, and the repository's reporters run as they do under vitest run, and Testkit's reporter runs beside them, so a run calls them once for each invocation. An invocation's deadline is the sum of its files' timeouts; Vitest's own test and hook timeouts apply inside it. A file that two projects select runs in both, its result combines both runs, and its evidence names each run's project. A file with a setup module, locks, or isolation other than shared runs in its own Vitest invocation with its own environment. Invocations run one after another unless --task-concurrency or execution.taskConcurrency is above 1. To run one group's files alone, select it with --group.

Plain files see the host environment Testkit started with: HOME, PATH, temporary and XDG directories, locale, CI, FORCE_COLOR, NO_COLOR, and any other variable reach config loading, setup files, and the tests as they do under vitest run. Unlike vitest run, ambient BASE_URL and DATABASE_URL never reach them; those come only from the group's environment, from provisioned services or supplied values. Testkit loads no env files for plain files, and Vitest's own env handling and test.env work as usual. Testkit puts its own --import in front of the shell's NODE_OPTIONS. Preloads in it see the shell's environment and the group's values; as under vitest run, Vitest sets TEST, VITEST, and NODE_ENV after them. --vitest-runner managed leaves the shell's NODE_OPTIONS out.

When Vitest detects a coding agent from the environment, such as CLAUDECODE, AI_AGENT, or CURSOR_AGENT, and the repository's Vitest config names no reporter, Vitest uses its agent reporter. That reporter prints no console output from a passing test, so the output is missing from the file's node-vitest-stdout artifact. A failing test's output still appears. Name a reporter in the Vitest config, such as reporters: ["default"], to keep it.

Two settings narrow the files Vitest selects: discovery.exclude in testkit.config.ts, and a declared group's exclude. Testkit's built-in exclusions for typed files, such as dist, build, and coverage, and discovery.roots do not apply to plain files.

A plain file belongs to the group whose roots contain it, or to the implicit root group, which uses no service. Testkit starts no service for a plain file in root.

The plain files in one directory form a suite named after that directory, relative to the group's root. In root, --suite src selects src/sum.test.ts and src/sum.spec.ts. A file directly in a group's root names its own suite, so sum.test.ts in a group rooted at src is in suite sum. When two suites in a group would share a name, Testkit lengthens the later ones with parent directories, such as api-src, and numbers them if that is not enough. testkit typecheck checks plain TypeScript test files and skips plain JavaScript ones, as tsc does without allowJs. It adds the TypeScript files in each Vitest project's setupFiles to the program, so a test can use the matchers a setup file declares, such as those from @testing-library/jest-dom/vitest.

Typed suites

testkit discovers typed suites from __testkit__/ directories.

Place each __testkit__/ directory as a peer of the thing being tested. The target is the smallest owned directory or package surface that honestly contains the behavior under test. This is a single target-relative rule, not a list of allowed or forbidden locations.

Examples:

• tests for src/api/routes/* live in src/api/routes/__testkit__/*
• tests for src/db/* live in src/db/__testkit__/*
• tests for frontend/src/components/layout/* live in frontend/src/components/layout/__testkit__/*
• tests for package-root behavior such as package.json, root scripts, release shape, or the whole package contract live in __testkit__/*

Do not put tests in a higher-level __testkit__/ merely because it is convenient. For example, a repository-root __testkit__/ is appropriate for repository/package-level behavior, but it is a dumping ground if it contains tests for src/api/routes/*, src/db/*, or a specific frontend component tree.

Do not create child directories inside a __testkit__/ directory. The __testkit__/ directory already names the owner boundary; additional grouping folders create a second, non-production hierarchy that weakens ownership. Use clear flat filenames instead, and colocate another __testkit__/ at a lower production owner when a narrower owner exists.

Concrete layouts:

• frontend/src/components/layout/__testkit__/navigation.ui.testkit.ts

testkit uses the *.ui.testkit.ts suffix automatically.

The suffix never chooses an engine. Each discovered typed file must also contain one root-package defineFile(...) declaration with a supported literal engine. Unsupported pairs fail discovery; for example, a *.ui.testkit.ts file cannot declare engine: "node-vitest".

See docs/test-types.md for the canonical type model.

Test groups own test files by directory. Declare them under groups in testkit.config.ts. Each group lists its repo-relative roots and the services its tests use:

export default defineConfig({
  services: {
    api: app.node({ cwd: "services/api", start: "node dist/server.js" }),
    web: app.next({ cwd: "apps/web", dependsOn: ["api"] }),
  },
  groups: {
    api: { roots: ["services/api"], uses: ["api"] },
    web: { roots: ["apps/web"], uses: ["web", "api"] },
  },
});

• A file belongs to the group with the deepest root that contains it. When two groups have roots at the same depth, discovery reports ambiguous_test.
• Files that no group's roots claim belong to the implicit group root, which uses no service. With no testkit.config.ts, every file belongs to root.
• A run starts the services that the selected files' groups use, with their dependsOn, and nothing else. A test gets BASE_URL, DATABASE_URL, and env values only from the services its group uses. When two of them set the same name, the first in uses wins. Each used service's URL and database are also available by name; see Named endpoints and databases.
• skip, requirements, and runtimeIsolation are group fields. See the config reference.

Services own no test files. A config with services and no groups key fails to load when a service's local.cwd or discovery.roots holds a typed test file, which that service owned before 0.6.0. The error lists the groups entries that keep those files with their services. status, destroy, cleanup, and db schema still run on it. Otherwise every file belongs to root.

Suite names are inferred from the colocated path:

• auth/__testkit__/*.ui.testkit.ts => auth
• routes/__testkit__/checkout.ui.testkit.ts => routes

Discovery is also a first-class CLI/API surface:

• testkit discover
  • human-first compact output with group -> type -> suite -> file hierarchy
• testkit discover --output-mode verbose
  • group roots and uses, explicit paths, IDs, locks, and skip reasons
• testkit discover --json
  • machine-readable output with stable enums, canonical paths, and summary data
• testkit discover --output .testkit/discovery.json
  • writes the same machine-readable JSON document to a file artifact

Compact mode prefers derived human labels such as Agent Configs Auth Gate instead of printing long file paths as the primary row label. Exact paths remain available in verbose and JSON output.

The public API is exported from @gfdelarue/testkit/workspace:

import { discoverTests } from "@gfdelarue/testkit/workspace";

const result = await discoverTests({
  dir: process.cwd(),
  runnableOnly: true,
  diagnostics: "report",
});

JSON and file output are machine-first. Each discovered file carries a stable identifier plus canonical metadata such as:

• id
• path
• service
• suiteName
• testType, absent for a plain Vitest test file
• engineId
• profileId
• contract
• skipped
• skipReason
• locks
• dependsOn
• displayName

Discovery output has no cross-run counts, last outcomes, durations, or trends. Testkit 0.2.1 removed testkit.status.json, .testkit/history/, and discovery's history, files[].history, and summary.latestResults fields. To check one run, read its bundle under .testkit/results/runs/<run-id>/. Testkit ignores an old .testkit/history/ directory and leaves it in place.

Local Databases

@gfdelarue/testkit provisions Docker-managed local Postgres automatically for services that define database: database.postgres(...).

Docker is also the required Postgres client runtime for Testkit-owned database operations. Testkit runs psql and pg_dump inside the configured Postgres client image instead of using host-installed binaries, which keeps local runs consistent across Linux, macOS, and native Windows. Override the client image with TESTKIT_POSTGRES_CLIENT_IMAGE when needed.

Template steps, schema snapshots, and database.steps helpers work the same under Docker Desktop, rootless Docker, a