@drak-marketing/mcp-test-harness

Shared test harness for MCP server repos. One call registers 40+ parameterized tests covering build integrity, smoke, protocol conformance, credential security, resilience, and adversarial inputs.

Install

npm install --save-dev @drak-marketing/mcp-test-harness vitest

vitest is a peer dependency — your repo supplies its own version.

Usage

Add a single test file (e.g. src/harness.test.ts) to your MCP server repo:

import { registerMcpTests } from "@drak-marketing/mcp-test-harness";
import { fileURLToPath } from "url";
import path from "path";

const __dirname = path.dirname(fileURLToPath(import.meta.url));

registerMcpTests({
  name: "mcp-example",
  repoRoot: path.resolve(__dirname, ".."),
  toolPrefix: "example_",
  minTools: 10,
  requiredTools: ["example_get_context", "example_list_items"],
  binEntries: { "mcp-example": "dist/index.js" },
  hasAuthCli: false,
  hasCredentials: true,
  hasResilience: true,
  hasPlatform: true,
  requiredEnvVars: ["EXAMPLE_API_KEY"],
  envPrefix: "EXAMPLE_",
  startupEnv: { EXAMPLE_API_KEY: "fake-key" },
});

Run with your normal vitest command:

npm test

What you get

Suites are selected based on the capabilities you declare in the config:

• Always on: build integrity, smoke, cross-platform, MCP protocol, dependency health, version/publish checks, observability, input sanitization, output contract, concurrency, adversarial (Ralph/Ender/Mayhem)
• hasCredentials: true → credential security, token lifecycle, backward-compat
• hasResilience: true → network failure scenarios
• requiredEnvVars.length > 0 → config loading
• startupEnv provided → graceful shutdown

See src/types.ts for the full config shape.

Publishing

Publishing happens in CI via npm Trusted Publishing (OIDC). There is no npm token to create, store, or rotate — .github/workflows/publish.yml authenticates with a short-lived credential npm issues in exchange for the workflow's OIDC token, matched against the Trusted Publisher registered for this package on npmjs.com (GitHub Actions; owner mharnett, repo mcp-test-harness, workflow publish.yml).

To publish a new version:

1. Bump version in package.json and merge that to main.

2. Cut a GitHub Release on the matching tag — that fires publish.yml:

   gh release create v0.2.2 --repo mharnett/mcp-test-harness --generate-notes

3. Watch the run: gh run list --repo mharnett/mcp-test-harness --workflow publish.yml

If a Release is created by automation rather than by hand, GitHub suppresses the downstream release trigger and the publish silently never runs. Use the manual fallback in that case:

gh workflow run publish.yml --repo mharnett/mcp-test-harness -f ref=v0.2.2

Two pins in publish.yml are load-bearing and documented inline — npm@11 (Trusted Publishing needs npm ≥ 11.5, but npm@12 drops Node 20), and the deliberate absence of --provenance (attestation requires a public source repo; this one is private). Read the comments before changing either.

Do not add NPM_TOKEN or NODE_AUTH_TOKEN to this repo. If publishing fails, the OIDC trust is what needs fixing.

License

MIT