mcp-contract-tester

mcp-contract-tester is a black-box contract test harness for MCP servers. It validates advertised tools, probes invalid inputs, records error-shape stability, and emits deterministic reports.

Features

• Validates duplicate names, missing schemas, and malformed metadata
• Runs happy-path assertions against expected output fragments
• Fuzzes tool inputs by deleting required keys, widening primitive types, and injecting junk fields
• Produces JSON and Markdown summaries for CI or human review

Usage

import { runContractSuite, createMarkdownReport } from "mcp-contract-tester";

const adapter = {
  async listTools() {
    return [
      {
        name: "sum",
        title: "Sum",
        description: "Add two numbers",
        inputSchema: {
          type: "object",
          required: ["a", "b"],
          properties: {
            a: { type: "number" },
            b: { type: "number" }
          }
        }
      }
    ];
  },
  async callTool(name, args) {
    if (name !== "sum") {
      throw new Error("unknown tool");
    }
    if (typeof args.a !== "number" || typeof args.b !== "number") {
      throw new Error("invalid args");
    }
    return { content: [{ type: "text", text: String(args.a + args.b) }] };
  }
};

const suite = await runContractSuite(adapter, {
  tools: {
    sum: {
      sampleArgs: { a: 2, b: 3 },
      expectContentTextIncludes: ["5"]
    }
  }
});

console.log(createMarkdownReport(suite));