MCP Test Kit

CLI testing framework for MCP (Model Context Protocol) servers - validate, benchmark, and security-check your MCP implementations

Why MCP Test Kit?

The MCP ecosystem is growing rapidly, but 88% of MCP servers have security gaps (Source: Astrix Security). MCP Test Kit helps you:

• Validate your MCP server works correctly before deployment
• Discover tools, resources, and prompts exposed by your server
• Security audit for common vulnerabilities and misconfigurations
• Benchmark connection and tool execution performance
• CI/CD integration with JSON output for automated testing

Installation

# Global installation
npm install -g mcp-test-kit

# Or use npx
npx mcp-test-kit test "node ./my-server.js"

Quick Start

# Run all tests on an MCP server
mcp-test test "node ./dist/server.js"

# Quick connection check
mcp-test quick "npx @modelcontextprotocol/server-filesystem ."

# List available tools
mcp-test list "python server.py"

# Security assessment
mcp-test security "node ./my-server.js" --verbose

Commands

mcp-test test <server>

Run full test suite on an MCP server.

mcp-test test "node server.js" [options]

Options:
  -a, --args <args...>    Arguments to pass to server
  -e, --env <vars...>     Environment variables (KEY=VALUE)
  -t, --timeout <ms>      Connection timeout (default: 10000)
  -v, --verbose           Enable verbose output
  --connection-only       Only run connection tests
  --protocol-only         Only run protocol tests
  --tools-only            Only run tool validation
  --security-only         Only run security checks
  -f, --format <type>     Output: console, json, markdown

Example:

# Test with environment variables
mcp-test test "node server.js" -e "API_KEY=xxx" -e "DEBUG=true"

# Output as JSON for CI
mcp-test test "node server.js" -f json > results.json

# Run only security checks
mcp-test test "node server.js" --security-only

mcp-test quick <server>

Quick connection check - verify server is reachable.

mcp-test quick "node server.js"

Output:

✅ Server is reachable
   Name: my-mcp-server
   Version: 1.0.0
   Protocol: 2024-11-05

mcp-test list <server>

List all tools exposed by the server.

mcp-test list "node server.js"

# Output as JSON
mcp-test list "node server.js" --json

mcp-test security <server>

Run security assessment and get a score.

mcp-test security "node server.js" --verbose

Output:

Security Assessment Results
══════════════════════════════════════════════════

Score: [████████████████████░░░░░░░░░░░░░░░░░░░] 80%

🚨 Critical Issues (0):

⚠️  High Severity (1):
   • Tool "execute_query" input patterns: Properties may need validation
     → Add pattern restrictions, enums, or length limits

✅ Passed Checks (5):
   • Tool "get_weather" naming
   • Tool "search_docs" naming
   ...

Programmatic Usage

import { MCPTestRunner, formatReport } from 'mcp-test-kit';

const runner = new MCPTestRunner({
  server: 'node ./dist/server.js',
  timeout: 10000
}, {
  connection: true,
  protocol: true,
  tools: true,
  security: true
});

const report = await runner.run();

console.log(formatReport(report, 'console'));

// Access individual results
for (const suite of report.suites) {
  console.log(`${suite.name}: ${suite.summary.passed}/${suite.summary.total} passed`);
}

What Gets Tested

Connection Tests

• Server startup and initialization
• MCP protocol handshake
• Server info (name, version, protocol version)

Protocol Compliance

• Ping response (optional)
• Tools list endpoint
• Resources list endpoint
• Prompts list endpoint

Tool Validation

• Tool schema completeness
• Required fields validation
• Input schema JSON Schema compliance
• Error handling for invalid arguments

Security Checks

• Dangerous naming patterns (exec, shell, sql, file access)
• Input validation presence
• Attack surface assessment (tool count)
• Error information leakage
• Protocol version currency

CI/CD Integration

GitHub Actions

name: MCP Server Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - run: npm install
      - run: npm run build

      - name: Test MCP Server
        run: npx mcp-test-kit test "node ./dist/server.js" -f json > results.json

      - name: Check Results
        run: |
          if grep -q '"overallStatus": "fail"' results.json; then
            echo "MCP tests failed!"
            cat results.json
            exit 1
          fi

Security Scoring

| Score | Rating | Description | |-------|--------|-------------| | 80-100% | ✅ PASS | Server follows security best practices | | 50-79% | ⚠️ WARN | Some security concerns need attention | | 0-49% | ❌ FAIL | Critical security issues detected |

What We Check

1. Tool Naming - Flags dangerous patterns like exec, shell, sql, eval
2. Input Schemas - Ensures tools have proper input validation
3. Attack Surface - Warns if too many tools increase risk
4. Error Handling - Checks for information leakage in errors
5. Protocol Version - Ensures server uses current MCP protocol

Contributing

Contributions welcome! Please read our contributing guidelines first.

# Clone and install
git clone https://github.com/cybeleri/mcp-test-kit
cd mcp-test-kit
npm install

# Run tests
npm test

# Build
npm run build

License

MIT © cybeleri

Built for the MCP community - Help make MCP servers safer and more reliable.