mutant-mcp

Mutation test-gap finder as an MCP server — "would my suite even notice if this broke?"

A green suite tells you the tests pass. It doesn't tell you the tests would catch a bug. mutant systematically breaks your changed code — one small mutation at a time — reruns the suite, and reports the survivors: lines a test executes but no assertion actually checks. Those are your real test gaps.

find_gaps
→ 32 mutations · 28 run on covered lines · killed 17 · survived 11 · caught 61%

  SURVIVED — a test executes these lines but nothing catches the change (real gaps):
    shard/receipt.py:131  [Lt->LtE@1]  0 <= lo < hi <= layer_count → 0 <= lo <= hi <= layer_count
    shard/receipt.py:141  [NotEq->Eq]  lo != cursor → lo == cursor
    …
  → add/extend an assertion that would fail under each change above

(That first survivor is a real one, found live on leyten/shard: weakening lo < hi to lo <= hi lets a zero-length receipt block into a paid tiling, and the whole suite stayed green.)

How it works

1. Green + coverage pre-pass. Runs the suite once under coverage.py. If it isn't green, mutation testing is meaningless — mutant aborts and says so. The pass also records which lines each test executes.
2. Mutate. For each changed .py file, generates surgical single-node mutations via the ast module: comparison flips (<↔<=, ==↔!=, including each position of a chained compare like 0 <= lo < hi), off-by-one boundary swaps, arithmetic (+↔-, *↔/), and↔or, return X → return None, and constant tweaks. Every mutation is re-parsed before use, so a mutant never fails to compile.
3. Split by coverage. Mutations on lines no test executes survive trivially — they're reported as "uncovered" for free, without a run. Only mutations on covered lines get the expensive per-mutant suite run.
4. Run & classify. Each covered mutant is applied in place, the suite runs, the file is restored: suite fails → KILLED (good), suite passes → SURVIVED (a gap).

Files are mutated in place and restored from an in-memory snapshot in a finally, so an interrupted run never leaves a mutated tree — but don't run find_gaps concurrently with edits to the same files.

Tools

| tool | purpose | | --- | --- | | find_gaps | mutate changed (or named) files, run the suite per mutant, report survivors | | list_mutations | dry run: preview the mutations that would be generated, no test runs |

find_gaps defaults to the files changed vs HEAD; pass files=[...] to target specific modules, base=<ref> to change the diff base, cmd=... to set the test command, and max_mutants to bound the per-mutant runs (default 30).

Install

// .mcp.json
{
  "mcpServers": {
    "mutant": {
      "command": "npx",
      "args": ["-y", "@ahmedshaikh/mutant-mcp"]
    }
  }
}

Needs coverage.py in the interpreter under test (pip install coverage). Interpreter auto-detected (python3/python, override with MUTANT_PYTHON); project root defaults to the server cwd (MUTANT_ROOT or per-call cwd).

Notes & limits

• Python only in v1. JS/TS mutation needs a real parser to avoid garbage mutants; deferred rather than done badly.
• Cost is O(covered mutations × suite time). Scope cmd to the relevant tests and use max_mutants on big changes. A narrow cmd also narrows what "caught" means — a survivor may just mean "this line is covered by a different test file you didn't run."
• Mutations on unexecuted lines are reported as uncovered, not run — they mark code your tests never reach at all (a different, coarser kind of gap).
• Requires a green baseline; a red suite aborts the run.

Development

npm install
npm test        # node:test; needs python3 + coverage.py (set MUTANT_PYTHON to override)
npm run build

MIT