ABTestResult MCP Server

Statistical tools for A/B testing, available as a Model Context Protocol (MCP) server. Powered by abtestresult.com.

Give any AI assistant (Claude, Codex, Cursor, VS Code, etc.) the ability to analyze A/B tests, calculate sample sizes, run Bayesian analysis, and more. All calculations run locally on your machine — no API keys, your test data never leaves your device.

This package ships readable source. It is source-available for local personal use only and is not licensed for commercial use.

Setup

Add to your MCP configuration:

{
  "mcpServers": {
    "abtestresult": {
      "command": "npx",
      "args": ["-y", "abtestresult-mcp"]
    }
  }
}

Where to add this:

• Claude Desktop: ~/Library/Application Support/Claude/claude_desktop_config.json
• Codex: ~/.codex/config.toml
• Claude Code: .mcp.json in your project root
• Cursor: .cursor/mcp.json
• VS Code: .vscode/settings.json (use mcp.servers instead of mcpServers)

That's it. No API keys, no authentication. The server starts automatically when your AI assistant needs it.

Tools

| Tool | Description | |------|-------------| | analyze_ab_test | Z-test for rate metrics (conversion rate, CTR, etc.) | | analyze_ab_test_average | T-test for continuous metrics (revenue, time on page, etc.) | | calculate_sample_size | Required sample size for rate or average metrics | | calculate_mde | Minimum detectable effect given a fixed sample size | | bayesian_ab_test | Bayesian analysis with probability to be best & expected loss | | check_srm | Sample Ratio Mismatch detection | | paired_test | Paired T-test or Wilcoxon signed-rank for before/after data | | survey_sample_size | Cochran's formula with finite population correction |

Shareable result links

Every analysis tool (except paired_test) returns a view_url in its response — a deeplink to abtestresult.com that pre-fills the matching calculator with your inputs and auto-runs it. Open it to inspect the result visually (charts, confidence intervals, recommendations) or share it with your team. No data is stored server-side: the inputs are encoded directly into the URL.

Usage Examples

Once connected, just ask your AI assistant in natural language:

• "My A/B test had 5,000 users per group. Control converted at 5%, variant at 6%. Is it significant?"
• "How many users do I need to detect a 10% lift on a 3% conversion rate?"
• "Run a Bayesian analysis on my test: control 10,000 users / 500 conversions, variant 10,000 users / 580 conversions"
• "Check if my traffic split is valid: control has 10,234 users, variant has 9,766"
• "What's the smallest effect I can detect with 50,000 total users and a 5% baseline?"

Privacy

Your test data never leaves your machine — all calculations run locally. We collect anonymous usage telemetry (tool name and client type only) to monitor performance and guide development. No test inputs, results, or user identifiers are tracked.

Learn More

• Full documentation — setup guides, tool reference, and rate limits
• Web calculators — use the same tools in your browser

License

Copyright (c) 2026 ABTestResult. All rights reserved.

This package is source-available, not open source.

You may use and modify it only for your own local personal, non-commercial use. Any commercial use, business use, client use, hosted use, redistribution, or money-making activity requires a separate written license from ABTestResult: [email protected].

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.