ab-test-sample-size

Zero-dependency A/B test sample-size, test-duration, and minimum-detectable-effect calculations using the two-proportion z-test. Written in TypeScript, runs anywhere (Node, Deno, Bun, browsers).

This is the same calculation engine that powers the free A/B test traffic & sample-size estimator at optipilot.com — extracted so you can run the numbers in your own code, CI checks, or experiment-planning dashboards.

Why

Most "is my A/B test big enough?" bugs come from eyeballing significance instead of computing the sample size up front. This library gives you the one number that matters — how many visitors per variation you need — plus how long that will take at your traffic level, with no dependencies and a formula you can audit in ~30 lines.

Install

npm install ab-test-sample-size

Usage

import { calculateSampleSize, calculateFullEstimate } from 'ab-test-sample-size'

// How many visitors per variation do I need?
calculateSampleSize({
  baselineRate: 0.05, // current conversion rate (5%)
  mdeRelative: 0.1,   // smallest relative lift worth detecting (10%)
  significance: 95,   // confidence level (90 | 95 | 99)
  power: 80,          // statistical power (70 | 80 | 90)
  variations: 2,      // control + variant(s)
})
// => 31240

// Full plan, including how long the test will run at 1,000 visitors/day:
calculateFullEstimate(
  { baselineRate: 0.05, mdeRelative: 0.1, significance: 95, power: 80, variations: 2 },
  1000,
)
// => { sampleSizePerVariation: 31240, totalSampleSize: 62480, estimatedDays: 63 }

API

calculateSampleSize(input): number

Required visitors per variation, rounded up. Returns 0 for degenerate inputs (rates outside (0, 1) or a zero effect size).

calculateFullEstimate(input, dailyTraffic?): SampleSizeResult

Adds the total across all variations and — when dailyTraffic is supplied — an estimatedDays duration.

input: SampleSizeInput

| Field | Type | Meaning | | --- | --- | --- | | baselineRate | number | Current conversion rate as a decimal (0.05 = 5%). | | mdeRelative | number | Minimum detectable effect as a relative lift (0.1 = 10% relative). | | significance | number | Confidence level: 90, 95, or 99. | | power | number | Statistical power: 70, 80, or 90. | | variations | number | Number of variations including control. |

Unrecognised significance / power values fall back to 95 / 80. The z-score lookup tables are exported as Z_ALPHA and Z_BETA if you need them.

The formula

It's a standard two-proportion z-test:

n = ((z_α + z_β)² · (p₁(1 − p₁) + p₂(1 − p₂))) / (p₂ − p₁)²

where p₁ is the baseline rate, p₂ = p₁ · (1 + mdeRelative) is the expected rate under the alternative hypothesis, z_α is the two-tailed z-score for the significance level, and z_β is the z-score for the power level.

Testing

npm test

Zero runtime dependencies; the test suite uses Node's built-in test runner.

Related

• A/B test traffic & sample-size estimator — the interactive version of this library.
• Experiment QA checklist — 27-item pre-launch checklist.
• optipilot.com — guides for running trustworthy experiments with Optimizely.

License

MIT © optipilot.com