What this calculator does
This sample size calculator tells you how many visitors each variant of your A/B test needs before you start. You give it four numbers — your baseline conversion rate, the minimum detectable effect (MDE) you care about, your significance level, and your desired statistical power — and it returns the required sample size per variant, with a chart showing how that requirement explodes as the MDE shrinks.
Sizing the test up front is the single highest-leverage habit in experimentation: it prevents both the underpowered test that wastes four weeks to conclude nothing, and the endless test that gets stopped the day it flickers significant.
The formula
For a two-proportion test comparing baseline rate against target rate (baseline + MDE), the required sample size per variant is:
where is the average rate, is the normal quantile for your significance level (1.96 at 95%, two-tailed), and is the quantile for your power (0.84 at 80%). The denominator is the squared effect — which is why halving the MDE roughly quadruples the required sample.
With more than one variant, is divided by the number of variants before the quantile is taken (a Bonferroni adjustment). Each variant is compared against control, so this keeps the plan consistent with the multiple-comparison correction applied when the test is analyzed.
A worked example
Baseline conversion rate 10%, and you want to detect a 5% relative lift (10% → 10.5%) at 95% significance and 80% power, two-tailed. Plugging in: , , . The formula gives 57,763 visitors per variant, or about 115,500 total. If your site gets 20,000 visitors a week into the test, that is a six-week runtime — knowledge worth having before you build the variant, not after. (Check the MDE & duration calculator to see what is detectable at your actual traffic.)
How to choose each input
- Baseline rate: pull the last 2–4 weeks of the exact metric and audience your test will use. A stale or mismeasured baseline skews everything.
- MDE: work backwards from business value — the smallest lift that pays for the engineering time. Anything smaller than that is not worth detecting.
- Significance: 95% is standard; use 99% for risky, hard-to-reverse changes.
- Power: 80% is standard; 90% for decisions where missing a real winner is costly.
Common mistakes
- Stopping early at the first significant reading. The sample size is the contract that makes your error rates real. Breaking it by peeking voids the warranty — use the sequential testing calculator if you need early stopping.
- Relative/absolute MDE confusion. A "5% lift" on a 10% baseline could mean 10.5% or 15% depending on interpretation: 57,763 visitors per variant versus 686, a difference of about 84x. This calculator labels both explicitly.
- Sizing for the hoped-for effect instead of the minimum worthwhile effect. Optimism makes tests small and inconclusive.
- Ignoring extra variants. An A/B/C test needs the quoted n in each of the three groups, and a stricter significance level for each comparison. Set the number of variants and the calculator accounts for both.
- Not running full weeks. Round your runtime up to whole weeks so weekday/weekend mix does not bias the sample.