A/B test sample size calculator
An A/B test needs a sample size fixed before it starts, calculated from your current conversion rate, the smallest improvement worth detecting, and how often you are willing to be wrong. Running until a result looks significant is not a test.
A/B test sample size, step by step
The baseline rate is what the control currently converts at. The minimum detectable effect is the smallest change you would act on — and it is the input that dominates everything else: halving the effect you want to detect roughly quadruples the sample you need. Significance (usually 95%) is how often you accept a false positive; power (usually 80%) is how often you are willing to miss a real effect.
The output is per variant, so a two-sided A/B test needs twice that number of visitors, and a test with three arms needs three times. Dividing by your weekly traffic gives the honest duration, which is usually longer than anyone hoped.
Where this number misleads you
The number assumes you look once, at the end. Checking an in-flight test every morning and stopping when it crosses 95% inflates your false-positive rate well past 5% — the result is that roughly a third of "winning" variants stopped early are noise. Either fix the duration in advance and do not look, or use a sequential test designed for peeking.
Questions people ask
How many visitors do I need for an A/B test?
It depends almost entirely on your baseline conversion rate and the size of the effect you want to detect. A site converting at 3% looking for a 10% relative lift needs roughly 50,000 visitors per variant at 95% significance and 80% power. A 50% lift needs around 2,000.
Can I stop an A/B test early if it looks significant?
Not with a fixed-sample test. Repeatedly checking and stopping at the first significant moment is called peeking, and it turns a 5% false-positive rate into something nearer 30%. Decide the sample size in advance, run to it, then look once — or use a sequential or Bayesian method built for continuous monitoring.
What minimum detectable effect should I use?
Use the smallest improvement that would change your decision, not the smallest you can imagine. If a 2% lift would not justify shipping and maintaining the change, do not pay for the sample size to detect it. Working backwards from the decision usually shortens the test considerably.