A/B Testing Sample Size Calculator
Your current conversion rate.
Relative lift you want to detect (e.g. 20% increase).
90%
95% (Standard)
99%
Confidence level to avoid false positives.
80% (Standard)
90%
Probability of finding an effect if it exists.
Sample Size Per Variation
–
Visitors needed for Control and Variant each
Total Visitors Required: –
What is a Conversion Rate Sample Size Calculator?
Before launching an A/B test, it is crucial to know how many visitors you need to reach statistical validity. This Conversion Rate Sample Size Calculator determines the specific number of users required for each variation of your test to ensure your results are accurate and not a product of random chance.
Running a test with insufficient traffic can lead to "underpowered" tests, where you fail to detect a meaningful improvement even if one exists. Conversely, running a test too long wastes resources. This tool bridges that gap using standard statistical formulas.
Why Sample Size Matters in A/B Testing
In the world of Conversion Rate Optimization (CRO), math is just as important as design. If you stop a test too early, you might see a "false positive"—thinking a change improved conversions when it was actually just random noise. If your sample size is too small, your test lacks "power," meaning you might miss a winning variation.
Key Inputs Explained
- Baseline Conversion Rate: This is the current performance of your control page (e.g., if 5 out of 100 people buy, your rate is 5%).
- Minimum Detectable Effect (MDE): The smallest improvement you care about detecting. For example, if you want to detect at least a 20% increase in conversions, your MDE is 20%. Smaller MDEs require much larger sample sizes.
- Statistical Significance (Confidence): Typically set at 95%, this means there is only a 5% chance that a winning result is a false alarm.
- Statistical Power: Typically set at 80%, this represents the probability that your test will correctly identify a winning variation if the improvement actually exists.
How the Calculation Works
This calculator uses a standard two-tailed hypothesis test formula for proportions. It accounts for the variance in your baseline rate and the expected rate of your variation. The formula balances the risk of Type I errors (false positives) and Type II errors (false negatives).
Formula Note: As the Baseline Conversion Rate gets closer to 50%, the sample size required decreases. As the MDE gets smaller (e.g., trying to find a 1% lift vs a 50% lift), the sample size required increases exponentially.
Best Practices for Interpreting Results
Once you have your required sample size:
- Do not stop early: Even if one variation seems to be winning by a landslide, wait until the calculated sample size is reached to avoid "peeking" errors.
- Account for duration: If the calculator says you need 100,000 visitors, but you only get 1,000 per month, the test will take 100 months. You may need to increase your MDE (look for bigger wins) to make testing feasible.
- Full weeks only: Always run tests for full week cycles (e.g., 2 weeks, 3 weeks) to account for differences in user behavior between weekdays and weekends.