A/B Test Sample Size Calculator
Calculate the minimum sample size needed for statistically significant A/B test results. Plan your experiments with confidence.
Your current (control) conversion rate
The variant conversion rate you want to detect
Use when testing if variant is different (better or worse)
Including control (A/B = 2)
Required Sample Size
To detect a 20.0% relative lift
Sample Size by Effect Size
Compare sample requirements for different minimum detectable effects at 5% baseline
| MDE | Target Rate | Per Variant | Total |
|---|---|---|---|
| +5% | 5.25% | 121,987 | 243,974 |
| +10% | 5.50% | 31,200 | 62,400 |
| +15% | 5.75% | 14,178 | 28,356 |
| +20% | 6.00% | 8,150 | 16,300 |
| +25% | 6.25% | 5,328 | 10,656 |
| +30% | 6.50% | 3,777 | 7,554 |
Where Z_alpha = 1.96 (95% significance), Z_beta = 0.84 (80% power), and p_pooled = average of baseline and expected rates.
Understanding A/B Test Sample Sizes
Sample size is the foundation of statistical validity in A/B testing. Running a test without enough visitors leads to unreliable results - either missing real improvements or seeing false positives.
Factors Affecting Sample Size
Baseline Conversion Rate
Higher baseline rates require smaller samples. A page converting at 10% needs fewer visitors than one at 1%.
Minimum Detectable Effect
Smaller effects need larger samples. Detecting a 5% lift requires more data than detecting 50%.
Statistical Significance
Higher confidence requires more data. 99% significance needs more visitors than 90%.
Statistical Power
Higher power (chance of detecting a real effect) requires larger samples.
Sample Size Rules of Thumb
- Minimum conversions: You need at least 100 conversions per variant for reliable results.
- Effect size matters most: The difference between rates has the biggest impact on required sample.
- 95% + 80% is standard: Most teams use 95% significance and 80% power.
- More variants = more visitors: Each additional variant increases total sample needed.
- Pre-calculate, don't guess: Always calculate before starting a test.
Common Sample Size Mistakes
- Using pageviews instead of visitors: Sample size is unique visitors, not page views.
- Ignoring return visitors: The same person visiting twice is still one sample.
- Testing too many variants: Each variant dilutes your sample. Start with A/B.
- Expecting small effects: If you need to detect a 2% lift, you probably need more traffic.
- Not accounting for segments: If you plan to segment results, you need more data.
What If You Don't Have Enough Traffic?
- Test bigger changes: A radical redesign is easier to detect than button color changes.
- Use a higher-traffic page: Test on your homepage instead of a deep product page.
- Lower your significance level: 90% might be acceptable for lower-risk decisions.
- Combine similar pages: Test across a category instead of a single product.
- Use qualitative methods: User testing and surveys can complement limited quantitative data.
Need More Traffic for Your Tests?
We drive qualified traffic that converts. More visitors means statistically significant results, faster.
Book Your Strategy Call