How to calculate sample size
- For an A/B test, enter the current conversion rate and the smallest improvement worth detecting, as a relative change (20% of a 10% rate means 12%) or in percentage points.
- Keep 95% confidence and 80% power unless you have a reason to change them, and add your daily traffic to estimate how long the test must run.
- For a survey, choose the margin of error and confidence level, add the population size if it is small, and the response rate you expect.
Why sample size matters
An A/B test with too few visitors cannot tell a real improvement from random noise, so it either finds nothing or, if you stop early when it looks good, finds things that are not there. The required sample depends mostly on the effect you want to detect: halving the minimum detectable effect roughly quadruples the sample. Detecting a lift from 10% to 12% at 95% confidence and 80% power needs about 3,841 visitors per variant with the standard two-proportion formula used here. Testing more than two variants splits traffic further and needs a stricter significance level, applied here with a Bonferroni correction.
For surveys, the classic figure is 385 responses for ±5% at 95% confidence from a large population, assuming the most conservative 50% split of answers. Small populations need fewer responses thanks to the finite population correction, but non-response bias matters more than sample size if only enthusiastic people reply. Calculated in your browser.
What is statistical power?
The chance of detecting an effect of the size you specified if it really exists. 80% is the usual standard.
Why is my required sample so large?
Small effects on low conversion rates need a lot of data. Test bigger changes, or use a metric with a higher base rate.
How many survey responses do I need?
About 385 for ±5% at 95% confidence, 1,068 for ±3%, assuming a large population.