Why do you need a minimum number of visits to compute the results? How many visits is the minimum?

Nelio A/B Testing for WordPress

With a very low number of visits, you could not trust the results since they can be just due to pure chance. Imagine that only two people visit the variation of your tested page and both of them convert (100% conversion rate). Can you infer from this that everybody that will visit your site will convert? Clearly not. Maybe the variation has a conversion rate of around 20% and it just happened by chance that the first two visitors converted. We need more visitors to see if this is the real trend or not.

To get a minimum level of confidence in the results, they are not computed until you get at least 10 visits per variation.

Posted in: Statistics Antonio VillegasWhy do you need a minimum number of visits to compute the results? How many visits is the minimum?