The key to proper A-B testing lies in complicated statistical processes that are a foreign language to many marketers. Statisticians know that raw conversion rates alone are not enough to claim victory for one landing page over another, but many marketers don’t.
As data becomes more important to validate marketing campaigns, marketers must learn to speak the language of stats.
However, until there is a more formal fusion of marketing and statistician functions, marketers can be content with cutting some corners to produce data-driven results—as long as those results are statistically significant. This means data analysis must take into account test preparation, conditions, sample sizes and the probability of finding results solely based on chance. While a somewhat daunting task, this is a necessary process marketers must battle through in order to actually prove what works and what doesn’t, and then follow up with optimization.
Download the free Ultimate A-B Testing Tool here »
As you may have guessed, this statistical process for A-B testing gets pretty technical, so we recreated it in a simple template that will turn any marketer into a stats wiz. What was once a “2-sample z-test for difference in population proportions” is now a “2-step plug-n-chug for ultimate A-B testing convenience.” Simply choose your confidence level, plug in your number of conversions and total visits, and watch your results magically appear.
A conclusion statement will be given in terms of the confidence level you choose. The higher the confidence level, the more difficult it will be to prove a statistical difference between the conversion rates, but the less likely that your conclusion is a result of random chance. For example, a 98 percent confidence level would yield a conclusion with 98 percent certainty, meaning there is only a 2 percent (100-98) probability that you would find these results due to random sampling variability, or chance. And as a rule of thumb: The larger your sample size, the more accurate and actionable your results will be.
Most marketing automation software will track conversion percentages, but the simple truth is that most don’t give you enough information to actually prove which strategies are the most successful. For that, you need something more powerful and scientifically-backed.
Our simple solution works for just about everything from landing page conversions to click-through rates on emails and calls-to-action. Download the 2-step A-B Testing Tool and become a Marketician.
