Marketing Matrix: 10 Dos and Don'ts of A/B Testing

When was the last time you tested some part of your marketing? If it takes a bit of thought, you're probably not doing it enough. The problem with that is, even just a minor test can go a long way, improving traffic, leads, and even sales if done right. And if you're not taking advantage of some simple A/B testing tactics, you could be missing out on some major marketing improvements.

So without further ado, I present to you 10 great dos and don'ts of A/B testing to motivate you to get moving and start doing more testing.

10 A/B Testing Dos and Don'ts

1. Do Test Tons of Different Variables: There are so many variables you can A/B test in your marketing. Color, size, subject line, content offer, templates, layout, timing, frequency -- the possibilities are almost endless! The more variables you test, the better you'll be able to refine your marketing approaches and generate better results from your initiatives.

2. Don't Test Them All at Once: Smart A/B testers understand the importance of focusing on just one variable for each test they conduct. If you're testing the effectiveness of one call-to-action over another and are modifying two different variables such as design and messaging, how will you be able to isolate the reason for whether one CTA was more effective than the other? Stick to one variable per test so there's absolutely no guesswork involved in determining why one test subject worked better than another.

3. Do Start Simple: Dip your toes into the A/B testing pond by testing something simple like CTA button color or size on a webpage. While there are definitely more advanced tests you can conduct like testing more complex website features, there's nothing wrong with starting small and working your way up. In fact, even a minor change can cause major results. WikiJob, for example, conducted an A/B test by adding testimonials to one of its pages to see if it would increase that page's conversions. As a result of this simple addition, the web page including testimonials performed 34% better in sales than the web page without them. Starting simply will also help you learn some valuable A/B testing skills and build up the confidence to try more advanced testing.

4. Don't Be Afraid to Try More Advanced Testing: Once you feel comfortable with simple A/B tests, don't be afraid to graduate to more complex tests such as A/B/C tests or page level test designs, which pits two dramatically differently designed pages against one another. While this may seem to conflict with the direction of not testing more than one variable at once, it actually makes sense if you consider the whole page as the variable you're testing.

5. Do Always Be Testing: There's no reason why you can't constantly be testing something, and if you have the bandwidth to do so, there's no reason why you shouldn't be testing two different things -- like a CTA on one page and the layout on another page -- at the same time. The more intelligence you can generate at one time, the quicker you can optimize your marketing.

6. Don't Limit Your Tests to One Marketing Channel: Furthermore, don't think you can only test one channel in your marketing mix like website design or email marketing. You can conduct testing in every marketing channel at your disposal -- on your blog, in social media, in your search engine optimization -- everywhere! Just be sure you're using the right metrics for the channel you're testing.

7. Don't Assume Your Results are Statistically Significant: Not every test you conduct will generate significant results, so don't assume they all will (you know what they say happens when you assume, right?). Just because one email subject line performed slightly better than another doesn't mean that test was significant. And you don't want to determine next steps using data that didn't truly mean anything, do you?

8. Do Analyze Whether Results are Statistically Significant: Instead of assuming, actually analyze whether your test results are statistically significant. This blog post offers some tips and tricks on how to determine statistical significance.

9. Don't Give Up if Results Aren't Significant: If, through your analysis, you determine that your test results aren't significant, don't despair. This may happen from time to time, but it shouldn't deter you from conducting future tests. Determining that your test wasn't statistically significant can teach you something in itself: that the variable you were testing doesn't have a big impact on the results. So if the color of your CTA button didn't affect click-through rate, perhaps changing the text on the button will.

10. Do Make Changes Based on Test Results: Once you've analyzed the results of your test, act accordingly! If your test was statistically significant, implement the winning change. If it wasn't significant, recognize that the variable you tested didn't impact the results, and try testing a new variable.

Are you regularly incorporating A/B testing into your marketing strategy?

Image Credit: Horia Varlan