Top A/B Test Do’s and Don’ts

A/B testing, or split testing, should be a core part of your marketing and advertising strategies. Otherwise, how do you know that every element of your campaigns is working toward your goals?

But even if you have incorporated A/B testing into your campaign implementation and monitoring, you may not be doing it right. Then, you may as well not be doing it at all since you’re wasting time and resources on testing that’s not getting you the answers you need.

Here are some top A/B test do’s and don’ts to help you make sure you are getting the right results to maximize your marketing and advertising strategies:

DO Repeat Tests

The point of your split testing is to get answers. So when you think you’ve got the answers you need, you might be tempted to stop the testing.

You might think you have exactly what you need to start getting results.

But there’s a good chance that you don’t have the right answers. Are you sure that you ran the test for long enough? Or that you had a significant sample size?

Make sure that you run the test for a full business cycle. What determines a cycle will depend on your business model and operations. It can be a full week, a full month, or a full quarter. It can be bookmarked by an email campaign or a promotional campaign. You decide what determines a business cycle.

After a few weeks or a few months, you should run the same test again for another full business cycle to make sure that external factors did not skew the results of the test.

DON’T Abandon Inconclusive Tests

Ideally, your split testing will focus on one element of your campaign and will produce clear answers about which gets better results.

Unfortunately, this doesn’t always happen. You might run the test for a full cycle and still not get any statistically significant results. Both might seem to perform about the same. You might think that the test wasn’t effective, or you might think that the element you chose for the test isn’t important enough to have an impact.

What might be the truth is that you haven’t run the test for long enough or that you haven’t isolated the item to test the way you thought you had.

You might need to sharpen your focus. For example, if you are testing something that is hard to identify, such as emotional appeal in your ads, you might get inconclusive tests. You might need to focus instead on something like specific words used in the headline or a specific anecdote that you use to convey the emotion.

You might not need to change anything about the tests, but you might need to change the way you look at the analytics. For example, you might find that looking specifically at mobile users versus desktop users, you see some differences emerge between the two items being tested.

Don’t give up because your test seems inconclusive at first. Go back and see what you need to look at more closely or what you need to change.

DO Get User Feedback

A/B testing is often seen as a quantitative and analytical task. You look at the cold, hard numbers and you get your answers.

But there is a lot that the numbers can’t tell you. You need the user feedback to help you get a complete understanding of how your campaigns are performing.

You can do this by running a poll or survey alongside your test. The poll or survey can ask them specific questions about why they took the actions that they did or about what they thought of specific elements of your campaign. Or you can ask users for general feedback and let them tell you what they think is important.

Consider offering an incentive to fill out the survey by giving a discount or a freebie.

DON’T Forget to Segment

A lot of people run split testing on a campaign at large, but they forget to segment into mobile and desktop users.

However, the way that mobile users interact with contact is MUCH different than the way that desktop users do.

Run specific tests for mobile users and for desktop users separately. Your tests should look very different for each of them. Just treat each audience separately, and create tests that are independent of one another.

You’ll need to create different campaigns for mobile and desktop users anyway, so make sure you run separate tests also.

A/B testing can be a highly effective way to determine the winning elements of your marketing and advertising strategies. You just need to take the right steps to get accurate results from your tests. If you aren’t getting the results you need, check this list to see what you can change.


Marketers Valley Team

Marketing Industry Experts >

Leave a Reply

Your email address will not be published. Required fields are marked *

Next Post

Case Studies vs. Data Driven Marketing

Thu Jul 26 , 2018
Research is key to the success of your marketing. Without the right research, you will be throwing a lot of money after bad ideas that you only know are bad once they fail. With good research, you have a better idea of what strategies are likely to succeed with your […]