4 Basic A/B Testing Tips for Beginners

Jeremy Ellens
Jeremy Ellens



Take a moment to think of your favorite meal. Now, picture eating that meal everyday for the next year. For those of you thinking you could, in fact, eat the same meal every day and be okay with it, picture drinking your favorite alcoholic beverage instead; day, after day, after day. Feeling sick yet? Good, because that is what you are doing to your prospects and subscribers with the same email and landing page elements day after day.

If your business is at a standstill for improving open and click-through rates, it’s time you start making changes in your email and landing pages. Even for those of you seeing improvements, you can always do better. A/B testing is the answer for optimizing your email marketing and landing page performance.

Get rid of the mindset that change is scary. You can add variety to your emails and landing pages without the fear of it making a negative impact on your brand performance by using A/B testing to make safe, incremental changes that keep your buyers engaged and moving further down the buying cycle.

This post will cover the different email and landing page elements you should be testing, some best practices for A/B testing, and the importance of acting on your findings. You will quickly see why businesses with the correct testing strategy in place are increasing conversion rates by up to 300%.

For those of you unfamiliar with A/B testing, it works just as it sounds. Your business should have an A and B version of each element of your email copy (i.e. blue CTA button vs. red CTA button), and then use metrics to measure the success of each variation.

A/B testing example

1) Commonly Tested Elements

Subject line 

This is typically the most vital element of your email copy to be testing. Regardless of the quality of your business’s content, the subject line will be the deciding factor in whether or not your email gets opened. Test different subject line phrasing to see what gages the highest click-through rates from your subscribers. You might try explicitly stating your offer vs. using ambiguity to tease the subscriber, or funny subject lines vs. urgent ones.

Below is a graph depicting how marketer Neil Patel tested different variations of his subject line copy to more than double open rates.

quicksprout email open rate graphHeadline

Your headline is another important element that needs to be constantly tested. Similar to your subject line, your headline phrasing can be the deciding factor in your landing page’s success. Assuming your subject line was appealing enough to get subscribers to open your email, your headline will play a vital role in improving click-through rates.

headline example
The header above was changed from “Start a Highrise Account” and generated 30% more sign-ups.

For more tips on improving the quality of your email copy, check out a previous Yazamo Insights article, “Stay Out of the Spam Folder With More Effective Email Copy”.

Call-to-Action (CTA)

Your business should be experimenting with different ways to present your call-to-action. This could mean changing the wording used, adjusting the size and color, and testing different places to position the CTA in your email or landing page.

CTA example
The CTA on the right increased one business’s conversion rate by 34%.


Believe it or not, consumer expectations have gotten so high that simply offering discounts or promotions is not enough to catch their attention. That’s right--we now live in a day and age where giving out free money is not good enough.

You should be testing different phrasings in your promotional efforts to see which offers are most enticing to subscribers. For instance, you could test whether an offer for a percentage off vs. a specific dollar amount off gets higher click-through rates. 


Your business should be testing different email templates to see which trigger the highest performance rate. You could test whether subscribers prefer a single column body vs. a double column, or one color template vs. another, etc.


Where you place your image, what your images are of, and how many there are all play a role in how effective your email is going to be. Run different tests to find out which one triggers the best performance metrics. Having a person in your images as opposed to a product is just one example of a test you might want to try out.

You should be beginning to see a trend in the benefits that A/B testing can provide that other marketing research methods cannot. This quantitative approach can measure your visitors’ behavioral patterns and provide the insights necessary for coming up with solutions for implementing more effective email and landing page copy in the future.

2) Incremental Changes

The truth is that most pages do not have a traffic problem. What they have is a conversion problem. In general, conversion rates range from about 1-3%. What are you going to do to engage the 97% of people visiting your site that are not interested in what you are offering them?

Incremental changes may be your first step in figuring out what exactly it is that these leads interact with best.

When conducting A/B tests, make sure that your business is making incremental changes. In other words, only test one variation of an element at a time. For instance, if you want to see how the color of your CTA button influences click-through rates, your Version A should stay the same, and the only difference in your Version B should be a different colored CTA button.

You cannot kill two birds with one stone here. Trying to measure the influence of multiple different elements at once will complicate your ability to confirm which element triggered the change.

If you were to try measuring the impact of your CTA, header, and layout all at once, and you found an improvement in click-through rates, you wouldn’t be able to differentiate which element was responsible for the improvement.

Not knowing this might lead to unexpected decreases in the future if you were to change something without knowing the positive impact it was making in your emails.

For more examples of elements you should be incrementally testing and other mistakes to avoid in your landing page copy, check out another previous Yazamo Insights article, “Don’t Let These 8 Landing Page Design Mistakes Haunt Your Business”.

3) Value of Time

Your company should be using A/B testing to determine the optimal times for sending out emails. GetResponse, found that 26.63% of all email opens take place within the first hour that the email was sent. With that being said, it becomes very obvious how important it is that you are sending your emails at the right time.

Do not make the mistake of listening to published findings from other companies that state the best time for sending marketing emails. This optimal time is different for every business! Depending on what your business offers and who your subscribers are, the optimal time for subscriber engagement varies significantly by industry and target market.

Fortunately, your marketing automation software can typically suggest optimal times for sending emails based off of previous email open and click-through activity among your subscribers.

Many businesses fear A/B testing because they think that in the process of testing different elements they might lose prospects. The truth is that your business can use A/B testing for determining the optimal time for reaching your subscribers in a way that will not hurt your current CTR and open rates.

You can do this by using A/B testing on a small percentage of your subscribers and using your sample findings to predict how those results will translate to your entire subscriber list. For instance, you could send 10% of your subscribers an email at your regularly scheduled time (Version A) and then send another 10% of your subscribers an email at a different time (Version B).

Once you know which time triggered better performance rates, you can then use that winner as the designated time for emailing the remaining 80% the following day.

4) Implement Solutions

Only 22% of businesses are satisfied with their current conversion rates. This could be a result of businesses not running A/B tests, businesses not running enough A/B tests, or businesses not properly acting on the findings that they obtain with their tests.

Do not cut your A/B tests short. Ensure that they are running for a sufficient amount of time, which can typically be calculated by obtaining a sample large enough to have statistical significance (most testing tools can report this).

Once you are confident in the results that your A/B tests have obtained, it is time to act on your findings. You want to organize your findings into a binder or spreadsheet in order to avoid testing the same things again in the future. Additionally, organizing your findings will make it easier for you to come up with individual solutions that can be further capitalized on in the future.

Let’s say you find that using images of people on your landing page as opposed to images of products improved click-through rates. Aside from just ensuring that you continue using images of people moving forward, this finding can be applied to other elements of your marketing content, such as in your ads or on your website homepage. The only way to find out if this works is to test and test again.

You will begin to see patterns among the different elements that increased email performance. You can then use these patterns to better understand your subscribers and continue delivering compelling email copy in the future.

So what else can you do right now to optimize performance and start producing happier customers? Stay ahead of the curve and download our free ebook going over inbound marketing trends this year and our predictions for 2015. Click the image below to get it now!

New Call-to-action

We're committed to your privacy. HubSpot uses the information you provide to us to contact you about our relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our Privacy Policy.

Outline your company's marketing strategy in one simple, coherent plan.