How to A/B Test a CTA Redesign in HubSpot [Customer Story]

The Bellingham Electric blog has gone through quite a few iterations of CTA design using HubSpot’s PowerPoint method since 2014. We have about 10 different button-style CTAs for downloadable content, which are featured on over 300 blog posts.

In our case, the typical A/B testing scenario didn’t seem effective, given that these CTAs were each used on somewhere between 30 to 40 posts that generated different amounts of traffic. So, we decided to try something different. 

Before and After

We changed two of our CTAs to the new design and compared them against the other CTAs in the original style.

Here’s an example of one of our old button-style CTAs. The entire image is clickable, and the minimal text and height-to-width ratio made it mobile-friendly. That said, it was definitely in need of a face-lift. All our CTAs shared this style, and all of them are content offers for buying guides for different products.

 Our new CTA:

We replaced two of our existing CTAs with the new design and compared their performance against other CTAs that had a similar number of views and clicks. This helped us create a control and an experimental group, so we could observe how the new CTAs performed.

This method allowed us to successfully evaluate the new CTAs against the old without manually changing and monitoring dozens of blog posts. It’s important to note that this method may not be as effective if you have only a handful of CTAs, or if you have CTAs that are applied differently throughout your site. Because our CTAs were more or less identical in style except for the content they were directing readers to, we could easily compare our results. Let’s take a look at how we did it.

Step 1: Getting the data

First, we had to document the existing clickthrough ratio (CTR). HubSpot makes this easy: just navigate to Marketing > Lead Capture > CTAs, and click Export Data. This gives you the data for all time, which is useful for choosing which options to pair for the A/B test.

Once you start the test, you’ll be analyzing the data for the month or two after the A/B test against the previous period.

The blue highlighted rows in the screenshot are the CTAs that were replaced with the new design. Extraneous CTAs have been clipped out so that we can focus on the button CTA design being tested.

Step 2: Finding test subjects and controls

We chose test CTAs that had a lot of exposure. We also decided to use two other CTAs as a benchmark based on similar CTRs and overall views. 

We decided to compare December performance against January and picked two benchmarks that are noted in red above with similar views and CTRs. This allowed us to compare them against the new CTA style. We also applied the new style to the Refrigerator CTA and compared it against the Washing Machine CTA. We did the same with the Range and Dishwasher CTAs.

Step 3: Now we wait!

After applying the new CTA style to the experimental group, we waited for one month. We saw results very quickly. We knew that the new CTA was more modern and appealing, so we didn’t feel the need to continue the test beyond that month.

Step 4: Analyze the results

Rather than compare the new CTAs against the all-time data we exported from HubSpot, we looked at the month-to-month data. We used the details for each CTA to find the overall number of views as well as the CTR for the first month — before we started the test — and for the test month.

Now, if we had looked at these two redesigned CTAs alone, we would have been impressed: The CTR increased by 47% for one, from 1.27% to 1.87%, and by 20% for the other, from 1.94% to 2.33%.

Comparing the results across the board was even more exciting. Although overall views went up for almost every button-style CTA, the CTR decreased on almost all of them. The only notable exception was our Grilling CTA, which we inferred was mostly due to seasonality. Seeing the growth in CTR for the new CTAs in the experimental group in the context of declining CTR across the board further validated that our redesign worked.

We were relatively certain that the new CTA would outperform the old one simply because it was a more modern design that followed best practices, but seeing that data really drove that assumption home.

Conclusion

If you have a handful or more of similarly styled CTAs with similar content, this method might work well for you. It’s not quite as convenient as Hubspot’s built-in A/B test options, but it was still a pretty easy process.

Want to connect with others on HubSpot tips, tricks, and updates? Head over to the HubSpot Community to join a conversation or start one of your own.