Does the Word 'Free' Affect Email Deliverability & Click-Through Rates? [A/B Test]

Melissa Miller
Melissa Miller

Updated:

Published:

Email marketing can be a tricky balancing act. You want to use copy that will get your recipients' attention, but you don’t want to select words that will raise red flags with SPAM filters.

Email Test

But there's some good news! If you've been email marketing for a while (or even if you haven't), you might have heard that “free” is no longer a dirty word with the email overlords. Still, some email marketers are hesitant to use the word, just in case there's some truth to the former rumors.

Well, we decided to channel our inner Sherlock Holmes and do a little investigating. We tested it ourselves ... with two questions in mind, actually:

  1. Does using the word “free” hurt my deliverability?
  2. Does my click-through rate improve if I use the word “free?”

And, if the answer to both of these questions is “yes,” then what's the net effect of these two forces acting against each other? That is to say, even if my deliverability suffers, does the higher click-through rate make up for the hit? Okay, I guess that's three questions. Let's dig in ...

Download Now: Free State of Marketing Report [Updated for 2024]

The Test

We set up the test by creating two separate emails.

  • Version A had the word “free” in the subject line, sprinkled into the body three times, and included as text on the supporting image.
  • Version B was completely void of the word “free.” Instead of “Free Guide,” we used “SEO Guide” in the subject line.


email free ab test resized 600

The results were truly fascinating. Here's what we found:

Deliverability Results

We measured deliverability with two software tools: 1) Return Path and 2) HubSpot.

When we looked at Return Path's SPAM Filter Check results, we found that the two emails performed similarly. Of the nine filters checked, version A -- the email including the word "free" -- passed seven, while version B passed eight (78% versus 89%). This difference was attributed to one particular server filter that identifies itself as a "'carrier grade' filtering solution, used by some of the biggest ISPs in the business."

Return Path, however, also indicated that both emails had identical SpamAssassin Scores of 1.3. (As a general guideline, Return Path recommends aiming for a score lower than 5 in order to avoid deliverability issues.)

HubSpot's email software revealed similar results. Version A (with "free") got delivered to 99.25% of selected recipients, and version B got delivered to 99.24%. This means that version A did 0.01% better.

Considering all three data points, it looks like "free" raised an additional (small) red flag, but it was not severe enough to make version A perform statistically worse than version B. In other words, using the word "free" made no significant difference.

Click-Through Rate Results

The results based on click-through rates were a different story. One email was the clear winner with a click-through rate that was 17% better at 99% statistical significance.

Can you guess which one?

It was variation B, the version that excluded the word "free"! Even when you look at the click-through rate out of only those who opened the email, variation B performed 5% better than A.

Now, this doesn't mean that all emails without the word "free" will necessarily get a better click-through rate. It does, however, tell us that HubSpot's email subscribers are more responsive to the word "SEO" than the word "free." I can think of a couple reasons why this would be true. Either HubSpot subscribers are especially keen on SEO, or they already know that our content is free, and therefore, the word does not add value. Either way, it's fascinating to see that "free" -- a term that seems to be regarded as an all-powerful, silver bullet in the marketing world -- did not win this competition.

Marketing Takeaways

This test revealed two key findings:

  1. The word "free" on its own will not hurt your emails' deliverability.
  2. Even though it's safe to use "free," it might not be the most effective term for your audience.

The Lowdown on Keyword Filtering & Identifying SPAM

I spoke with HubSpot's own deliverability specialists about point #1, or more specifically, keywords' effect on email deliverability. They explained that keyword filtering is the old school method for identifying SPAM, but no longer is it the standard to outright reject emails based on keywords alone.

Instead, keyword filtering is simply used to help score your emails. And it is just one factor among several. Keyword filtering only hurts you when you have other problems, like a poor Sender Score (i.e. your IP reputation), email lists with low engagement, or a poor domain reputation. While keywords do impact the way you are perceived by a server, it's not as heavy handed as the rumors have suggested, and you will not get rejected just because the word "free" is pervasive in your email.

Now that we've cleared that up, we can all go forth and safely test "free" against whichever keywords we fancy!

state-of-marketing-2024

 

Topics: a-b-testing

Related Articles

We're committed to your privacy. HubSpot uses the information you provide to us to contact you about our relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our Privacy Policy.

Outline your company's marketing strategy in one simple, coherent plan.

Marketing software that helps you drive revenue, save time and resources, and measure and optimize your investments — all on one easy-to-use platform

START FREE OR GET A DEMO