Does the Word 'Free' Affect Email Deliverability & Click-Through Rates? [A/B Test]

by Melissa Miller

Date

September 5, 2012 at 1:00 PM

Email Testintroductory3

Email marketing can be a tricky balancing act. You want to use copy that will get your recipients' attention, but you don’t want to select words that will raise red flags with SPAM filters.

But there's some good news! If you've been email marketing for a while (or even if you haven't), you might have heard that “free” is no longer a dirty word with the email overlords. Still, some email marketers are hesitant to use the word, just in case there's some truth to the former rumors.

Well, we decided to channel our inner Sherlock Holmes and do a little investigating. We tested it ourselves ... with two questions in mind, actually:

  1. Does using the word “free” hurt my deliverability?
  2. Does my click-through rate improve if I use the word “free?”

And, if the answer to both of these questions is “yes,” then what's the net effect of these two forces acting against each other? That is to say, even if my deliverability suffers, does the higher click-through rate make up for the hit? Okay, I guess that's three questions. Let's dig in ...

The Test

We set up the test by creating two separate emails.

  • Version A had the word “free” in the subject line, sprinkled into the body three times, and included as text on the supporting image.
  • Version B was completely void of the word “free.” Instead of “Free Guide,” we used “SEO Guide” in the subject line.


email free ab test resized 600

The results were truly fascinating. Here's what we found:

Deliverability Results

We measured deliverability with two software tools: 1) Return Path and 2) HubSpot.

When we looked at Return Path's SPAM Filter Check results, we found that the two emails performed similarly. Of the nine filters checked, version A -- the email including the word "free" -- passed seven, while version B passed eight (78% versus 89%). This difference was attributed to one particular server filter that identifies itself as a "'carrier grade' filtering solution, used by some of the biggest ISPs in the business."

Return Path, however, also indicated that both emails had identical SpamAssassin Scores of 1.3. (As a general guideline, Return Path recommends aiming for a score lower than 5 in order to avoid deliverability issues.)

HubSpot's email software revealed similar results. Version A (with "free") got delivered to 99.25% of selected recipients, and version B got delivered to 99.24%. This means that version A did 0.01% better.

Considering all three data points, it looks like "free" raised an additional (small) red flag, but it was not severe enough to make version A perform statistically worse than version B. In other words, using the word "free" made no significant difference.

Click-Through Rate Results

The results based on click-through rates were a different story. One email was the clear winner with a click-through rate that was 17% better at 99% statistical significance.

Can you guess which one?

It was variation B, the version that excluded the word "free"! Even when you look at the click-through rate out of only those who opened the email, variation B performed 5% better than A.

Now, this doesn't mean that all emails without the word "free" will necessarily get a better click-through rate. It does, however, tell us that HubSpot's email subscribers are more responsive to the word "SEO" than the word "free." I can think of a couple reasons why this would be true. Either HubSpot subscribers are especially keen on SEO, or they already know that our content is free, and therefore, the word does not add value. Either way, it's fascinating to see that "free" -- a term that seems to be regarded as an all-powerful, silver bullet in the marketing world -- did not win this competition.

Marketing Takeaways

This test revealed two key findings:

  1. The word "free" on its own will not hurt your emails' deliverability.
  2. Even though it's safe to use "free," it might not be the most effective term for your audience.

The Lowdown on Keyword Filtering & Identifying SPAM

I spoke with HubSpot's own deliverability specialists about point #1, or more specifically, keywords' effect on email deliverability. They explained that keyword filtering is the old school method for identifying SPAM, but no longer is it the standard to outright reject emails based on keywords alone.

Instead, keyword filtering is simply used to help score your emails. And it is just one factor among several. Keyword filtering only hurts you when you have other problems, like a poor Sender Score (i.e. your IP reputation), email lists with low engagement, or a poor domain reputation. While keywords do impact the way you are perceived by a server, it's not as heavy handed as the rumors have suggested, and you will not get rejected just because the word "free" is pervasive in your email.

Now that we've cleared that up, we can all go forth and safely test "free" against whichever keywords we fancy!

Have you tried using "free" in your emails? What kind of results have you seen?

optimizing-email-marketing-ebook like-what-youaposve-read-click-here

Search Inbound Hub

Subscribe to Marketing Articles by Email

Subscribe by RSS

Follow HubSpot

Call Us: 1-888-HUBSPOT