Email marketing can be a tricky balancing act. You want to use copy that will get your recipients' attention, but you don’t want to select words that will raise red flags with SPAM filters.
But there's some good news! If you've been email marketing for a while (or even if you haven't), you might have heard that “free” is no longer a dirty word with the email overlords. Still, some email marketers are hesitant to use the word, just in case there's some truth to the former rumors.
Well, we decided to channel our inner Sherlock Holmes and do a little investigating. We tested it ourselves ... with two questions in mind, actually:
- Does using the word “free” hurt my deliverability?
- Does my click-through rate improve if I use the word “free?”
And, if the answer to both of these questions is “yes,” then what's the net effect of these two forces acting against each other? That is to say, even if my deliverability suffers, does the higher click-through rate make up for the hit? Okay, I guess that's three questions. Let's dig in ...
The Test
We set up the test by creating two separate emails.
- Version A had the word “free” in the subject line, sprinkled into the body three times, and included as text on the supporting image.
- Version B was completely void of the word “free.” Instead of “Free Guide,” we used “SEO Guide” in the subject line.
The results were truly fascinating. Here's what we found:
Deliverability Results
We measured deliverability with two software tools: 1) Return Path and 2) HubSpot.
When we looked at Return Path's SPAM Filter Check results, we found that the two emails performed similarly. Of the nine filters checked, version A -- the email including the word "free" -- passed seven, while version B passed eight (78% versus 89%). This difference was attributed to one particular server filter that identifies itself as a "'carrier grade' filtering solution, used by some of the biggest ISPs in the business."
Return Path, however, also indicated that both emails had identical SpamAssassin Scores of 1.3. (As a general guideline, Return Path recommends aiming for a score lower than 5 in order to avoid deliverability issues.)
HubSpot's email software revealed similar results. Version A (with "free") got delivered to 99.25% of selected recipients, and version B got delivered to 99.24%. This means that version A did 0.01% better.
Considering all three data points, it looks like "free" raised an additional (small) red flag, but it was not severe enough to make version A perform statistically worse than version B. In other words, using the word "free" made no significant difference.
Click-Through Rate Results
The results based on click-through rates were a different story. One email was the clear winner with a click-through rate that was 17% better at 99% statistical significance.
Can you guess which one?
It was variation B, the version that excluded the word "free"! Even when you look at the click-through rate out of only those who opened the email, variation B performed 5% better than A.
Now, this doesn't mean that all emails without the word "free" will necessarily get a better click-through rate. It does, however, tell us that HubSpot's email subscribers are more responsive to the word "SEO" than the word "free." I can think of a couple reasons why this would be true. Either HubSpot subscribers are especially keen on SEO, or they already know that our content is free, and therefore, the word does not add value. Either way, it's fascinating to see that "free" -- a term that seems to be regarded as an all-powerful, silver bullet in the marketing world -- did not win this competition.
Marketing Takeaways
This test revealed two key findings:
- The word "free" on its own will not hurt your emails' deliverability.
- Even though it's safe to use "free," it might not be the most effective term for your audience.
The Lowdown on Keyword Filtering & Identifying SPAM
I spoke with HubSpot's own deliverability specialists about point #1, or more specifically, keywords' effect on email deliverability. They explained that keyword filtering is the old school method for identifying SPAM, but no longer is it the standard to outright reject emails based on keywords alone.
Instead, keyword filtering is simply used to help score your emails. And it is just one factor among several. Keyword filtering only hurts you when you have other problems, like a poor Sender Score (i.e. your IP reputation), email lists with low engagement, or a poor domain reputation. While keywords do impact the way you are perceived by a server, it's not as heavy handed as the rumors have suggested, and you will not get rejected just because the word "free" is pervasive in your email.
Now that we've cleared that up, we can all go forth and safely test "free" against whichever keywords we fancy!
Have you tried using "free" in your emails? What kind of results have you seen?
Inbound Marketing Blog Feed
Lynn Dalsing 1:11 PM on September 05, 2012
Love this post! This is a "fact" that gets thrown out often in email marketing, and it's good to see someone taking the time to test it.
That said, I think it's great to see that "free" isn't as important as most marketers think.
Any ideas on how you would test if "free" is actually dissuading people from clicking through (i.e. if people are thinking that a free guide must not be worth it)?
Loren McDonald 1:12 PM on September 05, 2012
Melissa, I'm not sure that how you've constructed the test is actually a fair test of the use of free. All of HubSpot's white papers and guides are free as far as I know. So there is little value proposition to the recipient in the use of "free" in this context versus the topical value of "SEO." Just a thought.
Atilla Vekony 1:22 PM on September 05, 2012
Thanks for the tip. Been wondering about this every time I sent an email ... looks like I can relax...
Ed Gray 1:27 PM on September 05, 2012
Interesting post. Would love to see something like it in a B2C environment.
Ellis 1:30 PM on September 05, 2012
This test simply answers the question: Do our e-mails get delivered even if we use the word "free" within the subject / body of the e-mail (and it does!)
In my opinion, what we should take from this is that different things work for different audiences. For example, if you were blasting an e-mail for a Supermarket, the word free might make a difference, especially if the target market is a young, mid-low income mother.
As mentioned, Click-through rate will always vary depending on your target market, which is why you should have data regarding your actual customers, this way you don't waste time sending the wrong content.
I personally loved this study! We are in the process of getting clients on the web and keywords are always tricky business.
Thanks for posting this!
Silviu Preoteasa 1:34 PM on September 05, 2012
Thank you for the point.
I second Loren there - there is no base for expanding the findings outside a clearly defined subset, in this case "subscribers to Hubspot's newletter". From experience, and again an interesting subset to look at, if we do "Free printer with selected Dell laptops" it gets more traction - but I prefer to put forward it's the perceived value of the offer.
Melissa, how about you go with two more recipes - "Free SEO Guide" and "Free SEO Guide worth $xxx" - as you can see, I am interested if the qualifier "free" has actually different perceived value in relation to the terms it associates with.
Great post, cheers!
Tony Smith 1:41 PM on September 05, 2012
This is an awesome post! I am currently building out my site, and was positioning it as FREE advice for the would be entrepreneur.
However, after reading this post, I believe I will rethink this strategy!
Thanks for sharing this extremely useful content;-)
I will definitely put it to good use!
Sincerely,
Tony Smith
Websmith Marketing
Internet Business Professional & Coach
Herb Lawrence 1:41 PM on September 05, 2012
Great post and very helpful I have been working on "traditional knowledge" that Free in subject line equals dead in the water. I wonder though if it might be because folks who did us the word free in the subject line were so excited the capitalized it? FREE
I guess it will be time to try it out in an A/B test as well. Thanks again Hubspot you always make me think
Jeremy Knight 2:06 PM on September 05, 2012
Three important thingqs to remember when using the word free in email are:
1) avoid using it in capital letters
2) avoid using in conjunction punctuation like an exclamation mark
3) avoid using it in conjunction with other spam trigger words like 'trial' or 'quote'
Just a thought
Jeremy
Chandler Turner 2:14 PM on September 05, 2012
I also second Loren's opinion. And there is a big difference in "free" from a known entity vs an unknown. I have seen a good bit of logical information to suspect that the word "free" actually carries a marketing-speak connotation or an automatic doubt - questioning the veracity or value of the offer. We have seen too much that has no apparent value and are suspect. It is nonetheless a good starting test question.
Kristen 2:38 PM on September 05, 2012
Thanks for doing the hard work for me. :)
Cyndi 3:13 PM on September 05, 2012
I think that these results need to be carefully weighed before a marketer decides to start throwing the word "FREE" into their email communications. For example:
1. Since this was tested with a house file subscriber base, it might have impacted the results, and others may not achieve the same with a different customer base.
2. FREE was tested against no reference. Would it have been a better to test FREE against another word such as complimentary to see the power of the word itself against the potential spam risk?
Cyndi
I would suggest that
Dean 4:35 PM on September 05, 2012
I struggled using 'Free' to entice an audience to visit a link to a webcast we just delivered. The work performed behind developing the presentation, the thought leadership of the presenter, and also the cost to run the webcast by our provider were all very valuable to our marketing budget. I felt saying 'Free' ultimately diminished the value of the content.
Raul Bataller 4:52 PM on September 05, 2012
Buen post, me encanto leer sobre email-marketing y el estudio realizado
title="Servicios Social Media">
title="Social Media">Community
Manager
Jeet Banerjee 5:44 PM on September 05, 2012
This is a great article. I think people are more prone to checking something out just because it is free.
Ray Cassidy 6:56 PM on September 05, 2012
Interesting. I've tended to avoid the liberal use of free. AT one point I was dipping into the equally spammy looking fr.ee but I just wasn't comfortable with such blatant fudgery.
I think what this article does point out is the value of testing. Whatever the keywords you use - it's what the receipient responds to that's important.
Victor Lopez 11:22 PM on September 05, 2012
Awesome split test. And it produced great content too! I will try a similar post on my site.
Bridget 1:30 AM on September 06, 2012
Really found value in this article... and it was easy to understand, especially for a "newbie" ... Thanks!
Tom Sather 3:37 PM on September 06, 2012
Based on our own analysis here at Return Path, we have found that 78% of issues are due to reputation and the other 22% is due to content. When we looked at content more closely, actual keywords played little role in the filtering. In fact most were due to the actual HTML design of the email. The keywords we saw being tripped were more along the lines of things like "bad credit" and "advance fees"...words that most marketers will never use. FREE never even appeared in the top 20. Ironically, I would say marketers should treat their subject lines with SEO in mind since Gmail is including emails in search results based on the keywords in the subject line (and not so much in the body).
Don Montgomery 12:55 PM on September 07, 2012
Great post.
One nit: Re: "version A did 0.01% better". No, version A did 0.1% better, not 0.01%.
Pamela Vaughan 1:29 PM on September 07, 2012
Thanks for catching that, Don. All fixed!
Melissa Miller 2:11 PM on September 07, 2012
Thanks for the note Don! It was due to rounding. The numbers are now more precise.
Stephan 5:07 PM on September 07, 2012
Count me in with Loren's argument. Plus, it's a long subject line, so you don't know what kind of guide you're getting until the very end of Email A, so it will get cut off in most people's inbox views. Especially on mobile devices, and HubSpot sends a majority of their guide emails first thing in the AM.
A more appropriate split would be [FREE SEO Guide] vs. [SEO Guide].
Eric Chau 5:59 PM on September 11, 2012
I think the open rate is a better indicator in terms of how filters treat the word "Free" The bounce rate only reports any bounces (hard / soft / or spam) from the mail server. You don't know if the email actually reaches a user's inbox as client side filtering (i.e., outlook, etc) or mail servers not reporting the SPAM as bounces will still be reported as delivered in your deliverability report.. Based on your comment that the click through is 17% better for B without considering the open rate and only 5% better when open rate is considered, it actually implies the open rate for B is quite a bit better. It could be the subject line, spam filter (client or server) or other reasons, but I think if anything it shows that one must exercise very good judgment when using the word Free in the subject line.