How HubSpot Customers Use Subscriptions: Insights From a Product Manager

tl:dr

In late 2018, the product team at HubSpot began investigating our email subscriptions usage data, along with the data in HubSpot’s email product, to examine what subscription types are used for and their impact on communication for HubSpot customers. Our goal was to find a correlation between subscriptions usage and email performance to see if there is any best practice to recommend.

We found that email performance is more or less the same across customer accounts when it comes to open rate; however when examining customers by size, we found that larger customers perform the worst (*albeit by a small margin). We also found that the more subscription types a customer has, the more emails they send (normalized by contact), leading us to conclude that subscription types on their own do not lead to success with email. In fact, reviewing the data feels a lot like this: 

However, instituting feedback mechanisms for contacts to manage their subscriptions and communication more effectively may improve customer<>contact relationships.

Our starting point

If there’s one thing I’ve found as a product manager, it’s that there is a great deal of literature on the internet about getting subscribers, but not quite so much on what to send to those subscribers. For example, here are some search results when googling “types of email to send subscribers”

How many is it Google, 15 or 7?

Not to mention that subscriptions is still very much tied to email despite us heading towards a world where subscriptions can apply to any form of communication from carrier pigeon to telekinesis (though more likely channels like phone, text, or chat).

We worked with our Product Insights team to dive deeper into our customer’s email usage and performance data. We pulled aggregate data from 26K accounts that sent email between October 2017-October 2018. We excluded accounts with fewer than 1000 email sends on the year so as to ensure that the data we pulled was representative of more active companies in the email tool so as to not skew our results.

The ultimate goal of our research was to understand where Email Subscriptions fit in the communication story, and what success with subscriptions looks like. For those customers that do create subscription types, we wanted to see how successful they were across the app, namely in email. Do the customers with 10 subscription types perform better through email than ones with 3?

Our findings

Let’s break our research down into a series of key steps we took. First we looked at subscriptions usage on its own, then in conjunction with email, and finally we dug deeper into email usage and sending habits.

Subscriptions usage:

In analyzing subscriptions usage (ie, number of subscriptions created by a customer) we can see a breakdown by portal in this distribution chart. Recalling that the mode is 3 and the mean is 4.42 this chart makes sense. The bulk of portals fall around 3-5 subscription types, with very few (comparatively) having 10+ active subscription types.

With this overview we can see that subscription types just aren’t much used beyond the minimum particularly often. Our customers may come up with one or two more, but they do not seem to get very creative past that point.

This leads us to our next question: of the subscriptions that our customers are making, what types of subscriptions are they actually creating? We pulled the name of every active subscription type across the portals we analyzed to gather this data. We were able to pull together a breakdown of common words in Excel and the results were not all that surprising:

As expected the top three are

• Marketing Information
• Blog
• One:One

These are defaults so it is natural they would appear the most often. Additionally, we found the other common words used are:

• Event
• News
• Update

Slightly less common were

• Product Information
• Offer
• Resource

Essentially companies, beyond the standard three, are sending email regarding events and industry news, with smaller sample sizes sending emails about offers. This is expected – there aren’t many types of email that a company would send beyond the basics, though at a higher level these six buckets make sense as a benchmark.

We can see how subscriptions are used, and what they are used for, our next task was to examine usage alongside email performance. Perhaps the key isn’t tied so much to the variety of email so much as frequency and relevancy.

Open Rate

To make things simple we’re looking at a scale from 1-12 subscription types given the vast majority of portals fall in that scale.

Going in, my hypothesis was that the more subscription types you had, the better you would be at email. The reason being (I thought) that with more subscription types you were conscientious as to how you structured your email strategy. Instead what we found is that there is no such direct correlation.

In looking at open rates, they actually appear to drop once you get past 5 subscription types, with a slight spike uptick at 12 (likely due to the smaller sample size). Overall though open rate hovers around 20%-30% whether or not you have 2 subscription types or 10. As to why open rates would drop with the addition of subscription types we’ll get to that a bit later, for now let’s look at the other metrics.

By Portal Tier

While the above findings are interesting, they are a bit skewed. Our Enterprise portals email very differently than Basic. The following charts examine the same metrics while taking pricing tier into account. Then we’ll normalize some data based on the number of contacts per portal.

Open Rate

More subscriptions = lower open rates, but how about by portal tier? Between the Marketing tiers, we can see that Basic portals have the highest open rates whereas Enterprise portals have the lowest.

This is both surprising and unsurprising: we know that Enterprise portals are more sophisticated but also with more contacts comes broader mass communication and difficulty in targeting. 

Sends by # Subscriptions

If large customers perform relatively poorly with email, and these accounts are generally more likely to have more subscription types, let’s take a look at how they are using them. When examining sends by the number of subscription types we can see a pretty clear correlation here - more subscription types seems to mean more email.

Is it that more subscription types are the cause for poor performance? Or rather, do more subscription types lead to heavy email usage?

This is an interesting view, though normalizing the data by the number of contacts provides us with greater insight. Here we can see whether or not more subscription types lead to emailing individual contacts more. When normalizing the data by the number of contacts in a portal we can see that as the number of subscriptions increases we do see an increase in sends per contact.

Now, when it comes to portals sending the most email per contact (not just the most in general), we can take a look at a breakdown by product family.

It comes at no surprise then, that Enterprise portals once again lead the pack in sending the most email per contact.

Perhaps then, the key to subscriptions is not simply to provide customers with a method of communication to ensure they do not annoy their customers, but instead to provide contacts with the power and means to say exactly how, when, and how frequently they want to be communicated with.

Next Steps For the Product Team

Armed with our research there is a great amount of work to do in educating customers while setting them up for success with subscriptions. Of course, we’ll also take our findings and work to make the subscriptions feature better and easier to use.

Working with Marketing

We plan on putting these findings to practice so our customers can get themselves in the right habits when it comes to email.

A quick glance through our top blog posts on email marketing shows that, much like the rest of the internet, we have quite a bit of literature on how to style emails, optimize them by client, or write engaging subject lines - but not much on the types of email to send and best ways to target emails. Providing customer-facing materials will be carried out in a series of phases.

Best practice guides in conjunction with new features

As much fun as research is to read research we realize that 3000 word documents aren’t everyone’s cup of tea. We’ll produce a series of guides and posts to encourage best practices in communication so our customers have an easy and approachable resource to aid their marketing efforts.

Working with HubSpot Academy

Coupled with our work with the Marketing teams, we’ll also partner with Academy in producing training materials to guide our customers to a better communications strategy.

Our Academy team does an excellent job of coaching our customers on best practices in marketing principles as well as in the tool. Our goal will be to educate customers on how to develop an empathetic communication strategy that forges good relationships with their contacts. The goal is to nurture subscribers to become customers, not drive them away, after all.

To infinity and beyond

Where email publishing may be the star of the show (have you seen the new Drag and Drop editor?), I like to think that Subscriptions is the best supporting actor (if we had Oscars for Product), meaning that Subscriptions has the opportunity to elevate communication and make it better/more impactful. While it may be a modest takeaway it’s groundbreaking for HubSpot, I see Subscriptions heading to a place where customers are able to optimize their communications strategy and adjust on the fly.

--------

Research Process and Data

For our research we settled on a sample size of roughly 26K customers that had sent email between October 2017 and October 2018. While Subscriptions covers all types of communication, the bulk of our data and history lies in email, as a result the evidence of subscriptions usage will lie in email data and performance.

We focused on several key metrics broken into the following categories:

Customer statistics

• Tier
• Number of contacts

Email performance

• Sends
• Delivered
• Open Rate
• Clicks
• Unsubscribes
• Unsubscribe Rate

Email sends

• Average number of lists per email send
• Median number of lists per email send
• 75th percentile on lists per email send
• Max number of lists per email send

Subscriptions usage

• Number of active subscription types
• The names of the active subscriptions and clusters they fall into