3 Tests Our Content Team Ran in 2021 & How They Impacted 2022 Planning

2021 was a big year for the HubSpot content team.

Our team grew, we faced challenges, and we had some fun with experiments.

Experiments can teach you a lot about your audience and help you unlock growth opportunities. Here are some experiments we ran in 2021 and what we learned from them.

Conversion Rate Optimization

In January 2021, the web strategy team decided to run a form optimization experiment to understand how altering our forms would affect our users, ahead of a blog redesign.

When evaluating the current form, the team found that it was breaking several best practices relating to user experience (UX). For instance, the form used asterisks when all fields were required and users were unable to tab through the fields, taking longer to go through the form.

This issue with the UX was further validated with high dropoff rates on offer pages. This meant that a secondary goal was clear: Improve the user experience of the forms on our content offer pages, which would then improve our conversion rate.

With every experiment comes a hypothesis and this one had two, one for each goal:

• By redesigning the forms, we will learn the best approach to form design ahead of the blog redesign.
• By optimizing content offer forms’ design and functionality, we will improve the user experience and increase user clarity, which will lead to an increase in content leads’ CVR.

When it came to designing the experiment, the team chose an A/B/C/D/E test, leading 20% of traffic to each of the five treatments:

• 20% would see A, the control – a modal form.
  
• 20% would see B – a redesigned, standard version of the form.
  
• 20% would see C – the split-screen with a two-column form.

• 20% would see D – the multi-step form.

• 20% would see E – the split-screen with single-column form.

Wondering how they came up with these variants? User testing. Earlier in the year, the team had already gathered valuable insights from a user test, in which they learned about users’ preferences and expectations when using forms.

They used this data to design their experiment, something that Principal Marketing Manager at HubSpot Rebecca Hinton says highly contributed to their success.

The test ran on the top 20 converting offer pages and had to run for two weeks to get a reliable sample size for each variant and launched only 33% of traffic to mitigate risk to content lead goals. It ramped up to 60% of traffic during the experiment.

The primary metric used in this experiment was the conversion rate on the content offer form submission and the second was engagement.

"What we found is that displaying a multi-step form vastly outperformed the other treatments we tested, showing a 20% improvement over the control," said Hinton.

It’s worth noting that the winning variant, D, had a much higher conversion on mobile than it did on desktop. However, the multi-step form performed well on both device types.

"The multi-step form was so successful that we’ve decided to implement it ahead of the pages being redesigned, so we can capitalize on its strong performance," said Hinton.

A key takeaway here is that one test can (and should) inform another. If you collect user data for a particular feature or project, make sure you keep good documentation as that can serve a purpose later on.

Channel Promotions

Back in June 2021, HubSpot community manager Jenni Kim, then marketing manager on the Channel Promotions team, ran an experiment to explore opportunities for cross-promotion between the YouTube team and the blog team.

Kim described both channels as having an on-and-off relationship, crossing paths from time to time covering the same content and even collaborating at times. However, there was no consistency. This experiment would assess the value of embedding YouTube videos from HubSpot’s channel into relevant blog posts.

The hypothesis was that adding videos to blog posts would enhance the blog reader’s experience and drive meaningful growth for both channels.

Setting up this experiment required cross-collaboration between YouTube, Blog, and SEO teams to design a process while keeping in mind content lead goals, user reading experience, and SEO implications.

Now, let’s dive into the specifics.

One consideration they had to make was which YouTube videos would go and where. Here’s the breakdown they settled on:

• Existing Blog Post + Existing YouTube Video (9 blog posts, 9 videos)
• Existing Blog Post + New YouTube Video (6 blog posts, 3 videos)
• New Blog Post + Existing YouTube Video (4 blog posts, 4 videos)
• New Blog Post + New YouTube Video (5 blog posts, 3 videos)

They found that using existing blog posts and videos would offer the most clear results, as you could compare pre- and post-experiment metrics.

Key metrics:

• Total organic clicks (TOC) – The blog traffic coming in from the web results tab on Google
• Total organic clicks from the video tab – The blog traffic coming in specifically from the video tab on the SERP.
• Content leads/CVR – The number of viewers who converted through a blog post by signing up for an offer and the view-to-lead conversion rate on the blog on a post-level.
• Organic video views – Videos that came directly from the blog post embed.

The experiment ran for roughly three months, as that would be enough time to measure the SEO impact.

The results found that embedding YouTube videos into relevant blog posts had a positive impact on both YouTube views, contributing 15% of total views. Leads and CVR stayed consistent, which is considered a win, as the experiment didn’t negatively impact conversion.

As for TOC from the video tab, an average of 8% of clicks came from this section – making it a solid SEO opportunity for both channels.

The most challenging part for Kim was the adaptability they needed to exercise to start this experiment.

"A lot of the teams were all working pretty independently," said Kim, "So, we had to understand everyone's processes and then try to find that middle point to bring everyone together."

As a result of this experiment, both teams developed a sustainable collaborative process to help both teams generate more traffic.

Audience Growth

When it comes to blogging, one of the most important metrics is organic traffic. This refers to the non-paid traffic that comes from search engines.

On the HubSpot Blog, the team also leverages non-organic content coming from sources like email, social media, and other websites.

Pamela Bump, senior marketing manager responsible for audience growth on the blog team, shares that the HubSpot Blogs already had incredible success from our non-organic content, creating clickable, shareable blog posts with original research, quotes from experts, and insights on industry trends.

"Not only did our non-organic program help to increase our non-search traffic, but we also were able to write posts that eventually began to rank on search because we were ahead of trends," said Bump.

To capitalize on this success, Bump led an experiment to create hybrid posts, combining the shareability of non-organic content with the SEO of organic content.

"The goal of this experiment was to see if our blog posts could pull in the sudden bursts in traffic from non-search channels that non-organic blog posts achieve soon after publishing," Bump said, "while also gaining more evergreen traffic over time search as these posts begin to rank on search result pages."

She called this the "Hybrid Effect."

"These assignments were laid out by both myself and our SEO strategist and combined SEO elements, like keyword optimizations and search-driven formatting," said Bump, "while still including non-organic elements like quotes from experts, original data, news mentions, and trend coverage."

The result? In the first year of testing hybrid content, Bump says it has led to huge benefits for the blog, pulling in strong numbers comparable to organic traffic.

"The average views we get from hybrids in their first month is about 10% lower than that of non-organic piece," said Bump. "However, it can be more than 30% higher than the first month’s traffic of a completely organic piece of content."

In the long term, Bump says that the average hybrids and non-organic piece published in 2021 only had about a 1,000 to 2,000 view difference while gaining more keywords on average than a non-organic piece and gaining faster traffic than an organic piece in its first year.

Because of this success, the team has made the hybrid model a permanent strategy on the blog team.

"In 2022, we plan to increase our investment in hybrid content by 10% while also training writers on how to optimize organic pieces with non-organic elements for hybrid growth," said Bump.

In every experiment mentioned above, there’s something to learn – whether it’s the importance of collaboration or the value of historical data. If there’s an experiment you’ve been thinking of running, take this as your sign and use these insights to guide you.