Optimization Testing: Don't Obsess Over Conversion Rates

Download Now: Free Marketing Plan Template
Michael Redbord
Michael Redbord



This is a guest article written by Linda Bustos, director of eCommerce research for Elastic Path Software . Linda is chief blogger at the Get Elastic Ecommerce Blog , the Internet's most read eCommerce blog. You can find Linda on Twitter @getelastic and @roxyyo .

Like this post? Register for the upcoming HubSpot & Get Elastic Webinar on 6/9 at 4pm ET: Lose Your Gut (Feel) in 60 Minutes: Site Optimization Testing Boot Camp

Are you measuring the right thing with your website optimization tests?

One of the biggest mistakes you can make with A/B or multivariate testing is to have conversion rate myopia. It can cause you to draw the wrong conclusions about success, and apply the wrong elements to your site which can actually cause you to lose money.

While it’s always important to measure conversion rate improvement with any test, it’s not the only (and not always the most important) metric to consider in testing.

What does conversion rate mean, anyway?

There are different data points you can use to construct a conversion rate, but essentially it is expressed as the number of conversion actions (sales, completed forms, Facebook Likes, email sign ups, etc) divided by the number of visitors. Where conversion rate can become convoluted is when you have to decide whether a conversion rate applies to total visitors or unique visitors, or to a particular traffic segment (e.g. exclude customers outside of your shipping area).

Know a Test Page’s Goal

For most folks, the conversion goal measured is sales. So a home page test would be tied to ultimate sales. But is it fair to hold, for example, a banner on a home page responsible for higher or lower conversion percentages? The home page is simply one page in a long conversion funnel. Unless you can purchase directly from the home page, it does not influence ultimate conversion. What a home page  can do is entice the visitor to stay on your site for more than one page view. This means a successful home page test reduces bounce rate and wins a higher percentage of clicks deeper into your site.

Some sites’ conversion goals are a completed “contact us” form, for example. In cases where there is a very short conversion funnel, home pages may have more influence on the holy-grail-conversion for your business.

Understand the Relationship Between Metrics

It’s always important to consider the revenue improvement, not just conversion rate improvement. Removing product recommendations from shopping cart pages, for example, may improve conversion by 20%, but if average order value fell by 30%, it’s not a winner.

Other examples of analytics relationships:

  • Price and promotions test: More people bought, but at a lower price – profit did not increase
  • Merchandising test: More people bought, but items per sale were lower – profit did not increase
  • Email price, promotions or coupon test: More people purchased, but a percentage would have purchased anyway without the discount
  • Pre-checked email opt in test: More people signed up for email, but reported your messages as spam because they signed up unwittingly
  • Cart button test: More people initiated checkout, but abandonment the same because the real problem lies in the funnel
  • Remove negative reviews test: More people purchased the item, but more tried to refund because item wasn’t explained truthfully, negative reviews suppressed
  • Banner ad test: More traffic was driven to your site, conversion rate decreased, revenue increased

The Other Other Metrics

The cure for conversion rate myopia is to embrace the other important metrics, like:

  • Revenue
  • Revenue per visitor
  • Average order value
  • Items per order
  • Gross margin
  • Margin per visit (profit)
  • Margin per customer
  • Revenue less returns
  • Return rate
  • Repurchase rate
  • Lifetime customer value

The challenge is, some of these gold nuggets of data are not available from your analytics tool out-of-the-box with Google Analytics. (Some tools like Yahoo Web Analytics and Omniture allow you to import COGS - cost of goods sold - for example). You may need to integrate your data sources together to get a clearer picture. At the very minimum, you should be looking at revenue, revenue per visitor and average order value.

Related Articles

We're committed to your privacy. HubSpot uses the information you provide to us to contact you about our relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our Privacy Policy.

Outline your company's marketing strategy in one simple, coherent plan.

Marketing software that helps you drive revenue, save time and resources, and measure and optimize your investments — all on one easy-to-use platform