How to A/B Test Workflows and Automated Emails

Do you have a 3 month-long nurturing campaign that you want to start testing sometime before next quarter? Or a single email follow-up that you want to optimize? Well, take it off of your holiday wishlist, because you're about to learn how to A/B test those emails and workflows.

One word of warning: the following process is a little bit technical. But by the end of this article, you'll be an A/B testing ninja.

At a high level, here's what you'll be implementing:

You're going to be adding a short script to a form. This script will tag new contacts when they submit that form, and then determines which workflow the contact will receive.  By creating two (or more) workflows, you can send people down one or the other as they submit the form.  And voila - you're A/B testing your workflows and emails. 

For the visual learners out there, it looks like this:

In this example, Trial Nurturing Workflow 1 performed 15% better than Trial Nurturing Workflow 2 (298 conversions versus 259).

The implementation process for this takes 3 steps:

1. Set up the landing pages that will be rotating new leads
2. Set up the forms that will be accepting the submissions to be rotated
3. Set up your workflows that will be tested

Let's start at the top: configuring your landing page where leads will be rotated. 

Step 1: The Landing Page

On the landing page of your choosing, adjust your template by swapping the form area with a Custom HTML area. You're going to be embedding a slightly customized HubSpot form, so you need the HTML area for the customization. 

We'd recommend cloning your landing page template before making any changes to it. Use that cloned copy for the landing page that you've chosen.  You don't want to break any other landing pages that may be using the original template.

Step 2: The Form

Go to the form that you plan on using for your rotated contacts. Start by creating a new property (under Fields > Create new) and call it "Workflow Group." Make the Field type a number property, like in the image below:

Add this new property to the form.  Then make it a hidden field, so that your contacts won't see it when they're filling out the form.  Click on the Pencil icon next to the field and check off the "Make field hidden" box.

Save the form and click on the Embed navigation icon.  Copy the embed code after you set up the appropriate redirect for the form. With the embed code on your clipboard, head back to the landing page with the new template that you created in step 1.

Back to the Landing Page

Click the module to edit your Custom HTML and paste the form embed code into it.

Now for the fun part: customization! You're going to use the code below to assign a random number (minimum of 2, no maximum) to every lead that comes through this form.  This number should equal the number of workflows that you're planning on testing.  If you are testing three workflows, you'll be assigning random numbers between 1 - 3 to each contact. 

Customization #1: the script. Copy and paste the following code in its entirety to the Custom HTML section, placing it above the Form embed code.

<script>
  function setWorkflow() {
    var groups = 2
    workflowGroup = Math.ceil(Math.random() * groups)
    $('input[name="workflow_group"]').val(workflowGroup).change()
  }
</script>

If you have more than one workflow group that you want to test, change the value of the "var groups", or the groups variable to that number. So if you're testing 3 groups, change that line to
var groups = 3.

Now that you've set up your function to randomly assign a "workflow groups" number to your contacts, you need to execute it after the form has loaded. To do that, use this code as follows:

onFormReady($form, ctx){
  setWorkflow()
}

Place this code just below the "formId" line in your Form embed code script, placing a comma at the end of the formId line. The finished product should look like this (without all the red ink, of course):

Again, the only customization you make to the Form embed code itself is the onFormReady code above and the comma that precedes it. Don't change the formId or portalId from what you get on the Embed form code page. That plus the custom script above will set you up to create new leads with randomly assigned workflow group numbers.

Step 3: Workflow Setup

Congrats! The hard part is over.

Now that new leads are coming in with randomly assigned numbers attached to them, you can kick off workflows based on those random numbers, evenly dividing traffic between your different campaigns or emails. Let's set those up.

Step 1: Create the workflows that you want to test. If you want to have two nurturing tracks for new trials, for instance, create both workflows in their entirety. The splitting will happen with the starting conditions.

Step 2: Set up the starting conditions, utilizing the new property "Workflow Group" to start people down one track or the other. Here's what the starting conditions should look like: 

And for the second workflow, just change the number in the Workflow Group requirement.

For optimal results, set the same goal list for both workflows.  This will allow you to see which one attains the best overall conversion rate. By viewing the two workflows in separate tabs, you'll be able to see which emails get the highest click rate, if you're testing them head-to-head.

Have any questions? Leave them in the comments below. Happy testing!