8 Tips for Conducting Cognitive Walkthroughs

I remember when I learned how to drive a car with a manual transmission, or "stick-shift." I knew how to drive an automatic, but when I sat down in the car, I saw an extra foot pedal and a gear shifter that went left to right rather than up and down. Confused, I put the car in first gear without stepping on the clutch, resulting in a ghastly screech followed by the car shutting down.

This was my version of a cognitive walkthrough. But when used in technical applications, there's a much more methodical process than the one I used.

In this post, let's break down what a cognitive walkthrough is and how it differs from other usability tests. Then, we'll provide some best practices you can use when conducting this test at your business.

This form of user testing analyzes how easy it is to operate a product or service. It involves asking participants to complete a series of tasks, then recording the actions they take to do them. This uncovers flaws in usability and outlines how customers are interacting with your products.

During a cognitive walkthrough, participants are asked to follow a series of instructions known as the "happy path." Researchers refer to the happy path as the scenario where the participant doesn't make a mistake and achieves the walkthrough's goal without experiencing any roadblocks. Using this happy path, research teams compare user results against their ideal hypothetical to see how user-friendly the product is.

While each is a type of user testing, cognitive walkthroughs are different from heuristic evaluations and other usability tests. A heuristic evaluation uses industry professionals to measure usability against a set of predetermined principles. In comparison, a cognitive walkthrough simply records the actions that target consumers take to complete a task. This lets product management know if their product meets the needs of their customer base.

Benefits of Conducting a Cognitive Walkthrough

The most significant benefit of cognitive walkthroughs is that they're cost-effective. All your team needs to do is create a series of tasks then record what the user does. You don't need to set up any complex experiments or pay for customer feedback. You don't even need to find external participants as employees from other departments can substitute as your target audience.

Another benefit of this test is that it's flexible. You can conduct a cognitive walkthrough at almost any time during product development. Even when the product is in the design stages, you can run a test to see if customers can use it based on its blueprints. This can save your team from pursuing dead ends that cost your business both time and money.

Now that we've highlighted some of the benefits of this test, you may be interested in conducting a cognitive walkthrough at your business.

Before you begin, take a look at the list of tips below to ensure your test produces successful results.

Tips for Conducting Cognitive Walkthroughs

1. Identify a goal for the test.

If you're having trouble coming up with tasks for the test, think about the purpose of running a cognitive walkthrough. Why are you conducting this test? What about your product or services do you want to learn more about? By identifying a goal, the instructions to hand out become much clearer.

For example, let's say my goal is to see if my website's navigation bar is easy to use. I could run a cognitive walkthrough that instructs participants to visit different pages of my website using my navigation bar. By recording each user's actions, we can assess different aspects of my website's design and achieve our goal of improving its usability.

2. Test with both internal and external participants.

Earlier in this post, we discussed how cognitive walkthroughs are a cost-effective means of user testing. One reason is that you can use internal stakeholders to perform the test, rather than acquiring volunteer or paid participants. This saves you time and money, but it doesn't give you a completely accurate picture of your product or service. That's why you should reserve employees for earlier stages of development when product and design changes are at the least cost to your business.

For the most reliable results, you need to test your product with your target audience. This should be done in the mid-stages of product development when design changes are still cheap. By testing with your target audience, you'll have a direct picture of how customers perceive your product and what they like and dislike about it.

3. Consider the participant's perspective.

When you're coming up with tasks, it's important to consider the participant's perspective. Think about the environment they're testing in and how developed your product is. If it's earlier in its design you may need to make your instructions more specific. Or, you may have to explain how to use the product if it's something customers have never seen before.

4. Create both specific and general tasks.

It's tempting to create tasks that are too specific . Instead, make them extremely easy for customers to complete. Remember, your cognitive walkthrough should be unbiased, allowing the participant to pass or fail.

Tests should include a mix of both specific and general instructions. If the task is more complex, provide some additional details for clarity. Don't be afraid of failing results, as these will only make your product more user-friendly in the long run.

5. Use some internal jargon.

When creating tasks, it's okay to use some internal jargon in the instructions. So long as the task is still clear to the participant, this will help familiarize them with the product and the language they'll need when using it. This is important to customer service teams as they can learn the best way to talk about the product's features when working with your customers.

For example, when I was on the HubSpot Support Team, we would troubleshoot cases involving our Sales email integrations tool. However, if I picked up the phone, a customer wouldn't call it an email integration tool. Instead, they would say their email tracking wasn't working or their connected inbox was down. Over time, I learned to use the same language as our users which led to faster case resolutions and improved customer satisfaction.

6. Review each question before the test.

Before you invite participants to take your cognitive walkthrough, you should evaluate your instructions using four fundamental questions.

1. Will the participant complete the task?

Put yourself in the participant's shoes. Will they be able to complete this task in a reasonable amount of time? This is where you can use an internal employee to gauge the difficulty of each step in your walkthrough.

2. Is the right action clear to the participant?

You should also consider whether there are multiple ways to complete steps and if there's a "right" way to achieve the goal. If so, determine if that path is clear to the participant and remove any distractions that could interfere with the test.

3. Is there a direct correlation between the user's action and the correct outcome?

If there are multiple ways to complete a step, you need to be sure that the correct action leads to the right outcome. There needs to be a direct link between the participant's decisions and the result of the experiment. Be sure to remove any variables that could influence the test and reduce the accuracy of your findings.

4. If the correct action is taken, will the participant move on to the next step?

Finally, this question asks once a step is completed, will the participant know how to advance to the next task? If your walkthrough has a lot of steps, be sure to list each one out individually and avoid grouping them together. This will improve the accuracy of your results and your participants will know exactly how to finish the test.

7. Record results in terms of success and failure.

During the walkthrough, it's important to note whether the participant failed or completed the goal. Recording the results in these terms will help you categorize the data making it easier to assess later. This will also give you a completion rate as well as qualitative information describing the failed attempts. With this format, research teams can quickly analyze data and relay findings to product management.

8. Survey participants after the walkthrough.

Once participants complete the walkthrough, don't just let them walk out the front door. Take advantage of the opportunity and ask them to complete a brief survey regarding their experience. This will give your team valuable qualitative data that will be used later during the assessment period.

Keep your walkthrough's goal in mind when deciding on survey questions. Ask about specific details and encourage participants to elaborate on their experiences. After all, this is your team's chance to determine the reasoning behind user decisions and gain a deeper understanding of customer behavior.

Cognitive Walkthrough Example

A cognitive walkthrough should be tabled as a series of tests that your participants will take when testing your product. In the example below, there are five columns:

• Heuristic: The task being performed and tested.
• Severity Rating: How serious the task is to the end goal of the test.
• Pass/Fail: Whether the participant passed or failed the test.
• Explanations: Why the participant passed or failed the test.
• Design Change Recommendation: How your team can fix the issue and prevent it from happening again.

By outlining the cognitive walkthrough test and results in a similar way as the example, you'll have a clear understanding of how your users will interact with your product when it's released.

Image Source

Conducting Your Own Comprehensive Cognitive Walkthrough

Now that you understand the steps to conducting a cognitive walkthrough, it's time to start your own. Whether you're testing an existing product to increase user adoption or launching a new feature to the market, a cognitive walkthrough can give you the insights you need to release the best version of your product.

Editor's note: This post was originally published August 2019 and has been updated for comprehensiveness.