What is user testing?
User testing is the process of getting end users and customers to test and evaluate a product, feature, or prototype. Test subjects use the item and identify various pain points and positive attributes so the business that creates it can enhance its user experience before going to market.
Why is user testing valuable?
User testing is a simple way to gauge the success of a product, feature, or prototype from the point of view of one of your customers because it allows you to watch, hear, and review their interactions with your design.
While your employees, designers, and other people involved in the creation of your product, feature, or prototype might find it to be easy to use, your target customers may not. User testing allows you to see where your product falls short for your intended audience. It tells you where they get confused or frustrated. Since user testing takes place prior to releasing your product, you can use this information to iterate on the product until you’ve created a user experience (UX) that you’re proud to stand behind.
Now, you might be wondering how to perform user testing on your product. Well, every product, prototype, and feature is unique — as is every company — so there’s a variety of different methods of user testing you can employ.
We’ve compiled this list of five commonly used user testing methods to help kickstart your product testing.
1. Usability Testing
Usability testing is when you provide a real customer or member of your target audience with a product, prototype, or feature so they can assess and test its usability. Usability refers to different factors like a product’s ease of operation, usefulness, and utility. Usability testing allows you to determine how intuitive your product, prototype, or feature is in the eyes of your real customers.
You might be thinking usability testing sounds a lot like the definition of user testing we covered earlier — if so, you’re not wrong. In fact, some people use the terms interchangeably. However, for the sake of this piece, we’re going to differentiate them and continue with the notion that usability testing is a method of user testing. Here’s why:
User Testing vs. Usability Testing
There are some underlying factors that differentiate user testing and usability testing.
User testing covers the entire range of user experience a customer has with your product, prototype, or feature. This includes all of their perceptions, emotions, preferences, responses, and behaviors in response to that item from the moment they obtain it to the moment they stop using it.
Usability testing is a method of user testing that focuses on the way that — and the extent to which — a customer can use your product, prototype, or feature to accomplish a specific goal. This plays a part in the user experience but is not the entire experience.
Here’s a visual to help differentiate between user testing and usability testing.
If usability testing sounds like it might be of interest to you and your team, there’s one other factor to consider: whether you’ll run a moderated or unmoderated usability test.
Moderated vs. Unmoderated Usability Testing
Moderated usability testing involves one of your employees sitting down with a real user, explaining what they want that user to do, and then listening to their feedback during and/or after their use of the product. Unmoderated usability testing doesn’t involve any communication or interaction with the user. Instead, you’ll likely have a video recording of the user working with your product, prototype, or feature and then another video with a review of their interactions and their assessment of the item.
Moderated usability testing is great if you want to watch customers use the item in real-time to ensure you’re getting the information you need and want to obtain. Unmoderated usability testing is a good option if you want to test a large group of real customers from several different areas at the same point in time.
When to Use Usability Testing
Usability testing is ideal during the early to mid design phase of a product, prototype, or feature’s life. That’s because you can ensure each aspect of your design is working appropriately for your customers as you go and prior to investing more time in its construction.
Surveys are a great form of user testing if you want to obtain real, quantitative data from your customers about your product. You might ask them to complete a survey about your new product, or your latest update to an existing product. And since you create and ask the questions, you can be as broad or as granular as you want.
Additionally, surveys are a simple way to obtain feedback because your participants can complete and submit them from anywhere, via any device. They’re also easy to send out to large groups of participants at once and pull data from once submitted.
When to Use Survey Testing
Surveys are ideal if you want to receive a wide range of responses regarding your product in a short amount of time. Surveys also make it easy to dive deep into details about specific features because you can ask your test subjects tailored questions of your choice. They can also help you obtain, organize, and analyze both quantitative and qualitative results quickly so you can begin improving your product for your customers.
3. A/B Testing
A/B testing is when you split your test subjects into groups and test different versions of a product to determine their preferences. This way you can understand which version best meets your customers’ needs and provides them with better user experience.;
A/B testing allows you to share different versions of a specific aspect of a product with your customers and learn what really works for them rather than just guessing based on your buyer personas and information about your target audience. Just because you may think you know your customers well doesn’t mean your test subjects won’t surprise you in an A/B user test scenario.
When to Use A/B Testing
A/B testing is ideal if you want to get feedback about which version of an aspect of a product is most popular among your target audience. For example, you might conduct an A/B test for a CTA button on one of your landing pages with slightly different wording. This way you can learn which option gets you more clicks and conversions.
4. Focus Groups
Focus groups are where you sit down with a small group of real users (usually 10-12 people total) to discuss various attributes of your product, prototype, or feature. The group discussion is usually led by one of your designers, researchers, or other employees to ensure you’re receiving your desired type of feedback. A focus group discussion will typically last an hour or two and cover the test subjects’ concerns about a specific aspect of a product that’s brought up by the focus group facilitator.
For example, if your company is testing a part of your software that’s new or updated, you might have one of your web designers or engineers facilitate the discussion to ensure your test subjects provide you with details about their interactions with that specific aspect of the product. Your focus group leader can ask questions about the updated portion of the software with your participants. This cuts out all unnecessary fluff in the discussion and makes for a straightforward and impactful discussion.
When to Use Focus Group Testing
Focus groups are a flexible form of user testing because you can hold a discussion at any point in your product’s lifecycle. They’re ideal for getting a deep understanding of what your customers want from and hope to get out of your product through a guided discussion. However, if you do hold a focus group discussion, it’s recommended that you hold multiple other focus groups, too — this is because you only chat with a small group of your customer base during a focus group discussion.
5. Beta Testing
Beta testing happens during the last stages of your product, prototype, or feature’s design. It’s when you provide your customers and target audience with the item you’ve created in a state that closely resembles what it’ll look like upon distribution. Beta testing is a great way to get a final thumbs up on your product from customers prior to going to market.
When to Use Beta Testing
Beta testing is ideal if you’ve pretty much finalized your product’s design and want to conduct one last review of it prior to distribution. This way you can make any last-minute changes to enhance its UX if necessary and ensure it’s ready for your customers.
There should be other user tests that occur throughout your design process, prior to a beta test. That’s because this is only used for final and critical updates to the product — so you’ll want to get your customers’ opinions prior to the final stage of design. This way, once you reach the time for beta testing, you’ll likely only have minor changes that need to be made (if any at all).
Now that you have a better understanding of the various methods of user testing, let’s talk about how it works so you can begin testing your products, prototypes, and features.
How does user testing work?
- Create a product, prototype, or feature.
- Decide on a method of testing.
- Recruit people to complete the test.
- Determine a location to conduct the test.
- Decide how long the test will take to complete.
- Distribute the test.
- Analyze test results.
1. Create a product, prototype, or feature.
The first step in user testing is fairly obvious — you’ll need a product, prototype, or feature to test. You may choose to test a brand new product, a new version of a product, a sample, or a feature you’re thinking about adding to an existing product.
2. Decide on a method of testing.
Once you’ve designed your product, prototype, or feature, it’s time to determine which method of user testing you want to implement. To do this, be sure to consider which test method is best suited for the type of information you’re looking to obtain and how quickly you’re looking to receive that feedback. This way, you can get the results you need in a way that makes sense for your company and the product being tested.
Once you’ve decided on your method of testing, you can dive a little deeper. Consider which factors you’ll be looking at and testing for so you can ensure you have the type of feedback you want and need from customers. Here are some examples of things you can look for and incorporate during testing no matter the method you’ve chosen:
Problem discovery is a process in which your customers and test subjects surface various issues regarding the usability of a product, prototype, or feature. You may have them do this while they complete a given set of tasks with the item. Additionally, you might choose to use the system usability scale, a click test, or a heuristic evaluation to get deeper answers and results related to the problem customers have discovered.
Learnability is when you observe how difficult it is for a test subject to complete a task with, or within, the given product. You’re able to watch test subjects work through a specific task as many times as they need until they can complete it with ease. If the learning curve is shallow, they may only need to work through a task once. However, if the product has a greater learning curve, they might need to do this in multiple attempts.
Comparing two similar products, prototypes, or features can help you determine whether or not you’re building a better version of a product for your customers. The products being compared can both be created by your company or one of your company’s products can be compared to a similar one created by a competitor.
For example, you wouldn’t have test subjects compare new, high-tech running shoes to a photo editing software — this wouldn’t be helpful for anyone involved. Instead, you’d have testers compare your photo editing software to the older version you initially sold them, or that of a competitor’s, like Adobe Lightroom. This allows you to get real feedback and context around the ways you can enhance your software’s user experience and improve its capabilities.
You can have a group of your test subjects provide you with a benchmark to build off of and improve your product from. Subjects will use your product, prototype, or feature and provide you with advice on how to enhance it. Once you make those modifications, you have the subjects re-test to see how well you improved the item.
When you use video to watch a test subject’s eye movements throughout their use of your product, you’re using the testing technique of eye tracking. This is common for businesses testing software because you can easily see where that person’s line of sight goes on a screen.
3. Recruit people to complete the test.
Next, you’ll need to find and recruit people to complete your user test. No matter your chosen method of testing, user testing calls for real customers, real members of your target audience, and individuals who fit your buyer personas and empathy maps to complete the test. This allows you to get accurate data and results from the people who matter most — the ones who will be buying, using, and promoting your products. (We’ll go over how and where to recruit test subjects if you don’t already have a group.)
4. Determine a location to conduct the test.
Once you have your product, method of test, and test subjects, you’ll need to determine the time and location of your test. Common locations for user tests to be conducted include the following:
- Community space
- Your office
- Test subjects’ homes
This decision depends on what your product, prototype, or feature is, what it’s used for, and what its capabilities are. When choosing your test location, be sure to consider all of your subjects’ surroundings and any possible distractions they might encounter.
For example, if Airbnb was having users test their new, online vacation rental sign up process, they might have their test subjects do this from the comfort of their own homes. This is most likely where their customers would be working through these forms in real life. Also, with the ability to watch users work through the form via webcam and a screen sharing software — like Zoom — there really wouldn’t be a need for the company to ask test subjects to come into a community space, sit down with their computers, and work through the vacation rental sign up process.
However, if Nike were conducting a user test for the newest version of their Metcon weightlifting shoes, they may conduct the test in a gym with equipment subjects can use to lift, squat, and box jump.
Have you ever been asked to complete a task you know fairly little about? One of the first questions you might have is, “How long is this going to take?”
That’s most likely going to be something your test subjects are wondering about your test, too. After all, they’re the people who are taking time out of their days to complete the tasks you’re asking them to work through.
Different user tests will take different amounts of time. Try to stick around 30-60 minutes per test, per participant if possible. Longer tests have been said to cause a drop-off in the quality of responses and feedback received from test subjects due to fatigue, boredom, and lack of interest.
If your test has multiple parts or requires participants to go back in and work through a test more than once, be sure you allow for sufficient time to do this — whether that means conducting a few short tests repeatedly or allowing your subjects to take breaks between tests. Either way, just be sure to communicate with your test subjects throughout so they feel informed and cared for — remember, these are members of your real target audience, so you want to leave a great impression on them.
6. Distribute the test.
Now it’s time to distribute your test. Be sure your subjects have all test instructions they need including directions on how they are to provide you with their feedback and analysis.
Depending on your product, company, industry, and test method of choice, the various directions and pieces of information you’ll communicate to your test subjects prior to and/or during the test will differ. However, let’s review some points that are commonly reviewed by companies prior to and upon distribution of the test:
- Make sure they know what you’re trying to achieve with the user test so they can focus on specific aspects of your product and/ or can understand why you’re testing the product in the first place.
- Ensure they know how long the test will most likely take to complete so they’re aware of the schedule and what’s expected of them.
- Ask if they have any last minute questions prior to taking the test.
- Make sure they have all necessary equipment. For example, does their laptop have a webcam and is there screen share software downloaded?
- Ask if the participants need any unnecessary distractions removed from the area?
- Ensure the participants understand how you’ll expect them to provide their feedback and analysis after the test (verbal, written, or through a form).
7. Analyze the test results.
The final step is the most important: analyze your results. These results are what you conducted your user testing experiment for in the first place — to obtain critical feedback from real customers to improve your product and enhance its UX.
Consider everything your test subjects have to say, look at what the majority of them have concluded, and don’t be afraid to go back to the drawing board with minor or drastic modifications and changes, if necessary. After all, that’s what the process of user testing is for — perfecting your creation and design before it’s sold.
Here are some common user testing metrics that are reviewed upon the completion of a user test:
Task completion is when a test subject is able to indicate they’ve successfully completed the test or the end goal associated with the test.
Critical errors prevent your test subjects from having the ability to successfully complete your test.
Non-critical errors are made by the test subject yet don’t impact their overall ability to complete the test successfully. These errors typically mean a part of your product, prototype, or feature caused the test subject to be less efficient at a point in time.
This is the percentage of test participants who can complete the test without any errors, whether or not they made errors along the way.
Time On Task
Time on task is how long it takes your test subject to complete your test.
These are the opinions of your test subjects such as their likes and dislikes, recommendations, personal feeling of satisfaction upon completion of the test, and simply, their level of enjoyment while using your product.
Once you have your results and have associated them with the metrics listed above, you can analyze them in depth, look for patterns, and find trends among your participants. You should also consider reporting on the severity of the problems that surfaced throughout your tests. This will help you determine how imperative it is to rectify specific issues, as well as decide the order in which you should fix the issues (if there are multiple).
Here’s a scale to help you determine this:
From here, you can compile your findings and recommendations and turn them into actionable takeaways to help you improve your product. You should organize these findings so you can share them with your team members, designers, and anyone else involved in the product’s creation and development.
Some pieces of information to include in your list of findings and recommendations include:
- The number of people who completed the test successfully.
- The average amount of time it took test subjects to complete the test.
- Helpful participant comments, opinions, and recommendations.
- Specific details about the average point in time when most test subjects ran into issues.
- Positive things about your product — although user testing is used to identify issues with a product, it’s important to highlight the great things about it, what your test subjects enjoyed, and what they liked about it as well.
How do you recruit test subjects who represent your buyer personas and target audience to complete your user testing? Here are three main ways to do this:
1. Recruit existing customers.
A great way to recruit for user testing is by searching through your current customer database for contacts who are willing to help. You can also check your email subscription list or look for customers who’ve left you reviews and ratings on sites like Google and Facebook.
2. Recruit through your network.
You can also recruit test subjects through social platforms, like Twitter and Instagram, as well as ask for referrals from your fellow employees and people within your professional network.
3. Recruit through user testing services.
If you’re working on a large scale project and your business has the resources, you might choose to sign up for a user testing service. These third party sites connect your business with the right test subjects for your specific situation. Here are a few of the most popular user testing service sites to consider.
- Usertesting.com connects you with real customers, allows you to watch and hear them interact with your products in real-time, get their direct feedback, and share your results with your team.
- Validately.com offers recruiting services to help you find your ideal test subjects. It includes a dedicated testing platform to ensure obtaining results is simple and has automated reporting so you can analyze, manage, and organize your feedback.
- Verifyapp.com allows you to send out tests to gain insight into your customer reviews in real time to achieve actionable results.
Begin User Testing
User testing is a powerful tactic for all businesses to take advantage of because it provides you with real, actionable data that comes directly from your customers and members of your target audience. It allows you to ensure the design, UX, and all other aspects of your product, prototype, or feature, are ideal for customers prior to selling it. So, prepare your new product, determine which method of testing works for your specific situation, and recruit real customers to help you improve your creation and enhance its UX.
Originally published Apr 24, 2019 7:30:00 AM, updated July 12 2019