How to Avoid Nonresponse Bias When Creating Surveys

Clint Fontanella
Clint Fontanella



History question. Do you know what it means to plea the fifth amendment?


It means that you have the right to refuse to answer a question because you might incriminate yourself.

While this is mostly used in courtrooms and to spice up mid-day Law and Order episodes, the fifth amendment is subtly at play in the customer service realm as well. We call it, "nonresponse bias," and use it to categorize the group of people who don't answer surveys for intentional or unintentional reasons.

In this post, we'll discuss everything you need to know about nonresponse bias, including what it is, what causes it, and what you can do to avoid it when crafting surveys at your business.

→ Download Now: Market Research Templates [Free Kit]

Response Bias & Nonresponse Bias

Nonresponse bias differs from response bias because it refers to the group of people who don't respond to your survey, while response bias focuses on the participants who did respond.

Response bias also looks at the reasons why people would answer survey questions inaccurately or untruthfully. For example, if a participant felt pressured by a survey maker to say their experience with a brand was great, that would be an example of response bias. But, if the participant felt intimidated by the question and decided not to answer it at all, that would be considered nonresponse bias.

Nonresponse bias is caused by a few different factors that your business should be aware of. Let's review each one, as well as some examples of nonresponse bias, in the sections below.

Common Causes of Nonresponse Bias

Poor Survey Construction

If your survey is too long or too difficult to understand, participants will be quick to abandon it. In fact, research shows that completion rates for surveys drop between 5%-20% when it takes longer than seven or eight minutes to complete it. So, be sure that your surveys are brief, user-friendly, and engaging if want to keep your response rates high.

Incorrect Target Audience

Some people just aren't the right audience for surveys. Maybe they don't have the time to fill them out, or maybe they're just not interested in providing your brand with personal information. Regardless of the reason, it's important to leverage your customer data when distributing surveys and target groups that would be most likely to participate. Or, you can try changing up your survey's trigger and timing if you notice that your primary audience doesn't seem engaged.

Refusal to Participate

For some customers, the very last thing that they want to do is participate in a survey — and, if we're being fair, we've all fallen into this category at one point or another.

In these cases, timing is critical. If a person is adamant about not participating in your survey, then it may not be the right time for them. It doesn't mean that they'll never fill out one of your surveys. Rather, you just need to find the right time and channel to connect with these individuals to gather their feedback.

Failed Survey Delivery

Sometimes the survey doesn't always make it to the recipient. For example, if you're sending a survey via email, it could be lost in a spam folder and the intended participant never sees it. This still gets recorded as a nonresponse and it doesn't help you create a better survey. In fact, if you never find out that the email was buried in a spam folder, you might end up changing your survey's design for no meaningful reason.

The best surveys have safeguards in place that account for this variable. For instance, you can add a tracking pixel to your emails to confirm that your messages are reaching your participants. Or, if you're managing a mail-in survey, you can include a return address for any surveys that don't reach a mailbox. A simple system like this will help you keep track of surveys that were never completed because they never were seen by a survey taker.

Accidental Omission

Another common cause of nonresponse bias is forgetting to complete the survey altogether. It may not be that your survey was dull or irrelevant; instead, it may be that the participant was distracted and pulled away from the form before they could complete it. While it's hard to prevent these incidents, ideally, they should only make up a small fraction of your nonresponses.

Let's see how these causes work in real-life by reviewing some common examples where nonresponse bias can affect your surveys.

Examples of Nonresponse Bias

1. Outdated Customer Information

Let's say you wanted to send a survey to a group of customers who attended one of your events two years ago. While you have all of their email addresses recorded in your CRM, you haven't gathered any new information on them since their last event. When you send your survey, you notice that your delivery and open rates are extremely low. Turns out, your customers have changed their email addresses and aren't checking the inboxes that you have listed under their names.

This would be considered nonresponse bias because while these customers were technically sent the survey, they never had a real chance to interact with it.

2. Requesting Sensitive Information

In this example, you're running a cybersecurity company that's working on a new way to encrypt email addresses. To get an idea of your customers' current security setup, you distribute a survey asking them to provide various passwords to different accounts as well as their reasoning for why they chose those words or phrases.

To your surprise, no one completes your survey. In fact, a few customers even call your support team to report your survey as potential fraud.

This would be nonresponse bias because of the subject matter that you're requesting from customers. While these questions help your company accomplish its goal, it's not fair to ask customers to provide sensitive information especially when it's unrelated to a support ticket or sales inquiry.

3. Forgotten Survey

For this example, let's say you sent a mail-in survey to your customers about a month ago. Your survey's instructions are to submit the form by the end of the month so participants can receive a special promo offer.

Just as the end of the month approaches, a global crisis strikes and your company quickly pivots its marketing, sales, and customer service efforts to adapt to the new environment. You decide to put your survey on hold because customers are more focused on other needs than providing feedback.

This would also be considered nonresponse bias because participants never got a fair chance to submit the survey. If they waited until the end of the month, they wouldn't have had the same opportunity to collect the promo offer as those who submitted the survey earlier.

Now that you're familiar with how nonresponse bias can affect your surveys, let's review what you can do to avoid it.

1. Reconsider Your Survey's Trigger and Timing.

One of the most effective was to increase survey participation is to reconsider its trigger and timing. You want to present your survey at a moment where participants will be most likely to help and it should seem like it should only take a few minutes to complete. Approaching people at the right time and in the right way will make it easier for you to encourage them to participate.

Your customer journey map is an excellent resource to use when reviewing your survey's timing. You can look for moments of delight where customers would want to share an experience, or you can look for pain points where people would want to voice their criticisms. Targeting these memorable interactions is the key to approaching participants at the optimal point in their customer journey.

2. Optimize Your Survey's Design.

Survey length can be one of the biggest factors that determine whether participants fill out your survey. The ideal survey length is no more than 30 questions long and should take roughly seven or eight minutes to complete. Anything longer than that will lead to lower response and completion rates.

Another important factor to consider is your survey's design. If you stack questions on top of each other, viewers may be intimated by all of the scrolling they'll have to do just to reach the end of the page. If you have access to customer feedback software, you can trigger questions one at a time, so customers can focus on each one individually and won't be overwhelmed by a 30-question list.

3. Review Your Survey Questions.

The type of questions that you're asking will also influence participation. Long-response answers and comment boxes get exhausting over time, and double-barrel questions, like the example below, can be confusing for participants and lead to abandonment.


Image Source

Your survey should primarily include close-ended questions like multiple choice or Likert scales. These give customers a fixed number of responses to choose from which makes the survey much easier to fill out.

4. Leverage Customer Data.

Targeting the right participants for your survey can also play a major role in its success. If you have a CRM, you can leverage customer data and send your survey to people who would be more willing to complete it.

For example, buyer personas would be very helpful in this situation because you can use them to identify target audiences that would be more likely to participate in a survey. You can also look at past interactions with individual accounts to see if anyone recently interacted with your brand and may want to provide feedback. Not only can this help you attain valuable insights from your customers, but it can also help you connect with accounts that are potentially at risk of churn.

5. Provide Options for Omission.

While your survey should primarily offer closed-ended questions, it's important to include options to omit individual answers as well. You can either not require all questions to be answered or you can provide a dedicated multiple-choice option that participants can choose for omitting each question.

Here's an example.


Image Source

When asking for feedback, your customers shouldn't feel pressured or forced to provide information. Most would rather abandon your survey altogether than willfully answer a question that makes them feel uncomfortable. So, including this omission option provides a safety net for those who may want to skip a specific section. After all, it's not worth losing all of their answers over just one survey question.

6. Reward Customers With Incentives.

A sure-fire way to boost engagement is to offer incentives for completed surveys. This could be a discount or a small prize that's obtained once a participant fills out your form.

If you're not sure what to offer, a good place to start is with your customer loyalty program. You can borrow some of its rewards or promotions and apply them to your survey. Or, if your loyalty program is point-based, you can offer participants loyalty points that can be added to their accounts. This is a great way to acquire new signups for your loyalty program and increase overall customer retention.

Once you've handled nonresponse bias, you can continue to master different types of surveys by reading this guide to questionnaires.New Call-to-action

Related Articles

We're committed to your privacy. HubSpot uses the information you provide to us to contact you about our relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our Privacy Policy.

Free Guide & Templates to Help Your Market Research