Often when a transaction is completed online or a customer service call has ended, we are asked to rate our experience -- on a scale of 1 to 10 or by giving a number of “stars.” Services such as Uber even require a driver rating in order to book another ride.
Many agencies have adopted this quantitative feedback practice, and it is commonplace for agencies to ask clients to rate their agency annually, if not more often. This client feedback is important in proactively managing the agency-client relationship. So, why are we at Hallway Talk, a practice founded on the importance of client feedback, advising agencies to be wary of quantitative studies?
Through our work, we see that results from quantitative-only studies can be dangerously misleading. We’ve been retained by agencies who have conducted such studies for years, and we find that results from qualitative studies often reveal important contradictions. In fact, when we interview clients asking both quantitative and qualitative questions, we get contradicting answers from the very same client in the very same interview. How does this happen?
6 Reasons Why Your Quantitative Feedback Study May Not Be Reporting the Truth
1) Truth has consequences
Regardless of the agency’s performance, agency personnel and clients often form a strong bond. As a result, ratings can be inflated because the client giving the rating is concerned that his honest response may result in hurt feelings or worse yet, get his agency counterpart in “trouble.” To avoid the fallout that a frank but negative rating might create, the client may rate the agency’s people and services higher than what he really feels they warrant.
2) A 5 is not a 5 is not a 5
People vary significantly in how they rate. We’ve talked to clients who speak glowingly of the agency and then rate them a 4. When asked, we hear things like, “No one is a 5. That’s perfection.” Whereas other clients may consistently rate high and only deviate from a 5 if there is a serious problem. Understanding personal bias in rating systems provides huge insight on the scores.
3) But is this really confidential?
Clients have often told us that they are cautious when responding to surveys because they don’t believe that their responses will actually be kept confidential. As a result, they “hold back” and are less critical because they don’t want to be viewed negatively or create a personal issue with the agency. It may sound paranoid, but it is a reality of feedback studies. In fact, if the clients are told up front that the feedback is not confidential, we’ve been told by clients that it affects their scores.
4) Point in time
Let’s face it, the agency-client relationship is almost always changing. Agency turnover and account shuffles affect relationships and performance. Three great projects can be forgotten if one goes horribly wrong. So, when clients are asked to rate the agency -- their people, strategy, creative, delivery etc. -- what goes through their head?
This is an example of where quantitative feedback can be misleading. Through qualitative follow-up and probing, we’ve uncovered ratings that can lead to the wrong conclusion due to timing. Consider this actual client response, “I gave them a 5 on creative because the previous creative director on our business did a great job. I can’t stand the new CD, but it wouldn’t be fair to ding the agency since it did deliver for most of the past year.”
Without further probing, the agency would assume that all is well with creative for this client when in fact it’s not. Understanding the point in time behind the quantitative rating is critical.
5) Scratch that off my to-do list
Some clients take the feedback process very seriously while others view it as a chore. As a result, some client feedback surveys are not taken seriously. People want to get them done and off their to-do list, so they’ll scroll through the survey responding with ratings without even thinking about, or in some instances, even reading the question. Agencies that act on this feedback can make some big mistakes.
6) Optional fosters extremes
We’ve found that when quantitative feedback studies are optional, there are two groups of clients who tend to respond: those who are exceptionally happy or those who are exceptionally unhappy. While this feedback is valuable, it ignores the less engaged clients and skews the results. Optional feedback should be evaluated with that filter in mind.
How to Make Your Client Feedback More Accurate & Actionable
1) Keep it confidential
Clients must be assured that the agency is soliciting their feedback because they value their honest assessments and that their feedback cannot be traced back to them. The trade-off of confidentiality versus honesty is worth it.
2) Add open-ended questions
Open-ended questions or opportunities to comment on ratings “in their own words” will provide the agency with a much better picture of what is behind the numbers ratings.
3) Bring in professional listeners
Consider an experienced third party to do confidential, in-depth interviews with your clients. This qualitative data will uncover your clients’ true perceptions about your agency. Consultants are trained to push the boundaries to discover not only how the client feels but also how others in the agency perceive the agency as well.
4) Be open-minded
It takes progressive agency management to truly listen to and act on the perceptions of the client. Agencies are quick to get defensive and, in our experience, those agencies teams that take the time to understand client perceptions and are nimble and humble enough to adjust are best able to weather issues and retain clients.
5) Close the loop
Feedback is a loop. It is more destructive to solicit client feedback and then not address issues than it is to not solicit feedback at all. Share the results of your findings with your client, and reveal your action plan to address the client issues -- or why some issues will not be addressed. The important thing is to demonstrate that the client has been heard. This not only fosters collaboration but also encourages the client to continue to provide honest feedback in the future.
6) Monitor progress
Remember a one-time survey reflects a point in time. Repeat your study annually, or more often if issues warrant. Use your studies as benchmarks to measure progress and accountability.
Done right, the combination of quantitative and qualitative client surveys are an invaluable client retention tool that helps you proactively manage your client relationships. They allow you to reveal issues that may not have surfaced, reward employees who are valued by the client, and create an open dialogue for continued agency success. Craft your surveys wisely, interpret them intelligently, and make sure you let the client know that they’ve been heard.