COMMENTS
Interesting data. Less time spent on each question probably equals less genuine responses (i.e. inaccurate data). It's probably better to break longer surveys up into smaller ones and send them more frequently. I've had the best results with surveys that are 10-15 questions long.
Kip- This is a great topic, thanks for addressing it. Our users have taken in the neighborhood of 300 million quizzes (polls, surveys and personality tests) over the last several years and we have indeed noticed a correlation between length of quiz/survey and completion rate (we don't measure time per question). Part of our observations are that the number of questions is only one factor in determining completion rate. Our data indicates that there is a negligible decrease in quiz completion rate between ~10 and ~18 questions. Roughly we break out quiz length to <10, 10-20 and 20+ questions.
The other important factor that is not addressed is the payoff for the user. Incentives are one way to drive completions, but users expect engagement and the natural hook of results is very effective to driving people through the completion process. A user naturally asks "if I invest my time to take this quiz, what's in it for me?". One item in the payoff is the results- give users the ability to not only see what their answers were, but how they compare to the aggregate population. People crave the comparative nature of what they know or think and the relationship of that to others.
[note: Pangea Media runs a network of consumer destination quiz web sites as well as
SnapApp.com a self serve platform to use quizzes, surveys and polls to build engagement and drive customer acquisition]
Great data. These results show the average time spent on each question, but it would also be interesting to know how much the time/question dropped between the first and last questions. My assumption is that respondents spent more time on earlier questions than later ones, especially for the longer surveys. So I might add to the above best practices list to try and display your most important questions early in your survey, while you have your respondent's full attention.
Interesting topic. One of my main fears when conducting quantitative research is that to make it easy and quick for the user you need to recourse to multiple-choice type questions. This is great when you want your consumers to vote but you don't get insights on why, you don't know how they got to that conclusion, why, or if there is a "but" involved.
I understand the requirement for surveys, but they really do blight the internet. There's nothing more frustrating than being asked to complete one.
Another tip when creating surveys (and this also relates to Guadalupe's comment on the vagueness of multiple choice questions) is to use conditional logic to keep survey questions relevant to your respondent. Applying conditions to your questions will take the respondent along a specific path tailored to them based on their answers. That way you won’t have to include too many questions with “N/A” options and risk losing their attention. More on conditions here: http://bit.ly/aPN3Hj. And even though they take up space, it doesn’t hurt to include optional open response questions to allow respondents to explain themselves.
This is good information, Kip. One suggestion I've got regarding the first presentation of the survey to the reader: Let them know how many questions and how long this will take up front.
Personally, unless a survey states that up front, I won't take it. I've been trapped into the Never Ending surveys on numerous occasions only to become frustrated halfway (1/4, 3/4 of the way, who knows?) and just closing the window.
On a side note, I'm chalking this up to lack of sleep and not enough coffee, when I read your headline I thought your post was going to be on how long to leave a survey online, not how long should an online survey be. ;-)
Another interesting post - though I had also thought it was going to be about how long to leave a survey online. (My thought, no longer than a week - and perhaps that is also too long?) Also always see the inevitable decline in answer quality as the survey goes on - and we never ask more than 10 (also possibly too long?)
Moreover, selecting a target population correctly is essential. Many a times, the population taking the survey feels disconnected to the Survey Topic itself. That may result in itself wrong results due to lack of interest.
www.facebook.com/ClassifiedDuniya
www.twitter.com/ClassifiedDunia
www.classifiedduniya.com
Very interesting post, Thanks. :)
I am totally agree with that. I lead an online survey last November and 50% of respondents were abandonning around the 10th questions. It was a 19-questions survey and this is not too long regarding a marketer point of view but for online respondents it is way too long. I think online surveys are being more like big polls than surveys as people were used 10 years ago. As the world become faster and faster it is the marketer's job to adapt him/her to the market.
Retitle and reissue as:
How Long Should an Online Marketing Survey Be?
Thanks Kipp,
Interesting reading.
I think the availability of tools like surveymonkey and zoomerang has meant that anyone can put out a survey in seconds. It's free. It's easy. You just need an audience. This is a good thing because we can get quicker access to answers.
What we lose though is the intellectual rigour of reviewing surveys to make sure the questions are logical and the results are valid. A lot of surveys seem to have inane questions that don't flow from the previous answers.
Those old market research experts that would take months to come up with the information you wanted had some value.
We sometimes trade off speed of access to data, with integrity of data.
Just a thought.
Tim