A bad dream for those crafting questionnaires for online surveys might go as follows:
Panel members take an online survey critiquing a previous online survey they participated, and that you wrote. The results are overwhelmingly negative. The panel members hate it and admit they offered slapdash information that will not assist in research.
You don’t have to wake up. It hasn’t happened…yet…but if you want to avoid this reality it’s wise to understand two concepts in order to craft the best possible questionnaire. The first one is:
Also known as survey fatigue, respondent fatigue basically refers to the mental state when respondents become weary during an online survey, to the point their answers are rushed and even dishonest—thereby reducing statistical accuracy. Furthermore, an article in Great Brook explains that:
Fatigue means that those with extreme views are more likely to respond, leading to a serious survey bias, known as non-response bias. That is, those who don’t respond likely have different views from those who do. The survey data that winds up in the survey data base don’t properly reflect all customers’ views. The data are biased.
Respondent fatigue can be seen as more of a direct threat than just a negative aspect of online surveys. The New York Times reported that respondent fatigue has caused “declining response rates over the last decade.”
A chief reason for respondent fatigue is the length and wordiness of a survey, and science supports this. A study found survey fatigue often sets in after 20 minutes of a survey. It states: “They found survey respondents exert less effort and spend less time thinking about their answers as respondents get deeper into the survey.”
Here is a breakdown chart from research on revealing the risks of longer surveys:
As qSample’s own Director of Business Development, Connor Duffey, said: “Respondents will look at the scrolling bar during a survey. If it’s not moving fast enough for them, respondent fatigue becomes a self-fulfilling prophecy.”
Duffey further said that a sound way to get around respondent fatigue—even if a questionnaire has to be extensive —is by simply placing the most relevant questions at the beginning. Also, never taking incentive to respondents for granted is also important, Duffey added.
Length, clarity, and approachability are effective tools to combat respondent fatigue, but ultimately are not enough. The second notion is essential:
Putting yourself in a respondent’s shoes goes a long way into getting into their minds and hearts. Sharing their interests is not enough, but understanding that online respondents are real people with busy lives and little time (just like you). They are just as essential as those above you seeking data, and just as part of the process.
qSample’s own president Rudly Raphael encapsulated this issue in a recent interview:
Companies need to have more empathy for the research participant. The person(s) who writes the survey instrument should ask themselves if they could sit through that survey for 25-30 minutes. Companies should make surveys fun and engaging, regardless of the topic. They should test their surveys over and over again to identify the fatigue points in the survey. This is usually the area where data integrity is compromised.
There are sensible solutions that qSample, along with other data research companies, proffer to cultivate empathy with respondents:
– Be communicative from the beginning—as in explaining the number of questions, privacy policies, and purpose of the online survey. Communicating survey results during the survey is also an effective of holding respondent interest.
– Use more than one panel for the same survey or even future ones to ensure they are fresh. Often, companies pass around panels to different departments, and it wears them out.
– Ask yourself if every question is absolutely necessary—if you were to personally take it in a real-time event (on your smartphone, waiting for the train, for example)—and get rid of anything else.
In addition, there are practical techniques that can increase empathy and improve the overall architecture of a questionnaire. For example, Duffey states that minimizing matrices questions is a vital tactic to producing efficient questionnaires, as they tend to exhaust respondents and produce sloppy answers. Another expert in the online survey industry called matrices: “The laziest type of survey writing.”
An additional practical technique is including more opt-out choices, as explained by market research expert :
This is usually in the form of a “Don’t Know,” “Not Sure” or “Undecided.” Not only will adding the opt-out choice eliminate a lot of inaccurate answers from your study, but it will also provide you with valuable information. You can learn how many people have not made up their mind or are uneducated on a topic.
Of course, there are other issues that compromise online surveys—such as response acquiescence (the tendency to agree with survey questions regardless of content) and respondent bias (the inability to answer survey questions because of perceived social pressure). These issues can be moderated by various means that online survey providers should be able to recommend; they are commonly addressed by tweaking questions and providing some safeguards in the programming of the questionnaire software platform.
However, having empathy is an alpha and omega for the best possible questionnaire, certainly for decreasing respondent fatigue. This should not be surprising, in the end, as empathy is the solution to so many other problems, in both the virtual and real world.