Perception choice and control in forms vs. manipulation
How much control do you believe you possess over your actions, your thoughts, and the thousands of decisions you make every day, from deciding what to eat for breakfast to deciding how to conduct yourself at your job or how to solve conflicts at work and at home?
Psychologists agree that a sense of control (whether it’s real or imagined) impacts your general outlook on life and your overall happiness and success. A Scientific American study found that the majority of people surveyed (60 percent) believe that free will—the ability to make one’s own choices and decisions—exists. But even if we believe we have a choice, even if we need to believe it to lead a happy, healthy, and fulfilled life, do we actually have any say or sway over the things we do?
Depending on who you ask and what studies you read, the answers are different, but the general consensus on the “How much choice do we have?” question is “Far less than you may think.”
When forms tread the ethical line
There’s quite a bit of convincing scientific evidence explaining why this is the case, but we’re going to look at just one example today that shows how something as tiny as the wording of a question in a form (yep, you knew we were going to tie in forms somehow) can influence people to make a decision they may not have intended to.
In 2009, psychology and economics professor Dan Ariely gave a TED talk entitled “Are we in control of our own decisions?” As one of the examples of how little choice we have compared to how we might perceive our lives, Ariely gave the example of a DMV form that asked about one’s willingness to be an organ donor and compared the findings in two groups of countries that took two different approaches to wording the question.
Among the countries were the Netherlands, France, the UK, Sweden, Denmark and Austria. One group of countries saw far greater rates of organ donor participation while the other group saw a very low participation rate. What set one group of countries apart from the other? Not differing religious or social values, as Dan pointed out first. Nope. The big difference was something far more subtle, and at first glance, almost trivial.
Here’s an excerpt from the transcript of the talk where Ariely describes the two ways that countries introduced the option of becoming an organ donor or not:
The countries on the left have a form at the DMV that looks something like this. “Check the box below if you want to participate in the organ donor program.” And what happens? People don’t check, and they don’t join. The countries on the right, the ones that give a lot, have a slightly different form. It says, “Check the box below if you don’t want to participate …” Interestingly enough when people get this, they again don’t check, but now they join. (Dan Ariely, TED Talks, 2009)
According to the transcript, this anecdote creates laughter from the crowd, and at first it is a pretty funny story. If you think about it, it’s almost a clever ploy on the part of the form’s creators to get people to make what many might call an ethical choice without even knowing it. (For many religions, organ donation is a selfless act of giving.)
Ethical or unethical?
But let’s dig a little deeper. Is it good or bad that DMVs in certain countries to essentially trick people into making such a big choice, one they wouldn’t be around to regret, but still, an important choice that many would argue shouldn’t be made for them
The case for good
Whatever you think about the underlying principle of the DMV questions, it’s almost impossible to divorce the kind of choice involved. If a choice has little to no effect on the life of the person who makes it or if the choice would, in fact, benefit the greater good, would that change your answer?
Hypothetically, using the tactic employed in the DMV form could prompt people to engage in various positive actions, such as donating to charity or supporting a worthy humanitarian cause. Taking a Utilitarian point of view, which prioritizes “the greatest good for the greatest number of people” when making ethical decisions, using the wording of a question to influence more people to take a socially beneficial action is the ethical choice.
The argument for bad
But what if the action that people are tricked/influenced into taking isn’t something they’d approve of? Chances are you wouldn’t approve of that behavior but can think of multiple examples of internet scammers tricking people into falling for their scam or taking a certain action.
While it’s not exactly criminal (though it certainly could be risky if enough people noticed and disapproved of the behavior), this tactic could easily be used to get people to sign up for an email list they’re not interested in. As much as you want people to sign up for your list and engage with your content, is it really worth it if you had to trick them into doing it?
Our take
The debate of how much choice is an illusion aside, we believe an effort should be made on the part of anyone creating a form to give the form recipients as much information and control over their answers as possible. In the case of a form, people generally assume that not taking an action (not checking a box, not selecting an option) means nothing will change, nothing will occur that they didn’t want to happen. Because this is such an accepted convention, we believe it’s the more considerate (and safer) choice.
Imagine presenting someone with the organ donation options in person. You wouldn’t ask “Do you not want to be an organ donor?” right? You’d ask, plain and simple, “Do you want to be an organ donor?” And a simple “Yes” or “No” would be their answer.
What should I look for?
After all this, you might want to take a look at your own forms and think about what your questions are or aren’t prompting the form recipients to do. If they are influential in some way, what are people being influenced to do? Ethical issues like this don’t often have a clear right or wrong answer, and the answer to what you should do in this kind of decision may depend on your exact situation, the nature of the form, the kinds of people you’re sending it to, and a number of other factors.
Here are a few examples in which creator bias can creep into the construction of forms or surface in their responses. Awareness of these behaviors within your data collection campaigns, whether that be quizzes, surveys, polls, or web forms, can help you reduce the likeliness of their occurrence. Knowing that even data results need to be taken with a grain of salt can reduce conflict over polarizing topics and help you to construct forms with multiple versions of questions to eliminate outlier respondents.
| Bias Type | Description | Example |
|---|---|---|
| Selection Bias | This bias occurs when certain groups within the population are systematically excluded from the sample, leading to results that may not be representative of the entire population. For example, conducting a survey on smartphone usage but only sampling individuals who are tech-savvy may lead to biased results that overestimate the prevalence of smartphone usage in the general population. | Conducting a survey on smartphone usage but only sampling individuals who are tech-savvy may lead to biased results that overestimate the prevalence of smartphone usage in the general population. |
| Response Bias | Response bias refers to the tendency of respondents to answer questions inaccurately or misleadingly, often due to social desirability, memory recall issues, or misunderstanding the question. For instance, in a survey about charitable donations, respondents may over-report their contributions to appear more socially responsible, leading to inflated donation figures. | In a survey about charitable donations, respondents may over-report their contributions to appear more socially responsible, leading to inflated donation figures. |
| Order Bias | This bias occurs when the sequence or order of questions influences respondents’ answers. For example, in a customer satisfaction survey, if respondents are asked about the quality of service before being asked about their overall satisfaction, they may rate their satisfaction higher because the positive experience with service quality influences their overall perception. | In a customer satisfaction survey, if respondents are asked about the quality of service before being asked about their overall satisfaction, they may rate their satisfaction higher because the positive experience with service quality influences their overall perception. |
| Framing Effect | The framing effect refers to how the presentation or wording of a question can influence respondents’ answers. For instance, in a survey about environmental conservation, asking “Do you support measures to reduce carbon emissions?” may yield different responses than asking “Do you oppose measures that would harm the economy to reduce carbon emissions?” despite both questions addressing the same issue. | Asking “Do you support measures to reduce carbon emissions?” may yield different responses than asking “Do you oppose measures that would harm the economy to reduce carbon emissions?” despite both questions addressing the same issue. |
| Double-Barreled | These are questions that combine multiple issues or topics into one, making it challenging for respondents to provide accurate answers. For example, asking “Do you enjoy attending our company events and find them informative?” combines two distinct issues—enjoyment and informativeness—making it difficult for respondents to provide a clear answer if they have different opinions on each aspect. It would be better to ask separate questions about enjoyment and informativeness to get accurate responses. | Asking “Do you enjoy attending our company events and find them informative?” combines two distinct issues—enjoyment and informativeness—making it difficult for respondents to provide a clear answer if they have different opinions on each aspect. It would be better to ask separate questions about enjoyment and informativeness to get accurate responses. |
Build better forms
We’re sharing our top defenses against form bias so you can design your way out of creator bias and respondent manipulation.
Selection bias
- Ensure the sample population is representative of the entire target population by using random sampling methods.
- Expand the recruitment efforts to include diverse groups to minimize the risk of excluding certain demographics.
- Use stratified sampling techniques to ensure adequate representation from different segments of the population.
Response bias
- Use neutral and non-leading language in form questions to reduce the influence of social desirability.
- Provide clear instructions and examples to help respondents accurately interpret and recall information.
- Consider using anonymous surveys to encourage honest responses without fear of judgment.
Order bias
- Randomize the order of questions to minimize the impact of question sequence on responses.
- Group related questions together to maintain logical flow while avoiding priming effects.
- Use skip logic or branching to personalize the survey experience based on previous responses, reducing the risk of order bias.
Framing effect
- Use balanced and neutral language to present questions without bias towards a particular response.
- Avoid loaded or emotionally charged language that may influence respondents’ perceptions.
- Pilot test the survey with a small sample to identify and revise any questions that exhibit framing effects before launching the full survey.
Double-barreled questions
- Break down complex questions into single, focused inquiries to ensure clarity and accuracy.
- Use separate questions to address each issue or topic individually, allowing respondents to provide meaningful responses to each aspect.
- Consider the context and purpose of the survey to determine the most effective way to structure questions and gather relevant data without overwhelming respondents.
In summary, tackling biases in form creation is essential for obtaining accurate data. With FormAssembly’s powerful features like conditional processing for skip logic, randomization, and customizable question formatting, users can effectively address selection bias tendencies before a form, survey, or quiz ever goes live. By leveraging these features strategically, users can ensure their forms yield reliable insights for informed decision-making. Want to learn more? Schedule a demo to see the potential of FormAssembly’s intuitive platform, today.
———-
