There are many different methodologies to gain a representative overview of opinions in society. These can, for example, be used to prove a hypothesis you set up in your academic writing or to get feedback on services. While you can always conduct interviews with each participant individually, it may be quicker and cheaper to rely on questionnaires. This research tool consists of a set number of questions that are asked to participants, often with multiple-choice options to answer. The following article will provide all the relevant information about of questionnaires and how to use them.
Questionnaire – In a Nutshell
A questionnaire is a set of questions used to get specific information from the respondents.
Definition: Questionnaire
A questionnaire is a research tool consisting of structured questions to collect specific information from respondents. They enable researchers to understand patterns, behaviors, and underpinning factors in a more systematic and standardized manner, providing valuable insights into diverse topics.
Questionnaires vs. surveys
A questionnaire refers to the batch of questions that are asked to the participants. A survey, on the other hand, includes the whole process, including sampling participants, the questionnaire, and the analysis and interpretation of the results. So, a questionnaire is a part of a survey.
Methods
There are two different types of questionnaires, a self-administered one and a researcher-administered one. While the self-administered questionnaire is more common, both types have their pros and cons.
- Self-administered questionnaires consist of a fixed set of mostly simple questions, which are identical for each participant.
| ✔ Advantages | ✘ Disadvantages |
| Cost-effectiveness | It is limited by literacy and verbal communication skills |
| Ease of administration to large groups | It may be prone to nonresponse bias where some people fail to complete or submit altogether |
| They are apt for sensitive cases | It relies on volunteer responders since impersonal questions often elicit limited responses |
| They can be adjusted for specific timeframes |
- Researcher-administered questionnaires are usually conducted by the researcher themselves on the phone, in person or online live with the participant.
| ✔ Advantages | ✘ Disadvantages |
| It helps to ensure the most representative sample | It can be expensive and time-intensive |
| It allows for the explanation of unclear questions and responses | It may be hard to analyse large amounts of qualitative data |
| It offers more answers as respondents may feel obligated to respond in personal interaction | It may be susceptible to undue influence by the researchers |
| It is prone to incorrect responses as respondents may have social desirability bias |
Open-ended vs. closed-ended questions
The questions in a questionnaire can be phrased open-ended or closed-ended. As each of the types of questions has their advantages, it may be helpful to balance them in your survey in order to get the results you strive for.
Close-ended questions
- Closed-ended questions are also called restricted-choice questions because these include either yes/no answers or multiple-choice ones with fixed options to choose the answer from. The benefits of this type are that the questions are easy to understand and quickly to answer. On the other hand, participants lack the ability to fully express their opinion or perspective. This problem, however, can be solved by giving an extra space below the question for an alternative, individual answer.
Variables in close-ended questionnaires determine the type of statistical analysis to generate conclusive findings. These variables include:
- Nominal variables:
These are categories that can be ordered, such as religion or race.
Example
What is your religion?
- Christian
- Hindu
- Islam
- Judaic
- Buddhist
- Ordinal variables:
Include ranges and other categories that can be ranked within an array.
Example
How many years have you worked here?
- 10 or longer
- 7-10
- 5-7
- 2-5
- 2 or less
- Likert-type questions:
They use an ordering scale with between 5 to 7 points.
ㅤ
Example
Do you consider yourself an introvert?
- Strongly agree
- Agree
- Neutral
- Disagree
- Strongly disagree
Open-ended questions
- Open-ended or long-form questions allow the participant to elaborate on their answer and phrase it in their own words. While open-ended questions do give the respondents more freedom and thus comfort to get their point across, they are also more time expensive to analyse afterward. The results cannot be compared to one another easily, which is why for most questionnaires, closed-ended questions are more commonly used.
Examples of open-ended questions
- What is your biggest motivation?
- Do you believe in life outside earth?
Questionnaire: Question-wording
Question-wording can affect the respondents’ responses to a set of questions.
Use clear language
While phrasing the questions, it is important to keep the language simple and clear for your participants to understand. You may also consider the level of familiarity with your respondents and adjust the wording accordingly. For example, phrasing questions targeting university students might be more casual while the questionnaire for business companies needs to have a certain level of professionalism.
Balanced framing
Questions can always be phrased positively and negatively, which can highly affect the answers of your participants and lead them in a certain direction.
Example
- Positive frames are considered, for example: “Should schools be less strict with bullies?”
- Negative frames are perceived to be less neutral. For example: “Should schools be stricter with bullies?”
Avoid leading questions
Make sure that your questions are not leading to specific answers so that your results stay valid.
Example
Do you agree or disagree that traffic offenders should pay a fine and go to jail?
- Strongly agree
- Agree
- Neutral
- Disagree
- Strongly disagree
Keep your questions focused
Especially when asking open-ended questions, it is essential to stick to your topic and not drift off due to the participant’s answers. Furthermore, you should avoid double-barreled questions that might confuse or lead the respondents in a certain direction.
Example
Some experts say that a lean diet is not necessary for active individuals.
- Do you agree or disagree that a lean diet is essential for a healthy lifestyle?
Questionnaire: Question order
The order in which you ask the questions might also be important for the results of your questionnaire. Sometimes each question can give a little more context to the next one, that is needed to understand the situation. It may also be useful to rank them by complexity, starting with a few easy ones to make the participants feel more comfortable.
Logical flow
A questionnaire with a logical flow structures its questions from the simplest and most basic questions to complex or sensitive ones. The question order can influence the respondents by leading them in specific directions.
Example
Environmental groups are putting pressure on the government to enact laws to protect the environment. In a survey, the order of questions can affect the outlook of the public on the government:
- Do you know any laws protecting the environment?
- Do you think the government is doing enough to protect the environment?
- Do you think the government is fulfilling its duties?
Randomization
Another option would be to ask the questions randomly, which solves the question order effect. The order effect is a phenomenon where the order in which the questions are asked highly influences the answers of the respondents. This can have many reasons, for example leading them to a certain mindset or social desirability bias. However, mixing up the order can also cause the participants to think outside the box and challenge their views to get better answers.
How to design a questionnaire step-by-step
The following represents the 4 steps that are used:
Step 1: Clarify goals
After setting the topic of your questionnaire, you also need to be clear about what kind of information you want to gain. As the respondents are more likely to give valid results when the survey is shorter and thus takes less time to conduct, try to keep your questions to a minimum number without risking to lack any important data afterward.
Step 2: Set questions
To create an ideal list of questions, you may have to think from the perspective of your respondents, consider their familiar language or all possible options for multiple-choice questions. A lack of response options could make them feel excluded or dissatisfied with your survey.
Step 3: Determine the length and order
Depending on your target group, you also have to consider the length and order of questions in your survey. If your participants do not get compensated for their efforts, the questionnaire should be short and quickly to get done, while a reward for completion could motivate them for longer and more time-intensive surveys.
The question order should also be taken into consideration, whether you use a logical flow or randomized order, depending on your aims in the survey. This can highly improve your response rates.
Step 4: Pretest
Test your questionnaire on your friends and family to gain feedback on the clarity and logic of your questions. This can help avoid misunderstandings with your actual participants.
FAQs
A questionnaire is a part of a survey, consisting of a set of questions that are asked to the participants.
Do you like the snow?
- Yes
- No
- Indifferent
These allow the respondent to answer questions in their own phrases. They’re used to collect data such as opinions and attitudes.
Begin by identifying a specific research problem. Create short or detailed questions depending on how motivated the respondents are.
