ABENAN
A questionnaire is not just an online form. It is a measurement instrument, and any ambiguity or bias in a question passes straight into the data and the results. That is why the time spent planning before distribution saves a great deal of effort after the responses come in.
1. Start with the research problem and its objectives
Write the aim of the study in one clear sentence, then turn each objective into the information you need to measure. If the aim is to understand users’ satisfaction with a digital service, you may need to measure ease of use, speed of completion, quality of support, and overall satisfaction. These become your initial dimensions, not your final questions yet.
Any question that does not serve an objective, a hypothesis, or a necessary description makes the questionnaire longer for no benefit, and may increase drop-out or quick, careless answers.
2. Define the concepts you want to measure
Terms such as quality, satisfaction, and commitment are broad. Define what they mean in the context of your study. For example, “e-service quality” may consist of speed, reliability, ease of use, and security. This definition helps you build items that cover the concept instead of relying on one general question.
3. Identify the population and the right respondent
Ask: who actually has the information you need? Do not ask an employee to rate an aspect they do not deal with, and do not ask a user about internal procedures they do not know. Also take language level, experience, and age into account; a question that is clear to a specialist group may be ambiguous to the general public.
4. Build an outline of dimensions before writing questions
Write a simple table containing the objective, the dimension, its definition, the expected number of items, and the method of analysis. This prevents one dimension from swelling at the expense of another, and it reveals gaps before you draft the instrument.
| Objective | Dimension | What does it measure? | Answer type |
| Evaluate the user experience | Ease of use | Clarity of steps and ease of access | Five-point Likert |
| Evaluate the services | Speed of completion | Time and responsiveness | Five-point Likert |
| Identify improvements | Suggestions | Open opinions | Open-ended question |
5. Choose the right type of question
Closed: suitable for comparison and quick analysis.
Multiple choice: when the alternatives are known.
Rating scale: to measure attitude, satisfaction, or frequency.
Open-ended: to discover ideas the options do not cover, although it takes time to analyze.
Do not make every question open-ended, and do not close a question whose possible options you do not know. Use “Other, please specify” only when there is a real need for it.
6. Write one question for each idea
The item “The service is fast, easy, and secure” measures three things, and a respondent may agree with two of them and reject the third. Split it into three items. AAPOR recommends that a question be specific and measure one concept at a time, using words the target audience understands.
7. Keep the wording short and neutral
Avoid words that push the respondent toward an answer, such as “Don’t you think the excellent system has helped improve performance?” A better version is: “To what extent has the system helped improve performance?” Also avoid absolute terms such as “always” and “never” unless they are precisely what you need.
8. Set a time frame
Instead of “Do you use the platform a lot?”, write “In the last 30 days, how many times did you use the platform?” A time frame reduces differences in how people interpret “a lot” and helps them recall the answer.
9. Design balanced answer options
On a satisfaction scale, make the options cover both directions to a similar degree, for example: very dissatisfied, dissatisfied, neutral, satisfied, very satisfied. Do not offer three positive levels and only one negative one. Decide whether you will use a midpoint, and whether you need a “not applicable” option instead of forcing respondents to give an opinion they do not have.
10. Order the questions like a logical conversation
Guidance from the Pew Research Center suggests that the best questionnaires move through their topics like a conversation. Start with easy questions related to the topic, then move on to the details, and usually place sensitive and demographic questions near the end. Use filter questions so that respondents do not see questions that do not apply to them.
11. Watch the length of the questionnaire and the response burden
There is no magic number of questions. The right length is whatever achieves the objectives of the study without repetition. Pilot the questionnaire while timing the answers, and watch for questions that require long thought or complicated reading. Sometimes removing five similar items is better than keeping a long scale that does not suit the audience.
12. Plan the coding and the analysis before publishing
Before distribution, picture what the data file will look like. What is the name of each variable? How will you code the answers? Which items make up each dimension? Which test will answer each research question? If you cannot link a question to an analysis plan, it may need to be revised or removed.
13. Review face validity and content validity
Ask specialists to review how clear the items are, how well they fit the dimensions, and how well they cover the concept. Expert review is not just a matter of collecting cosmetic comments; record the feedback, the decisions you made, and your reasons. If you adapt a published instrument, explain what changed and check that it suits the new context.
14. Run a pilot test
Try the instrument with a small group similar to the target population. Ask them what they understood from each question, which words were unclear, and whether the answer options cover their situation. This is close to cognitive testing of questions, and it helps reveal differences in understanding before wide distribution.
15. Pay attention to ethics and privacy
Explain to participants the purpose of the study, what taking part involves, and how the data will be used. Do not collect a name, a number, or sensitive information unless it is needed, and protect it properly. Make sensitive questions optional when the study allows it, and make clear that participants can withdraw.
An example of building a dimension
Dimension: ease of use of the platform. It could include items such as:
The steps for using the platform are clear.
I can reach the service I need easily.
The instructions inside the platform are understandable.
I can complete the procedure without repeated help.
Each item measures one aspect, and a single agreement scale can be used for all of them. After the pilot test, the items are reviewed for repetition, clarity, and relevance to the dimension.
A checklist before distribution
Every question is linked to a clear objective.
Every item measures one idea.
The language is understandable to the target group.
The answer options are balanced and do not overlap.
A time frame is specified where needed.
The order is logical and the filter questions work.
The coding and the analysis plan are ready.
Expert review and piloting have been done.
Privacy and consent are explained.
You can browse ABENAN’s services for support in designing and reviewing your questionnaire and preparing it for analysis.
Frequently asked questions
How many items should each dimension have?
There is no fixed number that suits every study. The number depends on how broad the concept is, the quality of the items, and the scale used. It is better for a dimension to cover its main aspects without repetition, and for the items to be reviewed scientifically and in a pilot, rather than relying on the count alone.
Should all the questions use a Likert scale?
No. Demographic data may be a single choice, behavior may be measured by frequency, and some objectives need an open-ended question. Putting everything on a Likert scale may make the instrument look simple, but it does not guarantee correct measurement.
Is the supervisor’s review enough instead of a pilot test?
Expert review is important, but it does not always show how the audience understands a question. A pilot tests the language, the order, the time needed, and the answer options, so it complements expert review rather than replacing it.
Loading comments…