Chances are you have filled out a questionnaire at least once in your life. But have you ever created one yourself? Do you know the steps involved in questionnaire construction? Let's not make a questionnaire out of this introduction and get to the answers faster.
- We will start by exploring the construction of questionnaires in research methodology.
- Then we will explore the questionnaire construction process.
- After, we will explore the principles of questionnaire construction, covering topics including what makes a good or a bad questionnaire.
- Finally, we will delve into a questionnaire construction example.
Questionnaire Construction: Psychology
Researchers often use questionnaires because they are relatively inexpensive, and it is easy to recruit and analyse many responses and their data.
Questionnaires are a research method involving respondents responding to researchers' pre-determined questions.
Most often, researchers use established questionnaires in research. But what if no questionnaire effectively measures what the researcher intends to measure? Well, this is where questionnaire construction comes in.
Psychology research aims to be scientific, so researchers can't randomly select questions and form questionnaires. Instead, researchers have to follow a standardised process, which we'll get into later.
Construction of Questionnaire in Research Methodology
Researchers use questionnaires when there are no established questionnaires that can be used to measure the variables of interest. Thus, the researchers must develop their own reliable and valid questionnaire.
A valid questionnaire is when the questions in a questionnaire measure what they intend to.
And a reliable questionnaire is when a questionnaire consistently measures the same thing. E.g. if results change over time, across situations or contexts, a questionnaire is considered unreliable.
Questionnaire construction is likely needed when studying novel ideas or specific or investigating unique cases.
Imagine a study investigating childhood imagination and how it affects later careers; to measure imagination, the researcher will likely need to construct a questionnaire.
Questionnaire Construction: Process
When it comes to the questionnaire construction process, it's not as simple as just putting a bunch of questions together. There are more sophisticated and standardised procedures involved in the questionnaire construction process.
Let's explore some things researchers should remember during the questionnaire construction process.
The first thing a researcher needs to determine is the variable(s) that will be measured using the questionnaire.
A good questionnaire will measure each aspect of a variable rather than parts of it. E.g. a questionnaire should not measure half of the symptoms of depression; it should measure each of them.
Next, the researcher should determine whether they want to collect qualitative (non-numerical) or quantitative (numerical) data.
Open-ended questions in questionnaires collect qualitative data, and close-ended questions collect quantitative data.
The type of question used is crucial because it influences later analyses.
When the researcher wants to understand the respondents' thoughts or behaviours, they use open-ended questions. In contrast, a close-ended question is used when a simple, fixed response is sufficient, e.g. yes or no.
An example of an open-ended question is, describe how that video made you feel.
And an example of a close-ended question is on a scale of 1-5, how anxious do you feel one hour after drinking a cup of coffee?
Fixed-response questions should have no overlaps.
How often do you drink a cup of coffee? 2 - 3 times or 3 - 5 times?
If participants drink three cups, they must select both responses; this is not a good example of a fixed-response questionnaire.
Figure 1: Some questionnaires contain a mixture of both open- and close-ended types of questions.
Another essential factor researchers should remember during the questionnaire construction process is that the questions should not be leading, as bias will reduce the validity of later findings.
The questions in questionnaires should be written in a clear and easy-to-understand way that is not open to interpretation.
And the order of the questions is essential. The order should be logical to prevent confusing or frustrating and confusing participants, as this can lead to unreliable and invalid thoughts.
Questionnaire Construction: Comparison of Open/Close-Ended Questions
Let's compare the strengths and weaknesses of open- and close-ended questions in questionnaires.
| Open-Ended Question | Close-Ended Question |
| Cost-efficient | No | Yes |
| Time-efficient | No | Yes |
| Provides in-depth detail | Yes | No |
| Easy to analyse | No | Yes |
| Easy to determine the reliability | No | Yes |
Although the table suggests that close-ended questions have the most advantages, that is not necessarily true. If researchers are trying to get in detail knowledge of participants' thoughts, feelings, behaviours, and motives, wouldn't it be more appropriate to use open-ended questions?
Principles of Questionnaire Construction
As we've learned earlier, there are standardised procedures to ensure that questionnaires are constructed to a high scientific standard.
Before a questionnaire is used in an investigation, a pilot study must be conducted.
A pilot study is a small-scale preliminary investigation of the later research that'll be conducted to ensure that the study and its measures are reliable and valid and to determine if any adjustments are needed.
In terms of reliability, researchers want to ensure that questionnaires have high internal reliability.
Internal reliability means that all questions in the questionnaire should consistently measure the same thing, i.e. no irrelevant questions.
There are several types of validity, one of which is constructed validity. The questionnaire measures the variables stated in the hypothesis and if the scores measured reflect the hypothesis. Researchers must also ensure that the questions cover all the variables under study. If this is the case, the experiment has high validity.
Principles of Questionnaire Construction: Testing Reliability and Validity
One way to measure a questionnaire's reliability is the test-retest method. The process involves the same participants completing the same questionnaire but on different days. If there is a high correlation between test scores on each data, then it has high internal reliability.
A common way to measure the validity of a questionnaire is to distribute a questionnaire to two groups:
- One group consists of experts.
- The other has limited knowledge about the phenomenon under investigation.
If the two groups have similar results, the questionnaire is easy to understand, and the questions are not open to interpretation. Therefore, the questionnaire has high validity.
Another way to measure the validity of a questionnaire is to check whether it has concurrent validity. It is verified by inspecting whether the new questionnaire's results match those of a previously created questionnaire that measures a similar concept.
It is important to note that the measure used as a comparison tool is established to have high reliability and validity.
Questionnaire Construction: Strengths and Issues
Overall, questionnaires are an inexpensive and quick way to collect data for analysis, and as they are usually, anonymous respondents are more likely to be truthful. Additionally, a higher response rate is likely in comparison to interviews.
It becomes problematic when questions are not carefully constructed, as it can lead to biased, invalid and unreliable results.
Ultimately, questionnaires should avoid influencing or suggesting the correct answers (to avoid participants making the socially desirable choice).
Social desirability refers to when respondents respond in a way that they think is how the ideal society member would respond; it is often a result of perceived social pressure and wanting to be seen in a positive light.
Questionnaire Construction: Example
Let's look at an example of what type of questions may be asked when researchers are trying to measure how depressed respondents' are.
There are previously well-established questionnaires that measure depression, so it is unlikely that researchers would need to construct a new questionnaire to measure it.
Q1: In the past two weeks, have you felt depressed or hopeless?
Q2: How many days a week do you find it difficult to get out of bed?
Q3: In the last six months, have you found that you are less interested in activities you usually find enjoyable?
Q4: In the past two weeks, have you noticed that you have less energy than usual?
Q5: In the past two weeks, have you noticed a change in your appetite?
In the example, only close-ended questions are used, i.e. they only rely on fixed responses. Thus, the question collects quantitative data.
So what's good and bad about this questionnaire construction example?
The good things are that:
- It quantifies how far back their responses should think. Imagine if you don't ask this, someone may reflect on how they've been feeling in the past two weeks, and someone else may think back to the past two years; this isn't consistent. Thus, it causes reliability issues.
- The questions are not leading and not open to interpretation.
- The questions are written coherently, logically, and easy to follow.
However, the issue is that the questionnaire does not measure each symptom of depression; therefore, it lacks validity.
Questionnaire Construction - Key takeaways
- Questionnaire construction in psychology is important when no previously established questionnaires can measure the variables of interest.
- The construction of questionnaires in research methodology follows a standardised protocol similar to other scientific research method approaches.
- During the questionnaire construction, process researchers should ensure that questions are written logically, easy to follow, and not open to interpretation.
- The principles of questionnaire construction indicate that a pilot study should be conducted before it is used in an investigation; it should also be reliable and valid.
Explanations 
Textbooks 
Exams 
Magazine 
