limitations of questionnaires in research

12 advantages and disadvantages of questionnaires

• Written June 7, 2023
• by Nigel Lindemann

Have you ever wondered if conducting a questionnaire is a viable option for your research?

There are tons of options you can use for your research. In this article, we will focus on the one we know best: questionnaires.

As with every research method, there are pros and cons. We’ve gathered the biggest treasures and pitfalls so you can make the best possible decision.

Get ready to discover the 12 biggest advantages like cost efficiency, scalability and quick results. But also disadvantages such as respondents with their own agenda and a possible lack of personalization.

Did you know?  Pointerpro offers a free professional account to students

What is a questionnaire?

We define a questionnaire as a tool used to collect information from people by asking them a series of questions.

The questions are designed to gather specific data or opinions on a particular topic.

For example, imagine a company wants to understand customer satisfaction with its products. They could create a questionnaire with questions like “On a scale of 1 to 10, how satisfied are you with our product?” or “What features do you like most about our product?” Customers would then respond by selecting a number or providing written feedback.

Questionnaire results can be invaluable in a business setting as they provide actionable insights and inform decision-making processes. By analyzing the data collected through questionnaires, businesses can gain a deep understanding of their customers’ needs, preferences, and satisfaction levels. 

This knowledge can guide product development, marketing strategies , customer service improvements, and overall business operations. 

They can also help measure customer satisfaction and identify areas of improvement to enhance customer loyalty.

Additionally, questionnaire data can aid in market research , competitor analysis, and identifying trends and patterns that impact business performance.

What is the difference between a questionnaire and a survey?

The terms “questionnaire” and “ survey ” are often used interchangeably, but there can be some nuances in their usage.

A questionnaire is a specific type of survey instrument that consists of a set of questions designed to gather information or data from individuals or groups. It is a structured tool used to collect responses systematically and standardized. Questionnaires can be administered in various formats, such as paper-based forms, online surveys, or face-to-face interviews. They are typically used in research, assessments, or evaluations to gather specific data or opinions.

On the other hand, a survey is a broader term encompassing various data collection methods, including questionnaires. A survey can refer to any systematic approach to collecting information or data from a sample population. It can include questionnaires, interviews, focus groups, observational studies, or a combination of methods. Surveys aim to gather insights, opinions, or facts about a specific topic or research question.

In essence, a questionnaire is a type of survey that uses a standardized set of questions . In contrast, a survey can encompass a broader range of data collection methods beyond just questionnaires.

12 Advantages of questionnaires

Considering using questionnaires? Here are 15 reasons why questionnaires are your go to solution for data collection or marketing research .

1. Large-scale data collection

Questionnaires are a great way to collect information from a lot of people at once.

Let’s say a company wants to know what their customers think about their products. Instead of interviewing each customer individually, they can create a questionnaire,  generate high-quality content and send it to a large number of customers.

They can send the questionnaire through a business email platform or an online survey platform, allowing customers from different locations to participate. This saves time and resources compared to conducting individual interviews. And you don’t even have to do this manually. this could be done through an automated email in customer onboarding or lead nurturing campaigns. Before starting an automated email campaign, find the right email addresses of your leads. You can do this by using an email finder tool . Or when you face the problem of finding prospects on LinkedIn, the sales navigator can be your assistant in getting more leads.

Large-scale questionnaires offer several advantages in data collection. Firstly, they allow researchers to gather data from a large and diverse sample, providing a broader representation of the target population. This enhances the generalizability and external validity of the findings.

Secondly, they facilitate the analysis of trends and patterns across subgroups within the population, enabling researchers to identify variations and make more nuanced interpretations.

Additionally, large-scale surveys often generate a substantial amount of data, allowing for robust statistical analyses and more precise estimations of relationships between variables.

The extensive data collected through large-scale surveys can support longitudinal studies and provide valuable insights into changes and developments over time.

Furthermore, the scalability of large-scale surveys makes them suitable for addressing complex research questions or investigating multiple dimensions of a phenomenon.

Ultimately, large-scale surveys contribute to the accumulation of knowledge, inform evidence-based decision-making, and support the development of effective policies and interventions.

2. Standardized responses

Standardized responses in questionnaires refer to predetermined options or scales provided to respondents for their answers. These options are carefully designed and presented in a consistent format to ensure uniformity in data collection. Standardized response formats can take various forms, including multiple-choice questions, Likert scales, rating scales, or semantic differential scales.

By offering standardized response options, questionnaires provide a structured framework for respondents to select their answers. This helps to eliminate ambiguity, ensure consistency, and facilitate easier analysis and comparison of responses. Standardized responses allow researchers to quantitatively measure and categorize data, making it easier to identify patterns, trends, and relationships among variables.

For example, in a customer satisfaction survey, respondents may be asked to rate their level of satisfaction on a scale of 1 to 5, with 1 being “Very Dissatisfied” and 5 being “Very Satisfied.” By using standardized response options, the survey collects consistent and quantifiable data that can be statistically analyzed to measure overall customer satisfaction, identify areas for improvement, or compare satisfaction levels across different customer segments.

Standardized responses not only streamline the data collection process but also enhance the reliability, consistency, and objectivity of the collected data. They ensure that respondents are provided with the same set of response options, enabling researchers to draw meaningful conclusions based on the aggregated and standardized data.

3. Anonymity and confidentiality

Anonymity and confidentiality are crucial aspects of questionnaires that provide several benefits for both respondents and researchers.

Firstly, anonymity ensures that respondents can provide honest and unbiased answers without fear of repercussions or judgment. When respondents remain anonymous, they are more likely to express their true opinions, feelings, and experiences, even on sensitive or personal topics. 

For example, in a survey on workplace satisfaction, employees may feel more comfortable sharing their concerns or dissatisfaction if they know their responses will be kept anonymous. This fosters a higher level of transparency and authenticity in the data collected.

Confidentiality, on the other hand, focuses on the protection of respondents’ personal information. It assures participants that their data will be treated with strict confidentiality and will not be disclosed to unauthorized individuals. Maintaining confidentiality helps build trust between respondents and researchers, encouraging greater participation and cooperation. For example, in a healthcare survey that collects sensitive medical information, participants are more likely to provide accurate and detailed responses if they trust that their data will be handled securely and confidentially.

Both anonymity and confidentiality contribute to the integrity and quality of the data collected. They create a safe and non-judgmental environment for respondents to freely express their thoughts and opinions. Additionally, it helps prevent potential biases that may arise if respondents are influenced by social desirability or fear of repercussions. Researchers can then obtain more accurate and reliable insights from the data, leading to better-informed decisions and actions based on the findings.

Overall, the main benefit of anonymity and confidentiality in questionnaires is the assurance of privacy and protection for respondents, which promotes honesty, trust, and more accurate data collection.

4. Cost-effective

Questionnaires are perfect for cost-effective data collection due to their ability to reach a large number of respondents at a relatively low cost.

For instance, imagine a non-profit organization surveying to gather feedback from its supporters. By using online questionnaires, they can avoid expenses associated with printing and postage, as well as the need for manual data entry.

Instead, they can distribute the questionnaire via email or share it on their website through a digital flipbook or social media platforms . This significantly reduces costs compared to conducting in-person interviews or focus groups.

Additionally, questionnaires enable researchers to collect data from a large sample size, providing more robust and representative results. The ability to administer questionnaires remotely and automate data collection and analysis further enhances cost-effectiveness.

Overall, questionnaires offer a cost-efficient method of data collection by reducing expenses related to logistics, data entry, and participant recruitment, while still allowing for a comprehensive and reliable analysis of data.

5. Data quantification

When we talk about data quantification in questionnaires, we mean the process of assigning numerical values to the responses provided by participants. This helps researchers analyze the data more effectively and derive meaningful insights.

By converting responses into numbers, researchers can identify patterns and trends in the data. 

Let’s say a survey asks people about their favorite ice cream flavors and provides options such as chocolate, vanilla, and strawberry. By assigning numerical values to these options, researchers can determine the most popular flavor based on the frequency of responses. This allows them to identify common preferences and make informed decisions based on the findings.

Data quantification also enables statistical analysis. Researchers can use various statistical methods to examine relationships between variables, calculate averages, and measure the significance of findings. For example, if a questionnaire asks participants to rate their agreement with a statement on a scale of 1 to 5, researchers can calculate the average rating and determine if there are any significant differences between groups or demographic segments. This helps researchers understand the data more comprehensively and draw more reliable conclusions.

Additionally, using numbers makes it easier to communicate and visualize the findings. Graphs, charts, and summary statistics can be used to present the data in a visually appealing and easily understandable manner. This allows researchers to effectively convey their findings to stakeholders, decision-makers, or the general public, facilitating better understanding and interpretation of the results.

In summary, data quantification in questionnaires allows researchers to assign numerical values to responses, enabling analysis, statistical exploration, and clearer communication of the findings. This process helps researchers uncover insights, identify patterns, and make informed decisions based on the numerical representation of the data.

6. Flexibility

Flexibility allows researchers to customize the questionnaire design and content to gather specific information, accommodate diverse respondent needs, and ensure the relevance and accuracy of data collected.

One advantage of flexibility is the opportunity to include open-ended questions. These questions allow respondents to provide detailed, in-depth responses that go beyond predetermined options. For example, in a feedback survey , including an open-ended question like “Please provide any additional comments or suggestions” allows participants to freely express their thoughts, ideas, or concerns. This qualitative data provides valuable insights and a deeper understanding of respondents’ perspectives.

Let’s take the example of an employee survey. If you want to learn more about the ambitions of your employees, you can include a question like “ What are your life plans or long-term aspirations?”. Flexibility in survey design ensures that respondents have the space to articulate their personal and professional objectives without feeling constrained by predefined categories.

Flexibility also allows for the inclusion of skip patterns or branching logic. Skip patterns are used to direct respondents to specific questions based on their previous responses. This ensures that participants only answer relevant questions, making the questionnaire more efficient and personalized. 

For example, in a customer satisfaction survey, if a respondent indicates they have never used a particular feature of a product, the questionnaire can skip questions related to that feature, saving time and reducing unnecessary response burden.

Another benefit of flexibility is the ability to modify or update questionnaires over time. Researchers can make adjustments based on feedback or changing research needs, ensuring the questionnaire remains relevant and up-to-date. For example, if a survey on consumer preferences is conducted annually, flexibility allows for the inclusion of new product options or emerging trends in subsequent surveys.

Flexibility in questionnaires provides researchers with the means to gather more nuanced and accurate data. By customizing the questionnaire design, incorporating open-ended questions, utilizing skip patterns, and adapting to evolving research needs, flexibility enhances the overall quality and effectiveness of data collection efforts.

7. Quantitative and qualitative data collection simultaneously

The main benefit of combining quantitative and qualitative data collection in questionnaires is the comprehensive and nuanced understanding it provides. Both quantitative and qualitative data offer unique insights that, when used together, create a more holistic picture of the research topic.

Quantitative data, obtained through structured questionnaires with predefined response options, provides numerical information that can be analyzed statistically. This type of data allows researchers to measure trends, make comparisons, and draw objective conclusions. 

For instance, in a survey about smartphone usage, quantitative data can reveal statistics such as the percentage of respondents who own smartphones, their preferred brands, or the average time spent on various activities. These quantifiable insights are valuable for understanding trends and patterns within a larger population.

On the other hand, qualitative data, gathered through open-ended questions or free-text responses, offers rich and in-depth insights into participants’ thoughts, feelings, and experiences. Qualitative data provides a deeper understanding of individual perspectives, motivations, and subjective experiences that cannot be easily captured by quantitative measures alone. For example, in the same smartphone survey, qualitative responses may reveal specific reasons why participants prefer certain brands, their satisfaction with particular features, or their frustrations with user interfaces. These qualitative insights provide context and depth, helping researchers understand the “why” behind quantitative trends.

By combining quantitative and qualitative data collection in questionnaires, researchers can gain a more comprehensive understanding of their research topic. The quantitative data provides a broad overview and statistical evidence, while qualitative data offers a deeper exploration of individual experiences and perceptions. This mixed-methods approach allows for triangulation, where findings from both types of data can be compared and corroborated, enhancing the validity and reliability of the research. Ultimately, the combination of quantitative and qualitative data collection in questionnaires enables researchers to gain a more holistic and nuanced understanding of the research topic, leading to more informed decision-making and actionable insights.

8. Ease of analysis

Questionnaires provide structured and standardized responses, making the analysis process more streamlined and less time-consuming.

One advantage of ease of analysis is the ability to use statistical software and tools to process the data quickly. With questionnaires, researchers can easily input the data into statistical programs, such as SPSS or Excel, and perform various analyses. For example, if a survey includes multiple-choice questions, the data can be easily coded and analyzed using frequency distributions or cross-tabulations. This allows for efficient exploration of patterns, relationships, and trends within the data.

Additionally, questionnaires often have clear response options and scales, making it easier to quantify and measure the data. This facilitates the calculation of descriptive statistics, such as means, standard deviations, or percentages. Researchers can analyze the data in a structured and systematic manner, enabling them to identify key findings and draw meaningful conclusions. 

For example, if a questionnaire measures customer satisfaction on a scale of 1 to 5, researchers can calculate the average satisfaction score and compare it to a benchmark to assess overall satisfaction levels.

Ease of analysis in questionnaires also allows for efficient data visualization. Researchers can generate charts, graphs, and summary tables to present the findings in a visually appealing and easy-to-understand format. This aids in the communication of results to stakeholders or decision-makers. Visual representations make it simpler to identify trends, compare groups, and convey key insights effectively.

In summary, the main benefit of ease of analysis in questionnaires is the convenience and efficiency it provides in processing and interpreting data. By offering structured responses, standardized scales, and clear response options, questionnaires enable researchers to analyze the data quickly using statistical tools, quantify the responses, and visualize the findings. This ease of analysis enhances the research process, saves time, and facilitates the extraction of meaningful insights from the collected data.

9. Standardization across studies

Standardization allows for uniformity in questionnaire design, response options, and measurement scales, enhancing the reliability and validity of research findings.

One advantage of standardization is the ability to compare results across studies. When questionnaires are standardized, researchers can use similar or identical items to measure constructs of interest. For example, in the field of psychology, standardized questionnaires like the Beck Depression Inventory or the Big Five Personality Inventory are widely used across studies. This allows researchers to compare results and draw conclusions based on a common framework, facilitating the advancement of knowledge and the replication of findings.

Moreover, standardization promotes the accumulation of data across different populations or time periods. By using consistent measures, researchers can compare data collected from diverse samples or over multiple time points. This enables the identification of trends, changes, or variations in responses. 

For instance, if a questionnaire measures job satisfaction using a standardized scale, researchers can compare satisfaction levels across different industries, organizations, or generations, providing insights into the broader patterns and dynamics of employee well-being.

Standardization also contributes to the overall reliability and validity of the research. By using established and validated measures, researchers can have confidence in the quality and accuracy of the data collected. Standardized questionnaires have undergone rigorous testing and validation processes, ensuring that they effectively measure the intended constructs. This enhances the credibility and trustworthiness of the research outcomes.

In summary, standardization across studies in questionnaires allows for consistency, comparability, and replicability of data collected. By using standardized measures, researchers can compare results across different studies, accumulate data across diverse populations or time periods, and ensure the reliability and validity of the research. This promotes a more robust and cumulative knowledge base in various fields of study.

10. Accessibility

Accessibility measures in questionnaires enable individuals with different abilities, languages, or technological access to participate, enhancing the inclusiveness and validity of the research.

One advantage of accessibility is the opportunity to accommodate participants with disabilities. Questionnaires can be designed with accessibility features such as screen reader compatibility, alternative text for images, or keyboard navigation. These measures ensure that individuals with visual impairments or motor disabilities can access and complete the questionnaire independently. By removing barriers, researchers can gather data from a more diverse and representative sample, allowing for a more comprehensive understanding of the research topic. For example, an online survey with accessible features can include closed-captioning for videos, making it accessible to participants with hearing impairments.

Furthermore, language accessibility is crucial for reaching participants who are non-native English speakers or those who prefer to communicate in their native language. Questionnaires can be translated into multiple languages to ensure comprehension and inclusivity. By offering questionnaires in participants’ preferred languages, researchers can engage a broader audience and gather more accurate data. 

For instance, a healthcare survey could be provided in multiple languages to gather feedback from diverse cultural communities and improve healthcare services for all.

Accessibility in questionnaires also extends to the choice of data collection methods. Researchers can offer various options such as online surveys, social media walls , paper-based questionnaires, or phone interviews to cater to participants’ technological access or preferences. This flexibility allows individuals without internet access or those who prefer traditional methods to participate. By offering multiple modes of data collection, researchers can reduce barriers and ensure equal opportunities for individuals with different technological resources.

In summary, accessibility in questionnaires ensures inclusivity, reaching a diverse range of participants and maximizing the representation of different perspectives. By incorporating accessibility features for individuals with disabilities, offering language options, and providing multiple data collection methods, researchers create an environment that accommodates various needs and promotes equal participation. This enhances the validity and reliability of the research outcomes by incorporating the voices of individuals who may have otherwise been excluded or overlooked.

11. Ease of replication

Ease of replication ensures that other researchers can follow the same methodology and replicate the study with minimal effort and resources.

One advantage of ease of replication is the ability to verify research findings and strengthen the evidence base. When questionnaires are designed with clear instructions, standardized measures, and well-defined procedures, other researchers can replicate the study to assess the consistency and generalizability of the results. Replication studies help confirm or challenge previous findings, enhancing the scientific rigor and reliability of the research.

For example, suppose a questionnaire measures customer satisfaction using a validated scale and a specific methodology. In that case, other researchers can replicate the study to validate the original findings or identify potential contextual differences.

Moreover, ease of replication fosters cumulative knowledge building within a field. When studies can be easily replicated, researchers can build upon existing research, refine methodologies, and identify gaps or inconsistencies in the literature. Replication studies allow for exploring factors that may influence the research outcomes in different contexts or populations.

This iterative replication and extension of research findings leads to a more comprehensive understanding of the topic over time.

Ease of replication also promotes transparency and openness in research. When questionnaires are well-documented and accessible, other researchers can access the questionnaire materials, study design, and data collection procedures.

This transparency allows for scrutiny and evaluation of the research methods, increasing confidence in the research outcomes. It also facilitates collaboration and the sharing of knowledge among researchers working on similar topics.

In summary, ease of replication in questionnaires enhances the reliability, credibility, and cumulative nature of research. By enabling other researchers to replicate studies with ease, it ensures the verification of findings, the building of cumulative knowledge, and the promotion of transparency within the scientific community. Replication studies contribute to the robustness and validity of research outcomes, fostering a stronger evidence base for decision-making and further exploration of research topics.

12. Time efficiency

Time efficiency ensures that researchers can gather a substantial amount of data within a reasonable timeframe, allowing for timely analysis and interpretation of the results.

One advantage of time efficiency is the ability to reach a larger sample size. Questionnaires offer a scalable approach to data collection, allowing researchers to distribute surveys to a wide audience simultaneously. This enables the inclusion of a diverse range of participants, ensuring a more representative sample. For example, an online questionnaire distributed through social media platforms can reach thousands of participants within a short span, facilitating a large-scale data collection effort.

Moreover, questionnaires typically have structured formats with predefined response options, making it quicker for participants to provide their answers. Compared to other data collection methods like interviews or focus groups, questionnaires can be completed at a self-paced and convenient time for the participants. This reduces the time burden on both the researchers and participants, making it easier to collect data from a larger number of individuals. 

For instance, a customer satisfaction survey that utilizes multiple-choice questions can be completed by respondents within a few minutes, allowing for efficient data collection from a broad customer base.

Time efficiency in questionnaires also extends to the data analysis phase. With standardized responses and structured formats, questionnaires enable researchers to streamline the analysis process. Data can be easily coded, entered into statistical software, and analyzed using automated procedures. This accelerates the data processing and allows researchers to obtain results more quickly. 

For example, researchers can generate statistical summaries, charts, or visualizations within a short period, facilitating timely insights and decision-making.

In summary, the main benefit of time efficiency in questionnaires is the ability to collect data from a large number of participants within a relatively short timeframe. By reaching a larger sample size, ensuring quick completion by participants, and streamlining the data analysis process, questionnaires expedite the data collection and analysis stages. This enables researchers to obtain timely results, make informed decisions, and advance knowledge in their respective fields.

These advantages highlight the value of questionnaires as a versatile and efficient tool for data collection in various research, academic, and professional contexts.

Most survey and questionnaire providers are quantitative and allow easy analysis of results. With built-in tools, it’s easy to analyze your results without a background in statistics or scientific research. For example, photographers often use a wedding photography questionnaire to understand clients’ needs as accurately as possible and ensure they are on the same page.

You might think questionnaires are not fun and that getting respondents to complete them can be tricky. However, you can use interactive forms that will give you visual data insights to draw experienced user stories organized in dynamic timelines.

Tools like Pointerpro offer easy-to-interpret reports and visualizations, meaning that you’ll quickly be turning your data into results. These results can be put in a wide variety of charts and tables to present to your boss, colleagues, clients or customers.

12 Disadvantages of questionnaires

To balance things out, we’ve gathered 12 disadvantages, so you can outweigh both the pros and cons of a questionnaire to make an informed decision.

1. Limited depth of information

The main downside of limited depth of information is the potential inability to capture detailed and nuanced insights from participants. Questionnaires typically rely on structured and predefined response options, which can constrain participants’ ability to provide in-depth responses. This limitation restricts the level of detail that can be obtained, potentially resulting in a loss of important context and richness of information.

One of the consequences of limited depth of information is the difficulty in understanding the underlying reasons or motivations behind participants’ responses. For example, a questionnaire on customer satisfaction may provide overall ratings and rankings but may not delve into the specific factors influencing satisfaction levels. Without this deeper understanding, it becomes challenging for researchers to identify actionable insights or make informed decisions based solely on the questionnaire data.

Furthermore, certain topics or experiences may require open-ended questions or qualitative approaches to capture the complexity involved.

For instance, a questionnaire on individuals’ perceptions of their work environment may fail to comprehensively capture the specific challenges or opportunities that impact their experience. By relying solely on closed-ended questions, the questionnaire may miss valuable insights that could be gained through qualitative interviews or focus groups.

Additionally, the limited depth of information in questionnaires can hinder the exploration of unexpected findings or emerging themes. Since questionnaires are typically designed in advance, they may not have the flexibility to adapt or probe further during data collection.

As a result, researchers may miss out on valuable insights or unique perspectives that could arise during the questionnaire completion process.

To mitigate this downside, researchers may consider supplementing questionnaires with qualitative methods such as interviews or observation, allowing for a more comprehensive understanding of the research topic. By combining the strengths of both quantitative and qualitative approaches, researchers can overcome the limitations of limited depth of information and gain a more holistic view of the subject matter.

2. Response bias

Response bias refers to systematic errors in how participants interpret and respond to questionnaire items, leading to biased results.

One common example of response bias is social desirability bias. Participants may feel compelled to provide responses they perceive as socially acceptable or desirable rather than their true thoughts or behaviors.

For instance, participants may overstate their adherence to a healthy diet in a questionnaire about healthy eating habits due to a desire to present themselves favorably. This bias can lead to inflated or inaccurate data, compromising the validity of the findings.

Another example is acquiescence bias, where participants tend to agree or endorse statements regardless of their true beliefs or experiences. This bias can arise from a desire to please the researcher or avoid confrontation. For instance, participants may agree with positively worded statements without critically evaluating their validity. This can distort the true distribution of opinions or attitudes within the population.

To address response bias in questionnaires, researchers can employ various strategies. One approach is to use randomized response techniques or indirect questioning methods to mitigate the impact of social desirability bias. These techniques give respondents anonymity or reduce the perceived judgment associated with their responses, encouraging more honest answers.

Additionally, researchers can employ attention checks or validation questions within the questionnaire to identify participants who may not be fully engaged or who exhibit response bias tendencies. By identifying and excluding such participants from the analysis, researchers can enhance the accuracy and reliability of the data.

Furthermore, careful questionnaire design, including clear instructions and neutral wording, can minimize the potential for response bias. Piloting the questionnaire with a small sample before full-scale implementation allows researchers to identify and address any ambiguities or potential biases in the items.

3. Low response rates

The downside of low response rates is the potential for non-response bias and the potential impact on the representativeness and generalizability of the collected data. Low response rates occur when a significant proportion of individuals invited to participate in a questionnaire choose not to respond or do not complete the survey. This can lead to biased results and affect the overall quality of the study.

One example of non-response bias is when individuals who choose not to respond to a questionnaire differ systematically from those who do respond. In a customer satisfaction survey, if individuals with a negative experience are less likely to respond, the survey results may overestimate overall satisfaction levels. This bias can distort the findings and lead to inaccurate conclusions.

Low response rates can also introduce a lack of representativeness in the sample. If certain subgroups of the population are less likely to respond, their perspectives and experiences may be underrepresented in the data. This can limit the generalizability of the findings and raise concerns about the validity of the conclusions drawn from the questionnaire data.

To address low response rates, researchers can employ several strategies. Firstly, it is crucial to ensure clear and concise communication with potential participants, highlighting the purpose and importance of the survey. Providing incentives such as small rewards or entry into a prize draw can also increase motivation to participate.

Additionally, researchers can use multiple modes of data collection, such as online surveys, mail surveys, or phone interviews, to reach a wider range of individuals and improve response rates. Following up with reminders or personalized invitations can also prompt non-respondents to complete the questionnaire.

Furthermore, careful attention should be paid to the length and complexity of the questionnaire to minimize respondent burden. Long or complex questionnaires are more likely to discourage participation. Shorter, focused questionnaires are more likely to yield higher response rates.

While achieving a 100% response rate is impossible, researchers can use statistical techniques such as weighting or propensity score adjustment to mitigate the potential impact of non-response bias. These techniques aim to adjust the data to make it more representative of the target population. We’ve listed 35 ways to improve your response rate.

In summary, low response rates in questionnaires can introduce non-response bias and limit the representativeness of the data. By employing effective communication, using multiple modes of data collection, keeping the questionnaire concise, and employing statistical techniques to account for non-response bias, researchers can minimize the impact of low response rates and enhance the validity and reliability of the findings.

4. Misinterpretation of questions

The downside of misinterpretation of questions is the potential for respondents to misunderstand or misinterpret the intended meaning of the questionnaire items, leading to inaccurate or unreliable responses. Misinterpretation can occur due to unclear wording, ambiguous phrasing, or the use of technical jargon that is unfamiliar to participants.

One example of misinterpretation is when respondents misunderstand the context or scope of a question. For instance, respondents may interpret a questionnaire item asking about “annual income” differently. Some may include only their salary, while others may include additional sources of income or exclude certain types of earnings. This can lead to inconsistent or incomparable data across respondents, compromising the validity and reliability of the results.

Another example is when respondents encounter double-barreled questions, where multiple concepts or issues are combined into a single question. For instance, a question asking, “Do you find the product useful and affordable?” combines two distinct concepts. Respondents may find the product useful but not affordable, leading to difficulty in providing a clear and accurate response. This can introduce confusion and bias into the data collected.

To address misinterpretation of questions in questionnaires, researchers can take several steps. Firstly, it is essential to pilot test the questionnaire with a small group of individuals representative of the target population. This helps identify potential ambiguities or areas of confusion in the questionnaire items. Based on the feedback received, the wording can be clarified, and any problematic questions can be revised or removed.

Using clear and unambiguous language is crucial. Questions should be concise, specific, and free of technical jargon whenever possible. Providing clear instructions or definitions for key terms can also help ensure that respondents have a common understanding of the question.

Additionally, researchers can consider using visual aids, such as diagrams or images, to support the comprehension of complex or abstract concepts. Visual representations can assist respondents in understanding the question more accurately and provide a reference point for their responses.

It is also important to offer response options that cover a comprehensive range of possibilities. Including “don’t know” or “not applicable” options can allow respondents to indicate situations where they are unsure or the question does not apply to them.

By taking these measures to minimize misinterpretation, researchers can enhance the clarity and precision of the questionnaire items, reducing the likelihood of respondents misunderstanding the questions and improving the quality of the collected data.

5. Inability to capture non-verbal cues

The downside of the inability to capture non-verbal cues is the potential loss of valuable information conveyed through non-verbal communication. Non-verbal cues, such as facial expressions, body language, and tone of voice, play a significant role in communication, providing additional context and insights that may be missed in a questionnaire format. This limitation can result in a lack of depth and richness in the data collected.

For example, when asking about emotions or attitudes in a questionnaire, respondents may provide verbal responses that do not fully capture the nuances and subtleties of their actual emotions. Non-verbal cues, such as facial expressions or gestures, can convey additional emotional information that words alone cannot capture. Researchers may miss important nuances and complexities in participants’ responses without observing these cues.

Furthermore, non-verbal cues can indicate respondent engagement, hesitation, or confidence in their responses. For instance, during an interview or face-to-face interaction, researchers can observe changes in body language or tone of voice that may suggest uncertainty or lack of conviction. In a questionnaire, these cues are absent, making it difficult to gauge the level of certainty or sincerity behind respondents’ answers.

Researchers can consider combining different data collection methods to address the limitation of capturing non-verbal cues in questionnaires. For example, follow-up interviews or focus groups can be conducted with a subset of respondents to delve deeper into their questionnaire responses and gather additional qualitative insights. These methods allow for exploring non-verbal cues and provide a more comprehensive understanding of participants’ experiences and perspectives.

Another approach is to include open-ended questions in the questionnaire, allowing respondents to provide qualitative descriptions or explanations alongside their quantitative responses. This can help capture some of the non-verbal aspects of their experiences, providing richer insights beyond the limitations of structured response options.

While questionnaires may not directly capture non-verbal cues, researchers can employ supplementary techniques to better understand participants’ thoughts, emotions, and behaviors. By incorporating qualitative methods or additional data collection approaches, researchers can mitigate the downside of the inability to capture non-verbal cues and gain a more comprehensive understanding of the research topic.

6. Limited engagement

Limited engagement refers to respondents’ lack of interest, attention, or active involvement during the questionnaire completion process.

When participants approach a questionnaire with limited engagement, they may rush through the questions, provide incomplete or inaccurate responses, or exhibit response biases. This can result in low-quality data that does not accurately reflect their true thoughts, opinions, or behaviors. For example, respondents may choose random response options without thoughtful consideration, leading to unreliable results.

Limited engagement can stem from various factors, such as survey fatigue, lack of personal relevance, or perceived time constraints. If respondents do not find the questionnaire interesting, meaningful, or relevant to their lives, they may feel less motivated to provide thoughtful and accurate responses. This can lead to high dropout rates or low completion rates, further exacerbating the issue.

Researchers can employ strategies to enhance participant motivation and involvement to address limited engagement in questionnaires. One approach is to communicate the purpose and importance of the questionnaire, highlighting how the data will be used and the potential impact of the research. This can help participants understand the significance of their contribution and increase their sense of engagement.

Additionally, researchers can consider incorporating interactive elements or gamification techniques into the questionnaire design. For example, using progress bars, visual stimuli, or interactive features can make the questionnaire more visually appealing and engaging. Including interesting and relevant questions tailored to the participants’ experiences can also increase their motivation to provide thoughtful responses.

Furthermore, ensuring a user-friendly and intuitive questionnaire layout can enhance engagement. Clear instructions, logical flow, and manageable question lengths can contribute to a positive user experience, reducing the respondent burden and increasing their willingness to engage with the questionnaire.

Personalizing the questionnaire experience by addressing participants by name or tailoring the content to their specific characteristics or interests can also boost engagement. This demonstrates that their input is valued and relevant, increasing their motivation to provide accurate and meaningful responses.

Overall, limited questionnaire engagement can impact the collected data’s quality. By employing strategies to enhance participant motivation, highlighting the relevance and importance of the questionnaire, and incorporating interactive elements, researchers can mitigate the downside of limited engagement and improve the overall quality and validity of the responses.

7. Lack of context

Without proper context, participants may make assumptions, misinterpret the intent of the questions, or provide incomplete or inaccurate responses, leading to compromised data quality and validity.

For example, consider a questionnaire asking about customer satisfaction with a product or service. Without providing specific details about the features, usage scenarios, or recent experiences with the product/service, respondents may struggle to evaluate their satisfaction level accurately. Their general perceptions or biases may influence their responses rather than reflecting their actual experiences.

Another example is asking about sensitive topics, such as political beliefs or personal experiences. Respondents may hesitate to provide truthful responses due to concerns about privacy, social desirability, or fear of judgment without providing a clear context or framing for these questions. This can lead to biased or incomplete data, as respondents may choose to withhold or provide socially acceptable answers rather than express their true thoughts or experiences.

To address the lack of context in questionnaires, researchers can provide clear and concise instructions that set the stage and provide relevant background information. This can include brief descriptions of the research objectives, definitions of key terms, or explanations of the specific context or situation being referred to in the questions.

Researchers may also consider including vignettes or hypothetical scenarios to provide context for certain questions. By presenting realistic situations and asking respondents to imagine themselves in those scenarios, participants can provide more informed responses grounded in specific contexts.

Additionally, researchers can employ mixed-method approaches, combining questionnaires with qualitative interviews or focus groups. These qualitative methods allow for in-depth exploration and clarification of responses, providing a richer understanding of the context and uncovering nuances that may not be captured through closed-ended questionnaire items alone.

By addressing the lack of context in questionnaires through clear instructions, providing relevant background information, and utilizing mixed-method approaches, researchers can mitigate the downside and improve the accuracy and depth of the data collected.

8. Difficulty in capturing complex or nuanced information

Questionnaires typically rely on structured questions with predefined response options, which may not allow respondents to fully express complex or nuanced thoughts, opinions, or experiences.

For example, when exploring intricate emotions, personal beliefs, or subjective experiences, respondents may find it challenging to convey the full complexity of their perspectives through simple rating scales or multiple-choice options. Their responses may oversimplify their true feelings or fail to capture the subtle nuances of their experiences.

Moreover, complex topics often require contextual information, follow-up questions, or probing for deeper understanding. In a questionnaire, there is a lack of immediate interaction or clarification, making it difficult to capture the complexity of participants’ viewpoints.

Researchers can incorporate open-ended questions or free-text response options to address the difficulty in capturing complex or nuanced information in questionnaires. This allows participants to provide detailed and personalized responses, providing a deeper understanding of their thoughts and experiences. Open-ended questions allow for the expression of individual perspectives and the inclusion of unique insights that may not be captured through closed-ended questions.

Researchers can also consider utilizing qualitative research methods, such as interviews or focus groups, in combination with questionnaires. These methods provide opportunities for probing and follow-up questions, allowing participants to elaborate on complex topics and share in-depth narratives. Researchers can gain a more comprehensive understanding of complex issues by integrating qualitative data collection alongside questionnaires.

Furthermore, researchers can employ cognitive testing techniques to ensure that questionnaire items effectively capture complex concepts. Cognitive testing involves pretesting the questionnaire with a sample of participants to identify any comprehension difficulties or areas where the questions fail to capture the desired complexity. This iterative process allows for refining and improving the questionnaire to enhance the capture of nuanced information.

While questionnaires may have limitations in capturing complex or nuanced information, researchers can employ strategies such as open-ended questions, qualitative methods, and cognitive testing to mitigate these downsides and obtain a more comprehensive understanding of the research topic.

9. Inability to probe for clarification

Unlike in interviews or focus groups, where researchers can ask follow-up questions in real-time, questionnaires lack the opportunity for immediate clarification, which can lead to potential misunderstandings and incomplete or inaccurate responses.

For example, when respondents encounter ambiguous or unclear questions in a questionnaire, they may interpret them differently based on their assumptions or understanding. This can result in inconsistent or unreliable data, as participants may provide responses that do not align with the researcher’s intended meaning or question objectives.

Furthermore, participants may have varying levels of familiarity with the subject matter or require further explanation to grasp the context of certain questions fully. Without the ability to probe for clarification, researchers cannot address these knowledge gaps or ensure that respondents clearly understand the concepts being explored.

Researchers can employ several strategies to address the downside of the inability to probe for clarification in questionnaires. First, it is crucial to carefully design and pilot test the questionnaire to ensure the questions are clear, concise, and unambiguous. This reduces the likelihood of participants misinterpreting the intended meaning of the questions.

Additionally, researchers can provide detailed instructions or explanations before specific sections or questions to enhance participants’ understanding. Including examples or visual aids can also help clarify complex concepts and reduce potential confusion.

To mitigate the limitation of being unable to probe for clarification directly, researchers can consider including an optional open-ended comment section at the end of the questionnaire. This allows respondents to elaborate on their responses, raise concerns, or provide additional context that they feel may be important but was not addressed in the structured questions. While not a substitute for real-time probing, this allows participants to provide additional information that may improve the overall quality and validity of the data.

Overall, the inability to probe for clarification in questionnaires can limit researchers’ ability to ensure accurate and comprehensive responses. However, by designing unambiguous questions, providing detailed instructions, and incorporating optional open-ended comment sections, researchers can minimize the downsides and enhance the quality of the collected data.

10. Sampling limitations

Questionnaires often rely on a specific sample of participants. If this sample is not truly representative of the target population, the results may not accurately reflect the larger group’s characteristics or opinions.

For example, if a questionnaire is distributed online and only accessible to individuals with internet access, it automatically excludes those without internet connectivity. This can result in a biased sample that does not represent the broader population, leading to skewed or inaccurate results.

Additionally, self-selection bias can occur when individuals choose whether or not to participate in the questionnaire. Those more interested or with stronger opinions on the topic may be more likely to respond, while others may opt-out. This can lead to a non-representative sample and result in biased or unbalanced data.

To address sampling limitations in questionnaires, researchers can employ various strategies. First, it is essential to carefully define the target population and use appropriate sampling techniques to ensure representativeness. Random sampling methods, such as stratified or cluster sampling, can help mitigate biases and increase the likelihood of obtaining a diverse and representative sample.

When using online questionnaires, researchers can employ quota sampling to ensure a more balanced representation of various demographic groups. Quota sampling sets specific quotas for each group based on known population proportions, helping to overcome the limitations of self-selection bias.

Researchers can also consider using multiple data collection methods, such as combining online questionnaires with telephone or face-to-face interviews, to reach individuals who may be excluded from the online sample. This mixed-method approach helps capture a more diverse range of perspectives and improves the overall representativeness of the findings.

Furthermore, researchers should be transparent about the limitations of their sampling methods and acknowledge any potential biases in their interpretations. By clearly stating the characteristics of the sample and acknowledging any limitations, the findings can be appropriately contextualized, and the potential impact of sampling limitations can be considered.

In summary, sampling limitations in questionnaires can lead to biased or unrepresentative samples, impacting the generalizability of the findings. Researchers can address these limitations by employing appropriate sampling techniques, considering mixed-method approaches, and being transparent about the limitations and potential biases in their interpretations. These strategies help mitigate the downsides of sampling limitations and improve the overall quality and validity of the data collected.

11. Potential for measurement error

Measurement error refers to any deviation between the true value of the concept being measured and the actual response provided by the respondent. This can occur due to various factors, such as respondent bias, misunderstanding of the question, or errors in data entry or processing.

For example, respondents may have difficulty recalling specific details accurately or may provide socially desirable responses that do not reflect their true thoughts or behaviors. They may also misinterpret the meaning of certain questions or struggle to rate their feelings or experiences on a numerical scale accurately. Additionally, errors can occur during data entry or processing, leading to incorrect or distorted data.

To address the issue of measurement error in questionnaires, researchers can employ several strategies. First, it is crucial to design unambiguous questions that minimize potential misunderstandings. Careful wording, using examples, and providing clear response options can help reduce measurement error caused by respondent confusion or misinterpretation.

Researchers can also implement validation measures within the questionnaire to assess the reliability and validity of the responses. This can include including duplicate questions or items that measure the same construct in different ways and incorporating established scales or measures with known reliability and validity.

Researchers can emphasize the importance of honest and anonymous feedback to reduce socially desirable responses, ensuring respondents feel comfortable providing genuine responses without fear of judgment or consequences. Anonymity can help alleviate concerns about self-presentation bias and encourage participants to provide more accurate and truthful information.

Furthermore, researchers can employ techniques such as cognitive interviewing or pilot testing to identify potential sources of measurement error. Cognitive interviewing involves conducting in-depth interviews with participants to understand their thoughts and interpretations of the questionnaire items. This helps identify potential areas of confusion or misunderstanding that can be addressed through revisions to the questionnaire.

Lastly, data cleaning and validation procedures should be implemented to identify and correct errors during data entry or processing. This can involve checking for logical inconsistencies, missing responses, or outliers that may indicate data entry or measurement errors.

While measurement error is an inherent limitation in questionnaire-based research, researchers can employ clear question design, validation measures, emphasis on honest feedback, cognitive interviewing, and rigorous data cleaning procedures to minimize its impact. By addressing measurement errors, researchers can improve the accuracy and reliability of the data collected through questionnaires.

12. Potential for response fatigue

Response fatigue can occur when questionnaires are lengthy, repetitive, or overly complex, overwhelming participants and negatively affecting the quality of the data obtained.

For example, if a questionnaire consists of numerous pages with densely packed questions or repetitive items, respondents may lose interest and motivation. They may rush through the remaining questions, provide random or arbitrary responses, or abandon the questionnaire altogether. This can compromise the validity and reliability of the collected data.

Researchers can employ several strategies to address the issue of response fatigue in questionnaires. First, keeping the questionnaire concise and focused is important, avoiding unnecessary or redundant questions. Researchers should carefully consider the relevance and importance of each question to ensure that only essential information is collected.

Additionally, researchers can employ skip logic or branching to customize the questionnaire based on respondents’ previous answers. This allows participants to skip irrelevant sections or questions, making the questionnaire more efficient and reducing the risk of response fatigue.

Breaking the questionnaire into sections or pages with clear headings or progress indicators can help alleviate response fatigue. This creates a sense of accomplishment as participants move through the questionnaire, making it more manageable and less overwhelming.

Researchers can also consider using visual aids, such as charts, tables, or images, to present information more engaging and visually appealingly. This can help sustain participants’ interest and reduce the monotony of text-based questions.

Moreover, researchers should be mindful of the time required to complete the questionnaire and provide an estimated time commitment to participants upfront. This allows respondents to allocate sufficient time and mental energy, minimizing the likelihood of response fatigue.

Lastly, piloting the questionnaire with a small sample of participants can help identify potential issues related to response fatigue. Feedback from pilot participants can inform necessary revisions to the questionnaire structure, length, or wording to ensure a more engaging and user-friendly experience.

By implementing these strategies, researchers can mitigate the risk of response fatigue in questionnaires, encouraging participants to remain engaged, focused, and motivated throughout the process. This ultimately leads to more reliable and valid data.

Why use an online questionnaire

There are numerous advantages to using online questionnaires.

Firstly, as mentioned, they can be sent out quickly and the turnaround can be relatively short. You’re able to reach your audience in the moments that matter – while they’re browsing your products, as they read your content, as soon as they’ve made a purchase.

Online questionnaires allow users time to consider responses – a distinct advantage over face-to-face or telephone methods. Besides these great perks, the cost of online questionnaires is low and actionable items produced by the data can produce a high return on investment.

There are additional benefits as well.

User responses can be precoded, eliminating errors that are possible with human transcription . The data is already in an electronic format, allowing for easy analysis without the hassle of digitizing data.

Lastly, guidance and/or software is available through providers such as Pointerpro at an affordable cost and with a wide range of possibilities to make it fit your brand and your requirements.

How to motivate respondents to participate

Questionnaires have a bad rep and over the past years, many researchers saw their response rate decline due to their unpopularity. That’s why we’ve compiled an extensive list of tips that can help you  boost your response rate .

The most obvious way to do this is to reward respondents for their time through gifts, credits or payment. Pointerpro even offers fun reward tools like a digital scratch card or slot machine.

Rewarding a respondent is just one means of getting responses. Respondents also want to make sure their privacy is protected, make it clear that answers are confidential and make sure the privacy policy is up-to-date. Emphasize how the data will be useful to the user experience or helpful to society. People like to feel like they’re doing a good thing.

They also want to know that they can complete your questionnaire without the hassle of dinner time calls. Respecting the respondent’s time goes a long way towards showing you appreciate their participation.

Be sure to send reminders as well – many people forget anything that isn’t at the top of their minds.

Find inspiration for your questionnaire design

Designing a questionnaire may sound simple until you sit down to write the questions. Planning,  content creation  and  graphic design  are all important. Not everyone has the time or professional design skills to create a questionnaire.

That’s why some software companies like Pointerpro offer templates to create questions in minutes.

Now that you know how helpful and effective questionnaires and surveys can be, get started with Pointerpro’s 14-day free trial!

Create your own assessment for free!

About the author:.

Nigel Lindemann

54 responses.

Generally I don’t read article on blogs, however I wish to say that this write-up very compelled me to take a look at and do it! Your writing style has been surprised me. Thanks, quite nice article.

Thanks so much for this article,it was an eye opener when i was writing my assignment on this topic.

Informative and easy to follow. Far much less jargon!

nice informative blog on Questionnaire and surveys.

Very informative. Thank you very much

Great article, helped me a lot.

The information is well understand thaks

The information is clear & well stated. Thanks

Great it was a helpful read

Awesome, so simple and well understood, thanks alot

well understood, thank you

nice…..keep up!

nice article i love it so interesting

nice article,thanks alot

Very informative, thanks very much

Wow…nice article

Comprehensive article. It has really helped

Very nice explanation. all points are easy to learn.

Thanks this article really helped with my assignment ♥️

Well understood.

Please help me with the disadvantages of mail questionnaires as a method of primary data collection in business research.

thanks it really helped me

Thank you It was helpful

great research info

Thank you ,it was helpfull in my assignments.

Hi! It would be nice if you gave information about your sources, so that students could learn a bit more. Thanks!

this is a great article thanks a lot

Thanks… So clear and simple.

thank you so much!!!really helped me in attempting assignment questions

very simple yet comprehensive article… it was a huge help ❤

This really helped. Thank you!

Thank you …… really helped

Great stuff surveyors.

This was really helpful indeed. I like.

Good job. But I want to know the disadvantages of community data collection. Thanks.

So helpful……

Market research can no longer be done manually because of the advancement in technology and the internet. Without a market research tool, marketers will not be able to keep up with the increasing demand for data-driven strategies.

such an amazing article helped me loads!

Thanks so much. I’ve learnt a lot from this piece as I prepare for my exams

Great Effort… this is so helpful for me…Thanks

Thanks for sharing the significant information. It helped me and will help others a lot. Nice.. …

Wonderful article

good explination

Great article Stefan. Very much appreciated.

Thank you so much, This has really helped me.

The answers are OK but there is lack of references, thank you

Thanks for your contribution

Great article????it was very helpful and well ????explained.Thank you so much.????Keep it up.10/10

Great.How do I reference this??

so helpful very accurate and on point

Really good article

Actually I can say thank you for helping me to attempt my questions and exams notes are well understood.

Recommended reading

How youth development specialist Halogen reaches thousands of young Singaporeans with Pointerpro

Singapore, in Southeast Asia, is a global, economic powerhouse with a population of a little less than 6 million people.

Accountancy marketing: 3 ways to make the numbers human again [case study]

Like so many aspects of business, accountancy practices have made their ascent into the cloud. For accountants, that means more

Business writing to grab attention in the digital world: Write like a human

During the first years of my professional career, I was no copywriter. But I was notorious for writing colossal emails.

Want to know more?

Subscribe to our newsletter and get hand-picked articles directly to your inbox.

The Pointerpro newsletter brings you the latest market trends and will give you an overview of compelling testimonials from our community. Register now!

Popular Articles

Get to know us.

Contact us Careers (We’re hiring!)

Help Center

Careers (We’re hiring!)

Let’s stay in touch, we’d like that

• Privacy Policy

Buy Me a Coffee

Home » Questionnaire – Definition, Types, and Examples

Questionnaire – Definition, Types, and Examples

Table of Contents

Questionnaire

Definition:

A Questionnaire is a research tool or survey instrument that consists of a set of questions or prompts designed to gather information from individuals or groups of people.

It is a standardized way of collecting data from a large number of people by asking them a series of questions related to a specific topic or research objective. The questions may be open-ended or closed-ended, and the responses can be quantitative or qualitative. Questionnaires are widely used in research, marketing, social sciences, healthcare, and many other fields to collect data and insights from a target population.

History of Questionnaire

The history of questionnaires can be traced back to the ancient Greeks, who used questionnaires as a means of assessing public opinion. However, the modern history of questionnaires began in the late 19th century with the rise of social surveys.

The first social survey was conducted in the United States in 1874 by Francis A. Walker, who used a questionnaire to collect data on labor conditions. In the early 20th century, questionnaires became a popular tool for conducting social research, particularly in the fields of sociology and psychology.

One of the most influential figures in the development of the questionnaire was the psychologist Raymond Cattell, who in the 1940s and 1950s developed the personality questionnaire, a standardized instrument for measuring personality traits. Cattell’s work helped establish the questionnaire as a key tool in personality research.

In the 1960s and 1970s, the use of questionnaires expanded into other fields, including market research, public opinion polling, and health surveys. With the rise of computer technology, questionnaires became easier and more cost-effective to administer, leading to their widespread use in research and business settings.

Today, questionnaires are used in a wide range of settings, including academic research, business, healthcare, and government. They continue to evolve as a research tool, with advances in computer technology and data analysis techniques making it easier to collect and analyze data from large numbers of participants.

Types of Questionnaire

Types of Questionnaires are as follows:

Structured Questionnaire

This type of questionnaire has a fixed format with predetermined questions that the respondent must answer. The questions are usually closed-ended, which means that the respondent must select a response from a list of options.

Unstructured Questionnaire

An unstructured questionnaire does not have a fixed format or predetermined questions. Instead, the interviewer or researcher can ask open-ended questions to the respondent and let them provide their own answers.

Open-ended Questionnaire

An open-ended questionnaire allows the respondent to answer the question in their own words, without any pre-determined response options. The questions usually start with phrases like “how,” “why,” or “what,” and encourage the respondent to provide more detailed and personalized answers.

Close-ended Questionnaire

In a closed-ended questionnaire, the respondent is given a set of predetermined response options to choose from. This type of questionnaire is easier to analyze and summarize, but may not provide as much insight into the respondent’s opinions or attitudes.

Mixed Questionnaire

A mixed questionnaire is a combination of open-ended and closed-ended questions. This type of questionnaire allows for more flexibility in terms of the questions that can be asked, and can provide both quantitative and qualitative data.

Pictorial Questionnaire:

In a pictorial questionnaire, instead of using words to ask questions, the questions are presented in the form of pictures, diagrams or images. This can be particularly useful for respondents who have low literacy skills, or for situations where language barriers exist. Pictorial questionnaires can also be useful in cross-cultural research where respondents may come from different language backgrounds.

Types of Questions in Questionnaire

The types of Questions in Questionnaire are as follows:

Multiple Choice Questions

These questions have several options for participants to choose from. They are useful for getting quantitative data and can be used to collect demographic information.

• a. Red b . Blue c. Green d . Yellow

Rating Scale Questions

These questions ask participants to rate something on a scale (e.g. from 1 to 10). They are useful for measuring attitudes and opinions.

• On a scale of 1 to 10, how likely are you to recommend this product to a friend?

Open-Ended Questions

These questions allow participants to answer in their own words and provide more in-depth and detailed responses. They are useful for getting qualitative data.

• What do you think are the biggest challenges facing your community?

Likert Scale Questions

These questions ask participants to rate how much they agree or disagree with a statement. They are useful for measuring attitudes and opinions.

How strongly do you agree or disagree with the following statement:

“I enjoy exercising regularly.”

• a . Strongly Agree
• c . Neither Agree nor Disagree
• d . Disagree
• e . Strongly Disagree

Demographic Questions

These questions ask about the participant’s personal information such as age, gender, ethnicity, education level, etc. They are useful for segmenting the data and analyzing results by demographic groups.

• What is your age?

Yes/No Questions

These questions only have two options: Yes or No. They are useful for getting simple, straightforward answers to a specific question.

Have you ever traveled outside of your home country?

Ranking Questions

These questions ask participants to rank several items in order of preference or importance. They are useful for measuring priorities or preferences.

Please rank the following factors in order of importance when choosing a restaurant:

• a. Quality of Food
• c. Ambiance
• d. Location

Matrix Questions

These questions present a matrix or grid of options that participants can choose from. They are useful for getting data on multiple variables at once.

Dichotomous Questions

These questions present two options that are opposite or contradictory. They are useful for measuring binary or polarized attitudes.

Do you support the death penalty?

How to Make a Questionnaire

Step-by-Step Guide for Making a Questionnaire:

• Define your research objectives: Before you start creating questions, you need to define the purpose of your questionnaire and what you hope to achieve from the data you collect.
• Choose the appropriate question types: Based on your research objectives, choose the appropriate question types to collect the data you need. Refer to the types of questions mentioned earlier for guidance.
• Develop questions: Develop clear and concise questions that are easy for participants to understand. Avoid leading or biased questions that might influence the responses.
• Organize questions: Organize questions in a logical and coherent order, starting with demographic questions followed by general questions, and ending with specific or sensitive questions.
• Pilot the questionnaire : Test your questionnaire on a small group of participants to identify any flaws or issues with the questions or the format.
• Refine the questionnaire : Based on feedback from the pilot, refine and revise the questionnaire as necessary to ensure that it is valid and reliable.
• Distribute the questionnaire: Distribute the questionnaire to your target audience using a method that is appropriate for your research objectives, such as online surveys, email, or paper surveys.
• Collect and analyze data: Collect the completed questionnaires and analyze the data using appropriate statistical methods. Draw conclusions from the data and use them to inform decision-making or further research.
• Report findings: Present your findings in a clear and concise report, including a summary of the research objectives, methodology, key findings, and recommendations.

Questionnaire Administration Modes

There are several modes of questionnaire administration. The choice of mode depends on the research objectives, sample size, and available resources. Some common modes of administration include:

• Self-administered paper questionnaires: Participants complete the questionnaire on paper, either in person or by mail. This mode is relatively low cost and easy to administer, but it may result in lower response rates and greater potential for errors in data entry.
• Online questionnaires: Participants complete the questionnaire on a website or through email. This mode is convenient for both researchers and participants, as it allows for fast and easy data collection. However, it may be subject to issues such as low response rates, lack of internet access, and potential for fraudulent responses.
• Telephone surveys: Trained interviewers administer the questionnaire over the phone. This mode allows for a large sample size and can result in higher response rates, but it is also more expensive and time-consuming than other modes.
• Face-to-face interviews : Trained interviewers administer the questionnaire in person. This mode allows for a high degree of control over the survey environment and can result in higher response rates, but it is also more expensive and time-consuming than other modes.
• Mixed-mode surveys: Researchers use a combination of two or more modes to administer the questionnaire, such as using online questionnaires for initial screening and following up with telephone interviews for more detailed information. This mode can help overcome some of the limitations of individual modes, but it requires careful planning and coordination.

Example of Questionnaire

Title of the Survey: Customer Satisfaction Survey

Introduction:

We appreciate your business and would like to ensure that we are meeting your needs. Please take a few minutes to complete this survey so that we can better understand your experience with our products and services. Your feedback is important to us and will help us improve our offerings.

Instructions:

Please read each question carefully and select the response that best reflects your experience. If you have any additional comments or suggestions, please feel free to include them in the space provided at the end of the survey.

1. How satisfied are you with our product quality?

• Very satisfied
• Somewhat satisfied
• Somewhat dissatisfied
• Very dissatisfied

2. How satisfied are you with our customer service?

3. How satisfied are you with the price of our products?

4. How likely are you to recommend our products to others?

• Very likely
• Somewhat likely
• Somewhat unlikely
• Very unlikely

5. How easy was it to find the information you were looking for on our website?

• Somewhat easy
• Somewhat difficult
• Very difficult

6. How satisfied are you with the overall experience of using our products and services?

7. Is there anything that you would like to see us improve upon or change in the future?

…………………………………………………………………………………………………………………………..

Conclusion:

Thank you for taking the time to complete this survey. Your feedback is valuable to us and will help us improve our products and services. If you have any further comments or concerns, please do not hesitate to contact us.

Applications of Questionnaire

Some common applications of questionnaires include:

• Research : Questionnaires are commonly used in research to gather information from participants about their attitudes, opinions, behaviors, and experiences. This information can then be analyzed and used to draw conclusions and make inferences.
• Healthcare : In healthcare, questionnaires can be used to gather information about patients’ medical history, symptoms, and lifestyle habits. This information can help healthcare professionals diagnose and treat medical conditions more effectively.
• Marketing : Questionnaires are commonly used in marketing to gather information about consumers’ preferences, buying habits, and opinions on products and services. This information can help businesses develop and market products more effectively.
• Human Resources: Questionnaires are used in human resources to gather information from job applicants, employees, and managers about job satisfaction, performance, and workplace culture. This information can help organizations improve their hiring practices, employee retention, and organizational culture.
• Education : Questionnaires are used in education to gather information from students, teachers, and parents about their perceptions of the educational experience. This information can help educators identify areas for improvement and develop more effective teaching strategies.

Purpose of Questionnaire

Some common purposes of questionnaires include:

• To collect information on attitudes, opinions, and beliefs: Questionnaires can be used to gather information on people’s attitudes, opinions, and beliefs on a particular topic. For example, a questionnaire can be used to gather information on people’s opinions about a particular political issue.
• To collect demographic information: Questionnaires can be used to collect demographic information such as age, gender, income, education level, and occupation. This information can be used to analyze trends and patterns in the data.
• To measure behaviors or experiences: Questionnaires can be used to gather information on behaviors or experiences such as health-related behaviors or experiences, job satisfaction, or customer satisfaction.
• To evaluate programs or interventions: Questionnaires can be used to evaluate the effectiveness of programs or interventions by gathering information on participants’ experiences, opinions, and behaviors.
• To gather information for research: Questionnaires can be used to gather data for research purposes on a variety of topics.

When to use Questionnaire

Here are some situations when questionnaires might be used:

• When you want to collect data from a large number of people: Questionnaires are useful when you want to collect data from a large number of people. They can be distributed to a wide audience and can be completed at the respondent’s convenience.
• When you want to collect data on specific topics: Questionnaires are useful when you want to collect data on specific topics or research questions. They can be designed to ask specific questions and can be used to gather quantitative data that can be analyzed statistically.
• When you want to compare responses across groups: Questionnaires are useful when you want to compare responses across different groups of people. For example, you might want to compare responses from men and women, or from people of different ages or educational backgrounds.
• When you want to collect data anonymously: Questionnaires can be useful when you want to collect data anonymously. Respondents can complete the questionnaire without fear of judgment or repercussions, which can lead to more honest and accurate responses.
• When you want to save time and resources: Questionnaires can be more efficient and cost-effective than other methods of data collection such as interviews or focus groups. They can be completed quickly and easily, and can be analyzed using software to save time and resources.

Characteristics of Questionnaire

Here are some of the characteristics of questionnaires:

• Standardization : Questionnaires are standardized tools that ask the same questions in the same order to all respondents. This ensures that all respondents are answering the same questions and that the responses can be compared and analyzed.
• Objectivity : Questionnaires are designed to be objective, meaning that they do not contain leading questions or bias that could influence the respondent’s answers.
• Predefined responses: Questionnaires typically provide predefined response options for the respondents to choose from, which helps to standardize the responses and make them easier to analyze.
• Quantitative data: Questionnaires are designed to collect quantitative data, meaning that they provide numerical or categorical data that can be analyzed using statistical methods.
• Convenience : Questionnaires are convenient for both the researcher and the respondents. They can be distributed and completed at the respondent’s convenience and can be easily administered to a large number of people.
• Anonymity : Questionnaires can be anonymous, which can encourage respondents to answer more honestly and provide more accurate data.
• Reliability : Questionnaires are designed to be reliable, meaning that they produce consistent results when administered multiple times to the same group of people.
• Validity : Questionnaires are designed to be valid, meaning that they measure what they are intended to measure and are not influenced by other factors.

Advantage of Questionnaire

Some Advantage of Questionnaire are as follows:

• Standardization: Questionnaires allow researchers to ask the same questions to all participants in a standardized manner. This helps ensure consistency in the data collected and eliminates potential bias that might arise if questions were asked differently to different participants.
• Efficiency: Questionnaires can be administered to a large number of people at once, making them an efficient way to collect data from a large sample.
• Anonymity: Participants can remain anonymous when completing a questionnaire, which may make them more likely to answer honestly and openly.
• Cost-effective: Questionnaires can be relatively inexpensive to administer compared to other research methods, such as interviews or focus groups.
• Objectivity: Because questionnaires are typically designed to collect quantitative data, they can be analyzed objectively without the influence of the researcher’s subjective interpretation.
• Flexibility: Questionnaires can be adapted to a wide range of research questions and can be used in various settings, including online surveys, mail surveys, or in-person interviews.

Limitations of Questionnaire

Limitations of Questionnaire are as follows:

• Limited depth: Questionnaires are typically designed to collect quantitative data, which may not provide a complete understanding of the topic being studied. Questionnaires may miss important details and nuances that could be captured through other research methods, such as interviews or observations.
• R esponse bias: Participants may not always answer questions truthfully or accurately, either because they do not remember or because they want to present themselves in a particular way. This can lead to response bias, which can affect the validity and reliability of the data collected.
• Limited flexibility: While questionnaires can be adapted to a wide range of research questions, they may not be suitable for all types of research. For example, they may not be appropriate for studying complex phenomena or for exploring participants’ experiences and perceptions in-depth.
• Limited context: Questionnaires typically do not provide a rich contextual understanding of the topic being studied. They may not capture the broader social, cultural, or historical factors that may influence participants’ responses.
• Limited control : Researchers may not have control over how participants complete the questionnaire, which can lead to variations in response quality or consistency.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Survey Research – Types, Methods, Examples

Questionnaire Method In Research

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

A questionnaire is a research instrument consisting of a series of questions for the purpose of gathering information from respondents. Questionnaires can be thought of as a kind of written interview . They can be carried out face to face, by telephone, computer, or post.

Questionnaires provide a relatively cheap, quick, and efficient way of obtaining large amounts of information from a large sample of people.

Data can be collected relatively quickly because the researcher would not need to be present when completing the questionnaires. This is useful for large populations when interviews would be impractical.

However, a problem with questionnaires is that respondents may lie due to social desirability. Most people want to present a positive image of themselves, and may lie or bend the truth to look good, e.g., pupils exaggerate revision duration.

Questionnaires can effectively measure relatively large subjects’ behavior, attitudes, preferences, opinions, and intentions more cheaply and quickly than other methods.

Often, a questionnaire uses both open and closed questions to collect data. This is beneficial as it means both quantitative and qualitative data can be obtained.

Closed Questions

A closed-ended question requires a specific, limited response, often “yes” or “no” or a choice that fit into pre-decided categories.

Data that can be placed into a category is called nominal data. The category can be restricted to as few as two options, i.e., dichotomous (e.g., “yes” or “no,” “male” or “female”), or include quite complex lists of alternatives from which the respondent can choose (e.g., polytomous).

Closed questions can also provide ordinal data (which can be ranked). This often involves using a continuous rating scale to measure the strength of attitudes or emotions.

For example, strongly agree / agree / neutral / disagree / strongly disagree / unable to answer.

Closed questions have been used to research type A personality (e.g., Friedman & Rosenman, 1974) and also to assess life events that may cause stress (Holmes & Rahe, 1967) and attachment (Fraley, Waller, & Brennan, 2000).

• They can be economical. This means they can provide large amounts of research data for relatively low costs. Therefore, a large sample size can be obtained, which should represent the population from which a researcher can then generalize.
• The respondent provides information that can be easily converted into quantitative data (e.g., count the number of “yes” or “no” answers), allowing statistical analysis of the responses.
• The questions are standardized. All respondents are asked exactly the same questions in the same order. This means a questionnaire can be replicated easily to check for reliability . Therefore, a second researcher can use the questionnaire to confirm consistent results.

Limitations

• They lack detail. Because the responses are fixed, there is less scope for respondents to supply answers that reflect their true feelings on a topic.

Open Questions

Open questions allow for expansive, varied answers without preset options or limitations.

Open questions allow people to express what they think in their own words. Open-ended questions enable the respondent to answer in as much detail as they like in their own words. For example: “can you tell me how happy you feel right now?”

Open questions will work better if you want to gather more in-depth answers from your respondents. These give no pre-set answer options and instead, allow the respondents to put down exactly what they like in their own words.

Open questions are often used for complex questions that cannot be answered in a few simple categories but require more detail and discussion.

Lawrence Kohlberg presented his participants with moral dilemmas. One of the most famous concerns a character called Heinz, who is faced with the choice between watching his wife die of cancer or stealing the only drug that could help her.

Participants were asked whether Heinz should steal the drug or not and, more importantly, for their reasons why upholding or breaking the law is right.

• Rich qualitative data is obtained as open questions allow respondents to elaborate on their answers. This means the research can determine why a person holds a certain attitude .
• Time-consuming to collect the data. It takes longer for the respondent to complete open questions. This is a problem as a smaller sample size may be obtained.
• Time-consuming to analyze the data. It takes longer for the researcher to analyze qualitative data as they have to read the answers and try to put them into categories by coding, which is often subjective and difficult. However, Smith (1992) has devoted an entire book to the issues of thematic content analysis that includes 14 different scoring systems for open-ended questions.
• Not suitable for less educated respondents as open questions require superior writing skills and a better ability to express one’s feelings verbally.

Questionnaire Design

With some questionnaires suffering from a response rate as low as 5%, a questionnaire must be well designed.

There are several important factors in questionnaire design.

Pilot Study

Question order.

Questions should progress logically from the least sensitive to the most sensitive, from the factual and behavioral to the cognitive, and from the more general to the more specific.

The researcher should ensure that previous questions do not influence the answer to a question.

Question order effects

• Question order effects occur when responses to an earlier question affect responses to a later question in a survey. They can arise at different stages of the survey response process – interpretation, information retrieval, judgment/estimation, and reporting.
• Types of question order effects include: unconditional (subsequent answers affected by prior question topic), conditional (subsequent answers depend on the response to the prior question), and associational (correlation between two questions changes based on order).
• Question order effects have been found across different survey topics like social and political attitudes, health and safety studies, vignette research, etc. Effects may be moderated by respondent factors like age, education level, knowledge and attitudes about the topic.
• To minimize question order effects, recommendations include avoiding judgmental dependencies, separating potentially reactive questions, randomizing questions, following good survey design principles, considering respondent characteristics, and intentionally examining question context and order.

Terminology

• There should be a minimum of technical jargon. Questions should be simple, to the point, and easy to understand. The language of a questionnaire should be appropriate to the vocabulary of the group of people being studied.
• Use statements that are interpreted in the same way by members of different subpopulations of the population of interest.
• For example, the researcher must change the language of questions to match the social background of the respondent’s age / educational level / social class/ethnicity, etc.

Presentation

Ethical issues.

• The researcher must ensure that the information provided by the respondent is kept confidential, e.g., name, address, etc.
• This means questionnaires are good for researching sensitive topics as respondents will be more honest when they cannot be identified.
• Keeping the questionnaire confidential should also reduce the likelihood of psychological harm, such as embarrassment.
• Participants must provide informed consent before completing the questionnaire and must be aware that they have the right to withdraw their information at any time during the survey/ study.

Problems with Postal Questionnaires

At first sight, the postal questionnaire seems to offer the opportunity to get around the problem of interview bias by reducing the personal involvement of the researcher. Its other practical advantages are that it is cheaper than face-to-face interviews and can quickly contact many respondents scattered over a wide area.

However, these advantages must be weighed against the practical problems of conducting research by post. A lack of involvement by the researcher means there is little control over the information-gathering process.

The data might not be valid (i.e., truthful) as we can never be sure that the questionnaire was completed by the person to whom it was addressed.

That, of course, assumes there is a reply in the first place, and one of the most intractable problems of mailed questionnaires is a low response rate. This diminishes the reliability of the data

Also, postal questionnaires may not represent the population they are studying. This may be because:

• Some questionnaires may be lost in the post, reducing the sample size.
• The questionnaire may be completed by someone not a member of the research population.
• Those with strong views on the questionnaire’s subject are more likely to complete it than those without interest.

Benefits of a Pilot Study

A pilot study is a practice / small-scale study conducted before the main study.

It allows the researcher to try out the study with a few participants so that adjustments can be made before the main study, saving time and money.

It is important to conduct a questionnaire pilot study for the following reasons:

• Check that respondents understand the terminology used in the questionnaire.
• Check that emotive questions are not used, as they make people defensive and could invalidate their answers.
• Check that leading questions have not been used as they could bias the respondent’s answer.
• Ensure the questionnaire can be completed in an appropriate time frame (i.e., it’s not too long).

Frequently Asked Questions 

How do psychological researchers analyze the data collected from questionnaires.

Psychological researchers analyze questionnaire data by looking for patterns and trends in people’s responses. They use numbers and charts to summarize the information.

They calculate things like averages and percentages to see what most people think or feel. They also compare different groups to see if there are any differences between them.

By doing these analyses, researchers can understand how people think, feel, and behave. This helps them make conclusions and learn more about how our minds work.

Are questionnaires effective in gathering accurate data?

Yes, questionnaires can be effective in gathering accurate data. When designed well, with clear and understandable questions, they allow individuals to express their thoughts, opinions, and experiences.

However, the accuracy of the data depends on factors such as the honesty and accuracy of respondents’ answers, their understanding of the questions, and their willingness to provide accurate information. Researchers strive to create reliable and valid questionnaires to minimize biases and errors.

It’s important to remember that while questionnaires can provide valuable insights, they are just one tool among many used in psychological research.

Can questionnaires be used with diverse populations and cultural contexts?

Yes, questionnaires can be used with diverse populations and cultural contexts. Researchers take special care to ensure that questionnaires are culturally sensitive and appropriate for different groups.

This means adapting the language, examples, and concepts to match the cultural context. By doing so, questionnaires can capture the unique perspectives and experiences of individuals from various backgrounds.

This helps researchers gain a more comprehensive understanding of human behavior and ensures that everyone’s voice is heard and represented in psychological research.

Are questionnaires the only method used in psychological research?

No, questionnaires are not the only method used in psychological research. Psychologists use a variety of research methods, including interviews, observations , experiments , and psychological tests.

Each method has its strengths and limitations, and researchers choose the most appropriate method based on their research question and goals.

Questionnaires are valuable for gathering self-report data, but other methods allow researchers to directly observe behavior, study interactions, or manipulate variables to test hypotheses.

By using multiple methods, psychologists can gain a more comprehensive understanding of human behavior and mental processes.

What is a semantic differential scale?

The semantic differential scale is a questionnaire format used to gather data on individuals’ attitudes or perceptions. It’s commonly incorporated into larger surveys or questionnaires to assess subjective qualities or feelings about a specific topic, product, or concept by quantifying them on a scale between two bipolar adjectives.

It presents respondents with a pair of opposite adjectives (e.g., “happy” vs. “sad”) and asks them to mark their position on a scale between them, capturing the intensity of their feelings about a particular subject.

It quantifies subjective qualities, turning them into data that can be statistically analyzed.

Ayidiya, S. A., & McClendon, M. J. (1990). Response effects in mail surveys. Public Opinion Quarterly, 54 (2), 229–247. https://doi.org/10.1086/269200

Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000). An item-response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350-365.

Friedman, M., & Rosenman, R. H. (1974). Type A behavior and your heart . New York: Knopf.

Gold, R. S., & Barclay, A. (2006). Order of question presentation and correlation between judgments of comparative and own risk. Psychological Reports, 99 (3), 794–798. https://doi.org/10.2466/PR0.99.3.794-798

Holmes, T. H., & Rahe, R. H. (1967). The social readjustment rating scale. Journal of psychosomatic research, 11(2) , 213-218.

Schwarz, N., & Hippler, H.-J. (1995). Subsequent questions may influence answers to preceding questions in mail surveys. Public Opinion Quarterly, 59 (1), 93–97. https://doi.org/10.1086/269460

Smith, C. P. (Ed.). (1992). Motivation and personality: Handbook of thematic content analysis . Cambridge University Press.

Further Information

• Questionnaire design and scale development
• Questionnaire Appraisal Form

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Psychol Med
• v.42(6); 2020 Nov

The Limitations of Online Surveys

Chittaranjan andrade.

1 Dept. of Clinical Psychopharmacology and Neurotoxicology, National Institute of Mental Health and Neurosciences, Bangalore, Karnataka, India.

Online surveys are growing in popularity, perhaps because they are an easy, convenient, and inexpensive means of data collection. Online surveys commonly suffer from two serious methodological limitations: the population to which they are distributed cannot be described, and respondents with biases may select themselves into the sample. Research is of value only when the findings from a sample can be generalized to a meaningful population. When the population addressed by the survey cannot be described, and when the sample is contaminated by respondents with biases, findings from online surveys cannot be generalized and may therefore mislead.

A n online survey collects information from people who respond to a form or instrument that is distributed through internet channels. Online surveys have several advantages. 1 They are easy to conduct using free platforms such as SurveyMonkey, Google Forms, and Typeform. They can be quickly created and distributed with global reach; this is important when information is urgently sought, such as in the context of the current Covid-19 pandemic. There are few to no administrative costs. Analysis is usually easy to perform on the same platform at which the survey was conducted.

Online surveys are becoming increasingly popular. There were 1682 PubMed hits for “online survey” (search phrase entered with quotes) in 2016; this number increased to 1994 in 2016, 2425 in 2017, 2872 in 2018, and 3182 in 2019. On August 15, 2020, the number of hits for 2020 was already 2742; when annualized, this number projects to 4387. In contrast, the growth for the search term “meta-analysis” has been lower, from 17,806 in 2015 to 24,033 in 2019; and “randomized controlled trial” has shown negative growth, from 31,689 in 2015 to 26,285 in 2019. This article explains why, despite their popularity, online surveys may be methodologically unsound.

Generalizing from Sample to Population

In a hypothetical study that we conducted at our hospital in 2019, patients with schizophrenia expressed positive attitudes toward long-acting injectable antipsychotic medications. This finding truly applies only to the patients whom we studied. There is no assurance that the same finding will be obtained in a different set of patients studied in the same hospital, let alone in patients studied elsewhere in the world and in later years.

Research findings are of scientific value only if they can be generalized. At the very least, it should be possible to generalize from the sample to the population from which the sample was drawn. This can happen only if the sample is representative of the population, which requires two conditions to be fulfilled. The first condition is that the population must be known; it is not possible to generalize the findings of a study to a population that cannot be defined. The second condition is that a valid method of sampling should have been adopted; a method that recruits a sample that is overrepresented for a particular characteristic cannot represent the population.

An Absurd Example

Imagine that we create a questionnaire to study how Covid-19 has affected the mental health of the population. We leave copies of our questionnaire on a park bench. These sheets of paper are blown by the wind to different destinations. Most people regard the questionnaire as litter. Some pick it up, look at it, and drop it. A few take the trouble to answer the questions and return the questionnaire. We analyze the data in the returned forms and conclude, perhaps, that Covid-19 has compromised the mental health of persons in our city or country.

These conclusions are not valid because we have no idea at all about the population that had access to the questionnaire, let alone whom the questionnaire actually reached; so we cannot define or describe the population to which the findings can be generalized. The questionnaire may have collected demographic details, but we cannot even generalize the findings to persons with these demographics because we did not select a random sample of persons with these demographics (we cannot assume that persons who responded to the survey were representative of their population demographics).

Sampling bias could have arisen in several ways. For example, the questionnaire would only have reached persons who were downwind and would only have been completed by those who were literate and those who were sufficiently interested in the topic to take the time and trouble to respond. What is the value of findings that can only be generalized to “persons who are sufficiently interested in the topic to respond to it”?

Online Surveys

The absurd example in the previous section mirrors what happens in online surveys. Online surveys are commonly distributed almost by the wind, through individual emails, mailing lists, and social media platforms. Requests are made by the investigators to forward the survey to other channels. There is no way of identifying, understanding, and describing the population that could have accessed and responded to the survey, and to whom the results of the survey can be generalized.

Furthermore, online surveys are completed only by persons who are literate and who have access to the internet, and by those who are sufficiently biased to be interested in the subject; why else would they take the time and trouble to respond? As an example, patients who were traumatized by a medical procedure are more likely to respond to an online survey about that procedure than patients who underwent the procedure uneventfully. The traumatized patients want to be heard; other patients feel no pressure to respond. Thus, persons with bias are overrepresented in online survey samples. The survey findings are thereby skewed. Because there is no way of knowing the motives of those who responded, there is no way of understanding the extent of bias in online surveys. 2 – 4

The Best Case Scenario

If the online survey is individually distributed to, say, all the members of a society whose names are listed in a membership directory, we know the population that the survey has reached. If most of the members respond to the survey, we can reasonably conclude that the opinions of the sample are similar to those of most if not all of the population. This is the best case scenario; the sample and population can both be described, and the sample is probably representative of the population. However, if only a small proportion of members respond, it becomes possible that the results are contaminated by respondent bias and are therefore untrustworthy.

Concluding Notes

The results of online surveys must be regarded as tentative when the surveys are distributed to an unknown audience and when voluntary participation in the survey can result in respondents with biases selecting themselves into the sample.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors received no financial support for the research, authorship, and/or publication of this article.