external validity in research meaning

• Privacy Policy

Home » External Validity – Threats, Examples and Types

External Validity – Threats, Examples and Types

Table of Contents

External Validity

Definition:

External validity refers to the extent to which the results of a study can be generalized or applied to a larger population, settings, or conditions beyond the specific context of the study. It is a measure of how well the findings of a study can be considered representative of the real world.

How To Increase External Validity

To increase external validity in research, researchers can employ several strategies to enhance the generalizability of their findings. Here are some common approaches:

Representative Sampling

Ensure that the sample used in the study is representative of the target population of interest. Random sampling techniques, such as simple random sampling or stratified sampling, can help reduce sampling bias and increase the likelihood of obtaining a representative sample.

Diverse Participant Characteristics

Include participants with diverse demographic characteristics, such as age, gender, socioeconomic status, and cultural backgrounds. This helps to ensure that the findings are applicable to a wider range of individuals.

Multiple Settings

Conduct the study in multiple settings or contexts to assess the robustness of the findings across different environments. This could involve replicating the study in different geographical locations, institutions, or organizations.

Large Sample Size

Increasing the sample size can improve the statistical power of the study and enhance the reliability of the findings. Larger samples are generally more representative of the population, making it easier to generalize the results.

Longitudinal Studies

Consider conducting longitudinal studies that span a longer duration. By observing changes and trends over time, researchers can provide a more comprehensive understanding of the phenomenon under investigation and increase the applicability of their findings.

Real-world Conditions

Strive to create conditions in the study that closely resemble real-world situations. This can be achieved by conducting field experiments, using naturalistic observation, or implementing interventions in real-life settings.

External Validation of Measures

Use established and validated measurement instruments to assess variables of interest. By employing recognized measures, researchers increase the likelihood that their findings can be compared and replicated in other studies.

Meta-Analysis

Conducting a meta-analysis, which involves systematically analyzing and combining the results of multiple studies on the same topic, can provide a more comprehensive view and increase the external validity by pooling findings from various sources.

Replication

Encourage replication of the study by other researchers. When multiple studies yield similar results, it strengthens the external validity of the findings.

Transparent Reporting

Clearly document the study design, methodology, and limitations in research publications. Transparent reporting allows readers to evaluate the study’s external validity and consider the potential generalizability of the findings.

Threats to External Validity

There are several threats to external validity that researchers should be aware of when interpreting the generalizability of their findings. These threats include:

Selection Bias

Participants in a study may not be representative of the target population due to the way they were selected or recruited. This can limit the generalizability of the findings to the broader population.

Sampling Bias

Even with random sampling techniques, there is a possibility of sampling bias. This occurs when certain segments of the population are underrepresented or overrepresented in the sample, leading to a skewed representation of the population.

Reactive or Interaction Effects of Testing

The act of participating in a study or being exposed to a specific experimental condition can influence participants’ behaviors or responses. This can lead to artificial results that may not occur in natural settings.

Experimental Setting

The controlled environment of a laboratory or research setting may differ significantly from real-world situations, potentially influencing participant behavior and limiting the generalizability of the findings.

Demand Characteristics

Participants may alter their behavior based on their perception of the study’s purpose or the researcher’s expectations. This can introduce biases and limit the external validity of the findings.

Novelty Effects

Participants may respond differently to novel or unusual conditions in a study, which may not accurately reflect their behavior in everyday life.

Hawthorne Effect

Participants may change their behavior simply because they are aware they are being observed. This effect can distort the findings and limit generalizability.

Experimenter Bias

The actions or behaviors of the researchers conducting the study can inadvertently influence participant responses or outcomes, impacting the generalizability of the findings.

Time-related Threats

The passage of time can affect the external validity of findings. Social, cultural, or technological changes that occur between the study and the application of the findings may limit their relevance.

Specificity of the Intervention or Treatment

If the study involves a specific intervention or treatment, the findings may be limited to that particular intervention and may not generalize to other similar interventions or treatments.

Publication Bias

The tendency of researchers or journals to publish studies with significant or positive findings can introduce a bias in the literature and limit the generalizability of research findings.

Types of External Validity

Types of External Validity are as follows:

Population Validity

Population validity refers to the extent to which the findings of a study can be generalized to the larger target population from which the study sample was drawn. If the sample is representative of the population in terms of relevant characteristics, such as age, gender, socioeconomic status, and ethnicity, the study’s findings are more likely to have high population validity.

Ecological Validity

Ecological validity refers to the extent to which the findings of a study can be generalized to real-world settings or conditions. It assesses whether the experimental conditions and procedures accurately represent the complexity and dynamics of the natural environment. High ecological validity suggests that the findings are applicable to everyday situations.

Temporal Validity

Temporal validity, also known as historical validity or generalizability over time, refers to the extent to which the findings of a study can be generalized across different time periods. It assesses whether the relationships or effects observed during the study remain consistent or change over time.

Cross-Cultural Validity

Cross-cultural validity refers to the extent to which the findings of a study can be generalized to different cultural contexts or populations. It examines whether the relationships or effects observed in one culture hold true in other cultures. Conducting research in multiple cultural settings can help establish cross-cultural validity.

Setting Validity

Setting validity refers to the extent to which the findings of a study can be generalized to different settings or environments. It assesses whether the relationships or effects observed in one specific setting can be replicated in other similar settings.

Task Validity

Task validity refers to the extent to which the findings of a study can be generalized to different tasks or activities. It examines whether the relationships or effects observed during a specific task are applicable to other tasks that share similar characteristics.

Measurement Validity

Measurement validity refers to the extent to which the chosen measurements or instruments accurately capture the constructs or variables of interest. It examines whether the relationships or effects observed are robust across different measurement tools or techniques.

Examples of External Validity

Here are some real-time examples of external validity:

Medical Research: A pharmaceutical company conducts a clinical trial to test the efficacy of a new drug on a specific population group (e.g., adults with diabetes). To ensure external validity, the company includes participants from diverse backgrounds, ages, and geographical locations to ensure that the results can be generalized to a broader population.

Educational Research: A study examines the effectiveness of a teaching method in improving student performance in mathematics. Researchers choose a sample of schools from different regions, representing various socioeconomic backgrounds, to ensure the findings can be applied to a wider range of schools and students.

Opinion Polls: A polling agency conducts a survey to understand public opinion on a particular political issue. To ensure external validity, the agency ensures a representative sample of respondents, considering factors such as age, gender, ethnicity, education level, and geographic location. This approach allows the findings to be generalized to the broader population.

Social Science Research: A study investigates the impact of a social intervention program on reducing crime rates in a specific neighborhood. To enhance external validity, researchers select neighborhoods that represent diverse socio-economic conditions and urban and rural settings. This approach increases the likelihood that the findings can be applied to similar neighborhoods in other locations.

Psychological Research: A psychology study examines the effects of a therapy technique on reducing anxiety levels in individuals. To improve external validity, the researchers recruit a diverse sample of participants, including individuals of different ages, genders, and cultural backgrounds. This ensures that the findings can be applicable to a broader range of individuals experiencing anxiety.

Applications of External Validity

External validity has several practical applications across various fields. Here are some specific applications of external validity:

Policy Development:

External validity helps policymakers make informed decisions by considering research findings from different contexts and populations. By examining the external validity of studies, policymakers can determine the applicability and generalizability of research results to their target population and policy goals.

Program Evaluation:

External validity is crucial in evaluating the effectiveness of programs or interventions. By assessing the external validity of evaluation studies, policymakers and program administrators can determine if the findings are applicable to their target population and whether similar interventions can be implemented in different settings.

Market Research:

External validity is essential in market research to understand consumer behavior and preferences. By conducting studies with representative samples, companies can extrapolate the findings to the broader consumer population, allowing them to make informed marketing and product development decisions.

Health Interventions:

External validity plays a significant role in healthcare research. It helps researchers and healthcare practitioners understand the generalizability of treatment outcomes to diverse patient populations. By considering external validity, healthcare providers can determine if a specific treatment or intervention will be effective for their patients.

Education and Training:

External validity is important in educational research to ensure that instructional methods, educational interventions, and training programs are effective across diverse student populations and different educational settings. It helps educators and trainers make evidence-based decisions about instructional strategies that are likely to have positive outcomes in different contexts.

Public Opinion Research:

External validity is crucial in public opinion research, such as political polling or survey research. By ensuring a representative sample and considering external validity, researchers can generalize their findings to the larger population, providing insights into public sentiment and informing decision-making processes.

Advantages of External Validity

Here are some advantages of external validity:

• Generalizability: External validity allows researchers to generalize their findings to broader populations, settings, or conditions. It enables them to make inferences about how the results of a study might hold true in real-world situations beyond the controlled environment of the study.
• Real-world applicability: When a study has high external validity, the findings are more likely to be applicable and relevant to real-world scenarios. This is particularly important in fields such as medicine, psychology, and social sciences, where the goal is often to understand and improve human behavior and well-being.
• Increased confidence in findings: Studies with high external validity provide stronger evidence and increase confidence in the findings. When the results can be generalized to diverse populations or different contexts, it suggests that the observed effects are more robust and reliable.
• Enhanced ecological validity: External validity enhances ecological validity, which refers to the degree to which a study reflects real-life situations. When a study has good external validity, it increases the likelihood that the findings accurately represent the complexities and nuances of the real world.
• Policy implications: Research findings with high external validity are more likely to have practical implications for policy-making. Policymakers are interested in studies that can inform decisions and interventions on a larger scale. Studies with strong external validity provide a basis for making informed decisions and implementing effective policies.
• Replication and meta-analysis: External validity facilitates replication studies and meta-analyses, which involve combining the results of multiple studies. When studies have high external validity, it becomes easier to replicate the findings in different contexts or conduct meta-analyses to examine the overall effects across a range of studies.
• Improved understanding of causal relationships: External validity allows researchers to test the generalizability of causal relationships. By replicating studies in different settings or populations, researchers can examine whether the causal relationships observed in one context hold true in other contexts, providing a more comprehensive understanding of the phenomenon under investigation.

Limitations of External Validity

While external validity offers several advantages, it also has limitations that researchers need to consider. Here are some limitations of external validity:

• Specificity of conditions: The specific conditions and settings of a study may limit the generalizability of the findings. Factors such as the time period, location, and sample characteristics can influence the results. For example, cultural, socioeconomic, or geographical differences between the study sample and the target population may affect the generalizability of the findings.
• Selection bias: In many studies, participants are recruited through convenience sampling or other non-random methods, which can introduce selection bias. This means that the sample may not be representative of the larger population, reducing the external validity of the findings. Selection bias can limit the generalizability of the results to other populations or contexts.
• Artificiality of experimental settings: Studies conducted in controlled laboratory or experimental settings may lack ecological validity. The artificial conditions and controlled variables may not accurately reflect real-world complexities. Participants’ behavior in a laboratory setting may differ from their behavior in naturalistic settings, leading to limited generalizability.
• Novelty and awareness effects: Participants in research studies may behave differently simply because they are aware they are being studied. This awareness can lead to the novelty effect or demand characteristics, where participants alter their behavior in response to the study context or the researchers’ expectations. As a result, the observed effects may not accurately represent real-world behavior.
• Time-dependent effects: The relevance and applicability of research findings can change over time due to societal, technological, or cultural shifts. What may be true and valid today may not hold true in the future. Therefore, the external validity of a study’s findings may diminish as time progresses.
• Lack of contextual variation: Studies often focus on a narrow range of contexts or populations, limiting the understanding of how findings may vary across different contexts. The external validity of a study may be compromised if it fails to account for contextual variations that can influence the generalizability of the results.
• Replication challenges: While replication is important for assessing the external validity of a study, it can be challenging to replicate studies in different contexts or with diverse populations. Replication studies may encounter practical constraints, such as resource limitations, time constraints, or ethical considerations, which can limit the ability to establish external validity.

Also see Validity

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Internal Consistency Reliability – Methods...

Internal Validity – Threats, Examples and Guide

Reliability – Types, Examples and Guide

Test-Retest Reliability – Methods, Formula and...

Criterion Validity – Methods, Examples and...

Parallel Forms Reliability – Methods, Example...

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Sweepstakes
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Internal Validity vs. External Validity in Research

What they tell us about the meaningfulness and trustworthiness of research

Verywell / Bailey Mariner

• Internal Validity
• External Validity

How do you determine whether a psychology study is trustworthy and meaningful? Two characteristics that can help you assess research findings are internal and external validity.

• Internal validity measures how well a study is conducted (its structure) and how accurately its results reflect the studied group.
• External validity relates to how applicable the findings are in the real world.

These two concepts help researchers gauge if the results of a research study are trustworthy and meaningful.

Conclusions are warranted

Controls extraneous variables

Eliminates alternative explanations

Focus on accuracy and strong research methods

Findings can be generalized

Outcomes apply to practical situations

Results apply to the world at large

Results can be translated into another context

What Is Internal Validity in Research?

Internal validity is the extent to which a research study establishes a trustworthy cause-and-effect relationship. This type of validity depends largely on the study's procedures and how rigorously it is performed.

Internal validity is important because once established, it makes it possible to eliminate alternative explanations for a finding. If you implement a smoking cessation program, for instance, internal validity ensures that any improvement in the subjects is due to the treatment administered and not something else.

Internal validity is not a "yes or no" concept. Instead, we consider how confident we can be with study findings based on whether the research avoids traps that may make those findings questionable. The less chance there is for "confounding," the higher the internal validity and the more confident we can be.

Confounding refers to uncontrollable variables that come into play and can confuse the outcome of a study, making us unsure of whether we can trust that we have identified the cause-and-effect relationship.

In short, you can only be confident that a study is internally valid if you can rule out alternative explanations for the findings. Three criteria are required to assume cause and effect in a research study:

• The cause preceded the effect in terms of time.
• The cause and effect vary together.
• There are no other likely explanations for the relationship observed.

Factors That Improve Internal Validity

To ensure the internal validity of a study, you want to consider aspects of the research design that will increase the likelihood that you can reject alternative hypotheses. Many factors can improve internal validity in research, including:

• Blinding : Participants—and sometimes researchers—are unaware of what intervention they are receiving (such as using a placebo on some subjects in a medication study) to avoid having this knowledge bias their perceptions and behaviors, thus impacting the study's outcome
• Experimental manipulation : Manipulating an independent variable in a study (for instance, giving smokers a cessation program) instead of just observing an association without conducting any intervention (examining the relationship between exercise and smoking behavior)
• Random selection : Choosing participants at random or in a manner in which they are representative of the population that you wish to study
• Randomization or random assignment : Randomly assigning participants to treatment and control groups, ensuring that there is no systematic bias between the research groups
• Strict study protocol : Following specific procedures during the study so as not to introduce any unintended effects; for example, doing things differently with one group of study participants than you do with another group

Internal Validity Threats

Just as there are many ways to ensure internal validity, a list of potential threats should be considered when planning a study.

• Attrition : Participants dropping out or leaving a study, which means that the results are based on a biased sample of only the people who did not choose to leave (and possibly who all have something in common, such as higher motivation)
• Confounding : A situation in which changes in an outcome variable can be thought to have resulted from some type of outside variable not measured or manipulated in the study
• Diffusion : This refers to the results of one group transferring to another through the groups interacting and talking with or observing one another; this can also lead to another issue called resentful demoralization, in which a control group tries less hard because they feel resentful over the group that they are in
• Experimenter bias : An experimenter behaving in a different way with different groups in a study, which can impact the results (and is eliminated through blinding)
• Historical events : May influence the outcome of studies that occur over a period of time, such as a change in the political leader or a natural disaster that occurs, influencing how study participants feel and act
• Instrumentation : This involves "priming" participants in a study in certain ways with the measures used, causing them to react in a way that is different than they would have otherwise reacted
• Maturation : The impact of time as a variable in a study; for example, if a study takes place over a period of time in which it is possible that participants naturally change in some way (i.e., they grew older or became tired), it may be impossible to rule out whether effects seen in the study were simply due to the impact of time
• Statistical regression : The natural effect of participants at extreme ends of a measure falling in a certain direction due to the passage of time rather than being a direct effect of an intervention
• Testing : Repeatedly testing participants using the same measures influences outcomes; for example, if you give someone the same test three times, it is likely that they will do better as they learn the test or become used to the testing process, causing them to answer differently

What Is External Validity in Research?

External validity refers to how well the outcome of a research study can be expected to apply to other settings. This is important because, if external validity is established, it means that the findings can be generalizable to similar individuals or populations.

External validity affirmatively answers the question: Do the findings apply to similar people, settings, situations, and time periods?

Population validity and ecological validity are two types of external validity. Population validity refers to whether you can generalize the research outcomes to other populations or groups. Ecological validity refers to whether a study's findings can be generalized to additional situations or settings.

Another term called transferability refers to whether results transfer to situations with similar characteristics. Transferability relates to external validity and refers to a qualitative research design.

Factors That Improve External Validity

If you want to improve the external validity of your study, there are many ways to achieve this goal. Factors that can enhance external validity include:

• Field experiments : Conducting a study outside the laboratory, in a natural setting
• Inclusion and exclusion criteria : Setting criteria as to who can be involved in the research, ensuring that the population being studied is clearly defined
• Psychological realism : Making sure participants experience the events of the study as being real by telling them a "cover story," or a different story about the aim of the study so they don't behave differently than they would in real life based on knowing what to expect or knowing the study's goal
• Replication : Conducting the study again with different samples or in different settings to see if you get the same results; when many studies have been conducted on the same topic, a meta-analysis can also be used to determine if the effect of an independent variable can be replicated, therefore making it more reliable
• Reprocessing or calibration : Using statistical methods to adjust for external validity issues, such as reweighting groups if a study had uneven groups for a particular characteristic (such as age)

External Validity Threats

External validity is threatened when a study does not take into account the interaction of variables in the real world. Threats to external validity include:

• Pre- and post-test effects : When the pre- or post-test is in some way related to the effect seen in the study, such that the cause-and-effect relationship disappears without these added tests
• Sample features : When some feature of the sample used was responsible for the effect (or partially responsible), leading to limited generalizability of the findings
• Selection bias : Also considered a threat to internal validity, selection bias describes differences between groups in a study that may relate to the independent variable—like motivation or willingness to take part in the study, or specific demographics of individuals being more likely to take part in an online survey
• Situational factors : Factors such as the time of day of the study, its location, noise, researcher characteristics, and the number of measures used may affect the generalizability of findings

While rigorous research methods can ensure internal validity, external validity may be limited by these methods.

Internal Validity vs. External Validity

Internal validity and external validity are two research concepts that share a few similarities while also having several differences.

Similarities

One of the similarities between internal validity and external validity is that both factors should be considered when designing a study. This is because both have implications in terms of whether the results of a study have meaning.

Both internal validity and external validity are not "either/or" concepts. Therefore, you always need to decide to what degree a study performs in terms of each type of validity.

Each of these concepts is also typically reported in research articles published in scholarly journals . This is so that other researchers can evaluate the study and make decisions about whether the results are useful and valid.

Differences

The essential difference between internal validity and external validity is that internal validity refers to the structure of a study (and its variables) while external validity refers to the universality of the results. But there are further differences between the two as well.

For instance, internal validity focuses on showing a difference that is due to the independent variable alone. Conversely, external validity results can be translated to the world at large.

Internal validity and external validity aren't mutually exclusive. You can have a study with good internal validity but be overall irrelevant to the real world. You could also conduct a field study that is highly relevant to the real world but doesn't have trustworthy results in terms of knowing what variables caused the outcomes.

Examples of Validity

Perhaps the best way to understand internal validity and external validity is with examples.

Internal Validity Example

An example of a study with good internal validity would be if a researcher hypothesizes that using a particular mindfulness app will reduce negative mood. To test this hypothesis, the researcher randomly assigns a sample of participants to one of two groups: those who will use the app over a defined period and those who engage in a control task.

The researcher ensures that there is no systematic bias in how participants are assigned to the groups. They do this by blinding the research assistants so they don't know which groups the subjects are in during the experiment.

A strict study protocol is also used to outline the procedures of the study. Potential confounding variables are measured along with mood , such as the participants' socioeconomic status, gender, age, and other factors. If participants drop out of the study, their characteristics are examined to make sure there is no systematic bias in terms of who stays in.

External Validity Example

An example of a study with good external validity would be if, in the above example, the participants used the mindfulness app at home rather than in the laboratory. This shows that results appear in a real-world setting.

To further ensure external validity, the researcher clearly defines the population of interest and chooses a representative sample . They might also replicate the study's results using different technological devices.

Setting up an experiment so that it has both sound internal validity and external validity involves being mindful from the start about factors that can influence each aspect of your research.

It's best to spend extra time designing a structurally sound study that has far-reaching implications rather than to quickly rush through the design phase only to discover problems later on. Only when both internal validity and external validity are high can strong conclusions be made about your results.

Andrade C. Internal, external, and ecological validity in research design, conduct, and evaluation .  Indian J Psychol Med . 2018;40(5):498-499. doi:10.4103/IJPSYM.IJPSYM_334_18

San Jose State University. Internal and external validity .

Kemper CJ. Internal validity . In: Zeigler-Hill V, Shackelford TK, eds. Encyclopedia of Personality and Individual Differences . Springer International Publishing; 2017:1-3. doi:10.1007/978-3-319-28099-8_1316-1

Patino CM, Ferreira JC. Internal and external validity: can you apply research study results to your patients?   J Bras Pneumol . 2018;44(3):183. doi:10.1590/S1806-37562018000000164

Matthay EC, Glymour MM. A graphical catalog of threats to validity: Linking social science with epidemiology .  Epidemiology . 2020;31(3):376-384. doi:10.1097/EDE.0000000000001161

Amico KR. Percent total attrition: a poor metric for study rigor in hosted intervention designs .  Am J Public Health . 2009;99(9):1567-1575. doi:10.2105/AJPH.2008.134767

Kemper CJ. External validity . In: Zeigler-Hill V, Shackelford TK, eds. Encyclopedia of Personality and Individual Differences . Springer International Publishing; 2017:1-4. doi:10.1007/978-3-319-28099-8_1303-1

Desjardins E, Kurtz J, Kranke N, Lindeza A, Richter SH. Beyond standardization: improving external validity and reproducibility in experimental evolution . BioScience. 2021;71(5):543-552. doi:10.1093/biosci/biab008

Drude NI, Martinez Gamboa L, Danziger M, Dirnagl U, Toelch U. Improving preclinical studies through replications .  Elife . 2021;10:e62101. doi:10.7554/eLife.62101

Michael RS. Threats to internal & external validity: Y520 strategies for educational inquiry .

Pahus L, Burgel PR, Roche N, Paillasseur JL, Chanez P. Randomized controlled trials of pharmacological treatments to prevent COPD exacerbations: applicability to real-life patients . BMC Pulm Med . 2019;19(1):127. doi:10.1186/s12890-019-0882-y

By Arlin Cuncic, MA Arlin Cuncic, MA, is the author of The Anxiety Workbook and founder of the website About Social Anxiety. She has a Master's degree in clinical psychology.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• The 4 Types of Validity in Research | Definitions & Examples

The 4 Types of Validity in Research | Definitions & Examples

Published on September 6, 2019 by Fiona Middleton . Revised on June 22, 2023.

Validity tells you how accurately a method measures something. If a method measures what it claims to measure, and the results closely correspond to real-world values, then it can be considered valid. There are four main types of validity:

• Construct validity : Does the test measure the concept that it’s intended to measure?
• Content validity : Is the test fully representative of what it aims to measure?
• Face validity : Does the content of the test appear to be suitable to its aims?
• Criterion validity : Do the results accurately measure the concrete outcome they are designed to measure?

In quantitative research , you have to consider the reliability and validity of your methods and measurements.

Note that this article deals with types of test validity, which determine the accuracy of the actual components of a measure. If you are doing experimental research, you also need to consider internal and external validity , which deal with the experimental design and the generalizability of results.

Table of contents

Construct validity, content validity, face validity, criterion validity, other interesting articles, frequently asked questions about types of validity.

Construct validity evaluates whether a measurement tool really represents the thing we are interested in measuring. It’s central to establishing the overall validity of a method.

What is a construct?

A construct refers to a concept or characteristic that can’t be directly observed, but can be measured by observing other indicators that are associated with it.

Constructs can be characteristics of individuals, such as intelligence, obesity, job satisfaction, or depression; they can also be broader concepts applied to organizations or social groups, such as gender equality, corporate social responsibility, or freedom of speech.

There is no objective, observable entity called “depression” that we can measure directly. But based on existing psychological research and theory, we can measure depression based on a collection of symptoms and indicators, such as low self-confidence and low energy levels.

What is construct validity?

Construct validity is about ensuring that the method of measurement matches the construct you want to measure. If you develop a questionnaire to diagnose depression, you need to know: does the questionnaire really measure the construct of depression? Or is it actually measuring the respondent’s mood, self-esteem, or some other construct?

To achieve construct validity, you have to ensure that your indicators and measurements are carefully developed based on relevant existing knowledge. The questionnaire must include only relevant questions that measure known indicators of depression.

The other types of validity described below can all be considered as forms of evidence for construct validity.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Content validity assesses whether a test is representative of all aspects of the construct.

To produce valid results, the content of a test, survey or measurement method must cover all relevant parts of the subject it aims to measure. If some aspects are missing from the measurement (or if irrelevant aspects are included), the validity is threatened and the research is likely suffering from omitted variable bias .

A mathematics teacher develops an end-of-semester algebra test for her class. The test should cover every form of algebra that was taught in the class. If some types of algebra are left out, then the results may not be an accurate indication of students’ understanding of the subject. Similarly, if she includes questions that are not related to algebra, the results are no longer a valid measure of algebra knowledge.

Face validity considers how suitable the content of a test seems to be on the surface. It’s similar to content validity, but face validity is a more informal and subjective assessment.

You create a survey to measure the regularity of people’s dietary habits. You review the survey items, which ask questions about every meal of the day and snacks eaten in between for every day of the week. On its surface, the survey seems like a good representation of what you want to test, so you consider it to have high face validity.

As face validity is a subjective measure, it’s often considered the weakest form of validity. However, it can be useful in the initial stages of developing a method.

Criterion validity evaluates how well a test can predict a concrete outcome, or how well the results of your test approximate the results of another test.

What is a criterion variable?

A criterion variable is an established and effective measurement that is widely considered valid, sometimes referred to as a “gold standard” measurement. Criterion variables can be very difficult to find.

What is criterion validity?

To evaluate criterion validity, you calculate the correlation between the results of your measurement and the results of the criterion measurement. If there is a high correlation, this gives a good indication that your test is measuring what it intends to measure.

A university professor creates a new test to measure applicants’ English writing ability. To assess how well the test really does measure students’ writing ability, she finds an existing test that is considered a valid measurement of English writing ability, and compares the results when the same group of students take both tests. If the outcomes are very similar, the new test has high criterion validity.

Prevent plagiarism. Run a free check.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Face validity and content validity are similar in that they both evaluate how suitable the content of a test is. The difference is that face validity is subjective, and assesses content at surface level.

When a test has strong face validity, anyone would agree that the test’s questions appear to measure what they are intended to measure.

For example, looking at a 4th grade math test consisting of problems in which students have to add and multiply, most people would agree that it has strong face validity (i.e., it looks like a math test).

On the other hand, content validity evaluates how well a test represents all the aspects of a topic. Assessing content validity is more systematic and relies on expert evaluation. of each question, analyzing whether each one covers the aspects that the test was designed to cover.

A 4th grade math test would have high content validity if it covered all the skills taught in that grade. Experts(in this case, math teachers), would have to evaluate the content validity by comparing the test to the learning objectives.

Criterion validity evaluates how well a test measures the outcome it was designed to measure. An outcome can be, for example, the onset of a disease.

Criterion validity consists of two subtypes depending on the time at which the two measures (the criterion and your test) are obtained:

• Concurrent validity is a validation strategy where the the scores of a test and the criterion are obtained at the same time .
• Predictive validity is a validation strategy where the criterion variables are measured after the scores of the test.

Convergent validity and discriminant validity are both subtypes of construct validity . Together, they help you evaluate whether a test measures the concept it was designed to measure.

• Convergent validity indicates whether a test that is designed to measure a particular construct correlates with other tests that assess the same or similar construct.
• Discriminant validity indicates whether two tests that should not be highly related to each other are indeed not related. This type of validity is also called divergent validity .

You need to assess both in order to demonstrate construct validity. Neither one alone is sufficient for establishing construct validity.

The purpose of theory-testing mode is to find evidence in order to disprove, refine, or support a theory. As such, generalizability is not the aim of theory-testing mode.

Due to this, the priority of researchers in theory-testing mode is to eliminate alternative causes for relationships between variables . In other words, they prioritize internal validity over external validity , including ecological validity .

It’s often best to ask a variety of people to review your measurements. You can ask experts, such as other researchers, or laypeople, such as potential participants, to judge the face validity of tests.

While experts have a deep understanding of research methods , the people you’re studying can provide you with valuable insights you may have missed otherwise.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Middleton, F. (2023, June 22). The 4 Types of Validity in Research | Definitions & Examples. Scribbr. Retrieved August 29, 2024, from https://www.scribbr.com/methodology/types-of-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, reliability vs. validity in research | difference, types and examples, construct validity | definition, types, & examples, external validity | definition, types, threats & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

External Validity

• Living reference work entry
• First Online: 10 February 2017
• Cite this living reference work entry

• Christoph J. Kemper 3  

475 Accesses

External validity refers to the degree to which conclusions from experimental scientific studies can be generalized from the specific set of conditions under which the study is conducted to other populations, settings, treatments, measurements, times, and experimenters.

Introduction

The ultimate goal of experimental scientific studies is to advance our understanding of real-life processes and phenomena. In research on individual differences it is rarely feasible to design experiments that involve thousands of participants and conditions that closely resemble the real world. Researchers usually seek to study an assumed cause-effect relationship without the interference of myriads of extraneous variables in real-life settings. To this purpose, they set up an experimental situation which allows to focus on the assumed cause-effect relationship and to control potentially confounding effects of extraneous variables. As a result, an artificial situation that differs from the real...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Educational Research Journal, 5 (4), 437–474.

Article   Google Scholar  

Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research . Chicago: Rand-McNally.

Google Scholar  

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Most people are not WEIRD. Nature, 466 (7302), 29–29.

Article   PubMed   Google Scholar  

Lynch, J. G. (1982). On the external validity of experiments in consumer research. Journal of Consumer Research, 9 (3), 225–239.

Download references

Author information

Authors and affiliations.

University of Luxembourg, Esch-sur-Alzette, Belval, Luxembourg

Christoph J. Kemper

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Christoph J. Kemper .

Editor information

Editors and affiliations.

Oakland University, Rochester, USA

Virgil Zeigler-Hill

Todd K. Shackelford

Section Editor information

Humboldt University, Germany, Berlin, Germany

Matthias Ziegler

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this entry

Cite this entry.

Kemper, C.J. (2017). External Validity. In: Zeigler-Hill, V., Shackelford, T. (eds) Encyclopedia of Personality and Individual Differences. Springer, Cham. https://doi.org/10.1007/978-3-319-28099-8_1303-1

Download citation

DOI : https://doi.org/10.1007/978-3-319-28099-8_1303-1

Received : 03 February 2017

Accepted : 03 February 2017

Published : 10 February 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-28099-8

Online ISBN : 978-3-319-28099-8

eBook Packages : Springer Reference Behavioral Science and Psychology Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

External Validity (Psychology): Definition and Examples

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

External validity refers to the extent to which the results of a study can be generalized or applied to settings, people, times, and measures other than the ones used in the study.

A study with high external validity will be generalizable beyond the specific conditions or participants of the original study. This means the study will likely be of great value in the real world because it means it is likely to be applicable to other people’s specific contexts, as well.

Definition of External Validity in Psychology

Generally, we think of external validity in psychology as a measure of how likely a cause-and-effect relationship will hold in settings external to the setting in which the study took place.

If you are writing a research paper, methodology chapter, or essay on external validity, be sure to use a scholarly definition.

Here are two that you could use:

• “External validity is the extent to which there is confidence that a study’s result may be generalized to hold over variations in populations, settings, treatments, and outcomes.” (Kviz, 2019)
• “External validity is defined as the extent to which results generalize to other participants, settings, follow-up times, and so on.” (Hagger-Johnson, 2014)

It is important to know how this differs from internal validity , outlined below.

Internal vs External Validity

If external validity refers to the extent to which we can be sure the cause-and-effect relationship between variables will hold up in external settings, internal validity refers to the extent to which we can sure it holds up within the study itself (Kenny, 2019; Kviz, 2019).

In other words, a poorly designed study may make claims to cause-and-effect that are inaccurate or misguided:

Internal validity is assessed as the extent to which plausible alternative explanations may be ruled out that a change in a dependent variable is not caused by the independent variable.” (Kviz, 2019)

These problems generally emerge through poor research design , such as failure to control variables, causing the third variable problem . This is a problem where a third variable, known as the confounding variable , is exerting an effect on the dependent and/or independent variable unknown to the researchers.

If a study has low internal validity then its results will be questioned, while a study with high internal validity is considered to have sound results.

Read More: Threats to Internal Validity

Types of External Validity

There are two types of external validity: population validity and ecological validity.

1. Population Validity

Population validity describes the degree to which the cohort examined in the study reflect the broader population (Kenny, 2019).

For example, if researchers want to test the viability of a new pill that will be rolled-out across the entire nation, then their research participants need to be a representative sample of the whole population (Findley, Kikuta & Denly, 2021). There would need to be people of all ages, genders, and races, ideally of equal proportion to the national population, included in the trial.

Meanwhile, if researchers are testing a new intervention that is exclusively intended for people who suffer from anxiety, then we would expect that the population in the research study would all have anxiety. We wouldn’t want to include people without anxiety in the study, because they would not reflect the target cohort in the real world. 

2. Ecological Validity

Ecological validity describes the degree to which the setting in which the study takes place reflects the real-world environment (Andrade, 2018).

 Oftentimes, lab-based studies suffer from low ecological validity because it is difficult to replicate real-world conditions in a lab setting.

Imagine, for example, a study where parents’ focus is observed in an observation room. In this room, there are no real-world distractions such as phones, television, or children to look after.

This study doesn’t reflect the real-world setting at all. The results would likely not truly reflect the actual degree of focus these parents have in a real-world environment. So, in this context, perhaps an observation study would make more sense.

There is often a trade-off between ecological validity and the ability to control variables. A controlled environment can improve the likelihood of identifying how well an independent variable can influence the dependent variable because it controls for extraneous variables ; however, when controlling for extraneous variables in a manufactured setting, we sometimes don’t see what actually would take place in the real world (Andrade, 2018). So, multiple studies with different designs may need to be conducted to get a holistic understanding of the topic.

Threats to External Validity

Kviz (2019) points to four key threats to external validity. In each case, the threat emerges because elements of the study would be unrepresentative in other contexts.

• Unrepresentative units: The units, by which Kviz means people or groups within the study, may not represent the broader population. For example, if the participants in a study of school performance were all boys between ages 15 and 17, then there would be little justification for a claim that these results would be valid in mainstream co-educational K-12 classrooms. The ages and genders of the participants don’t match the setting to which you would be wanting to generalize the claims.
• Unrepresentative setting: Similarly, the setting of a study may not match the generalizable population. To take the above example, if the school that the study took place in was an all-boys’ school, then this is unrepresentative of most mainstream state schools which are co-educational. As such, external validity would be diminished. Similarly, a study in a lab setting is not likely representative of the complexity of a real-world setting. 
• Unrepresentative treatment: A treatment is any manipulation of an independent variable in order to determine its effects on a dependent variable to identify a causal relationship. If the treatment in a test setting does not reflect practice settings, then external validity is diminished. To take the above school example, if our study involved a classroom intervention in which students got a lesson from a highly-trained specialist, then we can’t expect generalizability unless populations outside of the study also get an equivalent lesson from an equivalently trained specialist.   
• Unrepresentative outcome: Generalizability may not be likely if the outcome is not measured in the same way in the test and practice settings. For example, researchers may use a different measurement tool in the lab setting than practitioners do in a real-life setting; or, practitioners may not be able to measure outcomes at the same time and with the same rigor in real-life settings.

Read About More Threats to External Validity Here

Strategies for Increasing External Validity in Psychological Studies

Enhancing external validity is crucial for ensuring that research findings can be generalized beyond the specific settings, groups, or conditions of a study.

Here are five strategies to increase external validity:

• Random Sampling:  This involves selecting participants for a study from the larger population in a way that every individual has an equal chance of being chosen (Gall, Gall, & Borg, 2014). By ensuring that the study sample is representative of the larger population, the results can be more confidently generalized to the broader group.
• Using Real-world Settings (Field Experiments):  Instead of conducting experiments in controlled laboratory settings, researchers can carry out their studies in natural, real-world settings where participants behave more naturally (Darlington & Scott, 2015). The findings are more likely to be applicable in everyday situations when the study context mirrors real-world conditions.
• Replication:  This involves repeating the same study in different settings, with different populations, or under different conditions (Kenny, 2019). If similar results are obtained across various replications, it strengthens the belief that the findings are not limited to the specific conditions of a single study and can be generalized more broadly.
• Using Heterogeneous Participants:  Instead of focusing on a narrow or specific group of participants, researchers can include a diverse range of participants in terms of age, gender, ethnicity, socioeconomic status, etc., that might best reflect the real world (Kenny, 2019). A diverse participant pool ensures that findings are not just applicable to a specific subgroup but can be generalized to a broader population.
• Cross-cultural Studies:  Researchers conduct studies across different cultures or countries to see if the findings hold true in different cultural settings (Gall, Gall, & Borg, 2014). This ensures that the results are not bound by cultural nuances and can be generalized across different cultural contexts.

While there are strategies to enhance external validity in research (explained above), achieving perfect external validity is very challenging.

Often, psychology researchers have to balance between internal validity (ensuring that the study measures what it intends to measure without interference from other variables) and external validity.

The key is to always be aware of the potential limitations and address them as much as possible in the study design and interpretation of results.

Andrade, C. (2018). Internal, external, and ecological validity in research design, conduct, and evaluation.  Indian journal of psychological medicine ,  40 (5), 498-499.

Darlington, Y., & Scott, D. (2015).  Understanding qualitative research and ethnomethodology . London: Sage.

Findley, M. G., Kikuta, K., & Denly, M. (2021). External validity.  Annual Review of Political Science ,  24 , 365-393. ( Source )

Gall, M. D., Gall, J. P., & Borg, W. R. (2014).  Applying educational research: How to read, do, and use research to solve problems of practice . Sydney: Pearson.

Hagger-Johnson, G. (2014). Introduction to Research Methods and Data Analysis in the Health Sciences . Taylor & Francis.

Kenny, D. A. (2019). Enhancing validity in psychological research.  The American Psychologist ,  74 (9), 1018–1028.  https://doi.org/10.1037/amp0000531

Kviz, F. J. (2019). Conducting Health Research: Principles, Process, and Methods. SAGE Publications.

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 10 Reasons you’re Perpetually Single
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 20 Montessori Toddler Bedrooms (Design Inspiration)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 21 Montessori Homeschool Setups
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 101 Hidden Talents Examples

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Internal vs External Validity | Understanding Differences & Examples

Internal vs External Validity | Differences & Examples

Published on 5 May 2022 by Raimo Streefkerk . Revised on 10 October 2022.

When testing cause-and-effect relationships, validity can be split up into two types: internal validity and external validity .

Table of contents

Trade-off between internal and external validity, threats to internal validity, threats to external validity, frequently asked questions about internal and external validity.

Better internal validity often comes at the expense of external validity (and vice versa). The type of study you choose reflects the priorities of your research.

A solution to this trade-off is to conduct the research first in a controlled (artificial) environment to establish the existence of a causal relationship, followed by a field experiment to analyse whether the results hold in the real world.

Prevent plagiarism, run a free check.

There are eight factors that can threaten the   internal validity of your research. They are explained below using the following example:

They set up an experiment with two groups: 1) a control group   of employees with fixed working hours, and 2) an experiment group with employees with flexible working hours.

There are three main factors that might threaten the external validity of our study example.

There are various other   threats to external validity   that can apply to different kinds of experiments.

I nternal validity is the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables .

External validity is the extent to which your results can be generalised to other contexts.

The validity of your experiment depends on your experimental design .

There are eight threats to internal validity : history, maturation, instrumentation, testing, selection bias , regression to the mean, social interaction, and attrition .

The two types of external validity are population validity (whether you can generalise to other groups of people) and ecological validity (whether you can generalise to other situations and settings).

There are seven threats to external validity : selection bias , history, experimenter effect, Hawthorne effect , testing effect, aptitude-treatment, and situation effect.

Experimental designs are a set of procedures that you plan in order to examine the relationship between variables that interest you.

To design a successful experiment, first identify:

• A testable hypothesis
• One or more independent variables that you will manipulate
• One or more dependent variables that you will measure

When designing the experiment, first decide:

• How your variable(s) will be manipulated
• How you will control for any potential confounding or lurking variables
• How many subjects you will include
• How you will assign treatments to your subjects

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Streefkerk, R. (2022, October 10). Internal vs External Validity | Differences & Examples. Scribbr. Retrieved 30 August 2024, from https://www.scribbr.co.uk/research-methods/internal-vs-external-validity/

Is this article helpful?

Raimo Streefkerk

10 Things You Need to Know About External Validity

After months or years under development and implementation, navigating the practical, theoretical and inferential pitfalls of experimental social science research, your experiment has finally been completed. Comparing the treatment and control groups, you find a substantively and statistically significant result on an outcome of theoretical interest. Before you can pop the champagne in celebration of an intervention well evaluated, a friendly colleague asks: “But what does this tell us about the world?”

1 What is external validity?

External validity is another name for the generalizability of results, asking “whether a causal relationship holds over variation in persons, settings, treatments and outcomes.” ( Shadish, Cook, and Campbell 2002 ) . A classic example of an external validity concern is whether traditional economics or psychology lab experiments carried out on college students produce results that are generalizable to the broader public. In the political economy of development, we might consider how a community-driven development program in India might apply (or not) in West Africa, or Central America.

External validity becomes particularly important when making policy recommendations that come from research. Extrapolating causal effects from one or more studies to a given policy context requires careful consideration of both theory and empirical evidence. This methods guide discusses some key concepts, pitfalls to avoid, and useful references to consider when going from a Local Average Treatment Effect to the larger world.

2 How is this different than internal validity?

Internal validity refers to the quality of causal inferences being made for a given subject pool. As originally posited by Campbell ( Campbell 1957 ) , internal validity asks, “did in fact the experimental stimulus make some significant difference in this specific instance.” This concept dovetails with the counterfactual approach to causality that experimentalists typically use, which asks whether outcomes change depending on the presence or absence of a treatment. 1

Before you can extrapolate a causal effect to a distinct population, it is vital that the original Average Treatment Effect be based on a well-identified result. For most experimentalists, random assignment provides the requisite identifying variation, provided no attrition, interference, spillovers, or other threats to inference. For observational studies, additional identifying assumptions are needed, such as conditional independence of the treatment from potential outcomes.

3 Navigating the trade-offs between internal and external validity

There has been an ongoing debate within the social sciences regarding the relative importance of identifying internally valid results, which by definition apply to a local sample, and generating results that can be extrapolated to broader populations of interest. It is helpful to be familiar with this discussion when considering design trade-offs that inevitably crop up in resource-limited interventions. That both sides of the argument include luminaries of econometrics attests to the importance of the topic.

On one side of the argument fall advocates of “identification first,” who argue that without internally valid results, a study simply does not contribute useful information, regardless of whether it is a local or general population or context. As put by Imbens ( 2013 ) “without strong internal validity studies have little to contribute to policy debates, whereas [internally valid] studies with very limited external validity often are, and in my view should be, taken seriously in such discussions.”

Others argue that even without full identification of an internally valid result, useful information can be salvaged, especially if it is relevant for important questions that affect a broad context. Manski ( 2013 ) writes that “what matters is the informativeness of a study for policy making, which depends jointly on internal and external validity.” With data from a broad but a poorly identified study, Manski argues, bounds on the estimand of interest can be generated that, while not as useful as a precise point estimate, still moves science forward.

4 Theory and generalization

Extrapolating a result to a distinct context, outcome, population or treatment is not a mechanical process. As discussed by Samii ( 2016 ) and Rosenbaum ( 1999 ) relevant theory should be used to guide generalization, taking the relevant existing evidence and making predictions for other contexts in a principled fashion. Theories boil down complex problems into more parsimonious representations, and help to elucidate what factors matter. Just as theory guides the content of interventions and research designs, theoretical propositions can tell you which scope conditions are relevant for extrapolating a result. What covariates matter? What contextual information matters?

5 How can I determine where my results apply?

There are two primary means of generalizing results, one based on the covariates of units in the study and the other based on actual experimental manipulation of moderating variables. Observing how a treatment effect varies over a non-randomized pre-treatment variable can describe treatment effect heterogeneity, which can be highly suggestive about where or for whom the intervention is likely to be most effective, beyond the original sample. Note, however, that this type of analysis cannot pin down whether the treatment-effect heterogeneity is caused by that pre-treatment variable. The concern—endemic to observational research—is that the non-randomized covariate may be correlated with an unobserved variable, and it is this “unseen” factor that in fact is responsible for the heterogeneous impacts of the treatment ( Gerber and Green 2012 ) . Ideally, therefore, we want to leverage exogenous variation in the moderator of interest, thereby ruling out the possibility of such confounding. A factorial experimental design in which the researcher assigns the moderator independently of the main treatment of interest can generate especially compelling evidence about a moderator’s role. Though, of course, considerations of cost and statistical power may preclude this approach in practice.

Because generalization is primarily a prediction exercise, asking where we can expect a causal relationship similar to one observed locally, extrapolating heterogeneous effects based on similar covariates is often reasonable, provided theory does not indicate sources of confounding ( Bisbee et al. 2016 ) . Nonetheless, the strongest evidence for the generalizability of a result comes from a well-identified interaction between an exogenous moderator and the treatment, then projected across the covariate profile of a target population. Indeed, with some strong assumptions extrapolation can provide as good or better results than carrying out a second experiment in situ ( Bisbee et al. 2016 ) . The calculation of an extrapolated estimate can often be best performed using machine learning, although linear regression also performs reasonably well ( Kern et al. 2016 ) .

6 Strategic behavior can scuttle your extrapolations

Extrapolating a local result to a different context can prove challenging even with a compelling covariate profile to which you want to generalize effects. A randomized experimental manipulation in a local area generates a “partial equilibrium effect.” Strategic dynamics, including compensatory behavior or backlashes, outside the local context of an experimental intervention can complicate efforts to generalize a result. Suppose, for example, that an unconditional cash transfer intervention is shown to increase welfare, entrepreneurship, and employment in a sample of 200 villages. What would happen if the intervention were extended to encompass 1000 villages? At this point, one could imagine that regions excluded from the program are more likely to learn about it. Untreated units may start to demand other types of transfers from the government, giving rise to effects similar to those produced by the direct cash transfer. In a similar vein, sometimes causal relationships only work when they are applied to some people. For example, imagine a job skills program that functions very well (as compared to those who did not receive it), what would happen if it were extended to all workers? Even if there are positive effects across all participants, there could be reduced or no average effects as higher skilled jobs are already filled by the first batch and the second batch is forced to remain in their previous jobs, now overqualified. In short, under general equilibrium conditions we might expect different results even where the covariate profile matches.

7 Don’t confuse external validity with construct validity or ecological validity

Internal and external validity are not the only ‘validity’ concerns that can be leveled at experimental work, and though relevant, they are also distinct. Ecological validity, as defined by Shadish, Cook, and Campbell ( 2002 ) concerns whether an intervention appears artificial or out of place when deployed in a new context. For example, does an information workshop in a rural town carried out by experimenters resemble the kinds of information sharing that the population may experience in regular life? Similarly, if the same workshop were held in a large city, would it appear out of place?

Construct validity considers whether a theoretical concept being tested in a study is appropriately operationalized by the treatment(s). If your experiment is testing the effect of anger on political reciprocity and you are in fact manipulating fear or trust in your treatment, construct validity may be violated. Both construct and ecological validity are relevant for generalizations, and thus useful for making claims about external validity.

8 Extrapolation across treatments and outcomes

While much of this guide has implicitly focused on porting a given treatment to a new place or time, external validity also considers variations in treatments and outcomes. That is, imagine we did the same experiment on the same sample, but with a variation on the treatment, would we predict the local causal effect to be similar? Similarly, can we predict if a given treatment will produce the same or different causal effects on a different outcome? Sometimes we can address these concerns by conducting experiments that assess alternative treatments and outcomes. When follow-up experiments are in short supply, such issues have to be settled analytically. Rather than considering the features of subjects, extrapolation in this case requires thinking through, aided by theory, the characteristics of the treatments or outcomes and making reasonable predictions.

9 Replication is important

No single study represents the final word on a scholarly question. Following the logic of Bayesian updating, additional evidence in favor of or against a given theory allows the scientific and policy community to update their beliefs about the strength and validity of a causal relationship.

Replication of studies is an important part of this: scholars should replicate studies in contexts that look very different, but also in some contexts that look very similar. The former allows us to identify local causal relationships that can be triangulated with existing evidence and generalized as appropriate. At the same time, it is important to directly replicate existing studies under conditions that are as close as possible to the original in order to verify that local effects one may be interested in extrapolating are indeed reliable. The Collaboration ( 2015 ) found, for example, that when reproducing 100 major psychology experiments, just 47% of the original reported effect sizes fell within the 95% confidence interval of the effect size shown in the replication.

10 Don’t forget time

When thinking about causal relationships of interest, it is important also to consider time: do things we learn about the past extend to the future? How do an individual’s potential outcomes change over time? Immutable laws govern the physical and chemical worlds; hence what we learn about these laws today will always remain true. By contrast, we understand far less about the underlying drivers of social behavior and whether they hold constant in the same way. The answer may well be no. When making decisions about the policy relevance and generalizability of results, these considerations can help scholars determine a reasonable level of uncertainty and help policy makers adjust accordingly.

11 References

More details can be found in the causal inference methods guide . ↩︎

Internal vs. External Validity In Psychology

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

Internal validity centers on demonstrating clear casual relationships within the bounds of a specific study and external validity relates to demonstrating the applicability of findings beyond that original study situation or population.

Researchers have to weigh these considerations in designing methodologically rigorous and generalizable studies.

Internal Validity 

Internal validity refers to the degree of confidence that the causal relationship being tested exists and is trustworthy.

It tests how likely it is that your treatment caused the differences in results that you observe. Internal validity is largely determined by the study’s experimental design and methods . 

Studies that have a high degree of internal validity provide strong evidence of causality, so it makes it possible to eliminate alternative explanations for a finding.

Studies with low internal validity provide weak evidence of causality. The less chance there is for confounding or extraneous variables , the higher the internal validity and the more confident we can be in our findings. 

In order to assume cause and effect in a research study, the cause must precede the effect in terms of time, the cause and effect must vary together, and there must be no other explanations for the relationship observed. If these three criteria are observed, you can be confident that a study is internally valid. 

An example of a study with high internal validity would be if you wanted to run an experiment to see if using a particular weight-loss pill will help people lose weight.

To test this hypothesis, you would randomly assign a sample of participants to one of two groups: those who will take the weight-loss pill and those who will take a placebo pill.

You can ensure that there is no bias in how participants are assigned to the groups by blinding the research assistants , so they don’t know which participants are in which groups during the experiment. The participants are also blinded, so they do not know whether they are receiving the intervention or not.

If participants drop out of the study, their characteristics are examined to ensure there is no systematic bias regarding who left.

It is important to have a well-thought-out research procedure to mitigate the threats to internal validity.

External Validity

External validity refers to the extent to which the results of a research study can be applied or generalized to another context.

This is important because if external validity is established, the studies’ findings can be generalized to a larger population as opposed to only the relatively few subjects who participated in the study. Unlike internal validity, external validity doesn’t assess causality or rule out confounders.

There are two types of external validity: ecological validity and population validity.

• Ecological validity refers to whether a study’s findings can be generalized to other situations or settings. A high ecological validity means that there is a high degree of similarity between the experimental setting and another setting, and thus we can be confident that the results will generalize to that other setting.
• Population validity refers to how well the experimental sample represents other populations or groups. Using random sampling techniques , such as stratified sampling or cluster sampling, significantly helps increase population validity. 

An example of a study with high external validity would be if you hypothesize that practicing mindfulness two times per week will improve the mental health of those diagnosed with depression.

You recruit people who have been diagnosed with depression for at least a year and are between 18–29 years old. Choosing this representative sample with a clearly defined population of interest helps ensure external validity. 

You give participants a pre-test and a post-test measuring how often they experienced symptoms of depression in the past week.

During the study, all participants were given individual mindfulness training and asked to practice mindfulness daily for 15 minutes as part of their morning routine. 

You can also replicate the study’s results using different methods of mindfulness or different samples of participants. 

Trade-off Between Internal and External Validity

There tends to be a negative correlation between internal and external validity in experimental research. This means that experiments that have high internal validity will likely have low external validity and vice versa. 

This happens because experimental conditions that produce higher degrees of internal validity (e.g., artificial labs) tend to be highly unlikely to match real-world conditions. So, the external validity is weaker because a lab environment is much different than the real world. 

On the other hand, to produce higher degrees of external validity, you want experimental conditions that match a real-world setting (e.g., observational studies ).

However, this comes at the expense of internal validity because these types of studies increase the likelihood of confounding variables and alternative explanations for differences in outcomes. 

A solution to this trade-off is replication! You want to conduct the research in multiple environments and settings – first in a controlled, artificial environment to establish the existence of a causal relationship and then in a “real-world” setting to analyze if the results are generalizable. 

Threats to Internal Validity

Attrition refers to the loss of study participants over time. Participants might drop out or leave the study which means that the results are based solely on a biased sample of only the people who did not choose to leave.

Differential rates of attrition between treatment and control groups can skew results by affecting the relationship between your independent and dependent variables and thus affect the internal validity of a study. 

Confounders

A confounding variable is an unmeasured third variable that influences, or “confounds,” the relationship between an independent and a dependent variable by suggesting the presence of a spurious correlation.

Confounders are threats to internal validity because you can’t tell whether the predicted independent variable causes the outcome or if the confounding variable causes it.

Participant Selection Bias

This is a bias that may result from the selection or assignment of study groups in such a way that proper randomization is not achieved.

If participants are not randomly assigned to groups, the sample obtained might not be representative of the population intended to be studied. For example, some members of a population might be less likely to be included than others due to motivation, willingness to take part in the study, or demographics. 

Experimenter Bias

Experimenter bias occurs when an experimenter behaves in a different way with different groups in a study, impacting the results and threatening internal validity. This can be eliminated through blinding. 

Social Interaction (Diffusion)

Diffusion refers to when the treatment in research spreads within or between treatment and control groups. This can happen when there is interaction or observation among the groups.

Diffusion poses a threat to internal validity because it can lead to resentful demoralization. This is when the control group is less motivated because they feel resentful over the group that they are in. 

Historical Events

Historical events might influence the outcome of studies that occur over longer periods of time. For example, changes in political leadership, natural disasters, or other unanticipated events might change the conditions of the study and influence the outcomes.

Instrumentation

Instrumentation refers to any change in the dependent variable in a study that arises from changes in the measuring instrument used. This happens when different measures are used in the pre-test and post-test phases. 

Maturation refers to the impact of time on a study. If the outcomes of the study vary as a natural result of time, it might not be possible to determine whether the effects seen in the study were due to the study treatment or simply due to the impact of time. 

Statistical Regression

Regression to the mean refers to the fact that if one sample of a random variable is extreme, the next sampling of the same random variable is likely going to be closer to its mean.

This is a threat to internal validity as participants at extreme ends of treatment can naturally fall in a certain direction due to the passage of time rather than being a direct effect of an intervention. 

Repeated Testing

Testing your research participants repeatedly with the same measures will influence your research findings because participants will become more accustomed to the testing. Due to familiarity, or awareness of the study’s purpose, many participants might achieve better results over time.

Threats to External Validity 

Sample features.

If some feature(s) of the sample used were responsible for the effect, this could lead to limited generalizability of the findings.

This is a bias that may result from the selection or assignment of study groups in such a way that proper randomization is not achieved. If participants are not randomly assigned to groups, the sample obtained might not be representative of the population intended to be studied.

For example, some members of a population might be less likely to be included than others due to motivation, willingness to take part in the study, or demographics. 

Situational Factors

Factors such as the setting, time of day, location, researchers’ characteristics, noise, or the number of measures might affect the generalizability of the findings.

Aptitude-Treatment Interaction → Aptitude-Treatment Interaction to the concept that some treatments are more or less effective for particular individuals depending upon their specific abilities or characteristics. 

Hawthorne Effect

The Hawthorne Effect refers to the tendency for participants to change their behaviors simply because they know they are being studied.

Experimenter Effect

Experimenter bias occurs when an experimenter behaves in a different way with different groups in a study, impacting the results and threatening the external validity.

John Henry Effect

The John Henry Effect refers to the tendency for participants in a control group to actively work harder because they know they are in an experiment and want to overcome the “disadvantage” of being in the control group.

Factors that Improve Internal Validity

Blinding refers to a practice where the participants (and sometimes the researchers) are unaware of what intervention they are receiving.

This reduces the influence of extraneous factors and minimizes bias, as any differences in outcome can thus be linked to the intervention and not to the participant’s knowledge of whether they were receiving a new treatment or not. 

Random Sampling

Using random sampling to obtain a sample that represents the population that you wish to study will improve internal validity. 

Random Assignment

Using random assignment to assign participants to control and treatment groups ensures that there is no systematic bias among the research groups. 

Strict Study Protocol

Highly controlled experiments tend to improve internal validity. Experiments that occur in lab settings tend to have higher validity as this reduces variability from sources other than the treatment. 

Experimental Manipulation

Manipulating an independent variable in a study as opposed to just observing an association without conducting an intervention improves internal validity. 

Factors that Improve External Validity

Replication.

Conducting a study more than once with a different sample or in a different setting to see if the results will replicate can help improve external validity.

If multiple studies have been conducted on the same topic, a meta-analysis can be used to determine if the effect of an independent variable can be replicated, thus making it more reliable.

Replication is the strongest method to counter threats to external validity by enhancing generalizability to other settings, populations, and conditions.

Field Experiments

Conducting a study outside the laboratory, in a natural, real-world setting will improve external validity (however, this will threaten the internal validity) 

Probability Sampling

Using probability sampling will counter selection bias by making sure everyone in a population has an equal chance of being selected for a study sample.

Recalibration

Recalibration is the use of statistical methods to maintain accuracy, standardization, and repeatability in measurements to assure reliable results.

Reweighting groups, if a study had uneven groups for a particular characteristic (such as age), is an example of calibration. 

Inclusion and Exclusion Criteria

Setting criteria as to who can be involved in the research and who cannot be involved will ensure that the population being studied is clearly defined and that the sample is representative of the population.

Psychological Realism

Psychological realism refers to the process of making sure participants perceive the experimental manipulations as real events so as to not reveal the purpose of the study and so participants don’t behave differently than they would in real life based on knowing the study’s goal.

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research Research Tools and Apps

External Validity: Types, Research Methods & Examples

External validity is one of the main goals of researchers who want to find reliable cause-and-effect relationships in qualitative research.

When research has this validity, the results can be used with other people in different situations or places. Because without this validity, analysis can’t be generalized, and researchers can’t apply the results of studies to the real world. So, psychology research needs to be conducted outside a lab setting.

Still, sometimes they prefer to research how variables cause each other instead of being able to generalize the results.

In this article, we’ll talk about what external validity means, its types, and its research design methods.

LEARN ABOUT: Theoretical Research

What is external validity?

External validity describes how effectively the findings of an experiment may be generalized to different people, places, or times. Most scientific investigations do not intend to obtain outcomes that only apply to the few persons who participated in the study.

Instead, researchers want to be able to take the results of an experiment and use them with a larger group of people. It is a big part of what inferential statistics try to do.

For example, if you’re looking at a new drug or educational program, you don’t want to know that it works for only a few people. You want to use those results outside the experiment and beyond those participating. It is called “generalizability,” the essential part of this validity.

Types of external validity

Generally, there are three main types of this validity. We’ll discuss each one below and give examples to help you understand.

Population validity

Population validity is a kind of external validity that looks at how well the study’s results applied to a larger group of people. In this case, “population” refers to the group of people about whom a researcher is trying to conclude. On the other hand, a sample is a particular group of people who participate in the research.

If the results from the sample can apply to a larger group of people, then the study is valid for a large population.

Example: low population validity

You want to test the theory about how exercise and sleep are linked. You think that adults will sleep better when they do physical activities regularly. Your target group is adults in the United States, but your sample comprises about 300 college students. 

Even though they are all adults, it might be hard to ensure the population validity in this case because the sampling model of students only represents some adults in the US.

So, your study has a limited amount of population validity, and you can only apply the results to some of the population.

Ecological validity

Ecological validity is another type of external validity that shows how well the research results can be used in different situations. In simple terms, ecological validity is about whether or not your results can be used in the real world.

So, if a study has a lot of ecological validity, the results can be used in the real world. On the other hand, low validity means that the results can’t be used outside the experiment.

Example: low ecological validity

The Milgram Experiment is a classic example of low ecological validity.

Stanley Milgram studied authority in the 1960s. He randomly chose participants and directed them to employ higher and higher voltage shocks to penalize wrong-answering actors. The study showed great obedience to authorities despite fake shock and victim behaviors.

The results of this study are revolutionary for the field of social psychology. However, it is often criticized because it has little ecological validity. Milgram’s set-up was not like real-life situations.

In the experiment, he set up a situation where the participants couldn’t avoid obeying the rules. But the reality of the issue can be very different.

Temporal validity

When figuring out external validity, time is just as important as the number of people involved and confusing factors.

The concept of temporal validity refers to how findings evolve. Particularly, this form of validity refers to how well the research results can be extended to another period.

High temporal validity means that research results can be used correctly in different times and places and that factors will be important in the future.

Imagine that you’re a psychologist, and you’re studying how people act the same.

You found out that social pressure from the majority group has a big effect on the choices of the minority. Because of this, people act similarly. Even though famous psychologist Solomon Asch did this research in the 1950s, the results can still be used in the real world today. 

This study, therefore, has temporal validity even after nearly a century.

Research methods of external validity

There are a lot of methods you can do to improve the external validity of your research. Some things that can improve are given below:

Field experiments

Field experiments are like conducting research outside rather than in a controlled environment like a laboratory.

Criteria for inclusion and exclusion

Establishing criteria for who can participate in the research and ensuring that the group being examined is properly identified

Realism in psychology

If you want the participants to believe that the events that take place throughout the study are true, you should provide them with a cover story regarding the purpose of the research. So that they don’t behave any differently than they would in real life based on the fact.

Replication

Doing the study again with different samples or in a different place to see if you get the same results. When many studies have been done on the same topic, a meta-analysis can be used to see if the effect of an independent variable can be repeated to make it more reliable.

Reprocessing

It is like using statistical methods to fix problems with external validity, like reweighting groups if they were different in a certain way, such as age.

LEARN ABOUT: 12 Best Tools for Researchers

As stated in the article, the ability to replicate the results of an experiment is a key component of its external validity. Using the sampling methods the external validity can be improved in the research.

Researchers compare the results to other relevant data to determine the external validity. They can also do the research with more people from the target population. It’s hard to figure out external validity in research, but it’s important to use the results in the future.

The QuestionPro research suite is an enterprise-level research tool that can help you with your research process and surveys.

We at QuestionPro provide tools for data collection, such as our survey software, and a library of insights for any lengthy study. If you’re interested in seeing a demo or learning more, visit the Insight Hub.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Why You Should Attend XDAY 2024

Aug 30, 2024

Alchemer vs Qualtrics: Find out which one you should choose

Target Population: What It Is + Strategies for Targeting

Aug 29, 2024

Microsoft Customer Voice vs QuestionPro: Choosing the Best

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

Neag School of Education

Educational Research Basics by Del Siegle

External validity.

Note to EPSY 5601 Students: An understanding of the difference between population and ecological validity is sufficient. Mastery of the sub categories for each is not necessary for this course.

External Validity (Generalizability) –to whom can the results of the study be applied–

There are two types of study validity: internal (more applicable with experimental research) and external. This section covers external validity.

External validity involves the extent to which the results of a study can be generalized (applied) beyond the sample. In other words, can you apply what you found in your study to other people (population validity) or settings (ecological validity).  A study of fifth graders in a rural school that found one method of teaching spelling was superior to another may not be applicable with third graders (population) in an urban school (ecological).

Threats to External Validity

Population Validity the extent to which the results of a study can be generalized from the specific sample that was studied to a larger group of subjects

• the extent to which one can generalize from the study sample to a defined population– If  the sample is drawn from an accessible population, rather than the target population, generalizing the research results from the accessible population to the target population is risky. 2. the extent to which personological variables interact with treatment effects– If the study is an experiment, it may be possible that different results might be found with students at different grades (a personological variable).

Ecological Validity the extent to which the results of an experiment can be generalized from the set of environmental conditions created by the researcher to other environmental conditions (settings and conditions).

• Explicit description of the experimental treatment (not sufficiently described for others to replicate) If the researcher fails to adequately describe how he or she conducted a study, it is difficult to determine whether the results are applicable to other settings.
• Multiple-treatment interference (catalyst effect) If a researcher were to apply several treatments, it is difficult to determine how well each of the treatments would work individually. It might be that only the combination of the treatments is effective.
• Hawthorne effect (attention causes differences) Subjects perform differently because they know they are being studied. “…External validity of the experiment is jeopardized because the findings might not generalize to a situation in which researchers or others who were involved in the research are not present” (Gall, Borg, & Gall, 1996, p. 475)
• Novelty and disruption effect (anything different makes a difference) A treatment may work because it is novel and the subjects respond to the uniqueness, rather than the actual treatment. The opposite may also occur, the treatment may not work because it is unique, but given time for the subjects to adjust to it, it might have worked.
• Experimenter effect (it only works with this experimenter) The treatment might have worked because of the person implementing it. Given a different person, the treatment might not work at all.
• Pretest sensitization (pretest sets the stage) A treatment might only work if a pretest is given. Because they have taken a pretest, the subjects may be more sensitive to the treatment. Had they not taken a pretest, the treatment would not have worked.
• Posttest sensitization (posttest helps treatment “fall into place”) The posttest can become a learning experience. “For example, the posttest might cause certain ideas presented during the treatment to ‘fall into place’ ” (p. 477). If the subjects had not taken a posttest, the treatment would not have worked.
• Interaction of history and treatment effec t (…to everything there is a time…) Not only should researchers be cautious about generalizing to other population, caution should be taken to generalize to a different time period. As time passes, the conditions under which treatments work change.
• Measurement of the dependent variable (maybe only works with M/C tests) A treatment may only be evident with certain types of measurements. A teaching method may produce superior results when its effectiveness is tested with an essay test, but show no differences when the effectiveness is measured with a multiple choice test.
• Interaction of time of measurement and treatment effect (it takes a while for the treatment to kick in) It may be that the treatment effect does not occur until several weeks after the end of the treatment. In this situation, a posttest at the end of the treatment would show no impact, but a posttest a month later might show an impact.

Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Education Research Journal, 5, 437-474. Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. White Plains, NY: Longman.

Del Siegle, Ph.D. Neag School of Education – University of Connecticut [email protected] www.delsiegle.com

• Foundations
• Write Paper

Search form

• Experiments
• Anthropology
• Self-Esteem
• Social Anxiety

External Validity

External validity is one the most difficult of the validity types to achieve, and is at the foundation of every good experimental design.

This article is a part of the guide:

• Validity and Reliability
• Types of Validity
• Definition of Reliability
• Content Validity
• Construct Validity

Browse Full Outline

• 1 Validity and Reliability
• 2 Types of Validity
• 3.1 Population Validity
• 3.2 Ecological Validity
• 4 Internal Validity
• 5.1.1 Concurrent Validity
• 5.1.2 Predictive Validity
• 6 Content Validity
• 7.1 Convergent and Discriminant Validity
• 8 Face Validity
• 9 Definition of Reliability
• 10.1 Reproducibility
• 10.2 Replication Study
• 11 Interrater Reliability
• 12 Internal Consistency Reliability
• 13 Instrument Reliability

Many scientific disciplines, especially the social sciences, face a long battle to prove that their findings represent the wider population in real world situations.

The main criteria of external validity is the process of generalization , and whether results obtained from a small sample group, often in laboratory surroundings, can be extended to make predictions about the entire population.

The reality is that if a research program has poor external validity, the results will not be taken seriously, so any  research design  must justify  sampling  and selection methods.

What is External Validity?

In 1966, Campbell and Stanley proposed the commonly accepted definition of external validity.

“External validity asks the question of generalizability: To what populations, settings, treatment variables and measurement variables can this effect be generalized?”

External validity is usually split into two distinct types, population validity and ecological validity , and they are both essential elements in judging the strength of an experimental design.

Psychology and External Validity

The battle lines are drawn.

External validity often causes a little friction between clinical psychologists and research psychologists.

Clinical psychologists often believe that research psychologists spend all of their time in laboratories, testing mice and humans in conditions that bear little resemblance to the outside world. They claim that the data produced has no external validity, and does not take into account the sheer complexity and individuality of the human mind.

Before we are flamed by irate research psychologists, the truth lies somewhere between the two extremes! Research psychologists find out trends and generate sweeping generalizations that predict the behavior of groups. Clinical psychologists end up picking up the pieces, and study the individuals who lie outside the predictions, hence the animosity.

In most cases, research psychology has a very high population validity , because researchers take meticulously randomly select groups and use large sample sizes , allowing meaningful statistical analysis.

However, the artificial nature of research psychology means that ecological validity is usually low.

Clinical psychologists, on the other hand, often use focused case studies , which cause minimum disruption to the subject and have strong ecological validity. However, the small sample sizes mean that the population validity is often low.

Ideally, using both approaches provides useful generalizations , over time!

Randomization in External Validity and Internal Validity

It is also important to distinguish between external and internal validity , especially with the process of randomization, which is easily misinterpreted. Random selection is an important tenet of external validity.

For example, a research design , which involves sending out survey questionnaires to students picked at random, displays more external validity than one where the questionnaires are given to friends. This is randomization to improve external validity.

Once you have a representative sample, high internal validity involves randomly assigning subjects to groups, rather than using pre-determined selection factors.

With the student example, randomly assigning the students into test groups, rather than picking pre-determined groups based upon degree type, gender, or age strengthens the internal validity.

Campbell, D.T., Stanley, J.C. (1966). Experimental and Quasi-Experimental Designs for Research. Skokie, Il: Rand McNally.

• Psychology 101
• Flags and Countries
• Capitals and Countries

Martyn Shuttleworth (Aug 7, 2009). External Validity. Retrieved Aug 30, 2024 from Explorable.com: https://explorable.com/external-validity

You Are Allowed To Copy The Text

The text in this article is licensed under the Creative Commons-License Attribution 4.0 International (CC BY 4.0) .

This means you're free to copy, share and adapt any parts (or all) of the text in the article, as long as you give appropriate credit and provide a link/reference to this page.

That is it. You don't need our permission to copy the article; just include a link/reference back to this page. You can use it freely (with some kind of link), and we're also okay with people reprinting in publications like books, blogs, newsletters, course-material, papers, wikipedia and presentations (with clear attribution).

Want to stay up to date? Follow us!

Save this course for later.

Don't have time for it all now? No problem, save it as a course and come back to it later.

Footer bottom

• Privacy Policy

• Subscribe to our RSS Feed
• Like us on Facebook
• Follow us on Twitter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Bras Pneumol
• v.44(3); May-Jun 2018

Internal and external validity: can you apply research study results to your patients?

Cecilia maria patino.

1 . Methods in Epidemiologic, Clinical, and Operations Research-MECOR-program, American Thoracic Society/Asociación Latinoamericana del Tórax, Montevideo, Uruguay.

2 . Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.

Juliana Carvalho Ferreira

3 . Divisão de Pneumologia, Instituto do Coração, Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo (SP) Brasil.

CLINICAL SCENARIO

In a multicenter study in France, investigators conducted a randomized controlled trial to test the effect of prone vs. supine positioning ventilation on mortality among patients with early, severe ARDS. They showed that prolonged prone-positioning ventilation decreased 28-day mortality [hazard ratio (HR) = 0.39; 95% CI: 0.25-0.63]. 1

STUDY VALIDITY

The validity of a research study refers to how well the results among the study participants represent true findings among similar individuals outside the study. This concept of validity applies to all types of clinical studies, including those about prevalence, associations, interventions, and diagnosis. The validity of a research study includes two domains: internal and external validity.

Internal validity is defined as the extent to which the observed results represent the truth in the population we are studying and, thus, are not due to methodological errors. In our example, if the authors can support that the study has internal validity, they can conclude that prone positioning reduces mortality among patients with severe ARDS. The internal validity of a study can be threatened by many factors, including errors in measurement or in the selection of participants in the study, and researchers should think about and avoid these errors.

Once the internal validity of the study is established, the researcher can proceed to make a judgment regarding its external validity by asking whether the study results apply to similar patients in a different setting or not ( Figure 1 ). In the example, we would want to evaluate if the results of the clinical trial apply to ARDS patients in other ICUs. If the patients have early, severe ARDS, probably yes, but the study results may not apply to patients with mild ARDS . External validity refers to the extent to which the results of a study are generalizable to patients in our daily practice, especially for the population that the sample is thought to represent.

Lack of internal validity implies that the results of the study deviate from the truth, and, therefore, we cannot draw any conclusions; hence, if the results of a trial are not internally valid, external validity is irrelevant. 2 Lack of external validity implies that the results of the trial may not apply to patients who differ from the study population and, consequently, could lead to low adoption of the treatment tested in the trial by other clinicians.

INCREASING VALIDITY OF RESEARCH STUDIES

To increase internal validity, investigators should ensure careful study planning and adequate quality control and implementation strategies-including adequate recruitment strategies, data collection, data analysis, and sample size. External validity can be increased by using broad inclusion criteria that result in a study population that more closely resembles real-life patients, and, in the case of clinical trials, by choosing interventions that are feasible to apply. 2

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Volume 23, Issue 1
• External validity, generalisability, applicability and directness: a brief primer
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

https://doi.org/10.1136/ebmed-2017-110800

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

You do not have access to the full text of this article, the first page of the PDF of this article appears above.

Contributors MHM and VMM conceived the idea and drafted the manuscript. All authors critically revised the manuscript and approved it.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

• Basics of Research Process
• Methodology
• External Validity: Definition, Types, Threats & Examples
• Speech Topics
• Basics of Essay Writing
• Essay Topics
• Other Essays
• Main Academic Essays
• Research Paper Topics
• Basics of Research Paper Writing
• Miscellaneous
• Chicago/ Turabian
• Data & Statistics
• Admission Writing Tips
• Admission Advice
• Other Guides
• Student Life
• Studying Tips
• Understanding Plagiarism
• Academic Writing Tips
• Basics of Dissertation & Thesis Writing

• Essay Guides
• Research Paper Guides
• Formatting Guides
• Admission Guides
• Dissertation & Thesis Guides

External Validity: Definition, Types, Threats & Examples

Table of contents

Use our free Readability checker

You probably have already heard about external validity in research. As you know, the critical goal for each research is to make it useful for people or institutions. Or in other words, we are talking about the ability to extend your study results to a broader audience. This is external validity – a research term we are going to discuss in this blog. 

External validity is the extent to which results from a study can be generalized to other settings. It shows whether or not the findings of a single experiment are applicable beyond the immediate research context. Achieving high levels of external validity requires researchers to replicate their studies in a range of different environments to assure that the results are consistent. 

How to make your academic work important? How to ensure that your results can be generalized for bigger targets? Here, we are going to discuss:

• Types of external validity
• How to establish it
• External validity threats
• How to minimize potential risks .

Our researh paper writing service will also share valuable external validity examples to support your study and future research ideas. We hope this guide will benefit your work and scientific career.

What Is External Validity: Definition

First and foremost, what is external validity in research? When you understand its fundamental principles, it will be easier to get externally valid data for study results in the future. It helps you define whether you can apply your research results to real-world examples. 

External validity is the extent to which you can expand your results for other target audiences, circumstances, or settings. Unlike internal validity , it deals with generalizability of study outcomes. In other words, the external validity of a study is the ability to use your insights from quantitative or qualitative research to make a conclusion for the broader public. 

To achieve high external validity, researchers must meet 2 criteria:

• Sample should be representative of the population.
• Methods and measures are appropriate for the research question.

Note that whenever your external validity increases, there is a potential risk of decreasing internal validity. Make sure to find a balance between internal vs external validity in your study design.

Why Is External Validity Important?

Another essential question is why you even need to care about it. You have research questions , and you have your answers. But this is not a goal for almost all academics. Your aim is to change this situation and help essential institutions to be more productive. 

You need to ensure external validity for valuable research to achieve such goals:

• Ensure that findings can be applied to similar populations or circumstances
• Make your study more valuable
• Extend future research
• Make a connection between scientific findings and real-world issues
• Complete a stronger study.

External Validity Example

To understand the transferability of your research results better, and go deeper into different types of validity , let’s start with a basic example. 

Here you can see a common example of external validity, and next, we will focus on its types and specifics.

For instance, let's say you're conducting research on how using mobile phones, iPads, or computers 2 hours before bedtime can affect the quality of sleep. To investigate this, you have recruited 100 participants to spend a night in a sleep lab while using their electronic devices before bed. You're measuring their brain activity during sleep to determine any effects from using blue screens before bedtime. After analyzing the data, you discover that using blue screens 2 hours before going to bed makes it harder to fall asleep. The generalizability of results allows you to apply findings to all people, not only to your experimental group .

Types of External Validity 

There are 2 main types of external validity that you may face while working on research:

• Population validity
• Ecological validity.

In the following paragraphs, we will briefly discuss each type and share examples of external validity.

Population Validity

Population validity means that you can generalize your findings from your smaller samplings to a larger group of people with the same characteristics. Your results may be limited to the population you are researching in case of applying a non-probability sampling method. However, in some situations, you can easily extend your results to bigger groups of people.

Example of Low Population Validity

Let's take a look at an example of research that examines the usage of recreation centers and gyms among students in a US school. For instance, the results of a survey research of 1000 students from this school show that regular gym usage is positively correlated with higher grades in other disciplines. However, it does not mean that these results can be applied to all students in high schools worldwide, as the representation of sampling was too narrow. This study has low external validity. But it’s still valuable for improving grade scores in this particular school, even if not all students were surveyed.

Why do we define this type of validity as low, and how to ensure strong external validity for your study? A problem with these examples is that they did not consider other institutions. In other universities, the number of male and female students can be different, and some cultural differences may also exist. This is why we call this type of external validity low. Researchers should consider conducting this study in multiple settings to ensure that the results are consistent across different contexts. Additionally, using multiple data collection techniques and analysis methods can help increase the generalizability of study results.

Ecological Validity

Another type of external validity is ecological validity – the extent to which your result can be generalized for other circumstances or environmental conditions. In other words, can we make a conclusion and apply insights to the same sampling but in other situations? This type of external validity is very similar to population validity, but instead of examining different groups of people, it examines different settings.

In some cases, your study results can not be applied to all settings, as the experiment was conducted in quite specific circumstances (e.g.,using a computer instead of a simulation). We would call it low ecological validity.

Below, you will find an exact example of ecological validity. 

Example of Low Ecological Validity

To understand external validity, we need to look precisely into the test environment. Let’s use the study of establishing driving habits as an example. You have a group of people and a computer with a driving simulation. For this experiment, you tested how different circumstances affect the drivers’ accuracy. For instance, you looked at how music would affect their focus and how noise will influence a driving technique. You also may try to change the road from the village to see how aggressive a driver can be because of city noise. You may have pretty interesting results!

Why would we call this experiment a low ecological validity? First of all, scientists used a simulation on a computer instead of real driving. You may think that there is no difference, but you are wrong. When a person is really driving, they may have more focus just because they assess the danger of the road and the possible consequences of driving accidents. In simulation, people know that they are sitting in front of a computer, and they will still be alive in any circumstances. That is why we can’t use these test results for a real-world situation with drivers, as changing the circumstances can change insights.

How to Establish External Validity

It can be challenging for some research to ensure its external validity. Scientists need to understand all details of assessing external validity and examine all options for improving it. The only way to ensure that your results can be used for solving real-life problems is to repeat your study for other target audiences or use other settings. You may increase the validity by expanding the criteria for your samplings. However, before you try to enhance validity, you need to figure it out and define if you have any problems.

How to Increase External Validity?

As we have already mentioned, increasing external validity can be challenging and require a lot of additional research work. However, if the purpose of the study is to expand the insight for a broad audience, you need to focus on how to improve the external validity of the research. 

Here’s how to improve external validity and make it more applicable for real-live:

• Use random selection for your samplings if it’s possible.
• Bring more focus on researching the group for your samples, and outline similarities of different groups that are a part of your work.
• Use conceptual framework to understand the degree of similarities between target groups.
• Conduct your study with different groups of people in various circumstances and time slots.

Threats to External Validity

Finally, we came to the most important part of your research – identifying threats to external validity 

External validity can be compromised by various extraneous variables that may influence the causal relationship. These variables may include external factors such as the research setting, participant characteristics, and the method of data collection.

Your opponents can use external validity threats to make your work weak. That’s why you need to know all possible threats to be able to avoid them.

1. Sampling Bias

One of the common threats to external validity is when research samples are not representative and do not illustrate the whole population you want to generalize the study. In other words, your sampling group includes only one type of people but not the average data about the population you are looking for. 

You are doing research on how the citizens support one of the candidates for mayor. You have a survey, but your sampling is mostly women in age 60+. However, this is a city where 40% of the population is students, and 55% are male. Your survey data won’t represent the real situation of how your candidate will be supported on election day. 

2. History Threat

Other external threats to validity are some events that occur during your tests or gathering samples that affect study results but are not directly related to your study. It can be natural disasters or political rallies that make your target audience answer in a different way. As a result, your data can be compromised. 

You are researching a stress level among people who live on Hawaii islands. You get your surveys that represent the target audience. But during this research, a tsunami destroyed one of the small islands not far from Hawaii. This event definitely affected how people answered questions about stress.

3. Observer Bias

One common threat to external validity is how the behavior or other characteristics of the person who is conducting research influence your results. For instance, people often react to an instructor instead of research questions during focus groups.

You have a focus group for researching the best marketing advertisement campaign. Initially, the instructor tells people that one of the firms will happily give everyone presents if they choose their work. And after that, they point to this work. Of course, people will be focused on how to get presents instead of being honest. 

4. Pre- and Post-Test Effects

Pre and post-test communication also threatens the external validity of a study. In other words, how your research team or instructor organizes pre and post-test directly influences how people behave during the survey. For instance, how stressed they will be about tests or experiments. 

A great example is the study of how effective people can be in taking standardized tests. In most cases, when people are aware of how the test will be going, for instance, they have a detailed overview of how the test will be going. It means their stress level will be different than in case the whole test is a surprise for them.

5. Hawthorne Effect

One more threat that affects external validity is the Hawthorne effect. It stands on the point that people can change their behavior if they know that they are part of an experiment or study. People usually try to behave better than usual in case they know that researchers observe this behavior.  

For example, you are studying how people form good habits, like going jogging every morning. In case they know that they are part of an experiment, they will more actively try to form this habit and not skip it. But it does not mean that their habit will be formed the same in case people will know that nobody is watching. 

6. Novelty and Disruption Effect

In some cases, researchers can’t apply external validation of their study. For example, they are not sure if the results depend on the novelty of the treatment. Or if any study disruptions affect the insights of the study. This is often a threat in medicine and ecology studies. 

In medicine, scientists may test a new treatment on patients but not be sure if positive results are the cause of this new treatment or if the organism just reacted positively to something new. It can be a huge problem for hospitals. 

7. Aptitude Treatment

This is one more threat to external validity! It is also possible that group characteristics, together with some individual variables, influence your dependent variable . That is why the insights from the study can be criticized and cannot be expanded to a broader audience or different circumstances. 

The study focuses on how positive thinking and meditation can help people to organize their everyday life. However, you may find that some participants of this study have a problem with panic attacks, but also meditation helped them with organizational issues. But does it mean that it will help people with depression, for example?

8. Situational Factors

There are a lot of factors, like day time, weather, specific season, or other circumstances, that can undermine your study outcomes. In this case, the scientists should ensure that all independent variables are analyzed and nothing can influence the final results. However, this thread is often used to manipulate data. 

For example, you are looking at what type of bank advertisement is perceived better by people. However, you need to know that people will be more likely to click the link with bank advertisements right after they get a salary or in the early morning. And if you run this ad research only in the mornings, your results can be limited.

How to Minimize Threats to External Validity?

After you are aware of key threats to your research, you also need to know how to counter threats to external validity. We would say that the essential part is to understand all the threats and try to prevent them. However, there are a few things to consider for minimizing threats:

• Random sampling By selecting participants randomly from the target population, you can ensure that no participants are underrepresented.
• Increase sample size The larger your sample size, the more likely it is that your outcomes will be accurate.
• Diverse samples Try to recruit participants from various backgrounds to investigate multiple perspectives.
• Naturalistic settings Use natural context for experiments you are going to conduct.
• Replication By conducting the same study several times with different samples, researchers can determine if findings remain consistent.

Bottom Line on External Validity

External validity is referred to the ability to extend the research results to a broad audience or various circumstances. It helps scientists to understand how their work can be applied to solving real-life problems. This is why every researcher needs to understand its importance and know how to improve the data and results. In this text, we also outlined key threats to external validity and focused on some steps that can help you to minimize them. 

However, every time you are launching research, you need to start with your aims. After you understand it and have a clear vision of what you are going to study, you can easily navigate external validity and make your findings applicable to broader cases.

Hire an expert paper writer to ensure that your study is conducted with strong external validity, and delivered within your specified timeframe. Trust our writing service to get reliable results effort-free.

FAQ About External Validity

1. what is external validity in research.

External validity in research is the extent to which you can generalize your study findings. In other words, it shows if your results are applicable to a broad audience and can be used for solving a real-life problem. This is the key point for any type of research in various fields, as all scientists aim to be helpful to society.

2. What factors determine external validity?

There are a few threats that determine the external validity of the research. It can be sampling bias, history threat, observer bias, pre, and post-test effect, Hawthorne effect, novelty effect, and others. You need to focus on all factors that can have an influence on the result's objectivity in your study.

3. How do you ensure external validity?

If you want your study expanded to a broader audience and applied to other circumstances, you need to look carefully at all validity threats, use natural context for your field experiment, ensure that the whole population is represented in your experiment, and use some algorithms to correct all the factors that can change your results. 

Joe Eckel is an expert on Dissertations writing. He makes sure that each student gets precious insights on composing A-grade academic writing.

You may also like