research study bias types

Research Bias 101: What You Need To Know

By: Derek Jansen (MBA) | Expert Reviewed By: Dr Eunice Rautenbach | September 2022

If you’re new to academic research, research bias (also sometimes called researcher bias) is one of the many things you need to understand to avoid compromising your study. If you’re not careful, research bias can ruin the credibility of your study. 

In this post, we’ll unpack the thorny topic of research bias. We’ll explain what it is , look at some common types of research bias and share some tips to help you minimise the potential sources of bias in your research.

Overview: Research Bias 101

• What is research bias (or researcher bias)?
• Bias #1 – Selection bias
• Bias #2 – Analysis bias
• Bias #3 – Procedural (admin) bias

So, what is research bias?

Well, simply put, research bias is when the researcher – that’s you – intentionally or unintentionally skews the process of a systematic inquiry , which then of course skews the outcomes of the study . In other words, research bias is what happens when you affect the results of your research by influencing how you arrive at them.

For example, if you planned to research the effects of remote working arrangements across all levels of an organisation, but your sample consisted mostly of management-level respondents , you’d run into a form of research bias. In this case, excluding input from lower-level staff (in other words, not getting input from all levels of staff) means that the results of the study would be ‘biased’ in favour of a certain perspective – that of management.

Of course, if your research aims and research questions were only interested in the perspectives of managers, this sampling approach wouldn’t be a problem – but that’s not the case here, as there’s a misalignment between the research aims and the sample .

Now, it’s important to remember that research bias isn’t always deliberate or intended. Quite often, it’s just the result of a poorly designed study, or practical challenges in terms of getting a well-rounded, suitable sample. While perfect objectivity is the ideal, some level of bias is generally unavoidable when you’re undertaking a study. That said, as a savvy researcher, it’s your job to reduce potential sources of research bias as much as possible.

To minimize potential bias, you first need to know what to look for . So, next up, we’ll unpack three common types of research bias we see at Grad Coach when reviewing students’ projects . These include selection bias , analysis bias , and procedural bias . Keep in mind that there are many different forms of bias that can creep into your research, so don’t take this as a comprehensive list – it’s just a useful starting point.

Bias #1 – Selection Bias

First up, we have selection bias . The example we looked at earlier (about only surveying management as opposed to all levels of employees) is a prime example of this type of research bias. In other words, selection bias occurs when your study’s design automatically excludes a relevant group from the research process and, therefore, negatively impacts the quality of the results.

With selection bias, the results of your study will be biased towards the group that it includes or favours, meaning that you’re likely to arrive at prejudiced results . For example, research into government policies that only includes participants who voted for a specific party is going to produce skewed results, as the views of those who voted for other parties will be excluded.

Selection bias commonly occurs in quantitative research , as the sampling strategy adopted can have a major impact on the statistical results . That said, selection bias does of course also come up in qualitative research as there’s still plenty room for skewed samples. So, it’s important to pay close attention to the makeup of your sample and make sure that you adopt a sampling strategy that aligns with your research aims. Of course, you’ll seldom achieve a perfect sample, and that okay. But, you need to be aware of how your sample may be skewed and factor this into your thinking when you analyse the resultant data.

Need a helping hand?

Bias #2 – Analysis Bias

Next up, we have analysis bias . Analysis bias occurs when the analysis itself emphasises or discounts certain data points , so as to favour a particular result (often the researcher’s own expected result or hypothesis). In other words, analysis bias happens when you prioritise the presentation of data that supports a certain idea or hypothesis , rather than presenting all the data indiscriminately .

For example, if your study was looking into consumer perceptions of a specific product, you might present more analysis of data that reflects positive sentiment toward the product, and give less real estate to the analysis that reflects negative sentiment. In other words, you’d cherry-pick the data that suits your desired outcomes and as a result, you’d create a bias in terms of the information conveyed by the study.

Although this kind of bias is common in quantitative research, it can just as easily occur in qualitative studies, given the amount of interpretive power the researcher has. This may not be intentional or even noticed by the researcher, given the inherent subjectivity in qualitative research. As humans, we naturally search for and interpret information in a way that confirms or supports our prior beliefs or values (in psychology, this is called “confirmation bias”). So, don’t make the mistake of thinking that analysis bias is always intentional and you don’t need to worry about it because you’re an honest researcher – it can creep up on anyone .

To reduce the risk of analysis bias, a good starting point is to determine your data analysis strategy in as much detail as possible, before you collect your data . In other words, decide, in advance, how you’ll prepare the data, which analysis method you’ll use, and be aware of how different analysis methods can favour different types of data. Also, take the time to reflect on your own pre-conceived notions and expectations regarding the analysis outcomes (in other words, what do you expect to find in the data), so that you’re fully aware of the potential influence you may have on the analysis – and therefore, hopefully, can minimize it.

Bias #3 – Procedural Bias

Last but definitely not least, we have procedural bias , which is also sometimes referred to as administration bias . Procedural bias is easy to overlook, so it’s important to understand what it is and how to avoid it. This type of bias occurs when the administration of the study, especially the data collection aspect, has an impact on either who responds or how they respond.

A practical example of procedural bias would be when participants in a study are required to provide information under some form of constraint. For example, participants might be given insufficient time to complete a survey, resulting in incomplete or hastily-filled out forms that don’t necessarily reflect how they really feel. This can happen really easily, if, for example, you innocently ask your participants to fill out a survey during their lunch break.

Another form of procedural bias can happen when you improperly incentivise participation in a study. For example, offering a reward for completing a survey or interview might incline participants to provide false or inaccurate information just to get through the process as fast as possible and collect their reward. It could also potentially attract a particular type of respondent (a freebie seeker), resulting in a skewed sample that doesn’t really reflect your demographic of interest.

The format of your data collection method can also potentially contribute to procedural bias. If, for example, you decide to host your survey or interviews online, this could unintentionally exclude people who are not particularly tech-savvy, don’t have a suitable device or just don’t have a reliable internet connection. On the flip side, some people might find in-person interviews a bit intimidating (compared to online ones, at least), or they might find the physical environment in which they’re interviewed to be uncomfortable or awkward (maybe the boss is peering into the meeting room, for example). Either way, these factors all result in less useful data.

Although procedural bias is more common in qualitative research, it can come up in any form of fieldwork where you’re actively collecting data from study participants. So, it’s important to consider how your data is being collected and how this might impact respondents. Simply put, you need to take the respondent’s viewpoint and think about the challenges they might face, no matter how small or trivial these might seem. So, it’s always a good idea to have an informal discussion with a handful of potential respondents before you start collecting data and ask for their input regarding your proposed plan upfront.

Let’s Recap

Ok, so let’s do a quick recap. Research bias refers to any instance where the researcher, or the research design , negatively influences the quality of a study’s results, whether intentionally or not.

The three common types of research bias we looked at are:

• Selection bias – where a skewed sample leads to skewed results
• Analysis bias – where the analysis method and/or approach leads to biased results – and,
• Procedural bias – where the administration of the study, especially the data collection aspect, has an impact on who responds and how they respond.

As I mentioned, there are many other forms of research bias, but we can only cover a handful here. So, be sure to familiarise yourself with as many potential sources of bias as possible to minimise the risk of research bias in your study.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

This is really educational and I really like the simplicity of the language in here, but i would like to know if there is also some guidance in regard to the problem statement and what it constitutes.

Do you have a blog or video that differentiates research assumptions, research propositions and research hypothesis?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Incorporate STEM journalism in your classroom

• Exercise type: Activity
• Topic: Science & Society
• Category: Research & Design
• Category: Diversity in STEM

How bias affects scientific research

• Download Student Worksheet

Purpose: Students will work in groups to evaluate bias in scientific research and engineering projects and to develop guidelines for minimizing potential biases.

Procedural overview: After reading the Science News for Students article “ Think you’re not biased? Think again ,” students will discuss types of bias in scientific research and how to identify it. Students will then search the Science News archive for examples of different types of bias in scientific and medical research. Students will read the National Institute of Health’s Policy on Sex as a Biological Variable and analyze how this policy works to reduce bias in scientific research on the basis of sex and gender. Based on their exploration of bias, students will discuss the benefits and limitations of research guidelines for minimizing particular types of bias and develop guidelines of their own.

Approximate class time: 2 class periods

How Bias Affects Scientific Research student guide

Computer with access to the Science News archive

Interactive meeting and screen-sharing application for virtual learning (optional)

Directions for teachers:

One of the guiding principles of scientific inquiry is objectivity. Objectivity is the idea that scientific questions, methods and results should not be affected by the personal values, interests or perspectives of researchers. However, science is a human endeavor, and experimental design and analysis of information are products of human thought processes. As a result, biases may be inadvertently introduced into scientific processes or conclusions.

In scientific circles, bias is described as any systematic deviation between the results of a study and the “truth.” Bias is sometimes described as a tendency to prefer one thing over another, or to favor one person, thing or explanation in a way that prevents objectivity or that influences the outcome of a study or the understanding of a phenomenon. Bias can be introduced in multiple points during scientific research — in the framing of the scientific question, in the experimental design, in the development or implementation of processes used to conduct the research, during collection or analysis of data, or during the reporting of conclusions.

Researchers generally recognize several different sources of bias, each of which can strongly affect the results of STEM research. Three types of bias that often occur in scientific and medical studies are researcher bias, selection bias and information bias.

Researcher bias occurs when the researcher conducting the study is in favor of a certain result. Researchers can influence outcomes through their study design choices, including who they choose to include in a study and how data are interpreted. Selection bias can be described as an experimental error that occurs when the subjects of the study do not accurately reflect the population to whom the results of the study will be applied. This commonly happens as unequal inclusion of subjects of different races, sexes or genders, ages or abilities. Information bias occurs as a result of systematic errors during the collection, recording or analysis of data.

When bias occurs, a study’s results may not accurately represent phenomena in the real world, or the results may not apply in all situations or equally for all populations. For example, if a research study does not address the full diversity of people to whom the solution will be applied, then the researchers may have missed vital information about whether and how that solution will work for a large percentage of a target population.

Bias can also affect the development of engineering solutions. For example, a new technology product tested only with teenagers or young adults who are comfortable using new technologies may have user experience issues when placed in the hands of older adults or young children.

Want to make it a virtual lesson? Post the links to the  Science News for Students article “ Think you’re not biased? Think again ,” and the National Institutes of Health information on sickle-cell disease . A link to additional resources can be provided for the students who want to know more. After students have reviewed the information at home, discuss the four questions in the setup and the sickle-cell research scenario as a class. When the students have a general understanding of bias in research, assign students to breakout rooms to look for examples of different types of bias in scientific and medical research, to discuss the Science News article “ Biomedical studies are including more female subjects (finally) ” and the National Institute of Health’s Policy on Sex as a Biological Variable and to develop bias guidelines of their own. Make sure the students have links to all articles they will need to complete their work. Bring the groups back together for an all-class discussion of the bias guidelines they write.

Assign the Science News for Students article “ Think you’re not biased? Think again ” as homework reading to introduce students to the core concepts of scientific objectivity and bias. Request that they answer the first two questions on their guide before the first class discussion on this topic. In this discussion, you will cover the idea of objective truth and introduce students to the terminology used to describe bias. Use the background information to decide what level of detail you want to give to your students.

As students discuss bias, help them understand objective and subjective data and discuss the importance of gathering both kinds of data. Explain to them how these data differ. Some phenomena — for example, body temperature, blood type and heart rate — can be objectively measured. These data tend to be quantitative. Other phenomena cannot be measured objectively and must be considered subjectively. Subjective data are based on perceptions, feelings or observations and tend to be qualitative rather than quantitative. Subjective measurements are common and essential in biomedical research, as they can help researchers understand whether a therapy changes a patient’s experience. For instance, subjective data about the amount of pain a patient feels before and after taking a medication can help scientists understand whether and how the drug works to alleviate pain. Subjective data can still be collected and analyzed in ways that attempt to minimize bias.

Try to guide student discussion to include a larger context for bias by discussing the effects of bias on understanding of an “objective truth.” How can someone’s personal views and values affect how they analyze information or interpret a situation?

To help students understand potential effects of biases, present them with the following scenario based on information from the National Institutes of Health :

Sickle-cell disease is a group of inherited disorders that cause abnormalities in red blood cells. Most of the people who have sickle-cell disease are of African descent; it also appears in populations from the Mediterranean, India and parts of Latin America. Males and females are equally likely to inherit the condition. Imagine that a therapy was developed to treat the condition, and clinical trials enlisted only male subjects of African descent. How accurately would the results of that study reflect the therapy’s effectiveness for all people who suffer from sickle-cell disease?

In the sickle-cell scenario described above, scientists will have a good idea of how the therapy works for males of African descent. But they may not be able to accurately predict how the therapy will affect female patients or patients of different races or ethnicities. Ask the students to consider how they would devise a study that addressed all the populations affected by this disease.

Before students move on, have them answer the following questions. The first two should be answered for homework and discussed in class along with the remaining questions.

1.What is bias?

In common terms, bias is a preference for or against one idea, thing or person. In scientific research, bias is a systematic deviation between observations or interpretations of data and an accurate description of a phenomenon.

2. How can biases affect the accuracy of scientific understanding of a phenomenon? How can biases affect how those results are applied?

Bias can cause the results of a scientific study to be disproportionately weighted in favor of one result or group of subjects. This can cause misunderstandings of natural processes that may make conclusions drawn from the data unreliable. Biased procedures, data collection or data interpretation can affect the conclusions scientists draw from a study and the application of those results. For example, if the subjects that participate in a study testing an engineering design do not reflect the diversity of a population, the end product may not work as well as desired for all users.

3. Describe two potential sources of bias in a scientific, medical or engineering research project. Try to give specific examples.

Researchers can intentionally or unintentionally introduce biases as a result of their attitudes toward the study or its purpose or toward the subjects or a group of subjects. Bias can also be introduced by methods of measuring, collecting or reporting data. Examples of potential sources of bias include testing a small sample of subjects, testing a group of subjects that is not diverse and looking for patterns in data to confirm ideas or opinions already held.

4. How can potential biases be identified and eliminated before, during or after a scientific study?

Students should brainstorm ways to identify sources of bias in the design of research studies. They may suggest conducting implicit bias testing or interviews before a study can be started, developing guidelines for research projects, peer review of procedures and samples/subjects before beginning a study, and peer review of data and conclusions after the study is completed and before it is published. Students may focus on the ideals of transparency and replicability of results to help reduce biases in scientific research.

Obtain and evaluate information about bias

Students will now work in small groups to select and analyze articles for different types of bias in scientific and medical research. Students will start by searching the Science News or Science News for Students archives and selecting articles that describe scientific studies or engineering design projects. If the Science News or Science News for Students articles chosen by students do not specifically cite and describe a study, students should consult the Citations at the end of the article for links to related primary research papers. Students may need to read the methods section and the conclusions of the primary research paper to better understand the project’s design and to identify potential biases. Do not assume that every scientific paper features biased research.

Student groups should evaluate the study or engineering design project outlined in the article to identify any biases in the experimental design, data collection, analysis or results. Students may need additional guidance for identifying biases. Remind them of the prior discussion about sources of bias and task them to review information about indicators of bias. Possible indicators include extreme language such as all , none or nothing ; emotional appeals rather than logical arguments; proportions of study subjects with specific characteristics such as gender, race or age; arguments that support or refute one position over another and oversimplifications or overgeneralizations. Students may also want to look for clues related to the researchers’ personal identity such as race, religion or gender. Information on political or religious points of view, sources of funding or professional affiliations may also suggest biases.

Students should also identify any deliberate attempts to reduce or eliminate bias in the project or its results. Then groups should come back together and share the results of their analysis with the class.

If students need support in searching the archives for appropriate articles, encourage groups to brainstorm search terms that may turn up related articles. Some potential search terms include bias , study , studies , experiment , engineer , new device , design , gender , sex , race , age , aging , young , old , weight , patients , survival or medical .

If you are short on time or students do not have access to the Science News or Science News for Students archive, you may want to provide articles for students to review. Some suggested articles are listed in the additional resources  below.

Once groups have selected their articles, students should answer the following questions in their groups.

1. Record the title and URL of the article and write a brief summary of the study or project.

Answers will vary, but students should accurately cite the article evaluated and summarize the study or project described in the article. Sample answer: We reviewed the Science News article “Even brain images can be biased,” which can be found at www.sciencenews.org/blog/scicurious/even-brain-images-can-be-biased. This article describes how scientific studies of human brains that involve electronic images of brains tend to include study subjects from wealthier and more highly educated households and how researchers set out to collect new data to make the database of brain images more diverse.

2. What sources of potential bias (if any) did you identify in the study or project? Describe any procedures or policies deliberately included in the study or project to eliminate biases.

The article “Even brain images can be biased” describes how scientists identified a sampling bias in studies of brain images that resulted from the way subjects were recruited. Most of these studies were conducted at universities, so many college students volunteer to participate, which resulted in the samples being skewed toward wealthier, educated, white subjects. Scientists identified a database of pediatric brain images and evaluated the diversity of the subjects in that database. They found that although the subjects in that database were more ethnically diverse than the U.S. population, the subjects were generally from wealthier households and the parents of the subjects tended to be more highly educated than average. Scientists applied statistical methods to weight the data so that study samples from the database would more accurately reflect American demographics.

3. How could any potential biases in the study or design project have affected the results or application of the results to the target population?

Scientists studying the rate of brain development in children were able to recognize the sampling bias in the brain image database. When scientists were able to apply statistical methods to ensure a better representation of socioeconomically diverse samples, they saw a different pattern in the rate of brain development in children. Scientists learned that, in general, children’s brains matured more quickly than they had previously thought. They were able to draw new conclusions about how certain factors, such as family wealth and education, affected the rate at which children’s brains developed. But the scientsits also suggested that they needed to perform additional studies with a deliberately selected group of children to ensure true diversity in the samples.

In this phase, students will review the Science News article “ Biomedical studies are including more female subjects (finally) ” and the NIH Policy on Sex as a Biological Variable , including the “ guidance document .” Students will identify how sex and gender biases may have affected the results of biomedical research before NIH instituted its policy. The students will then work with their group to recommend other policies to minimize biases in biomedical research.

To guide their development of proposed guidelines, students should answer the following questions in their groups.

1. How have sex and gender biases affected the value and application of biomedical research?

Gender and sex biases in biomedical research have diminished the accuracy and quality of research studies and reduced the applicability of results to the entire population. When girls and women are not included in research studies, the responses and therapeutic outcomes of approximately half of the target population for potential therapies remain unknown.

2. Why do you think the NIH created its policy to reduce sex and gender biases?

In the guidance document, the NIH states that “There is a growing recognition that the quality and generalizability of biomedical research depends on the consideration of key biological variables, such as sex.” The document goes on to state that many diseases and conditions affect people of both sexes, and restricting diversity of biological variables, notably sex and gender, undermines the “rigor, transparency, and generalizability of research findings.”

3. What impact has the NIH Policy on Sex as a Biological Variable had on biomedical research?

The NIH’s policy that sex is factored into research designs, analyses and reporting tries to ensure that when developing and funding biomedical research studies, researchers and institutes address potential biases in the planning stages, which helps to reduce or eliminate those biases in the final study. Including females in biomedical research studies helps to ensure that the results of biomedical research are applicable to a larger proportion of the population, expands the therapies available to girls and women and improves their health care outcomes.

4. What other policies do you think the NIH could institute to reduce biases in biomedical research? If you were to recommend one set of additional guidelines for reducing bias in biomedical research, what guidelines would you propose? Why?

Students could suggest that the NIH should have similar policies related to race, gender identity, wealth/economic status and age. Students should identify a category of bias or an underserved segment of the population that they think needs to be addressed in order to improve biomedical research and health outcomes for all people and should recommend guidelines to reduce bias related to that group. Students recommending guidelines related to race might suggest that some populations, such as African Americans, are historically underserved in terms of access to medical services and health care, and they might suggest guidelines to help reduce the disparity. Students might recommend that a certain percentage of each biomedical research project’s sample include patients of diverse racial and ethnic backgrounds.

5. What biases would your suggested policy help eliminate? How would it accomplish that goal?

Students should describe how their proposed policy would address a discrepancy in the application of biomedical research to the entire human population. Race can be considered a biological variable, like sex, and race has been connected to higher or lower incidence of certain characteristics or medical conditions, such as blood types or diabetes, which sometimes affect how the body reponds to infectious agents, drugs, procedures or other therapies. By ensuring that people from diverse racial and ethnic groups are included in biomedical research studies, scientists and medical professionals can provide better medical care to members of those populations.

Class discussion about bias guidelines

Allow each group time to present its proposed bias-reducing guideline to another group and to receive feedback. Then provide groups with time to revise their guidelines, if necessary. Act as a facilitator while students conduct the class discussion. Use this time to assess individual and group progress. Students should demonstrate an understanding of different biases that may affect patient outcomes in biomedical research studies and in practical medical settings. As part of the group discussion, have students answer the following questions.

1. Why is it important to identify and eliminate biases in research and engineering design?

The goal of most scientific research and engineering projects is to improve the quality of life and the depth of understanding of the world we live in. By eliminating biases, we can better serve the entirety of the human population and the planet .

2. Were there any guidelines that were suggested by multiple groups? How do those actions or policies help reduce bias?

Answers will depend on the guidelines developed and recommended by other groups. Groups could suggest policies related to race, gender identity, wealth/economic status and age. Each group should clearly identify how its guidelines are designed to reduce bias and improve the quality of human life.

3. Which guidelines developed by your classmates do you think would most reduce the effects of bias on research results or engineering designs? Support your selection with evidence and scientific reasoning.

Answers will depend on the guidelines developed and recommended by other groups. Students should agree that guidelines that minimize inequities and improve health care outcomes for a larger group are preferred. Guidelines addressing inequities of race and wealth/economic status are likely to expand access to improved medical care for the largest percentage of the population. People who grow up in less economically advantaged settings have specific health issues related to nutrition and their access to clean water, for instance. Ensuring that people from the lowest economic brackets are represented in biomedical research improves their access to medical care and can dramatically change the length and quality of their lives.

Possible extension

Challenge students to honestly evaluate any biases they may have. Encourage them to take an Implicit Association Test (IAT) to identify any implicit biases they may not recognize. Harvard University has an online IAT platform where students can participate in different assessments to identify preferences and biases related to sex and gender, race, religion, age, weight and other factors. You may want to challenge students to take a test before they begin the activity, and then assign students to take a test after completing the activity to see if their preferences have changed. Students can report their results to the class if they want to discuss how awareness affects the expression of bias.

Additional resources

If you want additional resources for the discussion or to provide resources for student groups, check out the links below.

Additional Science News articles:

Even brain images can be biased

Data-driven crime prediction fails to erase human bias

What we can learn from how a doctor’s race can affect Black newborns’ survival

Bias in a common health care algorithm disproportionately hurts black patients

Female rats face sex bias too

There’s no evidence that a single ‘gay gene’ exists

Positive attitudes about aging may pay off in better health

What male bias in the mammoth fossil record says about the animal’s social groups

The man flu struggle might be real, says one researcher

Scientists may work to prevent bias, but they don’t always say so

The Bias Finders

Showdown at Sex Gap

University resources:

Project Implicit (Take an Implicit Association Tests)

Catalogue of Bias

Understanding Health Research

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews
• Research question
• Conceptual framework
• Conceptual vs. theoretical framework
• Data collection
• Qualitative research methods
• Focus groups
• Observational research
• Case studies
• Ethnographical research
• Ethical considerations
• Confidentiality and privacy

What is research bias?

Understanding unconscious bias, how to avoid bias in research, bias and subjectivity in research.

• Power dynamics
• Reflexivity

Bias in research

In a purely objective world, research bias would not exist because knowledge would be a fixed and unmovable resource; either one knows about a particular concept or phenomenon, or they don't. However, qualitative research and the social sciences both acknowledge that subjectivity and bias exist in every aspect of the social world, which naturally includes the research process too. This bias is manifest in the many different ways that knowledge is understood, constructed, and negotiated, both in and out of research.

Understanding research bias has profound implications for data collection methods and data analysis , requiring researchers to take particular care of how to account for the insights generated from their data .

Research bias, often unavoidable, is a systematic error that can creep into any stage of the research process , skewing our understanding and interpretation of findings. From data collection to analysis, interpretation , and even publication , bias can distort the truth we seek to capture and communicate in our research.

It’s also important to distinguish between bias and subjectivity, especially when engaging in qualitative research . Most qualitative methodologies are based on epistemological and ontological assumptions that there is no such thing as a fixed or objective world that exists “out there” that can be empirically measured and understood through research. Rather, many qualitative researchers embrace the socially constructed nature of our reality and thus recognize that all data is produced within a particular context by participants with their own perspectives and interpretations. Moreover, the researcher’s own subjective experiences inevitably shape how they make sense of the data. These subjectivities are considered to be strengths, not limitations, of qualitative research approaches, because they open new avenues for knowledge generation. This is also why reflexivity is so important in qualitative research. When we refer to bias in this guide, on the other hand, we are referring to systematic errors that can negatively affect the research process but that can be mitigated through researchers’ careful efforts.

To fully grasp what research bias is, it's essential to understand the dual nature of bias. Bias is not inherently evil. It's simply a tendency, inclination, or prejudice for or against something. In our daily lives, we're subject to countless biases, many of which are unconscious. They help us navigate our world, make quick decisions, and understand complex situations. But when conducting research, these same biases can cause significant issues.

Research bias can affect the validity and credibility of research findings, leading to erroneous conclusions. It can emerge from the researcher's subconscious preferences or the methodological design of the study itself. For instance, if a researcher unconsciously favors a particular outcome of the study, this preference could affect how they interpret the results, leading to a type of bias known as confirmation bias.

Research bias can also arise due to the characteristics of study participants. If the researcher selectively recruits participants who are more likely to produce desired outcomes, this can result in selection bias.

Another form of bias can stem from data collection methods . If a survey question is phrased in a way that encourages a particular response, this can introduce response bias. Moreover, inappropriate survey questions can have a detrimental effect on future research if such studies are seen by the general population as biased toward particular outcomes depending on the preferences of the researcher.

Bias can also occur during data analysis . In qualitative research for instance, the researcher's preconceived notions and expectations can influence how they interpret and code qualitative data, a type of bias known as interpretation bias. It's also important to note that quantitative research is not free of bias either, as sampling bias and measurement bias can threaten the validity of any research findings.

Given these examples, it's clear that research bias is a complex issue that can take many forms and emerge at any stage in the research process. This section will delve deeper into specific types of research bias, provide examples, discuss why it's an issue, and provide strategies for identifying and mitigating bias in research.

What is an example of bias in research?

Bias can appear in numerous ways. One example is confirmation bias, where the researcher has a preconceived explanation for what is going on in their data, and any disconfirming evidence is (unconsciously) ignored. For instance, a researcher conducting a study on daily exercise habits might be inclined to conclude that meditation practices lead to greater engagement in exercise because that researcher has personally experienced these benefits. However, conducting rigorous research entails assessing all the data systematically and verifying one’s conclusions by checking for both supporting and refuting evidence.

What is a common bias in research?

Confirmation bias is one of the most common forms of bias in research. It happens when researchers unconsciously focus on data that supports their ideas while ignoring or undervaluing data that contradicts their ideas. This bias can lead researchers to mistakenly confirm their theories, despite having insufficient or conflicting evidence.

What are the different types of bias?

There are several types of research bias, each presenting unique challenges. Some common types include:

Confirmation bias: As already mentioned, this happens when a researcher focuses on evidence supporting their theory while overlooking contradictory evidence.

Selection bias: This occurs when the researcher's method of choosing participants skews the sample in a particular direction.

Response bias: This happens when participants in a study respond inaccurately or falsely, often due to misleading or poorly worded questions.

Observer bias (or researcher bias): This occurs when the researcher unintentionally influences the results because of their expectations or preferences.

Publication bias: This type of bias arises when studies with positive results are more likely to get published, while studies with negative or null results are often ignored.

Analysis bias: This type of bias occurs when the data is manipulated or analyzed in a way that leads to a particular result, whether intentionally or unintentionally.

What is an example of researcher bias?

Researcher bias, also known as observer bias, can occur when a researcher's expectations or personal beliefs influence the results of a study. For instance, if a researcher believes that a particular therapy is effective, they might unconsciously interpret ambiguous results in a way that supports the efficacy of the therapy, even if the evidence is not strong enough.

Even quantitative research methodologies are not immune from bias from researchers. Market research surveys or clinical trial research, for example, may encounter bias when the researcher chooses a particular population or methodology to achieve a specific research outcome. Questions in customer feedback surveys whose data is employed in quantitative analysis can be structured in such a way as to bias survey respondents toward certain desired answers.

Turn your data into findings with ATLAS.ti

Key insights are at your fingertips with our powerful interface. See how with a free trial.

Identifying and avoiding bias in research

As we will remind you throughout this chapter, bias is not a phenomenon that can be removed altogether, nor should we think of it as something that should be eliminated. In a subjective world involving humans as researchers and research participants, bias is unavoidable and almost necessary for understanding social behavior. The section on reflexivity later in this guide will highlight how different perspectives among researchers and human subjects are addressed in qualitative research. That said, bias in excess can place the credibility of a study's findings into serious question. Scholars who read your research need to know what new knowledge you are generating, how it was generated, and why the knowledge you present should be considered persuasive. With that in mind, let's look at how bias can be identified and, where it interferes with research, minimized.

How do you identify bias in research?

Identifying bias involves a critical examination of your entire research study involving the formulation of the research question and hypothesis , the selection of study participants, the methods for data collection, and the analysis and interpretation of data. Researchers need to assess whether each stage has been influenced by bias that may have skewed the results. Tools such as bias checklists or guidelines, peer review , and reflexivity (reflecting on one's own biases) can be instrumental in identifying bias.

How do you identify research bias?

Identifying research bias often involves careful scrutiny of the research methodology and the researcher's interpretations. Was the sample of participants relevant to the research question ? Were the interview or survey questions leading? Were there any conflicts of interest that could have influenced the results? It also requires an understanding of the different types of bias and how they might manifest in a research context. Does the bias occur in the data collection process or when the researcher is analyzing data?

Research transparency requires a careful accounting of how the study was designed, conducted, and analyzed. In qualitative research involving human subjects, the researcher is responsible for documenting the characteristics of the research population and research context. With respect to research methods, the procedures and instruments used to collect and analyze data are described in as much detail as possible.

While describing study methodologies and research participants in painstaking detail may sound cumbersome, a clear and detailed description of the research design is necessary for good research. Without this level of detail, it is difficult for your research audience to identify whether bias exists, where bias occurs, and to what extent it may threaten the credibility of your findings.

How to recognize bias in a study?

Recognizing bias in a study requires a critical approach. The researcher should question every step of the research process: Was the sample of participants selected with care? Did the data collection methods encourage open and sincere responses? Did personal beliefs or expectations influence the interpretation of the results? External peer reviews can also be helpful in recognizing bias, as others might spot potential issues that the original researcher missed.

The subsequent sections of this chapter will delve into the impacts of research bias and strategies to avoid it. Through these discussions, researchers will be better equipped to handle bias in their work and contribute to building more credible knowledge.

Unconscious biases, also known as implicit biases, are attitudes or stereotypes that influence our understanding, actions, and decisions in an unconscious manner. These biases can inadvertently infiltrate the research process, skewing the results and conclusions. This section aims to delve deeper into understanding unconscious bias, its impact on research, and strategies to mitigate it.

What is unconscious bias?

Unconscious bias refers to prejudices or social stereotypes about certain groups that individuals form outside their conscious awareness. Everyone holds unconscious beliefs about various social and identity groups, and these biases stem from a tendency to organize social worlds into categories.

How does unconscious bias infiltrate research?

Unconscious bias can infiltrate research in several ways. It can affect how researchers formulate their research questions or hypotheses , how they interact with participants, their data collection methods, and how they interpret their data . For instance, a researcher might unknowingly favor participants who share similar characteristics with them, which could lead to biased results.

Implications of unconscious bias

The implications of unconscious research bias are far-reaching. It can compromise the validity of research findings , influence the choice of research topics, and affect peer review processes . Unconscious bias can also lead to a lack of diversity in research, which can severely limit the value and impact of the findings.

Strategies to mitigate unconscious research bias

While it's challenging to completely eliminate unconscious bias, several strategies can help mitigate its impact. These include being aware of potential unconscious biases, practicing reflexivity , seeking diverse perspectives for your study, and engaging in regular bias-checking activities, such as bias training and peer debriefing .

By understanding and acknowledging unconscious bias, researchers can take steps to limit its impact on their work, leading to more robust findings.

Why is researcher bias an issue?

Research bias is a pervasive issue that researchers must diligently consider and address. It can significantly impact the credibility of findings. Here, we break down the ramifications of bias into two key areas.

How bias affects validity

Research validity refers to the accuracy of the study findings, or the coherence between the researcher’s findings and the participants’ actual experiences. When bias sneaks into a study, it can distort findings and move them further away from the realities that were shared by the research participants. For example, if a researcher's personal beliefs influence their interpretation of data , the resulting conclusions may not reflect what the data show or what participants experienced.

The transferability problem

Transferability is the extent to which your study's findings can be applied beyond the specific context or sample studied. Applying knowledge from one context to a different context is how we can progress and make informed decisions. In quantitative research , the generalizability of a study is a key component that shapes the potential impact of the findings. In qualitative research , all data and knowledge that is produced is understood to be embedded within a particular context, so the notion of generalizability takes on a slightly different meaning. Rather than assuming that the study participants are statistically representative of the entire population, qualitative researchers can reflect on which aspects of their research context bear the most weight on their findings and how these findings may be transferable to other contexts that share key similarities.

How does bias affect research?

Research bias, if not identified and mitigated, can significantly impact research outcomes. The ripple effects of research bias extend beyond individual studies, impacting the body of knowledge in a field and influencing policy and practice. Here, we delve into three specific ways bias can affect research.

Distortion of research results

Bias can lead to a distortion of your study's findings. For instance, confirmation bias can cause a researcher to focus on data that supports their interpretation while disregarding data that contradicts it. This can skew the results and create a misleading picture of the phenomenon under study.

Undermining scientific progress

When research is influenced by bias, it not only misrepresents participants’ realities but can also impede scientific progress. Biased studies can lead researchers down the wrong path, resulting in wasted resources and efforts. Moreover, it could contribute to a body of literature that is skewed or inaccurate, misleading future research and theories.

Influencing policy and practice based on flawed findings

Research often informs policy and practice. If the research is biased, it can lead to the creation of policies or practices that are ineffective or even harmful. For example, a study with selection bias might conclude that a certain intervention is effective, leading to its broad implementation. However, suppose the transferability of the study's findings was not carefully considered. In that case, it may be risky to assume that the intervention will work as well in different populations, which could lead to ineffective or inequitable outcomes.

While it's almost impossible to eliminate bias in research entirely, it's crucial to mitigate its impact as much as possible. By employing thoughtful strategies at every stage of research, we can strive towards rigor and transparency , enhancing the quality of our findings. This section will delve into specific strategies for avoiding bias.

How do you know if your research is biased?

Determining whether your research is biased involves a careful review of your research design, data collection , analysis , and interpretation . It might require you to reflect critically on your own biases and expectations and how these might have influenced your research. External peer reviews can also be instrumental in spotting potential bias.

Strategies to mitigate bias

Minimizing bias involves careful planning and execution at all stages of a research study. These strategies could include formulating clear, unbiased research questions , ensuring that your sample meaningfully represents the research problem you are studying, crafting unbiased data collection instruments, and employing systematic data analysis techniques. Transparency and reflexivity throughout the process can also help minimize bias.

Mitigating bias in data collection

To mitigate bias in data collection, ensure your questions are clear, neutral, and not leading. Triangulation, or using multiple methods or data sources, can also help to reduce bias and increase the credibility of your findings.

Mitigating bias in data analysis

During data analysis , maintaining a high level of rigor is crucial. This might involve using systematic coding schemes in qualitative research or appropriate statistical tests in quantitative research . Regularly questioning your interpretations and considering alternative explanations can help reduce bias. Peer debriefing , where you discuss your analysis and interpretations with colleagues, can also be a valuable strategy.

By using these strategies, researchers can significantly reduce the impact of bias on their research, enhancing the quality and credibility of their findings and contributing to a more robust and meaningful body of knowledge.

Impact of cultural bias in research

Cultural bias is the tendency to interpret and judge phenomena by standards inherent to one's own culture. Given the increasingly multicultural and global nature of research, understanding and addressing cultural bias is paramount. This section will explore the concept of cultural bias, its impacts on research, and strategies to mitigate it.

What is cultural bias in research?

Cultural bias refers to the potential for a researcher's cultural background, experiences, and values to influence the research process and findings. This can occur consciously or unconsciously and can lead to misinterpretation of data, unfair representation of cultures, and biased conclusions.

How does cultural bias infiltrate research?

Cultural bias can infiltrate research at various stages. It can affect the framing of research questions , the design of the study, the methods of data collection , and the interpretation of results . For instance, a researcher might unintentionally design a study that does not consider the cultural context of the participants, leading to a biased understanding of the phenomenon being studied.

Implications of cultural bias

The implications of cultural bias are profound. Cultural bias can skew your findings, limit the transferability of results, and contribute to cultural misunderstandings and stereotypes. This can ultimately lead to inaccurate or ethnocentric conclusions, further perpetuating cultural bias and inequities.

As a result, many social science fields like sociology and anthropology have been critiqued for cultural biases in research. Some of the earliest research inquiries in anthropology, for example, have had the potential to reduce entire cultures to simplistic stereotypes when compared to mainstream norms. A contemporary researcher respecting ethical and cultural boundaries, on the other hand, should seek to properly place their understanding of social and cultural practices in sufficient context without inappropriately characterizing them.

Strategies to mitigate cultural bias

Mitigating cultural bias requires a concerted effort throughout the research study. These efforts could include educating oneself about other cultures, being aware of one's own cultural biases, incorporating culturally diverse perspectives into the research process, and being sensitive and respectful of cultural differences. It might also involve including team members with diverse cultural backgrounds or seeking external cultural consultants to challenge assumptions and provide alternative perspectives.

By acknowledging and addressing cultural bias, researchers can contribute to more culturally competent, equitable, and valid research. This not only enriches the scientific body of knowledge but also promotes cultural understanding and respect.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Keep in mind that bias is a force to be mitigated, not a phenomenon that can be eliminated altogether, and the subjectivities of each person are what make our world so complex and interesting. As things are continuously changing and adapting, research knowledge is also continuously being updated as we further develop our understanding of the world around us.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

8.4   Introduction to sources of bias in clinical trials

The reliability of the results of a randomized trial depends on the extent to which potential sources of bias have been avoided. A key part of a review is to consider the risk of bias in the results of each of the eligible studies. A useful classification of biases is into selection bias, performance bias, attrition bias, detection bias and reporting bias. In this section we describe each of these biases and introduce seven corresponding domains that are assessed in the Collaboration’s ‘Risk of bias’ tool. These are summarized in Table 8.4.a . We describe the tool for assessing the seven domains in Section   8.5 . We provide more detailed consideration of each issue in Sections 8.9 to 8.15 .

8.4.1 Selection bias

Selection bias refers to systematic differences between baseline characteristics of the groups that are compared. The unique strength of randomization is that, if successfully accomplished, it prevents selection bias in allocating interventions to participants.  Its success in this respect depends on fulfilling several interrelated processes.  A rule for allocating interventions to participants must be specified, based on some chance (random) process. We call this sequence generation . Furthermore, steps must be taken to secure strict implementation of that schedule of random assignments by preventing foreknowledge of the forthcoming allocations. This process if often termed allocation concealment , although could more accurately be described as allocation sequence concealment. Thus, one suitable method for assigning interventions would be to use a simple random (and therefore unpredictable) sequence, and to conceal the upcoming allocations from those involved in enrolment into the trial.

For all potential sources of bias, it is important to consider the likely magnitude and the likely direction of the bias. For example, if all methodological limitations of studies were expected to bias the results towards a lack of effect, and the evidence indicates that the intervention is effective, then it may be concluded that the intervention is effective even in the presence of these potential biases.

8.4.2 Performance bias

Performance bias refers to systematic differences between groups in the care that is provided, or in exposure to factors other than the interventions of interest. . After enrolment into the study, blinding (or masking) of study participants and personnel may reduce the risk that knowledge of which intervention was received, rather than the intervention itself, affects outcomes. Effective blinding can also ensure that the compared groups receive a similar amount of attention, ancillary treatment and diagnostic investigations. Blinding is not always possible, however. For example, it is usually impossible to blind people to whether or not major surgery has been undertaken.

8.4.3 Detection bias

Detection bias refers to systematic differences between groups in how outcomes are determined. Blinding (or masking) of outcome assessors may reduce the risk that knowledge of which intervention was received, rather than the intervention itself, affects outcome measurement. Blinding of outcome assessors can be especially important for assessment of subjective outcomes, such as degree of postoperative pain.

8.4.4 Attrition bias

Attrition bias refers to systematic differences between groups in withdrawals from a study. Withdrawals from the study lead to incomplete outcome data. There are two reasons for withdrawals or incomplete outcome data in clinical trials. Exclusions refer to situations in which some participants are omitted from reports of analyses, despite outcome data being available to the trialists. Attrition refers to situations in which outcome data are not available.

8.4.5 Reporting bias

Reporting bias refers to systematic differences between reported and unreported findings. Within a published report those analyses with statistically significant differences between intervention groups are more likely to be reported than non-significant differences. This sort of ‘within-study publication bias’  is usually known as outcome reporting bias or selective reporting bias, and may be one of the most substantial biases affecting results from individual studies (Chan 2005).

8.4.6 Other biases

In addition there are other sources of bias that are relevant only in certain circumstances. These relate mainly to particular trial designs (e.g. carry-over in cross-over trials and recruitment bias in cluster-randomized trials); some can be found across a broad spectrum of trials, but only for specific circumstances (e.g. contamination, whereby the experimental and control interventions get ‘mixed’, for example if participants pool their drugs); and there may be sources of bias that are only found in a particular clinical setting.

8 Types of Research Bias and How to Avoid Them?

Appinio Research · 18.10.2023 · 39min read

Curious about how to ensure the integrity of your research ? Ever wondered how research bias can impact your findings? How might it affect your data-driven decisions?

Join us on a journey through the intricate landscape of unbiased research as we delve deep into strategies and real-world examples to guide you toward more reliable insights.

What is Bias in Research?

Research bias, often simply referred to as bias, is a systematic error or deviation from the true results or inferences in research. It occurs when the design, conduct, or interpretation of a study systematically skews the findings in a particular direction, leading to inaccurate or misleading results. Bias can manifest in various forms and at different stages of the research process, and it can compromise the validity and reliability of research outcomes.

Key Aspects of Research Bias

• Systematic Error: Bias is not a random occurrence but a systematic error that consistently influences research outcomes.
• Influence on Results: Bias can lead to overestimating or underestimating effects, associations, or relationships studied.
• Unintentional or Intentional: Bias can be unintentional, stemming from flaws in study design, data collection, or analysis. In some cases, it can also be introduced intentionally, leading to deliberate distortion of results.
• Impact on Decision-Making: Research bias can have significant consequences, affecting decisions in fields ranging from healthcare and policy to marketing and academia.

Understanding and recognizing the various types and sources of bias is crucial for researchers to minimize its impact and produce credible, objective, and actionable research findings.

Importance of Avoiding Research Bias

Avoiding research bias is paramount for several compelling reasons, as it directly affects the quality and integrity of research outcomes. Here's why researchers and decision-makers should prioritize bias mitigation:

• Credibility and Trustworthiness: Research bias undermines the credibility and trustworthiness of research findings. Biased results can erode public trust, damage an organization's reputation, and hinder the acceptance of research in the scientific community.
• Informed Decision-Making: Research serves as the foundation for informed decision-making in various fields. Bias can lead to erroneous conclusions, potentially leading to misguided policies, ineffective treatments, or poor business strategies.
• Resource Allocation: Bias can result in the misallocation of valuable resources. When resources are allocated based on biased research, they may not effectively address the intended issues or challenges.
• Ethical Considerations: Introducing bias, whether intentionally or unintentionally, raises ethical concerns in research. Ethical research practices demand objectivity, transparency, and fairness in the pursuit of knowledge.
• Advancement of Knowledge: Research contributes to the advancement of knowledge and innovation. Bias hinders scientific progress by introducing errors and distorting the true nature of phenomena, hindering the development of accurate theories and solutions.
• Public Health and Safety: In fields like healthcare, bias can have life-and-death implications. Biased medical research can lead to the adoption of less effective or potentially harmful treatments, putting patient health and safety at risk.
• Economic Impact: In business and economics , biased research can result in poor investment decisions, market strategies, and financial losses. Avoiding bias is essential for achieving sound economic outcomes.

In the pursuit of unbiased research, having a robust data collection platform is crucial. Appinio offers a comprehensive solution that helps researchers gather data in a transparent and unbiased manner. With Appinio, you can access a diverse and representative pool of respondents, ensuring that your research reflects a broad spectrum of perspectives.   Appinio 's platform also facilitates data collection with rigorous quality control, reducing the risk of bias at the source. By partnering with Appinio, you can enhance the integrity and credibility of your work, ultimately contributing to more reliable and impactful research outcomes.   Explore the possibilities with Appinio – book a demo today!

Get a free demo and see the Appinio platform in action!

The importance of avoiding research bias cannot be overstated. Recognizing bias, implementing strategies to mitigate it, and promoting transparent and unbiased research practices are essential steps to ensure that research contributes meaningfully to advancing knowledge, informed decision-making, and the well-being of individuals and society as a whole.

Common Types of Research Bias

Research bias can manifest in various forms, each with unique characteristics and implications. Understanding these common types of research bias is essential for recognizing and mitigating their effects on your research.

Selection Bias

Selection bias occurs when the sample used in a study does not represent the target population , leading to distorted results. It can happen when certain groups are systematically more or less likely to be included in the study, introducing bias.

Causes of Selection Bias:

• Volunteer Bias: Participants self-select to participate in a study, and their motivations or characteristics differ from those who do not volunteer.
• Convenience Sampling: Researchers choose participants who are readily available but may not be representative of the broader population.
• Non-Response Bias: Occurs when a significant portion of selected participants does not respond or drops out during the study, potentially due to differing characteristics.

Mitigation Strategies:

• Random Sampling: Select participants randomly from the target population to ensure equal representation.
• Stratified Sampling: Divide the population into subgroups and sample proportionally from each subgroup.
• Use of Control Groups: Compare the study group to a control group to help account for potential selection bias.

Sampling Bias

Sampling bias arises when the individuals or items in your sample are not chosen randomly or are not representative of the broader population. It can lead to inaccurate generalizations and skewed conclusions.

Causes of Sampling Bias:

• Sampling Frame Issues: When the list or database used to select the sample is incomplete or outdated.
• Self-Selection: Participants choose to be part of the sample, introducing bias if their motivations differ from non-participants.
• Undercoverage: When certain groups are underrepresented in the sample due to difficulties in reaching or including them.
• Random Sampling: Employ random selection methods to ensure every individual or item has an equal chance of being included.
• Stratified Sampling: Divide the population into homogeneous subgroups and sample proportionally from each subgroup.
• Quota Sampling : Set quotas for specific demographics to ensure representation.

Measurement Bias

Measurement bias occurs when the methods used to collect data are inaccurate or systematically flawed, leading to incorrect conclusions. This bias can affect both quantitative and qualitative data .

Causes of Measurement Bias:

• Instrument Flaws: When the measurement tools used are inherently unreliable or imprecise.
• Data Collection Errors: Mistakes made during data collection, such as misinterpretation of responses or inconsistent recording.
• Response Bias: Participants may provide socially desirable responses , leading to measurement errors. Next to that are various types of bias that arise from the structure of the questionnaire and psychologically influence the participants' answers. We summarized those on our Instagram:

          View this post on Instagram                       A post shared by Appinio (@appinio)

• Use Reliable Instruments: Select measurement tools that have been validated and are known for their accuracy.
• Pilot Testing: Test data collection procedures to identify and address potential sources of measurement bias.
• Blinding: Keep researchers unaware of specific measurements to minimize subjectivity.

Reporting Bias

Reporting bias involves selectively reporting results that support a particular hypothesis while ignoring or downplaying contrary findings. It can lead to a skewed representation of the evidence.

Causes of Reporting Bias:

• Publication Pressure: Researchers may prioritize publishing positive or significant results, leaving negative or inconclusive findings unreported.
• Editorial Bias: Journals may preferentially accept studies with significant results, discouraging the publication of less exciting findings.
• Confirmation Bias: Researchers may unintentionally focus on, emphasize, or interpret data that aligns with their hypotheses.
• Transparent Reporting: Share all research findings, whether they support your hypotheses or not.
• Pre-Registration: Register your research design and hypotheses before data collection, reducing the temptation to selectively report.
• Peer Review: Engage in peer review to ensure a balanced and comprehensive presentation of your research.

Confirmation Bias

Confirmation bias is the tendency to seek out or interpret information in a way that confirms pre-existing beliefs or expectations. It can cloud objectivity and lead to the misinterpretation of data.

Causes of Confirmation Bias:

• Cognitive Biases: Researchers may unconsciously filter or interpret data in a way that aligns with their preconceptions.
• Selective Information Search: Researchers might seek out information that supports their hypotheses while ignoring contradictory evidence.
• Interpretation Bias: Even when presented with neutral data, researchers may interpret it to fit their expectations.
• Blinding: Keep researchers unaware of the study's hypotheses to prevent bias in data interpretation.
• Objectivity Training: Train researchers to approach research with open minds and to recognize and challenge their biases.
• Diverse Perspectives: Collaborate with colleagues with different viewpoints to reduce the impact of confirmation bias.

Publication Bias

Publication bias occurs when studies with positive or significant results are more likely to be published, skewing the overall literature. Unpublished studies with negative or null findings remain hidden.

Causes of Publication Bias:

• Journal Preferences: Journals may favor publishing studies with significant results, leading to the underrepresentation of negative or null findings.
• Researcher Publication Bias: Researchers may prioritize submitting and resubmitting studies with positive results for publication.
• Publication of Negative Results: Encourage publishing studies with negative or null findings.
• Meta-analysis: Combine results from multiple studies to assess the overall effect, considering both published and unpublished studies.
• Journal Policies: Support journals that promote balanced publication practices.

Recall Bias

Recall bias arises when participants in a study inaccurately remember or report past events or experiences. It can compromise the accuracy of historical data.

Causes of Recall Bias:

• Memory Decay: Memories naturally fade over time, making it challenging to recall distant events accurately.
• Social Desirability Bias: Participants may provide responses they believe are socially acceptable or favorable.
• Leading Questions: The phrasing of questions can influence participants' recollections.
• Use of Objective Data Sources: Whenever possible, rely on documented records, medical charts, or other objective sources of information.
• Minimize Leading Questions: Craft questions carefully to avoid suggesting specific responses.
• Consider Timing: Be aware of how the timing of data collection may affect participants' recall.

Observer Bias

Observer bias occurs when researchers' expectations or preconceived notions influence their observations and interpretations of data. It can introduce subjectivity into the research process.

Causes of Observer Bias:

• Expectation Effects: Researchers may see what they expect or want to see in their observations.
• Interpretation Biases: Researchers may interpret ambiguous data in a way that confirms their hypotheses.
• Confirmation Bias: Researchers may selectively focus on evidence that supports their expectations.
• Blinding: Keep researchers unaware of the study's hypotheses to minimize their influence on observations.
• Inter-rater Reliability: Ensure agreement among multiple observers by using consistent criteria for data collection.
• Training and Awareness: Train researchers to recognize and mitigate their biases, promoting more objective observations.

Understanding and identifying these common types of research bias is the first step toward conducting rigorous and reliable research. By implementing effective mitigation strategies and fostering a culture of transparency and objectivity, you can enhance the credibility and impact of your research. It's not just about avoiding pitfalls but also about ensuring that your findings stand up to scrutiny and contribute to the broader body of knowledge in your field.

Remember, research is a continuous journey of discovery, and the quest for unbiased, evidence-based insights is at its core. Embracing these principles will not only strengthen your research but also empower you to make more informed decisions, drive positive change, and ultimately, advance both your individual goals and the greater collective knowledge of society.

What Causes Research Bias?

Research bias can stem from various sources, and gaining a deeper understanding of these causes is vital for effectively addressing and preventing bias in your research endeavors. Let's explore these causes in detail:

Inherent Biases

Inherent biases are those that are an intrinsic part of the research process itself and can be challenging to eliminate entirely. They often result from limitations or constraints in a study's design, data collection, or analysis.

Key Characteristics:

• Inherent to Study Design : These biases are ingrained in the very design or structure of a study.
• Difficult to Eliminate: Since they are innate, completely eradicating them may not be feasible.
• Potential to Skew Findings: Inherent biases can lead to skewed or inaccurate results.

Examples of Inherent Biases:

• Sampling Bias: Due to inherent limitations in data collection methods .
• Selection Bias: As a result of constraints in participant recruitment.
• Time-Order Bias: Arising from changes over time, which may be challenging to control.

Systematic Biases

Systematic biases result from consistent errors or flaws in the research process, which can lead to predictable patterns of deviation from the truth. Unlike inherent biases, systematic biases can be addressed with deliberate efforts.

• Consistent Patterns: These biases produce recurring errors or distortions.
• Identifiable Sources: The sources of systematic biases can often be pinpointed and addressed.
• Amenable to Mitigation: Conscious adjustments can reduce or eliminate systematic biases.

Examples of Systematic Biases:

• Measurement Bias: When measurement tools are systematically flawed, leading to inaccuracies.
• Reporting Bias: Stemming from the selective reporting of results to favor certain outcomes.
• Confirmation Bias: Arising from researchers' preconceived notions or hypotheses.

Non-Systematic Biases

Non-systematic biases are random errors that can occur in the research process but are neither consistent nor predictable. They introduce variability and can affect individual data points but may not systematically impact the overall study.

• Random Occurrence: Non-systematic biases are not tied to specific patterns or sources.
• Unpredictable: They may affect one data point but not another unexpectedly.
• Potential for Random Variation: Non-systematic biases introduce noise into data.

Examples of Non-Systematic Biases:

• Sampling Error : Natural fluctuations in data points due to random chance.
• Non-Response Bias: When non-responders differ from responders randomly.

Cognitive Biases

Cognitive biases are biases rooted in human psychology and decision-making processes. They can influence how researchers perceive, interpret, and make sense of data, often unconsciously.

• Psychological Origin: Cognitive biases originate from the way our brains process information.
• Subjective Interpretation: They affect how researchers subjectively interpret data.
• Affect Decision-Making: Cognitive biases can influence researchers' decisions throughout the research process.

Examples of Cognitive Biases:

• Confirmation Bias: Seeking information that confirms pre-existing beliefs.
• Anchoring Bias: Relying too heavily on the first piece of information encountered.
• Hindsight Bias: Seeing events as having been predictable after they've occurred.

Understanding these causes of research bias is crucial for researchers at all stages of their work. It enables you to identify potential sources of bias, take proactive measures to minimize bias and foster a research environment that prioritizes objectivity and rigor. By acknowledging the inherent biases in research, recognizing systematic and non-systematic biases, and being aware of the cognitive biases that can affect decision-making, you can conduct more reliable and credible research.

How to Detect Research Bias?

Detecting research bias is a crucial step in maintaining the integrity of your study and ensuring the reliability of your findings. Let's explore some effective methods and techniques for identifying bias in your research.

Data Analysis Techniques

Utilizing appropriate data analysis techniques is crucial in detecting and addressing research bias. Here are some strategies to consider:

• Statistical Analysis : Employ rigorous statistical methods to examine the data. Look for anomalies, inconsistencies, or patterns that may indicate bias, such as skewed distributions or unexpected correlations.
• Sensitivity Analysis: Conduct sensitivity analyses by varying key parameters or assumptions in your analysis. This helps assess the robustness of your results and identifies whether bias may be influencing your findings.
• Subgroup Analysis: If your study involves diverse groups or populations, perform subgroup analyses to explore whether bias may be affecting specific subsets differently.

Peer Review

Peer review is a fundamental process for evaluating research and identifying potential bias. Here's how it can assist in detecting bias:

• External Evaluation: Involve independent experts in your field who can objectively assess your research methods , data, and interpretations. They may identify overlooked sources of bias or offer suggestions for improvement.
• Bias Assessment: Ask peer reviewers specifically to scrutinize your study for any signs of bias. Encourage them to assess the transparency of your methods and reporting.
• Replicability: Peer reviewers can also assess the replicability of your study, ensuring that others can reproduce your findings independently.

Cross-Validation

Cross-validation is a technique that involves comparing the results of your research with external or independent sources to identify potential bias:

• External Data Sources: Compare your findings with data from external sources, such as government statistics, industry reports , or previous research. Significant disparities may signal bias.
• Expert Consultation: Seek feedback from experts who are not directly involved in your research. Their impartial perspectives can help identify any biases in your study design, data collection, or interpretation.
• Historical Comparisons: If applicable, compare your current research with historical data to assess whether trends or patterns have changed over time, which could indicate bias.

By employing these methods and techniques, you can proactively detect and address research bias, ultimately enhancing the credibility and reliability of your research findings.

How to Avoid Research Bias?

Effectively avoiding research bias is a fundamental aspect of conducting high-quality research. Implementing specific strategies can help researchers minimize the impact of bias and enhance the validity and reliability of their findings. Let's delve into these strategies in detail:

1. Randomization

Randomization is a method used to allocate participants or data points to different groups or conditions in an entirely random way. It helps ensure that each participant has an equal chance of being assigned to any group, reducing the potential for bias in group assignments.

Key Aspects:

• Random Assignment: Randomly assigning participants to experimental or control groups .
• Equal Opportunity: Ensuring every participant has an equal likelihood of being in any group.
• Minimizing Bias: Reduces the risk of selection bias by distributing potential biases equally across groups.
• Balanced Groups: Randomization creates comparable groups in terms of potential confounding variables.
• Minimizes Selection Bias: Eliminates researcher or participant biases in group allocation.
• Enhanced Causality: Strengthens the ability to make causal inferences from research findings.
• Simple Randomization: Assign participants or data points to groups using a random number generator or drawing lots.
• Stratified Randomization: Divide the population into subgroups based on relevant characteristics (e.g., age, gender) and then randomly assign within those subgroups.
• Blocked Randomization: Create blocks of participants, ensuring each block contains an equal number from each group.

In a clinical trial testing a new drug, researchers use simple randomization to allocate participants into two groups: one receiving the new drug and the other receiving a placebo. This helps ensure that patient characteristics, such as age or gender, do not systematically favor one group over another, minimizing bias in the study's results.

2. Blinding and Double-Blinding

Blinding involves keeping either the participants or the researchers (single-blinding) or both (double-blinding) unaware of certain aspects of the study, such as group assignments or treatment conditions. This prevents the introduction of bias due to expectations or knowledge of the study's hypotheses.

• Single-Blinding: Either participants or researchers are unaware of crucial information.
• Double-Blinding: Both participants and researchers are unaware of crucial information.
• Placebo Control: Often used in pharmaceutical research to ensure blinding.
• Minimizes Observer Bias: Researchers' expectations do not influence data collection or interpretation.
• Prevents Participant Bias: Participants' awareness of their group assignment does not affect their behavior or responses.
• Enhances Study Validity: Blinding reduces the risk of bias influencing study outcomes.
• Use of Placebos: In clinical trials, a placebo group is often included to maintain blinding.
• Blinding Procedures: Establish protocols to ensure that those who need to be blinded are kept unaware of relevant information.
• Blinding Verification: Conduct assessments to confirm that blinding has been maintained throughout the study.

In a double-blind drug trial, neither the participants nor the researchers know whether they are receiving or administering the experimental drug or a placebo. This prevents biases in reporting and evaluating the drug's effects, ensuring that results are objective and reliable.

3. Standardization of Procedures

Standardization involves creating and following consistent, well-defined procedures throughout a study. This ensures that data collection, measurements, and interventions are carried out uniformly, minimizing potential sources of bias.

• Detailed Protocols: Developing clear and precise protocols for data collection or interventions.
• Consistency: Ensuring that all research personnel adhere to the established procedures.
• Reducing Variability: Minimizing variation in how processes are carried out.
• Increased Reliability: Standardized procedures lead to more consistent and reliable data.
• Minimized Measurement Bias: Reduces the likelihood of measurement errors or inconsistencies.
• Easier Replication: Standardization facilitates replication by providing a clear roadmap for future studies.
• Protocol Development: Create detailed step-by-step protocols for data collection, interventions, or experiments.
• Training: Train all research personnel thoroughly on standardized procedures.
• Quality Control: Implement quality control measures to monitor and ensure adherence to protocols.

In a psychological study, researchers standardize the procedure for administering a cognitive test to all participants. They use the same test materials, instructions, and environmental conditions for every participant to ensure that the data collected are not influenced by variations in how the test is administered.

4. Sample Size Considerations

Sample size considerations involve determining the appropriate number of participants or data points needed for a study. Inadequate sample sizes can lead to underpowered studies that fail to detect meaningful effects, while excessively large samples can be resource-intensive without adding substantial value.

• Power Analysis: Calculating the required sample size based on expected effect sizes and desired statistical power.
• Effect Size Considerations: Ensuring the sample size is sufficient to detect the effect size of interest.
• Resource Constraints: Balancing the need for a larger sample with available resources.
• Improved Statistical Validity: Adequate sample sizes increase the likelihood of detecting actual effects.
• Generalizability: Larger samples enhance the generalizability of study findings to the target population.
• Resource Efficiency: Avoiding extensive samples conserves resources.
• Power Analysis Software: Use statistical software to perform power analyses.
• Pilot Studies: Conduct pilot studies to estimate effect sizes and inform sample size calculations.
• Consider Practical Constraints: Factor in time, budget, and other practical limitations when determining sample sizes.

In a medical research study evaluating the efficacy of a new treatment, researchers conduct a power analysis to determine the required sample size. This analysis considers the expected effect size, desired level of statistical power, and available resources to ensure that the study can reliably detect the treatment's effects.

5. Replication

Replication involves conducting the same study or experiment multiple times to assess the consistency and reliability of the findings. Replication is a critical step in research, as it helps validate the results and ensures that they are not due to chance or bias.

• Exact or Conceptual Replication: Replicating the study with the same methods (exact) or similar methods addressing the same research question (conceptual).
• Independent Replication: Replication by different research teams or in other settings.
• Enhanced Confidence: Replication builds confidence in the robustness of research findings.
• Enhanced Reliability: Replicated findings are more reliable and less likely to be influenced by bias.
• Verification of Results: Replication verifies the validity of initial study results.
• Error Detection: Identifies potential sources of bias or errors in the original study.
• Plan for Replication: Include replication as part of the research design from the outset.
• Collaboration: Collaborate with other researchers or research teams to conduct independent replications.
• Transparent Reporting: Clearly document replication methods and results for transparency.

A psychology study that originally reported a significant effect of a particular intervention on memory performance is replicated by another research team using the same methods and procedures. If the replication study also finds a significant impact, it provides additional support for the initial findings and reduces the likelihood of bias influencing the results.

6. Transparent Reporting

Transparent reporting involves thoroughly documenting all aspects of a research study, from its design and methodology to its results and conclusions. Transparent reporting ensures that other researchers can assess the study's validity and detect any potential sources of bias.

• Comprehensive Documentation: Detailed reporting of study design, procedures, data collection, and analysis.
• Open Access to Data: Sharing data and materials to allow for independent verification and analysis.
• Disclosure of Conflicts: Transparent reporting includes disclosing any potential conflicts of interest that could introduce bias.
• Accountability: Transparent reporting holds researchers accountable for their methods and results.
• Enhanced Credibility: Transparent research is more credible and less likely to be influenced by bias.
• Reproducibility: Other researchers can replicate and verify study findings.
• Use of Reporting Guidelines: Follow established reporting guidelines specific to your field (e.g., CONSORT for clinical trials, STROBE for observational studies).
• Data Sharing: Make research data and materials available to others through data repositories or supplementary materials.
• Peer Review: Engage in peer review to ensure clear and comprehensive reporting.

A scientific journal article reporting the results of a research study includes detailed descriptions of the study design, methods, statistical analyses, and potential limitations. The authors also provide access to the raw data and materials used in the study, allowing other researchers to assess the study's validity and potential bias. This transparent reporting enhances the credibility of the research.

Real-World Examples of Research Bias

To better understand the pervasive nature of research bias and its implications, let's delve into additional real-world examples that illustrate various types of research bias beyond those previously discussed.

Pharmaceutical Industry Influence on Clinical Trials

Bias Type: Funding Bias, Sponsorship Bias

Example: The pharmaceutical industry often sponsors clinical trials to evaluate the safety and efficacy of new drugs. In some cases, studies sponsored by pharmaceutical companies have been found to report more favorable outcomes for their products compared to independently funded research.

Explanation: Funding bias occurs when the financial interests of the sponsor influence study design, data collection, and reporting. In these instances, there may be pressure to emphasize positive results or downplay adverse effects to promote the marketability of the drug.

Impact: This bias can have severe consequences for patient safety and public health, as it can lead to the approval and widespread use of drugs that may not be as effective or safe as initially reported.

Social Desirability Bias in Self-reported Surveys

Bias Type: Response Bias

Example: Researchers conducting surveys on sensitive topics such as drug use, sexual behavior, or income levels often encounter social desirability bias. Respondents may provide answers they believe are socially acceptable or desirable rather than accurate.

Explanation: Social desirability bias is rooted in the tendency to present oneself in a favorable light. Respondents may underreport stigmatized behaviors or overreport socially acceptable ones, leading to inaccurate data.

Impact: This bias can compromise the validity of survey research, especially in areas where honest reporting is crucial for public health interventions or policy decisions.

Non-Publication of Negative Clinical Trials

Bias Type: Publication Bias

Example: Clinical trials with negative or null results are less likely to be published than those with positive findings. This leads to an overrepresentation of studies showing treatment efficacy and an underrepresentation of trials indicating no effect.

Explanation: Publication bias occurs because journals often preferentially accept studies with significant results, while researchers and sponsors may be less motivated to publish negative findings. This skews the evidence base and can result in the overuse of specific treatments or interventions.

Impact: Patients and healthcare providers may make decisions based on incomplete or biased information, potentially exposing patients to ineffective or even harmful treatments.

Gender Bias in Medical Research

Bias Type: Gender Bias

Example: Historically, medical research has been biased toward male subjects, leading to a limited understanding of how diseases and treatments affect women. Clinical trials and studies often fail to include a representative number of female participants.

Explanation: Gender bias in research arises from the misconception that results from male subjects can be generalized to females. This bias can lead to treatments and medications that are less effective or safe for women.

Impact: Addressing gender bias is crucial for developing healthcare solutions that account for the distinct biological and physiological differences between genders and ensuring equitable access to effective treatments.

Political Bias in Climate Change Research

Bias Type: Confirmation Bias, Political Bias

Example: In climate change research, political bias can influence the framing, interpretation, and reporting of findings. Researchers aligned with certain political ideologies may downplay or exaggerate the significance of climate change based on their preconceptions.

Explanation: Confirmation bias comes into play when researchers seek data or interpretations that align with their political beliefs. This can result in less objective research and more susceptible to accusations of bias.

Impact: Political bias can undermine public trust in scientific research, impede policy-making, and hinder efforts to address critical issues such as climate change.

These diverse examples of research bias highlight the need for robust safeguards, transparency, and peer review in the research process. Recognizing and addressing bias is essential for maintaining the integrity of scientific inquiry and ensuring that research findings can be trusted and applied effectively.

Conclusion for Research Bias

Understanding and addressing research bias is critical in conducting reliable and trustworthy research. By recognizing the various types of bias, whether they are inherent, systematic, non-systematic, or cognitive, you can take proactive measures to minimize their impact. Strategies like randomization, blinding, standardization, and transparent reporting offer powerful tools to enhance the validity of your research.

Moreover, real-world examples highlight the tangible consequences of research bias and emphasize the importance of conducting research with integrity. Whether you're in the world of science, healthcare, marketing, or any other field, the pursuit of unbiased research is essential for making informed decisions that drive success. So, keep these insights in mind as you embark on your next research journey, and remember that a commitment to objectivity will always lead to better, more reliable outcomes.

How to Prevent Bias in Research?

Are you tired of lengthy, expensive, and potentially biased research processes? Appinio , the real-time market research platform, is here to revolutionize how you gather consumer insights. Say goodbye to research bias and hello to rapid, data-driven decision-making.

Here's why you should choose Appinio:

• Fast and Precise: Get from questions to insights in minutes, ensuring you have the most up-to-date information for your decisions.
• User-Friendly: No need for a PhD in research – our platform is so intuitive that anyone can use it to gather valuable insights.
• Global Reach: Survey your target audience from over 90 countries and define precise target groups from a pool of 1200+ characteristics.

Get free access to the platform!

Join the loop 💌

Be the first to hear about new updates, product news, and data insights. We'll send it all straight to your inbox.

Get the latest market research news straight to your inbox! 💌

Wait, there's more

16.05.2024 | 30min read

Time Series Analysis: Definition, Types, Techniques, Examples

14.05.2024 | 30min read

Experimental Research: Definition, Types, Design, Examples

07.05.2024 | 29min read

Interval Scale: Definition, Characteristics, Examples

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research Research Tools and Apps

Research bias: What it is, Types & Examples

The researcher sometimes unintentionally or actively affects the process while executing a systematic inquiry. It is known as research bias, and it can affect your results just like any other sort of bias.

When it comes to studying bias, there are no hard and fast guidelines, which simply means that it can occur at any time. Experimental mistakes and a lack of concern for all relevant factors can lead to research bias.

One of the most common causes of study results with low credibility is study bias. Because of its informal nature, you must be cautious when characterizing bias in research. To reduce or prevent its occurrence, you need to be able to recognize its characteristics. 

This article will cover what it is, its type, and how to avoid it.

Content Index

What is research bias?

How does research bias affect the research process, types of research bias with examples, how questionpro helps in reducing bias in a research process.

Research bias is a technique in which the researchers conducting the experiment modify the findings to present a specific consequence. It is often known as experimenter bias.

Bias is a characteristic of the research technique that makes it rely on experience and judgment rather than data analysis. The most important thing to know about bias is that it is unavoidable in many fields. Understanding research bias and reducing the effects of biased views is an essential part of any research planning process.

For example, it is much easier to become attracted to a certain point of view when using social research subjects, compromising fairness.

Research bias can majorly affect the research process, weakening its integrity and leading to misleading or erroneous results. Here are some examples of how this bias might affect the research process:

Distorted research design

When bias is present, study results can be skewed or wrong. It can make the study less trustworthy and valid. If bias affects how a study is set up, how data is collected, or how it is analyzed, it can cause systematic mistakes that move the results away from the true or unbiased values.

Invalid conclusions

It can make it hard to believe that the findings of a study are correct. Biased research can lead to unjustified or wrong claims because the results may not reflect reality or give a complete picture of the research question.

Misleading interpretations

Bias can lead to inaccurate interpretations of research findings. It can alter the overall comprehension of the research issue. Researchers may be tempted to interpret the findings in a way that confirms their previous assumptions or expectations, ignoring alternate explanations or contradictory evidence.

Ethical concerns

This bias poses ethical considerations. It can have negative effects on individuals, groups, or society as a whole. Biased research can misinform decision-making processes, leading to ineffective interventions, policies, or therapies.

Damaged credibility

Research bias undermines scientific credibility. Biased research can damage public trust in science. It may reduce reliance on scientific evidence for decision-making.

Bias can be seen in practically every aspect of quantitative research and qualitative research , and it can come from both the survey developer and the participants. The sorts of biases that come directly from the survey maker are the easiest to deal with out of all the types of bias in research. Let’s look at some of the most typical research biases.

Design bias

Design bias happens when a researcher fails to capture biased views in most experiments. It has something to do with the organization and its research methods. The researcher must demonstrate that they realize this and have tried to mitigate its influence.

Another design bias develops after the research is completed and the results are analyzed. It occurs when the researchers’ original concerns are not reflected in the exposure, which is all too often these days.

For example, a researcher working on a survey containing questions concerning health benefits may overlook the researcher’s awareness of the sample group’s limitations. It’s possible that the group tested was all male or all over a particular age.

Selection bias or sampling bias

Selection bias occurs when volunteers are chosen to represent your research population, but those with different experiences are ignored. 

In research, selection bias manifests itself in a variety of ways. When the sampling method puts preference into the research, this is known as sampling bias . Selection bias is also referred to as sampling bias.

For example, research on a disease that depended heavily on white male volunteers cannot be generalized to the full community, including women and people of other races or communities.

Procedural bias

Procedural bias is a sort of research bias that occurs when survey respondents are given insufficient time to complete surveys. As a result, participants are forced to submit half-thoughts with misinformation, which does not accurately reflect their thinking.

Another sort of study bias is using individuals who are forced to participate, as they are more likely to complete the survey fast, leaving them with enough time to accomplish other things.

For Example, If you ask your employees to survey their break, they may be pressured, which may compromise the validity of their results.

Publication or reporting bias

A sort of bias that influences research is publication bias. It is also known as reporting bias. It refers to a condition in which favorable outcomes are more likely to be reported than negative or empty ones. Analysis bias can also make it easier for reporting bias to happen.

The publication standards for research articles in a specific area frequently reflect this bias on them. Researchers sometimes choose not to disclose their outcomes if they believe the data do not reflect their theory.

As an example, there was seven research on the antidepressant drug Reboxetine. Among them, only one got published, and the others were unpublished.

Measurement of data collecting bias

A defect in the data collection process and measuring technique causes measurement bias. Data collecting bias is also known as measurement bias. It occurs in both qualitative and quantitative research methodologies. 

Data collection methods might occur in quantitative research when you use an approach that is not appropriate for your research population. Instrument bias is one of the most common forms of measurement bias in quantitative investigations. A defective scale would generate instrument bias and invalidate the experimental process in a quantitative experiment.

For example, you may ask those who do not have internet access to survey by email or on your website.

Data collection bias occurs in qualitative research when inappropriate survey questions are asked during an unstructured interview. Bad survey questions are those that lead the interviewee to make presumptions. Subjects are frequently hesitant to provide socially incorrect responses for fear of criticism.

For example, a topic can avoid coming across as homophobic or racist in an interview.

Some more types of bias in research include the ones listed here. Researchers must understand these biases and reduce them through rigorous study design, transparent reporting, and critical evidence review: 

• Confirmation bias: Researchers often search for, evaluate, and prioritize material that supports their existing hypotheses or expectations, ignoring contradictory data. This can lead to a skewed perception of results and perhaps biased conclusions.
• Cultural bias: Cultural bias arises when cultural norms, attitudes, or preconceptions influence the research process and the interpretation of results.
• Funding bias: Funding bias takes place when powerful motives support research. It can bias research design, data collecting, analysis, and interpretation toward the funding source.
• Observer bias: Observer bias arises when the researcher or observer affects participants’ replies or behavior. Collecting data might be biased by accidental clues, expectations, or subjective interpretations.

LEARN ABOUT: Theoretical Research

QuestionPro offers several features and functionalities that can contribute to reducing bias in the research process. Here’s how QuestionPro can help:

Randomization

QuestionPro allows researchers to randomize the order of survey questions or response alternatives. Randomization helps to remove order effects and limit bias from the order in which participants encounter the items.

Branching and skip logic

Branching and skip logic capabilities in QuestionPro allow researchers to design customized survey pathways based on participants’ responses. It enables tailored questioning, ensuring that only pertinent questions are asked of participants. Bias generated by such inquiries is reduced by avoiding irrelevant or needless questions.

Diverse question types

QuestionPro supports a wide range of questions kinds, including multiple-choice, Likert scale, matrix, and open-ended questions. Researchers can choose the most relevant question kinds to get unbiased data while avoiding leading or suggestive questions that may affect participants’ responses.

Anonymous responses

QuestionPro enables researchers to collect anonymous responses, protecting the confidentiality of participants. It can encourage participants to provide more unbiased and equitable feedback, especially when dealing with sensitive or contentious issues.

Data analysis and reporting

QuestionPro has powerful data analysis and reporting options, such as charts, graphs, and statistical analysis tools. These properties allow researchers to examine and interpret obtained data objectively, decreasing the role of bias in interpreting results.

Collaboration and peer review

QuestionPro supports peer review and researcher collaboration. It helps uncover and overcome biases in research planning, questionnaire formulation, and data analysis by involving several researchers and soliciting external opinions.

You must comprehend biases in research and how to deal with them. Knowing the different sorts of biases in research allows you to readily identify them. It is also necessary to have a clear idea to recognize it in any form.

QuestionPro provides many research tools and settings that can assist you in dealing with research bias. Try QuestionPro today to undertake your original bias-free quantitative or qualitative research.

FREE TRIAL         LEARN MORE

Frequently Asking Questions

Research bias affects the validity and dependability of your research’s findings, resulting in inaccurate interpretations of the data and incorrect conclusions.

Bias should be avoided in research to ensure that findings are accurate, valid, and objective.

 To avoid research bias, researchers should take proactive steps throughout the research process, such as developing a clear research question and objectives, designing a rigorous study, following standardized protocols, and so on.

MORE LIKE THIS

Data Information vs Insight: Essential differences

May 14, 2024

Pricing Analytics Software: Optimize Your Pricing Strategy

May 13, 2024

Relationship Marketing: What It Is, Examples & Top 7 Benefits

May 8, 2024

The Best Email Survey Tool to Boost Your Feedback Game

May 7, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

• How it works

What is Research Bias - Types & Examples

Research is crucial in generating knowledge and understanding the world around us. However, the validity and reliability of research findings can be compromised by various factors, including bias in research. This comprehensive guide will explain the different examples and types of research bias. But before that, let’s look into the research bias definition.

What is Research Bias?

Research bias refers to the systematic errors or deviations from the truth that can occur during the research process, leading to inaccurate or misleading results. It arises from flaws in the research design , data collection , analysis, and interpretation, which can distort the findings and conclusions. Bias in research can occur at any stage of the research process and may be unintentional or deliberate. Recognising and addressing research bias is crucial for maintaining the integrity and credibility of scientific research.

Example of Bias in Research

Suppose a researcher wants to investigate the relationship between coffee consumption and heart disease risk. They recruit participants for their study and ask them to self-report their coffee intake through a questionnaire. Bias can occur in this scenario due to self-reporting bias, where participants may provide inaccurate or biased information about their coffee consumption.

For example, health-conscious individuals might underreport their coffee intake because they perceive it as unhealthy, while coffee enthusiasts might overreport their consumption due to their positive attitude towards coffee.

Types of Research Bias

There are many different types of research bias. Some of them are discussed below.

Information Bias

Publication bias, interviewer bias, response bias, researcher bias, selection bias, cognitive bias.

Information bias is also known as measurement bias. It refers to a type of research bias that occurs when there are errors or distortions in gathering, interpreting, or reporting information in a research study or any other form of data collection.

Example of Information Bias In Research

Let's say you are studying the effectiveness of a new weight loss program. You recruit participants and ask them to keep a daily food diary to track their caloric intake. However, the participants know that they are being monitored and may alter their eating habits, consciously or unconsciously, to present a more favourable image of themselves.

In this case, the participants' awareness of being observed can lead to information bias in research. They might underreport their consumption of high-calorie foods or overreport their consumption of healthy foods, skewing the data collected. This research bias could make the weight loss program appear more effective than it actually is because the reported dietary intake doesn't accurately reflect the participants' true behaviour.

Types of Information Bias In Research

Information bias can manifest in different ways, such as:

1. Measurement Bias

Measurement Bias occurs when the measurement instruments or techniques used to collect data are flawed or inaccurate. For example, if a survey question is poorly worded or ambiguous, it may generate biased responses or misinterpretations of the respondents' answers.

2. Recall Bias

Recall bias arises when participants in a study inaccurately remember or recall past events, experiences, or behaviours. It can happen due to various factors, such as selective memory, social desirability bias, or the passage of time. Recall bias causes distorted or unreliable data.

3. Reporting Bias

Reporting bias occurs when there is selective or incomplete reporting of study findings. It can happen if researchers or organisations only publish or publicise results that support their preconceived notions or desired outcomes while omitting or downplaying contradictory or unfavourable findings. Reporting bias can lead to a skewed perception of the true state of knowledge in a particular field.

4. Publication Bias

Publication bias refers to the tendency of researchers, journals, or other publishing entities to publish studies with statistically significant or positive results preferentially. Studies with null or negative findings are often less likely to be published, leading to an overrepresentation of positive results in the literature and potentially distorting the overall understanding of a research topic.

5. Language Bias

This bias can transpire if research is conducted and reported in a specific language, leading to limited accessibility and potential exclusion of relevant studies or data published in other languages. Language bias can introduce distortions in systematic reviews, meta-analyses, or other forms of evidence synthesis.

Publication bias occurs due to the systematic tendency of scientific journals and researchers to preferentially publish studies with positive or significant results while overlooking or rejecting studies with negative or non-significant findings. It transpires when the decision to publish a study is influenced by the nature or direction of its results rather than its methodological rigour or scientific merit.

Publication bias in research can arise due to various factors, such as researchers' and journals' preferences for novel or groundbreaking findings, the pressure to present positive results to secure funding or advance academic careers, and the tendency of studies with positive results to generate more attention and citations. This research bias can distort the overall body of scientific literature, leading to an overrepresentation of studies with positive outcomes and an underrepresentation of studies with negative or inconclusive findings.

Example of Publication Bias In Research

Let's say a pharmaceutical company conducts a clinical trial to test the effectiveness of a new drug for treating a certain medical condition. The company conducts several trials but only submits the results of the trials that show positive outcomes that states that the drug is effective to scientific journals for publication, as the negative results may lead to rejection in funding.

Interviewer bias means the potential for bias or prejudice to influence the outcome of an interview. It happens when the interviewer's personal beliefs, preferences, stereotypes, or prejudices affect their evaluation of the interviewee's qualifications, skills, or suitability for a position.

Example of Interviewer Bias In Research

Imagine there is an interviewer named James conducting interviews for a sales position in a company. During one interview, a candidate named Aisha, who is a woman, showcases exceptional knowledge about the products, demonstrates excellent communication skills, and presents a strong sales track record.

However, James thinks women are generally less assertive or aggressive in sales roles than men. Due to this stereotype bias in research, James may subconsciously underestimate Aisha's abilities or question her suitability for the position, despite her impressive qualifications.

Types of Interviewer Bias In Research

The main types of interviewer bias are:

1. Stereotyping

Stereotyping refers to holding preconceived notions or stereotypes about certain groups of people based on their race, gender, age, religion, or other characteristics. These biases can lead to unfair judgments or assumptions about the interviewee's abilities.

2. Confirmation Bias

In confirmation bias , Interviewers may subconsciously seek information that confirms their pre-existing beliefs or initial impressions about the interviewee. This results in selectively noticing and emphasising certain responses or behaviours that align with their biases while disregarding contradictory evidence.

3. Similarity Bias

Similarity bias means unconsciously favouring candidates with similar backgrounds, experiences, or characteristics, resulting in a preference for more relatable or familiar candidates. This leads to overlooking qualified candidates from diverse backgrounds.

4. Halo and Horns Effect

The halo effect occurs when an interviewer forms an overall positive impression of a candidate based on one favourable characteristic, leading to a bias in favour of that candidate. Conversely, the horns effect occurs when a negative impression of a candidate's single attribute influences the overall evaluation, resulting in a bias against the candidate.

5. Contrast Effect

The contrast effect leads to evaluating candidates relative to each other rather than based on objective criteria, leading to biased judgments. If the previous candidate was exceptionally strong or weak, the current candidate might be evaluated more harshly or leniently.

6. Implicit Bias

Interviewers may have unconscious biases influencing their perceptions and decision-making. Societal stereotypes often form these biases and can affect evaluations and decisions without the interviewer's conscious awareness.

Response bias arises from a systematic error or distortion in how individuals respond to survey questions or provide information in research studies. It occurs when respondents consistently tend to answer questions inaccurately or in a particular direction, leading to a skewed or biased dataset.

Example of Response Bias In Research

You conduct a survey asking people about their exercise habits and distribute the survey to a group of individuals. You ask them to report the number of times they exercise per week. However, some respondents may feel pressured to provide answers they believe are more socially acceptable. They might overstate their exercise frequency to present themselves as more active and health-conscious. This would result in an overestimation of exercise habits in the data.

Types of Response Bias In Research

We have discussed a few common types of response bias below. Other major types include courtesy bias and extreme responding.

1. Social Desirability Bias

This occurs when respondents provide answers that they perceive to be more socially acceptable or desirable than their true beliefs or behaviours. They may modify their responses to conform to societal norms or present themselves favourably.

2. Acquiescence Bias

Also known as "yea-saying" or "nay-saying," Acquiescence bias in research is the tendency of respondents to agree or disagree with statements or questions without carefully considering their content. Some individuals are predisposed to consistently agree (acquiesce) or disagree with items, leading to skewed responses.

3. Non-Response Bias

This bias emerges when individuals who choose not to participate in a study or survey have different characteristics or opinions compared to those who do participate.

Researcher bias, also known as experimenter bias or investigator bias, refers to the influence or distortion of research findings or interpretations due to the personal beliefs, preferences, or expectations of the researcher conducting the study. It occurs when the researcher's subjective biases or preconceived notions unconsciously affect the research process, leading to flawed or biased results.

Example of Researcher Bias In Research

Assume that a researcher is conducting a study on the effectiveness of a new teaching method for improving student performance in mathematics. The researcher strongly believes the new teaching method will significantly enhance students' mathematical abilities.

To test the method, the researcher divides students into two groups: the control group, which receives traditional teaching methods, and the experimental group, which receives the new teaching method.

During the study, the researcher spends more time interacting with the experimental group, providing additional support and encouragement. They unintentionally convey their enthusiasm for the new teaching method to the students in the experimental group while giving a different level of attention or encouragement to the control group.

When the post-test results come in, the experimental group shows a statistically significant improvement in mathematical performance compared to the control group. Influenced by their initial beliefs and unintentional differential treatment, the researcher concludes that the new teaching method is highly effective in enhancing students' mathematical abilities.

Hire an Expert Writer

Proposal and research paper orders completed by our expert writers are

• Formally drafted in academic style
• Plagiarism free
• 100% Confidential
• Never Resold
• Include unlimited free revisions
• Completed to match exact client requirements

Selection bias refers to a systematic error or distortion that occurs in a research study when the participants or subjects included in the study are not representative of the target population. This research bias arises when the process of selecting participants for the study is flawed or biased in some way, leading to a sample that does not accurately reflect the characteristics of the broader population.

Example of Selection Bias In Research

Suppose a research team wants to evaluate the weight loss program's effectiveness and recruits participants by placing an advertisement in a fitness magazine. The advertisement attracts health-conscious individuals who are actively seeking ways to lose weight. As a result, the study sample primarily consists of highly motivated individuals to lose weight and may have already tried other weight loss methods.

The sample is biased towards individuals more likely to succeed in weight loss due to their pre-existing motivation and experience.

Types of Selection Bias In Research

Selection bias can occur in various forms and impact both observational and experimental studies. Some common types of selection bias include:

1. Non-Response Bias

This occurs when individuals chosen for the study do not participate or respond, leading to a sample that differs from the target population. Non-response bias can introduce bias in research if those who choose not to participate have different characteristics from those who do participate.

2. Volunteer Bias

Volunteer bias happens when participants self-select or volunteer to participate in a study. This can lead to a sample not representative of the broader population because volunteers may have different characteristics, motivations, or experiences compared to those who do not volunteer.

3. Healthy User Bias

This research bias can occur in studies that examine the effects of a particular intervention or treatment. It arises when participants who follow a certain lifestyle or treatment regimen are healthier or have better health outcomes than the general population, leading to overestimating the treatment's effectiveness.

4. Berkson's Bias

This research bias occurs in hospital-based studies where patients are selected based on hospital admission. Since hospital-based studies typically exclude healthy individuals, the sample may consist of patients with multiple conditions or diseases, leading to an artificial association between certain variables.

5. Survivorship Bias

Survivorship bias happens when the sample includes only individuals or entities that have survived a particular process or undergone a specific experience. This bias can lead to an inaccurate understanding of the entire population since it neglects those who did not survive or dropped out.

A cognitive bias refers to systematic patterns of deviation from rational judgment or decision-making processes, often influenced by subjective factors and unconscious mental processes. These research biases can affect how we interpret information, judge, and form beliefs. Cognitive biases can be thought of as shortcuts or mental filters that our brains use to simplify complex information processing.

Example of Cognitive Bias In Research

Assume that you are investigating the effects of a new drug on a particular medical condition. Due to prior experiences or personal beliefs, the researcher has a positive view of the drug's effectiveness. During the research process, the researcher may unconsciously focus on collecting and analysing data that supports their preconceived notion of the drug's efficacy. They may pay less attention to data that suggests the drug has limited or no impact.

Types of Cognitive Bias In Research

Some of the most common types of cognitive bias are discussed below.

1. Confirmation Bias

The tendency to seek, interpret, or remember information in a way that confirms one's existing beliefs or hypotheses while disregarding or downplaying contradictory evidence.

2. Availability Heuristic

This research bias occurs when you overestimate the importance or likelihood of events based on how easily they come to mind or how vividly they are remembered.

3. Anchoring Bias

Relying too heavily on the first piece of information encountered (the " anchor ) when making decisions or estimations, even if it is irrelevant or misleading.

4. Halo Effect

The halo effect happens when you generalise positive or negative impressions of a person, company, or brand based on a single characteristic or initial experience.

5. Overconfidence Effect

The tendency to overestimate one's abilities, knowledge, or the accuracy of one's beliefs and predictions.

6. Bandwagon Effect

Preferencing to adopt certain beliefs or behaviours because others are doing so, often without critical evaluation or independent thinking.

7. Framing Effect

The framing effect refers to how the information presented or "framed" can influence decision-making, emphasising the potential gains or losses, leading to different choices even when the options are objectively the same.

How to Avoid Research Bias?

Avoiding research bias is crucial for maintaining the integrity and validity of your research findings. Here are some strategies on how to minimise research bias:

• Formulate a clear and specific research question that outlines the objective of your study. This will help you stay focused and reduce the chances of introducing research bias.
• Perform a thorough literature review on your topic before starting your research. This will help you understand the current state of knowledge and identify potential biases or gaps in the existing research.
• Use randomisation and blinding techniques to ensure that participants or samples are assigned to groups unbiasedly. Blinding techniques, such as single-blind or double-blind procedures, can be used to prevent bias in data collection and analysis.
• Ensure that your sample is representative of the target population by using random or stratified sampling methods . Avoid selecting participants based on convenience, as it can introduce selection bias.
• Consider using random invitations or incentives to encourage a diverse range of participants.
• Clearly define and document the methods and procedures used for data collection to ensure consistency. This includes using standardised measurement tools, following specific protocols, and training research assistants to minimise variability and observer bias.
• Researchers can unintentionally introduce bias through preconceived notions, beliefs, or expectations. Be conscious of your biases and regularly reflect on how they influence your research process and interpretation of results.
• Relying on a single source can introduce bias. Triangulate your findings by using multiple methods ( quantitative and qualitative ) and collecting data from diverse sources to ensure a more comprehensive and balanced perspective.
• Use appropriate statistical techniques and avoid cherry-picking results that support your hypothesis. Be transparent about the limitations and uncertainties in your findings.

Frequently Asked Questions

What is bias in research.

Bias in research refers to systematic errors or preferences that can distort the results or conclusions of a study, leading to inaccuracies or unfairness due to factors such as sampling, measurement, or interpretation.

What causes bias in research?

Bias in research can be caused by various factors, such as the selection of participants, flawed study design, inadequate sampling methods, researcher's own beliefs or preferences, funding sources, publication bias, or the omission or manipulation of data.

How to avoid bias in research?

To avoid research bias, use random and representative sampling, blinding techniques, pre-registering hypotheses, conducting rigorous peer review, disclosing conflicts of interest, and promoting transparency in data collection and analysis.

How to address bias in research?

You can critically examine your biases, use diverse and inclusive samples, employ appropriate statistical methods, conduct robust sensitivity analyses, encourage replication studies, and engage in open dialogue about potential biases in your findings.

You May Also Like

Are you new to English or just want to revise your grammar skills? We have gathered the basic language rules that every person should know in English.

Unfamiliar with what plagiarism is? Learn the different types of plagiarism and how to avoid them in our comprehensive plagiarism guide.

Applying to a university and looking for a guide to write your UCAS personal statement? We have covered all aspects of the UCAS statement to help you get to your dream university.

More Interesting Articles

USEFUL LINKS

LEARNING RESOURCES

COMPANY DETAILS

• How It Works

The complete guide to selection bias

As the old saying goes, knowledge is power. But the quest for knowledge isn’t always as easy as we’d like it to be.

Researchers often find themselves delving into datasets, dissecting information, and uncovering insights that can shape entire fields of study. However, amongst the excitement of new discovery lies a subtle yet complicated obstacle: selection bias.

This phenomenon has the potential to warp conclusions, skew perceptions, and cast doubt on the integrity of your findings. But what is it, and how can you avoid it in your own research? Let’s take a look.

What is selection bias?

Also known as the selection effect, selection bias occurs when a sample used in a study isn’t completely representative of the population of interest or is sub-optimal for answering the specific research question. This could be introduced through different sampling methods or the way the participants were selected. Or it could just come down to the particular area of interest being researched.

This bias then distorts the results of the study, undermining its value and rendering it untrustworthy.

There are a variety of different types of bias, each bringing their own implications. These are:

1. Sampling bias

Sampling bias occurs when certain members of the population of interest have a higher or lower chance of being selected than others. When this happens, the research won’t give a very representative point of view.

2. Survivorship bias

Survivorship bias is when only successful subjects are included in the final analysis, leading to a skewed outcome. This is often seen in studies of successful people or companies, where failures are taken out of the equation.

3. Self-selection bias

Self-selection bias is where people nominate themselves to be part of a study, leading to a non-random sample of participants. This is often prevalent in surveys or online polls, where the people who take part may not represent the population as a whole.

4. Information bias

Information bias happens when there are systematic errors in the measurement or collection of data. This makes the outcomes just as unreliable.

5. Non-response bias

There will also be people who refuse to take part in or drop out of a study. It’s likely that there will be some kind of underlying commonality in these participants. For example, they might be male, or under the age of 20.

Examples of selection bias

To understand the more practical implications that selection bias can have on a study, let’s take a look at some real-life scenarios.

1. Clinical trials

Clinical trials can see a significant impact from selection bias, largely due to self-selection bias. This can impact the effectiveness of the drug being trialed. For example, where younger people are more likely to take part in clinical trials, the results may only show the impact in that age group. So, older people aren’t represented.

2. Job recruitment

A common selection bias example is recruitment. Hiring processes can also fall prey to selection bias. If your company relies on employee referrals as one of your main recruitment methods, this can mean that individuals from different backgrounds or networks are excluded from the process entirely. This may result in a workforce that lacks diversity and unique thinking.

3. Economic studies

These can be vulnerable to survivorship bias. Especially in studies of successful businesses or investment strategies. For example, if a study only examines companies that have achieved significant growth, it may overlook the failures and challenges faced by less successful enterprises. 

Survivorship bias distorts perceptions of risk and reward. This can lead to flawed investment decisions.

4. Educational research

In educational studies, selection bias can distort assessments of teaching interventions or educational programs. For example, you might want to look at the effectiveness of a tutoring program. But if you only include students from affluent areas, the findings will likely be irrelevant to students from disadvantaged backgrounds.

The impact of selection bias

Why is selection bias so concerning? Because it can result in misleading conclusions and send researchers down the wrong path. And when this happens, outcomes don’t align with reality.

The ramifications of selection bias extend beyond just the statistics. They can also result in wasted resources. Valuable time, money, and manpower are used up on research that doesn’t reflect the population of interest. This can stop progress and have negative or unfair impacts on certain groups of people.

It can also lead people to lose trust in science. When research comes across as biased or unfair, it makes people doubt if science can really help us. This can have hugely negative impacts on society as a whole.

How to avoid selection bias in your own research

To get accurate results and draw meaningful conclusions, you need to conduct research that's fair and minimizes bias. But how?

Here are some simple yet effective strategies to ensure you conduct research with integrity and impartiality:

1. Define your population

Clearly define the population you want to study and make sure you understand who should be included and excluded from your research. Let's say you want to gauge people's understanding of a new financial services product. You would define your population of interest (people who would use that product), then ensure your sample reflects that population.

2. Random sampling

Use random sampling to select participants from your population. Imagine you're conducting a survey on public opinion about a controversial social issue. Instead of selecting participants based on convenience or availability, use random sampling to ensure that every member of the population of interest has an equal chance of being included in your study. This helps reduce the risk of bias and ensures that your findings are representative of the population as a whole.

3. Stratified sampling

If your population has different subgroups, you can use  stratified sampling to ensure representation from each group. By sampling randomly within the strata, you can capture the diversity of your population more accurately. This avoids biases introduced by over or underrepresentation of certain groups.

4. Minimize exclusions

Try to reduce as many exclusions as possible from your study unless absolutely necessary. Excluding certain groups or individuals can introduce bias and limit the effectiveness of your findings. 

5. Transparent reporting

Be transparent about your selection process in your research reports. Clearly document how participants were selected, as well as any criteria used for exclusion. This information gives people a clear insight into your methodology. And it helps readers to build trust with your findings.

6. Consider alternatives

Explore alternative methods of data collection or sampling if traditional methods introduce bias. For example, if you're carrying out a study on consumer preferences for a new product, consider using a combination of online surveys and focus groups to reach a diverse range of participants. This approach helps reduce bias by relying solely on one method of data collection and ensures that your findings are robust and reliable.

7. Consult experts

Seek input from colleagues or experts in your field to review your research design and selection process. Fresh perspectives can help identify potential sources of bias that may have been overlooked, enhancing the credibility of your research and ensuring you offer diverse viewpoints and methodologies.

How Prolific can help

At Prolific, flexibility and control are right at the heart of everything we do. With our pool of 120,000+ active participants, all fully vetted and verified, you can rely on us to deliver definitive and varied data sets, no matter what your research topic is.

Sign up to Prolific today to gather balanced, representative samples for your research.

You might also like

High-quality human data to deliver world-leading research and AIs.

Follow us on

All Rights Reserved Prolific 2024

May 4, 2024

Implicit Bias Hurts Everyone. Here’s How to Overcome It

The environment shapes stereotypes and biases, but it is possible to recognize and change them

By Corey S. Powell & OpenMind Magazine

fotostorm/Getty Images

We all have a natural tendency to view the world in black and white—to the extent that it's hard not to hear "black" and immediately think "white." Fortunately, there are ways to activate the more subtle shadings in our minds. Kristin Pauker is a professor of psychology at the University of Hawaiʻi at Mānoa who studies stereotyping and prejudice, with a focus on how our environment shapes our biases. In this podcast and Q&A, she tells OpenMind co-editor Corey S. Powell how researchers measure and study bias, and how we can use their findings to make a more equitable world. (This conversation has been edited for length and clarity.)

When I hear “bias,” the first thing I think of is a conscious prejudice. But you study something a lot more subtle, which researchers call “implicit bias.” What is it, and how does it affect us?

Implicit bias is a form of bias that influences our decision-making, our interactions and our behaviors. It can be based on any social group membership, like race, gender, age, sexual orientation or even the color of your shirt. Often we’re not aware of the ways in which these biases are influencing us. Sometimes implicit bias gets called unconscious bias, which is a little bit of a misnomer. We can be aware of these biases, so it's not necessarily unconscious. But we often are not aware of the way in which they're influencing our behaviors and thoughts.

On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing . By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

You make it sound like almost anything can set us off. Why is bias so deeply ingrained in our heads?

Our brain likes to categorize things because it makes our world easier to process. We make categories as soon as we start learning about something. So we categorize fruits, we categorize vegetables, we categorize chairs, we categorize tables for their function—and we also categorize people. We know from research that categorization happens early in life, as early as 5 or 6, in some cases even 3 or 4. Categorization creates shortcuts that help us process information faster, but that also can lead us to make assumptions that may or may not hold in particular situations. What categories we use are directed by the environment that we're in. Our environment already has told us certain categories are really important, such as gender, age, race and ethnicity. We quickly form an association when we’re assigned to a particular group.

Listen to the Podcast

Kristin Pauker: We have to think about ways in which we can change the features of our environment—so that our weeds aren’t so prolific.

In your research, you use a diagnostic tool called an “ implicit association test .” How does it work, and what does it tell you?

Typically someone would show you examples of individuals who belong to categories, and then ask you to categorize those individuals. For example, you would see faces and you would categorize them as black and white. You’re asked to make a fast categorization, as fast as you can. Then you are presented with words that could be categorized as good or bad, like “hero” and “evil,” and again asked to categorize the words quickly. The complicated part happens when, say, good and white are paired together or bad and black are paired together. You're asked to categorize the faces and the words as you were before. Then it's flipped, so that bad and white are paired together, and good and black are paired together. You’re asked to make the categorizations once again with the new pairings.

The point of the test is, how quickly do you associate certain concepts together? Oftentimes if certain concepts are more closely paired in your mind, then it will be easier for you to make that association. Your response will be faster. When the pairing is less familiar to you or less closely associated, it takes you longer to respond. Additional processing needs to occur.

When you run this implicit association test on your test subjects or your students, are they often surprised by the results?

We’ve done it as a demonstration in the classroom, and I've had students come up and complain saying, “There’s something wrong with this test. I don't believe it.” They’ll try to poke all kinds of holes in the test because it gave them a score that wasn’t what they felt it should be according to what they think about themselves. This is the case, I think, for almost anyone. I've taken an implicit association test and found that I have a stronger association with men in science than women in science . And I'm a woman scientist! We can have and hold these biases because they’re prevalent in society, even if they’re biases that may not be beneficial to the group we belong to.

Studies show that even after you make people aware of their implicit biases, they can’t necessarily get rid of them. So are we stuck with our biases?

Those biases are hard to change and control, but that doesn't mean that they are un controllable and un changeable. It’s just that oftentimes there are many features in our environment that reinforce those biases. I was thinking about an analogy. Right now I’m struggling with weeds growing in my yard, invasive vines. It’s hard because there are so many things supporting the growth of these vines. I live in a place that has lots of sun and rain. Similarly, there’s so much in our environment that is supporting our biases. It’s hard to just cut them off and be like, OK, they're gone. We have to think about ways in which we can change the features of our environment—so that our weeds aren’t so prolific.

Common programs aimed at reducing bias, such as corporate diversity training workshops, often seem to stop at the stage of making people aware that bias exists. Is that why they haven’t worked very well ?

If people are told that they’re biased, the reaction that many of them have is, “Oh, that means I'm a racist? I'm not a racist!” Very defensive, because we associate this idea of being biased with a moral judgment that I'm a bad person. Because of that, awareness-raising can have the opposite of the intended effect. Being told that they're biased can make people worried and defensive, and they push back against that idea. They're not willing to accept it.

A lot of the diversity training models are based on the idea that you can just tell people about their biases and then get them to accept them and work on them. But, A, some people don't want to accept their biases. B, some people don't want to work on them. And C, the messaging around how we talk about these biases creates a misunderstanding that they can’t be changed. We talk about biases that are unconscious, biases that we all hold, that are formed early in life—it creates the idea, “Well, there’s nothing I can do, so why should I even try?”

How can we do better in talking about bias, so that people are more likely to embrace change instead of becoming defensive or defeated?

Some of it is about messaging. Biases are hard to change, but we should be discussing the ways in which these biases can change, even though it might take some time and work. You have to emphasize the idea that these things can change, or else why would we try? There is research showing that if you just give people their bias score, normally that doesn't result in them becoming more aware of their bias. But if you combine that score with a message that this is something controllable, people are less defensive and more willing to accept their biases.

What about concrete actions we can take to reduce the negative impact of implicit bias?

One thing is thinking about when we do interventions. A lot of times we’re trying to make changes in the workplace. We should be thinking more about how we're raising our children. The types of environments we're exposing them to, and the features that are in our schools , are good places to think about creating change. Prejudice is something that’s malleable.

Another thing is not always focusing on the person. So much of what we do in these interventions is try to change individual people's biases. But we can also think about our environment. What are the ways in which our environments are communicating these biases, and how can we make changes there? A clever idea people have been thinking about is trying to change consequences of biases. There's a researcher, Jason A. Okonofua , who talks about this and calls it “sidelining bias.” You're not targeting the person and trying to get rid of their biases. You're targeting the situations that support those biases. If you can change that situation and kind of cut it off, then the consequences of bias might not be as bad. It could lead to a judgment that is not so influenced by those biases.

There’s research showing that people make fairer hiring decisions when they work off tightly structured interviews and qualification checklists, which leave less room for subjective reactions. Is that the kind of “sidelining” strategy you’re talking about?

Yes, that’s been shown to be an effective way to sideline bias. If you set those criteria ahead of time, it's harder for you to shift a preference based on the person that you would like to hire. Another good example is finding ways to slow down the processes we're working on. Biases are more likely to influence our decision-making when we have to make really quick decisions or when we are stressed—which is the case for a lot of important decisions that we make.

Jennifer Eberhardt does research on these kinds of implicit biases. She worked with NextDoor (a neighborhood monitoring app) when they noticed a lot of racial profiling in the things people were reporting in their neighborhood. She worked with them to change the way that people report a suspicious person. Basically they added some extra steps to the checklist when you report something. Rather than just reporting that someone looks suspicious, a user had to indicate what about the behavior itself was suspicious. And then there was an explicit warning that they couldn't just say the reason for the suspicious behavior was someone's race. Including extra check steps slowed down the process and reduced the profiling.

It does feel like we’re making progress in addressing bias but, damn, it’s been a slow process. Where can we go from here?

A big part that’s missing in the research on implicit bias is creating tools that are useful for people. We still don’t know a lot about bias, but we know a lot more than we're willing to put into practice. For instance, creating resources for parents to be able to have conversations about bias , and to be aware that the everyday things we do are really important. This is something that many people want to tackle, but they don’t know how to do it. Just asking questions about what is usual and what is unusual has really interesting effects. We’ve done that with our son. He’d say something and I would ask, “Why is that something that only boys can do? You say girls can't do that, is that really the case? Can you think of examples where the opposite is true?”

This Q&A is part of a series of OpenMind essays, podcasts and videos supported by a generous grant from the Pulitzer Center 's Truth Decay initiative.

This story originally appeared on OpenMind , a digital magazine tackling science controversies and deceptions.

Understanding the different types of bias in research (2024 guide)

Last updated

6 October 2023

Reviewed by

Miroslav Damyanov

Research bias is an invisible force that overly highlights or dismisses the chosen study topic’s traits. When left unchecked, it can significantly impact the validity and reliability of your research.

In a perfect world, every research project would be free of any trace of bias—but for this to happen, you need to be aware of the most common types of research bias that plague studies.

Read this guide to learn more about the most common types of bias in research and what you can do to design and improve your studies to create high-quality research results.

• What is research bias?

Research bias is the tendency for qualitative and quantitative research studies to contain prejudice or preference for or against a particular group of people, culture, object, idea, belief, or circumstance.

Bias is rarely based on observed facts. In most cases, it results from societal stereotypes, systemic discrimination, or learned prejudice.

Every human develops their own set of biases throughout their lifetime as they interact with their environment. Often, people are unaware of their own biases until they are challenged—and this is why it’s easy for unintentional bias to seep into research projects .

Left unchecked, bias ruins the validity of research . So, to get the most accurate results, researchers need to know about the most common types of research bias and understand how their study design can address and avoid these outcomes.

• The two primary types of bias

Historically, there are two primary types of bias in research:

Conscious bias

Conscious bias is the practice of intentionally voicing and sharing a negative opinion about a particular group of people, beliefs, or concepts.

Characterized by negative emotions and opinions of the target group, conscious bias is often defined as intentional discrimination.

In most cases, this type of bias is not involved in research projects, as they are unjust, unfair, and unscientific.

Unconscious bias

An unconscious bias is a negative response to a particular group of people, beliefs, or concepts that is not identified or intentionally acted upon by the bias holder.

Because of this, unconscious bias is incredibly dangerous. These warped beliefs shape and impact how someone conducts themselves and their research. The trouble is that they can’t identify the moral and ethical issues with their behavior.

• Examples of commonly occurring research bias

Humans use countless biases daily to quickly process information and make sense of the world. But, to create accurate research studies and get the best results, you must remove these biases from your study design.

Here are some of the most common types of research biases you should look out for when planning your next study:

Information bias

During any study, tampering with data collection is widely agreed to be bad science. But what if your study design includes information biases you are unaware of?

Also known as measurement bias, information bias occurs when one or more of the key study variables are not correctly measured, recorded, or interpreted. As a result, the study’s perceived outcome may be inaccurate due to data misclassification, omission, or obfuscation (obscuring). 

Observer bias

Observer bias occurs when researchers don’t have a clear understanding of their own personal assumptions and expectations. During observational studies, it’s possible for a researcher’s personal biases to impact how they interpret the data. This can dramatically affect the study’s outcome.

The study should be double-blind to combat this type of bias. This is where the participants don’t know which group they are in, and the observers don’t know which group they are observing.

Regression to the mean (RTM)

Bias can also impact research statistics.

Regression of the mean (RTM) refers to a statistical bias that if a first clinical reading is extreme in value (i.e., it’s very high or very low compared to the average), the second reading will provide a more statistically normal result.

Here’s an example: you might be nervous when a doctor takes your blood pressure in the doctor’s surgery. The first result might be quite high. This is a phenomenon known as “white coat syndrome.” When your blood pressure is retaken to double-check the value, it is more likely to be closer to typical values.

So, which value is more accurate, and which should you record as the truth?

The answer depends on the specific design of your study. However, using control groups is usually recommended for studies with a high risk of RTM.

Performance bias

A performance bias can develop if participants understand the study’s nature or desired outcomes. This can harm the study’s accuracy, as participants may adjust their behavior outside of their normal to improve their performance. This results in inaccurate data and study results.

This is a common bias type in medical and health studies, particularly those studying the differences between two lifestyle choices.

To reduce performance bias, researchers should strive to keep members of the control and study groups unaware of the other group’s activities. This method is known as “blinding.”

Recall bias

How good is your memory? Chances are, it’s not as good as you think—and the older the memory, the more inaccurate and biased it will become.

A recall bias commonly occurs in self-reporting studies requiring participants to remember past information. While people can remember big-picture events (like the day they got married or landed their first job), routine occurrences like what they do after work every Tuesday are harder to recall.

To offset this type of bias, design a study that engages with participants on both short- and long-term periods to help keep the content more top of mind.

Researcher bias

Researcher bias (also known as interviewer bias) occurs due to the researcher’s personal beliefs or tendencies that influence the study’s results or outcomes.

These types of biases can be intentional or unintentional, and most are driven by personal feelings, historical stereotypes, and assumptions about the study’s outcome before it has even begun.

Question order bias

Survey design and question order is a huge area of contention for researchers. These elements are essential for quality study design and can prevent or invite answer bias.

When designing a research study that collects data via survey questions , the order of the questions presented can impact how the participants answer each subsequent question. Leading questions (questions that guide participants toward a particular answer) are perfect examples of this. When included early in the survey, they can sway a participant’s opinions and answers as they complete the questionnaire .

This is known as systematic distortion, meaning each question answered after the guiding questions is impacted or distorted by the wording of the questions before.

Demand characteristics

Body language and social cues play a significant role in human communication—and this also rings true for the validity of research projects . 

A demand characteristic bias can occur due to a verbal or non-verbal cue that encourages research participants to behave in a particular way.

Imagine a researcher is studying a group of new grad business students about their experience applying to new jobs one, three, and six months after graduation. They scowl every time a participant mentions they don’t use a cover letter. This reaction may encourage participants to change their answers, harming the study’s outcome and resulting in less accurate results.

Courtesy bias

Courtesy bias arises from not wanting to share negative or constructive feedback or answers—a common human tendency.

You’ve probably been in this situation before. Think of a time when you had a negative opinion or perspective on a topic, but you felt the need to soften or reduce the harshness of your feedback to prevent someone’s feelings from being hurt.

This type of bias also occurs in research. Without a comfortable and non-judgmental environment that encourages honest responses, courtesy bias can result in inaccurate data intake.

Studies based on small group interviews, focus groups , or any in-person surveys are particularly vulnerable to this type of bias because people are less likely to share negative opinions in front of others or to someone’s face.

Extreme responding

Extreme responding refers to the tendency for people to respond on one side of the scale or the other, even if these extreme answers don’t reflect their true opinion. 

This is a common bias in surveys, particularly online surveys asking about a person’s experience or personal opinions (think questionnaires that ask you to decide if you strongly disagree, disagree, agree, or strongly agree with a statement).

When this occurs, the data will be skewed. It will be overly positive or negative—not accurate. This is a problem because the data can impact future decisions or study outcomes.

Writing different styles of questions and asking for follow-up interviews with a small group of participants are a few options for reducing the impact of this type of bias.

Social desirability bias

Everyone wants to be liked and respected. As a result, societal bias can impact survey answers.

It’s common for people to answer questions in a way that they believe will earn them favor, respect, or agreement with researchers. This is a common bias type for studies on taboo or sensitive topics like alcohol consumption or physical activity levels, where participants feel vulnerable or judged when sharing their honest answers.

Finding ways to comfort participants with ensured anonymity and safe and respectful research practices are ways you can offset the impact of social desirability bias.

Selection bias

For the most accurate results, researchers need to understand their chosen population before accepting participants. Failure to do this results in selection bias, which is caused by an inaccurate or misrepresented selection of participants that don’t truly reflect the chosen population.

Self-selection bias

To collect data, researchers in many studies require participants to volunteer their time and experiences. This results in a study design that is automatically biased toward people who are more likely to get involved.

People who are more likely to voluntarily participate in a study are not reflective of the common experience of a broad, diverse population. Because of this, any information collected from this type of study will contain a self-selection bias .

To avoid this type of bias, researchers can use random assignment (using control versus treatment groups to divide the study participants after they volunteer).

Sampling or ascertainment bias

When choosing participants for a study, take care to select people who are representative of the overall population being researched. Failure to do this will result in sampling bias.

For example, if researchers aim to learn more about how university stress impacts sleep quality but only choose engineering students as participants, the study won’t reflect the wider population they want to learn more about.

To avoid sampling bias, researchers must first have a strong understanding of their chosen study population. Then, they should take steps to ensure that any person within that population has an equal chance of being selected for the study.

Attrition bias

People tend to be hard on themselves, so an attrition bias toward the impact of failure versus success can seep into research.

Many people find it easier to list things they struggle with rather than things they think they are good at. This also occurs in research, as people are more likely to value the impact of a negative experience (or failure) than that of a positive, successful outcome.

Survivorship bias

In medical clinical trials and studies, a survivorship bias may develop if only the results and data from participants who survived the trial are studied. Survivorship bias also includes participants who were unable to complete the entire trial, not just those who passed away during the duration of the study.

In long-term studies that evaluate new medications or therapies for high-mortality diseases like aggressive cancers, choosing to only consider the success rate, side effects, or experiences of those who completed the study eliminates a large portion of important information. This disregarded information may have offered insights into the quality, efficacy, and safety of the treatment being tested.

Nonresponse bias

A nonresponse bias occurs when a portion of chosen participants decide not to complete or participate in the study. This is a common issue in survey-based research (especially online surveys).

In survey-based research, the issue of response versus nonresponse rates can impact the quality of the information collected. Every nonresponse is a missed opportunity to get a better understanding of the chosen population, whether participants choose not to reply based on subject apathy, shame, guilt, or a lack of skills or resources.

To combat this bias, improve response rates using multiple different survey styles. These might include in-person interviews, mailed paper surveys, and virtual options. However, note that these efforts will never completely remove nonresponse bias from your study.

Cognitive bias

Cognitive biases result from repeated errors in thinking or memory caused by misinterpreting information, oversimplifying a situation, or making inaccurate mental shortcuts. They can be tricky to identify and account for, as everyone lives with invisible cognitive biases that govern how they understand and interact with their surrounding environment.

Anchoring bias

When given a list of information, humans have a tendency to overemphasize (or anchor onto) the first thing mentioned.

For example, if you ask people to remember a grocery list of items that starts with apples, bananas, yogurt, and bread, people are most likely to remember apples over any of the other ingredients. This is because apples were mentioned first, despite not being any more important than the other items listed.

This habit conflates the importance and significance of this one piece of information, which can impact how you respond to or feel about the other equally important concepts being mentioned.

Halo effect

The halo effect explains the tendency for people to form opinions or assumptions about other people based on one specific characteristic. Most commonly seen in studies about physical appearance and attractiveness, the halo effect can cause either a positive or negative response depending on how the defined trait is perceived.

Framing effect

Framing effect bias refers to how you perceive information based on how it’s presented to you. 

To demonstrate this, decide which of the following desserts sounds more delicious.

“Made with 95% natural ingredients!”

“Contains only 5% non-natural ingredients!”

Both of these claims say the same thing, but most people have a framing effect bias toward the first claim as it’s positive and more impactful.

This type of bias can significantly impact how people perceive or react to data and information.

The misinformation effect

The misinformation effect refers to the brain’s tendency to alter or misremember past experiences when it has since been fed inaccurate information. This type of bias can significantly impact how a person feels about, remembers, or trusts the authority of their previous experiences.

Confirmation bias

Confirmation bias occurs when someone unconsciously prefers or favors information that confirms or validates their beliefs and ideas.

In some cases, confirmation bias is so strong that people find themselves disregarding information that counters their worldview, resulting in poorer research accuracy and quality.

We all like being proven right (even if we are testing a research hypothesis ), so this is a commonly occurring cognitive bias that needs to be addressed during any scientific study.

Availability heuristic

All humans contextualize and understand the world around them based on their past experiences and memories. Because of this, people tend to have an availability bias toward explanations they have heard before. 

People are more likely to assume or gravitate toward reasoning and ideas that align with past experience. This is known as the availability heuristic . Information and connections that are more available or accessible in your memory might seem more likely than other alternatives. This can impact the validity of research efforts.

• How to avoid bias in your research

Research is a compelling, complex, and essential part of human growth and learning, but collecting the most accurate data possible also poses plenty of challenges.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 11 January 2024

Last updated: 15 January 2024

Last updated: 17 January 2024

Last updated: 12 May 2023

Last updated: 30 April 2024

Last updated: 18 May 2023

Last updated: 25 November 2023

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

Users report unexpectedly high data usage, especially during streaming sessions.

Users find it hard to navigate from the home page to relevant playlists in the app.

It would be great to have a sleep timer feature, especially for bedtime listening.

I need better filters to find the songs or artists I’m looking for.

Log in or sign up

Get started for free

• Open access
• Published: 10 May 2024

An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies

• Yixin Zhao 1   na1 ,
• Bilal Khan 1   na1 &
• Zelalem F. Negeri 1  

BMC Medical Research Methodology volume  24 , Article number:  111 ( 2024 ) Cite this article

193 Accesses

9 Altmetric

Metrics details

A Generalized Linear Mixed Model (GLMM) is recommended to meta-analyze diagnostic test accuracy studies (DTAs) based on aggregate or individual participant data. Since a GLMM does not have a closed-form likelihood function or parameter solutions, computational methods are conventionally used to approximate the likelihoods and obtain parameter estimates. The most commonly used computational methods are the Iteratively Reweighted Least Squares (IRLS), the Laplace approximation (LA), and the Adaptive Gauss-Hermite quadrature (AGHQ). Despite being widely used, it has not been clear how these computational methods compare and perform in the context of an aggregate data meta-analysis (ADMA) of DTAs.

We compared and evaluated the performance of three commonly used computational methods for GLMM - the IRLS, the LA, and the AGHQ, via a comprehensive simulation study and real-life data examples, in the context of an ADMA of DTAs. By varying several parameters in our simulations, we assessed the performance of the three methods in terms of bias, root mean squared error, confidence interval (CI) width, coverage of the 95% CI, convergence rate, and computational speed.

For most of the scenarios, especially when the meta-analytic data were not sparse (i.e., there were no or negligible studies with perfect diagnosis), the three computational methods were comparable for the estimation of sensitivity and specificity. However, the LA had the largest bias and root mean squared error for pooled sensitivity and specificity when the meta-analytic data were sparse. Moreover, the AGHQ took a longer computational time to converge relative to the other two methods, although it had the best convergence rate.

Conclusions

We recommend practitioners and researchers carefully choose an appropriate computational algorithm when fitting a GLMM to an ADMA of DTAs. We do not recommend the LA for sparse meta-analytic data sets. However, either the AGHQ or the IRLS can be used regardless of the characteristics of the meta-analytic data.

Peer Review reports

Meta-analysis is a statistical technique used in research to combine and analyze the results of multiple independent studies on a particular topic or research question [ 1 ]. A meta-analysis of diagnostic test accuracy (DTA) is a specific type of meta-analysis that focuses on combining and analyzing data from multiple studies assessing the performance of diagnostic tests, allowing for synthesizing diagnostic test characteristics, such as sensitivity (Se) and specificity (Sp) across multiple independent studies [ 2 , 3 ]. In an aggregate data meta-analysis (ADMA) of DTAs, one gathers information on the true positive (TP), true negative (TN), false positive (FP), and false negative (FN) results for a specific diagnostic test across various studies. From these data, the study-specific observed Se, Sp, and other relevant measures of diagnostic accuracy can be calculated. By pooling the results from multiple studies, researchers aim to derive summary estimates of these test characteristics, while considering the variability and potential biases present in the individual studies.

Researchers and practitioners usually use generalized linear mixed models (GLMM) such as the bivariate random-effects model of Chu and Cole [ 4 ] to meta-analyze DTA data and obtain the maximum likelihood estimates (MLEs) of the model parameters. However, unlike the linear mixed model version of Reitsma et al. (2005) [ 5 ], since Chu and Cole’s GLMM does not have a closed-form solution for the log-likelihood due to the complex random effects variance components, one needs to use numerical methods to approximate the log-likelihood function and obtain MLEs of the model parameters. Commonly used computational methods in the context of an ADMA of DTAs include the Adaptive Gaussian Hermite quadrature (AGHQ) [ 6 ], the Laplace approximation (LA) [ 6 ], and the iteratively re-weighted least squares (IRLS) [ 7 , 8 ].

There have been some attempts at comparing and evaluating some of these numerical methods in different contexts. Ju et al. (2020) [ 9 ] compared the AGHQ, LA and the penalized quasi-likelihood (PQL) for meta-analyzing sparse binary data, and found that the AGHQ and PQL did not show improved performance compared to the LA. However, Ju et al. did not take the IRLS into account, and compared the numerical methods only in terms of the pooled odds ratio but not concerning the between-study variance and covariance. Additionally, their study was focused on a meta-analysis of sparse binary intervention studies outcomes, not on DTA data. Thomas, Platt & Benedetti [ 10 ] studied the performances of the PQL and AGHQ algorithm for meta-analysis of binary outcomes in the context of an individual participant data meta-analysis (IPDMA) of intervention studies. They found that there were no appreciable differences between the two computational methods. However, Thomas et al. did not consider the LA and meta-analysis of DTAs.

However, to the best of our knowledge, there was no evidence in the literature that describes the performance of these widely used computational algorithms for GLMM in the context of either IPDMA or ADMA of DTAs, partly because DTA meta-analysis is a relatively newer area of research compared to the widely studied meta-analysis of intervention studies. Additionally, since diagnosis precedes intervention, it is crucial to establish the accuracy of diagnostic tests using sound statistical methods or algorithms to minimize misdiagnosis of patients due to flawed evidence. Moreover, since meta-analytic methods for intervention or treatment studies cannot be used to meta-analyze DTA data because of differences in data characteristics and model assumptions [ 11 ], establishing evidence on the performance of computational methods for ADMA of DTA studies is needed. Therefore, this paper aims to fill this important research gap by comparing and evaluating the AGHQ, IRLS, and LA performances for GLMM to meta-analyze DTAs using aggregate data. We will compare the numerical methods using an extensive simulation study in terms of absolute bias, root mean squared error (RMSE), coverage probability, 95% confidence interval (CI) width, convergence rate, and computational speed. We will also illustrate the methods using real-life meta-analytic data.

The rest of this article is organized as follows. Motivating examples  section presents motivating examples using two real-life data, Methods  section introduces the statistical methods, including the GLMM model, the numerical algorithms and a simulation study. In Simulation study results  section, we discuss our simulation study results, and in Illustrative examples  section, we illustrate the computational methods using the motivating examples data. We conclude the manuscript with a discussion and concluding remarks in Discussion and Conclusions  sections.

Motivating examples

This Section describes two real-life data sets (see Appendix Tables A 1 and A 2 ) to motivate the statistical methods we present in Methods section.

First, consider an article by Vonasek et al. (2021) [ 12 ], which studied the accuracy of screening tests (e.g., visually identifying early signs and symptoms) for active pulmonary tuberculosis in children. Figure  1 depicts the forest plots of the sensitivity and specificity measurements.

Forest plots of sensitivity (left) and specificity (right) of the meta-analysis from Vonasek et al. (2021) [ 12 ]. The a and b in Schwoebel 2020 denote the two distinct screening tests, “One or more of cough, fever, or poor weight gain in tuberculosis contacts” and “One or more of cough, fever, or decreased playfulness in children aged under five years, inpatient or outpatient,” respectively, utilized in the study

The meta-analysis of Vonasek et al. [ 12 ] included 19 studies with no indication of sparsity in either Se or Sp; that is, none of the included primary studies had observed Se or Sp close to 0 or 1. The average number of diseased ( \(n_1\) ) and non-diseased ( \(n_2\) ) participants were about 99 and 11,058, respectively, where the average \(n_2\) was affected by four potentially outlier studies whose respective number of non-diseased participants were 1,903 [ 13 ], 1,903 [ 13 ], 1,336 [ 14 ], and 200,580 [ 15 ]. In Illustrative examples  section, we will demonstrate how the three computational algorithms deal with the data since the existence of such outlying studies may potentially distort the meta-analysis results.

In the second example, we present the study by Jullien et al. (2020) that studied the diagnosing characteristics of “Rapid diagnostic tests for plague” [ 16 ]. As can be seen from the forest plots presented in Fig.  2 , this meta-analysis contained only nine studies and the average number of diseased and non-diseased participants were 188 and 223, respectively, with no indication of potentially outlying studies.

Forest plots of sensitivity (left) and specificity (right) of the meta-analysis from Jullien et al. (2020) [ 16 ]

However, the second meta-analysis had some sparse data, particularly in the diseased group. There were 4/9 (44%) primary studies with 100% sensitivity (i.e., with \(FN=0\) ). Thus, we will revisit this data set in Illustrative examples  section to examine how the numerical methods perform in the context of sparse DTAs.

In this Section, we describe the commonly used conventional meta-analytic model for ADMA of DTAs, the three computational methods used to estimate the parameters of this model and methods for our simulation study.

The standard model

The bivariate binomial-normal (BBN) model is a bivariate random-effects model first developed by Chu and Cole [ 4 ]. The BBN model assumes the binomial distribution for modelling the within-study variability and the bivariate normal distribution for modelling the between-study variability in Se and Sp across studies. The BBN is generally accepted as the preferred model for ADMA of DTAs because it models the within-study variability using the exact Binomial distribution, instead of approximating it with the normal distribution, and it does not require an ad hoc continuity correction when any of the four cell frequencies in a DTA contain zero counts. If we let \(\textbf{y}_i = [\text {logit}(Se_i), \text {logit}(Sp_i)]'\) denote the study-specific logit-transformed sensitivity and specificity vector, \(\textbf{b}_i\) the study-specific random-effects, \(\varvec{\mu }\) the pooled sensitivity and specificity vector, and \(\varvec{\Sigma }\) the between-study heterogeneity parameter, the marginal likelihood function of the BBN model can be given as in equation 1 . However, since this likelihood does not have closed-form expression because the integral cannot be evaluated analytically in a closed-form [ 4 ], one needs to use numerical approximation methods to estimate the likelihood.

where \(i=1,...,k\) denotes the i -th study in the meta-analysis.

The AGHQ [ 6 ] is a numerical method used to approximate log-likelihoods by numerical integration to obtain the MLEs of model parameters. Although estimation becomes more precise as the number of quadrature points increases, it often gives rise to computational difficulties for high-dimension random effects and convergence problems where variances are close to zero or cluster sizes are small [ 6 ]. Most of the time, the AGHQ [ 6 ] is the default estimation method and is regarded as the most accurate. Nonetheless, the LA [ 6 ] which is the Gauss-Hermite quadrature of order one [ 17 ] and the IRLS [ 7 , 8 ] that aims to find the solution to a weighted least squares iteratively, can also be used to find MLEs and usually have lower computational difficulties and faster computational speed.

Simulation study design

Data simulation.

To compare the three computational methods for each combination of model parameter settings, we simulated data based on each simulation scenario and fitted the BBN model using the AGHQ, LA, and IRLS algorithms. To inform our simulations, we scraped the Cochrane Database of Systematic Reviews and selected 64 reviews containing meta-analyses data. Unwrapping these reviews and performing data cleaning gave us access to 393 meta-analyses covering a wide range of medical diagnosis tests. We fitted the BBN model to each of the 393 meta-analyses to obtain the empirical distribution of the model parameters. Based on these results, we defined our true parameter settings as shown in Table 1 . Following Ju et al. (2020) [ 9 ] and Jackson et al. (2018) [ 18 ], we introduced sparsity into the meta-analysis by considering large values of ( Se ,  Sp ).

Accordingly, we considered a total of \(3^4\times 4 = 324\) total scenarios in our simulation study. For each parameter combination, we conducted our simulation study by (1) simulating 1000 DTA data based on normal random effects following the steps described by Negeri and Beyene [ 19 ], (2) fitting the BBN model to each simulated data using the three computational methods, and (3) comparing the estimated results by each numerical method with the true values in terms of absolute bias, RMSE, CI width, coverage probability, convergence rate, and computing time.

We used the R statistical language [ 20 ] version 4.2.2 and RStudio [ 21 ] version 2023.09.0+463 for all data analyses. We utilized the glmer() function from the lme4 R package [ 22 ] to apply the IRLS and LA by setting nAGQ to 0 and 1, respectively. We fitted the BBN model with the AGHQ algorithm using the mixed_model() function from the GLMMadaptive R package [ 23 ] by setting the number of quadrature points used in the approximation (nAGQ) to 5.

Performance evaluation criteria

In our simulation study, we defined the convergence rate of the BBN model as the number of converged fits over the total number of fits in an iteration. We counted fits with non-positive semi-definite covariance matrices and fits that did not meet optimality conditions as non-converging. While assessing the convergence rate, we found that the “converged” message provided in the model summary from the glmer() function is sometimes non-trustable. For example, we saw a warning message such as “boundary (singular) fit: see help(’isSingular’)” when fitting the BBN model, which indicates a fit that did not converge, but the “converged” option wrongly provided convergence. Thus, we treated those singular fits as non-convergence to calculate the convergence rate. We measured the computing speed for each numerical method using R ’s built-in function system.time() . The remaining metrics, such as the absolute bias, RMSE, coverage probability, and CI width were calculated following Burton et al. (2006) [ 24 ] and Morris et al. (2019) [ 25 ].

Simulation study results

In this Section, we use the different metrics described in Methods  section to evaluate the performance of the three computational methods and summarize our simulation study findings by metrics. Note that the solid line is IRLS, the dashed line is LA, the dotted line is AGHQ, and that the lines might overlap for some scenarios when there is no difference in results between the three computational methods.

Absolute bias

Figure 3 depicts the bias of the three computational methods for sensitivity and specificity. We found that when the true Se and Sp were far from perfect, there was barely any difference among these three numerical methods as the three lines overlap for the first two columns. However, for all variance-covariance settings, the LA had the largest absolute bias compared to the AGHQ and the IRLS (Fig.  3 , third pane). Moreover, when data is sparse (i.e. large Se and Sp closer to 100%), the IRLS and AGHQ were comparable, although IRLS had a slightly larger absolute bias. We observed consistent results for the other scenarios considered in our simulations (see the Appendix figures).

Bias for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Similarly, the three computational methods had comparable performance when it comes to the bias of the between-study variances \(\sigma _{1}^2\) and \(\sigma _{2}^2\) for relatively small Se and Sp (Fig.  4 , first two panes). However, for sparse DTA data (large Se and Sp), the LA still had the largest absolute bias, and the AGHQ had the smallest bias for between-study variances. Similar results were found for the other scenarios examined in our simulations (see the Appendix figures).

Bias for between-study variances based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Root mean squared error (RMSE)

Concerning RMSE (Fig.  5 ), we observed a similar trend to bias. That is, the three numerical methods were comparable when the DTA data was not sparse, but the LA yielded larger RMSE for all (Se, Sp) pairs. Furthermore, the IRLS and the AGHQ were comparable, although the AGHQ had a slightly larger RMSE. Consistent results were observed for the other scenarios considered in our simulations (see the Appendix figures).

RMSE for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Confidence interval (CI) width and coverage

For CI width (Fig.  6 ), the three numerical methods gave almost the same results when the true Se and Sp were small. However, there were marginal differences among the computational methods when DTA was sparse, as the IRLS had the smallest CI width for specificity and the LA yielded the smallest CI width for sensitivity. Moreover, as Se or Sp increased, the width of the CI decreased.

CI width for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Figure  7 presents the coverage probabilities of the three computational methods. Similar to the other metrics, the AGHQ, LA, and IRLS had comparable coverage probability when data were not sparse (i.e., small Se and Sp). However, the LA had the smallest coverage probability for sparse DTA data compared to the other two methods, and the AGHQ had a slightly larger coverage than the IRLS. Moreover, as the number of studies in a meta-analysis increased, the coverage probability of the methods decreased. We found similar results for the other simulation scenarios considered in our simulations (see the Appendix figures).

Coverage for sensitivity (Se) and specificity (Sp) based on the IRLS (solid line), Laplace approximation (dashed line) and Gauss-Hermite quadrature (dotted line) when \(\sigma _1^2=1.59\) , \(\sigma _2^2=1.83\) , \(\sigma _{12}=-0.34\) , \(n_1=300\) , and \(n_2=500\)

Convergence rate and computing time

Table 2 depicts the average convergence rate, average computing time, and the interquartile range (IQR) for computing time across all simulation scenarios for the three computational methods. Accordingly, on average, the AGHQ had the highest convergence rate but the longest computing time compared to the two methods. We also observed that longer computing times were associated with higher convergence rates. Moreover, the AGHQ also had the largest IQR of the three numerical methods.

Illustrative examples

This Section summarizes the results of fitting the BBN model to the two motivating examples presented in Motivating examples  section using the three computational algorithms.

Table 3 summarizes the results of applying the numerical algorithms to the Vonasek et al. (2021) [ 12 ] data. All three numerical algorithms converged to the MLEs. The AGHQ estimated both the pooled Se and pooled Sp very differently than the other two methods. The LA and IRLS approaches resulted in similar pooled Se and pooled Sp estimates, with their pooled Sp closer to the observed specificities of the outlying studies identified in Motivating examples  section than the non-outlying studies, indicating that the LA and IRLS estimates may be influenced by outlying studies [ 2 , 3 ]. These results suggest that the AGHQ yielded estimates that were less affected by the outlying studies in specificity. However, all three methods yielded comparable between-study variance-covariance estimates.

We present the results of fitting the BBN model to the meta-analysis of Jullien et al. (2020) [ 16 ] in Table 4 . The AGHQ algorithm failed to converge with its Hessian matrix being non-positive-definite. Despite that, all three methods produced comparable pooled Se and Sp estimates, \(\sigma _{12}\) and \(\sigma _2^2\) . However, the LA produced a very large between-study variance of logit-transformed sensitivity \((\sigma _1^2)\) , which could be attributed to the apparent data sparsity among the diseased participants, consistent with our simulation study results.

In this study, we compared three commonly used computational algorithms, the AGHQ, the LA, and the IRLS, that numerically approximate the log-likelihood function of a bivariate GLMM for ADMA of DTAs. To determine which method is more appropriate in practice, we compared the performance of these methods using extensive simulation studies and real-life data sets. Our simulation settings were informed after analyzing 393 real-life meta-analyses from the Cochrane Database of Systematic Reviews.

In almost all of our simulation scenarios, we observed no obvious difference among the three numerical methods when Se and Sp were relatively small and not close to 100%. However, when the DTA data were sparse or equivalently when Se and Sp were both large and close to 100%, there were appreciable differences among these three computational algorithms. The LA usually had the largest absolute bias and RMSE but the smallest coverage probability for Se and Sp compared to the IRLS and the AGHQ. The IRLS and AGHQ were comparable, but IRLS had the smallest convergence rate. Though the AGHQ had the largest convergence rate among the three algorithms, it had the longest computing time.

Unlike the results reported by Ju et al. (2020) [ 9 ] for meta-analysis of rare intervention studies, we found appreciable differences in bias and RMSE of the LA and the AGHQ for sparse data, albeit in the context of ADMA of DTAs. However, we were not able to make similar comparisons in terms of the between-study variances since it wasn’t reported in their study. Similarly, a comparison was impossible between our findings and those of Thomas et al. (2017) [ 10 ] since the latter study evaluated only the AGHQ, not the LA and IRLS algorithms.

Our real-life data analyses also revealed consistent results with our simulation studies. The AGHQ produced robust pooled Se and Sp estimates when applied to DTA data with a few outlying studies. The LA yielded the largest between-study variance estimates when a GLMM was fitted to sparse DTA data. Although the PQL approach has been discouraged by other researchers in the context of intervention studies meta-analysis with binary outcomes [ 9 ] and is not commonly used in the context of meta-analysis of DTA studies, following a Reviewer’s suggestion, we applied it to our motivating examples data sets (see Appendix Table C 3 ) and observed inferior results consistent with that of Ju et al. [ 9 ]. Thus, we opted not to investigate its performance in our simulation study. Moreover, it was not unexpected to find the LA and IRLS algorithms affected by outliers since they utilize methods known to be prone to unusual observations – the normal distribution and least squares, respectively. Whereas the LA works by approximating the integrand of the likelihood with the normal distribution, for example, the IRLS iteratively solves a system of score equations via weighted least squares. The AGHQ approximates the entire likelihood or integral via a numerical approach known as quadrature method, making it the least sensitive approach to outliers.

The strengths of our manuscript include being the first study to report on the evaluation and comparison of commonly used computational methods for ADMA of DTAs and considering several real-life scenarios by informing our simulation study with 393 meta-analysis results from the Cochrane Database of Systematic Reviews. Thus, our study has contributed to the literature by filling an existing gap in the body of knowledge and by producing results applicable to practical real-world situations. Although we considered only the frequently used numerical methods in ADMA of DTAs, not including more than three such computational algorithms can be considered a limitation of our study, which can be pursued in a future study. For example, it is worth evaluating and validating the performance of these numerical methods in comparison with the Newton-Raphson-based algorithms [ 26 ], the many procedures implemented in the metadta Stata tool [ 27 ], or in the context of IPDMA of DTA studies with or without multiple cut-offs [ 28 ]. Moreover, the LA and IRLS algorithms appeared to be impacted by outlying studies when applied to a real-life meta-analysis. Thus, it is worth a future study investigating this issue further via a simulation study to see if this property of the two algorithms repeats for different data settings.

In summary, the IRLS, AGHQ, and the LA had similar performances for non-sparse data, but the LA performed worse for sparse DTA data sets. Whereas the AGHQ had the best convergence rate but the longest computing time, the IRLS had the shortest computing time but the worst convergence rate. Therefore, we suggest practitioners and researchers use any of the three computational methods for conducting ADMA of DTAs without sparse data. However, the LA should be avoided and either the IRLS or the AGHQ should be used when sparsity is a concern.

Availability of data and materials

All data generated or analyzed during this study will be included in this published article and its supplementary information files.

Abbreviations

Aggregate Data Meta-Analysis

Adaptive Gaussian-Hermite Quadrature

Bivariate Binomial-Normal

Confidence Interval

Diagnostic Test Accuracy

Generalized Linear Mixed Models

Individual Participant Data Meta-Analysis

Interquartile Range

Iteratively Reweighted Least Squares

Laplace Approximation

Penalized Quasi-likelihood

Root Mean Squared Error

Sensitivity

Specificity

Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3–8.

Article   Google Scholar  

Negeri ZF, Beyene J. Statistical methods for detecting outlying and influential studies in meta-analysis of diagnostic test accuracy studies. Stat Methods Med Res. 2020;29(4):1227–42.

Article   PubMed   Google Scholar  

Negeri ZF, Beyene J. Robust bivariate random-effects model for accommodating outlying and influential studies in meta-analysis of diagnostic test accuracy studies. Stat Methods Med Res. 2020;29(11):3308–25.

Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59(12):1331–2.

Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982–90.

Pinheiro JC, Bates DM. Approximations to the log-likelihood function in the nonlinear mixed-effects model. J Comput Graph Stat. 1995;4(1):12–35.

Jorgensen M. Iteratively reweighted least squares. Encycl Environmetrics. 2006.

Burrus CS. Iterative reweighted least squares. 2012;12. OpenStax CNX Available online: http://cnx.org/contents/92b90377-2b34-49e4-b26f-7fe572db78a1 . Accessed 15 Nov 2023.

Ju K, Lin L, Chu H, Cheng LL, Xu C. Laplace approximation, penalized quasi-likelihood, and adaptive Gauss-Hermite quadrature for generalized linear mixed models: Towards meta-analysis of binary outcome with sparse data. BMC Med Res Methodol. 2020;20(1):1–11.

Thomas D, Platt R, Benedetti A. A comparison of analytic approaches for individual patient data meta-analyses with binary outcomes. BMC Med Res Methodol. 2017;17:1–12.

Rücker G. Network Meta-Analysis of Diagnostic Test Accuracy Studies. In: Biondi-Zoccai G, editor. Diagnostic Meta-Analysis. Cham: Springer; 2018. pp. 183–97.

Vonasek B, Ness T, Takwoingi Y, Kay AW, van Wyk SS, Ouellette L, et al. Screening tests for active pulmonary tuberculosis in children. Cochrane Database Syst Rev. 2021;(6). Art. No.: CD013693.

Schwoebel V, Koura KG, Adjobimey M, Gnanou S, Wandji AG, Gody J-C, et al. Tuberculosis contact investigation and short-course preventive therapy among young children in Africa. Int J Tuberc Lung Dis. 2020;24(4):452–60.

Article   CAS   PubMed   Google Scholar  

Sawry S, Moultrie H, Van Rie A. Evaluation of the intensified tuberculosis case finding guidelines for children living with HIV. Int J Tuberc Lung Dis. 2018;22(11):1322–8.

Vonasek B, Kay A, Devezin T, Bacha JM, Kazembe P, Dhillon D, et al. Tuberculosis symptom screening for children and adolescents living with HIV in six high HIV/TB burden countries in Africa. AIDS. 2021;35(1):73–9.

Jullien S, Dissanayake HA, Chaplin M. Rapid diagnostic tests for plague. Cochrane Database Syst. Rev. 2020;(6). Art. No.: CD013459.

Liu Q, Pierce DA. A note on Gauss-Hermite quadrature. Biometrika. 1994;81(3):624–9.

Google Scholar  

Jackson D, Law M, Stijnen T, Viechtbauer W, White IR. A comparison of seven random-effects models for meta-analyses that estimate the summary odds ratio. Stat Med. 2018;37(7):1059–85.

Article   PubMed   PubMed Central   Google Scholar  

Negeri ZF, Beyene J. Skew-normal random-effects model for meta-analysis of diagnostic test accuracy (DTA) studies. Biom J. 2020;62(5):1223–44.

R Core Team. R: A Language and Environment for Statistical Computing. Vienna; 2022. https://www.R-project.org/ . Accessed 10 Sept 2023.

RStudio Team. RStudio: Integrated Development Environment for R. Boston; 2020. http://www.rstudio.com/ . Accessed 10 Sept 2023.

Bates D, Mächler M, Bolker B, Walker S. Fitting Linear Mixed-Effects Models Using lme4. J Stat Softw. 2015;67(1):1–48. https://doi.org/10.18637/jss.v067.i01 .

Rizopoulos D. GLMMadaptive: Generalized Linear Mixed Models using Adaptive Gaussian Quadrature. 2023. R package version 0.9-1. https://CRAN.R-project.org/package=GLMMadaptive . Accessed 10 Sept 2023.

Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med. 2006;25(24):4279–92.

Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074–102.

Willis BH, Baragilly M, Coomar D. Maximum likelihood estimation based on Newton-Raphson iteration for the bivariate random effects model in test accuracy meta-analysis. Stat Methods Med Res. 2020;29(4):1197–211.

Nyaga VN, Arbyn M. Comparison and validation of metadta for meta-analysis of diagnostic test accuracy studies. Res Synth Methods. 2023;14(3):544–62.

Negeri ZF, Levis B, Ioannidis JP, Thombs BD, Benedetti A. An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data. BMC Med Res Methodol. 2024;24(1):28.

Download references

Acknowledgements

We are grateful to the Faculty of Mathematics, University of Waterloo, for providing us with computing resources.

Dr. Negeri, Yixin Zhao (through Dr. Negeri) and Bilal Khan (through Dr. Negeri) were supported by the University of Waterloo’s New Faculty Start-Up Grant. Bilal Khan was also supported by the University of Waterloo’s NSERC USRA award.

Author information

Yixin Zhao and Bilal Khan contributed equally to this work.

Authors and Affiliations

Department of Statistics and Actuarial Science, University of Waterloo, 200 University Ave W, Waterloo, N2L 3G1, Ontario, Canada

Yixin Zhao, Bilal Khan & Zelalem F. Negeri

You can also search for this author in PubMed   Google Scholar

Contributions

ZN contributed to the conception and design of the study, participated in data analyses, and provided critical revisions to the manuscript. YZ contributed to the writing of R code for data analyses, running and summarizing of the simulation study, and drafting of the manuscript; BK contributed to the writing of R code for data analyses, scraping the Cochrane Database of Systematic Reviews and designing of the simulation study, and drafting of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Zelalem F. Negeri .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Absolute bias, RMSE, CI width, and coverage probabilities of the three computational methods for additional simulation scenarios.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Zhao, Y., Khan, B. & Negeri, Z. An evaluation of computational methods for aggregate data meta-analyses of diagnostic test accuracy studies. BMC Med Res Methodol 24 , 111 (2024). https://doi.org/10.1186/s12874-024-02217-2

Download citation

Received : 01 January 2024

Accepted : 15 April 2024

Published : 10 May 2024

DOI : https://doi.org/10.1186/s12874-024-02217-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Meta-analysis
• Diagnostic test accuracy
• Generalized linear mixed models
• Computational methods
• Adaptive Gauss-Hermite
• Laplace approximation

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 14 May 2024

A burden of proof study on alcohol consumption and ischemic heart disease

• Sinclair Carr   ORCID: orcid.org/0000-0003-0421-3145 1 ,
• Dana Bryazka 1 ,
• Susan A. McLaughlin 1 ,
• Peng Zheng 1 , 2 ,
• Sarasvati Bahadursingh 3 ,
• Aleksandr Y. Aravkin 1 , 2 , 4 ,
• Simon I. Hay   ORCID: orcid.org/0000-0002-0611-7272 1 , 2 ,
• Hilary R. Lawlor 1 ,
• Erin C. Mullany 1 ,
• Christopher J. L. Murray   ORCID: orcid.org/0000-0002-4930-9450 1 , 2 ,
• Sneha I. Nicholson 1 ,
• Jürgen Rehm 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ,
• Gregory A. Roth 1 , 2 , 13 ,
• Reed J. D. Sorensen 1 ,
• Sarah Lewington 3 &
• Emmanuela Gakidou   ORCID: orcid.org/0000-0002-8992-591X 1 , 2  

Nature Communications volume  15 , Article number:  4082 ( 2024 ) Cite this article

1 Altmetric

Metrics details

• Cardiovascular diseases
• Epidemiology
• Risk factors

Cohort and case-control data have suggested an association between low to moderate alcohol consumption and decreased risk of ischemic heart disease (IHD), yet results from Mendelian randomization (MR) studies designed to reduce bias have shown either no or a harmful association. Here we conducted an updated systematic review and re-evaluated existing cohort, case-control, and MR data using the burden of proof meta-analytical framework. Cohort and case-control data show low to moderate alcohol consumption is associated with decreased IHD risk – specifically, intake is inversely related to IHD and myocardial infarction morbidity in both sexes and IHD mortality in males – while pooled MR data show no association, confirming that self-reported versus genetically predicted alcohol use data yield conflicting findings about the alcohol-IHD relationship. Our results highlight the need to advance MR methodologies and emulate randomized trials using large observational databases to obtain more definitive answers to this critical public health question.

Similar content being viewed by others

Alcohol consumption and risks of more than 200 diseases in Chinese men

Alcohol intake and the risk of chronic kidney disease: results from a systematic review and dose–response meta-analysis

Association of change in alcohol consumption with cardiovascular disease and mortality among initial nondrinkers

Introduction.

It is well known that alcohol consumption increases the risk of morbidity and mortality due to many health conditions 1 , 2 , with even low levels of consumption increasing the risk for some cancers 3 , 4 . In contrast, a large body of research has suggested that low to moderate alcohol intake – compared to no consumption – is associated with a decreased risk of ischemic heart disease (IHD). This has led to substantial epidemiologic and public health interest in the alcohol-IHD relationship 5 , particularly given the high prevalence of alcohol consumption 6 and the global burden of IHD 7 .

Extensive evidence from experimental studies that vary short-term alcohol exposure suggests that average levels of alcohol intake positively affect biomarkers such as apolipoprotein A1, adiponectin, and fibrinogen levels that lower the risk of IHD 8 . In contrast, heavy episodic drinking (HED) may have an adverse effect on IHD by affecting blood lipids, promoting coagulation and thus thrombosis risk, and increasing blood pressure 9 . With effects likely to vary materially by patterns of drinking, alcohol consumption must be considered a multidimensional factor impacting IHD outcomes.

A recent meta-analysis of the alcohol-IHD relationship using individual participant data from 83 observational studies 4 found, among current drinkers, that – relative to drinking less than 50 g/week – any consumption above this level was associated with a lower risk of myocardial infarction (MI) incidence and consumption between >50 and <100 g/week was associated with lower risk of MI mortality. When evaluating other subtypes of IHD excluding MI, the researchers found that consumption between >100 and <250 g/week was associated with a decreased risk of IHD incidence, whereas consumption greater than 350 g/week was associated with an increased risk of IHD mortality. Roerecke and Rehm further observed that low to moderate drinking was not associated with reduced IHD risk when accompanied by occasional HED 10 .

The cohort studies and case-control studies (hereafter referred to as ‘conventional observational studies’) used in these meta-analyses are known to be subject to various types of bias when used to estimate causal relationships 11 . First, neglecting to separate lifetime abstainers from former drinkers, some of whom may have quit due to developing preclinical symptoms (sometimes labeled ‘sick quitters’ 12 , 13 ), and to account for drinkers who reduce their intake as a result of such symptoms may introduce reverse causation bias 13 . That is, the risk of IHD in, for example, individuals with low to moderate alcohol consumption may be lower when compared to IHD risk in sick quitters, not necessarily because intake at this level causes a reduction in risk but because sick quitters are at higher risk of IHD. Second, estimates can be biased because of measurement error in alcohol exposure resulting from inaccurate reporting, random fluctuation in consumption over time (random error), or intentional misreporting of consumption due, for example, to social desirability effects 14 (systematic error). Third, residual confounding may bias estimates if confounders of the alcohol-IHD relationship, such as diet or physical activity, have not been measured accurately (e.g., only via a self-report questionnaire) or accounted for. Fourth, because alcohol intake is a time-varying exposure, time-varying confounding affected by prior exposure must be accounted for 15 . To date, only one study that used a marginal structural model to appropriately adjust for time-varying confounding found no association between alcohol consumption and MI risk 16 . Lastly, if exposure to a risk factor, such as alcohol consumption, did not happen at random – even if all known confounders of the relationship between alcohol and IHD were perfectly measured and accounted for – the potential for unmeasured confounders persists and may bias estimates 11 .

In recent years, the analytic method of Mendelian randomization (MR) has been widely adopted to quantify the causal effects of risk factors on health outcomes 17 , 18 , 19 . MR uses single nucleotide polymorphisms (SNPs) as instrumental variables (IVs) for the exposure of interest. A valid IV should fulfill the following three assumptions: it must be associated with the risk factor (relevance assumption); there must be no common causes of the IV and the outcome (independence assumption); and the IV must affect the outcome only through the exposure (exclusion restriction or ‘no horizontal pleiotropy’ assumption) 20 , 21 . If all three assumptions are fulfilled, estimates derived from MR are presumed to represent causal effects 22 . Several MR studies have quantified the association between alcohol consumption and cardiovascular disease 23 , including IHD, using genes known to impact alcohol metabolism (e.g., ADH1B/C and ALDH2 24 ) or SNP combinations from genome-wide association studies 25 . In contrast to the inverse associations found in conventional observational studies, MR studies have found either no association or a harmful relationship between alcohol consumption and IHD 26 , 27 , 28 , 29 , 30 , 31 .

To advance the knowledge base underlying our understanding of this major health issue – critical given the worldwide ubiquity of alcohol use and of IHD – there is a need to systematically review and critically re-evaluate all available evidence on the relationship between alcohol consumption and IHD risk from both conventional observational and MR studies.

The burden of proof approach, developed by Zheng et al. 32 , is a six-step meta-analysis framework that provides conservative estimates and interpretations of risk-outcome relationships. The approach systematically tests and adjusts for common sources of bias defined according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria: representativeness of the study population, exposure assessment, outcome ascertainment, reverse causation, control for confounding, and selection bias. The key statistical tool to implement the approach is MR-BRT (meta-regression—Bayesian, regularized, trimmed 33 ), a flexible meta-regression tool that does not impose a log-linear relationship between the risk and outcome, but instead uses a spline ensemble to model non-linear relationships. MR-BRT also algorithmically detects and trims outliers in the input data, takes into account different reference and alternative exposure intervals in the data, and incorporates unexplained between-study heterogeneity in the uncertainty surrounding the mean relative risk (RR) curve (henceforth ‘risk curve’). For those risk-outcome relationships that meet the condition of statistical significance using conventionally estimated uncertainty intervals (i.e., without incorporating unexplained between-study heterogeneity), the burden of proof risk function (BPRF) is derived by calculating the 5th (if harmful) or 95th (if protective) quantile risk curve – inclusive of between-study heterogeneity – closest to the log RR of 0. The resulting BPRF is a conservative interpretation of the risk-outcome relationship based on all available evidence. The BPRF represents the smallest level of excess risk for a harmful risk factor or reduced risk for a protective risk factor that is consistent with the data, accounting for between-study heterogeneity. To quantify the strength of the evidence for the alcohol-IHD relationship, the BPRF can be summarized in a single metric, the risk-outcome score (ROS). The ROS is defined as the signed value of the average log RR of the BPRF across the 15th to 85th percentiles of alcohol consumption levels observed across available studies. The larger a positive ROS value, the stronger the alcohol-IHD association. For ease of interpretation, the ROS is converted into a star rating from one to five. A one-star rating (ROS < 0) indicates a weak alcohol-IHD relationship, and a five-star rating (ROS > 0.62) indicates a large effect size and strong evidence. Publication and reporting bias are evaluated with Egger’s regression and by visual inspection with funnel plots 34 . Further conceptual and technical details of the burden of proof approach are described in detail elsewhere 32 .

Using the burden of proof approach, we systematically re-evaluate all available eligible evidence from cohort, case-control, and MR studies published between 1970 and 2021 to conservatively quantify the dose-response relationship between alcohol consumption and IHD risk, calculated relative to risk at zero alcohol intake (i.e., current non-drinking, including lifetime abstinence or former use). We pool the evidence from all conventional observational studies combined, as well as individually for all three study designs, to estimate mean IHD risk curves. Based on patterns of results established by previous meta-analyses 4 , 35 , we also use data from conventional observational studies to estimate risk curves by IHD endpoint (morbidity or mortality) and further by sex, in addition to estimating risk curves for MI overall and by endpoint. We follow PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines 36 through all stages of this study (Supplementary Information section  1 , Fig.  S1 and Tables  S1 and S2 ) and comply with GATHER (Guidelines on Accurate and Transparent Health Estimates Reporting) recommendations 37 (Supplementary Information section  2 , Table  S3 ). The main findings and research implications of this work are summarized in Table  1 .

We updated the systematic review on the dose-response relationship between alcohol consumption and IHD previously conducted for the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2020 1 . Of 4826 records identified in our updated systematic review (4769 from databases/registers and 57 by citation search and known literature), 11 were eligible based on our inclusion criteria and were included. In total, combined with the results of the previous systematic reviews 1 , 38 , information from 95 cohort studies 26 , 27 , 29 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 27 case-control studies 131 , 132 , 133 , 134 , 135 , 136 , 137 , 138 , 139 , 140 , 141 , 142 , 143 , 144 , 145 , 146 , 147 , 148 , 149 , 150 , 151 , 152 , 153 , 154 , 155 , 156 , 157 , and five MR studies 26 , 27 , 28 , 29 , 31 was included in our meta-analysis (see Supplementary Information section  1 , Fig.  S1 , for the PRISMA diagram). Details on the extracted effect sizes, the design of each included study, underlying data sources, number of participants, duration of follow-up, number of cases and controls, and bias covariates that were evaluated and potentially adjusted for can be found in the Supplementary Information Sections  4 , 5 , and 6 .

Table  2 summarizes key metrics of each risk curve modeled, including estimates of mean RR and 95% UI (inclusive of between-study heterogeneity) at select alcohol exposure levels, the exposure level and RR and 95% UI at the nadir (i.e., lowest RR), the 85th percentile of exposure observed in the data and its corresponding RR and 95% UI, the BPRF averaged at the 15th and 85th percentile of exposure, the average excess risk or risk reduction according to the exposure-averaged BPRF, the ROS, the associated star rating, the potential presence of publication or reporting bias, and the number of studies included.

We found large variation in the association between alcohol consumption and IHD by study design. When we pooled the results of cohort and case-control studies, we observed an inverse association between alcohol at average consumption levels and IHD risk; that is, drinking average levels of alcohol was associated with a reduced IHD risk relative to drinking no alcohol. In contrast, we did not find a statistically significant association between alcohol consumption and IHD risk when pooling results from MR studies. When we subset the conventional observational studies to those reporting on IHD by endpoint, we found no association between alcohol consumption and IHD morbidity or mortality due to large unexplained heterogeneity between studies. When we further subset those studies that reported effect size estimates by sex, we found that average alcohol consumption levels were inversely associated with IHD morbidity in males and in females, and with IHD mortality in males but not in females. When we analyzed only the studies that reported on MI, we found significant inverse associations between average consumption levels and MI overall and with MI morbidity. Visualizations of the risk curves for morbidity and mortality of IHD and MI are provided in Supplementary Information Section  9 (Figs.  S2a –c, S3a –c, and S4a–c ). Among all modeled risk curves for which a BPRF was calculated, the ROS ranged from −0.40 for MI mortality to 0.20 for MI morbidity. In the Supplementary Information, we also provide details on the RR and 95% UIs with and without between-study heterogeneity associated with each 10 g/day increase in consumption for each risk curve (Table  S10 ), the parameter specifications of the model (Tables  S11 and S12 ), and each risk curve from the main analysis estimated without trimming 10% of the data (Fig.  S5a–l and Table  S13 ).

Risk curve derived from conventional observational study data

The mean risk curve and 95% UI were first estimated by combining all evidence from eligible cohort and case-control studies that quantified the association between alcohol consumption and IHD risk. In total, information from 95 cohort studies and 27 case-control studies combining data from 7,059,652 participants were included. In total, 243,357 IHD events were recorded. Thirty-seven studies quantified the association between alcohol consumption and IHD morbidity only, and 44 studies evaluated only IHD mortality. The estimated alcohol-IHD association was adjusted for sex and age in all but one study. Seventy-five studies adjusted the effect sizes for sex, age, smoking, and at least four other covariates. We adjusted our risk curve for whether the study sample was under or over 50 years of age, whether the study outcome was consistent with the definition of IHD (according to the International Classification of Diseases [ICD]−9: 410-414; and ICD-10: I20-I25) or related to specified subtypes of IHD, whether the outcome was ascertained by self-report only or by at least one other measurement method, whether the study accounted for risk for reverse causation, whether the reference group was non-drinkers (including lifetime abstainers and former drinkers), and whether effect sizes were adjusted (1) for sex, age, smoking, and at least four other variables, (2) for apolipoprotein A1, and (3) for cholesterol, as these bias covariates were identified as significant by our algorithm.

Pooling all data from cohort and case-control studies, we found that alcohol consumption was inversely associated with IHD risk (Fig.  1 ). The risk curve was J-shaped – without crossing the null RR of 1 at high exposure levels – with a nadir of 0.69 (95% UI: 0.48–1.01) at 23 g/day. This means that compared to individuals who do not drink alcohol, the risk of IHD significantly decreases with increasing consumption up to 23 g/day, followed by a risk reduction that becomes less pronounced. The average BPRF calculated between 0 and 45 g/day of alcohol intake (the 15th and 85th percentiles of the exposure range observed in the data) was 0.96. Thus, when between-study heterogeneity is accounted for, a conservative interpretation of the evidence suggests drinking alcohol across the average intake range is associated with an average decrease in the risk of IHD of at least 4% compared to drinking no alcohol. This corresponds to a ROS of 0.04 and a star rating of two, which suggests that the association – on the basis of the available evidence – is weak. Although we algorithmically identified and trimmed 10% of the data to remove outliers, Egger’s regression and visual inspection of the funnel plot still indicated potential publication or reporting bias.

The panels show the log(relative risk) function, the relative risk function, and a modified funnel plot showing the residuals (relative to 0) on the x-axis and the estimated standard error that includes the reported standard error and between-study heterogeneity on the y-axis. RR relative risk, UI uncertainty interval. Source data are provided as a Source Data file.

Risk curve derived from case-control study data

Next, we estimated the mean risk curve and 95% UI for the relationship between alcohol consumption and IHD by subsetting the data to case-control studies only. We included a total of 27 case-control studies (including one nested case-control study) with data from 60,914 participants involving 16,892 IHD cases from Europe ( n  = 15), North America ( n  = 6), Asia ( n  = 4), and Oceania ( n  = 2). Effect sizes were adjusted for sex and age in most studies ( n  = 25). Seventeen of these studies further adjusted for smoking and at least four other covariates. The majority of case-control studies accounted for the risk of reverse causation ( n  = 25). We did not adjust our risk curve for bias covariates, as our algorithm did not identify any as significant.

Evaluating only data from case-control studies, we observed a J-shaped relationship between alcohol consumption and IHD risk, with a nadir of 0.65 (0.50–0.85) at 23 g/day (Fig.  2 ). The inverse association between alcohol consumption and IHD risk reversed at an intake level of 61 g/day. In other words, alcohol consumption between >0 and 60 g/day was associated with a lower risk compared to no consumption, while consumption at higher levels was associated with increased IHD risk. However, the curve above this level is flat, implying that the association between alcohol and increased IHD risk is the same between 61 and 100 g/day, relative to not drinking any alcohol. The BPRF averaged across the exposure range between the 15th and 85th percentiles, or 0–45 g/day, was 0.87, which translates to a 13% average reduction in IHD risk across the average range of consumption. This corresponds to a ROS of 0.14 and a three-star rating. After trimming 10% of the data, no potential publication or reporting bias was found.

The panels show the log(relative risk) function, the relative risk function, and a modified funnel plot showing the residuals (relative to 0) on the x-axis and the estimated standard deviation that includes the reported standard deviation and between-study heterogeneity on the y-axis. RR relative risk, UI uncertainty interval. Source data are provided as a Source Data file.

Risk curve derived from cohort study data

We also estimated the mean risk curve and 95% UI for the relationship between alcohol consumption and IHD using only data from cohort studies. In total, 95 cohort studies – of which one was a retrospective cohort study – with data from 6,998,738 participants were included. Overall, 226,465 IHD events were recorded. Most data were from Europe ( n  = 43) and North America ( n  = 33), while a small number of studies were conducted in Asia ( n  = 14), Oceania ( n  = 3), and South America ( n  = 2). The majority of studies adjusted effect sizes for sex and age ( n  = 76). Fifty-seven of these studies also adjusted for smoking and at least four other covariates. Out of all cohort studies included, 88 accounted for the risk of reverse causation. We adjusted our risk curve for whether the study outcome was consistent with the definition of IHD or related to specified subtypes of IHD, and whether effect sizes were adjusted for apolipoprotein A1, as these bias covariates were identified as significant by our algorithm.

When only data from cohort studies were evaluated, we found a J-shaped relationship between alcohol consumption and IHD risk that did not cross the null RR of 1 at high exposure levels, with a nadir of 0.69 (0.47–1.01) at 23 g/day (Fig.  3 ). The shape of the risk curve was almost identical to the curve estimated with all conventional observational studies (i.e., cohort and case-control studies combined). When we calculated the average BPRF of 0.95 between the 15th and 85th percentiles of observed alcohol exposure (0–50 g/day), we found that alcohol consumption across the average intake range was associated with an average reduction in IHD risk of at least 5%. This corresponds to a ROS of 0.05 and a two-star rating. We identified potential publication or reporting bias after 10% of the data were trimmed.

Risk curve derived from Mendelian randomization study data

Lastly, we pooled evidence on the relationship between genetically predicted alcohol consumption and IHD risk from MR studies. Four MR studies were considered eligible for inclusion in our main analysis, with data from 559,708 participants from China ( n  = 2), the Republic of Korea ( n  = 1), and the United Kingdom ( n  = 1). Overall, 22,134 IHD events were recorded. Three studies used the rs671 ALDH2 genotype found in Asian populations, one study additionally used the rs1229984 ADH1B variant, and one study used the rs1229984 ADH1B Arg47His variant and a combination of 25 SNPs as IVs. All studies used the two-stage least squares (2SLS) method to estimate the association, and one study additionally applied the inverse-variance-weighted (IVW) method and multivariable MR (MVMR). For the study that used multiple methods to estimate effect sizes, we used the 2SLS estimates for our main analysis. Further details on the included studies are provided in Supplementary Information section  4 (Table  S6 ). Due to limited input data, we elected not to trim 10% of the observations. We adjusted our risk curve for whether the endpoint of the study outcome was mortality and whether the associations were adjusted for sex and/or age, as these bias covariates were identified as significant by our algorithm.

We did not find any significant association between genetically predicted alcohol consumption and IHD risk using data from MR studies (Fig.  4 ). No potential publication or reporting bias was detected.

As sensitivity analyses, we modeled risk curves with effect sizes estimated from data generated by Lankester et al. 28 using IVW and MVMR methods. We also used effect sizes from Biddinger et al. 31 , obtained using non-linear MR with the residual method, instead of those from Lankester et al. 28 in our main model (both were estimated with UK Biobank data) to estimate a risk curve. Again, we did not find a significant association between genetically predicted alcohol consumption and IHD risk (see Supplementary Information Section  10 , Fig.  S6a–c and Table  S14 ). To test for consistency with the risk curve we estimated using all included cohort studies, we also pooled the conventionally estimated effect sizes provided in the four MR studies. We did not observe an association between alcohol consumption and IHD risk due to large unexplained heterogeneity between studies (see Supplementary Information Section  10 , Fig.  S7, and Table  S14 ). Lastly, we pooled cohort studies that included data from China, the Republic of Korea, and the United Kingdom to account for potential geographic influences. Again, we did not find a significant association between alcohol consumption and IHD risk (see Supplementary Information Section  10 , Fig.  S8, and Table  S14 ).

Conventional observational and MR studies published to date provide conflicting estimates of the relationship between alcohol consumption and IHD. We conducted an updated systematic review and conservatively re-evaluated existing evidence on the alcohol-IHD relationship using the burden of proof approach. We synthesized evidence from cohort and case-control studies combined and separately and from MR studies to assess the dose-response relationship between alcohol consumption and IHD risk and to compare results across different study designs. It is anticipated that the present synthesis of evidence will be incorporated into upcoming iterations of GBD.

Our estimate of the association between genetically predicted alcohol consumption and IHD runs counter to our estimates from the self-report data and those of other previous meta-analyses 4 , 35 , 158 that pooled conventional observational studies. Based on the conservative burden of proof interpretation of the data, our results suggested an inverse association between alcohol and IHD when all conventional observational studies were pooled (alcohol intake was associated with a reduction in IHD risk by an average of at least 4% across average consumption levels; two-star rating). In evaluating only cohort studies, we again found an inverse association between alcohol consumption and IHD (alcohol intake was associated with a reduction in IHD risk by an average of at least 5% at average consumption levels; two-star rating). In contrast, when we pooled only case-control studies, we estimated that average levels of alcohol consumption were associated with at least a 13% average decrease in IHD risk (three-star rating), but the inverse association reversed when consumption exceeded 60 g/day, suggesting that alcohol above this level is associated with a slight increase in IHD risk. Our analysis of the available evidence from MR studies showed no association between genetically predicted alcohol consumption and IHD.

Various potential biases and differences in study designs may have contributed to the conflicting findings. In our introduction, we summarized important sources of bias in conventional observational studies of the association between alcohol consumption and IHD. Of greatest concern are residual and unmeasured confounding and reverse causation, the effects of which are difficult to eliminate in conventional observational studies. By using SNPs within an IV approach to predict exposure, MR – in theory – eliminates these sources of bias and allows for more robust estimates of causal effects. Bias may still occur, however, when using MR to estimate the association between alcohol and IHD 159 , 160 . There is always the risk of horizontal pleiotropy in MR – that is, the genetic variant may affect the outcome via pathways other than exposure 161 . The IV assumption of exclusion restriction is, for example, violated if only a single measurement of alcohol consumption is used in MR 162 ; because alcohol consumption varies over the life course, the gene directly impacts IHD through intake at time points other than that used in the MR analysis. To date, MR studies have not succeeded in separately capturing the multidimensional effects of alcohol intake on IHD risk (i.e., effects of average alcohol consumption measured through frequency-quantity, in addition to the effects of HED) 159 because the genes used to date only target average alcohol consumption that encompasses intake both at average consumption levels and HED. In other words, the instruments used are not able to separate out the individual effects of these two different dimensions of alcohol consumption on IHD risk using MR. Moreover, reverse causation may occur through cross-generational effects 160 , 163 , as the same genetic variants predispose both the individual and at least one of his or her parents to (increased) alcohol consumption. In this situation, IHD risk could be associated with the parents’ genetically predicted alcohol consumption and not with the individual’s own consumption. None of the MR studies included accounted for cross-generational effects, which possibly introduced bias in the effect estimates. It is important to note that bias by ancestry might also occur in conventional observational studies 164 . In summary, estimates of the alcohol-IHD association are prone to bias in all three study designs, limiting inferences of causation.

The large difference in the number of available MR versus conventional observational studies, the substantially divergent results derived from the different study types, and the rapidly developing field of MR clearly argue for further investigation of MR as a means to quantify the association between alcohol consumption and IHD risk. Future studies should investigate non-linearity in the relationship using non-linear MR methods. The residual method, commonly applied in non-linear MR studies such as Biddinger et al. 31 , assumes a constant, linear relationship between the genetic IV and the exposure in the study population; a strong assumption that may result in biased estimates and inflated type I error rates if the relationship varies by population strata 165 . However, by log-transforming the exposure, the relationships between the genetic IV and the exposure as expressed on a logarithmic scale may be more homogeneous across strata, possibly reducing the bias effect of violating the assumption of a constant, linear relationship. Alternatively, or in conjunction, the recently developed doubly ranked method, which obviates the need for this assumption, could be used 166 . Since methodology for non-linear MR is an active field of study 167 , potential limitations of currently available methods should be acknowledged and latest guidelines be followed 168 . Future MR studies should further (i) employ sensitivity analyses such as the MR weighted median method 169 to relax the exclusion restriction assumption that may be violated, as well as applying other methods such as the MR-Egger intercept test; (ii) use methods such as g-estimation of structural mean models 162 to adequately account for temporal variation in alcohol consumption in MR, and (iii) attempt to disaggregate the effects of alcohol on IHD by dimension in MR, potentially through the use of MVMR 164 . General recommendations to overcome common MR limitations are described in greater detail elsewhere 159 , 163 , 170 , 171 and should be carefully considered. With respect to prospective cohort studies used to assess the alcohol-IHD relationship, they should, at a minimum: (i) adjust the association between alcohol consumption and IHD for all potential confounders identified, for example, using a causal directed acyclic graph, and (ii) account for reverse causation introduced by sick quitters and by drinkers who changed their consumption. If possible, they should also (iii) use alcohol biomarkers as objective measures of alcohol consumption instead of or in addition to self-reported consumption to reduce bias through measurement error, (iv) investigate the association between IHD and HED, in addition to average alcohol consumption, and (v) when multiple measures of alcohol consumption and potential confounders are available over time, use g-methods to reduce bias through confounding as fully as possible within the limitations of the study design. However, some bias – due, for instance, to unmeasured confounding in conventional observational and to horizontal pleiotropy in MR studies – is likely inevitable, and the interpretation of estimates should be appropriately cautious, in accordance with the methods used in the study.

With the introduction of the Moderate Alcohol and Cardiovascular Health Trial (MACH15) 172 , randomized controlled trials (RCTs) have been revisited as a way to study the long-term effects of low to moderate alcohol consumption on cardiovascular disease, including IHD. In 2018, soon after the initiation of MACH15, the National Institutes of Health terminated funding 173 , reportedly due to concerns about study design and irregularities in the development of funding opportunities 174 . Although MACH15 was terminated, its initiation represented a previously rarely considered step toward investigating the alcohol-IHD relationship using an RCT 175 . However, while the insights from an RCT are likely to be invaluable, the implementation is fraught with potential issues. Due to the growing number of studies suggesting increased disease risk, including cancer 3 , 4 , associated with alcohol use even at very low levels 176 , the use of RCTs to study alcohol consumption is ethically questionable 177 . A less charged approach could include the emulation of target trials 178 using existing observational data (e.g., from large-scale prospective cohort studies such as the UK Biobank 179 , Atherosclerosis Risk in Communities Study 180 , or the Framingham Heart Study 181 ) in lieu of real trials to gather evidence on the potential cardiovascular effects of alcohol. Trials like MACH15 can be emulated, following the proposed trial protocols as closely as the observational dataset used for the analysis allows. Safety and ethical concerns, such as those related to eligibility criteria, initiation/increase in consumption, and limited follow-up duration, will be eliminated because the data will have already been collected. This framework allows for hypothetical trials investigating ethically challenging or even untenable questions, such as the long-term effects of heavy (episodic) drinking on IHD risk, to be emulated and inferences to broader populations drawn.

There are several limitations that must be considered when interpreting our findings. First, record screening for our systematic review was not conducted in a double-blinded fashion. Second, we did not have sufficient evidence to estimate and examine potential differential associations of alcohol consumption with IHD risk by beverage type or with MI endpoints by sex. Third, despite using a flexible meta-regression tool that overcame several limitations common to meta-analyses, the results of our meta-analysis were only as good as the quality of the studies included. We were able, however, to address the issue of varying quality of input data by adjusting for bias covariates that corresponded to core study characteristics in our analyses. Fourth, because we were only able to include one-sample MR studies that captured genetically predicted alcohol consumption, statistical power may be lower than would have been possible with the inclusion of two-sample MR studies, and studies that directly estimated gene-IHD associations were not considered 23 . Finally, we were not able to account for participants’ HED status when pooling effect size estimates from conventional observational studies. Given established differences in IHD risk for drinkers with and without HED 35 and the fact that more than one in three drinkers reports HED 6 , we would expect that the decreased average risk we found at moderate levels of alcohol consumption would be attenuated (i.e., approach the IHD risk of non-drinkers) if the presence of HED was taken into account.

Using the burden of proof approach 32 , we conservatively re-evaluated the dose-response relationship between alcohol consumption and IHD risk based on existing cohort, case-control, and MR data. Consistent with previous meta-analyses, we found that alcohol at average consumption levels was inversely associated with IHD when we pooled conventional observational studies. This finding was supported when aggregating: (i) all studies, (ii) only cohort studies, (iii) only case-control studies, (iv) studies examining IHD morbidity in females and males, (v) studies examining IHD mortality in males, and (vi) studies examining MI morbidity. In contrast, we found no association between genetically predicted alcohol consumption and IHD risk based on data from MR studies. Our confirmation of the conflicting results derived from self-reported versus genetically predicted alcohol use data highlights the need to advance methodologies that will provide more definitive answers to this critical public health question. Given the limitations of randomized trials, we advocate using advanced MR techniques and emulating target trials using observational data to generate more conclusive evidence on the long-term effects of alcohol consumption on IHD risk.

This study was approved by the University of Washington IRB Committee (study #9060).

The burden of proof approach is a six-step framework for conducting meta-analysis 32 : (1) data from published studies that quantified the dose-response relationship between alcohol consumption and ischemic heart disease (IHD) risk were systematically identified and obtained; (2) the shape of the mean relative risk (RR) curve (henceforth ‘risk curve’) and associated uncertainty was estimated using a quadratic spline and algorithmic trimming of outliers; (3) the risk curve was tested and adjusted for biases due to study attributes; (4) unexplained between-study heterogeneity was quantified, adjusting for within-study correlation and number of studies included; (5) the evidence for small-study effects was evaluated to identify potential risks of publication or reporting bias; and (6) the burden of proof risk function (BPRF) – a conservative interpretation of the average risk across the exposure range found in the data – was estimated relative to IHD risk at zero alcohol intake. The BPRF was converted to a risk-outcome score (ROS) that was mapped to a star rating from one to five to provide an intuitive interpretation of the magnitude and direction of the dose-response relationship between alcohol consumption and IHD risk.

We calculated the mean RR and 95% uncertainty intervals (UIs) for IHD associated with levels of alcohol consumption separately with all evidence available from conventional observational studies and from Mendelian randomization (MR) studies. For the risk curves that met the condition of statistical significance when the conventional 95% UI that does not include unexplained between-study heterogeneity was evaluated, we calculated the BPRF, ROS, and star rating. Based on input data from conventional observational studies, we also estimated these metrics by study design (cohort studies, case-control studies), and by IHD endpoint (morbidity, mortality) for both sexes (females, males) and sex-specific. For sex-stratified analyses, we only considered studies that reported effect sizes for both females and males to allow direct comparison of IHD risk across different exposure levels; however, we did not collect information about the method each study used to determine sex. We also estimated risk curves for myocardial infarction (MI), overall and by endpoint, using data from conventional observational studies. As a comparison, we also estimated each risk curve without trimming 10% of the input data. We did not consider MI as an outcome or disaggregate findings by sex or endpoint for MR studies due to insufficient data.

With respect to MR studies, several statistical methods are typically used to estimate the associations between genetically predicted exposure and health outcomes (e.g., two-stage least squares [2SLS], inverse-variance-weighted [IVW], multivariable Mendelian randomization [MVMR]). For our main analysis synthesizing evidence from MR studies, we included the reported effect sizes estimated using 2SLS if a study applied multiple methods because this method was common to all included studies. In sensitivity analyses, we used the effect sizes obtained by other MR methods (i.e., IVW, MVMR, and non-linear MR) and estimated the mean risk curve and uncertainty. We also pooled conventionally estimated effect sizes from MR studies to allow comparison with the risk curve estimated with cohort studies. Due to limited input data from MR studies, we elected not to trim 10% of the observations. Furthermore, we estimated the risk curve from cohort studies with data from countries that corresponded to those included in MR studies (China, the Republic of Korea, and the United Kingdom). Due to a lack of data, we were unable to estimate a risk curve from case-control studies in these geographic regions.

Conducting the systematic review

In step one of the burden of proof approach, data for the dose-response relationship between alcohol consumption and IHD risk were systematically identified, reviewed, and extracted. We updated a previously published systematic review 1 in PubMed that identified all studies evaluating the dose-response relationship between alcohol consumption and risk of IHD morbidity or mortality from January 1, 1970, to December 31, 2019. In our update, we additionally considered all studies up to and including December 31, 2021, for eligibility. We searched articles in PubMed on March 21, 2022, with the following search string: (alcoholic beverage[MeSH Terms] OR drinking behavior[MeSH Terms] OR “alcohol”[Title/Abstract]) AND (Coronary Artery Disease[Mesh] OR Myocardial Ischemia[Mesh] OR atherosclerosis[Mesh] OR Coronary Artery Disease[TiAb] OR Myocardial Ischemia[TiAb] OR cardiac ischemia[TiAb] OR silent ischemia[TiAb] OR atherosclerosis Outdent [TiAb] OR Ischemic heart disease[TiAb] OR Ischemic heart disease[TiAb] OR coronary heart disease[TiAb] OR myocardial infarction[TiAb] OR heart attack[TiAb] OR heart infarction[TiAb]) AND (Risk[MeSH Terms] OR Odds Ratio[MeSH Terms] OR “risk”[Title/Abstract] OR “odds ratio”[Title/Abstract] OR “cross-product ratio”[Title/Abstract] OR “hazards ratio”[Title/Abstract] OR “hazard ratio”[Title/Abstract]) AND (“1970/01/01”[PDat]: “2021/12/31”[PDat]) AND (English[LA]) NOT (animals[MeSH Terms] NOT Humans[MeSH Terms]). Studies were eligible for inclusion if they met all of the following criteria: were published between January 1, 1970, and December 31, 2021; were a cohort study, case-control study, or MR study; described an association between alcohol consumption and IHD and reported an effect size estimate (relative risk, hazard ratio, odds ratio); and used a continuous dose as exposure of alcohol consumption. Studies were excluded if they met any of the following criteria: were an aggregate study (meta-analysis or pooled cohort); utilized a study design not designated for inclusion in this analysis: not a cohort study, case-control study, or MR study; were a duplicate study: the underlying sample of the study had also been analyzed elsewhere (we always considered the analysis with the longest follow-up for cohort studies or the most recently published analysis for MR studies); did not report on the exposure of interest: reported on combined exposure of alcohol and drug use or reported alcohol consumption in a non-continuous way; reported an outcome that was not IHD or a composite outcome that included but was not limited to IHD, or outcomes lacked specificity, such as cardiovascular disease or all-cause mortality; were not in English; and were animal studies. All screenings of titles and abstracts of identified records, as well as full texts of potentially eligible studies, and extraction of included studies, were done by a single reviewer (SC or HL) independently. If eligible, studies were extracted for study characteristics, exposure, outcome, adjusted confounders, and effect sizes and their uncertainty. While the previous systematic review only considered cohort and case-control studies, our update also included MR studies. We chose to consider only ‘one-sample’ MR studies, i.e., those in which genes, risk factors, and outcomes were measured in the same participants, and not ‘two-sample’ MR studies in which two different samples were used for the MR analysis so that we could fully capture study-specific information. We re-screened previously identified records for MR studies to consider all published MR studies in the defined time period. We also identified and included in our sensitivity analysis an MR study published in 2022 31 which used a non-linear MR method to estimate the association between genetically predicted alcohol consumption and IHD. When eligible studies reported both MR and conventionally estimated effect sizes (i.e., for the association between self-reported alcohol consumption and IHD risk), we extracted both. If studies used the same underlying sample and investigated the same outcome in the same strata, we included the study that had the longest follow-up. This did not apply when the same samples were used in conventional observational and MR studies, because they were treated separately when estimating the risk curve of alcohol consumption and IHD. Continuous exposure of alcohol consumption was defined as a frequency-quantity measure 182 and converted to g/day. IHD was defined according to the International Classification of Diseases (ICD)−9, 410-414, and ICD-10, I20-I25.

The raw data were extracted with a standardized extraction sheet (see Supplementary Information Section  3 , Table  S4 ). For conventional observational studies, when multiple effect sizes were estimated from differently adjusted regression models, we used those estimated with the model reported to be fully adjusted or the one with the most covariates. In the majority of studies, alcohol consumption was categorized based on the exposure range available in the data. If the lower end of a categorical exposure range (e.g., <10 g/day) of an effect size was not specified in the input data, we assumed that this was 0 g/day. If the upper end was not specified (e.g., >20 g/day), it was calculated by multiplying the lower end of the categorical exposure range by 1.5. When the association between alcohol and IHD risk was reported as a linear slope, the average consumption level in the sample was multiplied by the logarithm of the effect size to effectively render it categorical. From the MR study which employed non-linear MR 31 , five effect sizes and their uncertainty were extracted at equal intervals across the reported range of alcohol exposure using WebPlotDigitizer. To account for the fact that these effect sizes were derived from the same non-linear risk curve, we adjusted the extracted standard errors by multiplying them by the square root of five (i.e., the number of extracted effect sizes). Details on data sources are provided in Supplementary Information Section  4 .

Estimating the shape of the risk-outcome relationship

In step two, the shape of the dose-response relationship (i.e., ‘signal’) between alcohol consumption and IHD risk was estimated relative to risk at zero alcohol intake. The meta-regression tool MR-BRT (meta-regression—Bayesian, regularized, trimmed), developed by Zheng et al. 33 , was used for modeling. To allow for non-linearity, thus relaxing the common assumption of a log-linear relationship, a quadratic spline with two interior knots was used for estimating the risk curve 33 . We used the following three risk measures from included studies: RRs, odds ratios (ORs), and hazard ratios (HRs). ORs were treated as equivalent to RRs and HRs based on the rare outcome assumption. To counteract the potential influence of knot placement on the shape of the risk curve when using splines, an ensemble model approach was applied. Fifty component models with random knot placements across the exposure domain were computed. These were combined into an ensemble by weighting each model based on model fit and variation (i.e., smoothness of fit to the data). To prevent bias from outliers, a robust likelihood-based approach was applied to trim 10% of the observations. Technical details on estimating the risk curve, use of splines, the trimming procedure, the ensemble model approach, and uncertainty estimation are described elsewhere 32 , 33 . Details on the model specifications for each risk curve are provided in Supplementary Information section  8 . We first estimated each risk curve without trimming input data to visualize the shape of the curve, which informed knot placement and whether to set a left and/or right linear tail when data were sparse at low or high exposure levels (see Supplementary Information Section  10 , Fig.  S5a–l ).

Testing and adjusting for biases across study designs and characteristics

In step three, the risk curve was tested and adjusted for systematic biases due to study attributes. According to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria 183 , the following six bias sources were quantified: representativeness of the study population, exposure assessment, outcome ascertainment, reverse causation, control for confounding, and selection bias. Representativeness was quantified by whether the study sample came from a location that was representative of the underlying geography. Exposure assessment was quantified by whether alcohol consumption was recorded once or more than once in conventional observational studies, or with only one or multiple SNPs in MR studies. Outcome ascertainment was quantified by whether IHD was ascertained by self-report only or by at least one other measurement method. Reverse causation was quantified by whether increased IHD risk among participants who reduced or stopped drinking was accounted for (e.g., by separating former drinkers from lifetime abstainers). Control for confounding factors was quantified by which and how many covariates the effect sizes were adjusted for (i.e., through stratification, matching, weighting, or standardization). Because the most adjusted effect sizes in each study were extracted in the systematic review process and thus may have been adjusted for mediators, we additionally quantified a bias covariate for each of the following potential mediators of the alcohol-IHD relationship: body mass index, blood pressure, cholesterol (excluding high-density lipoprotein cholesterol), fibrinogen, apolipoprotein A1, and adiponectin. Selection bias was quantified by whether study participants were selected and included based on pre-existing disease states. We also quantified and considered as possible bias covariates whether the reference group was non-drinkers, including lifetime abstainers and former drinkers; whether the sample was under or over 50 years of age; whether IHD morbidity, mortality, or both endpoints were used; whether the outcome mapped to IHD or referred only to subtypes of IHD; whether the outcome mapped to MI; and what study design (cohort or case-control) was used when conventional observational studies were pooled. Details on quantified bias covariates for all included studies are provided in Supplementary Information section  5 (Tables  S7 and S8 ). Using a Lasso approach 184 , the bias covariates were first ranked. They were then included sequentially, based on their ranking, as effect modifiers of the ‘signal’ obtained in step two in a linear meta-regression. Significant bias covariates were included in modeling the final risk curve. Technical details of the Lasso procedure are described elsewhere 32 .

Quantifying between-study heterogeneity, accounting for heterogeneity, uncertainty, and small number of studies

In step four, the between-study heterogeneity was quantified, accounting for heterogeneity, uncertainty, and small number of studies. In a final linear mixed-effects model, the log RRs were regressed against the ‘signal’ and selected bias covariates, with a random intercept to account for within-study correlation and a study-specific random slope with respect to the ‘signal’ to account for between-study heterogeneity. A Fisher information matrix was used to estimate the uncertainty associated with between-study heterogeneity 185 because heterogeneity is easily underestimated or may be zero when only a small number of studies are available. We estimated the mean risk curve with a 95% UI that incorporated between-study heterogeneity, and we additionally estimated a 95% UI without between-study heterogeneity as done in conventional meta-regressions (see Supplementary Information Section  7 , Table  S10 ). The 95% UI incorporating between-study heterogeneity was calculated from the posterior uncertainty of the fixed effects (i.e., the ‘signal’ and selected bias covariates) and the 95% quantile of the between-study heterogeneity. The estimate of between-study heterogeneity and the estimate of the uncertainty of the between-study heterogeneity were used to determine the 95% quantile of the between-study heterogeneity. Technical details of quantifying uncertainty of between-study heterogeneity are described elsewhere 32 .

Evaluating potential for publication or reporting bias

In step five, the potential for publication or reporting bias was evaluated. The trimming algorithm used in step two helps protect against these biases, so risk curves found to have publication or reporting bias using the following methods were derived from data that still had bias even after trimming. Publication or reporting bias was evaluated using Egger’s regression 34 and visual inspection using funnel plots. Egger’s regression tested for a significant correlation between residuals of the RR estimates and their standard errors. Funnel plots showed the residuals of the risk curve against their standard errors. We reported publication or reporting bias when identified.

Estimating the burden of proof risk function

In step six, the BPRF was calculated for risk-outcome relationships that were statistically significant when evaluating the conventional 95% UI without between-study heterogeneity. The BPRF is either the 5th (if harmful) or the 95th (if protective) quantile curve inclusive of between-study heterogeneity that is closest to the RR line at 1 (i.e., null); it indicates a conservative estimate of a harmful or protective association at each exposure level, based on the available evidence. The mean risk curve, 95% UIs (with and without between-study heterogeneity), and BPRF (where applicable) are visualized along with included effect sizes using the midpoint of each alternative exposure range (trimmed data points are marked with a red x), with alcohol consumption in g/day on the x-axis and (log)RR on the y-axis.

We calculated the ROS as the average log RR of the BPRF between the 15th and 85th percentiles of alcohol exposure observed in the study data. The ROS summarizes the association of the exposure with the health outcome in a single measure. A higher, positive ROS indicates a larger association, while a negative ROS indicates a weak association. The ROS is identical for protective and harmful risks since it is based on the magnitude of the log RR. For example, a mean log BPRF between the 15th and 85th percentiles of exposure of −0.6 (protective association) and a mean log BPRF of 0.6 (harmful association) would both correspond to a ROS of 0.6. The ROS was then translated into a star rating, representing a conservative interpretation of all available evidence. A star rating of 1 (ROS: <0) indicates weak evidence of an association, a star rating of 2 (ROS: 0–0.14) indicates a >0–15% increased or >0–13% decreased risk, a star rating of 3 (ROS: >0.14–0.41) indicates a >15–50% increased or >13–34% decreased risk, a star rating of 4 (ROS: >0.41–0.62) indicates a >50–85% increased or >34–46% decreased risk, and a star rating of 5 (ROS: >0.62) indicates a >85% increased or >46% decreased risk.

Statistics & reproducibility

The statistical analyses conducted in this study are described above in detail. No statistical method was used to predetermine the sample size. When analyzing data from cohort and case-control studies, we excluded 10% of observations using a trimming algorithm; when analyzing data from MR studies, we did not exclude any observations. As all data used in this meta-analysis were from observational studies, no experiments were conducted, and no randomization or blinding took place.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The findings from this study were produced using data extracted from published literature. The relevant studies were identified through a systematic literature review and can all be accessed online as referenced in the current paper 26 , 27 , 28 , 29 , 31 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 , 136 , 137 , 138 , 139 , 140 , 141 , 142 , 143 , 144 , 145 , 146 , 147 , 148 , 149 , 150 , 151 , 152 , 153 , 154 , 155 , 156 , 157 . Further details on the relevant studies can be found on the GHDx website ( https://ghdx.healthdata.org/record/ihme-data/gbd-alcohol-ihd-bop-risk-outcome-scores ). Study characteristics of all relevant studies included in the analyses are also provided in Supplementary Information Section  4 (Tables  S5 and S6 ). The template of the data collection form is provided in Supplementary Information section  3 (Table  S4 ). The source data includes processed data from these studies that underlie our estimates. Source data are provided with this paper.

Code availability

Analyses were carried out using R version 4.0.5 and Python version 3.10.9. All code used for these analyses is publicly available online ( https://github.com/ihmeuw-msca/burden-of-proof ).

Bryazka, D. et al. Population-level risks of alcohol consumption by amount, geography, age, sex, and year: a systematic analysis for the Global Burden of Disease Study 2020. Lancet 400 , 185–235 (2022).

Article   Google Scholar  

World Health Organization. Global Status Report on Alcohol and Health 2018 . (World Health Organization, Geneva, Switzerland, 2019).

Bagnardi, V. et al. Alcohol consumption and site-specific cancer risk: a comprehensive dose–response meta-analysis. Br. J. Cancer 112 , 580–593 (2015).

Article   CAS   PubMed   Google Scholar  

Wood, A. M. et al. Risk thresholds for alcohol consumption: combined analysis of individual-participant data for 599 912 current drinkers in 83 prospective studies. Lancet 391 , 1513–1523 (2018).

Article   PubMed   PubMed Central   Google Scholar  

Goel, S., Sharma, A. & Garg, A. Effect of alcohol consumption on cardiovascular health. Curr. Cardiol. Rep. 20 , 19 (2018).

Article   PubMed   Google Scholar  

Manthey, J. et al. Global alcohol exposure between 1990 and 2017 and forecasts until 2030: a modelling study. Lancet 393 , 2493–2502 (2019).

Vos, T. et al. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 396 , 1204–1222 (2020).

Brien, S. E., Ronksley, P. E., Turner, B. J., Mukamal, K. J. & Ghali, W. A. Effect of alcohol consumption on biological markers associated with risk of coronary heart disease: systematic review and meta-analysis of interventional studies. BMJ 342 , d636 (2011).

Rehm, J. et al. The relationship between different dimensions of alcohol use and the burden of disease—an update. Addiction 112 , 968–1001 (2017).

Roerecke, M. & Rehm, J. Irregular heavy drinking occasions and risk of ischemic heart disease: a systematic review and meta-analysis. Am. J. Epidemiol. 171 , 633–644 (2010).

Hernan, M. A. & Robin, J. M. Causal Inference: What If . (CRC Press, 2023).

Marmot, M. Alcohol and coronary heart disease. Int. J. Epidemiol. 13 , 160–167 (1984).

Shaper, A. G., Wannamethee, G. & Walker, M. Alcohol and mortality in British men: explaining the U-shaped curve. Lancet 332 , 1267–1273 (1988).

Davis, C. G., Thake, J. & Vilhena, N. Social desirability biases in self-reported alcohol consumption and harms. Addict. Behav. 35 , 302–311 (2010).

Mansournia, M. A., Etminan, M., Danaei, G., Kaufman, J. S. & Collins, G. Handling time varying confounding in observational research. BMJ 359 , j4587 (2017).

Ilomäki, J. et al. Relationship between alcohol consumption and myocardial infarction among ageing men using a marginal structural model. Eur. J. Public Health 22 , 825–830 (2012).

Lawlor, D. A., Harbord, R. M., Sterne, J. A. C., Timpson, N. & Davey Smith, G. Mendelian randomization: using genes as instruments for making causal inferences in epidemiology. Stat. Med. 27 , 1133–1163 (2008).

Article   MathSciNet   PubMed   Google Scholar  

Burgess, S., Timpson, N. J., Ebrahim, S. & Davey Smith, G. Mendelian randomization: where are we now and where are we going? Int. J. Epidemiol. 44 , 379–388 (2015).

Sleiman, P. M. & Grant, S. F. Mendelian randomization in the era of genomewide association studies. Clin. Chem. 56 , 723–728 (2010).

Davies, N. M., Holmes, M. V. & Davey Smith, G. Reading Mendelian randomisation studies: a guide, glossary, and checklist for clinicians. BMJ 362 , k601 (2018).

de Leeuw, C., Savage, J., Bucur, I. G., Heskes, T. & Posthuma, D. Understanding the assumptions underlying Mendelian randomization. Eur. J. Hum. Genet. 30 , 653–660 (2022).

Sheehan, N. A., Didelez, V., Burton, P. R. & Tobin, M. D. Mendelian randomisation and causal inference in observational epidemiology. PLoS Med. 5 , e177 (2008).

Van de Luitgaarden, I. A. et al. Alcohol consumption in relation to cardiovascular diseases and mortality: a systematic review of Mendelian randomization studies. Eur. J. Epidemiol. 1–15 (2021).

Edenberg, H. J. The genetics of alcohol metabolism: role of alcohol dehydrogenase and aldehyde dehydrogenase variants. Alcohol Res. Health 30 , 5–13 (2007).

PubMed   PubMed Central   Google Scholar  

Gelernter, J. et al. Genome-wide association study of maximum habitual alcohol intake in >140,000 U.S. European and African American veterans yields novel risk loci. Biol. Psychiatry 86 , 365–376 (2019).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Millwood, I. Y. et al. Conventional and genetic evidence on alcohol and vascular disease aetiology: a prospective study of 500 000 men and women in China. Lancet 393 , 1831–1842 (2019).

Au Yeung, S. L. et al. Moderate alcohol use and cardiovascular disease from Mendelian randomization. PLoS ONE 8 , e68054 (2013).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Lankester, J., Zanetti, D., Ingelsson, E. & Assimes, T. L. Alcohol use and cardiometabolic risk in the UK Biobank: a Mendelian randomization study. PLoS ONE 16 , e0255801 (2021).

Cho, Y. et al. Alcohol intake and cardiovascular risk factors: a Mendelian randomisation study. Sci. Rep. 5 , 18422 (2015).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Holmes, M. V. et al. Association between alcohol and cardiovascular disease: Mendelian randomisation analysis based on individual participant data. BMJ 349 , g4164 (2014).

Biddinger, K. J. et al. Association of habitual alcohol intake with risk of cardiovascular disease. JAMA Netw. Open 5 , e223849–e223849 (2022).

Zheng, P. et al. The Burden of Proof studies: assessing the evidence of risk. Nat. Med. 28 , 2038–2044 (2022).

Zheng, P., Barber, R., Sorensen, R. J., Murray, C. J. & Aravkin, A. Y. Trimmed constrained mixed effects models: formulations and algorithms. J. Comput. Graph. Stat. 30 , 544–556 (2021).

Article   MathSciNet   Google Scholar  

Egger, M., Smith, G. D., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 315 , 629–634 (1997).

Roerecke, M. & Rehm, J. Alcohol consumption, drinking patterns, and ischemic heart disease: a narrative review of meta-analyses and a systematic review and meta-analysis of the impact of heavy drinking occasions on risk for moderate drinkers. BMC Med. 12 , 182 (2014).

Page, M. J. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst. Rev. 10 , 89 (2021).

Stevens, G. A. et al. Guidelines for accurate and transparent health estimates reporting: the GATHER statement. PLoS Med. 13 , e1002056 (2016).

Griswold, M. G. et al. Alcohol use and burden for 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 392 , 1015–1035 (2018).

Albert, C. M. et al. Moderate alcohol consumption and the risk of sudden cardiac death among US male physicians. Circulation 100 , 944–950 (1999).

Arriola, L. et al. Alcohol intake and the risk of coronary heart disease in the Spanish EPIC cohort study. Heart 96 , 124–130 (2010).

Bazzano, L. A. et al. Alcohol consumption and risk of coronary heart disease among Chinese men. Int. J. Cardiol. 135 , 78–85 (2009).

Bell, S. et al. Association between clinically recorded alcohol consumption and initial presentation of 12 cardiovascular diseases: population based cohort study using linked health records. BMJ 356 , j909 (2017).

Bergmann, M. M. et al. The association of pattern of lifetime alcohol use and cause of death in the European prospective investigation into cancer and nutrition (EPIC) study. Int. J. Epidemiol. 42 , 1772–1790 (2013).

Beulens, J. W. J. et al. Alcohol consumption and risk for coronary heart disease among men with hypertension. Ann. Intern. Med. 146 , 10–19 (2007).

Bobak, M. et al. Alcohol, drinking pattern and all-cause, cardiovascular and alcohol-related mortality in Eastern Europe. Eur. J. Epidemiol. 31 , 21–30 (2016).

Boffetta, P. & Garfinkel, L. Alcohol drinking and mortality among men enrolled in an American Cancer Society prospective study. Epidemiology 1 , 342–348 (1990).

Britton, A. & Marmot, M. Different measures of alcohol consumption and risk of coronary heart disease and all-cause mortality: 11-year follow-up of the Whitehall II Cohort Study. Addiction 99 , 109–116 (2004).

Camargo, C. A. et al. Moderate alcohol consumption and risk for angina pectoris or myocardial infarction in U.S. male physicians. Ann. Intern. Med. 126 , 372–375 (1997).

Chang, J. Y., Choi, S. & Park, S. M. Association of change in alcohol consumption with cardiovascular disease and mortality among initial nondrinkers. Sci. Rep. 10 , 13419 (2020).

Chiuve, S. E. et al. Light-to-moderate alcohol consumption and risk of sudden cardiac death in women. Heart Rhythm 7 , 1374–1380 (2010).

Colditz, G. A. et al. Moderate alcohol and decreased cardiovascular mortality in an elderly cohort. Am. Heart J. 109 , 886–889 (1985).

Dai, J., Mukamal, K. J., Krasnow, R. E., Swan, G. E. & Reed, T. Higher usual alcohol consumption was associated with a lower 41-y mortality risk from coronary artery disease in men independent of genetic and common environmental factors: the prospective NHLBI Twin Study. Am. J. Clin. Nutr. 102 , 31–39 (2015).

Dam, M. K. et al. Five year change in alcohol intake and risk of breast cancer and coronary heart disease among postmenopausal women: prospective cohort study. BMJ 353 , i2314 (2016).

Degerud, E. et al. Associations of binge drinking with the risks of ischemic heart disease and stroke: a study of pooled Norwegian Health Surveys. Am. J. Epidemiol. 190 , 1592–1603 (2021).

de Labry, L. O. et al. Alcohol consumption and mortality in an American male population: recovering the U-shaped curve–findings from the normative Aging Study. J. Stud. Alcohol 53 , 25–32 (1992).

Doll, R., Peto, R., Boreham, J. & Sutherland, I. Mortality in relation to alcohol consumption: a prospective study among male British doctors. Int. J. Epidemiol. 34 , 199–204 (2005).

Dyer, A. R. et al. Alcohol consumption and 17-year mortality in the Chicago Western Electric Company study. Prev. Med. 9 , 78–90 (1980).

Ebbert, J. O., Janney, C. A., Sellers, T. A., Folsom, A. R. & Cerhan, J. R. The association of alcohol consumption with coronary heart disease mortality and cancer incidence varies by smoking history. J. Gen. Intern. Med. 20 , 14–20 (2005).

Ebrahim, S. et al. Alcohol dehydrogenase type 1 C (ADH1C) variants, alcohol consumption traits, HDL-cholesterol and risk of coronary heart disease in women and men: British Women’s Heart and Health Study and Caerphilly cohorts. Atherosclerosis 196 , 871–878 (2008).

Friedman, L. A. & Kimball, A. W. Coronary heart disease mortality and alcohol consumption in Framingham. Am. J. Epidemiol. 124 , 481–489 (1986).

Fuchs, F. D. et al. Association between alcoholic beverage consumption and incidence of coronary heart disease in whites and blacks: the Atherosclerosis Risk in Communities Study. Am. J. Epidemiol. 160 , 466–474 (2004).

Garfinkel, L., Boffetta, P. & Stellman, S. D. Alcohol and breast cancer: a cohort study. Prev. Med. 17 , 686–693 (1988).

Gémes, K. et al. Alcohol consumption is associated with a lower incidence of acute myocardial infarction: results from a large prospective population-based study in Norway. J. Intern. Med. 279 , 365–375 (2016).

Gigleux, I. et al. Moderate alcohol consumption is more cardioprotective in men with the metabolic syndrome. J. Nutr. 136 , 3027–3032 (2006).

Goldberg, R. J., Burchfiel, C. M., Reed, D. M., Wergowske, G. & Chiu, D. A prospective study of the health effects of alcohol consumption in middle-aged and elderly men. The Honolulu Heart Program. Circulation 89 , 651–659 (1994).

Goldberg, R. J. et al. Lifestyle and biologic factors associated with atherosclerotic disease in middle-aged men. 20-year findings from the Honolulu Heart Program. Arch. Intern. Med. 155 , 686–694 (1995).

Gordon, T. & Doyle, J. T. Drinking and coronary heart disease: the Albany Study. Am. Heart J. 110 , 331–334 (1985).

Gun, R. T., Pratt, N., Ryan, P., Gordon, I. & Roder, D. Tobacco and alcohol-related mortality in men: estimates from the Australian cohort of petroleum industry workers. Aust. N.Z. J. Public Health 30 , 318–324 (2006).

Harriss, L. R. et al. Alcohol consumption and cardiovascular mortality accounting for possible misclassification of intake: 11-year follow-up of the Melbourne Collaborative Cohort Study. Addiction 102 , 1574–1585 (2007).

Hart, C. L. & Smith, G. D. Alcohol consumption and mortality and hospital admissions in men from the Midspan collaborative cohort study. Addiction 103 , 1979–1986 (2008).

Henderson, S. O. et al. Established risk factors account for most of the racial differences in cardiovascular disease mortality. PLoS ONE 2 , e377 (2007).

Hippe, M. et al. Familial predisposition and susceptibility to the effect of other risk factors for myocardial infarction. J. Epidemiol. Community Health 53 , 269–276 (1999).

Ikehara, S. et al. Alcohol consumption and mortality from stroke and coronary heart disease among Japanese men and women: the Japan collaborative cohort study. Stroke 39 , 2936–2942 (2008).

Ikehara, S. et al. Alcohol consumption, social support, and risk of stroke and coronary heart disease among Japanese men: the JPHC Study. Alcohol. Clin. Exp. Res. 33 , 1025–1032 (2009).

Iso, H. et al. Alcohol intake and the risk of cardiovascular disease in middle-aged Japanese men. Stroke 26 , 767–773 (1995).

Jakovljević, B., Stojanov, V., Paunović, K., Belojević, G. & Milić, N. Alcohol consumption and mortality in Serbia: twenty-year follow-up study. Croat. Med. J. 45 , 764–768 (2004).

PubMed   Google Scholar  

Keil, U., Chambless, L. E., Döring, A., Filipiak, B. & Stieber, J. The relation of alcohol intake to coronary heart disease and all-cause mortality in a beer-drinking population. Epidemiology 8 , 150–156 (1997).

Key, T. J. et al. Mortality in British vegetarians: results from the European Prospective Investigation into Cancer and Nutrition (EPIC-Oxford). Am. J. Clin. Nutr. 89 , 1613S–1619S (2009).

Kitamura, A. et al. Alcohol intake and premature coronary heart disease in urban Japanese men. Am. J. Epidemiol. 147 , 59–65 (1998).

Kivelä, S. L. et al. Alcohol consumption and mortality in aging or aged Finnish men. J. Clin. Epidemiol. 42 , 61–68 (1989).

Klatsky, A. L. et al. Alcohol drinking and risk of hospitalization for heart failure with and without associated coronary artery disease. Am. J. Cardiol. 96 , 346–351 (2005).

Kono, S., Ikeda, M., Tokudome, S., Nishizumi, M. & Kuratsune, M. Alcohol and mortality: a cohort study of male Japanese physicians. Int. J. Epidemiol. 15 , 527–532 (1986).

Kunutsor, S. K. et al. Self-reported alcohol consumption, carbohydrate deficient transferrin and risk of cardiovascular disease: the PREVEND prospective cohort study. Clin. Chim. Acta 520 , 1–7 (2021).

Kurl, S., Jae, S. Y., Voutilainen, A. & Laukkanen, J. A. The combined effect of blood pressure and C-reactive protein with the risk of mortality from coronary heart and cardiovascular diseases. Nutr. Metab. Cardiovasc. Dis. 31 , 2051–2057 (2021).

Larsson, S. C., Wallin, A. & Wolk, A. Contrasting association between alcohol consumption and risk of myocardial infarction and heart failure: two prospective cohorts. Int. J. Cardiol. 231 , 207–210 (2017).

Lazarus, N. B., Kaplan, G. A., Cohen, R. D. & Leu, D. J. Change in alcohol consumption and risk of death from all causes and from ischaemic heart disease. BMJ 303 , 553–556 (1991).

Lee, D.-H., Folsom, A. R. & Jacobs, D. R. Dietary iron intake and Type 2 diabetes incidence in postmenopausal women: the Iowa Women’s Health Study. Diabetologia 47 , 185–194 (2004).

Liao, Y., McGee, D. L., Cao, G. & Cooper, R. S. Alcohol intake and mortality: findings from the National Health Interview Surveys (1988 and 1990). Am. J. Epidemiol. 151 , 651–659 (2000).

Licaj, I. et al. Alcohol consumption over time and mortality in the Swedish Women’s Lifestyle and Health cohort. BMJ Open 6 , e012862 (2016).

Lindschou Hansen, J. et al. Alcohol intake and risk of acute coronary syndrome and mortality in men and women with and without hypertension. Eur. J. Epidemiol. 26 , 439–447 (2011).

Makelä, P., Paljärvi, T. & Poikolainen, K. Heavy and nonheavy drinking occasions, all-cause and cardiovascular mortality and hospitalizations: a follow-up study in a population with a low consumption level. J. Stud. Alcohol 66 , 722–728 (2005).

Malyutina, S. et al. Relation between heavy and binge drinking and all-cause and cardiovascular mortality in Novosibirsk, Russia: a prospective cohort study. Lancet 360 , 1448–1454 (2002).

Maraldi, C. et al. Impact of inflammation on the relationship among alcohol consumption, mortality, and cardiac events: the health, aging, and body composition study. Arch. Intern. Med. 166 , 1490–1497 (2006).

Marques-Vidal, P. et al. Alcohol consumption and cardiovascular disease: differential effects in France and Northern Ireland. The PRIME study. Eur. J. Cardiovasc. Prev. Rehabil. 11 , 336–343 (2004).

Meisinger, C., Döring, A., Schneider, A., Löwel, H. & KORA Study Group. Serum gamma-glutamyltransferase is a predictor of incident coronary events in apparently healthy men from the general population. Atherosclerosis 189 , 297–302 (2006).

Merry, A. H. H. et al. Smoking, alcohol consumption, physical activity, and family history and the risks of acute myocardial infarction and unstable angina pectoris: a prospective cohort study. BMC Cardiovasc. Disord. 11 , 13 (2011).

Miller, G. J., Beckles, G. L., Maude, G. H. & Carson, D. C. Alcohol consumption: protection against coronary heart disease and risks to health. Int. J. Epidemiol. 19 , 923–930 (1990).

Mukamal, K. J., Chiuve, S. E. & Rimm, E. B. Alcohol consumption and risk for coronary heart disease in men with healthy lifestyles. Arch. Intern. Med. 166 , 2145–2150 (2006).

Ng, R., Sutradhar, R., Yao, Z., Wodchis, W. P. & Rosella, L. C. Smoking, drinking, diet and physical activity-modifiable lifestyle risk factors and their associations with age to first chronic disease. Int. J. Epidemiol. 49 , 113–130 (2020).

Onat, A. et al. Moderate and heavy alcohol consumption among Turks: long-term impact on mortality and cardiometabolic risk. Arch. Turkish Soc. Cardiol. 37 , 83–90 (2009).

Google Scholar  

Pedersen, J. Ø., Heitmann, B. L., Schnohr, P. & Grønbaek, M. The combined influence of leisure-time physical activity and weekly alcohol intake on fatal ischaemic heart disease and all-cause mortality. Eur. Heart J. 29 , 204–212 (2008).

Reddiess, P. et al. Alcohol consumption and risk of cardiovascular outcomes and bleeding in patients with established atrial fibrillation. Can. Med. Assoc. J. 193 , E117–E123 (2021).

Article   CAS   Google Scholar  

Rehm, J. T., Bondy, S. J., Sempos, C. T. & Vuong, C. V. Alcohol consumption and coronary heart disease morbidity and mortality. Am. J. Epidemiol. 146 , 495–501 (1997).

Renaud, S. C., Guéguen, R., Schenker, J. & d’Houtaud, A. Alcohol and mortality in middle-aged men from eastern France. Epidemiology 9 , 184–188 (1998).

Ricci, C. et al. Alcohol intake in relation to non-fatal and fatal coronary heart disease and stroke: EPIC-CVD case-cohort study. BMJ 361 , k934 (2018).

Rimm, E. B. et al. Prospective study of alcohol consumption and risk of coronary disease in men. Lancet 338 , 464–468 (1991).

Roerecke, M. et al. Heavy drinking occasions in relation to ischaemic heart disease mortality– an 11-22 year follow-up of the 1984 and 1995 US National Alcohol Surveys. Int. J. Epidemiol. 40 , 1401–1410 (2011).

Romelsjö, A., Allebeck, P., Andréasson, S. & Leifman, A. Alcohol, mortality and cardiovascular events in a 35 year follow-up of a nationwide representative cohort of 50,000 Swedish conscripts up to age 55. Alcohol Alcohol. 47 , 322–327 (2012).

Rostron, B. Alcohol consumption and mortality risks in the USA. Alcohol Alcohol. 47 , 334–339 (2012).

Ruidavets, J.-B. et al. Patterns of alcohol consumption and ischaemic heart disease in culturally divergent countries: the Prospective Epidemiological Study of Myocardial Infarction (PRIME). BMJ 341 , c6077 (2010).

Schooling, C. M. et al. Moderate alcohol use and mortality from ischaemic heart disease: a prospective study in older Chinese people. PLoS ONE 3 , e2370 (2008).

Schutte, R., Smith, L. & Wannamethee, G. Alcohol - The myth of cardiovascular protection. Clin. Nutr. 41 , 348–355 (2022).

Sempos, C., Rehm, J., Crespo, C. & Trevisan, M. No protective effect of alcohol consumption on coronary heart disease (CHD) in African Americans: average volume of drinking over the life course and CHD morbidity and mortality in a U.S. national cohort. Contemp. Drug Probl. 29 , 805–820 (2002).

Shaper, A. G., Wannamethee, G. & Walker, M. Alcohol and coronary heart disease: a perspective from the British Regional Heart Study. Int. J. Epidemiol. 23 , 482–494 (1994).

Simons, L. A., McCallum, J., Friedlander, Y. & Simons, J. Alcohol intake and survival in the elderly: a 77 month follow-up in the Dubbo study. Aust. N.Z. J. Med. 26 , 662–670 (1996).

Skov-Ettrup, L. S., Eliasen, M., Ekholm, O., Grønbæk, M. & Tolstrup, J. S. Binge drinking, drinking frequency, and risk of ischaemic heart disease: a population-based cohort study. Scand. J. Public Health 39 , 880–887 (2011).

Snow, W. M., Murray, R., Ekuma, O., Tyas, S. L. & Barnes, G. E. Alcohol use and cardiovascular health outcomes: a comparison across age and gender in the Winnipeg Health and Drinking Survey Cohort. Age Ageing 38 , 206–212 (2009).

Song, R. J. et al. Alcohol consumption and risk of coronary artery disease (from the Million Veteran Program). Am. J. Cardiol. 121 , 1162–1168 (2018).

Streppel, M. T., Ocké, M. C., Boshuizen, H. C., Kok, F. J. & Kromhout, D. Long-term wine consumption is related to cardiovascular mortality and life expectancy independently of moderate alcohol intake: the Zutphen Study. J. Epidemiol. Community Health 63 , 534–540 (2009).

Suhonen, O., Aromaa, A., Reunanen, A. & Knekt, P. Alcohol consumption and sudden coronary death in middle-aged Finnish men. Acta Med. Scand. 221 , 335–341 (1987).

Thun, M. J. et al. Alcohol consumption and mortality among middle-aged and elderly U.S. adults. N. Engl. J. Med. 337 , 1705–1714 (1997).

Tolstrup, J. et al. Prospective study of alcohol drinking patterns and coronary heart disease in women and men. BMJ 332 , 1244–1248 (2006).

Wannamethee, G. & Shaper, A. G. Alcohol and sudden cardiac death. Br. Heart J. 68 , 443–448 (1992).

Wannamethee, S. G. & Shaper, A. G. Type of alcoholic drink and risk of major coronary heart disease events and all-cause mortality. Am. J. Public Health 89 , 685–690 (1999).

Wilkins, K. Moderate alcohol consumption and heart disease. Health Rep. 14 , 9–24 (2002).

Yang, L. et al. Alcohol drinking and overall and cause-specific mortality in China: nationally representative prospective study of 220,000 men with 15 years of follow-up. Int. J. Epidemiol. 41 , 1101–1113 (2012).

Yi, S. W., Yoo, S. H., Sull, J. W. & Ohrr, H. Association between alcohol drinking and cardiovascular disease mortality and all-cause mortality: Kangwha Cohort Study. J. Prev. Med. Public Health 37 , 120–126 (2004).

Younis, J., Cooper, J. A., Miller, G. J., Humphries, S. E. & Talmud, P. J. Genetic variation in alcohol dehydrogenase 1C and the beneficial effect of alcohol intake on coronary heart disease risk in the Second Northwick Park Heart Study. Atherosclerosis 180 , 225–232 (2005).

Yusuf, S. et al. Modifiable risk factors, cardiovascular disease, and mortality in 155 722 individuals from 21 high-income, middle-income, and low-income countries (PURE): a prospective cohort study. Lancet 395 , 795–808 (2020).

Zhang, Y. et al. Association of drinking pattern with risk of coronary heart disease incidence in the middle-aged and older Chinese men: results from the Dongfeng-Tongji cohort. PLoS ONE 12 , e0178070 (2017).

Augustin, L. S. A. et al. Alcohol consumption and acute myocardial infarction: a benefit of alcohol consumed with meals? Epidemiology 15 , 767–769 (2004).

Bianchi, C., Negri, E., La Vecchia, C. & Franceschi, S. Alcohol consumption and the risk of acute myocardial infarction in women. J. Epidemiol. Community Health 47 , 308–311 (1993).

Brenner, H. et al. Coronary heart disease risk reduction in a predominantly beer-drinking population. Epidemiology 12 , 390–395 (2001).

Dorn, J. M. et al. Alcohol drinking pattern and non-fatal myocardial infarction in women. Addiction 102 , 730–739 (2007).

Fan, A. Z., Ruan, W. J. & Chou, S. P. Re-examining the relationship between alcohol consumption and coronary heart disease with a new lens. Prev. Med. 118 , 336–343 (2019).

Fumeron, F. et al. Alcohol intake modulates the effect of a polymorphism of the cholesteryl ester transfer protein gene on plasma high density lipoprotein and the risk of myocardial infarction. J. Clin. Investig. 96 , 1664–1671 (1995).

Gaziano, J. M. et al. Moderate alcohol intake, increased levels of high-density lipoprotein and its subfractions, and decreased risk of myocardial infarction. N. Engl. J. Med. 329 , 1829–1834 (1993).

Genchev, G. D., Georgieva, L. M., Weijenberg, M. P. & Powles, J. W. Does alcohol protect against ischaemic heart disease in Bulgaria? A case-control study of non-fatal myocardial infarction in Sofia. Cent. Eur. J. Public Health 9 , 83–86 (2001).

CAS   PubMed   Google Scholar  

Hammar, N., Romelsjö, A. & Alfredsson, L. Alcohol consumption, drinking pattern and acute myocardial infarction. A case referent study based on the Swedish Twin Register. J. Intern. Med. 241 , 125–131 (1997).

Hines, L. M. et al. Genetic variation in alcohol dehydrogenase and the beneficial effect of moderate alcohol consumption on myocardial infarction. N. Engl. J. Med. 344 , 549–555 (2001).

Ilic, M., Grujicic Sipetic, S., Ristic, B. & Ilic, I. Myocardial infarction and alcohol consumption: a case-control study. PLoS ONE 13 , e0198129 (2018).

Jackson, R., Scragg, R. & Beaglehole, R. Alcohol consumption and risk of coronary heart disease. BMJ 303 , 211–216 (1991).

Kabagambe, E. K., Baylin, A., Ruiz-Narvaez, E., Rimm, E. B. & Campos, H. Alcohol intake, drinking patterns, and risk of nonfatal acute myocardial infarction in Costa Rica. Am. J. Clin. Nutr. 82 , 1336–1345 (2005).

Kalandidi, A. et al. A case-control study of coronary heart disease in Athens, Greece. Int. J. Epidemiol. 21 , 1074–1080 (1992).

Kaufman, D. W., Rosenberg, L., Helmrich, S. P. & Shapiro, S. Alcoholic beverages and myocardial infarction in young men. Am. J. Epidemiol. 121 , 548–554 (1985).

Kawanishi, M., Nakamoto, A., Konemori, G., Horiuchi, I. & Kajiyama, G. Coronary sclerosis risk factors in males with special reference to lipoproteins and apoproteins: establishing an index. Hiroshima J. Med. Sci. 39 , 61–64 (1990).

Kono, S. et al. Alcohol intake and nonfatal acute myocardial infarction in Japan. Am. J. Cardiol. 68 , 1011–1014 (1991).

Mehlig, K. et al. CETP TaqIB genotype modifies the association between alcohol and coronary heart disease: the INTERGENE case-control study. Alcohol 48 , 695–700 (2014).

Miyake, Y. Risk factors for non-fatal acute myocardial infarction in middle-aged and older Japanese. Fukuoka Heart Study Group. Jpn. Circ. J. 64 , 103–109 (2000).

Oliveira, A., Barros, H., Azevedo, A., Bastos, J. & Lopes, C. Impact of risk factors for non-fatal acute myocardial infarction. Eur. J. Epidemiol. 24 , 425–432 (2009).

Oliveira, A., Barros, H. & Lopes, C. Gender heterogeneity in the association between lifestyles and non-fatal acute myocardial infarction. Public Health Nutr. 12 , 1799–1806 (2009).

Romelsjö, A. et al. Abstention, alcohol use and risk of myocardial infarction in men and women taking account of social support and working conditions: the SHEEP case-control study. Addiction 98 , 1453–1462 (2003).

Schröder, H. et al. Myocardial infarction and alcohol consumption: a population-based case-control study. Nutr. Metab. Cardiovasc. Dis. 17 , 609–615 (2007).

Scragg, R., Stewart, A., Jackson, R. & Beaglehole, R. Alcohol and exercise in myocardial infarction and sudden coronary death in men and women. Am. J. Epidemiol. 126 , 77–85 (1987).

Tavani, A., Bertuzzi, M., Gallus, S., Negri, E. & La Vecchia, C. Risk factors for non-fatal acute myocardial infarction in Italian women. Prev. Med. 39 , 128–134 (2004).

Tavani, A. et al. Intake of specific flavonoids and risk of acute myocardial infarction in Italy. Public Health Nutr. 9 , 369–374 (2006).

Zhou, X., Li, C., Xu, W., Hong, X. & Chen, J. Relation of alcohol consumption to angiographically proved coronary artery disease in chinese men. Am. J. Cardiol. 106 , 1101–1103 (2010).

Yang, Y. et al. Alcohol consumption and risk of coronary artery disease: a dose-response meta-analysis of prospective studies. Nutrition 32 , 637–644 (2016).

Zheng, J. et al. Recent developments in Mendelian randomization studies. Curr. Epidemiol. Rep. 4 , 330–345 (2017).

Mukamal, K. J., Stampfer, M. J. & Rimm, E. B. Genetic instrumental variable analysis: time to call Mendelian randomization what it is. The example of alcohol and cardiovascular disease. Eur. J. Epidemiol. 35 , 93–97 (2020).

Verbanck, M., Chen, C.-Y., Neale, B. & Do, R. Detection of widespread horizontal pleiotropy in causal relationships inferred from Mendelian randomization between complex traits and diseases. Nat. Genet. 50 , 693–698 (2018).

Shi, J. et al. Mendelian randomization with repeated measures of a time-varying exposure: an application of structural mean models. Epidemiology 33 , 84–94 (2022).

Burgess, S., Swanson, S. A. & Labrecque, J. A. Are Mendelian randomization investigations immune from bias due to reverse causation? Eur. J. Epidemiol. 36 , 253–257 (2021).

Davey Smith, G., Holmes, M. V., Davies, N. M. & Ebrahim, S. Mendel’s laws, Mendelian randomization and causal inference in observational data: substantive and nomenclatural issues. Eur. J. Epidemiol. 35 , 99–111 (2020).

Burgess, S. Violation of the constant genetic effect assumption can result in biased estimates for non-linear mendelian randomization. Hum. Hered. 88 , 79–90 (2023).

Tian, H., Mason, A. M., Liu, C. & Burgess, S. Relaxing parametric assumptions for non-linear Mendelian randomization using a doubly-ranked stratification method. PLoS Genet. 19 , e1010823 (2023).

Levin, M. G. & Burgess, S. Mendelian randomization as a tool for cardiovascular research: a review. JAMA Cardiol. 9 , 79–89 (2024).

Burgess, S. et al. Guidelines for performing Mendelian randomization investigations: update for summer 2023. Wellcome Open Res. 4 , 186 (2019).

Bowden, J., Davey Smith, G., Haycock, P. C. & Burgess, S. Consistent estimation in Mendelian randomization with some invalid instruments using a weighted median estimator. Genet. Epidemiol. 40 , 304–314 (2016).

Holmes, M. V., Ala-Korpela, M. & Smith, G. D. Mendelian randomization in cardiometabolic disease: challenges in evaluating causality. Nat. Rev. Cardiol. 14 , 577–590 (2017).

Labrecque, J. A. & Swanson, S. A. Interpretation and potential biases of Mendelian randomization estimates with time-varying exposures. Am. J. Epidemiol. 188 , 231–238 (2019).

Spiegelman, D. et al. The Moderate Alcohol and Cardiovascular Health Trial (MACH15): design and methods for a randomized trial of moderate alcohol consumption and cardiometabolic risk. Eur. J. Prev. Cardiol. 27 , 1967–1982 (2020).

DeJong, W. The Moderate Alcohol and Cardiovascular Health Trial: public health advocates should support good science, not undermine it. Eur. J. Prev. Cardiol. 28 , e22–e24 (2021).

National Institutes of Health. NIH to End Funding for Moderate Alcohol and Cardiovascular Health Trial https://www.nih.gov/news-events/news-releases/nih-end-funding-moderate-alcohol-cardiovascular-health-trial (2018).

Miller, L. M., Anderson, C. A. M. & Ix, J. H. Editorial: from MACH15 to MACH0 – a missed opportunity to understand the health effects of moderate alcohol intake. Eur. J. Prev. Cardiol. 28 , e23–e24 (2021).

Anderson, B. O. et al. Health and cancer risks associated with low levels of alcohol consumption. Lancet Public Health 8 , e6–e7 (2023).

Au Yeung, S. L. & Lam, T. H. Unite for a framework convention for alcohol control. Lancet 393 , 1778–1779 (2019).

Hernán, M. A. & Robins, J. M. Using big data to emulate a target trial when a randomized trial is not available. Am. J. Epidemiol. 183 , 758–764 (2016).

Sudlow, C. et al. UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 12 , e1001779 (2015).

The ARIC Investigators. The Atherosclerosis Risk in Communit (ARIC) study: design and objectives. Am. J. Epidemiol. 129 , 687–702 (1989).

Mahmood, S. S., Levy, D., Vasan, R. S. & Wang, T. J. The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective. Lancet 383 , 999–1008 (2014).

Gmel, G. & Rehm, J. Measuring alcohol consumption. Contemp. Drug Probl. 31 , 467–540 (2004).

Guyatt, G. H. et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336 , 924–926 (2008).

Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58 , 267–288 (1996).

Biggerstaff, B. J. & Tweedie, R. L. Incorporating variability in estimates of heterogeneity in the random effects model in meta‐analysis. Stat. Med. 16 , 753–768 (1997).

Download references

Acknowledgements

Research reported in this publication was supported by the Bill & Melinda Gates Foundation [OPP1152504]. S.L. has received grants or contracts from the UK Medical Research Council [MR/T017708/1], CDC Foundation [project number 996], World Health Organization [APW No 2021/1194512], and is affiliated with the NIHR Oxford Biomedical Research Centre. The University of Oxford’s Clinical Trial Service Unit and Epidemiological Studies Unit (CTSU) is supported by core grants from the Medical Research Council [Clinical Trial Service Unit A310] and the British Heart Foundation [CH/1996001/9454]. The CTSU receives research grants from industry that are governed by University of Oxford contracts that protect its independence and has a staff policy of not taking personal payments from industry. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders. The funders of the study had no role in study design, data collection, data analysis, data interpretation, writing of the final report, or the decision to publish.

Author information

Authors and affiliations.

Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA, USA

Sinclair Carr, Dana Bryazka, Susan A. McLaughlin, Peng Zheng, Aleksandr Y. Aravkin, Simon I. Hay, Hilary R. Lawlor, Erin C. Mullany, Christopher J. L. Murray, Sneha I. Nicholson, Gregory A. Roth, Reed J. D. Sorensen & Emmanuela Gakidou

Department of Health Metrics Sciences, School of Medicine, University of Washington, Seattle, WA, USA

Peng Zheng, Aleksandr Y. Aravkin, Simon I. Hay, Christopher J. L. Murray, Gregory A. Roth & Emmanuela Gakidou

Clinical Trial Service Unit & Epidemiological Studies Unit, Nuffield Department of Population Health, University of Oxford, Oxford, Oxfordshire, UK

Sarasvati Bahadursingh & Sarah Lewington

Department of Applied Mathematics, University of Washington, Seattle, WA, USA

Aleksandr Y. Aravkin

Institute for Mental Health Policy Research, Centre for Addiction and Mental Health, Toronto, ON, Canada

Jürgen Rehm

Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, ON, Canada

Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

Department of Psychiatry, University of Toronto, Toronto, ON, Canada

Faculty of Medicine, Institute of Medical Science (IMS), University of Toronto, Toronto, ON, Canada

World Health Organization / Pan American Health Organization Collaborating Centre, Centre for Addiction and Mental Health, Toronto, ON, Canada

Center for Interdisciplinary Addiction Research (ZIS), Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany

Institute of Clinical Psychology and Psychotherapy, Technische Universität Dresden, Dresden, Germany

Division of Cardiology, Department of Medicine, University of Washington, Seattle, WA, USA

Gregory A. Roth

You can also search for this author in PubMed   Google Scholar

Contributions

S.C., S.A.M., S.I.H., and E.C.M. managed the estimation or publications process. S.C. wrote the first draft of the manuscript. S.C. had primary responsibility for applying analytical methods to produce estimates. S.C. and H.R.L. had primary responsibility for seeking, cataloging, extracting, or cleaning data; designing or coding figures and tables. S.C., D.B., S.B., E.C.M., S.I.N., J.R., and R.J.D.S. provided data or critical feedback on data sources. S.C., D.B., P.Z., A.Y.A., S.I.N., and R.J.D.S. developed methods or computational machinery. S.C., D.B., P.Z., S.B., S.I.H., E.C.M., C.J.L.M., S.I.N., J.R., R.J.D.S., S.L., and E.G. provided critical feedback on methods or results. S.C., D.B., S.A.M., S.B., S.I.H., C.J.L.M., J.R., G.A.R., S.L., and E.G. drafted the work or revised it critically for important intellectual content. S.C., S.I.H., E.C.M., and E.G. managed the overall research enterprise.

Corresponding author

Correspondence to Sinclair Carr .

Ethics declarations

Competing interests.

G.A.R. has received support for this manuscript from the Bill and Melinda Gates Foundation [OPP1152504]. S.L. has received grants or contracts from the UK Medical Research Council [MR/T017708/1], CDC Foundation [project number 996], World Health Organization [APW No 2021/1194512], and is affiliated with the NIHR Oxford Biomedical Research Centre. The University of Oxford’s Clinical Trial Service Unit and Epidemiological Studies Unit (CTSU) is supported by core grants from the Medical Research Council [Clinical Trial Service Unit A310] and the British Heart Foundation [CH/1996001/9454]. The CTSU receives research grants from industry that are governed by University of Oxford contracts that protect its independence and has a staff policy of not taking personal payments from industry. All other authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Shiu Lun Au Yeung, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Carr, S., Bryazka, D., McLaughlin, S.A. et al. A burden of proof study on alcohol consumption and ischemic heart disease. Nat Commun 15 , 4082 (2024). https://doi.org/10.1038/s41467-024-47632-7

Download citation

Received : 14 June 2023

Accepted : 08 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1038/s41467-024-47632-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Open access
• Published: 10 May 2024

Intraspecific and interspecific variations in the synonymous codon usage in mitochondrial genomes of 8 pleurotus strains

• Wei Gao 1 ,
• Xiaodie Chen 2 ,
• Jing He 2 ,
• Ajia Sha 2 ,
• Yingyong Luo 2 ,
• Wenqi Xiao 2 ,
• Zhuang Xiong 2 &
• Qiang Li 2 , 3  

BMC Genomics volume  25 , Article number:  456 ( 2024 ) Cite this article

152 Accesses

Metrics details

In this study, we investigated the codon bias of twelve mitochondrial core protein coding genes (PCGs) in eight Pleurotus strains, two of which are from the same species. The results revealed that the codons of all Pleurotus strains had a preference for ending in A/T. Furthermore, the correlation between codon base compositions and codon adaptation index (CAI), codon bias index (CBI) and frequency of optimal codons (FOP) indices was also detected, implying the influence of base composition on codon bias. The two P. ostreatus species were found to have differences in various base bias indicators. The average effective number of codons (ENC) of mitochondrial core PCGs of Pleurotus was found to be less than 35, indicating strong codon preference of mitochondrial core PCGs of Pleurotus . The neutrality plot analysis and PR2-Bias plot analysis further suggested that natural selection plays an important role in Pleurotus codon bias. Additionally, six to ten optimal codons (ΔRSCU > 0.08 and RSCU > 1) were identified in eight Pleurotus strains, with UGU and ACU being the most widely used optimal codons in Pleurotus . Finally, based on the combined mitochondrial sequence and RSCU value, the genetic relationship between different Pleurotus strains was deduced, showing large variations between them. This research has improved our understanding of synonymous codon usage characteristics and evolution of this important fungal group.

Peer Review reports

Introduction

Codon bias indicates the non-uniform or biased usage of synonymous codons that encode the same amino acid in a gene or genome [ 1 ]. The genetic information contained in DNA is transferred to the sequence of 20 amino acids through transcription and translation steps [ 2 ]. Among the 64 triplet codon arrangements contained in DNA, 61 triplets can encode 20 standard amino acids, while the other three are translation termination codons. Among the 20 amino acids encoded, 18 amino acids are encoded by multiple different codons, while tryptophan and methionine are encoded by only one codon in most species. The degeneration of the genetic code allows the same amino acid to be encoded by synonymous codons or different codons [ 3 , 4 ]. However, in most cases, the probability of synonymous codons being used is not random or equal. This common phenomenon is called codon usage bias (CUB) [ 5 , 6 , 7 ]. This phenomenon of synonymous codons appearing with different frequencies is often observed in different genes, different organisms, or even the same gene from different species [ 8 , 9 , 10 ]. CUB is mainly caused by mutations in the gene coding region, especially mutations in the second or third nucleotides of the codon in the gene coding region [ 11 , 12 , 13 ]. A synonymy mutation or “silent mutation” will lead to the variability of synonymous codons in organisms during evolution [ 14 , 15 ]. Since some codons are more prone to mutation than others, selection can sustain this bias [ 16 ]. As a result of GC heterogeneity and GC biased gene transformation (gBGC), codon usage bias may also be a result of local recombination rate-based codon usage bias [ 17 , 18 , 19 ]. Consequently, synonymous codons evolve through a combination of mutation, natural selection, and genetic drift of gene translation efficiency, which may play a significant role in genome evolution [ 20 , 21 ]. There is a mutation mechanism that explains the interspecific differences in codon usage by explaining codon bias by the rate or repair of nucleotide bias or point mutations [ 22 , 23 ]. Furthermore, the theory of natural selection assumes that synonymous mutations that affect biological adaptability will be favored or suppressed throughout the evolutionary process, leading to changes in the use of codons in genomes or genes [ 24 , 25 ].

Several cellular processes can be affected by codon bias, including transcription, translation efficiency and accuracy, mRNA stability, protein expression, structure, function, and folding during cotranslation [ 26 , 27 , 28 ]. Codon bias affects transcription by altering chromatin structure and mRNA folding, which then affects translation efficiency by affecting translation elongation rate [ 29 , 30 ]. Therefore, codon bias arises as the result of genome adaptation to transcription and translation mechanisms. The study of molecular evolution of genes benefits from selecting genes that do not change amino acids. The codon bias analysis can reveal evolutionary relationships between closely related organisms because codons are used similarly by closely related organisms [ 31 , 32 ]. Most highly expressed proteins are encoded by genes with the best codons. With the rapid development of high-throughput sequencing technology, codon bias analysis is now essential for understanding species evolution, environmental adaptation, and genetics, etc [ 33 , 34 , 35 ]. In fungal species, particularly in large higher fungi, however, genetic characteristics of codon bias remain unknown [ 36 ].

Pleurotus is one of the largest cultivated edible fungi in the world, which has rich species diversity [ 37 , 38 , 39 , 40 ]. Some species of Pleurotus are delicious edible fungi, which are widely welcomed by consumers [ 37 , 38 , 41 ]. In addition, Pleurotus species also contain a variety of bioactive ingredients, with anti-tumor, antioxidant, anti-inflammatory, anti-virus and other effects [ 42 , 43 , 44 , 45 ]. Mitochondrial genome, known as the second genome of eukaryotes, plays an important role in maintaining the energy supply of eukaryotic cells [ 46 ]. Most fungi have 15 core protein coding genes (PCGs), including atp6 , atp8 , atp9 , cob , cox1 , cox2 , cox3 , nad1 , nad2 , nad3 , nad4 , nad4L , nad5 , nad6 , and rps3 [ 47 , 48 ]. The variation of mitochondrial genome has an important impact on the homeostasis, stress resistance and tolerance, development of eukaryotic cells [ 49 , 50 , 51 ]. Our previous research found that the mitochondrial genomes of different Pleurotus species had undergone large-scale gene rearrangement, indicating that Pleurotus species had undergone significant genetic differentiation [ 52 ]. However, the codon bias, genetic characteristics, and evolution of the mitochondrial core PCGs of Pleurotus within and between species are still unknown.

In this study, we analyzed and compared the usage characteristics of synonymous codons of mitochondrial core PCGs within and between 8 Pleurotus strains, including P. citrinopileatus , P. cornucopiae , P. eryngii , P. giganteus , P. ostreatus P51, P. ostreatus , P. platypus , and P. pulmonarius . We also deduced the phylogenetic relationship of different Pleurotus strains based on relative synonymous codon usage (RSCU) data and compared it with the phylogenetic relationship based on mitochondrial genome sequence inference. This study is the first report to analyze the intraspecific and interspecific synonymous codon usage characteristics of important cultivated edible fungi, which will promote the understanding of the evolution, genetics, and species differentiation of Pleurotus species and other related species.

Materials and methods

Sequence processing.

A total of 8 complete Pleurotus mitochondrial genomes have been published in the National Center for Biotechnology Information (NCBI) database, 2 of which were reported by our previous studies [ 52 ]. The 8 Pleurotus mitochondrial genomes were first downloaded from the NCBI database under the accession numbers NC_036998, NC_038091, NC_033533, NC_062374, OX344747, NC_009905, NC_036999, and NC_061177 [ 53 , 54 , 55 , 56 , 57 ]. We further obtained the core protein coding sequence of the mitochondrial genomes of 8 Pleurotus strains. Those core protein coding genes whose sequence length is less than 300 bp were excluded from subsequent analysis [ 14 ]. Finally, we obtained 12 core protein coding genes in each Pleurotus strains for subsequent analysis, including atp6 , cob , cox1 , cox2 , cox3 , nad1 , nad2 , nad3 , nad4 , nad5 , nad6 , and rps3 .

Codon usage indices

The GC3s parameter is used to measure the amount of codons with guanine and cytosine at the third synonymous position, with the exception of Met, Trp, and termination codons [ 58 ]. The third base of a codon, which is often the least conserved and most variable position. The codon adaptation index (CAI) is a measure of the bias towards codons that are commonly found in highly expressed genes [ 59 ]. CAI reflects the adaptation of a gene’s codon usage to the tRNA pool of the organism, which affects translational efficiency. It is a numerical value ranging from 0 to 1.0, with larger values indicating a greater frequency of synonymous codon usage. The Codon Bias Index (CBI) is a metric for evaluating gene expression, which quantifies the deviation from a random or uniform distribution of codons encoding the same amino acid. The Frequency of Optimal Codons (FOP) is determined by dividing the amount of optimal codons by the total number of synonymous codons in a gene, which provides a direct measure of how often a gene uses the “best” or most efficiently translated codons. The Effective Number of Codons (ENC) is a measure of the number of codons used in a gene, ranging from 20 to 61. A value of 20 indicates that only one codon is used for each amino acid, while 61 indicates that each codon is used on average. A low ENC value (below 35) indicates a strong codon usage preference, while a higher value (above 35) indicates a weak preference. The Relative Synonymous Codon Usage (RSCU) value is calculated by dividing the amino acids encoded by the same codons and their probability of appearing in the same codons, which provides a direct comparison of codon usage across genes or species, accounting for differences in codon composition due to amino acid composition. A value greater than 1 indicates a positive codon bias, while a value less than 1 indicates a negative codon bias. The General Average Hydropathicity (GRAVY) value is determined by summing the hydropathy values of all of the amino acids in the polymerase gene sequences and multiplying them by the number of residues in the gene sequences, which provides insights into the potential membrane-spanning or intracellular localization of a protein. GRAVY values range from − 2 to 2, with positive and negative values representing hydrophobic and hydrophilic proteins, respectively. The Aromaticity (AROMO) value is an indicator of the frequency of aromatic amino acids (Phe, Tyr, and Trp). Aromatic amino acids have a unique chemical structure that confers stability and specific interactions with other molecules. The aromaticity of a protein can affect its structure, function, and interactions with other molecules. GRAVY and AROMO values are also indicators of amino acid usage, and changes in amino acid composition will also affect the results of codon usage analysis. All of these codon usage indicators can be calculated using CodonW1.4.2 [ 60 ] or CAIcal server [ 61 ].

Neutrality plot analysis

The neutrality plot (GC12 vs. GC3) can be used to analyze the balance between mutation and selection when codon bias is formed. GC12 represents the average GC content in the first and second positions of the codon (GC1 and GC2), while GC3 represents the GC content in the third position. Neutral evolution theory assumes that mutations occur randomly and have no effect on the fitness of the organism. However, selection pressure can introduce biases in the observed mutation frequencies, leading to deviations from neutrality [ 62 ]. A strong statistical correlation between GC12 and GC3 indicates that the species is mainly driven by mutation, whereas a lack of correlation implies the main driving force is natural selection.

ENC-GC3s plot analysis

The ENC-GC3s plot (ENC vs. GC3s) is typically employed to assess whether the codon usage of a particular gene is impacted solely by mutation or other factors, such as natural selection. This diagram consists of the ordinate ENC value and abscissa GC3s value, with an expected curve calculated via a specific formula [ 63 ]. If the corresponding points are distributed around the expected curve, mutation pressure is an independent force in the formation of codon bias. However, if the points are significantly lower or distant from the expected curve, some other factors, such as natural selection, likely play a key role in the formation of codon bias.

The ENC Ratio value reflects the variation range between the expected value and the actual value of ENC.

PR2-Bias plot analysis

Additionally, the Parity Rule 2 bias (PR2-Bias) plot analysis based on [A3/(A3 + U3) vs. G3/(G3 + C3)] can be utilized to determine the degree and direction of the gene bias. The center point in the plot is A = T and C = G, meaning the codon has no usage bias.

Correspondence analysis

Correspondence analysis (COA) is a widely accepted multivariate statistical analysis method used to identify codon usage patterns. All genes were placed in a 59-dimensional hyperspace, taking into account the 59 sense codons (Met and Trp excluded). This method can detect the main trends in codon usage in the core CDS of Pleurotus and arrange codons along the axis according to the RSCU value.

Determination of optimal codons

The genes were ordered from highest to lowest expression according to the ENC value, and 10% of the genes from the front and rear ends were selected to form a high- and low-expression gene dataset. The D-value between the RSCU of the two datasets (ΔRSCU) was then calculated, with ΔRSCU values greater than 0.08 being defined as codons with high expression. Codons with RSCU values greater than 1 were considered high-frequency codons. A codon with ΔRSCU > 0.08 and RSCU > 1 was defined as the optimal codon.

Phylogenetic analysis

The phylogenetic relationships of Pleurotus strains were compared between codon usage-based and mitochondrial sequence-based methods. Using the RSCU values of the 8 Pleurotus strains, SPSS v19.0 software was employed to generate a hierarchical clustering method to illustrate the relationship tree between the different species. We employed the method described in our previous studies [ 48 , 64 ] to construct phylogenetic trees of the 8 Pleurotus strains using the combined mitochondrial gene datasets. To do this, individual mitochondrial genes were aligned using MAFFT v7.037 [ 65 ], and then the aligned sequences were combined into a single set using Sequence Matrix v1.7.8 [ 66 ]. Potential phylogenetic conflicts between different mitochondrial genes were identified through a partition homogeneity test. Partition Finder 2.1.1 [ 67 ] was used to determine the most suitable model of partitioning and evolution for the combined mitochondrial gene set. The phylogenetic tree was constructed using the Bayesian inference (BI) method with MrBayes v3.2.6 [ 68 ]. Two independent runs with four chains (three heated and one cold) were conducted for 2 × 10 6 generations, with samples taken every 100 generations. The first 25% of samples were discarded as burn-in, and the remaining trees were used to calculate Bayesian posterior probabilities (BPP) in a 50% majority-rule consensus tree. Ganoderma lingzhi was set as the outgroup [ 69 , 70 ].

Nucleotide composition of Pleurotus core PCGs

The codon usage analysis of 12 mitochondrial core PCGs from 8 Pleurotus strains revealed that the average length of these genes ranged from 370 bp to 2262 bp, with the nad3 gene having the shortest average length and the rps3 gene having the longest. Out of these 12 core PCGs, 10 genes had varying sequence lengths among the different Pleurotus species, while the cox2 and nad6 genes had the same gene length across all 8 strains. The rps3 gene showed the greatest length variation, with a maximum difference of 318 bp. Different Pleurotus species show great differences in base composition, even between the same species ( P. ostreatus ). The base composition of these 12 core PCGs was found to be rich in T base, with an average content of 41.80%, followed by A base, with an average content of 31.70%. The G and C base contents were relatively low, with an average of 13.59% and 12.92%, respectively. The average GC content of core PCGs ranged from 19.36 to 33.57%, with the rps3 gene having the lowest GC content and the cox1 gene having the highest.

Codon usage analysis

The GC1, GC2 and GC3 contents of the 12 core PCGs in the 8 Pleurotus strains were 34.23%, 34.31% and 11.08%, respectively (Fig.  1 ). The average GC3s value of these 12 PCGs was 9.44%, indicating that the mitochondrial core PCGs of Pleurotus tend to end with an A or T base. Additionally, the indices of A3s, T3s, G3s, and C3s of the 12 core PCGs of Pleurotus species showed that the codons were more likely to end with A, followed by T, C and G, with values of 54.80%, 54.65%, 7.53%, and 2.27%, respectively. We conducted an analysis of the codon bias of 12 core PCGs in 8 Pleurotus strains. The CAI values of the core PCGs ranged from 0.12 to 0.20, with nad2 having the lowest value and nad3 having the highest. P. giganteus had the highest CAI value, while P. citrinopileatus and P. pulmonarius had the lowest, indicating that they had a strong codon bias. The CBI values of the 8 Pleurotus strains ranged from − 0.164 to -0.173, with P. giganteus having the lowest value and P. ostreatus having the highest. The average FOP values of the 12 core PCGs ranged from 0.25 to 0.37, with nad1 having the lowest value and nad3 having the highest. P. ostreatus and P. platypus had the lowest FOP value, while P. giganteus had the largest. The GRAVY values of the 12 core PCGs were mostly positive, indicating that they were likely hydrophobic proteins, with the exception of rps3 , which was considered hydrophilic. The AROMO values of the PCGs ranged from 0.08 to 0.17, with rps3 having the highest value and cox3 having the lowest. The AROMO values of the 8 Pleurotus strains were relatively similar, with an average of 0.14. The two P. ostreatus species showed differences in various base bias indicators, among which P. ostreatus P51 had high CBI, FOP, ENC, GC3s and Aromo values, while P. ostreatus had higher CAI and Gravy indicators, indicating that the frequency of base synonymous codon usage also changed within Pleurotus species.

Codon usage indicators of 12 mitochondrial core protein coding genes in different Pleurotus strains

Codon usage correlation analysis

A significant correlation was observed between the GC1 content of mitochondrial codons and GC2, GC3, GC3s, and AROMO values in all eight Pleurotus strains ( P  < 0.05) (Fig.  2 ). Furthermore, a significant correlation was found between the GC2 content and GC content and AROMO values ( P  < 0.05). GC3 content was significantly correlated with GC3s and GC content ( P  < 0.05), and it was also found to affect codon bias in two Pleurotus species ( P. citrinopileatus , and P. giganteus ). GC3s and GC content were significantly correlated in all eight Pleurotus strains ( P  < 0.05). Additionally, the GC content was found to be significantly correlated with the AROMO values in all Pleurotus strains ( P  < 0.01). Furthermore, the CAI index of mitochondrial codons was significantly correlated with the FOP index and CBI index in seven out of eight Pleurotus strains ( P  < 0.05). Lastly, a negative correlation was observed between the ENC value and GRAVY value in P. giganteus ( P  < 0.01).

Pearson’s correlation analysis heatmap of different codon usage indicators of 8 Pleurotus strains. The color of the color block changes from green to red, indicating that the correlation index is increasing. One asterisk indicates a significant correlation between the two indicators at the P  < 0.05 level, while two asterisks indicate a significant correlation between the two indicators at the P  < 0.01 level. The 8 Pleurotus species are P. citrinopileatus , P. cornucopiae , P. eryngii , P. giganteus , P. ostreatus P51, P. ostreatus , P. platypus , and P. pulmonarius , from left to right and from top to bottom

We calculated the relationships between GC12 and GC3 based on neutrality plot analysis (Fig.  3 ). The GC12 content varied from 23.29 to 41.25%, and the GC3 content varied from 6.37 to 20.53%. The analysis between GC12 and GC3 content in mitochondrial codons of Pleurotus revealed a weak positive correlation, with the regression coefficient ranging from 0.55 to 0.95 and the R 2 value ranging from 0.2219 to 0.4458. Statistical analysis showed that there was no significant correlation between GC12 and GC3 values ( P  > 0.05), indicating that natural selection played a major role in codon bias of Pleurotus .

Neutrality plot analysis of GC12 and the third codon position (GC3) for the entire coding DNA sequence of 8 Pleurotus strains. a, P. citrinopileatus ; b, P. cornucopiae ; c, P. eryngii ; d, P. giganteus ; e, P. ostreatus P51; f, P. ostreatus ; g, P. platypus ; h, P. pulmonarius

The average ENC value of all 12 core PCGs detected was found to be 29.86, which is lower than 35, indicating a strong codon usage preference (Fig.  1 ). Moreover, the ENC values of 8 Pleurotus strains ranged from 29.58 to 30.74, further confirming the strong codon usage preference of Pleurotus species. The ENC plot showed that all Pleurotus genes were below the expected ENC-plot curve (Fig.  4 ), indicating that factors other than mutation pressure, such as natural selection, play a role in codon bias formation.

Additionally, the ENC Ratio values for all core PCGs ranged from 18.59 to 20.55%, indicating that the expected values were greater than the actual values (Fig. S1 ). This demonstrates that GC3s have an important influence on the formation of codon bias. In conclusion, it can be inferred that natural selection is a major factor determining the formation of Pleurotus codon bias.

ENC-GC3 plot analysis of 12 core PCGs in 8 Pleurotus strains. The solid line represents the expected curve when codon usage bias is affected only by mutation pressure. a, P. citrinopileatus ; b, P. cornucopiae ; c, P. eryngii ; d, P. giganteus ; e, P. ostreatus P51; f, P. ostreatus ; g, P. platypus ; h, P. pulmonarius

We conducted a Parity Rule 2 (PR2) plot analysis to investigate whether Pleurotus mitochondrial genes have any biases (Fig.  5 ). Both axes were centered on 0.5 to divide the plot into four quadrants. The results showed that the third base of the mitochondrial codon of Pleurotus had a strong preference for T over A and C over G. Most of the dots were found to be distributed in the third quadrant, while six out of the eight strains were not distributed in the fourth quadrant (preferring A to T and C to G), with P. citrinopileatus and P. giganteus being the exception. All the 8 Pleurotus strains were not distributed in the first quadrant (preferring A to T and G to C). This suggests that strong codon usage preference exist in Pleurotus species.

Parity Rule 2 (PR2) plot analysis of 12 core PCGs in 8 Pleurotus strains. a, P. citrinopileatus ; b, P. cornucopiae ; c, P. eryngii ; d, P. giganteus ; e, P. ostreatus P51; f, P. ostreatus ; g, P. platypus ; h, P. pulmonarius

To further analyze codon biases in Pleurotus , we conducted a correspondence analysis (COA) based on the RSCU values of mitochondrial genes from the 8 Pleurotus strains (Fig.  6 ). Axis 1, Axis 2, Axis 3 and Axis 4 are the main contributors to variance, with average contribution rates of 45.61%, 15.62%, 8.24% and 6.30%, respectively. The results showed that Axis 1 was the largest contributor to variance. Pearson correlation analysis showed that Axis 1 had significant correlation with CAI and ENC values. Additionally, we observed large variation in the rps3 gene and other core PCGs, indicating the differentiation of synonymous codon usage of core PCGs.

Correspondence analysis (COA) based on the relative synonymous codon usage (RSCU) values of 12 mitochondrial genes from 8 Pleurotus strains. Purple represents the cox gene, red represents the nad gene, green represents the atp6 gene, blue represents the cob gene, and yellow represents the rps3 gene. a, P. citrinopileatus ; b, P. cornucopiae ; c, P. eryngii ; d, P. giganteus ; e, P. ostreatus P51; f, P. ostreatus ; g, P. platypus ; h, P. pulmonarius

Optimal codon analysis

Analysis of the Relative Synonymous Codon Usage (RSCU) of eight Pleurotus strains revealed 27 high-frequency codons in six species ( P. citrinopileatus , P. cornucopiae , P. eryngii , P. ostreatus P51, P. platypus and P. pulmonarius ), with P. giganteus containing 26 and P. ostreatus containing 28 (Fig.  7 ). AUA was found to be used at a low frequency in P. citrinopileatus and P. giganteus , but used at a high frequency in other Pleurotus species. Of the 28 frequently used codons, 15 ended in T, 11 in A, and only 2 in G, indicating a preference for codons ending in A/T. In addition, 22, 15, 23, 30, 19, 28, 28, and 20 highly expressed codons (ΔRSCU > 0.08) were identified in the 8 Pleurotus strains, including P. citrinopileatus , P. cornucopiae , P. eryngii , P. giganteus , P. ostreatus P51, P. ostreatus P. platypus , and P. pulmonarius , respectively (Fig.  8 ). Comparative analysis revealed that 6, 6, 7, 7, 10, 9, 8, and 6 optimal codons (ΔRSCU > 0.08 and RSCU > 1) were found in P. citrinopileatus , P. cornucopiae , P. eryngii , P. giganteus , P. ostreatus P51, P. ostreatus P. platypus , and P. pulmonarius , respectively. All of these optimal codons ended with A/T, with UGU and ACU being the most widely used, followed by GGA, AUU, and UUU, which were used as the optimal codons of five strains. GCA, GCU, AAU, UCU, UAA, and ACA were each used as the optimal codons of one species. Furthermore, P. ostreatus P51 and P. ostreatus showed great differences in the use of optimal codons. AAU, GGU, AUA, UUA, UAA, and GUA were used as the optimal codons in P. ostreatus P51, while GCA, GGA, AUU, CCU, and GUU were used as the optimal codons in P. ostreatus .

Relative synonymous codon usage (RSCU) analysis of 12 mitochondrial genes from 8 Pleurotus strains. The color blocks with different colors on the bottom vertical axis represent different codons in the image above. a, P. citrinopileatus ; b, P. cornucopiae ; c, P. eryngii ; d, P. giganteus ; e, P. ostreatus P51; f, P. ostreatus ; g, P. platypus ; h, P. pulmonarius

Optimal codons of 8 Pleurotus strains (ΔRSCU > 0.08 and RSCU > 1), which are marked in purple. Highly expressed codons (ΔRSCU > 0.08) were marked in yellow and high-frequency codons (RSCU > 1) were marked in green

The Bayesian inference (BI) method was employed to construct phylogenetic trees of 8 Pleurotus strains based on the combined mitochondrial gene set (Fig.  9 ). The results demonstrated that P. giganteus and P. citrinopileatus had diverged from the Pleurotus population earlier. P. cornucopiae was identified as the sister species of P. platypus . Furthermore, two P. ostreatus strains were grouped in the same evolutionary clade, which indicated their close phylogenetic relationship. In contrast to the phylogenetic relationship inferred from sequences, the species relationship inferred from RSCU had some discrepancies, such as the phylogenetic status of P. ostreatus , P. eryngii , and P. pulmonarius . Nevertheless, the RSCU-based species relationship also clearly revealed the close relationship between P. platypus and P. cornucopiae , as well as the early divergence of P. giganteus and P. citrinopileatus from the Pleurotus population.

Relationship inference of different Pleurotus strains based on the Bayesian inference (BI) ( a ) method and relative synonymous codon usage (RSCU) hierarchical clustering ( b )

The development of sequencing technology has enabled researchers to gain access to the genetic sequences of various species and types of genomes, including the nuclear genome, chloroplast genome and mitochondrial genome [ 71 , 72 , 73 ]. Through the analysis of genetic information, it has been observed that the usage of synonymous codons varies among different species, with some codons being used more frequently than others [ 74 ]. This codon usage bias is mainly affected by several factors, such as gene base composition, gene length, gene expression level, tRNA abundance, amino acid hydrophobicity, aromaticity, mutation, and selection, with mutation and selection being the most influential [ 75 , 76 ]. Examining the codon bias characteristics of different species can help to understand the genetic structure and evolution trend of species [ 77 , 78 ]. However, the codon usage of important organelle genomes of higher fungi has not been thoroughly studied.

The mitochondrial genome is often referred to as the ‘second genome’ of eukaryotes. In this study, it was found that the length and base composition of mitochondrial core PCGs of different Pleurotus strains varied significantly, even within the same Pleurotus species, indicating the differentiation of Pleurotus mitochondrial genes. The differences in synonymous codons were mainly reflected in the third codon. Additionally, it was observed that all core PCGs of Pleurotus species tend to end with A/T, which is in line with the rule of mitochondrial codon usage in many eukaryotes [ 79 , 80 ]. The majority of high-frequency codons parsed by RSCU also end with A/T, further confirming the tendency of using the third codon of Pleurotus . Moreover, variations in base usage were observed among different species and genes. The two P. ostreatus species also showed differences in various base bias indicators, including CAI, CBI, FOP, ENC, and GC3s values, indicating that the frequency of base synonymous codon usage also changed in the within Pleurotus species. Furthermore, correlations were detected between codon base composition and GC3s, CAI, CBI, and FOP, suggesting the influence of base composition on codon bias. An ENC value lower than 35 indicates a strong codon preference [ 81 , 82 ]. The average ENC value of the mitochondrial core PCGs of Pleurotus was found to be 29.86, which indicates strong codon preference. Furthermore, the expected and actual ENC values showed significant differences (18.59-20.55%). Neutrality plot analysis and PR2-Bias plot analysis also showed evidence of natural selection in Pleurotus codon bias. This is consistent with the results seen in the mitochondrial genomes of other species [ 83 , 84 , 85 ]. The findings of this study revealed that, despite some discrepancies in codon usage indicators between different Pleurotus species, they all experienced strong natural selection on their mitochondrial PCGs.

Mitochondria are believed to have been obtained from bacteria by the ancestors of eukaryotes [ 86 ], and most mitochondrial genes have since been transferred to the nuclear genome [ 87 ]. While most eukaryotes still retain some core PCGs, some tRNA genes and rRNA genes for energy metabolism [ 88 , 89 ], which can be used as a molecular marker for phylogeny. As such, the mitochondrial genome is considered a useful tool for inferring phylogenetic relationships of species [ 90 , 91 , 92 ]. In this study, the genetic relationship of different Pleurotus species was analyzed based on a combined mitochondrial gene set and high support rates were found for each evolutionary clade. Additionally, the relationship between different Pleurotus species was determined based on their RSCU values, which differed from the sequence-based relationships. The phylogenetic tree constructed with RSCU values can serve as a supplement and reference for constructing mitochondrial gene phylogenetic trees, which agreed with previous research [ 93 , 94 ]. Codon bias, the non-uniform usage of synonymous codons, plays a role in species biodiversity, physiology, morphology, and nutrition of fungi. It can contribute to species-specific genetic signatures, influence translational efficiency and protein expression levels, potentially affect protein structure and function related to morphology, and influence the ability of a species to utilize different nutrients. However, the precise mechanisms and causal relationships between codon bias and these biological characteristics remain incompletely understood [ 95 , 96 ]. Consequently, this research enhanced the comprehension of codon usage characteristics and genetic evolution of this higher fungal group.

Data availability

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Kurland CG. Codon bias and gene expression. FEBS Lett. 1991;285(2):165–9.

Article   CAS   PubMed   Google Scholar  

Parvathy ST, Udayasuriyan V, Bhadana V. Codon usage bias. Mol Biol Rep. 2022;49(1):539–65.

Holm L. Codon usage and gene expression. Nucleic Acids Res. 1986;14(7):3075–87.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Iriarte A, Lamolle G, Musto H. Codon usage Bias: an endless tale. J Mol Evol. 2021;89(9–10):589–93.

Chakraborty S, et al. Codon usage trend in genes associated with obesity. Biotechnol Lett. 2020;42(10):1865–75.

Deng Y, et al. Hidden patterns of codon usage bias across kingdoms. J R Soc Interface. 2020;17(163):20190819.

He Z et al. Synonymous Codon usage analysis of three Narcissus potyviruses. Viruses, 2022. 14(5).

Ahn I, Jeong BJ, Son HS. Comparative study of synonymous codon usage variations between the nucleocapsid and spike genes of coronavirus, and C-type lectin domain genes of human and mouse. Exp Mol Med. 2009;41(10):746–56.

Brandão PE. The evolution of codon usage in structural and non-structural viral genes: the case of avian coronavirus and its natural host Gallus gallus. Virus Res. 2013;178(2):264–71.

Article   PubMed   Google Scholar  

Jia R, et al. Analysis of synonymous codon usage in the UL24 gene of duck enteritis virus. Virus Genes. 2009;38(1):96–103.

Kokate PP, Techtmann SM, Werner T. Codon usage bias and dinucleotide preference in 29 Drosophila species G3 (Bethesda), 2021. 11(8).

Li G, Zhang L, Du N. Relative synonymous codon usage of ORF1ab in SARS-CoV-2 and SARS-CoV. Genes Genomics. 2021;43(11):1351–9.

Long S, et al. Analysis of compositional bias and codon usage pattern of the coding sequence in Banna virus genome. Virus Res. 2018;258:68–72.

Bu Y, et al. Codon usage bias predicts the functional MYB10 gene in Populus. J Plant Physiol. 2021;265:153491.

LaBella AL, et al. Signatures of optimal codon usage in metabolic genes inform budding yeast ecology. PLoS Biol. 2021;19(4):e3001185.

Arella D, Dilucca M, Giansanti A. Codon usage bias and environmental adaptation in microbial organisms. Mol Genet Genomics. 2021;296(3):751–62.

Delport W, Scheffler K, Seoighe C. Models of coding sequence evolution. Brief Bioinform. 2009;10(1):97–109.

Hugaboom M et al. Evolution and codon usage bias of mitochondrial and nuclear genomes in Aspergillus section Flavi G3 (Bethesda), 2022.

Liu Y. A code within the genetic code: codon usage regulates co-translational protein folding. Cell Commun Signal. 2020;18(1):145.

Plotkin JB, et al. Codon usage and selection on proteins. J Mol Evol. 2006;63(5):635–53.

Shackelton LA, Parrish CR, Holmes EC. Evolutionary basis of codon usage and nucleotide composition bias in vertebrate DNA viruses. J Mol Evol. 2006;62(5):551–63.

Schmid P, Flegel WA. Codon usage in vertebrates is associated with a low risk of acquiring nonsense mutations. J Transl Med. 2011;9:87.

Article   PubMed   PubMed Central   Google Scholar  

Stoletzki N, Eyre-Walker A. Synonymous codon usage in Escherichia coli: selection for translational accuracy. Mol Biol Evol. 2007;24(2):374–81.

Dhindsa RS, et al. Natural selection shapes codon usage in the Human Genome. Am J Hum Genet. 2020;107(1):83–95.

Dilucca M, et al. Co-evolution between codon usage and protein-protein interaction in bacteria. Gene. 2021;778:145475.

Wang Q et al. Codon usage provides insights into the adaptive evolution of mycoviruses in their Associated Fungi host. Int J Mol Sci, 2022. 23(13).

Wang YY, et al. Optimized codon usage enhances the expression and immunogenicity of DNA vaccine encoding Taenia solium oncosphere TSOL18 gene. Mol Med Rep. 2015;12(1):281–8.

Xu Y, et al. Codon usage bias regulates gene expression and protein conformation in yeast expression system P. pastoris. Microb Cell Fact. 2021;20(1):91.

Yang Q, et al. Effects of codon usage on gene expression are promoter context dependent. Nucleic Acids Res. 2021;49(2):818–31.

Zhao F et al. Genome-wide role of codon usage on transcription and identification of potential regulators. Proc Natl Acad Sci U S A, 2021. 118(6).

Tuller T. Codon bias, tRNA pools and horizontal gene transfer. Mob Genet Elem. 2011;1(1):75–7.

Article   Google Scholar  

Tuller T, et al. Association between translation efficiency and horizontal gene transfer within microbial communities. Nucleic Acids Res. 2011;39(11):4743–55.

Callens M et al. Read between the lines: diversity of Nontranslational Selection pressures on local Codon usage. Genome Biol Evol, 2021. 13(9).

Ma J, et al. METTL1/WDR4-mediated m(7)G tRNA modifications and m(7)G codon usage promote mRNA translation and lung cancer progression. Mol Ther. 2021;29(12):3422–35.

López JL et al. Codon Usage Optimization in the Prokaryotic Tree of Life: how synonymous codons are differentially selected in sequence domains with different expression levels and degrees of Conservation. mBio, 2020. 11(4).

Jiang W, et al. Analysis of synonymous codon usage patterns in the edible fungus volvariella volvacea. Biotechnol Appl Biochem. 2017;64(2):218–24.

Otieno OD, et al. Genetic diversity of Kenyan native oyster mushroom (Pleurotus). Mycologia. 2015;107(1):32–8.

Lin P et al. Genetic and Chemical Diversity of Edible Mushroom Pleurotus Species Biomed Res Int, 2022. 2022: p. 6068185.

Flores GA, et al. Diversity of Pleurotus spp. (Agaricomycetes) and their metabolites of Nutraceutical and Therapeutic Importance. Int J Med Mushrooms. 2023;25(6):1–20.

Hasan HA et al. Assessment of genetic diversity among Pleurotus Spp. Isolates from Jordan. J Fungi (Basel), 2018. 4(2).

Sánchez C. Cultivation of Pleurotus ostreatus and other edible mushrooms. Appl Microbiol Biotechnol. 2010;85(5):1321–37.

Krakowska A, et al. Selected edible medicinal mushrooms from Pleurotus Genus as an answer for human civilization diseases. Food Chem. 2020;327:127084.

Dos Reis EE, Schenkel PC, Camassola M. Effects of bioactive compounds from Pleurotus mushrooms on COVID-19 risk factors associated with the cardiovascular system. J Integr Med. 2022;20(5):385–95.

Cohen R, Persky L, Hadar Y. Biotechnological applications and potential of wood-degrading mushrooms of the genus Pleurotus. Appl Microbiol Biotechnol. 2002;58(5):582–94.

Li L, et al. Effect of Pleurotus Eryngii mycelial fermentation on the composition and antioxidant properties of tartary buckwheat. Heliyon. 2024;10(4):e25980.

Gray MW, Burger G, Lang BF. The origin and early evolution of mitochondria. Genome Biol. 2001;2(6):REVIEWS1018.

Li Q, et al. Comparative Mitogenome Analysis reveals mitochondrial genome differentiation in Ectomycorrhizal and Asymbiotic Amanita Species. Front Microbiol. 2020;11:1382.

Li Q, et al. First two mitochondrial genomes for the order Filobasidiales reveal novel gene rearrangements and intron dynamics of Tremellomycetes. IMA Fungus. 2022;13(1):7.

Cai N, et al. Mitochondrial DNA variants modulate N-formylmethionine, proteostasis and risk of late-onset human diseases. Nat Med. 2021;27(9):1564–75.

Chen K et al. Mitochondrial mutations and mitoepigenetics: focus on regulation of oxidative stress-induced responses in breast cancers. Semin Cancer Biol, 2020.

Hayashi JI, et al. Mutations in mitochondrial DNA regulate mitochondrial diseases and metastasis but do not regulate aging. Curr Opin Genet Dev. 2016;38:63–7.

Li Q, et al. Comparative mitogenomics reveals large-scale gene rearrangements in the mitochondrial genome of two Pleurotus species. Appl Microbiol Biotechnol. 2018;102(14):6143–53.

Liu Z, et al. The complete mitochondrial genome of the edible mushroom Pleurotus Giganteus (Agaricales, Pleurotus) and insights into its phylogeny. Mitochondrial DNA B Resour. 2022;7(7):1313–5.

Wang Y, et al. The mitochondrial genome of the Basidiomycete fungus Pleurotus Ostreatus (oyster mushroom). FEMS Microbiol Lett. 2008;280(1):34–41.

Xu LM, Hinsinger DD, Jiang GF. The complete mitochondrial genome of the Basidiomycete fungus Pleurotus cornucopiae (Paulet) Rolland. Mitochondrial DNA B Resour. 2018;3(1):73–5.

Yang R, et al. The complete mitochondrial genome of the Basidiomycete edible fungus Pleurotus Eryngii. Mitochondrial DNA B Resour. 2016;1(1):772–4.

Ye LY, et al. Mitochondrial genome and diverse inheritance patterns in Pleurotus Pulmonarius. J Microbiol. 2020;58(2):142–52.

Huo X, et al. Analysis of synonymous codon usage of transcriptome database in Rheum palmatum. PeerJ. 2021;9:e10450.

Sharp PM, Li WH. The codon Adaptation Index–a measure of directional synonymous codon usage bias, and its potential applications. Nucleic Acids Res. 1987;15(3):1281–95.

Peden JF. Analysis of codon usage PhD Thesis, University of Nottingham, 1999. UK.

Puigbo P, Bravo IG, Garcia-Vallve S. CAIcal: a combined set of tools to assess codon usage adaptation. Biol Direct. 2008;3:38.

Orr HA. The evolutionary genetics of adaptation: a simulation study. Genet Res. 1999;74(3):207–14.

Wright F. The ‘effective number of codons’ used in a gene. Gene. 1990;87(1):23–9.

Li Q, et al. The first two mitochondrial genomes for the genus Ramaria reveal mitochondrial genome evolution of Ramaria and phylogeny of Basidiomycota. IMA Fungus. 2022;13(1):16.

Katoh K, Rozewicki J, Yamada KD. MAFFT online service: multiple sequence alignment, interactive sequence choice and visualization. Brief Bioinform; 2017.

Vaidya G, Lohman DL, Meier R. SequenceMatrix: concatenation software for the fast assembly of multi-gene datasets with character set and codon information. Cladistics. 2011;27(2):171–80.

Lanfear R, et al. PartitionFinder 2: New methods for selecting Partitioned models of Evolution for Molecular and Morphological phylogenetic analyses. Mol Biol Evol. 2017;34(3):772–3.

CAS   PubMed   Google Scholar  

Ronquist F, et al. MrBayes 3.2: efficient bayesian phylogenetic inference and model choice across a large model space. Syst Biol. 2012;61(3):539–42.

Li Q et al. Comparative mitogenomic analysis reveals Intraspecific, Interspecific Variations and Genetic Diversity of Medical Fungus Ganoderma. J Fungi (Basel), 2022. 8(8).

Wu P, et al. Comprehensive analysis of codon bias in 13 Ganoderma mitochondrial genomes. Front Microbiol. 2023;14:1170790.

Chakraborty S, Yengkhom S, Uddin A. Analysis of codon usage bias of chloroplast genes in Oryza species: Codon usage of chloroplast genes in Oryza species. Planta. 2020;252(4):67.

Barbhuiya PA, Uddin A, Chakraborty S. Analysis of compositional properties and codon usage bias of mitochondrial CYB gene in anura, urodela and gymnophiona. Gene. 2020;751:144762.

Xu W, et al. Synonymous codon usage bias in plant mitochondrial genes is associated with intron number and mirrors species evolution. PLoS ONE. 2015;10(6):e0131508.

Liu H et al. Codon usage by chloroplast gene is bias in Hemiptelea Davidii. J Genet, 2020. 99.

Chen H, et al. Mutation and selection cause codon usage and bias in mitochondrial genomes of ribbon worms (Nemertea). PLoS ONE. 2014;9(1):e85631.

Trotta E. Selection on codon bias in yeast: a transcriptional hypothesis. Nucleic Acids Res. 2013;41(20):9382–95.

Yang C, et al. Comparative analysis of genomic and transcriptome sequences reveals divergent patterns of Codon Bias in Wheat and its ancestor species. Front Genet. 2021;12:732432.

Yu X, et al. Comprehensive Analysis of Synonymous Codon usage Bias for Complete genomes and E2 gene of atypical Porcine Pestivirus. Biochem Genet. 2021;59(3):799–812.

Mazumder GA, Uddin A, Chakraborty S. Analysis of codon usage bias in mitochondrial CO gene among platyhelminthes. Mol Biochem Parasitol. 2021;245:111410.

Montana-Lozano P, Balaguera-Reina SA, Prada-Quiroga CF. Comp Anal Codon Usage Mitochondrial Genomes Provides Evolutionary Insights into Reptiles Gene. 2022;851:146999.

Google Scholar  

Pepe D, K DEK. Codon bias analyses on thyroid carcinoma genes. Minerva Endocrinol. 2020;45(4):295–305.

PubMed   Google Scholar  

Prabha R, et al. Genome-wide comparative analysis of codon usage bias and codon context patterns among cyanobacterial genomes. Mar Genomics. 2017;32:31–9.

Barbhuiya PA, Uddin A, Chakraborty S. Understanding the codon usage patterns of mitochondrial CO genes among amphibians. Gene. 2021;777:145462.

Barbhuiya PA, Uddin A, Chakraborty S. Codon usage pattern and evolutionary forces of mitochondrial ND genes among orders of class Amphibia. J Cell Physiol. 2021;236(4):2850–68.

Li Q, et al. Analysis of synonymous codon usage patterns in mitochondrial genomes of nine Amanita species. Front Microbiol. 2023;14:1134228.

Lang BF, Gray MW, Burger G. Mitochondrial genome evolution and the origin of eukaryotes. Annu Rev Genet. 1999;33:351–97.

Adams KL, Palmer JD. Evolution of mitochondrial gene content: gene loss and transfer to the nucleus. Mol Phylogenet Evol. 2003;29(3):380–95.

Costa GG, et al. The mitochondrial genome of Moniliophthora Roreri, the frosty pod rot pathogen of cacao. Fungal Biol. 2012;116(5):551–62.

Bullerwell CE, Burger G, Lang BF. A novel motif for identifying rps3 homologs in fungal mitochondrial genomes. Trends Biochem Sci. 2000;25(8):363–5.

Johri P, et al. Population genetics of Paramecium mitochondrial genomes: recombination, mutation spectrum, and efficacy of selection. Genome Biol Evol; 2019.

Li H, et al. Higher-level phylogeny of paraneopteran insects inferred from mitochondrial genome sequences. Sci Rep. 2015;5:8527.

Li Q, et al. The first two mitochondrial genomes from Apiotrichum reveal mitochondrial evolution and different taxonomic assignment of Trichosporonales. IMA Fungus. 2023;14(1):7.

Crane A et al. Phylogenetic relationships and codon usage bias amongst cluster K mycobacteriophages G3 (Bethesda), 2021. 11(11).

Gupta S, Paul K, Roy A. Codon usage signatures in the genus Cryptococcus: a complex interplay of gene expression, translational selection and compositional bias. Genomics. 2021;113(1 Pt 2):821–30.

Hershberg R, Petrov DA. Selection on codon bias. Annu Rev Genet. 2008;42:287–99.

Gustafsson C, Govindarajan S, Minshull J. Codon bias and heterologous protein expression. Trends Biotechnol. 2004;22(7):346–53.

Download references

Acknowledgements

Not applicable.

This work was supported by National Natural Science Foundation of China (No. 82102738).

Author information

Authors and affiliations.

Clinical Medical College & Affiliated Hospital of Chengdu University, Chengdu University, Chengdu, Sichuan, China

Key Laboratory of Coarse Cereal Processing, Ministry of Agriculture and Rural Affairs, School of Food and Biological Engineering, Chengdu University, Chengdu, Sichuan, China

Xiaodie Chen, Jing He, Ajia Sha, Yingyong Luo, Wenqi Xiao, Zhuang Xiong & Qiang Li

School of Food and Biological Engineering, Chengdu University, 2025 # Chengluo Avenue, Longquanyi District, Chengdu, Sichuan, 610106, China

You can also search for this author in PubMed   Google Scholar

Contributions

Q.L and W.G designed the experiment; X.C., J.H., A.S., Y.L., W.X., and Z.X. analyzed the data; Q.L and W.G wrote and review the manuscript; Q.L. managed the project. All authors reviewed the manuscript.

Corresponding author

Correspondence to Qiang Li .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Adherence to national and international regulations

Additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Gao, W., Chen, X., He, J. et al. Intraspecific and interspecific variations in the synonymous codon usage in mitochondrial genomes of 8 pleurotus strains. BMC Genomics 25 , 456 (2024). https://doi.org/10.1186/s12864-024-10374-3

Download citation

Received : 12 June 2023

Accepted : 03 May 2024

Published : 10 May 2024

DOI : https://doi.org/10.1186/s12864-024-10374-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Codon usage
• Mitochondrial genome
• Natural selection

BMC Genomics

ISSN: 1471-2164

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Research bias

Types of Bias in Research | Definition & Examples

Research bias results from any deviation from the truth, causing distorted results and wrong conclusions. Bias can occur at any phase of your research, including during data collection , data analysis , interpretation, or publication. Research bias can occur in both qualitative and quantitative research .

Understanding research bias is important for several reasons.

• Bias exists in all research, across research designs , and is difficult to eliminate.
• Bias can occur at any stage of the research process.
• Bias impacts the validity and reliability of your findings, leading to misinterpretation of data.

It is almost impossible to conduct a study without some degree of research bias. It’s crucial for you to be aware of the potential types of bias, so you can minimise them.

For example, the success rate of the program will likely be affected if participants start to drop out. Participants who become disillusioned due to not losing weight may drop out, while those who succeed in losing weight are more likely to continue. This in turn may bias the findings towards more favorable results.  

Table of contents

Actor–observer bias.

• Confirmation bias

Information bias

Interviewer bias.

• Publication bias

Researcher bias

Response bias.

Selection bias

How to avoid bias in research

Other types of research bias, frequently asked questions about research bias.

Actor–observer bias occurs when you attribute the behaviour of others to internal factors, like skill or personality, but attribute your own behaviour to external or situational factors.

In other words, when you are the actor in a situation, you are more likely to link events to external factors, such as your surroundings or environment. However, when you are observing the behaviour of others, you are more likely to associate behaviour with their personality, nature, or temperament.

One interviewee recalls a morning when it was raining heavily. They were rushing to drop off their kids at school in order to get to work on time. As they were driving down the road, another car cut them off as they were trying to merge. They tell you how frustrated they felt and exclaim that the other driver must have been a very rude person.

At another point, the same interviewee recalls that they did something similar: accidentally cutting off another driver while trying to take the correct exit. However, this time, the interviewee claimed that they always drive very carefully, blaming their mistake on poor visibility due to the rain.

Confirmation bias is the tendency to seek out information in a way that supports our existing beliefs while also rejecting any information that contradicts those beliefs. Confirmation bias is often unintentional but still results in skewed results and poor decision-making.

Let’s say you grew up with a parent in the military. Chances are that you have a lot of complex emotions around overseas deployments. This can lead you to over-emphasise findings that ‘prove’ that your lived experience is the case for most families, neglecting other explanations and experiences.

Information bias , also called measurement bias, arises when key study variables are inaccurately measured or classified. Information bias occurs during the data collection step and is common in research studies that involve self-reporting and retrospective data collection. It can also result from poor interviewing techniques or differing levels of recall from participants.

The main types of information bias are:

• Recall bias
• Observer bias

Performance bias

Regression to the mean (rtm).

Over a period of four weeks, you ask students to keep a journal, noting how much time they spent on their smartphones along with any symptoms like muscle twitches, aches, or fatigue.

Recall bias is a type of information bias. It occurs when respondents are asked to recall events in the past and is common in studies that involve self-reporting.

As a rule of thumb, infrequent events (e.g., buying a house or a car) will be memorable for longer periods of time than routine events (e.g., daily use of public transportation). You can reduce recall bias by running a pilot survey and carefully testing recall periods. If possible, test both shorter and longer periods, checking for differences in recall.

• A group of children who have been diagnosed, called the case group
• A group of children who have not been diagnosed, called the control group

Since the parents are being asked to recall what their children generally ate over a period of several years, there is high potential for recall bias in the case group.

The best way to reduce recall bias is by ensuring your control group will have similar levels of recall bias to your case group. Parents of children who have childhood cancer, which is a serious health problem, are likely to be quite concerned about what may have contributed to the cancer.

Thus, if asked by researchers, these parents are likely to think very hard about what their child ate or did not eat in their first years of life. Parents of children with other serious health problems (aside from cancer) are also likely to be quite concerned about any diet-related question that researchers ask about.

Observer bias is the tendency of research participants to see what they expect or want to see, rather than what is actually occurring. Observer bias can affect the results in observationa l and experimental studies, where subjective judgement (such as assessing a medical image) or measurement (such as rounding blood pressure readings up or down) is part of the data collection process.

Observer bias leads to over- or underestimation of true values, which in turn compromise the validity of your findings. You can reduce observer bias by using double-  and single-blinded research methods.

Based on discussions you had with other researchers before starting your observations, you are inclined to think that medical staff tend to simply call each other when they need specific patient details or have questions about treatments.

At the end of the observation period, you compare notes with your colleague. Your conclusion was that medical staff tend to favor phone calls when seeking information, while your colleague noted down that medical staff mostly rely on face-to-face discussions. Seeing that your expectations may have influenced your observations, you and your colleague decide to conduct interviews with medical staff to clarify the observed events. Note: Observer bias and actor–observer bias are not the same thing.

Performance bias is unequal care between study groups. Performance bias occurs mainly in medical research experiments, if participants have knowledge of the planned intervention, therapy, or drug trial before it begins.

Studies about nutrition, exercise outcomes, or surgical interventions are very susceptible to this type of bias. It can be minimized by using blinding , which prevents participants and/or researchers from knowing who is in the control or treatment groups. If blinding is not possible, then using objective outcomes (such as hospital admission data) is the best approach.

When the subjects of an experimental study change or improve their behaviour because they are aware they are being studied, this is called the Hawthorne (or observer) effect . Similarly, the John Henry effect occurs when members of a control group are aware they are being compared to the experimental group. This causes them to alter their behaviour in an effort to compensate for their perceived disadvantage.

Regression to the mean (RTM) is a statistical phenomenon that refers to the fact that a variable that shows an extreme value on its first measurement will tend to be closer to the centre of its distribution on a second measurement.

Medical research is particularly sensitive to RTM. Here, interventions aimed at a group or a characteristic that is very different from the average (e.g., people with high blood pressure) will appear to be successful because of the regression to the mean. This can lead researchers to misinterpret results, describing a specific intervention as causal when the change in the extreme groups would have happened anyway.

In general, among people with depression, certain physical and mental characteristics have been observed to deviate from the population mean .

This could lead you to think that the intervention was effective when those treated showed improvement on measured post-treatment indicators, such as reduced severity of depressive episodes.

However, given that such characteristics deviate more from the population mean in people with depression than in people without depression, this improvement could be attributed to RTM.

Interviewer bias stems from the person conducting the research study. It can result from the way they ask questions or react to responses, but also from any aspect of their identity, such as their sex, ethnicity, social class, or perceived attractiveness.

Interviewer bias distorts responses, especially when the characteristics relate in some way to the research topic. Interviewer bias can also affect the interviewer’s ability to establish rapport with the interviewees, causing them to feel less comfortable giving their honest opinions about sensitive or personal topics.

Participant: ‘I like to solve puzzles, or sometimes do some gardening.’

You: ‘I love gardening, too!’

In this case, seeing your enthusiastic reaction could lead the participant to talk more about gardening.

Establishing trust between you and your interviewees is crucial in order to ensure that they feel comfortable opening up and revealing their true thoughts and feelings. At the same time, being overly empathetic can influence the responses of your interviewees, as seen above.

Publication bias occurs when the decision to publish research findings is based on their nature or the direction of their results. Studies reporting results that are perceived as positive, statistically significant , or favoring the study hypotheses are more likely to be published due to publication bias.

Publication bias is related to data dredging (also called p -hacking ), where statistical tests on a set of data are run until something statistically significant happens. As academic journals tend to prefer publishing statistically significant results, this can pressure researchers to only submit statistically significant results. P -hacking can also involve excluding participants or stopping data collection once a p value of 0.05 is reached. However, this leads to false positive results and an overrepresentation of positive results in published academic literature.

Researcher bias occurs when the researcher’s beliefs or expectations influence the research design or data collection process. Researcher bias can be deliberate (such as claiming that an intervention worked even if it didn’t) or unconscious (such as letting personal feelings, stereotypes, or assumptions influence research questions ).

The unconscious form of researcher bias is associated with the Pygmalion (or Rosenthal) effect, where the researcher’s high expectations (e.g., that patients assigned to a treatment group will succeed) lead to better performance and better outcomes.

Researcher bias is also sometimes called experimenter bias, but it applies to all types of investigative projects, rather than only to experimental designs .

• Good question: What are your views on alcohol consumption among your peers?
• Bad question: Do you think it’s okay for young people to drink so much?

Response bias is a general term used to describe a number of different situations where respondents tend to provide inaccurate or false answers to self-report questions, such as those asked on surveys or in structured interviews .

This happens because when people are asked a question (e.g., during an interview ), they integrate multiple sources of information to generate their responses. Because of that, any aspect of a research study may potentially bias a respondent. Examples include the phrasing of questions in surveys, how participants perceive the researcher, or the desire of the participant to please the researcher and to provide socially desirable responses.

Response bias also occurs in experimental medical research. When outcomes are based on patients’ reports, a placebo effect can occur. Here, patients report an improvement despite having received a placebo, not an active medical treatment.

While interviewing a student, you ask them:

‘Do you think it’s okay to cheat on an exam?’

Common types of response bias are:

Acquiescence bias

Demand characteristics.

• Social desirability bias

Courtesy bias

• Question-order bias

Extreme responding

Acquiescence bias is the tendency of respondents to agree with a statement when faced with binary response options like ‘agree/disagree’, ‘yes/no’, or ‘true/false’. Acquiescence is sometimes referred to as ‘yea-saying’.

This type of bias occurs either due to the participant’s personality (i.e., some people are more likely to agree with statements than disagree, regardless of their content) or because participants perceive the researcher as an expert and are more inclined to agree with the statements presented to them.

Q: Are you a social person?

People who are inclined to agree with statements presented to them are at risk of selecting the first option, even if it isn’t fully supported by their lived experiences.

In order to control for acquiescence, consider tweaking your phrasing to encourage respondents to make a choice truly based on their preferences. Here’s an example:

Q: What would you prefer?

• A quiet night in
• A night out with friends

Demand characteristics are cues that could reveal the research agenda to participants, risking a change in their behaviours or views. Ensuring that participants are not aware of the research goals is the best way to avoid this type of bias.

On each occasion, patients reported their pain as being less than prior to the operation. While at face value this seems to suggest that the operation does indeed lead to less pain, there is a demand characteristic at play. During the interviews, the researcher would unconsciously frown whenever patients reported more post-op pain. This increased the risk of patients figuring out that the researcher was hoping that the operation would have an advantageous effect.

Social desirability bias is the tendency of participants to give responses that they believe will be viewed favorably by the researcher or other participants. It often affects studies that focus on sensitive topics, such as alcohol consumption or sexual behaviour.

You are conducting face-to-face semi-structured interviews with a number of employees from different departments. When asked whether they would be interested in a smoking cessation program, there was widespread enthusiasm for the idea.

Note that while social desirability and demand characteristics may sound similar, there is a key difference between them. Social desirability is about conforming to social norms, while demand characteristics revolve around the purpose of the research.

Courtesy bias stems from a reluctance to give negative feedback, so as to be polite to the person asking the question. Small-group interviewing where participants relate in some way to each other (e.g., a student, a teacher, and a dean) is especially prone to this type of bias.

Question order bias

Question order bias occurs when the order in which interview questions are asked influences the way the respondent interprets and evaluates them. This occurs especially when previous questions provide context for subsequent questions.

When answering subsequent questions, respondents may orient their answers to previous questions (called a halo effect ), which can lead to systematic distortion of the responses.

Extreme responding is the tendency of a respondent to answer in the extreme, choosing the lowest or highest response available, even if that is not their true opinion. Extreme responding is common in surveys using Likert scales , and it distorts people’s true attitudes and opinions.

Disposition towards the survey can be a source of extreme responding, as well as cultural components. For example, people coming from collectivist cultures tend to exhibit extreme responses in terms of agreement, while respondents indifferent to the questions asked may exhibit extreme responses in terms of disagreement.

Selection bias is a general term describing situations where bias is introduced into the research from factors affecting the study population.

Common types of selection bias are:

Sampling or ascertainment bias

• Attrition bias

Volunteer or self-selection bias

• Survivorship bias
• Nonresponse bias
• Undercoverage bias

Sampling bias occurs when your sample (the individuals, groups, or data you obtain for your research) is selected in a way that is not representative of the population you are analyzing. Sampling bias threatens the external validity of your findings and influences the generalizability of your results.

The easiest way to prevent sampling bias is to use a probability sampling method . This way, each member of the population you are studying has an equal chance of being included in your sample.

Sampling bias is often referred to as ascertainment bias in the medical field.

Attrition bias occurs when participants who drop out of a study systematically differ from those who remain in the study. Attrition bias is especially problematic in randomized controlled trials for medical research because participants who do not like the experience or have unwanted side effects can drop out and affect your results.

You can minimize attrition bias by offering incentives for participants to complete the study (e.g., a gift card if they successfully attend every session). It’s also a good practice to recruit more participants than you need, or minimize the number of follow-up sessions or questions.

You provide a treatment group with weekly one-hour sessions over a two-month period, while a control group attends sessions on an unrelated topic. You complete five waves of data collection to compare outcomes: a pretest survey , three surveys during the program, and a posttest survey.

Volunteer bias (also called self-selection bias ) occurs when individuals who volunteer for a study have particular characteristics that matter for the purposes of the study.

Volunteer bias leads to biased data, as the respondents who choose to participate will not represent your entire target population. You can avoid this type of bias by using random assignment – i.e., placing participants in a control group or a treatment group after they have volunteered to participate in the study.

Closely related to volunteer bias is nonresponse bias , which occurs when a research subject declines to participate in a particular study or drops out before the study’s completion.

Considering that the hospital is located in an affluent part of the city, volunteers are more likely to have a higher socioeconomic standing, higher education, and better nutrition than the general population.

Survivorship bias occurs when you do not evaluate your data set in its entirety: for example, by only analyzing the patients who survived a clinical trial.

This strongly increases the likelihood that you draw (incorrect) conclusions based upon those who have passed some sort of selection process – focusing on ‘survivors’ and forgetting those who went through a similar process and did not survive.

Note that ‘survival’ does not always mean that participants died! Rather, it signifies that participants did not successfully complete the intervention.

However, most college dropouts do not become billionaires. In fact, there are many more aspiring entrepreneurs who dropped out of college to start companies and failed than succeeded.

Nonresponse bias occurs when those who do not respond to a survey or research project are different from those who do in ways that are critical to the goals of the research. This is very common in survey research, when participants are unable or unwilling to participate due to factors like lack of the necessary skills, lack of time, or guilt or shame related to the topic.

You can mitigate nonresponse bias by offering the survey in different formats (e.g., an online survey, but also a paper version sent via post), ensuring confidentiality , and sending them reminders to complete the survey.

You notice that your surveys were conducted during business hours, when the working-age residents were less likely to be home.

Undercoverage bias occurs when you only sample from a subset of the population you are interested in. Online surveys can be particularly susceptible to undercoverage bias. Despite being more cost-effective than other methods, they can introduce undercoverage bias as a result of excluding people who do not use the internet.

While very difficult to eliminate entirely, research bias can be mitigated through proper study design and implementation. Here are some tips to keep in mind as you get started.

• Clearly explain in your methodology section how your research design will help you meet the research objectives and why this is the most appropriate research design.
• In quantitative studies , make sure that you use probability sampling to select the participants. If you’re running an experiment, make sure you use random assignment to assign your control and treatment groups.
• Account for participants who withdraw or are lost to follow-up during the study. If they are withdrawing for a particular reason, it could bias your results. This applies especially to longer-term or longitudinal studies .
• Use triangulation to enhance the validity and credibility of your findings.
• Phrase your survey or interview questions in a neutral, non-judgemental tone. Be very careful that your questions do not steer your participants in any particular direction.
• Consider using a reflexive journal. Here, you can log the details of each interview , paying special attention to any influence you may have had on participants. You can include these in your final analysis.

Cognitive bias

• Baader–Meinhof phenomenon
• Availability heuristic
• Halo effect
• Framing effect
• Sampling bias
• Ascertainment bias
• Self-selection bias
• Hawthorne effect
• Omitted variable bias
• Pygmalion effect
• Placebo effect

Bias in research affects the validity and reliability of your findings, leading to false conclusions and a misinterpretation of the truth. This can have serious implications in areas like medical research where, for example, a new form of treatment may be evaluated.

Observer bias occurs when the researcher’s assumptions, views, or preconceptions influence what they see and record in a study, while actor–observer bias refers to situations where respondents attribute internal factors (e.g., bad character) to justify other’s behaviour and external factors (difficult circumstances) to justify the same behaviour in themselves.

Response bias is a general term used to describe a number of different conditions or factors that cue respondents to provide inaccurate or false answers during surveys or interviews . These factors range from the interviewer’s perceived social position or appearance to the the phrasing of questions in surveys.

Nonresponse bias occurs when the people who complete a survey are different from those who did not, in ways that are relevant to the research topic. Nonresponse can happen either because people are not willing or not able to participate.

Is this article helpful?

More interesting articles.

• Attrition Bias | Examples, Explanation, Prevention
• Demand Characteristics | Definition, Examples & Control
• Hostile Attribution Bias | Definition & Examples
• Observer Bias | Definition, Examples, Prevention
• Regression to the Mean | Definition & Examples
• Representativeness Heuristic | Example & Definition
• Sampling Bias and How to Avoid It | Types & Examples
• Self-Fulfilling Prophecy | Definition & Examples
• The Availability Heuristic | Example & Definition
• The Baader–Meinhof Phenomenon Explained
• What Is a Ceiling Effect? | Definition & Examples
• What Is Actor-Observer Bias? | Definition & Examples
• What Is Affinity Bias? | Definition & Examples
• What Is Anchoring Bias? | Definition & Examples
• What Is Ascertainment Bias? | Definition & Examples
• What Is Belief Bias? | Definition & Examples
• What Is Bias for Action? | Definition & Examples
• What Is Cognitive Bias? | Meaning, Types & Examples
• What Is Confirmation Bias? | Definition & Examples
• What Is Conformity Bias? | Definition & Examples
• What Is Correspondence Bias? | Definition & Example
• What Is Explicit Bias? | Definition & Examples
• What Is Generalisability? | Definition & Examples
• What Is Hindsight Bias? | Definition & Examples
• What Is Implicit Bias? | Definition & Examples
• What Is Information Bias? | Definition & Examples
• What Is Ingroup Bias? | Definition & Examples
• What Is Negativity Bias? | Definition & Examples
• What Is Nonresponse Bias?| Definition & Example
• What Is Normalcy Bias? | Definition & Example
• What Is Omitted Variable Bias? | Definition & Example
• What Is Optimism Bias? | Definition & Examples
• What Is Outgroup Bias? | Definition & Examples
• What Is Overconfidence Bias? | Definition & Examples
• What Is Perception Bias? | Definition & Examples
• What Is Primacy Bias? | Definition & Example
• What Is Publication Bias? | Definition & Examples
• What Is Recall Bias? | Definition & Examples
• What Is Recency Bias? | Definition & Examples
• What Is Response Bias? | Definition & Examples
• What Is Selection Bias? | Definition & Examples
• What Is Self-Selection Bias? | Definition & Example
• What Is Self-Serving Bias? | Definition & Example
• What Is Social Desirability Bias? | Definition & Examples
• What Is Status Quo Bias? | Definition & Examples
• What Is Survivorship Bias? | Definition & Examples
• What Is the Affect Heuristic? | Example & Definition
• What Is the Egocentric Bias? | Definition & Examples
• What Is the Framing Effect? | Definition & Examples
• What Is the Halo Effect? | Definition & Examples
• What Is the Hawthorne Effect? | Definition & Examples
• What Is the Placebo Effect? | Definition & Examples
• What Is the Pygmalion Effect? | Definition & Examples
• What Is Unconscious Bias? | Definition & Examples
• What Is Undercoverage Bias? | Definition & Example
• What Is Vividness Bias? | Definition & Examples

• Open access
• Published: 04 May 2024

Impacts of heat exposure in utero on long-term health and social outcomes: a systematic review

• Nicholas Brink 1 ,
• Darshnika P. Lakhoo 1 ,
• Ijeoma Solarin 1 ,
• Gloria Maimela 1 ,
• Peter von Dadelszen 2 ,
• Shane Norris 3 ,
• Matthew F. Chersich 1 &

Climate and Heat-Health Study Group

BMC Pregnancy and Childbirth volume  24 , Article number:  344 ( 2024 ) Cite this article

512 Accesses

8 Altmetric

Metrics details

Climate change, particularly global warming, is amongst the greatest threats to human health. While short-term effects of heat exposure in pregnancy, such as preterm birth, are well documented, long-term effects have received less attention. This review aims to systematically assess evidence on the long-term impacts on the foetus of heat exposure in utero.

A search was conducted in August 2019 and updated in April 2023 in MEDLINE(PubMed). We included studies on the relationship of environmental heat exposure during pregnancy and any long-term outcomes. Risk of bias was assessed using tools developed by the Joanna-Briggs Institute, and the evidence was appraised using the GRADE approach. Synthesis without Meta-Analysis (SWiM) guidelines were used.

Eighteen thousand six hundred twenty one records were screened, with 29 studies included across six outcome groups. Studies were mostly conducted in high-income countries ( n  = 16/25), in cooler climates. All studies were observational, with 17 cohort, 5 case-control and 8 cross-sectional studies. The timeline of the data is from 1913 to 2019, and individuals ranged in age from neonates to adults, and the elderly. Increasing heat exposure during pregnancy was associated with decreased earnings and lower educational attainment ( n  = 4/6), as well as worsened cardiovascular ( n  = 3/6), respiratory ( n  = 3/3), psychiatric ( n  = 7/12) and anthropometric ( n  = 2/2) outcomes, possibly culminating in increased overall mortality ( n  = 2/3). The effect on female infants was greater than on males in 8 of 9 studies differentiating by sex. The quality of evidence was low in respiratory and longevity outcome groups to very low in all others.

Conclusions

Increasing heat exposure was associated with a multitude of detrimental outcomes across diverse body systems. The biological pathways involved are yet to be elucidated, but could include epigenetic and developmental perturbations, through interactions with the placenta and inflammation. This highlights the need for further research into the long-term effects of heat exposure, biological pathways, and possible adaptation strategies in studies, particularly in neglected regions. Heat exposure in-utero has the potential to compound existing health and social inequalities. Poor study design of the included studies constrains the conclusions of this review, with heterogenous exposure measures and outcomes rendering comparisons across contexts/studies difficult.

Trial Registration

PROSPERO CRD 42019140136.

Peer Review reports

Introduction

Climate change is one of the most significant threats to human health [ 1 ], characterized by an increase in global temperatures amongst other environmental changes. Global temperatures have increased by approximately 1·2 °C, and are projected to increase beyond a critical threshold of 1·5 °C in the next 5–10 years [ 2 ]. Increasingly, heat exposure is being linked with a multitude of short- and long-term health effects in vulnerable populations, including children [ 3 ], the elderly, and pregnant women [ 4 ]. The effect on pregnant women extends to the health of the foetus, with significant detrimental effects associated with heat exposure including preterm birth, stillbirth, and decreased birth weight [ 5 ]. Impacts of heat exposure are increasingly important in populations in resource-constrained settings, where heat adaptation measures such as active (air-conditioning) and passive cooling (water, green and blue spaces) are limited, and often inaccessible [ 6 ]. These populations are often found in some of the hottest climates and in areas whose contribution to global warming is negligible, thus compounding inequities [ 7 ]. In addition, research in this field is biased towards Europe, North America and Asia and is profoundly underrepresented in Africa and South America [ 8 ]. Understanding the scope and distribution of research conducted is key to guiding future research, including biological studies to explore possible mechanisms, and interventional studies to alleviate any observed negative effects. Multiple previous systematic reviews have explored the short-term impacts of heat on the foetus [ 3 , 5 , 9 ] but only one has explored the long-term impacts of heat exposure on mental health [ 10 ]. The in-utero environment has long been considered important in the long-term health and wellbeing of individuals [ 11 , 12 ], although it has been challenging to delineate specific causal pathways. This study aims to systematically review the literature on the long-term effects of heat exposure in-utero on the foetus, and explore possible casual pathways.

Materials and methods

This review forms part of a larger systematic mapping survey of the effect of heat exposure, and adaptation interventions on health (PROSPERO CRD 42019140136) [ 13 ]. The initial literature search was conducted in September 2018, where the authors searched MEDLINE (PubMed), Science Citation Index Expanded, Social Sciences Citation Index, and Arts and Humanities Citation Index using a validated search strategy (Supplementary Text 1 ). This search was updated in April 2023 through a MEDLINE search, as all previous articles were located in this database. Screening of titles and abstracts was done independently in duplicate, with any differences reconciled by MFC, with subsequent updates conducted by NB and DL. The authors only included studies on humans, published in Chinese, English, German, or Italian. Studies on heat exposure from artificial and endogenous sources were excluded, and only exogenous, weather-related heat exposure during pregnancy was included. All study designs were eligible except modelling studies and systematic reviews. No date restrictions were applied. EPPI-Reviewer software [ 14 ] provided a platform for screening, reviewing of full text articles, and for data extraction. No additional information was requested or provided by the authors. Long-term effects were defined as any outcomes that were not apparent at birth.

Articles meeting the eligibility criteria were extracted in duplicate after the initial search and then by a single reviewer in the subsequent update (NB/DL). Data were extracted to include characteristics outlined in Supplementary file 1 .

This systematic review was conducted according to the Systematic Review without Meta-Analysis (SWiM) guidelines, broadly based on PRISMA [ 15 ], as the outcomes, statistical techniques, and heat exposure measurements were heterogenous, rendering a meta-analysis untenable. Outcomes were grouped clinically, reviewed for the magnitude and direction of effect, and their statistical significance, and included negative or null findings when reported on. A text-based summary of these findings was made. ‘Vote-counting’ was utilized to summarise direction of effect findings. Analysis was conducted on the geographical areas, climate zones [ 16 ], mean annual temperature and socioeconomic classification of the country where the studies were conducted. Furthermore, an attempt was made to identify at-risk population sub-groups.

The principal investigator assessed each study for a risk of bias using the tools developed by the Joanna-Briggs Institute (JBI) [ 17 ] (Supplementary file 1 ). Each study was classified as high or low risk of bias. Studies that did not score ‘yes’ on two or more applicable parameters were classified as high risk of bias [ 5 ]. Due to the limited research in this field, no studies were excluded based on risk of bias. The certainty of the evidence was assessed using the GRADE approach, with the body of evidence assessed on a scale of certainty: very low, low, moderate and high  [ 18 ]. Due to the heterogeneity of outcomes, and the reporting thereof, assessment of publication bias was not possible.

The funder of the study had no role in study design, data collection, analysis, interpretation, or writing of the report.

The updated search identified 18 621 non-duplicate records, and after screening 229 full-text articles were reviewed for inclusion, with a total of 29 studies included in the final analysis (Fig.  1 : flow chart). The included studies were conducted in 25 countries across six continents, including six Low-Income Countries (LIC), two Lower-Middle Income Countries (LMIC), one Upper-Middle Income Country (UMIC) and 16 High Income Countries (HIC) [ 19 ]. They included 25 Köppen-Geiger climate zones [ 16 ], and mean annual temperatures ranging from 2.1 °C in Norway to 30.0 °C in Burkina Faso [ 20 ] (Figs.  2 and 3 ). All studies were observational, with 17 cohort, five case-control and eight cross-sectional studies. The timeline of the data is from 1913 to 2019, and individuals included ranged in age from neonates to adults, and the elderly. The studies were grouped by outcomes as follows: behavioural, educational and socioeconomic ( n  = 6), cardiovascular disease ( n  = 6), respiratory disease ( n  = 3), growth and anthropometry ( n  = 2), mental health ( n  = 12) and longevity and mortality ( n  = 3). The measures of heat exposure were variable, with minimum, mean, maximum, and apparent temperature being utilized, as well as temperature variability, heat wave days and discreet shocks (number of times exposure exceeded a specific threshold). The majority of studies measured heat using mean temperature ( n  = 27/29). In addition, the statistical comparison was diverse, with some studies making a continuous linear comparison by degree Celsius, while others compared heat exposure by quartiles, amongst other categorical comparisons. Furthermore, heat exposure by any definition was not reported over the same timeframes, with some studies including variable periods before birth, during pregnancy and at birth in their analysis. Levels of temporal resolution of heat exposure were also diverse, ranging from monthly effects to effects observed over the entire gestational period, or year of birth. In addition, differing use of heat adaptation mechanisms was not uniformly described and adjusted for. Various confounders were adjusted for, and although not uniform, these were generally inadequate. The effect on female infants was greater than on male infants in eight of nine studies differentiating by sex, with increased effects on marginalised groups (African-Americans) in one further study. Overall, the quality of the evidence, as assessed by the GRADE approach, was low in respiratory and longevity outcome groups to very low in all other groups, primarily as a result of their observational nature and high risk of bias, due to insufficient consideration of confounders, and inadequate measures of heat exposure.

PRISMA flow diagram

Map showing countries where studies were conducted relative to mean annual temperature [ 21 ]

Map showing countries where studies were conducted relative to climate zones [ 16 ]

A total of six studies reported on behaviour, educational and socioeconomic outcomes, which were detrimentally affected by increases in heat exposure (Fig.  4 ; Table  1 ), although the quality of the evidence was very low . End-points were not uniform, but included earnings, completion of secondary school or higher education, number of years of schooling, and gamified cooperation-rates in a public-goods game (where test scores represent achieving maximal public benefit in hypothetical situations).

Two large studies reported a detrimental effect of heat exposure on adult income, with the greatest effect noted in first trimester exposure. These studies noted a reduction in earnings of up to 1·2% per 1 °C increase in temperature, with greater effects in females [ 22 ], and a decrease of $55.735 (standard error(SE): 15·425, P  < 0·01) annual earnings at 29–31 years old, per day exposure > 32 °C [ 26 ]. Two studies reported worse educational outcomes, with the greatest effect noted in the second trimester [ 23 ]. Rates of completing secondary education were found to be reduced by 0·2% per 1 °C increase in temperature ( P  = 0·05) [ 22 ], illiteracy was increased by 0·18% (SE=(0·0009); P  < 0·05) and mean years of schooling was lowered by 0.02 (SE=(0·009) P  = 0·07) [ 23 ]. Two studies reported a beneficial effect of heat exposure on educational outcomes, although both studies suffered from significant methodological flaws, and effects were < 0·01% when effect estimates were noted [ 24 , 27 ]. One small study reported lower cooperation rates by 20% ( P  < 0·01) in a public-goods game, with lower predicted public wellbeing [ 25 ].

The studies generally exhibited a dose-response effect with evidence for a critical threshold of effect of 28 °C in one study [ 22 ]. All studies were at a high risk of bias.

Figure showing vote counting across all outcome groups. No Effect = No direction of effect noted in study

Six studies reported on cardiovascular pathology and risk factors thereof, which were detrimentally affected by increased exposure to heat (Fig.  4 ; Table  2 ), although measures and surrogates of this outcome were heterogenous. The quality of the evidence was very low, and the sample sizes were small. Outcomes included blood pressure, a composite cardiovascular disease indicator, and specific cardiovascular disease risk factors such as diabetes mellitus (type I), insulin resistance, waste circumference, and triglyceride levels.

Three studies found a detrimental effect of heat exposure on hypertension rates, and increased blood pressure [ 31 ], with a maximum of 1·6 mm Hg increase noted per interquartile range (IQR) increase (95% Confidence interval (CI) = 0·2, 2·9, P  = 0·024) in children [ 30 ], with increased effects on women in the largest study ( N  = 11,237) [ 32 ]. Another study found increasing heat exposure at conception was detrimentally associated with an increase in coronary heart disease ( P  = 0·08) [ 32 ], although one of the smaller studies ( N  = 4286) found a beneficial effect of heat exposure at birth on diverse cardiovascular outcomes, including coronary heart disease ( P  = 0·03 for trend), triglyceride levels ( P  = 0·06 for trend) and insulin resistance ( P  = 0·04 for trend) [ 27 ]. One study found lower odds of type I diabetes mellitus with increasing heat exposure, with odds ratio (OR) = 0·73 (95%CI = 0·48, 1·09, P -value not stated) [ 28 ]. Another study did not detect statistically significant relationships between heat exposure and hypertension or a composite cardiovascular disease indicator, but did not provide effect estimates [ 29 ]. Five studies were at a high risk of bias [ 27 , 29 , 30 , 31 , 32 ], with only one case-control study at a low risk of bias [ 28 ].

Respiratory pathology was reported by three studies, assessing different outcomes. Outcomes were detrimentally associated with increasing heat (Fig.  4 ; Table  3 ), however the quality of the evidence was low . The outcomes were primarily measured in infants and children, with no studies on adult outcomes. The largest study ( N  = 1681) found that increasing heat exposure increased the odds of having childhood asthma [ 33 ], and another small study ( N  = 343) noted worsened lung function with increasing heat exposure [ 34 ].

An additional study noted increased odds of childhood pneumonia with increasing diurnal temperature variation (DTV) in pregnancy, with a maximum OR = 1·85 (95%CI = 1·24, 2·76) in the third trimester [ 35 ].

Exposure in the third trimester had the greatest effect across all three studies [ 33 , 34 , 35 ]. Females showed an increased susceptibility to heat exposure’s effects on lung function, but males were more susceptible to heat’s effect on childhood pneumonia. There was a critical threshold noted in the asthma study of 24·6 °C, with a dose-response effect. The asthma study was assessed as low risk of bias, however the other studies were at high risk.

Growth and anthropometry was reported on by two studies, with differing outcomes, although in both, heat exposure was associated with detrimental, although heterogenous, outcomes (Fig.  4 ; Table  4 ). The overall quality of the evidence was very low . One study found a positive association with heat exposure and increased body mass index (BMI), r  = 0·22 ( P  < 0·05) in the third trimester with greater effects noted in females and in African-Americans [ 36 ]. Another large study ( N  = 23 026) found increased odds of stunting (OR = 1·28, 95%CI = not stated, p  < 0·001) with a negative correlation with height noted ( r =-0·083 P  < 0·01) [ 37 ]. Effects were greatest in the first and third trimester. Both studies were at a high risk of bias.

Mental health was reported on by 12 studies. Increasing heat exposure generally had a detrimental association with mental health outcomes (Fig.  4 ; Table  5 ), although these were heterogenous. The overall quality of the evidence was very low . Five studies reported on schizophrenia rates, with only one study showing a strongly positive association of heat exposure at conception with schizophrenia rates ( r  = 0·50, p  < 0·025) [ 38 ]. Another study noted the same effect with increasing heat in the summer before birth, however this was not statistically significant [ 39 ]. The third study reported no association of this outcome [ 40 ], with another small study ( N  = 2985) showing a negative correlation with temperatures at birth, without reporting on heat exposure during other periods of gestation [ 41 ]. The fifth study failed to report direction of effect, but noted non-significant findings [ 42 ]. Six studies reported on eating disorders, with all six showing a detrimental effect with increasing heat exposure. Of the three studies on clinical anorexia nervosa, one reported increasing rates of anorexia nervosa compared to other eating disorders (χ²= 4·48, P  = 0·017) [ 43 ], another reported increasing rates of a restrictive-subtype (χ²= 3·18, P  = 0·04) as well as reporting worse assessments of restrictive behaviours [ 44 ], which was supported by a third study in a different setting [ 45 ]. Three studies examined non-clinical settings, with some inconsistent effects. The first study showed a weak positive association with heat exposure, and drive for thinness (Spearman’s ⍴ = 0·46, P  < 0·05) and bulimia scores (Spearman’s ⍴ = 0·25, P  < 0·05) [ 46 ], which was supported by a replication study [ 47 ], and one other study [ 48 ]. The most significant and consistent effects noted in the third trimester, at birth, and in females [ 47 , 48 ]. One study reported a beneficial effect of increased temperatures in the first trimester on rates of depression, however no other directions of effect were noted for other periods of exposure [ 49 ]. These studies were at a high risk of bias.

Increasing heat exposure had a detrimental effect on longevity and mortality across various outcomes (Fig.  4 ; Table  6 ), although despite large sample sizes, the quality of the evidence was low . One study found a negative correlation of heat exposure with longevity ( r =-0·667, P  < 0·001), with a greater effect on females [ 50 ]. A second study showed a detrimental effect on telomere length, as a predicter of longevity, with the greatest effect towards the end of gestation (3·29% shorter TL, 95%CI = − 4·67, − 1·88, per 1 C increase above 95th centile) [ 51 ]. Conversely, a third study noted no correlation with mortality [ 24 ]. All but the study on telomere length [ 51 ] were at a high risk of bias.

This study establishes significant patterns of effects amongst the outcomes reviewed, with increasing heat exposure being associated with an overall detrimental effect on multiple, diverse, long-term outcomes. These effects are likely to increase with rising temperatures, however modelling this is beyond the scope of this review.

The most notable detrimental outcomes are related to neurodevelopmental pathways, with behavioural, educational, socioeconomic and mental health outcomes consistently associated with increasing heat exposure, in addition to having the greatest body of literature to support this. Importantly, other systems such as the respiratory and cardiovascular systems also suggest harmful effects of heat exposure, culminating in detrimental associations with longevity and mortality. Some studies illustrated a possible beneficial effect in some disease-processes, such as coronary heart disease and depression showing the potential for shifting disease profiles with rising temperatures.

The detrimental effects of heat exposure became more significant with increasing temperatures, with many studies describing increasing effects beyond critical thresholds which, although varied across studies, suggest that there is a limit of heat adaption strategies, both biological and behavioural [ 52 , 53 ].

In addition, the effect of increasing heat exposure was associated with worse outcomes in already marginalised communities, such as women [ 22 , 32 , 34 , 36 , 44 , 47 , 48 , 50 ] and certain ethnic groups (African-Americans) [ 46 ]. The reasons for sub-population vulnerabilities are unclear and likely complex. In the case of female foetuses being more susceptible to changes in the in-utero environment, it is possible that there is a ‘survivorship bias’. This would occur if women with harmful exposure lose male infants during pregnancy at a higher rate, and thus the surviving female infants appear more at risk. However, despite an increased risk of early pregnancy loss, there are no studies that have assessed this differential vulnerability. This still has the effect of potentially increasing the burden of disease on an already marginalised group.

In the case of certain population groups being more at risk, it is likely that both physiological differences in vulnerability as well as socio-economic effect-modifiers exist to explain these differences, however, the included literature lacks sufficient evidence to assess this. The vulnerabilities of different populations to the long-term effects of heat exposure in-utero likely contributes to the unequal impacts of climate change that have already been established [ 54 ], and will be an important contributor to inequality with future increases in temperature. Further research in this area is critical to inform targeted redistributive interventions.

Although the associations may be clear, establishing causality is fraught with difficulty, with no consensus on an infallible approach [ 55 , 56 , 57 ]. However, it is prudent to highlight supporting evidence in this review.

The hypothesis that the in-utero environment had significant long-term impacts on the foetus was first suggested by Barker, in the context of maternal nutrition and cardiovascular disease [ 11 ]. Further studies supported this hypothesis, and expanded on the effects the in-utero environment has on the foetus and its long-term wellbeing [ 58 ]. Long-term heat exposure may also be associated with changes in nutritional availability [ 11 ], and is likely one of many complex but important environmental exposures in-utero.

Maternal comorbidities, associated with increasing heat exposure such as hypertensive disorders of pregnancy and gestational diabetes mellitus, are known to negatively affect the foetus in the long-term [ 59 , 60 ]. These comorbidities may be part of the long-term pathogenicity of heat exposure, through short-term exposure-outcome pathways. Placental dysfunction is central to the pathology of pre-eclampsia, and is a significant cause for foetal pathology [ 61 , 62 ]. The placenta is not auto-regulated and is therefore acutely affected by changes to blood volume, heart rate and blood pressure, culminating in cardiac output as it is delivered to the placenta as an end-organ with resultant negative effects on the foetus [ 63 ]. Heat-acclimatisation mechanisms are hypothesized to affect this delicate balance [ 52 , 64 ], with observational studies supporting this [ 64 ]. It has been suggested that heat exposure’s increase in inflammation is a possible causative mechanism for pre-term birth [ 5 , 52 ], but inflammation has numerous additional effects on the immune system and could prove an insult to the mother and developing foetus [ 62 , 65 ]. These effects may only manifest in the long-term.

Heat was one of the earliest described teratogens [ 66 ], with significant effects on neurodevelopment noted in animal models in keeping with the observed associations of this review [ 67 ]. Biological organisms are extremely dependent on heat as a trigger for various processes. Plants and animals undergo significant change in response to the seasons, which are often guided by fluctuations in temperature. These changes are often mediated by epigenetic mechanisms, allowing the modification and modulation of gene expression [ 68 , 69 ].

Thus, from an evolutionary perspective, DNA, is sensitive to changes in temperature. The mechanism of this sensitivity has been shown to be primarily epigenetic in nature [ 69 ]. Increasing heat results in modifications to histone deacetylation and DNA methylation [ 69 ]. This is required to provide fast-acting adaptions to acute stressors, but can have long-term effects too [ 70 ]. Thus, it is likely that humans are sensitive to changes in temperature, which can alter epigenetic modifications, and thus our exposome. This sensitivity, may have provided a survival benefit in times of increasing heat, or it may simply be a vestigial function which provides no survival benefit, and may in fact have detrimental effects [ 71 ]. Epigenetic changes have been shown to have significant effects on metabolic diseases and risk profiles, and an in-depth review is provided by Wu et al. [ 72 ]. The exact processes and genes involved would be an area requiring further research, where similar research exists on the effects of nutrition on exact epigenetic pathways [ 73 ]. An important pattern requiring further research involves the effect heat may have on neurodevelopment [ 67 , 74 ]. The above pathways provide additional mechanisms for the long-term lag between exposure-outcome pathways. In addition, acute heat exposure at the time of birth has been associated with various possibly pathogenic mechanisms such as preterm birth [ 5 ], low APGAR scores [ 75 ] and foetal distress [ 76 ], as well as a possible effect on the maternal microbiome and the seeding thereof to the neonate [ 10 , 64 , 77 , 78 ]. These effects, can all provide plausible causes for the long-term outcomes observed through short-term insults. The interplay of these, and additional factors is highlighted in Fig. 5 [ 79 ]. Importantly, the periods of vulnerability are likely different for these various pathways, but specific outcomes may have multiple periods of vulnerability through different pathways.

Causal pathways

The outcomes associated with increasing heat exposure highlight the health, social, and economic cost of global warming, establishing current estimates and future predictions for this are beyond the scope of this research but would provide a valuable area for future research. This would entail estimating disease-burden due to climate change through attribution studies. Traditional health impact studies conflate adverse outcomes from natural variations in climate (‘noise’) with adverse outcomes from anthropogenic climate change. However, not every climate-related adverse outcome is the result of anthropogenic climate change, and these effects are likely different in vulnerable populations. This highlights the benefit of studying and implementing effective heat adaptation strategies in areas where the greatest effect is likely to be observed, and where the greatest impacts in lessening the economic and human impact of global warming are possible [ 80 , 81 ].

Limitations

The difficulty in assessing the data is compounded by the heterogenous measures of heat exposure. No studies used widely accepted heat exposure indices that consider important environmental modifying factors like humidity and windspeed [ 82 , 83 ]. In addition, effect modifiers, heat acclimatisation and adaptation strategies were seldom considered [ 84 , 85 , 86 ]. It may be prudent for future studies to consider the measure of ionizing radiation exposure as an analogous environmental exposure, where different measures exist for the intensity, total quantity (a function of duration of exposure) and biologically-adjusted quantity absorbed [ 87 ]. Differing time-periods of exposure made it difficult to evaluate specific periods of sensitivity, which are likely different for various outcomes, depending on critical periods of development.

Despite consistency across different contexts in this review, the analysis of the distribution of the included studies highlights the unequal weight of studies towards relatively cooler climates, in regions with higher socioeconomic levels and likely greater heat adaptation uptake, and must therefore be interpreted in this context. It is possible that myriad factors that differ geographically, including physiological and socio-economic differences, will influence the effects of heat, and thus there is likely no underlying universal truth to associations and effect estimates.

Quantifying, describing and comparing the effect size across studies was rendered more difficult due to heterogenous statistical analyses.

Although some studies adjusted for possible confounding variables, not all reported on this, with the effects of seasonal, foetal, and maternal biological factors that may not lie on the causal pathway seldom considered [ 3 , 5 , 9 , 88 , 89 , 90 , 91 , 92 ].

Data extraction and assessment of risk of bias was not uniformly undertaken in duplicate due to resource constraints, which may predispose to extraction errors or bias. The high risk of bias of included studies, limits the utility of the overall assessment of effects and suggestions for further action. In addition, publication bias is likely skewing the results towards statistically significant detrimental results, with studies with smaller sample sizes not necessarily showing wider distribution of findings as would be expected.

Climate change, and in particular, global warming, is a significant emerging global public health threat, with far reaching, and disproportionate effects on the most vulnerable populations. The effects of increasing heat exposure in utero are associated with, and possibly causal in, wide-ranging long-term impacts on socioeconomic and health outcomes with a significant cost associated with increasing global temperatures. This association is as a result of a complex interplay of factors, including through direct and indirect effects on the mother and foetus. Further research is urgently required to elicit biological pathways, and targets for intervention as well as predicting future disease-burden and economic impacts through attribution studies.

Availability of data and materials

This study was a review of publicly available information data, with references to data sources made in the reference list.

Abbreviations

Apparent Temperature

Body Mass Index

Blood Pressure

Coronary Heart Disease

Confidence Interval

Diastolic Blood Pressure

Eating Disorder Inventory

Functional Residual Capacity

Interquartile Range

Non-Significant

Respiratory Rate

Systolic Blood Pressure

Standard Error

Telomere Length

World Health Organisation

Atwoli L, A HB, Benfield T, et al. Call for emergency action to limit global temperature increases, restore biodiversity and protect health. BMJ Lead. 2022;6(1):1–3.

Article   PubMed   Google Scholar  

Organization WM, World Meteorological O. WMO global annual to decadal climate update (Target years: 2023–2027). Geneva: WMO; 2023. p. 24.

Book   Google Scholar  

Lakhoo DP, Blake HA, Chersich MF, Nakstad B, Kovats S. The effect of high and low ambient temperature on infant health: a systematic review. Int J Environ Res Public Health. 2022;19(15):9109.

Article   PubMed   PubMed Central   Google Scholar  

Cissé G, McLeman R, Adams H, et al. Health, wellbeing and the changing structure of communities. In: Pörtner HO, Roberts DC, Tignor MMB, Poloczanska ES, Mintenbeck K, Alegría A, et al. editors. Climate change 2022: impacts, adaptation and vulnerability contribution of working group II to the sixth assessment report of the intergovernmental panel on climate change. Cambridge: Cambridge University Press; 2022.

Chersich MF, Pham MD, Areal A, et al. Associations between high temperatures in pregnancy and risk of preterm birth, low birth weight, and stillbirths: systematic review and meta-analysis. BMJ. 2020;371:m3811.

Birkmann J, Liwenga E, Pandey R, et al. Poverty, livelihoods and sustainable development. In: Pörtner HO, Roberts DC, Tignor MMB, Poloczanska ES, Mintenbeck K, Alegría A, et al. editors. Climate change 2022: impacts, adaptation and vulnerability contribution of working group II to the sixth assessment report of the intergovernmental panel on climate change. Cambridge: Cambridge University Press; 2022.

Watts N, Amann M, Ayeb-Karlsson S, et al. The Lancet countdown on health and climate change: from 25 years of inaction to a global transformation for public health. Lancet. 2018;391(10120):581–630.

Campbell-Lendrum D, Manga L, Bagayoko M, Sommerfeld J. Climate change and vector-borne diseases: what are the implications for public health research and policy? Philos Trans R Soc Lond B Biol Sci. 2015;370(1665):20130552.

Haghighi MM, Wright CY, Ayer J, et al. Impacts of high environmental temperatures on congenital anomalies: a systematic review. Int J Environ Res Public Health. 2021;18(9):4910.

Puthota J, Alatorre A, Walsh S, Clemente JC, Malaspina D, Spicer J. Prenatal ambient temperature and risk for schizophrenia. Schizophr Res. 2022;247:67–83.

Barker DJ. Intrauterine programming of coronary heart disease and stroke. Acta Paediatr Suppl. 1997;423:178–82; discussion 83.

Article   CAS   PubMed   Google Scholar  

Hales CN, Barker DJ. The thrifty phenotype hypothesis. Br Med Bull. 2001;60:5–20.

Manyuchi A, Dhana A, Areal A, et al. Title: systematic review to quantify the impacts of heat on health, and to assess the effectiveness of interventions to reduce these impacts. PROSPERO: International prospective register of systematic reviews. 2019, 42019140136. Available from: https://www.crd.york.ac.uk/PROSPEROFILES/118113_PROTOCOL_20181129.pdf .

Thomas J, Brunton J, Graziosi S. EPPI-Reviewer 4.0: software for research synthesis. EPPI Centre Software. London: Social Science Research Unit, Institute of Education, University of London; 2010.

Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:l6890.

Beck HE, Zimmermann NE, McVicar TR, Vergopolan N, Berg A, Wood EF. Present and future Köppen-Geiger climate classification maps at 1-km resolution. Sci Data. 2018;5(1):180214.

Barker TH, Stone JC, Sears K, et al. Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process. JBI Evid Synth. 2023;21(3):478.

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

World Bank Country and Lending Groups – World Bank Data Help Desk. Retrieved: June 01 2023. Available from: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups-files/112/906519-world-bank-country-and-lending-groups.html .

Home | Climate change knowledge portal. Retrieved: June 01 2023. Available from: https://climateknowledgeportal.worldbank.org/files/26/climateknowledgeportal.worldbank.org.html .

Harris I, Osborn TJ, Jones P, Lister D. Version 4 of the CRU TS monthly high-resolution gridded multivariate climate dataset. Sci Data. 2020;7(1):109.

Fishman R, Carrillo P, Russ J. Long-term impacts of exposure to high temperatures on human capital and economic productivity. J Environ Econ Manag. 2019;93:221–38.

Article   Google Scholar  

Hu Z, Li T. Too hot to handle: the effects of high temperatures during pregnancy on adult welfare outcomes. J Environ Econ Manag. 2019;94:236–53.

Wilde J, Apouey BH, Jung T. The effect of ambient temperature shocks during conception and early pregnancy on later life outcomes. Eur Econ Rev. 2017;97:87–107.

Duchoslav J. Prenatal temperature shocks reduce cooperation: evidence from public goods games in Uganda. Front Behav Neurosci. 2017;11: 249.

Isen A, Rossin-Slater M, Walker R. Relationship between season of birth, temperature exposure, and later life wellbeing. Proc Natl Acad Sci U S A. 2017;114(51):13447–52.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lawlor DA, Davey Smith G, Mitchell R, Ebrahim S. Temperature at birth, coronary heart disease, and insulin resistance: cross sectional analyses of the British women’s heart and health study. Heart. 2004;90(4):381–8.

Taha-Khalde A, Haim A, Karakis I, et al. Air pollution and meteorological conditions during gestation and type 1 diabetes in offspring. Environ Int. 2021;154: 106546.

Ho JY. Early-life environmental exposures and height, hypertension, and cardiovascular risk factors among older adults in India. Biodemography Soc Biol. 2015;61(2):121–46.

Warembourg C, Maitre L, Tamayo-Uria I, et al. Early-life environmental exposures and blood pressure in children. J Am Coll Cardiol. 2019;74(10):1317–28.

Warembourg C, Nieuwenhuijsen M, Ballester F, et al. Urban environment during early-life and blood pressure in young children. Environ Int. 2021;146: 106174.

Schreier N, Moltchanova E, Forsen T, Kajantie E, Eriksson JG. Seasonality and ambient temperature at time of conception in term-born individuals - influences on cardiovascular disease and obesity in adult life. Int J Circumpolar Health. 2013;72: 21466.

Zhang J, Bai S, Lin S, et al. Maternal apparent temperature during pregnancy on the risk of offspring asthma and wheezing: effect, critical window, and modifiers. Environ Sci Pollut Res Int. 2023;30(22):62924–37.

Guilbert A, Hough I, Seyve E, et al. Association of prenatal and postnatal exposures to warm or cold air temperatures with lung function in young infants. JAMA Netw Open. 2023;6(3):e233376.

Zheng X, Kuang J, Lu C, et al. Preconceptional and prenatal exposure to diurnal temperature variation increases the risk of childhood pneumonia. BMC Pediatr. 2021;21(1):192.

van Hanswijck de Jonge L, Stettler N, Kumanyika S, Stoa Birketvedt G, Waller G. Environmental temperature during gestation and body mass index in adolescence: new etiologic clues? Int J Obes Relat Metab Disord. 2002;26(6):765–9.

Randell H, Gray C, Grace K. Stunted from the start: early life weather conditions and child undernutrition in Ethiopia. Soc Sci Med. 2020;261: 113234.

Templer DI, Austin RK. Confirmation of relationship between temperature and the conception and birth of schizophrenics. J Orthomolecular Psychiatr. 1980;9(3):220-2.

Hare E, Moran P. A relation between seasonal temperature and the birth rate of schizophrenic patients. Acta Psychiatr Scand. 1981;63(4):396–405.

Watson CG, Kucala T, Tilleskjor C, Jacobs L. Schizophrenic birth seasonality in relation to the incidence of infectious diseases and temperature extremes. Arch Gen Psychiatry. 1984;41(1):85–90.

Tatsumi M, Sasaki T, Iwanami A, Kosuga A, Tanabe Y, Kamijima K. Season of birth in Japanese patients with schizophrenia. Schizophr Res. 2002;54(3):213–8.

McNeil T, Dalén P, Dzierzykray-Rogalska M, Kaij L. Birthrates of schizophrenics following relatively warm versus relatively cool summers. Arch Psychiat Nervenkr. 1975;221(1):1–10.

Watkins B, Willoughby K, Waller G, Serpell L, Lask B. Pattern of birth in anorexia nervosa I: early-onset cases in the United Kingdom. Int J Eat Disord. 2002;32(1):11–7.

Waller G, Watkins B, Potterton C, et al. Pattern of birth in adults with anorexia nervosa. J Nerv Ment Dis. 2002;190(11):752.

Willoughby K, Watkins B, Beumont P, Maguire S, Lask B, Waller G. Pattern of birth in anorexia nervosa. II: a comparison of early-onset cases in the southern and northern hemispheres. Int J Eat Disord. 2002;32(1):18–23.

van Hanswijck L, Meyer C, Smith K, Waller G. Environmental temperature during pregnancy and eating attitudes during teenage years: a replication and extension study. Int J Eat Disord. 2001;30(4):413–20.

van Hanswijck L, Waller G. Influence of environmental temperatures during gestation and at birth on eating characteristics in adolescence: a replication and extension study. Appetite. 2002;38(3):181–7.

Waller G, Meyer C, van Hanswijck de Jonge L. Early environmental influences on restrictive eating pathology among nonclinical females: the role of temperature at birth. Int J Eat Disord. 2001;30(2):204–8.

Boland MR, Parhi P, Li L, et al. Uncovering exposures responsible for birth season – disease effects: a global study. J Am Med Inform Assoc. 2018;25(3):275–88.

Flouris AD, Spiropoulos Y, Sakellariou GJ, Koutedakis Y. Effect of seasonal programming on fetal development and longevity: links with environmental temperature. Am J Hum Biol. 2009;21(2):214–6.

Martens DS, Plusquin M, Cox B, Nawrot TS. Early biological aging and fetal exposure to high and low ambient temperature: a birth cohort study. Environ Health Perspect. 2019;127(11):117001.

Samuels L, Nakstad B, Roos N, et al. Physiological mechanisms of the impact of heat during pregnancy and the clinical implications: review of the evidence from an expert group meeting. Int J Biometeorol. 2022;66(8):1505–13.

Ravanelli N, Casasola W, English T, Edwards KM, Jay O. Heat stress and fetal risk. Environmental limits for exercise and passive heat stress during pregnancy: a systematic review with best evidence synthesis. Br J Sports Med. 2019;53(13):799–805.

Islam SN, Winkel J. Climate change and social inequality, DESA working paper no. 152, United Nations Department of Economic and Social Affiars (2017). Available from: https://www.un.org/esa/desa/papers/2017/wp152_2017.pdf .

Rothman KJ, Greenland S. Causation and causal inference in epidemiology. Am J Public Health. 2005;95(S1):S144–150.

Fedak KM, Bernal A, Capshaw ZA, Gross S. Applying the Bradford Hill criteria in the 21st century: how data integration has changed causal inference in molecular epidemiology. Emerg Themes Epidemiol. 2015;12(1):14.

Hill AB. The environment and disease: association or causation? J R Soc Med. 2015;108(1):32–7.

Gluckman PD, Hanson MA, Cooper C, Thornburg KL. Effect of in Utero and early-life conditions on adult health and disease. N Engl J Med. 2008;359(1):61–73.

Duley L. The global impact of pre-eclampsia and eclampsia. Semin Perinatol. 2009;33(3):130–7.

Bianco ME, Kuang A, Josefson JL, et al. Hyperglycemia and adverse pregnancy outcome follow-up study: newborn anthropometrics and childhood glucose metabolism. Diabetologia. 2021;64(3):561–70.

Fisher SJ. The placental problem: linking abnormal cytotrophoblast differentiation to the maternal symptoms of preeclampsia. Reprod Biol Endocrinol. 2004;2: 53.

Steegers EAP, von Dadelszen P, Duvekot JJ, Pijnenborg R. Pre-eclampsia. Lancet. 2010;376(9741):631–44.

von Dadelszen P, Ornstein MP, Bull SB, Logan AG, Koren G, Magee LA. Fall in mean arterial pressure and fetal growth restriction in pregnancy hypertension: a meta-analysis. Lancet. 2000;355(9198):87–92.

Bonell A, Sonko B, Badjie J, et al. Environmental heat stress on maternal physiology and fetal blood flow in pregnant subsistence farmers in the Gambia, West Africa: an observational cohort study. Lancet Planet Health. 2022;6(12):e968–76.

Redline RW. Placental inflammation. Semin Neonatol. 2004;9(4):265–74.

Alsop FM. The effect of abnormal temperatures upon the developing nervous system in the chick embryos. Anat Rec. 1919;15(6):306–31.

Edwards MJ. Hyperthermia as a teratogen: a review of experimental studies and their clinical significance. Teratog Carcinog Mutagen. 1986;6(6):563–82.

Nicotra AB, Atkin OK, Bonser SP, et al. Plant phenotypic plasticity in a changing climate. Trends Plant Sci. 2010;15(12):684–92.

McCaw BA, Stevenson TJ, Lancaster LT. Epigenetic responses to temperature and climate. Integr Comp Biol. 2020;60(6):1469–80.

Horowitz M. Epigenetics and cytoprotection with heat acclimation. J Appl Physiol (1985). 2016;120(6):702–10.

Murray KO, Clanton TL, Horowitz M. Epigenetic responses to heat: from adaptation to maladaptation. Exp Physiol. 2022;107(10):1144–58.

Wu YL, Lin ZJ, Li CC, et al. Epigenetic regulation in metabolic diseases: mechanisms and advances in clinical study. Sig Transduct Target Ther. 2023;8:98.

Sookoian S, Gianotti TF, Burgueño AL, Pirola CJ. Fetal metabolic programming and epigenetic modifications: a systems biology approach. Pediatr Res. 2013;73(2):531–42.

Graham JM, Marshall J. Edwards: discoverer of maternal hyperthermia as a human teratogen. Birth Defects Res Clin Mol Teratol. 2005;73(11):857–64.

Article   CAS   Google Scholar  

Andalón M, Azevedo JP, Rodríguez-Castelán C, Sanfelice V, Valderrama-González D. Weather shocks and health at Birth in Colombia. World Dev. 2016;82:69–82.

Cil G, Cameron TA. Potential climate change health risks from increases in heat waves: abnormal birth outcomes and adverse maternal health conditions. Risk Anal. 2017;37(11):2066–79.

Huus KE, Ley RE. Blowing hot and cold: body temperature and the microbiome. mSystems. 2021;6(5):e00707–21.

Wen C, Wei S, Zong X, Wang Y, Jin M. Microbiota-gut-brain axis and nutritional strategy under heat stress. Anim Nutr. 2021;7(4):1329–36.

ACOG Committee. Physical activity and exercise during pregnancy and the postpartum period, ACOG Committee Opinion No. 804. Obstet Gynecol. 2020;135e:88.

Google Scholar  

Lee H, Calvin K, Dasgupta D, et al. Synthesis report of the IPCC Sixth Assessment Report (AR6). Geneva: IPCC; 2023. p. 35–115.

Stern N. Economics: current climate models are grossly misleading. Nature. 2016;530(7591):407–9.

McGregor GR, Vanos JK. Heat: a primer for public health researchers. Public Health. 2018;161:138–46.

Gao C, Kuklane K, Östergren P-O, Kjellstrom T. Occupational heat stress assessment and protective strategies in the context of climate change. Int J Biometeorol. 2018;62(3):359–71.

Alele F, Malau-Aduli B, Malau-Aduli A, Crowe M. Systematic review of gender differences in the epidemiology and risk factors of exertional heat illness and heat tolerance in the armed forces. BMJ Open. 2020;10(4): e031825.

Khosla R, Jani A, Perera R. Health risks of extreme heat. BMJ. 2021;375:n2438.

Spector JT, Masuda YJ, Wolff NH, Calkins M, Seixas N. Heat exposure and occupational injuries: review of the literature and implications. Curr Environ Health Rep. 2019;6(4):286–96.

Measuring Radiation. Centers for disease control and prevention. 2015. Retrieved: May 15 2023. Available from: https://www.cdc.gov/nceh/radiation/measuring.html .

Molina-Vega M, Gutiérrez-Repiso C, Muñoz-Garach A, et al. Relationship between environmental temperature and the diagnosis and treatment of gestational diabetes mellitus: an observational retrospective study. Sci Total Environ. 2020;744: 140994.

Su W-L, Lu C-L, Martini S, Hsu Y-H, Li C-Y. A population-based study on the prevalence of gestational diabetes mellitus in association with temperature in Taiwan. Sci Total Environ. 2020;714: 136747.

Part C, le Roux J, Chersich M, et al. Ambient temperature during pregnancy and risk of maternal hypertensive disorders: a time-to-event study in Johannesburg, South Africa. Environ Res. 2022;212: 113596 (Pt D)).

Shashar S, Kloog I, Erez O, et al. Temperature and preeclampsia: epidemiological evidence that perturbation in maternal heat homeostasis affects pregnancy outcome. PLoS One. 2020;15(5): e0232877.

Hajdu T, Hajdu G. Post-conception heat exposure increases clinically unobserved pregnancy losses. Sci Rep. 2021;11(1):1987.

Download references

This research was funded through the HE2AT Centre, a grant supported by the NIH Common Fund and NIEHS, which is managed by the Fogarty International Centre NIH award number: 1U54TW012083-01, and has received funding through the HIGH Horizons project from the European Union’s Horizon Framework Programme under Grant Agreement No. 101057843. Neither funding group influenced the methodology or reporting of this review.

Author information

Authors and affiliations.

Climate and Health Directorate, Wits RHI, University of the Witwatersrand, Johannesburg, South Africa

Nicholas Brink, Darshnika P. Lakhoo, Ijeoma Solarin, Gloria Maimela & Matthew F. Chersich

King’s College, London, United Kingdom

Peter von Dadelszen

MRC Developmental Pathways for Health Research Unit, University of the Witwatersrand, Johannesburg, South Africa

Shane Norris

You can also search for this author in PubMed   Google Scholar

• Admire Chikandiwa
• , Britt Nakstad
• , Caradee Y. Wright
• , Lois Harden
• , Nathalie Roos
• , Stanley M. F. Luchters
• , Cherie Part
• , Ashtyn Areal
• , Marjan Mosalam Haghighi
• , Albert Manyuchi
• , Melanie Boeckmann
• , Minh Duc Pham
• , Robyn Hetem
•  & Dilara Durusu

Contributions

NB updated the literature search, data extraction, created figures and compiled initial and final drafts, DL updated the literature search, data extraction and reviewed initial and final drafts, IS updated the literature search, data extraction and reviewed initial and final drafts, GM reviewed initial and final drafts, PvD and SN reviewed the final draft and provided input on the causal pathways. MFC conceptualised the research, conducted the initial literature search and data extraction, and reviewed initial and final drafts. The climate and heat-health study group contributed to conceptualisation, literature search, extraction and reviewed the final draft. All authors have reviewed and approve the final manuscript.

Corresponding author

Correspondence to Nicholas Brink .

Ethics declarations

Ethics approval and consent to participate.

This study was a review of publicly available information, and did not require review or approval by an ethics board.

Consent for publication

Not applicable.

Competing interests

NB holds investments in companies involved in the production, distribution and use of fossil-fuels through managed funds and indices. MFC and DL hold investments in the fossil fuel industry through their pension fund, as per the policy of the Wits Health Consortium. The University of the Witwatersrand holds investments in the fossil fuel industry through their endowments and other financial reserves.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Full data extraction table including risk of bias and GRADE assessments.

Additional file 2.

Search terms for Medline(PubMed) and Web of Science.

Additional file 3.

Author list for Climate and Heat-Health Study Group. Individual JBI risk of bias assessment forms, and excluded studies metadata from EPPI reviewer available on request.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Brink, N., Lakhoo, D.P., Solarin, I. et al. Impacts of heat exposure in utero on long-term health and social outcomes: a systematic review. BMC Pregnancy Childbirth 24 , 344 (2024). https://doi.org/10.1186/s12884-024-06512-0

Download citation

Received : 26 October 2023

Accepted : 11 April 2024

Published : 04 May 2024

DOI : https://doi.org/10.1186/s12884-024-06512-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Climate change
• Heat exposure
• Long-term effects
• Socioeconomic impact
• Maternal health
• Child health
• Epigenetics
• Metabolic disease

BMC Pregnancy and Childbirth

ISSN: 1471-2393

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Multidiscip Healthc

Information bias in health research: definition, pitfalls, and adjustment methods

Alaa althubaiti.

Department of Basic Medical Sciences, College of Medicine, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia

As with other fields, medical sciences are subject to different sources of bias. While understanding sources of bias is a key element for drawing valid conclusions, bias in health research continues to be a very sensitive issue that can affect the focus and outcome of investigations. Information bias, otherwise known as misclassification, is one of the most common sources of bias that affects the validity of health research. It originates from the approach that is utilized to obtain or confirm study measurements. This paper seeks to raise awareness of information bias in observational and experimental research study designs as well as to enrich discussions concerning bias problems. Specifying the types of bias can be essential to limit its effects and, the use of adjustment methods might serve to improve clinical evaluation and health care practice.

Introduction

Bias can be defined as any systematic error in the design, conduct, or analysis of a study. In health studies, bias can arise from two different sources; the approach adopted for selecting subjects for a study or the approach adopted for collecting or measuring data from a study. These are, respectively, termed as selection bias and information bias. 1 Bias can have different effects on the validity of medical research findings. In epidemiological studies, bias can lead to inaccurate estimates of association, or over- or underestimation of risk parameters. Allocating the sources of bias and their impacts on final results are key elements for making valid conclusions. Information bias, otherwise known as misclassification, is one of the most common sources of bias that affects the validity of health research. It originates from the approach that is utilized to obtain or confirm study measurements. These measurements can be obtained by experimentation (eg, bioassays) or observation (eg, questionnaires or surveys).

Medical practitioners are conscious of the fact that the results of their investigation can be deemed invalid if they do not account for major sources of bias. While a number of studies have discussed different types of bias, 2 – 4 the problem of bias is still frequently ignored in practice. Often bias is unintentionally introduced into a study by researchers, making it difficult to recognize, but it can also be introduced intentionally. Thus, bias remains a very sensitive issue to address and discuss openly. The aim of this paper is to raise the awareness of three specific forms of information bias in observational and experimental medical research study designs. These are self-reporting bias, and the often-marginalized measurement error bias, and confirmation bias. We present clear and simple strategies to improve the decision-making process. As will be seen, specifying the type of bias can be essential for limiting its implications. The “Self-reporting bias” section discusses the problem of bias in self-reporting data and presents two examples of self-reporting bias, social desirability bias and recall bias. The “Measurement error bias” section describes the problem of measurement error bias, while the “Confirmation bias” section discusses the problem of confirmation bias.

Self-reporting bias

Self-reporting is a common approach for gathering data in epidemiologic and medical research. This method requires participants to respond to the researcher’s questions without his/her interference. Examples of self-reporting include questionnaires, surveys, or interviews. However, relative to other sources of information, such as medical records or laboratory measurements, self-reported data are often argued to be unreliable and threatened by self-reporting bias.

The issue of self-reporting bias represents a key problem in the assessment of most observational (such as cross-sectional or comparative, eg, case–control or cohort) research study designs, although it can still affect experimental studies. Nevertheless, when self-reporting data are correctly utilized, they can help to provide a wider range of responses than many other data collection instruments. 5 For example, self-reporting data can be valuable in obtaining subjects’ perspectives, views, and opinions.

There are a number of aspects of bias that accompany self-reported data and these should be taken into account during the early stages of the study, particularly when designing the self-reporting instrument. Bias can arise from social desirability, recall period, sampling approach, or selective recall. Here, two examples of self-reporting bias are discussed: social desirability and recall bias.

Social desirability bias

When researchers use a survey, questionnaire, or interview to collect data, in practice, the questions asked may concern private or sensitive topics, such as self-report of dietary intake, drug use, income, and violence. Thus, self-reporting data can be affected by an external bias caused by social desirability or approval, especially in cases where anonymity and confidentiality cannot be guaranteed at the time of data collection. For instance, when determining drug usage among a sample of individuals, the results could underestimate the exact usage. The bias in this case can be referred to as social desirability bias.

Overcoming social desirability bias

The main strategy to prevent social desirability bias is to validate the self-reporting instrument before implementing it for data collection. 6 – 11 Such validation can be either internal or external. In internal validation, the responses collected from the self-reporting instrument are compared with other data collection methods, such as laboratory measurements. For example, urine, blood, and hair analysis are some of the most commonly used validation approaches for drug testing. 12 – 14 However, when laboratory measurements are not available or it is not possible to analyze samples in a laboratory for reasons such as cost and time, external validation is often used. There are different methods, including medical record checks or reports from family or friends to examine externally the validity of the self-reporting instrument. 12 , 15

Note that several factors must be accounted for in the design and planning of the validation studies, and in some cases, this can be very challenging. For example, the characteristics of the sample enrolled in the validation study should be carefully investigated. It is important to have a random selection of individuals so that results from the validation can be generalized to any group of participants. When the sampling approach is not random and subjective, the results from the validation study can only apply to the same group of individuals, and the differences between the results from validation studies and self-reporting instruments cannot be used to adjust for differences in any group of individuals. 12 , 16 Hence, when choosing a predesigned and validated self-reporting instrument, information on the group of participants enrolled in the validation process should be obtained. This information should be provided as part of the research paper and if not, further communication is needed with the authors of the work in order to obtain them. For example, if the target of the study is to examine drug use among the general population with no specific background, then a self-reporting instrument that has been validated on a sample of the population having general characteristics should be used. In addition, combining more than one validation technique or the use of multiple data sources may increase the validity of the results.

Moreover, the possible effects of social desirability on study outcomes should be identified during the design phase of the data collection method. As such, measurement scales such as Marlowe–Crowne Social Desirability Scale 17 or Martin–Larsen Approval Motivation score 18 would be useful to identify and measure the social desirability aspect of the self-reported information.

Recall bias

Occasionally, study participants can erroneously provide responses that depend on his/her ability to recall past event. The bias in this case can be referred to as recall bias, as it is a result of recall error. This type of bias often occurs in case–control or retrospective cohort study designs, where participants are required to evaluate exposure variables retrospectively using a self-reporting method, such as self-administered questionnaires. 19 – 21

While the problems posed by recall bias are no less than those caused by social desirability, recall bias is more common in epidemiologic and medical research. The effect of recall bias has been investigated extensively in the literature, with particular focus on survey methods for measuring dietary or food intake. 22 – 25 If not given proper consideration, it can either underestimate or overestimate the true effect or association. For example, a recall error in a dietary survey may result in underestimates of the association between dietary intake and disease risk. 24

Overcoming recall bias

To overcome recall bias, it is important to recognize cases where recall errors are more likely to occur. Recall bias was found to be related to a number of factors, including length of the recall period (ie, short or long times of clinical assessment), characteristics of the disease under investigation (eg, acute, chronic), patient/sample characteristics (eg, age, accessibility), and study design (eg, duration of study). 26 – 30 For example, in a case–control study, cases are often more likely to recall exposure to risk factors than healthy controls. As such, true exposure might be underreported in healthy controls and overreported in the cases. The size of the difference between the observed rates of exposure to risk factors in cases and controls will consequently be inflated, and, in turn, the observed odds ratio would also increase.

Many solutions have proven to be useful for minimizing and, in some cases, eliminating recall bias. For example, to select the appropriate recall period, all the above-mentioned factors should be considered in relation to recall bias. Previous literature showed that a short recall period is preferable to a long one, particularly when asking participants about routine or frequent events. In addition, the recall period can be stratified according to participant demographics and the frequency of events they experienced. For example, when participants are expected to have a number of events to recall, they can be asked to describe a shorter period than those who would have fewer events to recall. Other methods to facilitate participant’s recall include the use of memory aids, diaries, and interviewing of participants prior to initiating the study. 31

However, when it is not possible to eliminate recall errors, it is important to obtain information on the error characteristics and distribution. Such information can be obtained from previous or pilot studies and is useful when adjusting the subsequent analyses and choosing a suitable statistical approach for data analysis. It must be borne in mind that there are fundamental differences between statistical approaches to make adjustments that address different assumptions about the errors. 22 , 32 – 36 When conducting a pilot study to examine error properties, a high level of accuracy and careful planning are needed, as validation largely depends on biological testing or laboratory measurements, which, besides being costly to conduct, are often subject to measurement errors. For example, in a validation study to estimate sodium intake using a 24-hour urinary excretion method, the estimated sodium intake tended to be lower than the true amount. 25 Despite these potential shortcomings, the use of biological testing or laboratory measurements is one of the most credible approaches to validate self-reported data. More information on measurement errors is provided in the next section.

It is important to point out that overcoming recall bias can be difficult in practice. In particular, bias often accompanies results from case–control studies. Hence, case–control studies can be conducted in order to generate a research hypothesis, but not to evaluate prognoses or treatment effects. Finally, more research is needed to assess the impact of recall bias. Studies to evaluate the agreements between responses from self-reporting instruments and gold-standard data sources should be conducted. Such studies can provide medical researchers with information concerning the validity of the self-reporting instrument before utilizing it in a study or for a disease under investigation. Other demographic factors associated with recall bias can also be identified. For instance, a high agreement was found between self-reported questionnaires and medical record diagnoses of diseases such as diabetes, hypertension, myocardial infarction, and stroke but not for heart failure. 37

Measurement error bias

Device inaccuracy, environmental conditions in the laboratory, or self-reported measurements are all sources of errors. If these errors occur, observed measurements will differ from the actual values, and this is often referred to as measurement error, instrumental error, measurement imprecision, or measurement bias. These errors are encountered in both observational (such as cohort studies) and experimental (such as laboratory tests) study designs. For example, in an observational study of cardiovascular disease, measurements of blood cholesterol levels (as a risk factor) often included errors.

An analysis that ignores the effect of measurement error on the results can be referred to as a naïve analysis. 22 Results obtained from using naïve analysis can be potentially biased and misleading. Such results can include inconsistent (or biased) and/or inefficient estimators of regression parameters, which may yield poor inferences about confidence intervals and the hypothesis testing of parameters. 22 , 34

However, random sampling should not be confused with measurement error variability. Commonly used statistical methods can address the sampling variability during data analysis, but they do not account for uncertainty due to measurement error.

Measurement error bias has rarely been discussed or adjusted for in the medical research literature, except in the field of forensic medicine, where forensic toxicologists have undoubtedly the most theoretical understanding of measurement bias as it is particularly relevant for their type of research. 38 Known examples of measurement error bias have also been reported for blood alcohol content analyses. 38 , 39

Systematic and random error

Errors could occur in a random or systematic manner. When errors are systematic, the observed measurements deviate from true values in a consistent manner, that is, they are either consistently higher or lower than the true values. For example, a device could be calibrated improperly and subtract a certain amount from each measurement. By not accounting for this deviation in the measurement, the results will contain systematic errors and in this case, true measurements would be underestimated.

For random errors, the deviation of the observed from true values is not consistent, causing errors to occur in an unpredictable manner. Such errors will follow a distribution, in the simplest case a gaussian (also called normal or bell-shaped) distribution, and will have a mean and standard deviation. When the mean is zero, the measured value should be reported within an interval around zero and an estimated amount of deviation from the actual value. When the target value is reported to fall within a range or interval of minimum and maximum levels, the size of the interval depends mainly on the size of measurement errors, that is, the larger the errors, the larger the uncertainty and hence the wider the intervals, which could affect the precision level.

Random errors could also be proportional to the measured amount. In this case, errors can be referred to as multiplicative or non-gaussian errors. 36 These random errors occur due to uncontrollable and possibly unknown experimental factors, such as laboratory environment conditions that affect concentrations in biological experiments. Examples of non-gaussian errors can be found in breath alcohol measurements, in which the variability around the measurement increases with increasing alcohol concentrations. 40 – 42

Adjusting for measurement error bias

The type and distribution of measurement errors determines the type of adjusting method. 34 When errors are systematic, calibration methods can be used to reduce their effects on the results. These methods are based on a reference measurement that can be obtained from a previous or pilot study, and used as the correct quantity to calibrate the study measurements. As such, simple mathematical tools can be used if the errors are estimated. The adjustment methods for systematic errors are simpler to apply than those for random errors.

Significant efforts have been made to develop sophisticated statistical approaches that adjust for the effect of random measurement errors. 34 Commonly available and popular statistical software packages, such as R Software Package ( http://www.r-project.org ) and the Stata (Stata Corporation, College Station, TX, USA) include features that allow adjustments to be made for random measurement errors. Some of the bias adjustment methods include simulation–extrapolation, regression calibration, and the instrumental variable approach. 34 In order to select the best adjustment approach, knowledge of the error properties is essential. For example, the amount of standard deviation and the shape of error distribution should be identified through a previous or pilot study. Therefore, evaluation of the measuring technique is recommended to identify the error properties before starting the actual measuring procedure. Error properties should also be identified for survey measurement errors, in which methods for examining the reliability and validity of the survey can be used such as test–retest and record checks.

A simpler approach used by practitioners to minimize errors in epidemiologic studies is replication; in this method, replicates of the risk factor (eg, long-term average nutrients) are available and the mean of these values is calculated and used to present an approximate value relative to the actual value. 43 These replicates can also be used to estimate the measurement error variance and apply an adjusted statistical approach.

Confirmation bias

Placing emphasis on one hypothesis because it does not contradict investigator beliefs is called confirmation bias, otherwise known as confirmatory, ascertainment, or observer bias. Confirmation bias is a type of psychological bias in which a decision is made according to the subject’s preconceptions, beliefs, or preferences. Such bias results from human errors, including imprecision and misconception. Confirmation bias can also emerge owing to overconfidence, which results in contradictory evidence being ignored or overlooked. 44 In medicine, confirmation bias is one of the main reasons for diagnostic errors and may cause inaccurate diagnosis and improper treatment management. 45 – 47

An understanding of how the results of a medical investigation are affected by confirmation bias is important. Many studies have demonstrated that any aspect of investigation that requires human judgment is subject to confirmation bias, 48 – 50 which was also found to influence the inclusion and exclusion criteria of randomized controlled trial study designs. 51 There are many examples of confirmation bias in the medical literature, some of which are even illustrated in DNA matching. 16

Overcoming confirmation bias

Researchers have shown that not accounting for confirmation bias could affect the reliability of the investigation. Several studies in the literature also suggest a number of approaches for dealing with this type of bias. An approach that is often used is to conduct multiple and independent checks on study subjects across different laboratories or through consultation with other researchers who may have differing opinions. Through this approach, scientists can seek independent feedback and confirmation. 52 The use of blinding or masking procedures, whether single- or double-blinded, is important for enhancing the reliability of scientific investigations. These approaches have proven to be very useful in clinical trials, as they protect final conclusions from confirmation bias. The blinding may involve participant, treating clinician, recruiter, and/or assessor.

In addition, researchers should be encouraged to evaluate evidence objectively, taking into account contradictory evidence, and alter perspectives through specific education and training programs, 53 , 54 with no overcorrection or change in the researcher’s decision making. 55

However, the problem with the above suggestions is that they become ineffective if specific factors of bias are not accounted for. For example, researchers could reach conclusions in haste due to external pressure to obtain results, which can be particularly true in highly sensitive clinical trials. Bias in such cases is a very sensitive issue, as it might affect the validity of the investigation. We can, however, avoid the possibility of such bias by developing and following well-designed study protocols.

Finally, in order to overcome confirmation bias and enhance the reliability of investigations, it is important to accept that bias is a part of investigations. Quantifying this inevitable bias and its potential sources must be part of well-developed conclusions.

Bias in epidemiologic and medical research is a major problem. Understanding the possible types of bias and how they affect research conclusions is important to ensure the validity of findings. This work discussed some of the most common types of information bias, namely self-reporting bias, measurement error bias, and confirmation bias. Approaches for overcoming bias through the use of adjustment methods were also presented. A summary of study types with common data collection methods, type of information bias and adjusting or preventing strategies is presented in Table 1 . The framework described in this work provides epidemiologists and medical researchers with useful tools to manage information bias in their scientific investigations. The consequences of ignoring this bias on the validity of the results were also described.

Type of study designs, common data collection methods, type of bias, and adjusting strategies

Bias is often not accounted for in practice. Even though a number of adjustment and prevention methods to mitigate bias are available, applying them can be rather challenging due to limited time and resources. For example, measurement error bias properties might be difficult to detect, particularly if there is a lack of information about the measuring instrument. Such information can be tedious to obtain as it requires the use of validation studies and, as mentioned before, these studies can be expensive and require careful planning and management. Although conducting the usual analysis and ignoring measurement error bias may be tempting, researchers should always follow the practice of reporting any evidence of bias in their results.

In order to minimize or eliminate bias, careful planning is needed in each step of the research design. For example, several rules and procedures should be followed when designing self-reporting instruments. Training of interviewers is important in minimizing such type of bias. On the other hand, the effect of measurement error can be difficult to eliminate since measuring devices and algorithms are often imperfect. A general rule is to revise the level of accuracy of the measuring instrument before utilizing it for data collection. Such adjustments should greatly reduce any possible defects. Finally, confirmation bias can be eliminated from the results if investigators take into account different factors that can affect human judgment.

Researchers should be familiar with sources of bias in their results, and additional effort is needed to minimize the possibility and effects of bias. Increasing the awareness of the possible shortcomings and pitfalls of decision making that can result in bias should begin at the medical undergraduate level and students should be provided with examples to demonstrate how bias can occur. Moreover, adjusting for bias or any deficiency in the analysis is necessary when bias cannot be avoided. Finally, when presenting the results of a medical research study, it is important to recognize and acknowledge any possible source of bias.

The author reports no conflicts of interest in this work.