research methodology in medical education

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Singapore Med J
• v.59(12); 2018

Qualitative research essentials for medical education

Sayra m cristancho.

1 Department of Surgery and Faculty of Education, Schulich School of Medicine and Dentistry, Western University, Canada

2 Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University, Canada

Mark Goldszmidt

3 Department of Medicine, Schulich School of Medicine and Dentistry, Western University, Canada

Lorelei Lingard

Christopher watling.

4 Postgraduate Medical Education, Schulich School of Medicine and Dentistry, Western University, Canada

This paper offers a selective overview of the increasingly popular paradigm of qualitative research. We consider the nature of qualitative research questions, describe common methodologies, discuss data collection and analysis methods, highlight recent innovations and outline principles of rigour. Examples are provided from our own and other authors’ published qualitative medical education research. Our aim is to provide both an introduction to some qualitative essentials for readers who are new to this research paradigm and a resource for more experienced readers, such as those who are currently engaged in a qualitative research project and would like a better sense of where their work sits within the broader paradigm.

INTRODUCTION

Are you a medical education researcher engaged in qualitative research and wondering if you are on the right track? Are you contemplating a qualitative research project and not sure how to get started? Are you reading qualitative manuscripts and making guesses about their quality? This paper offers a selective overview of the increasingly popular domain of qualitative research. We consider the nature of qualitative research questions, describe common methodologies, discuss data collection and analysis methods, highlight recent innovations, and outline principles of rigour. The aim of this paper is to educate newcomers through introductory explanations while stimulating more experienced researchers through attention to current innovations and emerging debates.

WHAT IS QUALITATIVE RESEARCH?

Qualitative research is naturalistic; the natural setting – not the laboratory – is the source of data. Researchers go where the action is; to collect data, they may talk with individuals or groups, observe their behaviour and their setting, or examine their artefacts.( 1 ) As defined by leading qualitative researchers Denzin and Lincoln, qualitative research studies social and human phenomena in their natural settings, attempting to make sense of or interpret these phenomena in terms of the meanings participants bring to them.( 2 )

Because qualitative research situates itself firmly in the world it studies, it cannot aim for generalisability. Its aim is to understand, rather than erase, the influence of context, culture and perspective. Good qualitative research produces descriptions, theory or conceptual understanding that may be usefully transferred to other contexts, but users of qualitative research must always carefully consider how the principles unearthed might unfold in their own distinct settings.

WHAT QUESTIONS ARE APPROPRIATE FOR QUALITATIVE RESEARCH?

Meaningful education research begins with compelling questions. Research methods translate curiosity into action, facilitating exploration of those questions. Methods must be chosen wisely; some questions lend themselves to certain methodological approaches and not to others.

In recent years, qualitative research methods have become increasingly prominent in medical education. The reason is simple: some of the most pressing questions in the field require qualitative approaches for meaningful answers to be found.

Qualitative research examines how things unfold in real world settings. While quantitative research approaches that dominate the basic and clinical sciences focus on testing hypotheses, qualitative research explores processes, phenomena and settings ( Box 1 ). For example, the question “Does the introduction of a mandatory rural clerkship increase the rate of graduates choosing to practise in rural areas? ” demands a quantitative approach. The question embeds a hypothesis – that a mandatory rural clerkship will increase the rate of graduates choosing to practise in rural areas – and so the research method must test whether or not that hypothesis is true. But the question “ How do graduating doctors make choices about their practice location? ” demands a qualitative approach. The question does not embed a hypothesis; rather, it explores a process of decision-making.

Qualitative research questions:

Many issues in medical education could be examined from either a quantitative or qualitative approach; one approach is not inherently superior. The questions that drive the research as well as the products that derive from it are, however, fundamentally different. Consider two approaches to studying the issue of online learning. A quantitative researcher might ask, “ What is the effect of an online learning module on medical students’ end-of-semester OSCE [objective structured clinical examination] scores? ”, while a qualitative researcher might ask “ How do medical students make choices about using online learning resources? ” Although the underlying issue is the same – the phenomenon of online learning in medical school – the studies launched by these questions and the products of those studies will look very different.

WHAT ARE QUALITATIVE METHODOLOGIES AND WHY ARE THEY IMPORTANT?

Executing rigorous qualitative research requires an understanding of methodology – the principles and procedures that define how the research is approached. Far from being monolithic, the world of qualitative research encompasses a range of methodologies, each with distinctive approaches to inquiry and characteristic products. Methodologies are informed by the researcher’s epistemology – that is, their theory of knowledge. Epistemology shapes how researchers approach the researcher’s role, the participant-researcher relationship, forms of data, analytical procedures, measures of research quality, and representation of results in analysis and writing.( 3 )

In medical education, published qualitative work includes methodologies such as grounded theory, phenomenology, ethnography, case study, discourse analysis, participatory action research and narrative inquiry, although the list is growing as the field embraces researchers with diverse disciplinary backgrounds. This paper neither seeks to exhaustively catalogue all qualitative methodologies nor comprehensively describe any of them. Rather, we present a subset, with the aim of familiarising readers with its fundamental goals. In this article, we briefly introduce four common methodologies used in medical education research ( Box 2 ). Using one topic, professionalism, we illustrate how each methodology might be applied and how its particular features would yield different insights into that topic.

Common qualitative methodologies in medical education:

Grounded theory

Arguably the most frequently used methodology in medical education research today, grounded theory seeks to understand social processes. Core features of grounded theory include iteration, in which data collection and analysis take place concurrently with each informing the other, and a reliance on theoretical sampling to explore patterns as they emerge.( 4 ) While many different schools of grounded theory exist, they share the aim of generating theory that is grounded in empirical data.( 5 ) Theory, in this type of research, can be thought of as a conceptual understanding of the process under study, ideally affording a useful explanatory power. For example, if one were interested in the development of professionalism among senior medical students during clerkship, one might design a grounded theory study around the following question: “ What aspects of clerkship support or challenge professional behaviour among senior medical students? ” The resulting product would be a conceptual rendering of how senior medical students navigate thorny professionalism issues, which might in turn be useful to curriculum planners.

Phenomenology

This methodology begins with a phenomenon of interest and seeks to understand the subjective lived experience of that phenomenon.( 6 ) Core features of phenomenology include a focus on the individual experience (typically pursued through in-depth interviewing and/or examinations of personal narratives), inductive analysis and a particular attention to reflexivity.( 7 ) Phenomenological researchers typically enumerate their own ideas and preconceptions about the phenomenon under study and consider how these perceptions might influence their interpretation of data.( 8 ) A phenomenological study around professionalism in senior medical students, for example, might involve interviewing several students who have experienced a professionalism lapse about that experience. The resulting product might be an enhanced understanding of the emotional, social and professional implications of this phenomenon from the student’s perspective, which might in turn inform wellness or resilience strategies.

Ethnography

Ethnography aims to understand people in their contexts, exploring the influence of culture, social organisation and shared values on how people behave – their routines and rituals. Core features of ethnography include reliance on direct observation as a data source, and the use of sustained immersive engagement in the setting of interest in order to understand social dynamics from within.( 9 , 10 ) An ethnographic approach to studying how professional attitudes develop in senior medical students might gather data through observations of ward rounds, team meetings and clinical teaching sessions over a period of time. The resulting product – called an ethnography – would describe how professional values are socialised in junior learners in clinical settings, which could assist educators in understanding how the clinical experiences they programme for their learners are influencing their professional development.

Case study research seeks an in-depth understanding of an individual case (or series of cases) that is illustrative of a problem of interest. Like clinical case studies, the goal is not generalisation but a thorough exploration of one case, in hopes that the fruits of that exploration may prove useful to others facing similar problems. Core features of case study research include: thoughtful bounding or defining of the scope of the case at the outset; collection of data from multiple sources, ranging from interviews with key players to written material in policy documents and websites; and careful attention to both the phenomenon of interest and its particular context.( 11 ) A specific professionalism challenge involving medical students could provide fodder for a productive case study. For example, if a medical school had to discipline several students for inappropriately sharing personal patient information on social media, a case study might be undertaken. The ‘case’ would be the incident of social media misuse at a single medical school, and the data gathered might include interviews with students and school officials, examination of relevant policy documents, examination of news media coverage of the event, and so on. The product of this research might trigger similar institutions to carefully consider how they might approach – or prevent – a similar problem.

As these four examples illustrate, methodology is the backbone of qualitative research. Methodology shapes the way the research question is asked, defines the characteristics of an appropriate sample, and governs the way the data collection and analysis procedures are organised. The researcher’s role is also distinctive in each methodology; for instance, in constructivist grounded theory, the researcher actively constructs the theory,( 12 ) while in phenomenology, the researcher attempts to manage his or her ‘pre-understandings’ through either bracketing them off or being reflexive about them.( 13 ) Interested readers may wish to consult the reference list for recently published examples of research using grounded theory,( 14 ) phenomenology,( 15 ) ethnography( 16 ) and case study approaches( 17 ) in order to appreciate how researchers deploy these methodologies to tackle compelling questions in contemporary medical education.

WHAT ARE SOME COMMON METHODS OF QUALITATIVE DATA COLLECTION?

The most common methods of qualitative data collection are interview – talking to participants about their experiences relevant to the research question, and observation – watching participants while they are having those experiences. Depending on the research questions explored, a research design might combine interviews and observations.

Interview-based methods

Interviews are typically used for situations where a guided conversation with relevant participants can help provide insight into their lived experiences and how they view and interpret the world around them. Interviews are also particularly useful for exploring past events that cannot be replicated or phenomena where direct observation is impossible or unfeasible.

Participants may be interviewed individually or in groups. Focus group interviews are used when the researcher’s topic of interest is best explored through a guided, interactive discussion among the participants themselves. Therefore, when focus groups are used, the sample is conceptualised at the level of the group – three focus groups of five people constitutes a sample of three interactive discussions, not 15 individual participants. Because they centre on the group discussion and dynamic, focus groups are less well-suited for topics that are sensitive, highly personal or perceived to be culturally inappropriate to discuss publicly.( 18 )

Unlike quantitative interviews, where a set of structured, closed-ended (e.g. yes/no) questions are asked in the same order with the same wording every time, qualitative interviews typically involve a semi-structured design where a list of open-ended questions serves to guide, but not constrain, the interview. Therefore, at the interviewer’s discretion, the questions and their sequence may vary from interview to interview. This judgement is made based on both the interviewer’s understanding of the phenomenon under exploration and the emerging dynamic between the interviewer and participant.

The primary goal of a qualitative interview is to get the participants to think carefully about their experience and relate it to the interviewer with rich detail. Getting good data from interviewing relies on using creative strategies to avoid the common trap of getting politically correct answers – often called ‘cover stories’– or answers that are superficial rather than deep and reflective.( 19 ) A common design error occurs when researchers are overly explicit in their questioning, such as asking “ What are the top five criteria you use to assess student professionalism? ” A better approach involves questions that ask participants to describe what they do in practice, with follow-up probes that extend beyond the specific experience described. For example, starting with “ Tell me about a recent experience where you assessed a student’s professionalism ” allows the participant to relay an experience, to which the interviewer can respond with probes such as “ What was tricky about that? ” or “ How typical is that experience? ”

Another common strategy for prompting participants to engage in rich reflection on their experience and perceptions is to use vignettes as discussion prompts. Vignettes are often artificial scenarios presented to participants to read or watch on video, about which they are then asked probing questions.( 20 ) However, vignettes can also be used to recreate an authentic situation for the participant to engage with.( 21 ) For instance, in one interview study, we presented participants with a vignette in the form of the research assistant reading aloud a standard patient admission presentation that the interviewees would typically hear from their students on morning ward rounds. We then asked the participants to interact with the interviewer as though he or she was a student who had presented this case on morning rounds. Recreating this interaction in the context of the interview served as a stepping stone to questions such as “ Why did you ask the student ‘x’? ” and “ How would your approach have differed with a different student presenter, e.g. a stronger or weaker one? ”

Direct observation

Observation-based research can involve a wide spectrum of activities, ranging from brief observations of specific tasks (e.g. handover, preoperative team briefings) to prolonged field observations such as those seen in ethnography. When used effectively, direct observation can provide the researcher with powerful insight into the routines of a group.

Getting good data from observational research relies on several key components. First, it is essential to define the scope of the project upfront: limited budgets, the massive amount of detail to be attended to, and the ability of any individual or group of observers to attend to these make this essential. Good observational research therefore relies on collaboration between knowledgeable insiders and those with both methodological and theoretical expertise. Sampling demands particular attention; an initial purposive sampling approach is often followed by more targeted, theoretical sampling that is guided by the developing analysis. Observational research also typically involves a mix of data sources, including observational field notes, field interviews and document analysis. Audio and video may be helpful when the studied phenomena is particularly complex or nuances of interaction may be missed without the ability to review data, or when precision of verbal and nonverbal interactions is necessary to answer the research question.( 22 )

Field notes are often the dominant data source used for subsequent analysis in observational research. As such, they must be created with great diligence. Usually researchers will jot down brief notes during an observation and afterwards elaborate in as much detail as they can recall. Field notes have an important reflective component. In addition to the factual descriptions, researchers include comments about their feelings, reactions, hunches, speculations and working theories or interpretations. The content of field notes, therefore, usually includes: descriptions of the setting, people and activities; direct quotations or paraphrasing of what people said; and the observer’s reflections.( 23 ) Field notes are time-consuming when done well – even a single hour of observation can lead to several hours of reflective documentation.

An important aspect to consider when designing observation-based research is the ‘observer effect’, also known as the Hawthorne effect, more recently reframed as ‘participant reactivity’ by health professions education researchers Paradis and Sutkin.( 24 ) The Hawthorne effect is conventionally defined as “ when observed participants act differently from how they would act if the observer were not present ”.( 25 ) Researchers have implemented a number of strategies to mitigate this effect, including prolonged embedding of the observer, efforts to ‘fit in’ through dress or comportment, and careful recording of explicit instances of the effect.( 24 ) However, Paradis and Sutkin found that instances of the Hawthorne effect, as conventionally defined, have never been described in qualitative research manuscripts in the health professions education field, perhaps because, as they speculate, healthcare workers and trainees are accustomed to being observed. Based on this, they argued that researchers should worry less about mitigating the Hawthorne effect and instead invest in interpersonal relationships at their study site to mitigate the effects of altered behaviour and draw on theory to make sense of participants’ altered behaviour.( 23 ) Combining interviewing and observation is also common in qualitative research ( Box 3 ).

Combining interviews and observations:

WHAT ARE THE COMMON METHODS OF QUALITATIVE DATA ANALYSIS?

Qualitative data almost invariably takes the form of text; an interview is turned into a transcript and an observation is rendered into a field note. Analysing these qualitative texts is about uncovering meaning, developing understanding and discovering insights relevant to the research question. Analysis is not separated from data collection in qualitative research, and begins with the first interview, the first observation or the first reading of a document. In fact, the iterative nature of data collection and analysis is a hallmark of qualitative research, because it allows the researcher’s emerging insights about the study phenomena to inform subsequent rounds of data collection ( Box 4 ).

The iterative process of analysis:

Data that has been analysed while being collected is both parsimonious and illuminating. However, this process can extend indefinitely. There will always be another person to interview or another observation to record. Deciding when to stop depends on both practical and theoretical concerns. Practical concerns include deadlines and funding. More importantly, the decision should be guided by the theoretical concern of sufficiency.( 26 ) Sufficiency occurs when new data does not produce new insights into the phenomenon, in other words, when you keep hearing and seeing the same things you have heard and seen before.

Qualitative data analysis is primarily inductive and comparative. The overall process of data analysis begins by identifying segments in the data that are responsive to the research question. The next step is to compare one segment with the next, looking for recurring patterns in the data set. During this step, the focus is on sorting the raw data into categories that progressively build a coherent description or explanation of the phenomenon under study. This process of identifying pieces of data and grouping them into categories is called coding.( 14 ) Once a tentative scheme of categories is derived, it is applied to new data to see whether those categories continue to exist or not, or whether new categories arise – this step determines whether sufficiency has been reached. The final step in the analysis is to think about how categories interrelate. At this point, the analysis moves to interpreting the meaning of these categories and their interrelations.( 12 )

The process for data analysis laid out in this section is a basic inductive and comparative analysis strategy that is suitable for analysing data for most interpretive qualitative research methodologies, including the four featured in this paper – phenomenology, grounded theory, ethnography and case study – as well as others such as narrative analysis and action research. While each methodology attends to specific procedures, they all share the use of this basic inductive/comparative strategy. Overall, analysis should be guided by methodology, but different analytical procedures can be creatively combined across methodologies, as long as this combining is explicit and intentional.( 27 )

WHAT ARE SOME CURRENT INNOVATIONS IN QUALITATIVE RESEARCH?

Understanding the complex factors that influence clinical practice and medical education is not an easy research task. Many important issues may be difficult for the insider to articulate during interviews and impossible for the outsider to ‘see’ during observation. Innovations to address these challenges include guided walks,( 28 ) photovoice( 29 ) and point-of-view filming.( 30 ) Our own research has drawn intensively on the innovation termed ‘rich pictures’ to explore the features and implications of complexity in medical education.( 31 ) In one study, we asked medical students to draw pictures of clinical cases that they found complex: an exciting case and a frustrating one.( 32 ) Participants were given 30–60 minutes on their own to reflect on the situation and draw their pictures. This was followed by an in-depth interview using the pictures as triggers to explore the phenomenon under study – in this case, students’ experiences of and responses to complexity during their training.

Such innovations hold great promise for qualitative research in medical education. For instance, rich pictures can reveal emotional and organisational dimensions of complex clinical experiences, which are less likely to be emphasised in participants’ traditional interview responses.( 33 ) Methodological innovations, however, bring new challenges: they can be time-intensive for participants and researchers; they require new analytical procedures to be developed; and they necessitate efforts to educate audiences about the rigour and credibility of unfamiliar approaches.

WHAT ARE THE PRINCIPLES OF RIGOUR IN QUALITATIVE RESEARCH?

Like quantitative research, qualitative research has principles of rigour that are used to judge the quality of the work.( 34 ) Here, we discuss principles that appear in most criteria for rigour in the field: reflexivity, adequacy, authenticity, trustworthiness and resonance ( Box 5 ).

Principles of rigour in qualitative research:

The main data collection tool in qualitative research is the researcher. We talk to participants, observe their practices and interpret their documents. Consequently, a critical feature of rigour in qualitative data collection is researcher reflexivity: the ability to consider our own orientations towards the studied phenomenon, acknowledge our assumptions and articulate regularly our impressions of the data.( 35 ) Only this way can we assure others that our subjectivity has been thoughtfully considered and afford them the ability to judge its influence on the work for themselves. Qualitative research does not seek to remove this subjectivity; it treats research perspective as unavoidable and enriching, not as a form of bias to purge.

Every qualitative dataset is an approximation of a complex phenomenon – no study can capture all dimensions and nuances of situated social experiences, such as medical students’ negotiations of professional dilemmas in the clinical workplace. Therefore, two other important criteria of rigour relate to the adequacy and authenticity of the sampled experiences. Did the research focus on the appropriate participants and/or situations? Was the size and scope of the sample adequate to represent the scope of the phenomenon?( 36 ) Was the data collected an authentic reflection of the phenomenon in question? Qualitative researchers should thoughtfully combine different perspectives, methods and data sources (a process called ‘triangulation’) to intensify the richness of their representation.( 37 ) We should endeavour to draw on data in our written reports such that we provide what sociologist Geertz has termed a sufficiently ‘thick’ description( 38 ) for readers to judge the authenticity of our portrayal of the studied phenomenon.

Qualitative analysis embraces subjectivity: what the researcher ‘sees’ in the data is a product both of what participants told or showed us and of what we were oriented to make of those stories and situations. To some degree, a rhetorician will always see rhetoric and a systems engineer will always see systems. To fulfil the rigour criteria of trustworthiness, qualitative analysis should also be systematic and held to a principle of trustworthiness, which dictates that we should clearly describe: (a) what was done by whom during the inductive, comparative analytical process; (b) how the perspectives of multiple coders were negotiated; (c) how and when theoretical lenses were brought to bear in the iterative process of data collection and analysis; and (d) how discrepant instances in the data – those that fell outside the dominant thematic patterns – were handled.

Finally, the ultimate measure of quality in qualitative research is the resonance of the final product to those who live the social experience under study.( 4 ) As qualitative researchers presenting our work at conferences, we know we have met this bar if our audiences laugh, nod or scowl at the right moments, and if their response at the end is “ You nailed it. That’s my world. But you’ve given me a new way to look at it ”. The situatedness of qualitative research means that its transferability to other contexts is always a matter of the listener/reader’s judgement, based on their consideration of the similarities and differences between the research context and their own. Thus, there is a necessity for qualitative research to sufficiently describe its context, so that consumers of the work have the necessary information to gauge transferability. Ultimately, though, transferability remains an open question, requiring further inquiry to explore the explanatory power of one study’s insights in a new setting.

WHAT ELSE IS THERE TO KNOW?

This overview of qualitative research in medical education is not exhaustive. We have been purposefully selective, discussing in depth some common methodologies and methods, and leaving aside others. We have also passed over important issues such as qualitative research ethics, sampling and writing. There is much, much more for readers to know! Our selectivity notwithstanding, we hope that this paper will provide an accessible introduction to some qualitative essentials for readers who are new to this research domain, and that it may serve as a useful resource for more experienced readers, particularly those who are doing a qualitative research project and would like a better sense of where their work sits within the broader field of qualitative approaches.

Advertisement

• Previous Issue
• Previous Article
• Next Article
• Box 1. What to Look for in Research Using This Method

What Is Qualitative Research?

Qualitative versus quantitative research, conducting and appraising qualitative research, conclusions, research support, competing interests, qualitative research methods in medical education.

Submitted for publication January 5, 2018. Accepted for publication November 29, 2018.

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Open the PDF for in another window
• Cite Icon Cite
• Get Permissions
• Search Site

Adam P. Sawatsky , John T. Ratelle , Thomas J. Beckman; Qualitative Research Methods in Medical Education. Anesthesiology 2019; 131:14–22 doi: https://doi.org/10.1097/ALN.0000000000002728

Download citation file:

• Ris (Zotero)
• Reference Manager

Qualitative research was originally developed within the social sciences. Medical education is a field that comprises multiple disciplines, including the social sciences, and utilizes qualitative research to gain a broader understanding of key phenomena within the field. Many clinician educators are unfamiliar with qualitative research. This article provides a primer for clinician educators who want to appraise or conduct qualitative research in medical education. This article discusses a definition and the philosophical underpinnings for qualitative research. Using the Standards for Reporting Qualitative Research as a guide, this article provides a step-wise approach for conducting and evaluating qualitative research in medical education. This review will enable the reader to understand when to utilize qualitative research in medical education and how to interpret reports using qualitative approaches.

Image: J. P. Rathmell and Terri Navarette.

Qualitative research provides approaches to explore and characterize the education of future anesthesiologists. For example, the practice of anesthesiology is increasingly team-based; core members of the anesthesia care team include physicians, trainees, nurse anesthetists, anesthesiologist assistants, and other healthcare team members. 1   Understanding how to work within and how to teach learners about anesthesia care teams requires the ability to conceptualize the complexity of individual psychology and social interactions that occur within teams. Qualitative research is well suited to investigate complex issues like team-based care. For example, one qualitative study observed the interactions between members of the anesthesia care team during simulated stressful situations and conducted interviews of team members; they described limited understanding of each team member’s role and perceptions about appropriate roles and responsibilities, which provided insight for interprofessional team training. 2   Another qualitative study explored the hierarchy within the anesthesia care team, highlighting residents’ reluctance to challenge the established hierarchy and outlining the strategies they use to cope with fear and intimidation. 3   Key issues in medical education and anesthesiology, particularly when exploring human experience and social interactions, may be best studied using qualitative research methodologies and methods.

Medical education is a complex field, and medical education research and practice fittingly draws from many disciplines ( e.g. , medicine, psychology, sociology, education) and synthesizes multiple perspectives to explain how people learn and how medicine should be taught. 4 , 5   The concept of a field was well described by Cristancho and Varpio 5   in their tips for early career medical educators: “A discipline is usually guided by shared paradigms, assumptions, rules and methods to present their knowledge claims— i.e. , people from the same discipline speak the same language. A field brings people from multiple disciplines together.” Qualitative research draws from the perspectives of multiple disciplines and has provided methodologies to explore the complex research questions inherent to medical education.

When appraising qualitative research in medical education, do the authors:

Clearly state the study purpose and research question?

Describe the conceptual framework that inform the study and guide analysis?

Identify their qualitative methodology and research paradigm?

Demonstrate adequate reflexivity, conveying to the reader their values, assumptions and way of thinking, being explicit about the effects these ways of thinking have on the research process?

Choose data collection methods that are congruent with the research purpose and qualitative methodology?

Select an appropriate sampling strategy, choosing participants whose perspectives or experiences are relevant to the study question?

Define their method for determining saturation, how they decided to stop data collection?

Outline their process for data processing, including the management and coding of study data?

Conduct data analysis consistent with their chosen methodology?

Consider techniques to enhance trustworthiness of their study findings?

Synthesize and interpret their data with sufficient detail and supporting quotations to explain the phenomenon of study?

Current medical training is heavily influenced by the practice of evidence-based medicine. 6   Trainees are taught the “hierarchy of evidence” for evaluating studies of clinical interventions. 7   This hierarchy prioritizes knowledge gained through systematic reviews and meta-analyses, randomized controlled trials, and observational studies, but it does not include qualitative research methodologies. This means that because of their medical training and exposure to quantitative medical literature, clinician educators may be more familiar with quantitative research and feel more comfortable engaging in studies utilizing quantitative methodologies. However, many clinician educators are not familiar with the language and application of qualitative research and feel less comfortable engaging in studies using qualitative methodologies.

Because medical education is a diverse and complex field, qualitative research is a common approach in medical education research. Clinician educators who wish to understand the medical education literature need to be familiar with qualitative research. Clinician educators involved in research may also find themselves asking questions best answered by qualitative methodologies. Our goal is to provide a broad, practical overview of qualitative research in medical education. Our objectives are to:

1) Define qualitative research.

2) Compare and contrast qualitative and quantitative research.

3) Provide a framework for conducting and appraising qualitative research in medical education.

Qualitative research in medical education has a distinct vocabulary with terminology not commonly used in other biomedical research fields. Therefore, we have provided a glossary and definitions of the common terms that are used throughout this article ( table 1 ).

Glossary of Common Terms Used in Qualitative Research

Of the many attempts to provide a comprehensive definition of qualitative research, our favorite definition comes from Denzin and Lincoln:

“Qualitative research is a situated activity that locates the observer in the world. Qualitative research consists of a set of interpretive, material practices that make the world visible. These practices…turn the world into a series of representations, including field notes, interviews, conversations, photographs, recordings, and memos to the self. At this level, qualitative research involves an interpretive, naturalistic approach to the world. This means that qualitative researchers study things in their natural settings, attempting to make sense of or interpret phenomena in terms of the meanings people bring to them.” 12  

This definition reveals the following points: first, qualitative research is a “situated activity,” meaning that the research and observations are made in the real world, in this case a real life clinical or educational situation. Second, qualitative research “turns the world into a series of representations” by representing the observations, in this case of a clinical or educational situation, with qualitative data, usually taking the form of words, pictures, documents, and other symbols. Last, qualitative researchers seek to “make sense” of the meanings that research participants bring to different phenomena to allow for a greater understanding of those phenomena. Through qualitative research, observers comprehend participants’ beliefs and values and the way these beliefs and values are shaped by the context in which they are studied.

Because most clinician educators are familiar with quantitative methods, we will start by comparing qualitative and quantitative methods to gain a better understanding of qualitative research ( table 2 ). To illustrate the difference between qualitative and quantitative research in medical education, we pose the question: “What makes noon conference lectures effective for resident learning?” A qualitative approach might explore the learner perspective on learning in noon conference lectures during residency and conduct an exploratory thematic analysis to better understand what the learner thinks is effective. 13   A qualitative approach is useful to answer this question, especially if the phenomenon of interest is incompletely understood. If we wanted to compare types or attributes of conferences to assess the most effective methods of teaching in a noon conference setting, then a quantitative approach might be more appropriate, though a qualitative approach could be helpful as well. We could use qualitative data to inform the design of a survey 14   or even inform the design of a randomized control trial to compare two types of learning during noon conference. 15   Therefore, when discussing qualitative and quantitative research, the issue is not which research approach is stronger, because it is understood that each approach yields different types of knowledge when answering the research question.

Comparisons of Quantitative and Qualitative Research in Medical Education

Similarities

The first step of any research project, qualitative or quantitative, is to determine and refine the study question; this includes conducting a thorough literature review, crafting a problem statement, establishing a conceptual framework for the study, and declaring a statement of intent. 16   A common pitfall in medical education research is to start by identifying the desired methods ( e.g. , “I want to do a focus group study with medical students.”) without having a clearly refined research question, which is like putting the cart before the horse. In other words, the research question should guide the methodology and methods for both qualitative and quantitative research.

Acknowledging the conceptual framework for a study is equally important for both qualitative and quantitative research. In a systematic review of medical education research, only 55% of studies provided a conceptual framework, limiting the interpretation and meaning of the results. 17   Conceptual frameworks are often theories that represent a way of thinking about the phenomenon being studied. Conceptual frameworks guide the interpretation of data and situate the study within the larger body of literature on a specific topic. 9   Because qualitative research was developed within the social sciences, many qualitative research studies in medical education are framed by theories from social sciences. Theories from social science disciplines have the ability to “open up new ways of seeing the world and, in turn, new questions to ask, new assumptions to unearth, and new possibilities for change.” 18   Qualitative research in medical education has benefitted from these new perspectives to help understand fundamental and complex problems within medical education such as culture, power, identity, and meaning.

Differences

The fundamental difference between qualitative and quantitative methodologies centers on epistemology ( i.e. , differing views on truth and knowledge). Cleland 19   describes the differences between qualitative and quantitative philosophies of scientific inquiry: “quantitative and qualitative approaches make different assumptions about the world, about how science should be conducted and about what constitutes legitimate problems, solutions and criteria of ‘proof.’”

Quantitative research comes from objectivism , an epistemology asserting that there is an absolute truth that can be discovered; this way of thinking about knowledge leads researchers to conduct experimental study designs aimed to test hypotheses about cause and effect. 10   Qualitative research, on the other hand, comes from constructivism , an epistemology asserting that reality is constructed by our social, historical, and individual contexts, and leads researchers to utilize more naturalistic or exploratory study designs to provide explanations about phenomenon in the context that they are being studied. 10   This leads researchers to ask fundamentally different questions about a given phenomenon; quantitative research often asks questions of “What?” and “Why?” to understand causation, whereas qualitative research often asks the questions “Why?” and “How?” to understand explanations. Cook et al. 20   provide a framework for classifying the purpose of medical education research to reflect the steps in the scientific method—description (“What was done?”), justification (“Did it work?”), and clarification (“Why or how did it work?”). Qualitative research nicely fits into the categories of “description” and “clarification” by describing observations in natural settings and developing models or theories to help explain “how” and “why” educational methods work. 20  

Another difference between quantitative and qualitative research is the role of the researcher in the research process. Experimental studies have explicitly stated methods for creating an “unbiased” study in which the researcher is detached ( i.e. , “blinded”) from the analysis process so that their biases do not shape the outcome of the research. 21   The term “bias” comes from the positivist paradigm underpinning quantitative research. Assessing and addressing “bias” in qualitative research is incongruous. 22   Qualitative research, based largely on a constructivist paradigm, acknowledges the role of the researcher as a “coconstructer” of knowledge and utilizes the concept of “reflexivity.” Because researchers act as coconstructors of knowledge, they must be explicit about the perspectives they bring to the research process. A reflexive researcher is one who challenges their own values, assumptions, and way of thinking and who is explicit about the effects these ways of thinking have on the research process. 23   For example, when we conducted a study on self-directed learning in residency training, we were overt regarding our roles in the residency program as core faculty, our belief in the importance of self-directed learning, and our assumptions that residents actually engaged in self-directed learning. 24 , 25   We also needed to challenge these assumptions and open ourselves to alternative questions, methods of data collection, and interpretations of the data, to ultimately ensure that we created a research team with varied perspectives. Therefore, qualitative researchers do not strive for “unbiased” research but to understand their own roles in the coconstruction of knowledge. When assessing reflexivity, it is important for the authors to define their roles, explain how those roles may affect the collection and analysis of data, and how the researchers accounted for that effect and, if needed, challenged any assumptions during the research process. Because of the role of the researcher in qualitative research, it is vital to have a member of the research team with qualitative research experience.

A Word on Mixed Methods

In mixed methods research, the researcher collects and analyzes both qualitative and quantitative data rigorously and integrates both forms of data in the results of the study. 26   Medical education research often involves complex questions that may be best addressed through both quantitative and qualitative approaches. Combining methods can complement the strengths and limitations of each method and provide data from multiple sources to create a more detailed understanding of the phenomenon of interest. Examples of uses of mixed methods that would be applicable to medical education research include: collecting qualitative and quantitative data for more complete program evaluation, collecting qualitative data to inform the research design or instrument development of a quantitative study, or collecting qualitative data to explain the meaning behind the results of a quantitative study. 26   The keys to conducting mixed methods studies are to clearly articulate your research questions, explain your rationale for use of each approach, build an appropriate research team, and carefully follow guidelines for methodologic rigor for each approach. 27  

Toward Asking More “Why” Questions

We presented similarities and differences between qualitative and quantitative research to introduce the clinician educator to qualitative research but not to suggest the relative value of one these research methods over the other. Whether conducting qualitative or quantitative research in medical education, researchers should move toward asking more “why” questions to gain deeper understanding of the key phenomena and theories in medical education to move the field of medical education forward. 28   By understanding the theories and assumptions behind qualitative and quantitative research, clinicians can decide how to use these approaches to answer important questions in medical education.

There are substantial differences between qualitative and quantitative research with respect to the assessment of rigor; here we provide a framework for reading, understanding, and assessing the quality of qualitative research. O’Brien et al. 29   created a useful 21-item guide for reporting qualitative research in medical education, based upon a systematic review of reporting standards for qualitative research—the Standards for Reporting Qualitative Research. It should be noted, however, that just performing and reporting each step in these standards do not ensure research quality.

Using the Standards for Reporting Qualitative Research as a backdrop, we will highlight basic steps for clinician educators wanting to engage with qualitative research. If you use this framework to conduct qualitative research in medical education, then you should address these steps; if you are evaluating qualitative research in medical education, then you can assess whether the study investigators addressed these steps. Table 3 underscores each step and provides examples from our research in resident self-directed learning. 25  

Components of Qualitative Research: Examples from a Single Research Study

Refine the study question. As with any research project, investigators should clearly define the topic of research, describe what is already known about the phenomenon that is being studied, identify gaps in the literature, and clearly state how the study will fill that gap. Considering theoretical underpinnings of qualitative research in medical education often means searching for sources outside of the biomedical literature and utilizing theories from education, sociology, psychology, or other disciplines. This is also a critical time to engage people from other disciplines to identify theories or sources of information that can help define the problem and theoretical frameworks for data collection and analysis. When evaluating the introduction of a qualitative study, the researchers should demonstrate a clear understanding of the phenomenon being studied, the previous research on the phenomenon, and conceptual frameworks that contextualize the study. Last, the problem statement and purpose of the study should be clearly stated.

Identify the qualitative methodology and research paradigm. The qualitative methodology should be chosen based on the stated purpose of the research. The qualitative methodology represents the overarching philosophy guiding the collection and analysis of data and is distinct from the research methods ( i.e. , how the data will be collected). There are a number of qualitative methodologies; we have included a list of some of the most common methodologies in table 4 . Choosing a qualitative methodology involves examining the existing literature, involving colleagues with qualitative research expertise, and considering the goals of each approach. 32   For example, explaining the processes, relationships, and theoretical understanding of a phenomenon would point the researcher to grounded theory as an appropriate approach to conducting research. Alternatively, describing the lived experiences of participants may point the researcher to a phenomenological approach. Ultimately, qualitative research should explicitly state the qualitative methodology along with the supporting rationale. Qualitative research is challenging, and you should consult or collaborate with a qualitative research expert as you shape your research question and choose an appropriate methodology. 32  

Choose data collection methods. The choice of data collection methods is driven by the research question, methodology, and practical considerations. Sources of data for qualitative studies would include open-ended survey questions, interviews, focus groups, observations, and documents. Among the most important aspects of choosing the data collection method is alignment with the chosen methodology and study purpose. 33   For interviews and focus groups, there are specific methods for designing the instruments. 34 , 35   Remarkably, these instruments can change throughout the course of the study, because data analysis often informs future data collection in an iterative fashion.

Select a sampling strategy. After identifying the types of data to be collected, the next step is deciding how to sample the data sources to obtain a representative sample. Most qualitative methodologies utilize purposive sampling, which is choosing participants whose perspectives or experiences are relevant to the study question. 11   Although random sampling and convenience sampling may be simpler and less costly for the researcher than purposeful sampling, these approaches often do not provide sufficient information to answer the study question. 36   For example, in grounded theory, theoretical sampling means that the choice of subsequent participants is purposeful to aid in the building and refinement of developing theory. The criteria for selecting participants should be stated clearly. One key difference between qualitative and quantitative research is sample size: in qualitative research, sample size is usually determined during the data collection process, whereas in quantitative research, the sample size is determined a priori . Saturation is verified when the analysis of newly collected data no longer provides additional insights into the data analysis process. 10  

Plan and outline a strategy for data processing. Data processing refers to how the researcher organizes, manages, and dissects the study data. Although data processing serves data analysis, it is not the analysis itself. Data processing includes practical aspects of data management, like transcribing interviews, collecting field notes, and organizing data for analysis. The next step is coding the data, which begins with organizing the raw data into chunks to allow for the identification of themes and patterns. A code is a “word or short phrase that symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a portion of language-based or visual data.” 8   There is an artificial breakdown between data processing and analysis, because these steps may be conducted simultaneously; many consider coding as different from—yet a necessary step to facilitating—the analysis of data. 8   Qualitative software can support this process, by making it easier to organize, access, search, and code your data. However, it is noteworthy that these programs do not do the work for you, they are merely tools for supporting data processing and analysis.

Conduct the data analysis. When analyzing the data, there are several factors to consider. First, the process of data analysis begins with the initial data collection, which often informs future data collection. Researchers should be intentional when reading, reviewing, and analyzing data as it is collected, so that they can shape and enrich subsequent data collection ( e.g. , modify the interview questions). Second, data analysis is often conducted by a research team that should have the appropriate expertise and perspectives to bring to the analysis process. Therefore, when evaluating a qualitative study, you should consider the team’s composition and their reflexivity with respect to their potential biases and influences on their study subjects. Third, the overall goal is to move from the raw data to abstractions of the data that answer the research question. For example, in grounded theory, the research moves from the raw data, to the identification of themes, to categorization of themes, to identifying relationships between themes, and ultimately to the development of theoretical explanations of the phenomenon. 30   Consequently, the primary researcher or research team should be intimately involved with the data analysis, interrogating the data, writing analytic memos, and ultimately make meaning out of the data. There are differing opinions about the use of “counting” of codes or themes in qualitative research. In general, counting of themes is used during the analysis process to recognize patterns and themes; often these are not reported as numbers and percentages as in quantitative research, but may be represented by words like few , some , or many . 37  

Recognize techniques to enhance trustworthiness of your study findings. Ensuring consistency between the data and the results of data analysis, along with ensuring that the data and results accurately represent the perspectives and contexts related to the data source, are crucial to ensuring trustworthiness of study findings. Methods for enhancing trustworthiness include triangulation , which is comparing findings from different methods or perspectives, and member-checking , which is presenting research findings to study participants to provide opportunities to ensure that the analysis is representative. 10  

Synthesize and interpret your data. Synthesis of qualitative research is determined by the depth of the analysis and involves moving beyond description of the data to explaining the findings and situating the results within the larger body of literature on the phenomenon of interest. The reporting of data synthesis should match the research methodology. For instance, if the study is using grounded theory, does the study advance the theoretical understanding of the phenomenon being studied? It is also important to acknowledge that clarity and organization are paramount. 10   Qualitative data are rich and extensive; therefore, researchers must organize and tell a compelling story from the data. 38   This process includes the selection of representative data ( e.g. , quotations from interviews) to substantiate claims made by the research team.

Common Methodologies Used in Qualitative Research

For more information on qualitative research in medical education:

Qualitative Research and Evaluation Methods: Integrating Theory and Practice, by Michael Q. Patton (SAGE Publications, Inc., 2014)

Qualitative Inquiry and Research Design: Choosing Among Five Approaches, by John W. Cresswell (SAGE Publications, Inc. 2017)

Researching Medical Education, by Jennifer Cleland and Steven J. Durning (Wiley-Blackwell, 2015)

Qualitative Research in Medical Education, by Patricia McNally, in Oxford Textbook of Medical Education, edited by Kieren Walsh (Oxford University Press, 2013)

The Journal of Graduate Medical Education “Qualitative Rip Out Series” (Available at: http://www.jgme.org/page/ripouts )

The Standards for Reporting Qualitative Research (O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-51.)

The Wilson Centre Qualitative Atelier (For more information: http://thewilsoncentre.ca/atelier/ )

Qualitative research is commonly used in medical education but may be unfamiliar to many clinician educators. In this article, we provided a definition of qualitative research, explored the similarities and differences between qualitative and quantitative research, and outlined a framework for conducting or appraising qualitative research in medical education. Even with advanced training, it can be difficult for clinician educators to understand and conduct qualitative research. Leaders in medical education research have proposed the following advice to clinician educators wanting to engage in qualitative medical education research: (1) clinician educators should find collaborators with knowledge of theories from other disciplines ( e.g. , sociology, cognitive psychology) and experience in qualitative research to utilize their complementary knowledge and experience to conduct research—in this way, clinician educators can identify important research questions; collaborators can inform research methodology and theoretical perspectives; and (2) clinician educators should engage with a diverse range disciplines to generate new questions and perspectives on research. 4  

Support was provided solely from institutional and/or departmental sources.

The authors declare no competing interests.

Citing articles via

Most viewed, email alerts, related articles, social media, affiliations.

• ASA Practice Parameters
• Online First
• Author Resource Center
• About the Journal
• Editorial Board
• Rights & Permissions
• Online ISSN 1528-1175
• Print ISSN 0003-3022
• Anesthesiology
• ASA Monitor

• Terms & Conditions Privacy Policy
• Manage Cookie Preferences
• © Copyright 2024 American Society of Anesthesiologists

This Feature Is Available To Subscribers Only

Sign In or Create an Account

• Open access
• Published: 05 December 2023

A scoping review to identify and organize literature trends of bias research within medical student and resident education

• Brianne E. Lewis 1 &
• Akshata R. Naik 2  

BMC Medical Education volume  23 , Article number:  919 ( 2023 ) Cite this article

816 Accesses

1 Citations

2 Altmetric

Metrics details

Physician bias refers to the unconscious negative perceptions that physicians have of patients or their conditions. Medical schools and residency programs often incorporate training to reduce biases among their trainees. In order to assess trends and organize available literature, we conducted a scoping review with a goal to categorize different biases that are studied within medical student (MS), resident (Res) and mixed populations (MS and Res). We also characterized these studies based on their research goal as either documenting evidence of bias (EOB), bias intervention (BI) or both. These findings will provide data which can be used to identify gaps and inform future work across these criteria.

Online databases (PubMed, PsycINFO, WebofScience) were searched for articles published between 1980 and 2021. All references were imported into Covidence for independent screening against inclusion criteria. Conflicts were resolved by deliberation. Studies were sorted by goal: ‘evidence of bias’ and/or ‘bias intervention’, and by population (MS or Res or mixed) andinto descriptive categories of bias.

Of the initial 806 unique papers identified, a total of 139 articles fit the inclusion criteria for data extraction. The included studies were sorted into 11 categories of bias and showed that bias against race/ethnicity, specific diseases/conditions, and weight were the most researched topics. Of the studies included, there was a higher ratio of EOB:BI studies at the MS level. While at the Res level, a lower ratio of EOB:BI was found.

Conclusions

This study will be of interest to institutions, program directors and medical educators who wish to specifically address a category of bias and identify where there is a dearth of research. This study also underscores the need to introduce bias interventions at the MS level.

Peer Review reports

Physician bias ultimately impacts patient care by eroding the physician–patient relationship [ 1 , 2 , 3 , 4 ]. To overcome this issue, certain states require physicians to report a varying number of hours of implicit bias training as part of their recurring licensing requirement [ 5 , 6 ]. Research efforts on the influence of implicit bias on clinical decision-making gained traction after the “Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care” report published in 2003 [ 7 ]. This report sparked a conversation about the impact of bias against women, people of color, and other marginalized groups within healthcare. Bias from a healthcare provider has been shown to affect provider-patient communication and may also influence treatment decisions [ 8 , 9 ]. Nevertheless, opportunities within medical education curriculum are created to evaluate biases at an earlier stage of physician-training and provide instruction to intervene them [ 10 , 11 , 12 ]. We aimed to identify trends and organize literature on bias training provided during medical school and residency programs since the meaning of ‘bias’ is broad and encompasses several types of attitudes and predispositions [ 13 ].

Several reviews, narrative or systematic in nature, have been published in the field of bias research in medicine and healthcare [ 14 , 15 , 16 ]. Many of these reviews have a broad focus on implicit bias and they often fail to define the patient’s specific attributes- such as age, weight, disease, or condition against which physicians hold their biases. However, two recently published reviews categorized implicit biases into various descriptive characteristics albeit with research goals different than this study [ 17 , 18 ]. The study by Fitzgerald et al. reviewed literature focused on bias among physicians and nurses to highlight its role in healthcare disparities [ 17 ]. While the study by Gonzalez et al. focused on bias curricular interventions across professions related to social determinants of health such as education, law, medicine and social work [ 18 ]. Our research goal was to identify the various bias characteristics that are studied within medical student and/or resident populations and categorize them. Further, we were interested in whether biases were merely identified or if they were intervened. To address these deficits in the field and provide clarity, we utilized a scoping review approach to categorize the literature based on a) the bias addressed and b) the study goal within medical students (MS), residents (Res) and a mixed population (MS and Res).

To date no literature review has organized bias research by specific categories held solely by medical trainees (medical students and/or residents) and quantified intervention studies. We did not perform a quality assessment or outcome evaluation of the bias intervention strategies, as it was not the goal of this work and is standard with a scoping review methodology [ 19 , 20 ]. By generating a comprehensive list of bias categories researched among medical trainee population, we highlight areas of opportunity for future implicit bias research specifically within the undergraduate and graduate medical education curriculum. We anticipate that the results from this scoping review will be useful for educators, administrators, and stakeholders seeking to implement active programs or workshops that intervene specific biases in pre-clinical medical education and prepare physicians-in-training for patient encounters. Additionally, behavioral scientists who seek to support clinicians, and develop debiasing theories [ 21 ] and models may also find our results informative.

We conducted an exhaustive and focused scoping review and followed the methodological framework for scoping reviews as previously described in the literature [ 20 , 22 ]. This study aligned with the four goals of a scoping review [ 20 ]. We followed the first five out of the six steps outlined by Arksey and O’Malley’s to ensure our review’s validity 1) identifying the research question 2) identifying relevant studies 3) selecting the studies 4) charting the data and 5) collating, summarizing and reporting the results [ 22 ]. We did not follow the optional sixth step of undertaking consultation with key stakeholders as it was not needed to address our research question it [ 23 ]. Furthermore, we used Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia) that aided in managing steps 2–5 presented above.

Research question, search strategy and inclusion criteria

The purpose of this study was to identify trends in bias research at the medical school and residency level. Prior to conducting our literature search we developed our research question and detailed the inclusion criteria, and generated the search syntax with the assistance from a medical librarian. Search syntax was adjusted to the requirements of the database. We searched PubMed, Web of Science, and PsycINFO using MeSH terms shown below.

Bias* [ti] OR prejudice*[ti] OR racism[ti] OR homophobia[ti] OR mistreatment[ti] OR sexism[ti] OR ageism[ti]) AND (prejudice [mh] OR "Bias"[Mesh:NoExp]) AND (Education, Medical [mh] OR Schools, Medical [mh] OR students, medical [mh] OR Internship and Residency [mh] OR “undergraduate medical education” OR “graduate medical education” OR “medical resident” OR “medical residents” OR “medical residency” OR “medical residencies” OR “medical schools” OR “medical school” OR “medical students” OR “medical student”) AND (curriculum [mh] OR program evaluation [mh] OR program development [mh] OR language* OR teaching OR material* OR instruction* OR train* OR program* OR curricul* OR workshop*

Our inclusion criteria incorporated studies which were either original research articles, or review articles that synthesized new data. We excluded publications that were not peer-reviewed or supported with data such as narrative reviews, opinion pieces, editorials, perspectives and commentaries. We included studies outside of the U.S. since the purpose of this work was to generate a comprehensive list of biases. Physicians, regardless of their country of origin, can hold biases against specific patient attributes [ 17 ]. Furthermore, physicians may practice in a different country than where they trained [ 24 ]. Manuscripts were included if they were published in the English language for which full-texts were available. Since the goal of this scoping review was to assess trends, we accepted studies published from 1980–2021.

Our inclusion criteria also considered the goal and the population of the study. We defined the study goal as either that documented evidence of bias or a program directed bias intervention. Evidence of bias (EOB) had to originate from the medical trainee regarding a patient attribute. Bias intervention (BI) studies involved strategies to counter biases such as activities, workshops, seminars or curricular innovations. The population studied had to include medical students (MS) or residents (Res) or mixed. We defined the study population as ‘mixed’ when it consisted of both MS and Res. Studies conducted on other healthcare professionals were included if MS or Res were also studied. Our search criteria excluded studies that documented bias against medical professionals (students, residents and clinicians) either by patients, medical schools, healthcare administrators or others, and was focused on studies where the biases were solely held by medical trainees (MS and Res).

Data extraction and analysis

Following the initial database search, references were downloaded and bulk uploaded into Covidence and duplicates were removed. After the initial screening of title and abstracts, full-texts were reviewed. Authors independently completed title and abstract screening, and full text reviews. Any conflicts at the stage of abstract screening were moved to full-text screening. Conflicts during full-text screening were resolved by deliberation and referring to the inclusion and exclusion criteria detailed in the research protocol. The level of agreement between the two authors for full text reviews as measured by inter-rater reliability was 0.72 (Cohen’s Kappa).

A data extraction template was created in Covidence to extract data from included full texts. Data extraction template included the following variables; country in which the study was conducted, year of publication, goal of the study (EOB, BI or both), population of the study (MS, Res or mixed) and the type of bias studied. Final data was exported to Microsoft Excel for quantification. For charting our data and categorizing the included studies, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews(PRISMA-ScR) guidelines [ 25 ]. Results from this scoping review study are meant to provide a visual synthesis of existing bias research and identify gaps in knowledge.

Study selection

Our search strategy yielded a total of 892 unique abstracts which were imported into ‘Covidence’ for screening. A total of 86 duplicate references were removed. Then, 806 titles and abstracts were screened for relevance independently by the authors and 519 studies were excluded at this stage. Any conflicts among the reviewers at this stage were resolved by discussion and referring to the inclusion and exclusion criteria. Then a full text review of the remaining 287 papers was completed by the authors against the inclusion criteria for eligibility. Full text review was also conducted independently by the authors and any conflicts were resolved upon discussion. Finally, we included 139 studies which were used for data extraction (Fig.  1 ).

PRISMA diagram of the study selection process used in our scoping review to identify the bias categories that have been reported within medical education literature. Study took place from 2021–2022. Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Publication trends in bias research

First, we charted the studies to demonstrate the timeline of research focused on bias within the study population of our interest (MS or Res or mixed). Our analysis revealed an increase in publications with respect to time (Fig.  2 ). Of the 139 included studies, fewer studies were published prior to 2001, with a total of only eight papers being published from the years 1985–2000. A substantial increase in publications occurred after 2004, with 2019 being the peak year where most of the studies pertaining to bias were published (Fig.  2 ).

Studies matching inclusion criteria mapped by year of publication. Search criteria included studies addressing bias from 1980–2021 within medical students (MS) or residents (Res) or mixed (MS + Res) populations. * Publication in 2022 was published online ahead of print

Overview of included studies

We present a descriptive analysis of the 139 included studies in Table 1 based on the following parameters: study location, goal of the study, population of the study and the category of bias studied. All of the above parameters except the category of bias included a denominator of 139 studies. Several studies addressed more than one bias characteristic; therefore, we documented 163 biases sorted in 11 categories over the 139 papers. The bias categories that we generated and their respective occurrences are listed in Table 1 . Of the 139 studies that were included, most studies originated in the United States ( n  = 89/139, 64%) and Europe ( n  = 20/139, 20%).

Sorting of included research by bias category

We grouped the 139 included studies depending on the patient attribute or the descriptive characteristic against which the bias was studied (Table 1 ). By sorting the studies into different bias categories, we aimed to not only quantitate the amount of research addressing a particular topic of bias, but also reveal the biases that are understudied.

Through our analysis, we generated 11 descriptive categories against which bias was studied: Age, physical disability, education level, biological sex, disease or condition, LGBTQ + , non-specified, race/ethnicity, rural/urban, socio-economic status, and weight (Table 1 ). “Age” and “weight” categories included papers that studied bias against older population and higher weight individuals, respectively. The categories “education level” and “socio-economic status” included papers that studied bias against individuals with low education level and individuals belonging to low socioeconomic status, respectively. Within the bias category named ‘biological sex’, we included papers that studied bias against individuals perceived as women/females. Papers that studied bias against gender-identity or sexual orientation were included in its own category named, ‘LGBTQ + ’. The bias category, ‘disease or condition’ was broad and included research on bias against any patient with a specific disease, condition or lifestyle. Studies included in this category researched bias against any physical illnesses, mental illnesses, or sexually transmitted infections. It also included studies that addressed bias against a treatment such as transplant or pain management. It was not significant to report these as individual categories but rather as a whole with a common underlying theme. Rural/urban bias referred to bias that was held against a person based on their place of residence. Studies grouped together in the ‘non-specified bias’ category explored bias without specifying any descriptive characteristic in their methods. These studies did not address any specific bias characteristic in particular but consisted of a study population of our interest (MS or Res or mixed). Based on our analysis, the top five most studied bias categories in our included population within medical education literature were: racial or ethnic bias ( n  = 39/163, 24%), disease or condition bias ( n  = 29/163, 18%), weight bias ( n  = 22/163, 13%), LGBTQ + bias ( n  = 21/163, 13%), and age bias ( n  = 16/163, 10%) which are presented in Table 1 .

Sorting of included research by population

In order to understand the distribution of bias research based on their populations examined, we sorted the included studies in one of the following: medical students (MS), residents (Res) or mixed (Table 1 ). The following distributions were observed: medical students only ( n  = 105/139, 76%), residents only ( n  = 19/139, 14%) or mixed which consisted of both medical students and residents ( n  = 15/139, 11%). In combination, these results demonstrate that medical educators have focused bias research efforts primarily on medical student populations.

Sorting of included research by goal

A critical component of this scoping review was to quantify the research goal of the included studies within each of the bias categories. We defined the research goal as either to document evidence of bias (EOB) or to evaluate a bias intervention (BI) (see Fig.  1 for inclusion criteria). Some of the included studies focused on both, documenting evidence in addition to intervening biases and those studies were grouped separately. The analysis revealed that 69/139 (50%) of the included studies focused exclusively on documenting evidence of bias (EOB). There were fewer studies ( n  = 51/139, 37%) which solely focused on bias interventions such as programs, seminars or curricular innovations. A small minority of the included studies were more comprehensive in that they documented EOB followed by an intervention strategy ( n  = 19/139, 11%). These results demonstrate that most bias research is dedicated to documenting evidence of bias among these groups rather than evaluating a bias intervention strategy.

Research goal distribution

Our next objective was to calculate the distribution of studies with respect to the study goal (EOB, BI or both), within the 163 biases studied across the 139 papers as calculated in Table 1 . In general, the goal of the studies favors documenting evidence of bias with the exception of race/ethnic bias which is more focused on bias intervention (Fig.  3 ). Fewer studies were aimed at both, documenting evidence then providing an intervention, across all bias categories.

Sorting of total biases ( n  = 163) within medical students or residents or a mixed population based on the bias category . Dark grey indicates studies with a dual goal, to document evidence of bias and to intervene bias. Medium grey bars indicate studies which focused on documenting evidence of bias. Light grey bars indicate studies focused on bias intervention within these populations. Numbers inside the bars indicate the total number of biases for the respective study goal. * Non-specified bias includes studies which focused on implicit bias but did not mention the type of bias investigated

Furthermore, we also calculated the ratio of EOB, BI and both (EOB + BI) within each of our population of interest (MS; n  = 122, Res; n  = 26 and mixed; n  = 15) for the 163 biases observed in our included studies. Over half ( n  = 64/122, 52%) of the total bias occurrences in MS were focused on documenting EOB (Fig.  4 ). Contrastingly, a shift was observed within resident populations where most biases addressed were aimed at intervention ( n  = 12/26, 41%) rather than EOB ( n  = 4/26, 14%) (Fig.  4 ). Studies which included both MS and Res (mixed) were primarily focused on documenting EOB ( n  = 9/15, 60%), with 33% ( n  = 5/15) aimed at bias intervention and 7% ( n  = 1/15) which did both (Fig.  4 ). Although far fewer studies were documented in the Res population it is important to highlight that most of these studies were focused on bias intervention when compared to MS population where we documented a majority of studies focused on evidence of bias.

A ratio of the study goal for the total biases ( n  = 163) mapped within each of the study population (MS, Res and Mixed). A study goal with a) documenting evidence of bias (EOB) is depicted in dotted grey, b) bias intervention (BI) in medium grey, and c) a dual focus (EOB + BI) is depicted in dark grey. * N  = 122 for medical student studies. b N  = 26 for residents. c N  = 15 for mixed

Addressing biases at an earlier stage of medical career is critical for future physicians engaging with diverse patients, since it is established that bias negatively influences provider-patient interactions [ 171 ], clinical decision-making [ 172 ] and reduces favorable treatment outcomes [ 2 ]. We set out with an intention to explore how bias is addressed within the medical curriculum. Our research question was: how has the trend in bias research changed over time, more specifically a) what is the timeline of papers published? b) what bias characteristics have been studied in the physician-trainee population and c) how are these biases addressed? With the introduction of ‘standards of diversity’ by the Liaison Committee on Medical Education, along with the Association of American Medical Colleges (AAMC) and the American Medical Association (AMA) [ 173 , 174 ], we certainly expected and observed a sustained uptick in research pertaining to bias. As shown here, research addressing bias in the target population (MS and Res) is on the rise, however only 139 papers fit our inclusion criteria. Of these studies, nearly 90% have been published since 2005 after the “Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care” report was published in 2003 [ 7 ]. However, given the well documented effects of physician held bias, we anticipated significantly more number of studies focused on bias at the medical student or resident level.

A key component from this study was that we generated descriptive categories of biases. Sorting the biases into descriptive categories helps to identify a more targeted approach for a specific bias intervention, rather than to broadly intervene bias as a whole. In fact, our analysis found a number of publications (labeled “non-specified bias” in Table 1 ) which studied implicit bias without specifying the patient attribute or the characteristic that the bias was against. In total, we generated 11 descriptive categories of bias from our scoping review which are shown in Table 1 and Fig.  3 . Furthermore, our bias descriptors grouped similar kinds of biases within a single category. For example, the category, “disease or condition” included papers that studied bias against any type of disease (Mental illness, HIV stigma, diabetes), condition (Pain management), or lifestyle. We neither performed a qualitative assessment of the studies nor did we test the efficacy of the bias intervention studies and consider it a future direction of this work.

Evidence suggests that medical educators and healthcare professionals are struggling to find the appropriate approach to intervene biases [ 175 , 176 , 177 ] So far, bias reduction, bias reflection and bias management approaches have been proposed [ 26 , 27 , 178 ]. Previous implicit bias intervention strategies have been shown to be ineffective when biased attitudes of participants were assessed after a lag [ 179 ]. Understanding the descriptive categories of bias and previous existing research efforts, as we present here is only a fraction of the challenge. The theory of “cognitive bias” [ 180 ] and related branches of research [ 13 , 181 , 182 , 183 , 184 ] have been studied in the field of psychology for over three decades. It is only recently that cognitive bias theory has been applied to the field of medical education medicine, to explain its negative influence on clinical decision-making pertaining only to racial minorities [ 1 , 2 , 15 , 16 , 17 , 185 ]. In order to elicit meaningful changes with respect to targeted bias intervention, it is necessary to understand the psychological underpinnings (attitudes) leading to a certain descriptive category of bias (behaviors). The questions which medical educators need to ask are: a) Can these descriptive biases be identified under certain type/s of cognitive errors that elicits the bias and vice versa b) Are we working towards an attitude change which can elicit a sustained positive behavior change among healthcare professionals? And most importantly, c) are we creating a culture where participants voluntarily enroll themselves in bias interventions as opposed to being mandated to participate? Cognitive psychologists and behavioral scientists are well-positioned to help us find answers to these questions as they understand human behavior. Therefore, an interdisciplinary approach, a marriage between cognitive psychologists and medical educators, is key in targeting biases held by medical students, residents, and ultimately future physicians. This review may also be of interest to behavioral psychologists, keen on providing targeted intervening strategies to clinicians depending on the characteristics (age, weight, sex or race) the portrayed bias is against. Further, instead of an individualized approach, we need to strive for systemic changes and evidence-based strategies to intervene biases.

The next element in change is directing intervention strategies at the right stage in clinical education. Our study demonstrated that most of the research collected at the medical student level was focused on documenting evidence of bias. Although the overall number of studies at the resident level were fewer than at the medical student level, the ratio of research in favor of bias intervention was higher at the resident level (see Fig.  3 ). However, it could be helpful to focus on bias intervention earlier in learning, rather than at a later stage [ 186 ]. Additionally, educational resources such as textbooks, preparatory materials, and educators themselves are potential sources of propagating biases and therefore need constant evaluation against best practices [ 187 , 188 ].

This study has limitations. First, the list of the descriptive bias categories that we generated was not grounded in any particular theory so assigning a category was subjective. Additionally, there were studies that were categorized as “nonspecified” bias as the studies themselves did not mention the specific type of bias that they were addressing. Moreover, we had to exclude numerous publications solely because they were not evidence-based and were either perspectives, commentaries or opinion pieces. Finally, there were overall fewer studies focused on the resident population, so the calculated ratio of MS:Res studies did not compare similar sample sizes.

Future directions of our study include working with behavioral scientists to categorize these bias characteristics (Table 1 ) into cognitive error types [ 189 ]. Additionally, we aim to assess the effectiveness of the intervention strategies and categorize the approach of the intervention strategies.

The primary goal of our review was to organize, compare and quantify literature pertaining to bias within medical school curricula and residency programs. We neither performed a qualitative assessment of the studies nor did we test the efficacy of studies that were sorted into “bias intervention” as is typical of scoping reviews [ 22 ]. In summary, our research identified 11 descriptive categories of biases studied within medical students and resident populations with “race and ethnicity”, “disease or condition”, “weight”, “LGBTQ + ” and “age” being the top five most studied biases. Additionally, we found a greater number of studies conducted in medical students (105/139) when compared to residents (19/139). However, most of the studies in the resident population focused on bias intervention. The results from our review highlight the following gaps: a) bias categories where more research is needed, b) biases that are studied within medical school versus in residency programs and c) study focus in terms of demonstrating the presence of bias or working towards bias intervention.

This review provides a visual analysis of the known categories of bias addressed within the medical school curriculum and in residency programs in addition to providing a comparison of studies with respect to the study goal within medical education literature. The results from our review should be of interest to community organizations, institutions, program directors and medical educators interested in knowing and understanding the types of bias existing within healthcare populations. It might be of special interest to researchers who wish to explore other types of biases that have been understudied within medical school and resident populations, thus filling the gaps existing in bias research.

Despite the number of studies designed to provide bias intervention for MS and Res populations, and an overall cultural shift to be aware of one’s own biases, biases held by both medical students and residents still persist. Further, psychologists have recently demonstrated the ineffectiveness of some bias intervention efforts [ 179 , 190 ]. Therefore, it is perhaps unrealistic to expect these biases to be eliminated altogether. However, effective intervention strategies grounded in cognitive psychology should be implemented earlier on in medical training. Our focus should be on providing evidence-based approaches and safe spaces for an attitude and culture change, so as to induce actionable behavioral changes.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

• Medical student

Evidence of bias

• Bias intervention

Hagiwara N, Mezuk B, Elston Lafata J, Vrana SR, Fetters MD. Study protocol for investigating physician communication behaviours that link physician implicit racial bias and patient outcomes in Black patients with type 2 diabetes using an exploratory sequential mixed methods design. BMJ Open. 2018;8(10):e022623.

Article   Google Scholar  

Haider AH, Schneider EB, Sriram N, Dossick DS, Scott VK, Swoboda SM, Losonczy L, Haut ER, Efron DT, Pronovost PJ, et al. Unconscious race and social class bias among acute care surgical clinicians and clinical treatment decisions. JAMA Surg. 2015;150(5):457–64.

Penner LA, Dovidio JF, Gonzalez R, Albrecht TL, Chapman R, Foster T, Harper FW, Hagiwara N, Hamel LM, Shields AF, et al. The effects of oncologist implicit racial bias in racially discordant oncology interactions. J Clin Oncol. 2016;34(24):2874–80.

Phelan SM, Burgess DJ, Yeazel MW, Hellerstedt WL, Griffin JM, van Ryn M. Impact of weight bias and stigma on quality of care and outcomes for patients with obesity. Obes Rev. 2015;16(4):319–26.

Garrett SB, Jones L, Montague A, Fa-Yusuf H, Harris-Taylor J, Powell B, Chan E, Zamarripa S, Hooper S, Chambers Butcher BD. Challenges and opportunities for clinician implicit bias training: insights from perinatal care stakeholders. Health Equity. 2023;7(1):506–19.

Shah HS, Bohlen J. Implicit bias. In: StatPearls. Treasure Island (FL): StatPearls Publishing; 2023. Copyright © 2023, StatPearls Publishing LLC.

Google Scholar  

Institute of Medicine (US) Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. In: Smedley BD, Stith AY, Nelson AR, editors. Washington (DC): National Academies Press (US); 2003. PMID: 25032386.

Dehon E, Weiss N, Jones J, Faulconer W, Hinton E, Sterling S. A systematic review of the impact of physician implicit racial bias on clinical decision making. Acad Emerg Med. 2017;24(8):895–904.

Oliver MN, Wells KM, Joy-Gaba JA, Hawkins CB, Nosek BA. Do physicians’ implicit views of African Americans affect clinical decision making? J Am Board Fam Med. 2014;27(2):177–88.

Rincon-Subtirelu M. Education as a tool to modify anti-obesity bias among pediatric residents. Int J Med Educ. 2017;8:77–8.

Gustafsson Sendén M, Renström EA. Gender bias in assessment of future work ability among pain patients - an experimental vignette study of medical students’ assessment. Scand J Pain. 2019;19(2):407–14.

Hardeman RR, Burgess D, Phelan S, Yeazel M, Nelson D, van Ryn M. Medical student socio-demographic characteristics and attitudes toward patient centered care: do race, socioeconomic status and gender matter? A report from the medical student CHANGES study. Patient Educ Couns. 2015;98(3):350–5.

Greenwald AG, Banaji MR. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychol Rev. 1995;102(1):4–27.

Kruse JA, Collins JL, Vugrin M. Educational strategies used to improve the knowledge, skills, and attitudes of health care students and providers regarding implicit bias: an integrative review of the literature. Int J Nurs Stud Adv. 2022;4:100073.

Zestcott CA, Blair IV, Stone J. Examining the presence, consequences, and reduction of implicit bias in health care: a narrative review. Group Process Intergroup Relat. 2016;19(4):528–42.

Hall WJ, Chapman MV, Lee KM, Merino YM, Thomas TW, Payne BK, Eng E, Day SH, Coyne-Beasley T. Implicit racial/ethnic bias among health care professionals and its influence on health care outcomes: a systematic review. Am J Public Health. 2015;105(12):E60–76.

FitzGerald C, Hurst S. Implicit bias in healthcare professionals: a systematic review. BMC Med Ethics. 2017;18(1):19.

Gonzalez CM, Onumah CM, Walker SA, Karp E, Schwartz R, Lypson ML. Implicit bias instruction across disciplines related to the social determinants of health: a scoping review. Adv Health Sci Educ. 2023;28(2):541–87.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.

Pat C, Geeta S, Sílvia M. Cognitive debiasing 1: origins of bias and theory of debiasing. BMJ Qual Saf. 2013;22(Suppl 2):ii58.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Thomas A, Lubarsky S, Durning SJ, Young ME. Knowledge syntheses in medical education: demystifying scoping reviews. Acad Med. 2017;92(2):161–6.

Hagopian A, Thompson MJ, Fordyce M, Johnson KE, Hart LG. The migration of physicians from sub-Saharan Africa to the United States of America: measures of the African brain drain. Hum Resour Health. 2004;2(1):17.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Teal CR, Shada RE, Gill AC, Thompson BM, Frugé E, Villarreal GB, Haidet P. When best intentions aren’t enough: Helping medical students develop strategies for managing bias about patients. J Gen Intern Med. 2010;25(Suppl 2):S115–8.

Gonzalez CM, Walker SA, Rodriguez N, Noah YS, Marantz PR. Implicit bias recognition and management in interpersonal encounters and the learning environment: a skills-based curriculum for medical students. MedEdPORTAL. 2021;17:11168.

Hoffman KM, Trawalter S, Axt JR, Oliver MN. Racial bias in pain assessment and treatment recommendations, and false beliefs about biological differences between blacks and whites. Proc Natl Acad Sci U S A. 2016;113(16):4296–301.

Mayfield JJ, Ball EM, Tillery KA, Crandall C, Dexter J, Winer JM, Bosshardt ZM, Welch JH, Dolan E, Fancovic ER, et al. Beyond men, women, or both: a comprehensive, LGBTQ-inclusive, implicit-bias-aware, standardized-patient-based sexual history taking curriculum. MedEdPORTAL. 2017;13:10634.

Morris M, Cooper RL, Ramesh A, Tabatabai M, Arcury TA, Shinn M, Im W, Juarez P, Matthews-Juarez P. Training to reduce LGBTQ-related bias among medical, nursing, and dental students and providers: a systematic review. BMC Med Educ. 2019;19(1):325.

Perdomo J, Tolliver D, Hsu H, He Y, Nash KA, Donatelli S, Mateo C, Akagbosu C, Alizadeh F, Power-Hays A, et al. Health equity rounds: an interdisciplinary case conference to address implicit bias and structural racism for faculty and trainees. MedEdPORTAL. 2019;15:10858.

Sherman MD, Ricco J, Nelson SC, Nezhad SJ, Prasad S. Implicit bias training in a residency program: aiming for enduring effects. Fam Med. 2019;51(8):677–81.

van Ryn M, Hardeman R, Phelan SM, Burgess DJ, Dovidio JF, Herrin J, Burke SE, Nelson DB, Perry S, Yeazel M, et al. Medical school experiences associated with change in implicit racial bias among 3547 students: a medical student CHANGES study report. J Gen Intern Med. 2015;30(12):1748–56.

Chary AN, Molina MF, Dadabhoy FZ, Manchanda EC. Addressing racism in medicine through a resident-led health equity retreat. West J Emerg Med. 2020;22(1):41–4.

DallaPiazza M, Padilla-Register M, Dwarakanath M, Obamedo E, Hill J, Soto-Greene ML. Exploring racism and health: an intensive interactive session for medical students. MedEdPORTAL. 2018;14:10783.

Dennis SN, Gold RS, Wen FK. Learner reactions to activities exploring racism as a social determinant of health. Fam Med. 2019;51(1):41–7.

Gonzalez CM, Walker SA, Rodriguez N, Karp E, Marantz PR. It can be done! a skills-based elective in implicit bias recognition and management for preclinical medical students. Acad Med. 2020;95(12S Addressing Harmful Bias and Eliminating Discrimination in Health Professions Learning Environments):S150–5.

Motzkus C, Wells RJ, Wang X, Chimienti S, Plummer D, Sabin J, Allison J, Cashman S. Pre-clinical medical student reflections on implicit bias: Implications for learning and teaching. PLoS ONE. 2019;14(11):e0225058.

Phelan SM, Burke SE, Cunningham BA, Perry SP, Hardeman RR, Dovidio JF, Herrin J, Dyrbye LN, White RO, Yeazel MW, et al. The effects of racism in medical education on students’ decisions to practice in underserved or minority communities. Acad Med. 2019;94(8):1178–89.

Zeidan A, Tiballi A, Woodward M, Di Bartolo IM. Targeting implicit bias in medicine: lessons from art and archaeology. West J Emerg Med. 2019;21(1):1–3.

Baker TK, Smith GS, Jacobs NN, Houmanfar R, Tolles R, Kuhls D, Piasecki M. A deeper look at implicit weight bias in medical students. Adv Health Sci Educ Theory Pract. 2017;22(4):889–900.

Eymard AS, Douglas DH. Ageism among health care providers and interventions to improve their attitudes toward older adults: an integrative review. J Gerontol Nurs. 2012;38(5):26–35.

Garrison CB, McKinney-Whitson V, Johnston B, Munroe A. Race matters: addressing racism as a health issue. Int J Psychiatry Med. 2018;53(5–6):436–44.

Geller G, Watkins PA. Addressing medical students’ negative bias toward patients with obesity through ethics education. AMA J Ethics. 2018;20(10):E948-959.

Onyeador IN, Wittlin NM, Burke SE, Dovidio JF, Perry SP, Hardeman RR, Dyrbye LN, Herrin J, Phelan SM, van Ryn M. The value of interracial contact for reducing anti-black bias among non-black physicians: a Cognitive Habits and Growth Evaluation (CHANGE) study report. Psychol Sci. 2020;31(1):18–30.

Poustchi Y, Saks NS, Piasecki AK, Hahn KA, Ferrante JM. Brief intervention effective in reducing weight bias in medical students. Fam Med. 2013;45(5):345–8.

Ruiz JG, Andrade AD, Anam R, Taldone S, Karanam C, Hogue C, Mintzer MJ. Group-based differences in anti-aging bias among medical students. Gerontol Geriatr Educ. 2015;36(1):58–78.

Simpson T, Evans J, Goepfert A, Elopre L. Implementing a graduate medical education anti-racism workshop at an academic university in the Southern USA. Med Educ Online. 2022;27(1):1981803.

Wittlin NM, Dovidio JF, Burke SE, Przedworski JM, Herrin J, Dyrbye L, Onyeador IN, Phelan SM, van Ryn M. Contact and role modeling predict bias against lesbian and gay individuals among early-career physicians: a longitudinal study. Soc Sci Med. 2019;238:112422.

Miller DP Jr, Spangler JG, Vitolins MZ, Davis SW, Ip EH, Marion GS, Crandall SJ. Are medical students aware of their anti-obesity bias? Acad Med. 2013;88(7):978–82.

Gonzalez CM, Deno ML, Kintzer E, Marantz PR, Lypson ML, McKee MD. A qualitative study of New York medical student views on implicit bias instruction: implications for curriculum development. J Gen Intern Med. 2019;34(5):692–8.

Gonzalez CM, Kim MY, Marantz PR. Implicit bias and its relation to health disparities: a teaching program and survey of medical students. Teach Learn Med. 2014;26(1):64–71.

Gonzalez CM, Nava S, List J, Liguori A, Marantz PR. How assumptions and preferences can affect patient care: an introduction to implicit bias for first-year medical students. MedEdPORTAL. 2021;17:11162.

Hernandez RA, Haidet P, Gill AC, Teal CR. Fostering students’ reflection about bias in healthcare: cognitive dissonance and the role of personal and normative standards. Med Teach. 2013;35(4):e1082-1089.

Kushner RF, Zeiss DM, Feinglass JM, Yelen M. An obesity educational intervention for medical students addressing weight bias and communication skills using standardized patients. BMC Med Educ. 2014;14:53.

Nazione S, Silk KJ. Patient race and perceived illness responsibility: effects on provider helping and bias. Med Educ. 2013;47(8):780–9.

Ogunyemi D. Defeating unconscious bias: the role of a structured, reflective, and interactive workshop. J Grad Med Educ. 2021;13(2):189–94.

Phelan SM, Burke SE, Hardeman RR, White RO, Przedworski J, Dovidio JF, Perry SP, Plankey M, A Cunningham B, Finstad D, et al. Medical school factors associated with changes in implicit and explicit bias against gay and lesbian people among 3492 graduating medical students. J Gen Intern Med. 2017;32(11):1193–201.

Phelan SM, Puhl RM, Burke SE, Hardeman R, Dovidio JF, Nelson DB, Przedworski J, Burgess DJ, Perry S, Yeazel MW, et al. The mixed impact of medical school on medical students’ implicit and explicit weight bias. Med Educ. 2015;49(10):983–92.

Barber Doucet H, Ward VL, Johnson TJ, Lee LK. Implicit bias and caring for diverse populations: pediatric trainee attitudes and gaps in training. Clin Pediatr (Phila). 2021;60(9–10):408–17.

Burke SE, Dovidio JF, Przedworski JM, Hardeman RR, Perry SP, Phelan SM, Nelson DB, Burgess DJ, Yeazel MW, van Ryn M. Do contact and empathy mitigate bias against gay and lesbian people among heterosexual first-year medical students? A report from the medical student CHANGE study. Acad Med. 2015;90(5):645–51.

Johnston B, McKinney-Whitson V, Garrison V. Race matters: addressing racism as a health issue. WMJ. 2021;120(S1):S74–7.

Kost A, Akande T, Jones R, Gabert R, Isaac M, Dettmar NS. Use of patient identifiers at the University of Washington School of Medicine: building institutional consensus to reduce bias and stigma. Fam Med. 2021;53(5):366–71.

Madan AK, Aliabadi-Wahle S, Beech DJ. Ageism in medical students’ treatment recommendations: the example of breast-conserving procedures. Acad Med. 2001;76(3):282–4.

Marbin J, Lewis L, Kuo AK, Schudel C, Gutierrez JR. The power of place: travel to explore structural racism and health disparities. Acad Med. 2021;96(11):1569–73.

Phelan SM, Dovidio JF, Puhl RM, Burgess DJ, Nelson DB, Yeazel MW, Hardeman R, Perry S, van Ryn M. Implicit and explicit weight bias in a national sample of 4,732 medical students: the medical student CHANGES study. Obesity (Silver Spring). 2014;22(4):1201–8.

Van J, Aloman C, Reau N. Potential bias and misconceptions in liver transplantation for alcohol- and obesity-related liver disease. Am J Gastroenterol. 2021;116(10):2089–97.

White-Means S, Zhiyong D, Hufstader M, Brown LT. Cultural competency, race, and skin tone bias among pharmacy, nursing, and medical students: implications for addressing health disparities. Med Care Res Rev. 2009;66(4):436–55.

Williams RL, Vasquez CE, Getrich CM, Kano M, Boursaw B, Krabbenhoft C, Sussman AL. Racial/gender biases in student clinical decision-making: a mixed-method study of medical school attributes associated with lower incidence of biases. J Gen Intern Med. 2018;33(12):2056–64.

Cohen RW, Persky S. Influence of weight etiology information and trainee characteristics on physician-trainees’ clinical and interpersonal communication. Patient Educ Couns. 2019;102(9):1644–9.

Haider AH, Sexton J, Sriram N, Cooper LA, Efron DT, Swoboda S, Villegas CV, Haut ER, Bonds M, Pronovost PJ, et al. Association of unconscious race and social class bias with vignette-based clinical assessments by medical students. JAMA. 2011;306(9):942–51.

Lewis R, Lamdan RM, Wald D, Curtis M. Gender bias in the diagnosis of a geriatric standardized patient: a potential confounding variable. Acad Psychiatry. 2006;30(5):392–6.

Matharu K, Shapiro JF, Hammer RR, Kravitz RL, Wilson MD, Fitzgerald FT. Reducing obesity prejudice in medical education. Educ Health. 2014;27(3):231–7.

McLean ME, McLean LE, McLean-Holden AC, Campbell LF, Horner AM, Kulkarni ML, Melville LD, Fernandez EA. Interphysician weight bias: a cross-sectional observational survey study to guide implicit bias training in the medical workplace. Acad Emerg Med. 2021;28(9):1024–34.

Meadows A, Higgs S, Burke SE, Dovidio JF, van Ryn M, Phelan SM. Social dominance orientation, dispositional empathy, and need for cognitive closure moderate the impact of empathy-skills training, but not patient contact, on medical students’ negative attitudes toward higher-weight patients. Front Psychol. 2017;8:15.

Stone J, Moskowitz GB, Zestcott CA, Wolsiefer KJ. Testing active learning workshops for reducing implicit stereotyping of Hispanics by majority and minority group medical students. Stigma Health. 2020;5(1):94–103.

Symons AB, Morley CP, McGuigan D, Akl EA. A curriculum on care for people with disabilities: effects on medical student self-reported attitudes and comfort level. Disabil Health J. 2014;7(1):88–95.

Ufomata E, Eckstrand KL, Hasley P, Jeong K, Rubio D, Spagnoletti C. Comprehensive internal medicine residency curriculum on primary care of patients who identify as LGBT. LGBT Health. 2018;5(6):375–80.

Aultman JM, Borges NJ. A clinical and ethical investigation of pre-medical and medical students’ attitudes, knowledge, and understanding of HIV. Med Educ Online. 2006;11:1–12.

Bates T, Cohan M, Bragg DS, Bedinghaus J. The Medical College of Wisconsin senior mentor program: experience of a lifetime. Gerontol Geriatr Educ. 2006;27(2):93–103.

Chiaramonte GR, Friend R. Medical students’ and residents’ gender bias in the diagnosis, treatment, and interpretation of coronary heart disease symptoms. Health Psychol. 2006;25(3):255–66.

Friedberg F, Sohl SJ, Halperin PJ. Teaching medical students about medically unexplained illnesses: a preliminary study. Med Teach. 2008;30(6):618–21.

Gonzales E, Morrow-Howell N, Gilbert P. Changing medical students’ attitudes toward older adults. Gerontol Geriatr Educ. 2010;31(3):220–34.

Hinners CK, Potter JF. A partnership between the University of Nebraska College of Medicine and the community: fostering positive attitudes towards the aged. Gerontol Geriatr Educ. 2006;27(2):83–91.

Lee M, Coulehan JL. Medical students’ perceptions of racial diversity and gender equality. Med Educ. 2006;40(7):691–6.

Schmetzer AD, Lafuze JE. Overcoming stigma: involving families in medical student and psychiatric residency education. Acad Psychiatry. 2008;32(2):127–31.

Willen SS, Bullon A, Good MJD. Opening up a huge can of worms: reflections on a “cultural sensitivity” course for psychiatry residents. Harv Rev Psychiatry. 2010;18(4):247–53.

Dogra N, Karnik N. First-year medical students’ attitudes toward diversity and its teaching: an investigation at one U.S. medical school. Acad Med. 2003;78(11):1191–200.

Fitzpatrick C, Musser A, Mosqueda L, Boker J, Prislin M. Student senior partnership program: University of California Irvine School of Medicine. Gerontol Geriatr Educ. 2006;27(2):25–35.

Hoffman KG, Gray P, Hosokawa MC, Zweig SC. Evaluating the effectiveness of a senior mentor program: the University of Missouri-Columbia School of Medicine. Gerontol Geriatr Educ. 2006;27(2):37–47.

Kantor BS, Myers MR. From aging…to saging-the Ohio State Senior Partners Program: longitudinal and experiential geriatrics education. Gerontol Geriatr Educ. 2006;27(2):69–74.

Klamen DL, Grossman LS, Kopacz DR. Medical student homophobia. J Homosex. 1999;37(1):53–63.

Kopacz DR, Grossman LS, Klamen DL. Medical students and AIDS: knowledge, attitudes and implications for education. Health Educ Res. 1999;14(1):1–6.

Leiblum SR. An established medical school human sexuality curriculum: description and evaluation. Sex Relatsh Ther. 2001;16(1):59–70.

Rastegar DA, Fingerhood MI, Jasinski DR. A resident clerkship that combines inpatient and outpatient training in substance abuse and HIV care. Subst Abuse. 2004;25(4):11–5.

Roberts E, Richeson NA, Thornhill JTIV, Corwin SJ, Eleazer GP. The senior mentor program at the University of South Carolina School of Medicine: an innovative geriatric longitudinal curriculum. Gerontol Geriatr Educ. 2006;27(2):11–23.

Burgess DJ, Burke SE, Cunningham BA, Dovidio JF, Hardeman RR, Hou YF, Nelson DB, Perry SP, Phelan SM, Yeazel MW, et al. Medical students’ learning orientation regarding interracial interactions affects preparedness to care for minority patients: a report from medical student CHANGES. BMC Med Educ. 2016;16:254.

Burgess DJ, Hardeman RR, Burke SE, Cunningham BA, Dovidio JF, Nelson DB, Perry SP, Phelan SM, Yeazel MW, Herrin J, et al. Incoming medical students’ political orientation affects outcomes related to care of marginalized groups: results from the medical student CHANGES study. J Health Pol Policy Law. 2019;44(1):113–46.

Kurtz ME, Johnson SM, Tomlinson T, Fiel NJ. Teaching medical students the effects of values and stereotyping on the doctor/patient relationship. Soc Sci Med. 1985;21(9):1043–7.

Matharu K, Kravitz RL, McMahon GT, Wilson MD, Fitzgerald FT. Medical students’ attitudes toward gay men. BMC Med Educ. 2012;12:71.

Pearl RL, Argueso D, Wadden TA. Effects of medical trainees’ weight-loss history on perceptions of patients with obesity. Med Educ. 2017;51(8):802–11.

Perry SP, Dovidio JF, Murphy MC, van Ryn M. The joint effect of bias awareness and self-reported prejudice on intergroup anxiety and intentions for intergroup contact. Cultur Divers Ethnic Minor Psychol. 2015;21(1):89–96.

Phelan SM, Burgess DJ, Burke SE, Przedworski JM, Dovidio JF, Hardeman R, Morris M, van Ryn M. Beliefs about the causes of obesity in a national sample of 4th year medical students. Patient Educ Couns. 2015;98(11):1446–9.

Phelan SM, Puhl RM, Burgess DJ, Natt N, Mundi M, Miller NE, Saha S, Fischer K, van Ryn M. The role of weight bias and role-modeling in medical students’ patient-centered communication with higher weight standardized patients. Patient Educ Couns. 2021;104(8):1962–9.

Polan HJ, Auerbach MI, Viederman M. AIDS as a paradigm of human behavior in disease: impact and implications of a course. Acad Psychiatry. 1990;14(4):197–203.

Reuben DB, Fullerton JT, Tschann JM, Croughan-Minihane M. Attitudes of beginning medical students toward older persons: a five-campus study. J Am Geriatr Soc. 1995;43(12):1430–6.

Tsai J. Building structural empathy to marshal critical education into compassionate practice: evaluation of a medical school critical race theory course. J Law Med Ethics. 2021;49(2):211–21.

Weyant RJ, Bennett ME, Simon M, Palaisa J. Desire to treat HIV-infected patients: similarities and differences across health-care professions. AIDS. 1994;8(1):117–21.

Ross PT, Lypson ML. Using artistic-narrative to stimulate reflection on physician bias. Teach Learn Med. 2014;26(4):344–9.

Calabrese SK, Earnshaw VA, Krakower DS, Underhill K, Vincent W, Magnus M, Hansen NB, Kershaw TS, Mayer KH, Betancourt JR, et al. A closer look at racism and heterosexism in medical students’ clinical decision-making related to HIV Pre-Exposure Prophylaxis (PrEP): implications for PrEP education. AIDS Behav. 2018;22(4):1122–38.

Fitterman-Harris HF, Vander Wal JS. Weight bias reduction among first-year medical students: a quasi-randomized, controlled trial. Clin Obes. 2021;11(6):e12479.

Madan AK, Cooper L, Gratzer A, Beech DJ. Ageism in breast cancer surgical options by medical students. Tenn Med. 2006;99(5):37–8, 41.

Bikmukhametov DA, Anokhin VA, Vinogradova AN, Triner WR, McNutt LA. Bias in medicine: a survey of medical student attitudes towards HIV-positive and marginalized patients in Russia, 2010. J Int AIDS Soc. 2012;15(2):17372.

Dijkstra AF, Verdonk P, Lagro-Janssen AL. Gender bias in medical textbooks: examples from coronary heart disease, depression, alcohol abuse and pharmacology. Med Educ. 2008;42(10):1021–8.

Dobrowolska B, Jędrzejkiewicz B, Pilewska-Kozak A, Zarzycka D, Ślusarska B, Deluga A, Kościołek A, Palese A. Age discrimination in healthcare institutions perceived by seniors and students. Nurs Ethics. 2019;26(2):443–59.

Hamberg K, Risberg G, Johansson EE, Westman G. Gender bias in physicians’ management of neck pain: a study of the answers in a Swedish national examination. J Womens Health Gend Based Med. 2002;11(7):653–66.

Magliano L, Read J, Sagliocchi A, Oliviero N, D’Ambrosio A, Campitiello F, Zaccaro A, Guizzaro L, Patalano M. “Social dangerousness and incurability in schizophrenia”: results of an educational intervention for medical and psychology students. Psychiatry Res. 2014;219(3):457–63.

Reis SP, Wald HS. Contemplating medicine during the Third Reich: scaffolding professional identity formation for medical students. Acad Med. 2015;90(6):770–3.

Schroyen S, Adam S, Marquet M, Jerusalem G, Thiel S, Giraudet AL, Missotten P. Communication of healthcare professionals: Is there ageism? Eur J Cancer Care (Engl). 2018;27(1):e12780.

Swift JA, Hanlon S, El-Redy L, Puhl RM, Glazebrook C. Weight bias among UK trainee dietitians, doctors, nurses and nutritionists. J Hum Nutr Diet. 2013;26(4):395–402.

Swift JA, Tischler V, Markham S, Gunning I, Glazebrook C, Beer C, Puhl R. Are anti-stigma films a useful strategy for reducing weight bias among trainee healthcare professionals? Results of a pilot randomized control trial. Obes Facts. 2013;6(1):91–102.

Yertutanol FDK, Candansayar S, Seydaoğlu G. Homophobia in health professionals in Ankara, Turkey: developing a scale. Transcult Psychiatry. 2019;56(6):1191–217.

Arnold O, Voracek M, Musalek M, Springer-Kremser M. Austrian medical students’ attitudes towards male and female homosexuality: a comparative survey. Wien Klin Wochenschr. 2004;116(21–22):730–6.

Arvaniti A, Samakouri M, Kalamara E, Bochtsou V, Bikos C, Livaditis M. Health service staff’s attitudes towards patients with mental illness. Soc Psychiatry Psychiatr Epidemiol. 2009;44(8):658–65.

Lopes L, Gato J, Esteves M. Portuguese medical students’ knowledge and attitudes towards homosexuality. Acta Med Port. 2016;29(11):684–93.

Papadaki V, Plotnikof K, Gioumidou M, Zisimou V, Papadaki E. A comparison of attitudes toward lesbians and gay men among students of helping professions in Crete, Greece: the cases of social work, psychology, medicine, and nursing. J Homosex. 2015;62(6):735–62.

Papaharitou S, Nakopoulou E, Moraitou M, Tsimtsiou Z, Konstantinidou E, Hatzichristou D. Exploring sexual attitudes of students in health professions. J Sex Med. 2008;5(6):1308–16.

Roberts JH, Sanders T, Mann K, Wass V. Institutional marginalisation and student resistance: barriers to learning about culture, race and ethnicity. Adv Health Sci Educ. 2010;15(4):559–71.

Wilhelmi L, Ingendae F, Steinhaeuser J. What leads to the subjective perception of a ‘rural area’? A qualitative study with undergraduate students and postgraduate trainees in Germany to tailor strategies against physician’s shortage. Rural Remote Health. 2018;18(4):4694.

Herrmann-Werner A, Loda T, Wiesner LM, Erschens RS, Junne F, Zipfel S. Is an obesity simulation suit in an undergraduate medical communication class a valuable teaching tool? A cross-sectional proof of concept study. BMJ Open. 2019;9(8):e029738.

Ahadinezhad B, Khosravizadeh O, Maleki A, Hashtroodi A. Implicit racial bias among medical graduates and students by an IAT measure: a systematic review and meta-analysis. Ir J Med Sci. 2022;191(4):1941–9. https://doi.org/10.1007/s11845-021-02756-3 .

Hsieh JG, Hsu M, Wang YW. An anthropological approach to teach and evaluate cultural competence in medical students - the application of mini-ethnography in medical history taking. Med Educ Online. 2016;21:32561.

Poreddi V, Thimmaiah R, Math SB. Attitudes toward people with mental illness among medical students. J Neurosci Rural Pract. 2015;6(3):349–54.

Mino Y, Yasuda N, Tsuda T, Shimodera S. Effects of a one-hour educational program on medical students’ attitudes to mental illness. Psychiatry Clin Neurosci. 2001;55(5):501–7.

Omori A, Tateno A, Ideno T, Takahashi H, Kawashima Y, Takemura K, Okubo Y. Influence of contact with schizophrenia on implicit attitudes towards schizophrenia patients held by clinical residents. BMC Psychiatry. 2012;12:8.

Banwari G, Mistry K, Soni A, Parikh N, Gandhi H. Medical students and interns’ knowledge about and attitude towards homosexuality. J Postgrad Med. 2015;61(2):95–100.

Lee SY. Obesity education in medical school curricula in Korea. J Obes Metab Syndr. 2018;27(1):35–8.

Aruna G, Mittal S, Yadiyal MB, Acharya C, Acharya S, Uppulari C. Perception, knowledge, and attitude toward mental disorders and psychiatry among medical undergraduates in Karnataka: a cross-sectional study. Indian J Psychiatry. 2016;58(1):70–6.

Wong YL. Review paper: gender competencies in the medical curriculum: addressing gender bias in medicine. Asia Pac J Public Health. 2009;21(4):359–76.

Earnshaw VA, Jin H, Wickersham JA, Kamarulzaman A, John J, Lim SH, Altice FL. Stigma toward men who have sex with men among future healthcare providers in Malaysia: would more interpersonal contact reduce prejudice? AIDS Behav. 2016;20(1):98–106.

Larson B, Herx L, Williamson T, Crowshoe L. Beyond the barriers: family medicine residents’ attitudes towards providing Aboriginal health care. Med Educ. 2011;45(4):400–6.

Wagner AC, Girard T, McShane KE, Margolese S, Hart TA. HIV-related stigma and overlapping stigmas towards people living with HIV among health care trainees in Canada. AIDS Educ Prev. 2017;29(4):364–76.

Tellier P-P, Bélanger E, Rodríguez C, Ware MA, Posel N. Improving undergraduate medical education about pain assessment and management: a qualitative descriptive study of stakeholders’ perceptions. Pain Res Manage. 2013;18(5):259–65.

Loignon C, Boudreault-Fournier A, Truchon K, Labrousse Y, Fortin B. Medical residents reflect on their prejudices toward poverty: a photovoice training project. BMC Med Educ. 2014;14:1050.

Phillips SP, Clarke M. More than an education: the hidden curriculum, professional attitudes and career choice. Med Educ. 2012;46(9):887–93.

Jaworsky D, Gardner S, Thorne JG, Sharma M, McNaughton N, Paddock S, Chew D, Lees R, Makuwaza T, Wagner A, et al. The role of people living with HIV as patient instructors—Reducing stigma and improving interest around HIV care among medical students. AIDS Care. 2017;29(4):524–31.

Sukhera J, Wodzinski M, Teunissen PW, Lingard L, Watling C. Striving while accepting: exploring the relationship between identity and implicit bias recognition and management. Acad Med. 2018;93(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 57th Annual Research in Medical Education Sessions):S82-s88.

Harris R, Cormack D, Curtis E, Jones R, Stanley J, Lacey C. Development and testing of study tools and methods to examine ethnic bias and clinical decision-making among medical students in New Zealand: the Bias and Decision-Making in Medicine (BDMM) study. BMC Med Educ. 2016;16:173.

Cormack D, Harris R, Stanley J, Lacey C, Jones R, Curtis E. Ethnic bias amongst medical students in Aotearoa/New Zealand: findings from the Bias and Decision Making in Medicine (BDMM) study. PLoS ONE. 2018;13(8):e0201168.

Harris R, Cormack D, Stanley J, Curtis E, Jones R, Lacey C. Ethnic bias and clinical decision-making among New Zealand medical students: an observational study. BMC Med Educ. 2018;18(1):18.

Robinson EL, Ball LE, Leveritt MD. Obesity bias among health and non-health students attending an Australian university and their perceived obesity education. J Nutr Educ Behav. 2014;46(5):390–5.

Sopoaga F, Zaharic T, Kokaua J, Covello S. Training a medical workforce to meet the needs of diverse minority communities. BMC Med Educ. 2017;17:19.

Parker R, Larkin T, Cockburn J. A visual analysis of gender bias in contemporary anatomy textbooks. Soc Sci Med. 2017;180:106–13.

Gomes MdM. Doctors’ perspectives and practices regarding epilepsy. Arq Neuropsiquiatr. 2000;58(2):221–6.

Caixeta J, Fernandes PT, Bell GS, Sander JW, Li LM. Epilepsy perception amongst university students - A survey. Arq Neuropsiquiatr. 2007;65:43–8.

Tedrus GMAS, Fonseca LC, da Câmara Vieira AL. Knowledge and attitudes toward epilepsy amongst students in the health area: intervention aimed at enlightenment. Arq Neuropsiquiatr. 2007;65(4-B):1181–5.

Gomez-Moreno C, Verduzco-Aguirre H, Contreras-Garduño S, Perez-de-Acha A, Alcalde-Castro J, Chavarri-Guerra Y, García-Lara JMA, Navarrete-Reyes AP, Avila-Funes JA, Soto-Perez-de-Celis E. Perceptions of aging and ageism among Mexican physicians-in-training. Clin Transl Oncol. 2019;21(12):1730–5.

Campbell MH, Gromer J, Emmanuel MK, Harvey A. Attitudes Toward Transgender People Among Future Caribbean Doctors. Arch Sex Behav. 2022;51(4):1903-11. https://doi.org/10.1007/s10508-021-02205-3 .

Hatala R, Case SM. Examining the influence of gender on medical students’ decision making. J Womens Health Gend Based Med. 2000;9(6):617–23.

Deb T, Lempp H, Bakolis I, et al. Responding to experienced and anticipated discrimination (READ): anti -stigma training for medical students towards patients with mental illness – study protocol for an international multisite non-randomised controlled study. BMC Med Educ. 2019;19:41. https://doi.org/10.1186/s12909-019-1472-7 .

Morgan S, Plaisant O, Lignier B, Moxham BJ. Sexism and anatomy, as discerned in textbooks and as perceived by medical students at Cardiff University and University of Paris Descartes. J Anat. 2014;224(3):352–65.

Alford CL, Miles T, Palmer R, Espino D. An introduction to geriatrics for first-year medical students. J Am Geriatr Soc. 2001;49(6):782–7.

Stone J, Moskowitz GB. Non-conscious bias in medical decision making: what can be done to reduce it? Med Educ. 2011;45(8):768–76.

Nazione S. Slimming down medical provider weight bias in an obese nation. Med Educ. 2015;49(10):954–5.

Dogra N, Connin S, Gill P, Spencer J, Turner M. Teaching of cultural diversity in medical schools in the United Kingdom and Republic of Ireland: cross sectional questionnaire survey. BMJ. 2005;330(7488):403–4.

Aultman JM, Borges NJ. A clinical and ethical investigation of pre-medical and medical students’ attitudes, knowledge, and understanding of HIV. Med Educ Online. 2006;11(1):4596.

Deb T, Lempp H, Bakolis I, Vince T, Waugh W, Henderson C, Thornicroft G, Ando S, Yamaguchi S, Matsunaga A, et al. Responding to experienced and anticipated discrimination (READ): anti -stigma training for medical students towards patients with mental illness – study protocol for an international multisite non-randomised controlled study. BMC Med Educ. 2019;19(1):41.

Gonzalez CM, Grochowalski JH, Garba RJ, Bonner S, Marantz PR. Validity evidence for a novel instrument assessing medical student attitudes toward instruction in implicit bias recognition and management. BMC Med Educ. 2021;21(1):205.

Ogunyemi D. A practical approach to implicit bias training. J Grad Med Educ. 2021;13(4):583–4.

Dennis GC. Racism in medicine: planning for the future. J Natl Med Assoc. 2001;93(3 Suppl):1S-5S.

Maina IW, Belton TD, Ginzberg S, Singh A, Johnson TJ. A decade of studying implicit racial/ethnic bias in healthcare providers using the implicit association test. Soc Sci Med. 2018;199:219–29.

Blair IV, Steiner JF, Hanratty R, Price DW, Fairclough DL, Daugherty SL, Bronsert M, Magid DJ, Havranek EP. An investigation of associations between clinicians’ ethnic or racial bias and hypertension treatment, medication adherence and blood pressure control. J Gen Intern Med. 2014;29(7):987–95.

Stanford FC. The importance of diversity and inclusion in the healthcare workforce. J Natl Med Assoc. 2020;112(3):247–9.

Education LCoM. Standards on diversity. 2009. https://health.usf.edu/~/media/Files/Medicine/MD%20Program/Diversity/LCMEStandardsonDiversity1.ashx?la=en .

Onyeador IN, Hudson STJ, Lewis NA. Moving beyond implicit bias training: policy insights for increasing organizational diversity. Policy Insights Behav Brain Sci. 2021;8(1):19–26.

Forscher PS, Mitamura C, Dix EL, Cox WTL, Devine PG. Breaking the prejudice habit: mechanisms, timecourse, and longevity. J Exp Soc Psychol. 2017;72:133–46.

Lai CK, Skinner AL, Cooley E, Murrar S, Brauer M, Devos T, Calanchini J, Xiao YJ, Pedram C, Marshburn CK, et al. Reducing implicit racial preferences: II. Intervention effectiveness across time. J Exp Psychol Gen. 2016;145(8):1001–16.

Sukhera J, Watling CJ, Gonzalez CM. Implicit bias in health professions: from recognition to transformation. Acad Med. 2020;95(5):717–23.

Vuletich HA, Payne BK. Stability and change in implicit bias. Psychol Sci. 2019;30(6):854–62.

Tversky A, Kahneman D. Judgment under uncertainty: Heuristics and biases. Science. 1974;185(4157):1124–31.

Miller DT, Ross M. Self-serving biases in the attribution of causality: fact or fiction? Psychol Bull. 1975;82(2):213–25.

Nickerson RS. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol. 1998;2(2):175–220.

Suveren Y. Unconscious bias: definition and significance. Psikiyatride Guncel Yaklasimlar. 2022;14(3):414–26.

Dietrich D, Olson M. A demonstration of hindsight bias using the Thomas confirmation vote. Psychol Rep. 1993;72(2):377–8.

Green AR, Carney DR, Pallin DJ, Ngo LH, Raymond KL, Iezzoni LI, Banaji MR. Implicit bias among physicians and its prediction of thrombolysis decisions for black and white patients. J Gen Intern Med. 2007;22(9):1231–8.

Rushmer R, Davies HT. Unlearning in health care. Qual Saf Health Care. 2004;13 Suppl 2(Suppl 2):ii10-15.

Vu MT, Pham TTT. Gender, critical pedagogy, and textbooks: Understanding teachers’ (lack of) mediation of the hidden curriculum in the EFL classroom. Lang Teach Res. 2022;0(0). https://doi.org/10.1177/13621688221136937 .

Kalantari A, Alvarez A, Battaglioli N, Chung A, Cooney R, Boehmer SJ, Nwabueze A, Gottlieb M. Sex and race visual representation in emergency medicine textbooks and the hidden curriculum. AEM Educ Train. 2022;6(3):e10743.

Satya-Murti S, Lockhart J. Recognizing and reducing cognitive bias in clinical and forensic neurology. Neurol Clin Pract. 2015;5(5):389–96.

Chang EH, Milkman KL, Gromet DM, Rebele RW, Massey C, Duckworth AL, Grant AM. The mixed effects of online diversity training. Proc Natl Acad Sci U S A. 2019;116(16):7778–83.

Download references

Acknowledgements

The authors would like to thank Dr. Misa Mi, Professor and Medical Librarian at the Oakland University William Beaumont School of Medicine (OWUB) for her assistance with selection of databases and construction of literature search strategies for the scoping review. The authors also wish to thank Dr. Changiz Mohiyeddini, Professor in Behavioral Medicine and Psychopathology at Oakland University William Beaumont School of Medicine (OUWB) for his expertise and constructive feedback on our manuscript.

Author information

Authors and affiliations.

Department of Foundational Sciences, Central Michigan University College of Medicine, Mt. Pleasant, MI, 48859, USA

Brianne E. Lewis

Department of Foundational Medical Studies, Oakland University William Beaumont School of Medicine, 586 Pioneer Dr, Rochester, MI, 48309, USA

Akshata R. Naik

You can also search for this author in PubMed   Google Scholar

Contributions

A.R.N and B.E.L were equally involved in study conception, design, collecting data and analyzing the data. B.E.L and A.R.N both contributed towards writing the manuscript. A.R.N and B.E.L are both senior authors on this paper. All authors reviewed the manuscript.

Corresponding author

Correspondence to Akshata R. Naik .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lewis, B.E., Naik, A.R. A scoping review to identify and organize literature trends of bias research within medical student and resident education. BMC Med Educ 23 , 919 (2023). https://doi.org/10.1186/s12909-023-04829-6

Download citation

Received : 14 March 2023

Accepted : 01 November 2023

Published : 05 December 2023

DOI : https://doi.org/10.1186/s12909-023-04829-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Preclinical curriculum
• Evidence of bis

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Skip to content

Explore VP&S

Academic programs.

Join our MD program to become a physician with compassion, a sense of self, and true grit in all medical pursuits. 

For Researchers

Areas of research.

VP&S continues to be a world leader across the entire spectrum of basic science, translational, and clinical research

• Patient Care

Departments & Centers

Clinical departments and divisions.

Our mission is to provide world-class patient care, foster innovative research, and train the next generation of leaders in medicine.

Medical Education Program Objectives

DOMAIN #1. Patient Care:   Recognizing their role as a health care professional, students use knowledge and skills during clinical encounters to gather necessary information and apply evidence in a person-centered approach to develop appropriate diagnostic and therapeutic plans as well as health promotion efforts that address conditions seen in patients across the lifespan.

1.1 Demonstrates person-centered care and humility in clinical encounters

1.2 Obtains an accurate and thorough clinical history and performs a complete and focused physical examination 

1.3 Synthesizes patient information from history, exam, and other available data such as electronic health records to generate a broad and appropriate differential diagnosis

1.4 Develops and implements a person-centered management plan informed by the prioritized differential diagnosis as well as patient preferences and goals

1.5 Demonstrates recognition of urgent/critical clinical needs and responds rapidly

DOMAIN #2. Medical Knowledge: Students demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care.

2.1 Applies foundational scientific principles to the understanding of common and critical medical conditions across the continuum of the lifespan

2.2 Applies knowledge of community and public health principles to the care of patients and populations

DOMAIN #3. Practice-Based Learning and Improvement: Students identify and pursue clear learning goals and explore new opportunities for intellectual and professional growth and development. Foremost is the ability to investigate and evaluate one’s care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning

3.1 Continually incorporates feedback and self-reflection to identify personal strengths and goals for growth

3.2 Accesses, appraises, applies, and disseminates published evidence toward the solution of clinical questions that arise in the care of patients

DOMAIN #4. Interpersonal and Communications Skills: Students actively listen to and communicate clearly and with humility with patients, families, peers, faculty and all members of the patient support system and health care team. These skills result in the effective exchange of information and collaboration with patients, their families, and health professionals.

4.1 Communicates with patients and families to establish a trusting and therapeutic relationship

4.2 Communicates medical information effectively both verbally and through written documentation

4.3 Collaborates with a community of learners to advance understanding and education

DOMAIN #5. Professionalism: Students develop professional identities that demonstrate a commitment to professional and personal responsibilities exemplified by adherence to the highest standards of leadership, advocacy and integrity, consistent with the VP&S Honor Code, VP&S values, clinical site policies and procedures, and those of the profession.

5.1 Adheres to ethical behavior in all realms of professional life and commitment to integrity, advocacy, and leadership

5.2 Demonstrates initiative, responsiveness, reliability and accountability

5.3 Demonstrates awareness of the importance of balancing the varied demands and opportunities of professional life to promote personal wellbeing

DOMAIN #6. Systems-Based Practice: Students demonstrate an awareness of and responsiveness to how individual actions and interrelated systems affect health and healthcare. They are able to call effectively on other interdisciplinary/ interprofessional resources in the system to provide optimal health care. Students acknowledge their role and the advocacy required to act within these social and system dynamics.

6.1 Works and communicates effectively as a team member, including on interprofessional teams, to optimize outcomes

6.2 Contributes to a practice of systems-thinking, safety and continuous quality improvement

6.3 Demonstrates awareness of the components of the health care system and impact on patient health

6.4 Articulates the role of the physician in addressing the existential and emerging challenges of our time

DOMAIN #7. Anti-Racism, Inclusion, Diversity, and Equity: Students demonstrate an understanding of how bias, structural and interpersonal expressions of racism, as well as all types of oppression impact health and health care, whether manifesting as social determinants of health, policies that create health disparities, or diagnostic algorithms that bias treatment decisions. Students will adhere to principles of inclusion and diversity that mitigate bias and foster belonging

7.1 Recognizes personal biases and their impact on those around them and on patient care, and can apply strategies to mitigate the effects of these biases.

7.2 Demonstrates skills necessary to serve as an ally to others and to promote agency in others when there is historical injustice

7.3 Articulates structural and historical inequities and applies strategies to mitigate systems of oppression in order to achieve equitable health care and learning environments

DOMAIN #8. Inquiry: Students demonstrate an exploratory, creative mindset that includes curiosity, skepticism, and appreciation for ambiguity, that facilitates an awareness of the limits of current knowledge and builds skills and teams that challenge current concepts and develop new knowledge and understanding

8.1 Demonstrates understanding of methods of inquiry and investigation

8.2 Develops a research question, engages in investigation to advance understanding and contributes to the dissemination of knowledge and work

• Recommended

The role of AI in medical education: Embrace it or fear it?  

Artificial intelligence is upon us and likely will forever change the way we interact with learning and education. Despite this reality, educational institutions seem to fall into either of two camps. One camp seems loath to acknowledge that AI exists. A faculty member who helps with curriculum development at one medical school recently shared, “We don’t know what to do about AI. Do we act like it’s not there, or do we acknowledge it?”

The other camp embraces AI and encourages students to employ AI resources, such as ChatGPT. Given the possibility of plagiarism or simply allowing students to bypass any learning whatsoever, it’s understandable why medical schools and other institutions might be conflicted about AI. But even though AI is no replacement for a novel idea or human thought or in medicine placing one’s hands on patients, AI has value in medical education–and not just a little bit. AI may be used as a supplement, resource, or aid when we are learning, teaching, or creating something new.

Just how could medical schools and medical students use AI to assist in educating students? Prior articles have suggested how medical students can utilize chatbots, like ChatGPT, as online tutors to help answer questions or to create quizzes to test their knowledge . For example, bots like ChatGPT can help compare differences in diagnoses, treatments, or procedures that students may be confused about. Those same AI sites can offer a personalized learning experience that schools ought to acknowledge or promote. NYU Grossman School of Medicine has run with this idea and has fully embraced the idea of precision learning from AI by incorporating a “precision education” tool. Each NYU medical student is offered a personalized medical education, with an AI algorithm tailoring subject matter and content format.

In the research space, AI can also be invaluable in medical education. For example, faculty and students alike can also utilize AI to help create data analysis plans, code for various computer languages and scan literature. An online website called Elicit lets users pose a question and then, through AI, scans the internet to find papers and synthesizes their findings into a summary.

Outside of the student experience, professors may also use AI to create lecture outlines and predict the questions that students are most likely to have about certain material. Additionally, professors and faculty must be able to set standards and address the use of AI in the classroom. If they don’t, students may misunderstand the expectations for AI and when or if its use is permissible.

In our own experience, we have used AI to create study guides for courses, create outlines for lectures and book chapters, analyze CVs, and write initial drafts of promotion letters for fellow faculty members. We are certain that the uses of AI that will further simplify our work and assist in medical learning will become clearer and only be seen as greater assets going forward.

Medical schools already offer courses on a wide range of learning and research topics, such as “best study” practices or how to conduct a literature review. Going forward, AI-based tools should be included in these lectures and within the list of online resources for student learning and research. Additionally, schools should teach students what to watch out for when using AI, like bias or flat-out false information and/or non-existent references . Teaching how to use AI in one’s learning promotes a more prepared generation for future technological innovation. This approach may complement courses that explore innovation and AI in medicine.

To those who are hesitant to incorporate AI into education even after reading about NYUs approach and our own ideas, we encourage them to look at how AI has improved other aspects of medicine. From image analysis in radiology and pathology to quick retrieval of medical information and tracking infectious disease outbreaks, this technology has created greater efficiency in health care . Other studies have found that AI can help reduce racial disparities in health care, with one investigation finding that AI better predicted pain from X-rays for underserved patients when compared to radiologists . The technology can be used for good, including in education.

To illustrate that we are not just talking about the potential values of AI in medical school education, in thinking about writing this essay, we asked ChatGPT, “how can artificial intelligence be used to teach medicine and enhance learning in medical schools.” The answers ChatGPT provided included personalized learning, virtual patients, data analysis and research, smart tutors, and educating students about the limitations, biases, and potential risks of AI tools.

As anyone ought to do when using AI, we analyzed ChatGPT’s response, and ultimately–although this might not always be the case–we agree with its recommendations. Therefore, given that we are intent on practicing what we’re now preaching, we couldn’t have written our piece without emphasizing those elements, among others.

Amelia Mercado is a medical student.  J. Wesley Boyd  is a psychiatrist.

Empowering patients: Navigating medical information with AI

Integrated coaching in physician residency training [PODCAST]

Tagged as: Health IT , Medical school

Related Posts

The role of medical education in perpetuating health care disparities

The role of income in medical school acceptance

Business education’s role in preventing physician practice decline

3 ways to change your mindset in continuing medical education

COVID-19, medical education, and the role of medical students around the world

In the face of uncertainty, choose hope over fear

More in tech.

Top 10 commandments for the ethical and effective use of AI

How AI can save lives by simplifying communication

AI adoption in health care: early adopters to late adapters

Cryonics: advancements, ethics, and skepticism

Health care informatics and the future of patient care

Most popular.

Rise of mega payouts: Physicians are now the white whales

Why minorities need more representation in breast cancer research

A crisis of alcoholism is surging in the United States

Nurse burnout: causes, symptoms, and solutions by gender and generation

Mental health support for teens [PODCAST]

Past 6 months.

Do they care if women die? Exploring women’s rights.

Doctor charged after treating a DEA agent

Panic button: Escaping the broken health care escape room

Psychological safety: an overlooked factor in clinician burnout and moral injury in health care

Primary care for all Americans: What the U.S. health care system can learn from Scituate, Rhode Island

From passion to burnout: When a doctor’s love hurts

Recent posts.

Plaintiff attorneys and their contingency fees: causes of chaos in medical malpractice litigation

Dream jobs and dream contracts: restrictive covenant nightmares

Responsibility, burnout, and self-care [PODCAST]

Subscribe to kevinmd and never miss a story.

Get free updates delivered free to your inbox.

Find jobs at Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

CME Spotlights

Leave a Comment

Comments are moderated before they are published. Please read the comment policy .

Jazz at Drew returns 09/21

• Summer Research Program

Welcome to the Summer Research Program at Charles R. Drew University of Medicine and Science, where your journey into the forefront of medical and scientific inquiry begins. This program is a beacon for aspiring scientists and medical professionals dedicated to impactful research and the betterment of community health.

Engage in immersive research alongside esteemed faculty, exploring the depths of medical science and contributing to meaningful advancements. Our curriculum is tailored to foster not just your academic growth but to instill a deep commitment to serving underrepresented communities, aligning with our core mission of equity and access in healthcare.

As you step into this enriching environment, expect a summer filled with discovery, innovation, and opportunities to sharpen your skills for a future in science and medicine. Charles R. Drew University is excited to welcome you to a community of scholars poised to make a difference. Prepare for a transformative experience that extends beyond the laboratory, shaping you into a leader ready to tackle the challenges of tomorrow’s healthcare landscape.

• Office of Research Education – General Program Information
• Office of Research Education – For Third Year Students
• Office of Research Education – For Fourth Year Students

Questions? Contact Us.

This paper is in the following e-collection/theme issue:

Published on 8.4.2024 in Vol 8 (2024)

Development and Implementation of an eHealth Oncohematonootric Program: Descriptive, Observational, Prospective Cohort Pilot Study

Authors of this article:

Original Paper

• Beatriz Sánchez-Quiñones 1, 2 * , MD   ; 
• Cristina Antón-Maldonado 1, 2 * , MD   ; 
• Nataly Ibarra Vega 1, 2 * , MD   ; 
• Isabel Martorell Mariné 3 * , PhD   ; 
• Amparo Santamaria 1, 2 * , MD, PhD  

1 Hybrid Hematology Department, University Hospital Vinalopó, Alicante, Elche, Spain

2 Hematoinnova Unit, Fundación para el Fomento de la Investigación Sanitaria y Biomédica de la Comunidad Valenciana, Valencia, Spain

3 Nutrition Department, Nootric Inc, Barcelona, Spain

*all authors contributed equally

Corresponding Author:

Amparo Santamaria, MD, PhD

Hybrid Hematology Department

University Hospital Vinalopó

Calle Tonico Sansano Mora, 14

Alicante, Elche, 03293

Phone: 34 658981769

Fax:34 965781564

Email: [email protected]

Background: In oncohematology, both the development of the disease and the side effects of antineoplastic treatment often take a toll on patients’ physical and nutritional well-being. In this era of digital transformation, we launched a pioneering project for oncohematologic patients to promote adherence to a healthy lifestyle and improve their physical and nutritional well-being. We aim to achieve this goal by involving doctors and nutritionists through the Nootric app.

Objective: This study aims to assess the impact of the use of eHealth tools to facilitate nutrition and well-being in oncohematologic patients. We also aim to determine the usefulness of physical-nutritional management in improving tolerance to chemotherapy treatments within routine clinical practice.

Methods: We designed a descriptive, observational, longitudinal, prospective cohort pilot study that included a total of 22 patients from March to May 2022 in the Vinalopó University Hospital. The inclusion criteria were adults over 18 years of age diagnosed with oncohematological pathology in active chemotherapy treatment. An action plan was created to generate alerts between the doctor and the nutritionist. In the beginning, the patients were trained to use the app and received education highlighting the importance of nutrition and physical exercise. Sociodemographic, clinical-biological-analytical (eg, malnutrition index), health care impact, usability, and patient adherence data were collected. Tolerance to chemotherapy treatment and its health care impact were evaluated.

Results: We included 22 patients, 11 (50%) female and 11 (50%) male, ranging between 42 and 84 years of age. Among them, 13 (59%) were adherents to the program. The most frequent diseases were lymphoproliferative syndromes (13/22, 59%) and multiple myeloma (4/22, 18%). Moreover, 15 (68%) out of 22 patients received immunochemotherapy, while 7 (32%) out of 22 patients received biological treatment. No worsening of clinical-biological parameters was observed. Excluding dropouts and abandonments (n=9/22, 41%), the adherence rate was 81%, established by calculating the arithmetic mean of the adherence rates of 13 patients. No admission was observed due to gastrointestinal toxicity or discontinuation of treatment related to alterations in physical and nutritional well-being. In addition, only 5.5% of unscheduled consultations were increased due to incidents in well-being, mostly telematic (n=6/103 consultation are unscheduled). Additionally, 92% of patients reported an improvement in their nutritional habits (n=12/13), and up to 45% required adjustment of medical supportive treatment (n=5/11). There were no cases of grade 3 or greater gastrointestinal toxicity. All of this reflects improved tolerance to treatments. Patients reported a satisfaction score of 4.3 out of 5, while professionals rated their satisfaction at 4.8 out of 5.

Conclusions: We demonstrated the usefulness of integrating new technologies through a multidisciplinary approach. The Nootric app facilitated collaboration among the medical team, nutritionists, and patients. It enabled us to detect health issues related to physical-nutritional well-being, anticipate major complications, and mitigate potentially avoidable risks. Consequently, there was a decrease in unscheduled visits and admissions related to this condition.

Introduction

Hematological malignancies encompass a heterogeneous group of diseases that have different behaviors, evolution, treatments, and prognoses. However, all of them similarly compromise the patient’s nutritional and physical status. This is because both the development of the disease and antineoplastic treatment can lead to caloric-protein malnutrition, leading to a high prevalence of adverse effects in daily clinical practice [ 1 ]. New treatments and conventional chemotherapy lead to toxicity in the gastrointestinal tract, which has a direct impact on the patient's well-being and survival [ 2 - 9 ]. Medical management is often insufficient to carry out a comprehensive assessment of the patient’s physical and nutritional well-being. Therefore, professional support in this aspect through the application of information and communication technologies (ICTs) in patients’ everyday environment outside the hospital is a useful tool to improve well-being and reduce health care costs [ 6 , 10 - 12 ]. There is increasing evidence showing that lifestyle interventions can improve symptoms, quality of life, and even overall survival rates for patients with cancer. Digital interventions can help implement physical-nutritional behavior modifications and empower patients through healthy lifestyle education and support [ 13 ].

An effective system for patient physical-nutritional monitoring and treatment after nutritional risk assessment appears to be lacking. We identified the need for a standardized system to prevent and treat malnutrition related to these diseases. Currently, there are studies involving mobile apps for nutritional control and support to monitor dietary intake among patients who are hospitalized and face nutritional risks. These apps have demonstrated good acceptance among patients and have the potential to be useful dietary evaluation tools for use in clinical practice. These results suggest that such tools could be extrapolated to the field of oncohematology consultations [ 14 ].

The studies available so far confirm that the application of mobile apps, among other appropriately designed digital interventions, can be effective tools in nutritional interventions [ 2 , 3 , 15 ]. It is estimated that 59% or more of the currently available apps are health-related [ 2 ]. Some studies show that mobile app interventions can improve the quality of life of patients with malignant hemopathies by reducing symptoms [ 16 ].

However, many of these apps are not developed by nutrition experts, validated by official agencies, or part of routine use in the hospital setting. The use of mobile apps for nutritional interventions to improve dietary patterns, avoid or reduce side effects, and improve patient physical and nutritional well-being is a new challenge currently facing health care.

Therefore, we have launched a pioneering project with the aim of improving and mitigating malnutrition and side effects through proper nutritional and physical well-being monitoring. We aim to deliver this digital nutrition service via the Nootric app, thereby promoting patients’ adherence to a healthy lifestyle.

In this study, we included under the term “oncohematological patient” those who met the inclusion criteria: adults over 18 years of age with a diagnosis of oncohematologic pathology undergoing active treatment. The hematologic malignancies included were mostly lymphoproliferative syndromes and multiple myeloma. We analyzed only nutritional and physical parameters within this group.

We evaluated an intervention designed to support oncohematological patients in active treatment. The primary goal was to assess how eHealth tools in nutrition and well-being management impact patients in oncohematology. Additionally, we aimed to determine the usefulness of physical-nutritional management in improving tolerance to chemotherapy treatments within routine clinical practice.

As secondary objectives, on the one hand, we aimed to evaluate the usefulness of the application of a physical-nutritional intervention among oncohematological patients by observing serial cases. On the other hand, we wanted to evaluate the adequacy and acceptability of this app in this group of patients. We aimed to qualitatively evaluate how knowledge guides the reorientation of intervention strategies regarding physical-nutritional well-being in these patients. Finally, we aimed to understand the nutritional and physical requirements throughout each phase of these patients' treatment, considering the potential implications for their well-being. Measures of engagement with the intervention and semistructured interviews with intervention participants were used to evaluate the feasibility of the intervention.

Study Design

A descriptive, observational, prospective, longitudinal cohort pilot study was conducted among oncohematological patients in our hospital center ( Figure 1 ).

Recruitment took place between April and May 22. The exposure, follow-up, and data collection period lasted 3 months (from May 22 to August 22). The sample consisted of 22 patients and was not divided under any concept at the beginning of the study. During study development, patients who had good adherence were included in the exposed cohort. However, those who dropped out of the study and had a low adherence rate were included in the unexposed cohort. They were not taken into account in the analysis of the results, and no comparative study was performed. To ensure that older adults were not excluded due to the digital gap, we included patients aged above 80 years old, and during the first visit, we encouraged them to continue using the Nootric app.

Study Setting

This study was conducted in the province of Alicante, Spain, at Vialopó University Hospital, which tends to a culturally, linguistically, and socioeconomically diverse population. The participants were recruited from the hospital’s hematology department.

Participants

Patients were eligible to participate if they (1) were aged ≥18 years, (2) had a documented diagnosis of oncohematological pathology, and (3) were receiving medical treatment for cancer at the time of study initiation. Patients were excluded from the study if they (1) were children, adolescents, or pregnant women; (2) were due for surgery with hospital nutritional treatment; (3) were following nutritional care or hospital treatment; (4) had any acute or chronic condition that the practitioner believed limited their ability to participate in the study; (5) were unable to provide written consent; (6) were not literate; and (7) did not have a smartphone.

Intervention: OncohematoNootric Program

In recent years, there has been a growing use of eHealth tools, such as mobile apps, in nutritional interventions with good acceptance and results [ 2 , 3 ]. These technologies have been applied in different health care settings, such as mental health support and chronic disease management, to enable interaction with patients and promote engagement with health care interventions. Ultimately, they aim to increase the acceptability, use, and effectiveness of interventions [ 17 , 18 ]. However, to date, few studies have evaluated the effectiveness of these apps in routine clinical practice within the health care environment. There have been no studies in patients with hematological malignancies undergoing treatment [ 4 ].

The OncohematoNootric program involves a continuous approach and follow-up of patients by their physician and nutritionist through the Nootric app over 3 months via face-to-face/telematic consultations, voice calls, and direct chat through the app.

The intervention aims to provide nutritional-physical support tools to oncohematology patients receiving treatment who may develop adverse effects that put their physical and nutritional well-being at risk. The program provides training and information. It also facilitates risk assessment and clinical support, as needed, with the help of a physician-nutritionist alarm system.

Education Website

Nootric is a digital nutrition service that creates personalized nutrition plans tailored to the oncohematological patient, featuring recipes compatible with the potential side effects of treatment. It augments cognitive-behavioral therapy and provides guides and challenges that address aspects related to nutrition and its application in daily life. Physicians can monitor patients through the app using a dynamic panel that displays real-time actions. It also includes a chat feature to communicate with a dietitian-nutritionist.

In this study, all patients received dietary and exercise recommendations from professionals. The Nootric app aimed to help patients improve their health and well-being by facilitating behavioral changes.

Intervention Development and Patient Involvement

The intervention was designed in conjunction with patients, medical and nutritional health professionals, and professionals with expertise in eHealth and wellness management programs and technologists. No generative artificial intelligence (AI) was used in this study.

For program implementation, a training session on the use of the Nootric app was held with different medical teams. When the candidate patient was selected by the center, the team gave them a patient information sheet, an informed consent form, and an information leaflet. Once the patient agreed to participate in the pilot, they handed over the signed informed consent form and began to participate. The center registered the patient on the Nootric website by entering a code assigned to the patient. Once this registration was completed, the patient downloaded the app and logged in. Next, the patient completed a series of forms that served as a basis for the dietitian-nutritionist to establish their personalized plan. In addition, a clinical-biological test was performed via a blood test requested by the medical team and carried out at the hospital, after which the results were recorded, and the nutritional counseling intervention was initiated. At the beginning of the intervention, the health care professionals oriented the patient on the use of the Nootric application and emphasized the importance of good nutrition and physical exercise. Each patient received a weekly menu and shopping list, was able to upload photos of their meals during follow-up, and had direct access to an app-based chat with a nutritionist. During the study, the patient was able to contact their dietitian-nutritionist through the app’s chat function to solve nutritional doubts and receive motivational support to increase physical activities and food recording.

During the project, improvements were made to the app to provide better patient care, including adapting 70 menu prescriptions to be compatible with the potential side effects of the treatment, configuring menu items to ensure suitability for the most common side effects, and preparing and adapting informative guides and challenges. Other improvements included sending activity and hydration reminders, optimizing the internal messaging system for medical professionals to exchange information with nutritionists, and making functional changes to facilitate uploading files for medical professionals.

An action plan was created to generate alerts between the physician and nutritionist with all the possible adverse events that patients could present (eg, hyporexia, weight loss, skin and nail changes, diarrhea, dyspepsia, pain according to a visual analog scale, edema, fatigue, constipation, dysphagia, odynophagia, mucositis, canker sores, nausea, vomiting, diarrhea, insomnia, urinary and bladder problems, anuria, bleeding, flu-like symptoms, fever, xerostomia, rash, and pruritus), along with their severity criteria and the action plan to be followed by the doctor and nutritionist.

When any of these issues were detected, the nutritionist informed the doctor, who carried out an unscheduled telematic/presential consultation with the patient to resolve the issue with the nutritionist’s support. On the other hand, if the issue was detected by the doctor, the latter informed the nutritionist so that the patient could receive support from both. In this way, both professionals were always kept up to date on incidents and procedures. The communication channel between professionals was the Nootric web platform.

For patients who were observed to have low adherence to the program during the pilot, the professionals studied the potential causes and intensified their actions to avoid dropout.

During the follow-up period, a weekly evaluation of the variables under study was carried out by the professionals. Biweekly follow-up meetings were held with the team to track the program and make possible improvements. Patients completed weekly program evaluation forms. After the follow-up study, a satisfaction survey was conducted to qualitatively evaluate patient satisfaction and program usefulness.

A new clinical-biological test was performed to comparatively analyze the results obtained before and after the intervention ( Figure 2 ).

Study Outcomes

The primary outcome of the study was the health impact of the use of an eHealth tool related to nutrition and physical well-being on the oncohematological patient.This was measured by determining the following variables: nº alerts resolved, nº emergency visits, nº unscheduled consultations, nº treatments suspended, nº hospital admissions and improvement in nutritional habits according to the patient's perception, nº patients referred to the hospital’s Nutrition Unit, nº patients requiring adjustment of support treatment, and nº patients with gastrointestinal toxicity (which determines the impact on improving tolerance to treatments). The secondary outcomes included assessing the perceived improvement in the patients’ physical and nutritional well-being, determined through satisfaction and usefulness questionnaires at the end of the intervention. We also sought to assess the feasibility of the intervention, focusing on usability, acceptability, and adherence regarding different intervention components.

Data Collection and Study Procedures

The variables were collected on a form based on Microsoft Office Excel 2021 (Microsoft Corp), with each coded and subsequently exported to the SPSS statistical software (version 28.0; IBM Corp). To describe continuous variables with normal distribution, measures of central tendency were used, such as arithmetic mean. The qualitative variables were presented as frequency and proportion.

The variables collected at the beginning of the pilot study, during follow-up, and at the end were: (1) sociodemographic, including sex, age, level of education, main disease, comorbidities, and medication; (2) clinical-biological-analytical, with the following parameters included in the blood analysis to assess nutritional status and treatment toxicity and direct the professionals’ action plan: hemoglobin, creatinine clearance, calcium, total protein, albumin, vitamin B12, folic acid, ferritin, malnutrition index, total cholesterol, liver enzymes aspartate aminotransferase, alanine aminotransferase, alkaline phosphatase, and gamma-glutamyl transferase, cancer medical treatment scheme, food consumption patterns measured through the form provided by Nootric through the app, and BMI; (3) health care impact; (4) usability; and (5) adherence.

To weight the usability of the application, the following user interaction variables were considered: viewing, rating, comments on recipes, uploaded photos, access to the chat, review of a guide or completion of a challenge (framed in cognitive-behavioral training), points achieved (result of the gamification provided by Nootric), and exercises displayed. Adherence was determined by analyzing each patient’s use of the Nootric app during the program application period. For adherence, the percentage of access to the Nootric application was calculated, considering the 120-day access as 100% adherence.

The evaluations performed can be viewed in more detail in Tables 1 and 2 and Figures 3 - 5 . The data extracted from the patients' medical records included results of the blood analysis, main disease, comorbidities and their medication, cancer medical treatment scheme, and health care impact variables.

a Patients received medical treatment, physical-nutritional recommendations, and close monitoring by their multidisciplinary assistance program. Most of the study population consisted of patients without severe comorbidities (18/22, 80%).

b These include rituximab, cyclophosphamide, doxorubicin, vincristine, and prednisone, (R-CHOP); adriamycin, bleomycin, vinblastine, and dacarbazine (ABVD); rituximab, bortezomib, cyclophosphamide, adriamycin, and prednisone (VR-CAP); and methotrexate, regimens with melphalan, and 5-azacitidine.

c Therapeutic regimens included daratumumab, bortezomib, venetoclax, obinutuzumab, ibrutinib, imatinib, and nilotinib.

d These included 3 women and 3 men. Dropouts were considered as users with adherence rates below 7%, and they were not accounted for in the analytical results, body variables, and interactions with the app.

Ethical Considerations

This study was approved by the Ethics Committee of Vinalopó University Hospital on March 30, 2022. All research activities involving human patients in this study have been treated in accordance with the ethical guidelines established by The Organic Law on Data Protection. All necessary approvals were obtained, including for the analysis of the research data.

This study complies with the ethical provisions outlined in the informed consent, and any additional analysis has been conducted in accordance with the existing ethical approvals. Informed consent was obtained from all patients for the conduct of this study and publication of this article, and no compensation was provided. Patients were informed that the doctor would receive information about their progress.

To protect the privacy and confidentiality of the participants, all data collected in this study were anonymized before the analysis. Measures were taken to ensure that the participants’ identifiable details were not disclosed. We used an anonymous identification system that consisted of assigning an alphanumeric code to each patient registered on the Nootric web platform and in the Nootric app. No one else, apart from Ribera's medical team and Nootric's team of nutritionists, had access to the data in the Nootric app and website, which are confidential and encrypted.

The original informed consent allows for secondary analysis without additional consent; this includes data collected from the participants’ medical history.

Sociodemographic Variables

Of the 22 included patients, 11 (50%) were female and 11 (50%) were male, with an average age of 70 (range 42-84) years. Among them, 13 (60%) patients were under 65 years and 9 (40%) were over 65 years. The variables regarding sociodemographic characteristics, treatment received during the study, and program adherence are described in Table 1 .

Clinical-Biological-Analytical Variables

A clinical-biological test was conducted for all patients at the beginning of the study, but only 13 (59%) out of 22 patients completed it. On this population, we performed the analysis of the results. Table 2 presents all the analyzed laboratory parameters and the results at the beginning and end of the

intervention. Some of the relevant data are shown in Figure 3 . Some patients received corticosteroid therapy, which increases the risk of developing steroidal hyperglycemia. These patients benefited from an adapted physical-nutritional plan. There were no patients in the study who presented with alterations in blood glucose levels. Of the 2 (15%) patients who had dyslipidemia at baseline, 1 (8%) patient did not maintain dyslipidemia at the end of the study. Furthermore, 1 (8%) patient presented with iron overload secondary to a high transfusion requirement. Iron chelation therapy was initiated, which triggered a grade 3 hepatotoxicity. After discontinuing it, it improved clinically and analytically to grade 1. Of 6 (46%) patients with malnutrition at baseline, only 1 (8%) patient still had malnutrition at the end of the study. Of the 4 (30%) patients who presented with hypoproteinemia at baseline, 1 (23%) did not have it at the end. Moreover, 1 patient (8%) presented with a folic deficit at baseline and maintained it at baseline despite having received supportive treatment and dietary recommendations.

The clinical interview confirmed that treatment compliance was inadequate. Among 13 patients who had anemia at the beginning of the study, 3 (23%) did not maintain it at the end of the study. Conversely, 3 (23%) of the 13 patients who had anemia at the end of the study did not have it at baseline. None of the causes were caused by vitamin B12, folic acid, or iron deficiency but by myelotoxicity due to targeted cancer therapy.

Program Adherence and Usability

A total of 6 (27%) out of 22 patients abandoned the study due to a lack of adherence to the program, attributed to advanced age, insufficient socioeconomic level to ensure proper use of the app, lack of family support for improving adherence, and lack of initiative to establish a change in the physical-nutritional routine. They were not taken into account in the analysis of the results. In addition, users with less than 7% adherence were considered dropouts (3/22, 14%), and they were not taken into account in the analysis of the results. None of these patients were older adults or dropped out due to the digital gap.

Excluding dropouts and abandonments (n=9, 30%), the adherence rate was 81%, established by calculating the arithmetic mean of the adherence rates of 13 patients, much higher than the average rate in well-being. Of the 22 patients, between 7 and 11 (30%-50%) perceived an improvement in well-being, determined by the satisfaction survey conducted at the end of the study. To determine the adherence rate of each patient, the use of the app by the patients was evaluated and the parameters were nº interactions with the app, use of the chat with the nutritionist, nº interactions with the recipes, nº photos of recipes uploaded by the patients, and gamification points earned ( Figure 4 ). Regarding the impact of usability, we obtained an average of 655 impacts per app user ( Figure 4 ).

Impact on Health Care

Regarding the impact on health care quality, Table 2 and Figure 5 show data that demonstrates highly satisfactory results. None of the total emergency visits were related to physical and nutritional well-being.

No hospitalizations occurred during the study for any cause, including those related to physical and nutritional well-being. No patient had to be referred to the hospital’s Nutrition Unit. Of the total number of medical consultations carried out, only 5.5% (6/103) were unscheduled and none of them were carried out for physical or nutritional issues. A total of 7 patients presented toxicity, among which 5 (71%) were cases of digestive toxicity. Of the 11 patients who required adjustment of supportive or symptomatic treatment due to toxicity, 5 (45%) had digestive toxicity. No treatment was suspended due to physical or nutritional conditions.

The patients showed an improvement in tolerance to chemotherapy treatments since there were no cases of grade 3 or higher gastrointestinal toxicity, defined as complications requiring intravenous support treatment. There were also no hospitalizations, emergency visits, or chemotherapy treatment discontinuations for this reason. A high percentage of patients (12/13, 92%) perceived an improvement in their nutritional habits.

Impact on Patients’ Perceived Improvements in Physical and Nutritional Well-Being

At the end of the study, the Nootric team disseminated an anonymous survey to measure satisfaction and usefulness for the patients and medical professionals involved. This enabled us to evaluate the impact on the perceived improvement in the patients’ physical and nutritional well-being. We observed that the users who adhered adequately to the program showed an improvement in this aspect. A total of 12 questionnaires were filled out by patients.

The average satisfaction rating among professionals was 4.8 out of 5, while patients rated their satisfaction at 4.3 out of 5. Table 3 highlights these results.

Our study is based on a multidisciplinary nutritional and exercise support program for oncohematological patients undergoing active treatment using new technologies. This study provides initial data on the effectiveness of a novel physical and nutritional support program aimed at patients with malignant hemopathies (largely represented in our study as lymphoproliferative syndromes and multiple myeloma) receiving targeted cancer treatment. It also provides a detailed evaluation of the implementation, adoption, and overall acceptability of this digital care intervention through a mobile app. The evaluation design has been adapted to the study objectives to provide new data to enable a better estimation of such an intervention’s impact and inform further development of digital care interventions for malignant blood diseases under active treatment.

Principal Findings

We did not observe any hospital admissions or discontinuation of chemotherapy treatment related to the patients' physical-nutritional well-being, supporting the benefit of the program. Adequate nutritional support was provided to ensure patients' well-being and mitigate the need for referral to the hospital's Nutrition Unit. Regarding the impact on physical and nutritional well-being, we observed that users who adhered adequately to the program improved in this aspect.

We observed a reduction in the number of unplanned consultations related to physical and nutritional well-being. In terms of impact on health care quality, the results demonstrated highly satisfactory results. None of the total emergency visits were related to physical and nutritional well-being. Moreover, the intervention received a high satisfaction rating from both professionals and patients.

Regarding other works in the field, there is little information about the best nutritional support for patients with cancer [ 9 , 19 , 20 ]. Antineoplastic agents are known to be associated with gastrointestinal complications that lead to physical and nutritional repercussions, which can decrease well-being and result in death due to malnutrition [ 19 ]. Additionally, early nutritional intervention can improve prognosis and reduce the disease’s complication rate [ 12 , 19 ].

One study used a novel mobile app to assess and evaluate dietary behaviors in 39 oncologic patients. Although 5 patients dropped out prior to the study, the authors concluded that participants who tracked their daily dietary habits using a mobile phone app were more likely to reach their nutritional goals than the control patients. Other studies have used mobile apps to record nutritional status and activity levels in patients with breast cancer or other or other diseases, but none of them are similar to our study [ 20 ]. Our study was performed by a multidisciplinary team using both the app and the internet to maintain contact with the patients. Furthermore, the multidisciplinary team tailored each patient's diet to suit their individual needs throughout their cancer treatment journey, particularly addressing gastrointestinal toxicities associated with active chemotherapy. This underscores the effectiveness of such technologies for integration into clinical practice without compromising the human touch in health care delivery.

Limitations and Strengths

This study highlights the importance of eHealth programs in addressing nutrition and

well-being among oncohematology patients, offering significant value in multidisciplinary care management. The use of the Nootric app allowed for improved health care indicators and physical-nutritional well-being, promoting better patient outcomes.

Another notable strength of this study is the finding that over 50% (n=11) of the patients improved their physical-nutritional habits, leading to a considerable enhancement in their perception of well-being.

In terms of limitations, we must point out that this study has a small sample size of 22 patients, which may limit the generalizability of the results. Moreover, we experienced a 27% (6/22) dropout rate due to a lack of adherence to the program, which could have affected the overall results. In addition, not all of those who completed the study completed the clinical-biological tests. Finally, we acknowledge that the 12-week follow-up period might not adequately capture the program’s clinical impact on adherence to healthy habits and improved physical and nutritional well-being.

In conclusion, using targeted eHealth programs for nutrition and well-being among oncohematological patients undergoing active treatment offers significant value in multidisciplinary care management. This is achieved through enhanced interaction between physicians, dietitian-nutritionists, and patients via a digital nutrition service, such as the Nootric app. Supporting patients throughout their cancer journey, these technologies serve as valuable tools for integration into clinical practice without detracting from the human aspect of health care. Therefore, implementing projects that leverage new technologies in routine holistic clinical practice for oncohematological patients could prove cost-effective in both the short and long term. By facilitating the early detection of health issues related to physical-nutritional well-being and anticipating potential complications, these initiatives may help reduce unscheduled visits and admissions related to this condition.

Acknowledgments

We extend our gratitude to all the patients and families involved in this study.

Data Availability

Our data adheres to open science availability guidelines for broader research accessibility.

Conflicts of Interest

None declared.

• Martín Salces M, de Paz R, Hernández-Navarro F. Therapeutic recommendations in the oncohematological patient. Nutr Hosp. 2006;21(3):379-385. [ Medline ]
• Lee EY, Cha SA, Yun JS, Lim SY, Lee JH, Ahn YB, et al. Efficacy of personalized diabetes self-care using an electronic medical record-integrated mobile app in patients with type 2 diabetes: 6-month randomized controlled trial. J Med Internet Res. Jul 28, 2022;24(7):e37430. [ FREE Full text ] [ CrossRef ] [ Medline ]
• San Mauro Martín I, González Fernández M, Collado Yurrita L. Mobile applications for nutrition, dietetics and healthy habits; analysis and consequences of an increasing trend. Nutr Hosp. Jul 01, 2014;30(1):15-24. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sandín-Vázquez M, Sarría-Santamera A. Health Impact Assessment: assessing the effectiveness of policies in population health. Rev Esp Salud Publica. Jun 2008;82(3):261-272. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferlay J, Colombet M, Soerjomataram I, Parkin DM, Piñeros M, Znaor A, et al. Cancer statistics for the year 2020: An overview. Int J Cancer. Apr 05, 2021.:778-789. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guest DD, Cox T, Voss AC, Kelley K, Ma X, Nguyen A, et al. Assessing impact of nutrition care by registered dietitian nutritionists on patient medical and treatment outcomes in outpatient cancer clinics: a cohort feasibility study. Nutr Cancer. 2023;75(3):923-936. [ CrossRef ] [ Medline ]
• Planas M, Fernández-Ortega JF, Abilés J, Metabolism Nutrition Working Group of the Spanish Society of Intensive Care MedicineCoronary units. Guidelines for specialized nutritional and metabolic support in the critically-ill patient: update. Consensus SEMICYUC-SENPE: oncohematological patient. Nutr Hosp. Nov 2011;26 Suppl 2:50-53. [ CrossRef ] [ Medline ]
• Saigí Ullastre I, Pérez Pérez A. Hiperglucemia inducida por glucocorticoides. Semin de la Fund. Espanola de Reumatol. Jul 2011;12(3):83-90. [ CrossRef ] [ Medline ]
• Alonso Domínguez T, Civera Andrés M, Santiago Crespo JA, García Malpartida K, Botella Romero F. Digestive toxicity in cancer treatments. Bibliographic review. Influence on nutritional status. Endocrinol Diabetes Nutr (Engl Ed). Feb 2023;70(2):136-150. [ CrossRef ] [ Medline ]
• Kuijpers W, Groen WG, Aaronson NK, van Harten WH. A systematic review of web-based interventions for patient empowerment and physical activity in chronic diseases: relevance for cancer survivors. J Med Internet Res. Feb 20, 2013;15(2):e37. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Singleton AC, Raeside R, Hyun KK, Partridge SR, Di Tanna GL, Hafiz N, et al. Electronic health interventions for patients with breast cancer: systematic review and meta-analyses. J Clin Oncol. Jul 10, 2022;40(20):2257-2270. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lazar DE, Postolica R, Hanganu B, Mocanu V, Ioan BG. Web-based nutrition: a useful resource for cancer patients? Front Nutr. 2023;10:1134793. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gudmundsson GH, Mészáros J, Björnsdóttir AE, Ámundadóttir ML, Thorvardardottir GE, Magnusdottir E, et al. Evaluating the feasibility of a digital therapeutic program for patients with cancer during active treatment: pre-post interventional study. JMIR Form Res. Oct 13, 2022;6(10):e39764. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Paulsen MM, Hagen MLL, Frøyen MH, Foss-Pedersen RJ, Bergsager D, Tangvik RJ, et al. A dietary assessment app for hospitalized patients at nutritional risk: development and evaluation of the MyFood app. JMIR Mhealth Uhealth. Sep 07, 2018;6(9):e175. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Karimi N, Opie R, Crawford D, O'Connell S, Ball K. Digitally delivered interventions to improve nutrition behaviors among resource-poor and ethnic minority groups with type 2 diabetes: systematic review. J Med Internet Res. Feb 01, 2024;26:e42595. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Win H, Russell S, Wertheim BC, Maizes V, Crocker R, Brooks AJ, et al. Mobile app intervention on reducing the myeloproliferative neoplasm symptom burden: pilot feasibility and acceptability study. JMIR Form Res. Mar 31, 2022;6(3):e33581. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Franco-Carrero JA, Dominguez Padilla M, Domínguez-Fernandez JM. El uso de aplicaciones móviles como herramienta para abordar el estrés en trabajadores: revisión bibliográfica. Med Segur Trab. Mar 30, 2023;69(270):49-60. [ CrossRef ]
• Quevedo Rodríguez A, Wägner AM. Mobile phone applications for diabetes management: A systematic review. Endocrinol Diabetes Nutr (Engl Ed). May 2019;66(5):330-337. [ CrossRef ] [ Medline ]
• Molina Villaverde R. El paciente oncológico del siglo xxi. Maridaje terapéutico Nutrición-Oncología. Nutr Hosp. Jun 03, 2016;33(Suppl 1):174. [ CrossRef ] [ Medline ]
• Orlemann T, Reljic D, Zenker B, Meyer J, Eskofier B, Thiemt J, et al. A novel mobile phone app (OncoFood) to record and optimize the dietary behavior of oncologic patients: pilot study. JMIR Cancer. Nov 20, 2018;4(2):e10703. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani; submitted 03.06.23; peer-reviewed by C Herrero; comments to author 02.10.23; revised version received 13.10.23; accepted 14.02.24; published 08.04.24.

©Beatriz Sánchez-Quiñones, Cristina Antón-Maldonado, Nataly Ibarra Vega, Isabel Martorell Mariné, Amparo Santamaria. Originally published in JMIR Formative Research (https://formative.jmir.org), 08.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

This paper is in the following e-collection/theme issue:

Published on 8.4.2024 in Vol 10 (2024)

Importance of Patient History in Artificial Intelligence–Assisted Medical Diagnosis: Comparison Study

Authors of this article:

There are no citations yet available for this article according to Crossref .