qualitative research meaning medical

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Prev Med

Qualitative Methods in Health Care Research

Vishnu renjith.

School of Nursing and Midwifery, Royal College of Surgeons Ireland - Bahrain (RCSI Bahrain), Al Sayh Muharraq Governorate, Bahrain

Renjulal Yesodharan

1 Department of Mental Health Nursing, Manipal College of Nursing Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Judith A. Noronha

2 Department of OBG Nursing, Manipal College of Nursing Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Elissa Ladd

3 School of Nursing, MGH Institute of Health Professions, Boston, USA

Anice George

4 Department of Child Health Nursing, Manipal College of Nursing Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Healthcare research is a systematic inquiry intended to generate robust evidence about important issues in the fields of medicine and healthcare. Qualitative research has ample possibilities within the arena of healthcare research. This article aims to inform healthcare professionals regarding qualitative research, its significance, and applicability in the field of healthcare. A wide variety of phenomena that cannot be explained using the quantitative approach can be explored and conveyed using a qualitative method. The major types of qualitative research designs are narrative research, phenomenological research, grounded theory research, ethnographic research, historical research, and case study research. The greatest strength of the qualitative research approach lies in the richness and depth of the healthcare exploration and description it makes. In health research, these methods are considered as the most humanistic and person-centered way of discovering and uncovering thoughts and actions of human beings.

Introduction

Healthcare research is a systematic inquiry intended to generate trustworthy evidence about issues in the field of medicine and healthcare. The three principal approaches to health research are the quantitative, the qualitative, and the mixed methods approach. The quantitative research method uses data, which are measures of values and counts and are often described using statistical methods which in turn aids the researcher to draw inferences. Qualitative research incorporates the recording, interpreting, and analyzing of non-numeric data with an attempt to uncover the deeper meanings of human experiences and behaviors. Mixed methods research, the third methodological approach, involves collection and analysis of both qualitative and quantitative information with an objective to solve different but related questions, or at times the same questions.[ 1 , 2 ]

In healthcare, qualitative research is widely used to understand patterns of health behaviors, describe lived experiences, develop behavioral theories, explore healthcare needs, and design interventions.[ 1 , 2 , 3 ] Because of its ample applications in healthcare, there has been a tremendous increase in the number of health research studies undertaken using qualitative methodology.[ 4 , 5 ] This article discusses qualitative research methods, their significance, and applicability in the arena of healthcare.

Qualitative Research

Diverse academic and non-academic disciplines utilize qualitative research as a method of inquiry to understand human behavior and experiences.[ 6 , 7 ] According to Munhall, “Qualitative research involves broadly stated questions about human experiences and realities, studied through sustained contact with the individual in their natural environments and producing rich, descriptive data that will help us to understand those individual's experiences.”[ 8 ]

Significance of Qualitative Research

The qualitative method of inquiry examines the 'how' and 'why' of decision making, rather than the 'when,' 'what,' and 'where.'[ 7 ] Unlike quantitative methods, the objective of qualitative inquiry is to explore, narrate, and explain the phenomena and make sense of the complex reality. Health interventions, explanatory health models, and medical-social theories could be developed as an outcome of qualitative research.[ 9 ] Understanding the richness and complexity of human behavior is the crux of qualitative research.

Differences between Quantitative and Qualitative Research

The quantitative and qualitative forms of inquiry vary based on their underlying objectives. They are in no way opposed to each other; instead, these two methods are like two sides of a coin. The critical differences between quantitative and qualitative research are summarized in Table 1 .[ 1 , 10 , 11 ]

Differences between quantitative and qualitative research

Qualitative Research Questions and Purpose Statements

Qualitative questions are exploratory and are open-ended. A well-formulated study question forms the basis for developing a protocol, guides the selection of design, and data collection methods. Qualitative research questions generally involve two parts, a central question and related subquestions. The central question is directed towards the primary phenomenon under study, whereas the subquestions explore the subareas of focus. It is advised not to have more than five to seven subquestions. A commonly used framework for designing a qualitative research question is the 'PCO framework' wherein, P stands for the population under study, C stands for the context of exploration, and O stands for the outcome/s of interest.[ 12 ] The PCO framework guides researchers in crafting a focused study question.

Example: In the question, “What are the experiences of mothers on parenting children with Thalassemia?”, the population is “mothers of children with Thalassemia,” the context is “parenting children with Thalassemia,” and the outcome of interest is “experiences.”

The purpose statement specifies the broad focus of the study, identifies the approach, and provides direction for the overall goal of the study. The major components of a purpose statement include the central phenomenon under investigation, the study design and the population of interest. Qualitative research does not require a-priori hypothesis.[ 13 , 14 , 15 ]

Example: Borimnejad et al . undertook a qualitative research on the lived experiences of women suffering from vitiligo. The purpose of this study was, “to explore lived experiences of women suffering from vitiligo using a hermeneutic phenomenological approach.” [ 16 ]

Review of the Literature

In quantitative research, the researchers do an extensive review of scientific literature prior to the commencement of the study. However, in qualitative research, only a minimal literature search is conducted at the beginning of the study. This is to ensure that the researcher is not influenced by the existing understanding of the phenomenon under the study. The minimal literature review will help the researchers to avoid the conceptual pollution of the phenomenon being studied. Nonetheless, an extensive review of the literature is conducted after data collection and analysis.[ 15 ]

Reflexivity

Reflexivity refers to critical self-appraisal about one's own biases, values, preferences, and preconceptions about the phenomenon under investigation. Maintaining a reflexive diary/journal is a widely recognized way to foster reflexivity. According to Creswell, “Reflexivity increases the credibility of the study by enhancing more neutral interpretations.”[ 7 ]

Types of Qualitative Research Designs

The qualitative research approach encompasses a wide array of research designs. The words such as types, traditions, designs, strategies of inquiry, varieties, and methods are used interchangeably. The major types of qualitative research designs are narrative research, phenomenological research, grounded theory research, ethnographic research, historical research, and case study research.[ 1 , 7 , 10 ]

Narrative research

Narrative research focuses on exploring the life of an individual and is ideally suited to tell the stories of individual experiences.[ 17 ] The purpose of narrative research is to utilize 'story telling' as a method in communicating an individual's experience to a larger audience.[ 18 ] The roots of narrative inquiry extend to humanities including anthropology, literature, psychology, education, history, and sociology. Narrative research encompasses the study of individual experiences and learning the significance of those experiences. The data collection procedures include mainly interviews, field notes, letters, photographs, diaries, and documents collected from one or more individuals. Data analysis involves the analysis of the stories or experiences through “re-storying of stories” and developing themes usually in chronological order of events. Rolls and Payne argued that narrative research is a valuable approach in health care research, to gain deeper insight into patient's experiences.[ 19 ]

Example: Karlsson et al . undertook a narrative inquiry to “explore how people with Alzheimer's disease present their life story.” Data were collected from nine participants. They were asked to describe about their life experiences from childhood to adulthood, then to current life and their views about the future life. [ 20 ]

Phenomenological research

Phenomenology is a philosophical tradition developed by German philosopher Edmond Husserl. His student Martin Heidegger did further developments in this methodology. It defines the 'essence' of individual's experiences regarding a certain phenomenon.[ 1 ] The methodology has its origin from philosophy, psychology, and education. The purpose of qualitative research is to understand the people's everyday life experiences and reduce it into the central meaning or the 'essence of the experience'.[ 21 , 22 ] The unit of analysis of phenomenology is the individuals who have had similar experiences of the phenomenon. Interviews with individuals are mainly considered for the data collection, though, documents and observations are also useful. Data analysis includes identification of significant meaning elements, textural description (what was experienced), structural description (how was it experienced), and description of 'essence' of experience.[ 1 , 7 , 21 ] The phenomenological approach is further divided into descriptive and interpretive phenomenology. Descriptive phenomenology focuses on the understanding of the essence of experiences and is best suited in situations that need to describe the lived phenomenon. Hermeneutic phenomenology or Interpretive phenomenology moves beyond the description to uncover the meanings that are not explicitly evident. The researcher tries to interpret the phenomenon, based on their judgment rather than just describing it.[ 7 , 21 , 22 , 23 , 24 ]

Example: A phenomenological study conducted by Cornelio et al . aimed at describing the lived experiences of mothers in parenting children with leukemia. Data from ten mothers were collected using in-depth semi-structured interviews and were analyzed using Husserl's method of phenomenology. Themes such as “pivotal moment in life”, “the experience of being with a seriously ill child”, “having to keep distance with the relatives”, “overcoming the financial and social commitments”, “responding to challenges”, “experience of faith as being key to survival”, “health concerns of the present and future”, and “optimism” were derived. The researchers reported the essence of the study as “chronic illness such as leukemia in children results in a negative impact on the child and on the mother.” [ 25 ]

Grounded Theory Research

Grounded theory has its base in sociology and propagated by two sociologists, Barney Glaser, and Anselm Strauss.[ 26 ] The primary purpose of grounded theory is to discover or generate theory in the context of the social process being studied. The major difference between grounded theory and other approaches lies in its emphasis on theory generation and development. The name grounded theory comes from its ability to induce a theory grounded in the reality of study participants.[ 7 , 27 ] Data collection in grounded theory research involves recording interviews from many individuals until data saturation. Constant comparative analysis, theoretical sampling, theoretical coding, and theoretical saturation are unique features of grounded theory research.[ 26 , 27 , 28 ] Data analysis includes analyzing data through 'open coding,' 'axial coding,' and 'selective coding.'[ 1 , 7 ] Open coding is the first level of abstraction, and it refers to the creation of a broad initial range of categories, axial coding is the procedure of understanding connections between the open codes, whereas selective coding relates to the process of connecting the axial codes to formulate a theory.[ 1 , 7 ] Results of the grounded theory analysis are supplemented with a visual representation of major constructs usually in the form of flow charts or framework diagrams. Quotations from the participants are used in a supportive capacity to substantiate the findings. Strauss and Corbin highlights that “the value of the grounded theory lies not only in its ability to generate a theory but also to ground that theory in the data.”[ 27 ]

Example: Williams et al . conducted a grounded theory research to explore the nature of relationship between the sense of self and the eating disorders. Data were collected form 11 women with a lifetime history of Anorexia Nervosa and were analyzed using the grounded theory methodology. Analysis led to the development of a theoretical framework on the nature of the relationship between the self and Anorexia Nervosa. [ 29 ]

Ethnographic research

Ethnography has its base in anthropology, where the anthropologists used it for understanding the culture-specific knowledge and behaviors. In health sciences research, ethnography focuses on narrating and interpreting the health behaviors of a culture-sharing group. 'Culture-sharing group' in an ethnography represents any 'group of people who share common meanings, customs or experiences.' In health research, it could be a group of physicians working in rural care, a group of medical students, or it could be a group of patients who receive home-based rehabilitation. To understand the cultural patterns, researchers primarily observe the individuals or group of individuals for a prolonged period of time.[ 1 , 7 , 30 ] The scope of ethnography can be broad or narrow depending on the aim. The study of more general cultural groups is termed as macro-ethnography, whereas micro-ethnography focuses on more narrowly defined cultures. Ethnography is usually conducted in a single setting. Ethnographers collect data using a variety of methods such as observation, interviews, audio-video records, and document reviews. A written report includes a detailed description of the culture sharing group with emic and etic perspectives. When the researcher reports the views of the participants it is called emic perspectives and when the researcher reports his or her views about the culture, the term is called etic.[ 7 ]

Example: The aim of the ethnographic study by LeBaron et al . was to explore the barriers to opioid availability and cancer pain management in India. The researchers collected data from fifty-nine participants using in-depth semi-structured interviews, participant observation, and document review. The researchers identified significant barriers by open coding and thematic analysis of the formal interview. [ 31 ]

Historical research

Historical research is the “systematic collection, critical evaluation, and interpretation of historical evidence”.[ 1 ] The purpose of historical research is to gain insights from the past and involves interpreting past events in the light of the present. The data for historical research are usually collected from primary and secondary sources. The primary source mainly includes diaries, first hand information, and writings. The secondary sources are textbooks, newspapers, second or third-hand accounts of historical events and medical/legal documents. The data gathered from these various sources are synthesized and reported as biographical narratives or developmental perspectives in chronological order. The ideas are interpreted in terms of the historical context and significance. The written report describes 'what happened', 'how it happened', 'why it happened', and its significance and implications to current clinical practice.[ 1 , 10 ]

Example: Lubold (2019) analyzed the breastfeeding trends in three countries (Sweden, Ireland, and the United States) using a historical qualitative method. Through analysis of historical data, the researcher found that strong family policies, adherence to international recommendations and adoption of baby-friendly hospital initiative could greatly enhance the breastfeeding rates. [ 32 ]

Case study research

Case study research focuses on the description and in-depth analysis of the case(s) or issues illustrated by the case(s). The design has its origin from psychology, law, and medicine. Case studies are best suited for the understanding of case(s), thus reducing the unit of analysis into studying an event, a program, an activity or an illness. Observations, one to one interviews, artifacts, and documents are used for collecting the data, and the analysis is done through the description of the case. From this, themes and cross-case themes are derived. A written case study report includes a detailed description of one or more cases.[ 7 , 10 ]

Example: Perceptions of poststroke sexuality in a woman of childbearing age was explored using a qualitative case study approach by Beal and Millenbrunch. Semi structured interview was conducted with a 36- year mother of two children with a history of Acute ischemic stroke. The data were analyzed using an inductive approach. The authors concluded that “stroke during childbearing years may affect a woman's perception of herself as a sexual being and her ability to carry out gender roles”. [ 33 ]

Sampling in Qualitative Research

Qualitative researchers widely use non-probability sampling techniques such as purposive sampling, convenience sampling, quota sampling, snowball sampling, homogeneous sampling, maximum variation sampling, extreme (deviant) case sampling, typical case sampling, and intensity sampling. The selection of a sampling technique depends on the nature and needs of the study.[ 34 , 35 , 36 , 37 , 38 , 39 , 40 ] The four widely used sampling techniques are convenience sampling, purposive sampling, snowball sampling, and intensity sampling.

Convenience sampling

It is otherwise called accidental sampling, where the researchers collect data from the subjects who are selected based on accessibility, geographical proximity, ease, speed, and or low cost.[ 34 ] Convenience sampling offers a significant benefit of convenience but often accompanies the issues of sample representation.

Purposive sampling

Purposive or purposeful sampling is a widely used sampling technique.[ 35 ] It involves identifying a population based on already established sampling criteria and then selecting subjects who fulfill that criteria to increase the credibility. However, choosing information-rich cases is the key to determine the power and logic of purposive sampling in a qualitative study.[ 1 ]

Snowball sampling

The method is also known as 'chain referral sampling' or 'network sampling.' The sampling starts by having a few initial participants, and the researcher relies on these early participants to identify additional study participants. It is best adopted when the researcher wishes to study the stigmatized group, or in cases, where findings of participants are likely to be difficult by ordinary means. Respondent ridden sampling is an improvised version of snowball sampling used to find out the participant from a hard-to-find or hard-to-study population.[ 37 , 38 ]

Intensity sampling

The process of identifying information-rich cases that manifest the phenomenon of interest is referred to as intensity sampling. It requires prior information, and considerable judgment about the phenomenon of interest and the researcher should do some preliminary investigations to determine the nature of the variation. Intensity sampling will be done once the researcher identifies the variation across the cases (extreme, average and intense) and picks the intense cases from them.[ 40 ]

Deciding the Sample Size

A-priori sample size calculation is not undertaken in the case of qualitative research. Researchers collect the data from as many participants as possible until they reach the point of data saturation. Data saturation or the point of redundancy is the stage where the researcher no longer sees or hears any new information. Data saturation gives the idea that the researcher has captured all possible information about the phenomenon of interest. Since no further information is being uncovered as redundancy is achieved, at this point the data collection can be stopped. The objective here is to get an overall picture of the chronicle of the phenomenon under the study rather than generalization.[ 1 , 7 , 41 ]

Data Collection in Qualitative Research

The various strategies used for data collection in qualitative research includes in-depth interviews (individual or group), focus group discussions (FGDs), participant observation, narrative life history, document analysis, audio materials, videos or video footage, text analysis, and simple observation. Among all these, the three popular methods are the FGDs, one to one in-depth interviews and the participant observation.

FGDs are useful in eliciting data from a group of individuals. They are normally built around a specific topic and are considered as the best approach to gather data on an entire range of responses to a topic.[ 42 Group size in an FGD ranges from 6 to 12. Depending upon the nature of participants, FGDs could be homogeneous or heterogeneous.[ 1 , 14 ] One to one in-depth interviews are best suited to obtain individuals' life histories, lived experiences, perceptions, and views, particularly while exporting topics of sensitive nature. In-depth interviews can be structured, unstructured, or semi-structured. However, semi-structured interviews are widely used in qualitative research. Participant observations are suitable for gathering data regarding naturally occurring behaviors.[ 1 ]

Data Analysis in Qualitative Research

Various strategies are employed by researchers to analyze data in qualitative research. Data analytic strategies differ according to the type of inquiry. A general content analysis approach is described herewith. Data analysis begins by transcription of the interview data. The researcher carefully reads data and gets a sense of the whole. Once the researcher is familiarized with the data, the researcher strives to identify small meaning units called the 'codes.' The codes are then grouped based on their shared concepts to form the primary categories. Based on the relationship between the primary categories, they are then clustered into secondary categories. The next step involves the identification of themes and interpretation to make meaning out of data. In the results section of the manuscript, the researcher describes the key findings/themes that emerged. The themes can be supported by participants' quotes. The analytical framework used should be explained in sufficient detail, and the analytic framework must be well referenced. The study findings are usually represented in a schematic form for better conceptualization.[ 1 , 7 ] Even though the overall analytical process remains the same across different qualitative designs, each design such as phenomenology, ethnography, and grounded theory has design specific analytical procedures, the details of which are out of the scope of this article.

Computer-Assisted Qualitative Data Analysis Software (CAQDAS)

Until recently, qualitative analysis was done either manually or with the help of a spreadsheet application. Currently, there are various software programs available which aid researchers to manage qualitative data. CAQDAS is basically data management tools and cannot analyze the qualitative data as it lacks the ability to think, reflect, and conceptualize. Nonetheless, CAQDAS helps researchers to manage, shape, and make sense of unstructured information. Open Code, MAXQDA, NVivo, Atlas.ti, and Hyper Research are some of the widely used qualitative data analysis software.[ 14 , 43 ]

Reporting Guidelines

Consolidated Criteria for Reporting Qualitative Research (COREQ) is the widely used reporting guideline for qualitative research. This 32-item checklist assists researchers in reporting all the major aspects related to the study. The three major domains of COREQ are the 'research team and reflexivity', 'study design', and 'analysis and findings'.[ 44 , 45 ]

Critical Appraisal of Qualitative Research

Various scales are available to critical appraisal of qualitative research. The widely used one is the Critical Appraisal Skills Program (CASP) Qualitative Checklist developed by CASP network, UK. This 10-item checklist evaluates the quality of the study under areas such as aims, methodology, research design, ethical considerations, data collection, data analysis, and findings.[ 46 ]

Ethical Issues in Qualitative Research

A qualitative study must be undertaken by grounding it in the principles of bioethics such as beneficence, non-maleficence, autonomy, and justice. Protecting the participants is of utmost importance, and the greatest care has to be taken while collecting data from a vulnerable research population. The researcher must respect individuals, families, and communities and must make sure that the participants are not identifiable by their quotations that the researchers include when publishing the data. Consent for audio/video recordings must be obtained. Approval to be in FGDs must be obtained from the participants. Researchers must ensure the confidentiality and anonymity of the transcripts/audio-video records/photographs/other data collected as a part of the study. The researchers must confirm their role as advocates and proceed in the best interest of all participants.[ 42 , 47 , 48 ]

Rigor in Qualitative Research

The demonstration of rigor or quality in the conduct of the study is essential for every research method. However, the criteria used to evaluate the rigor of quantitative studies are not be appropriate for qualitative methods. Lincoln and Guba (1985) first outlined the criteria for evaluating the qualitative research often referred to as “standards of trustworthiness of qualitative research”.[ 49 ] The four components of the criteria are credibility, transferability, dependability, and confirmability.

Credibility refers to confidence in the 'truth value' of the data and its interpretation. It is used to establish that the findings are true, credible and believable. Credibility is similar to the internal validity in quantitative research.[ 1 , 50 , 51 ] The second criterion to establish the trustworthiness of the qualitative research is transferability, Transferability refers to the degree to which the qualitative results are applicability to other settings, population or contexts. This is analogous to the external validity in quantitative research.[ 1 , 50 , 51 ] Lincoln and Guba recommend authors provide enough details so that the users will be able to evaluate the applicability of data in other contexts.[ 49 ] The criterion of dependability refers to the assumption of repeatability or replicability of the study findings and is similar to that of reliability in quantitative research. The dependability question is 'Whether the study findings be repeated of the study is replicated with the same (similar) cohort of participants, data coders, and context?'[ 1 , 50 , 51 ] Confirmability, the fourth criteria is analogous to the objectivity of the study and refers the degree to which the study findings could be confirmed or corroborated by others. To ensure confirmability the data should directly reflect the participants' experiences and not the bias, motivations, or imaginations of the inquirer.[ 1 , 50 , 51 ] Qualitative researchers should ensure that the study is conducted with enough rigor and should report the measures undertaken to enhance the trustworthiness of the study.

Conclusions

Qualitative research studies are being widely acknowledged and recognized in health care practice. This overview illustrates various qualitative methods and shows how these methods can be used to generate evidence that informs clinical practice. Qualitative research helps to understand the patterns of health behaviors, describe illness experiences, design health interventions, and develop healthcare theories. The ultimate strength of the qualitative research approach lies in the richness of the data and the descriptions and depth of exploration it makes. Hence, qualitative methods are considered as the most humanistic and person-centered way of discovering and uncovering thoughts and actions of human beings.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Open access
• Published: 27 May 2020

How to use and assess qualitative research methods

• Loraine Busetto   ORCID: orcid.org/0000-0002-9228-7875 1 ,
• Wolfgang Wick 1 , 2 &
• Christoph Gumbinger 1  

Neurological Research and Practice volume  2 , Article number:  14 ( 2020 ) Cite this article

729k Accesses

294 Citations

84 Altmetric

Metrics details

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 , 8 , 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 , 10 , 11 , 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 , 25 , 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 , 32 , 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 , 38 , 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Availability of data and materials

Not applicable.

Abbreviations

Endovascular treatment

Randomised Controlled Trial

Standard Operating Procedure

Standards for Reporting Qualitative Research

Philipsen, H., & Vernooij-Dassen, M. (2007). Kwalitatief onderzoek: nuttig, onmisbaar en uitdagend. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Qualitative research: useful, indispensable and challenging. In: Qualitative research: Practical methods for medical practice (pp. 5–12). Houten: Bohn Stafleu van Loghum.

Chapter   Google Scholar  

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches . London: Sage.

Kelly, J., Dwyer, J., Willis, E., & Pekarsky, B. (2014). Travelling to the city for hospital care: Access factors in country aboriginal patient journeys. Australian Journal of Rural Health, 22 (3), 109–113.

Article   Google Scholar  

Nilsen, P., Ståhl, C., Roback, K., & Cairney, P. (2013). Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Science, 8 (1), 1–12.

Howick J, Chalmers I, Glasziou, P., Greenhalgh, T., Heneghan, C., Liberati, A., Moschetti, I., Phillips, B., & Thornton, H. (2011). The 2011 Oxford CEBM evidence levels of evidence (introductory document) . Oxford Center for Evidence Based Medicine. https://www.cebm.net/2011/06/2011-oxford-cebm-levels-evidence-introductory-document/ .

Eakin, J. M. (2016). Educating critical qualitative health researchers in the land of the randomized controlled trial. Qualitative Inquiry, 22 (2), 107–118.

May, A., & Mathijssen, J. (2015). Alternatieven voor RCT bij de evaluatie van effectiviteit van interventies!? Eindrapportage. In Alternatives for RCTs in the evaluation of effectiveness of interventions!? Final report .

Google Scholar  

Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299 (10), 1182–1184.

Article   CAS   Google Scholar  

Christ, T. W. (2014). Scientific-based research and randomized controlled trials, the “gold” standard? Alternative paradigms and mixed methodologies. Qualitative Inquiry, 20 (1), 72–80.

Lamont, T., Barber, N., Jd, P., Fulop, N., Garfield-Birkbeck, S., Lilford, R., Mear, L., Raine, R., & Fitzpatrick, R. (2016). New approaches to evaluating complex health and care systems. BMJ, 352:i154.

Drabble, S. J., & O’Cathain, A. (2015). Moving from Randomized Controlled Trials to Mixed Methods Intervention Evaluation. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 406–425). London: Oxford University Press.

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science : IS, 8 , 117.

Hak, T. (2007). Waarnemingsmethoden in kwalitatief onderzoek. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Observation methods in qualitative research] (pp. 13–25). Houten: Bohn Stafleu van Loghum.

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36 , 717–732.

Yanow, D. (2000). Conducting interpretive policy analysis (Vol. 47). Thousand Oaks: Sage University Papers Series on Qualitative Research Methods.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

van der Geest, S. (2006). Participeren in ziekte en zorg: meer over kwalitatief onderzoek. Huisarts en Wetenschap, 49 (4), 283–287.

Hijmans, E., & Kuyper, M. (2007). Het halfopen interview als onderzoeksmethode. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [The half-open interview as research method (pp. 43–51). Houten: Bohn Stafleu van Loghum.

Jansen, H. (2007). Systematiek en toepassing van de kwalitatieve survey. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Systematics and implementation of the qualitative survey (pp. 27–41). Houten: Bohn Stafleu van Loghum.

Pv, R., & Peremans, L. (2007). Exploreren met focusgroepgesprekken: de ‘stem’ van de groep onder de loep. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Exploring with focus group conversations: the “voice” of the group under the magnifying glass (pp. 53–64). Houten: Bohn Stafleu van Loghum.

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41 (5), 545–547.

Boeije H: Analyseren in kwalitatief onderzoek: Denken en doen, [Analysis in qualitative research: Thinking and doing] vol. Den Haag Boom Lemma uitgevers; 2012.

Hunter, A., & Brewer, J. (2015). Designing Multimethod Research. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 185–205). London: Oxford University Press.

Archibald, M. M., Radil, A. I., Zhang, X., & Hanson, W. E. (2015). Current mixed methods practices in qualitative research: A content analysis of leading journals. International Journal of Qualitative Methods, 14 (2), 5–33.

Creswell, J. W., & Plano Clark, V. L. (2011). Choosing a Mixed Methods Design. In Designing and Conducting Mixed Methods Research . Thousand Oaks: SAGE Publications.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320 (7226), 50–52.

O'Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine : Journal of the Association of American Medical Colleges, 89 (9), 1245–1251.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52 (4), 1893–1907.

Moser, A., & Korstjens, I. (2018). Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. European Journal of General Practice, 24 (1), 9–18.

Marlett, N., Shklarov, S., Marshall, D., Santana, M. J., & Wasylak, T. (2015). Building new roles and relationships in research: A model of patient engagement research. Quality of Life Research : an international journal of quality of life aspects of treatment, care and rehabilitation, 24 (5), 1057–1067.

Demian, M. N., Lam, N. N., Mac-Way, F., Sapir-Pichhadze, R., & Fernandez, N. (2017). Opportunities for engaging patients in kidney research. Canadian Journal of Kidney Health and Disease, 4 , 2054358117703070–2054358117703070.

Noyes, J., McLaughlin, L., Morgan, K., Roberts, A., Stephens, M., Bourne, J., Houlston, M., Houlston, J., Thomas, S., Rhys, R. G., et al. (2019). Designing a co-productive study to overcome known methodological challenges in organ donation research with bereaved family members. Health Expectations . 22(4):824–35.

Piil, K., Jarden, M., & Pii, K. H. (2019). Research agenda for life-threatening cancer. European Journal Cancer Care (Engl), 28 (1), e12935.

Hofmann, D., Ibrahim, F., Rose, D., Scott, D. L., Cope, A., Wykes, T., & Lempp, H. (2015). Expectations of new treatment in rheumatoid arthritis: Developing a patient-generated questionnaire. Health Expectations : an international journal of public participation in health care and health policy, 18 (5), 995–1008.

Jun, M., Manns, B., Laupacis, A., Manns, L., Rehal, B., Crowe, S., & Hemmelgarn, B. R. (2015). Assessing the extent to which current clinical research is consistent with patient priorities: A scoping review using a case study in patients on or nearing dialysis. Canadian Journal of Kidney Health and Disease, 2 , 35.

Elsie Baker, S., & Edwards, R. (2012). How many qualitative interviews is enough? In National Centre for Research Methods Review Paper . National Centre for Research Methods. http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf .

Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18 (2), 179–183.

Sim, J., Saunders, B., Waterfield, J., & Kingstone, T. (2018). Can sample size in qualitative research be determined a priori? International Journal of Social Research Methodology, 21 (5), 619–634.

Download references

Acknowledgements

no external funding.

Author information

Authors and affiliations.

Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Loraine Busetto, Wolfgang Wick & Christoph Gumbinger

Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Wolfgang Wick

You can also search for this author in PubMed   Google Scholar

Contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

Corresponding author

Correspondence to Loraine Busetto .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Busetto, L., Wick, W. & Gumbinger, C. How to use and assess qualitative research methods. Neurol. Res. Pract. 2 , 14 (2020). https://doi.org/10.1186/s42466-020-00059-z

Download citation

Received : 30 January 2020

Accepted : 22 April 2020

Published : 27 May 2020

DOI : https://doi.org/10.1186/s42466-020-00059-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Mixed methods
• Quality assessment

Neurological Research and Practice

ISSN: 2524-3489

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 55, Issue 2
• Understanding qualitative research in health care
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

Qualitative studies are often used to research phenomena that are difficult to quantify numerically. 1,2 These may include concepts, feelings, opinions, interpretations and meanings, or why people behave in a certain way. Although qualitative research is often described in opposition to quantitative research, the approaches are complementary, and many researchers use mixed methods in their projects, combining the strengths of both approaches. 2 Many comprehensive texts exist on qualitative research methodology including those with a focus on healthcare related research. 2-4 Here we give a brief introduction to the rationale, methods and quality assessment of qualitative research.

https://doi.org/10.1136/dtb.2017.2.0457

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Key issues in qualitative research

Qualitative research allows deeper understanding of the richness and complexity of social phenomena. Qualitative methods can provide evidence on health and illness and can be used in various ways: 3

To complement quantitative methods, or when quantitative methods are impractical (e.g. when the topic is sensitive, poses measurement problems or is concerned with process and/or interaction; the research population is very small; or for intensive understanding of an innovation before widespread introduction). 3

In the exploratory stages of an applied health research programme, when they may clarify the research question and generate hypotheses. 3 Study design is often described as flexible or ‘emergent’ and researchers may have to adapt a study through a process of ‘progressive focussing’ in response to important but unanticipated findings. 3

To assess a pre-specified hypothesis, as in quantitative research. 3

To study particular experiences in depth rather than population averages or generalisations, and explore experiences from different perspectives. 3 For example, assessment of patient satisfaction is complex and varies between cultures or settings, which can make it difficult to develop quantitative measurement instruments. 5

To involve potential users in development and implementation of interventions. 6

To interpret the findings of quantitative research. 3

For example, in one study, alongside a quantitative review on the efficacy of telephone counselling for informal carers of people with dementia, qualitative methods were used to explore carers' experiences of receiving telephone counselling and counsellors' experiences of conducting telephone counselling. 7 Analysis of quantitative and qualitative results identified areas for improvement.

While the same ethical principles (autonomy, non-maleficence [doing no harm], beneficence and justice) apply equally to both quantitative and qualitative research, the application of these principles should take into account the nature of the research proposed. 3 The risks of serious physical harm arising from qualitative research are rarely as high as in quantitative medical studies. However, such research may cause psychological harm, damage to self-esteem and have an adverse impact on interpersonal relationships. Qualitative researchers should seek to minimise such risks. A ‘good’ interviewer builds rapport, encouraging trust and disclosure. In addition, the interviewer also needs to allow opportunities for the interviewee to withdraw or cease disclosure. 4

People, settings, processes, events or data are sampled in qualitative research in order to understand a particular problem, rather than to statistically represent a wider population. In contrast to quantitative methods, where many participants are sampled by randomisation, the small number of participants in qualitative research are often deliberately chosen for their ability to illuminate the research question (known as purposive sampling). 2 , 3 Such sampling may include the selection of outlier cases, cases with maximum variation or homogeneous cases.

Data collection

Methods of collecting data include observation, interviews and focus groups. 3  It is usual to write field notes, but methods can also include video or audio recordings.

Observation

Observation allows researchers to notice events and relationships in a ‘natural’ setting, which may produce insights about which the participants may not be aware. Observation (with varying levels of participation by the researcher in the events being observed) may require a prolonged time period but allows the observer to check their record against an informant's description and note discrepancies. 3

One-to-one interviews are used when the researcher wants to find out something that cannot be directly observed (e.g. how consumers evaluate health services, what understanding and attitudes underlie particular kinds of health behaviour). 3

Good questions in qualitative interviews should be open-ended, neutral, sensitive and clear to the interviewee. 8 , 9 It is usually best to start with questions that the interviewee can answer easily and then proceed to more difficult or sensitive topics. 8 Open questions allow the respondent to answer in their own words and interviews where there is a good rapport may be more likely to create a context in which respondents are prepared to discuss sensitive topics and penetrate beyond a superficial level. 2 , 3 Researchers should monitor their interviewing technique, critically appraise recordings of their interviews and ask others for their comments, noting in particular whether leading questions are being asked, whether cues are picked up or ignored, and whether interviewees are given enough time to explain what they mean. 8

Interviews can be tightly structured or more flexible: 3

A structured (standardised) interview is one in which the wording and order of all questions are exactly the same for every respondent. This aims to ensure that any differences between respondents are not attributable to differences in the questionnaire or research procedures.

A semi-structured interview covers topics on the researcher's agenda, but the phrasing and ordering of questions are tailored to the vocabulary and understanding of respondents, rather than imposing a uniform set of questions, which may be interpreted differently by respondents.

A ‘depth’ or ‘long’ interview, involves no specific set of questions. The participant's perspective on the phenomenon of interest should unfold as the participant and not the researcher views it. 3 The flexibility of such interviews may uncover new unanticipated areas or ideas.

Focus groups

Focus groups (or group interviews) are informal discussions in which participants focus collectively on a topic or issue, usually presented to them (either verbally or in written form) as a vignette to discuss. 3 , 10 Focus group participants (usually 6-8 people) may be existing clusters of people (e.g. family members, friends or work colleagues) or assembled specifically for the research. 10

The advantage of a focus group is the interaction of group participants with each other, as well as with the researcher/moderator. Such interaction is helpful to:

gain access to participants' own assumptions, meanings, vocabulary and thinking patterns;

identify concepts and practices central to the respondents, and to design subsequent questionnaires using respondents' own words or phrases, enhancing future participants' understanding of research questions;

observe how people interact to construct meanings attributed to health and illness, how opinions are formed, expressed, defended and sometimes modified in discussion and debate with others;

enable participants to have more control over the research agenda, which may generate unexpected insights into areas important to them; and

identify obstacles or objections that prevent or discourage individuals from healthy behaviours, and explore individuals' understanding of, and responses to, health-related media messages. 10

Data analysis

Whereas quantitative analysis seeks to reduce data, generating a series of statistics, qualitative analysis aims to represent data using detailed knowledge of cases. Qualitative research studies typically produce very large amounts of data, which are reduced and summarised into codes, concepts, categories, themes and theory. 3

There are many different approaches to analysis in qualitative research, including thematic analysis, framework analysis, grounded theory, interpretive phenomenological analysis, conversation analysis and discourse analysis. 2 , 3 , 11 , 12

Grounded theory derives theory from data using a constant comparative method. 2 , 3 , 11 , 12 Items in the raw data (e.g. interviews, field notes) are broken down, so that as many ideas and concepts as possible are identified and systematically coded, while at the same time comparing them with previous incidents. 2 , 3 Standardised rules are used for transcribing data to limit the risk of misinterpretation. Codes and categories can be based on the respondent's own words. 3 The data are simultaneously organised and reassembled, so that the researcher can identify relevant elements and relationships by cataloguing the substantive topics and attaching labels to concepts and themes that they see emerging from the material (rather than reflecting prior theoretical assumptions), developing core categories and ultimately explaining phenomena. 2 , 3 The analysis is iterative with the findings of preliminary analyses of data from the first phases of fieldwork informing subsequent work. 3

There should be careful documentation of all analytic decisions that lead to the creation of categories, forming an audit trail that provides evidence to support the integrity of the coding, categorisation and interpretive choices made throughout the qualitative data analysis process. 2

Researchers should also demonstrate continuous reflection on the fit between data and reality, by regarding all theoretical explanations, categories, hypotheses and questions about the data as being provisional until they are compared with the data. 3 They should give clear explanations of methods and demonstrate that they have considered alternative possible explanations for their data. 3

A study may be adapted through a process of iterative ‘progressive focussing’ in response to important unanticipated findings. 3 ‘Data saturation’ is a point at which no new themes or ideas arise from data analysis, and data collection can stop at that point.

Researchers' pre-existing assumptions are often challenged by qualitative research, as ideas are raised by respondents and new concepts and theories can develop that help to explain nuance and complexity. 13 Qualitative studies are not seeking statistical generalisability, but to provide insights with applicability to other settings and situations. It is helpful to have the qualitative researchers' description of what they have done and how it was done, so that readers can judge how transferable the findings are to other contexts. 14

Quality assessment

Qualitative methodologies are derived from many fields of social research with different philosophies and underlying assumptions. 3 There is, therefore, more than one correct way to conduct qualitative research, and there are debates and disagreements about qualitative methodology that can be confusing. 15

Quality assessment of qualitative research studies remains a contested area. 16–18 In addition, as qualitative researchers may have an impact on the data, they must show the efforts they undertook to prevent this from influencing the data. 7

Just as the quality of quantitative studies may be assessed using checklists, many checklists are available to help assess qualitative studies. 16 , 17 For example, the Critical Skills Appraisal Programme (CASP) has produced a checklist to aid evaluation of qualitative research (see Box). 19

CASP checklist 19

Screening questions:

Was there a clear statement of the aims of the research?

Is a qualitative methodology appropriate? Is it worth continuing? (Only continue if both these questions are answered ‘yes’).

Was the research design appropriate to address the aims of the research?

Was the recruitment strategy appropriate to the aims of the research?

Were the data collected in a way that addressed the research issue?

Has the relationship between researcher and participants been adequately considered?

Have ethical issues been taken into consideration?

Was the data analysis sufficiently rigorous?

Is there a clear statement of findings?

How valuable is the research?

A more detailed checklist (the COREQ) consists of 32 items in three domains: 20

The research team and reflexivity (covering items such as the researchers' sex, credentials, occupation, experience or training; the relationship with the participants; researchers' biases and assumptions).

The study design (e.g. the theoretical framework, sampling, how participants were approached, the setting, description of the sample, data collection and the audit trail)

Data analysis and reporting (e.g. the number of data coders and description of coding method, whether participant quotations were presented to illustrate the themes, whether data were consistent, and whether themes were clearly presented in the findings).

Comparison between qualitative and quantitative research

The table below summarises some key differences between qualitative and quantitative research.

• View inline

Comparison of quantitative and qualitative research 3

How can qualitative and quantitative research complement each other?

Qualitative research can complement quantitative approaches. Two examples are given below:

Survey research

Qualitative field work (e.g. using observations, interviews or focus groups) to achieve a better understanding of health problems; generation of questions and response options to capture the reality of participants' experiences; qualitative interviews to seek feedback on the perceived meanings and wording of proposed survey questions; quantitative analysis of survey responses from a large, randomly selected sample.

Intervention development and evaluation

Qualitative fieldwork to illuminate the complexity of a health problem, explore barriers and facilitators for change and explore suitable types and designs of treatment or intervention; qualitative methods to involve users in intervention design to ensure that it meets recipient needs and that it is engaging, feasible and sustainable; randomised controlled trial to assess the effectiveness of an intervention; qualitative study to explore how interventions work (and why they don't work) in the real world.

Qualitative research is a technical discipline that is important in providing valuable descriptions of complex phenomena. Qualitative studies differ from quantitative studies in that they often generate rather than test hypotheses; focus on processes, thoughts, feelings and experiences rather than outcomes; and assess small numbers of participants in depth rather than large sample sizes. Such research is designed to capture concepts that are hard to enumerate with quantitative research methods. Findings from qualitative research can enhance the design of other types of research and the implementation of health interventions. Data are often collected using observation, interviews or focus groups, usually recorded and transcribed, which allow the exploration of topics in the respondents' own words and understanding. Analysis of the information must be systematic and transparent; checklists exist to help assess the quality of qualitative studies. Qualitative methods can complement quantitative methods to add a further dimension to research, although there may be issues around the generalisability of results from one context to another. Evidence from qualitative studies can lead to a richer understanding of the complex reality of people's lives and their experience of illness and medical care.

• Understanding statistical terms: 3 . DTB 2009 ; 47 : 59 – 60 . OpenUrl Abstract / FREE Full Text
• Murphy E , et al
• Thorogood N
• Almaida RS , et al
• Bailey J , et al
• Lins S , et al
• Wilkinson S
• Glaser BG ,
• Strauss A ,
• Silverman D
• ↵ Cochrane Methods , 2011 . Supplemental handbook guidance [online]. Available: http://methods.cochrane.org/qi/supplemental-handbook-guidance [Accessed 18 January 2017] .
• ↵ Cochrane Qualitative and Implementation Methods Group , 2013 . Qualitative and implementation evidence and Cochrane reviews [online]. Available: http://methods.cochrane.org/sites/methods.cochrane.org.qi/files/public/uploads/Handbook52_QQ_Qualitative_web%20update%20Oct%202015.pdf [Accessed 18 January 2017] .
• Crabtree BF
• ↵ Critical Appraisal Skills Programme , 2013 . Qualitative research checklist [online]. Available: http://media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf [Accessed 18 January 2017] .
• Tong A , et al

Read the full text or download the PDF:

Loading metrics

Open Access

Qualitative Research: Understanding Patients' Needs and Experiences

• The PLoS Medicine Editors

Published: August 28, 2007

• https://doi.org/10.1371/journal.pmed.0040258
• Reader Comments

Citation: The PLoS Medicine Editors (2007) Qualitative Research: Understanding Patients' Needs and Experiences. PLoS Med 4(8): e258. https://doi.org/10.1371/journal.pmed.0040258

Copyright: © 2007 The PLoS Medicine Editors. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Why do up to half of all patients with tuberculosis (TB) fail to adhere to drug treatment [ 1 ]? The answer to this question is a matter of life and death, since nonadherence contributes to disease relapse and mortality [ 2 ]. In last month's PLoS Medicine, Salla Munro and colleagues argue that qualitative studies—in which researchers listen to what patients, care givers, and health care providers have to say—can provide important insights into why nonadherence occurs [ 3 ]. Their paper is a “meta-ethnography” [ 4 ], a systematic review and synthesis of qualitative studies on adherence to TB medication. The review found a wide array of factors to explain nonadherence, such as the belief that if one's symptoms have disappeared there is no need to finish a course of treatment. We published this review because we thought it would play a role in improving the delivery of TB treatment and ultimately in reducing the enormous global burden of the disease.

PLoS Medicine has now published two such meta-ethnographies (the first looked at adherence to HIV medication [ 5 ]). We have also published a small number of individual qualitative studies. For example, in our special issue on social medicine ( http://collections.plos.org/plosmedicine/socialmedicine-2006.php ), we published a qualitative study of migrant workers in the US that found that farm working and housing conditions are organized according to ethnicity and citizenship and that this hierarchy determines health disparities [ 6 ]. We have been very selective in our editorial decisions about which qualitative studies to publish. In our decision-making process, we have been guided by two crucial questions.

The first question is whether a qualitative approach was the right way to answer the research question. Quantitative research strives to be objective: human beings, health, and illness are the objects of investigation. Such investigation has led to extraordinary biomedical advances—yet patients often fail to reap the benefits because health professionals may not understand how best to deliver them in the context of patients' multifaceted lives. The academic editor of Salla Munro and colleagues' study commented that thinking of TB drugs simply as a “biomedical intervention” without factoring in patients' needs and broader social contexts creates circumstances that increase the likelihood of poor adherence to treatment. Qualitative research is the best way to understand these needs and contexts.

Astrid Fletcher and colleagues, for example, used quantitative methods to objectively determine who (in terms of age, sex, and education level) did not use the eye-care services available in India [ 7 ]. But they adopted a qualitative approach to answer the question of why people did not use these services. David Leon and colleagues, during a quantitative study on hazardous alcohol drinking in Russia, learned that much alcohol was consumed in the form of what were described as “surrogates” [ 8 ]. Qualitative research helped to identify what these surrogates were—they included eau de Cologne and over-the-counter medications.

When researchers investigate the experiences of people receiving or failing to receive health care, identify themes in these subjective stories, and integrate these themes into the greater context of human life experience, the results are informative to care providers. The usefulness of these results lies precisely in their subjectivity: the subjects are telling us, or we are finding out through more subtle observation, what matters to them.

The results of qualitative research can also help to inform the very process of research itself. Qualitative approaches can help us to understand, for example, why some patients decline to participate in clinical trials [ 9 ], or how patients experience the trial process itself. They can even be used to refine or improve a clinical trial in “real time.” In a trial of a computerized decision support tool for patients with atrial fibrillation being considered for anticoagulation treatment, Madeleine Murtagh and colleagues used qualitative evidence in deciding to discontinue one arm of the trial (the intervention in that arm was causing confusion amongst the patients and was unlikely to produce valid data) [ 10 ]. When a quantitative study is assessing the effectiveness of a complex multifaceted intervention, qualitative methods can help to tease out why such an intervention works or fails [ 11 ]. Qualitative approaches can also help to identify which of many possible research questions should receive priority for investigation, often by asking the research participants themselves. For example, patients with asthma may value easy-to-use inhalers more highly than a new class of drug.

Once it is clear that qualitative methods constitute the right approach for a study submitted to PLoS Medicine, the second question is whether the study meets our criteria for rigor and relevance. For a study to be suitable, regardless of the methodology, it should address an important topic in clinical medicine or public health and it should have the potential to transform our understanding of the causes or treatment of disease. In assessing any study, quantitative or qualitative, we are always on the lookout for biases, poorly described methods, and limited generalizability or overinterpretation of the data. In specifically assessing qualitative studies, we additionally wish to be reassured that the researchers used some type of “quality control” in analyzing the data—for example, were the data independently analyzed by at least two researchers and did consistent themes emerge from the data each time?

One characteristic of PLoS Medicine is the very broad range of research that we have published to date. We feel that such a range is appropriate for a medical journal, since understanding the complex nature of illness and health care requires a variety of different research approaches. “What is involved is not a crossroads where we have to go left or right,” Martyn Hammersley has argued in a discussion of the false dichotomy between quantitative and qualitative research. “A better analogy is a complex maze where we are repeatedly faced with decisions, and where paths wind back on one another” [ 12 ].

• View Article
• Google Scholar
• 2. Volmink J, Garner P (2006) Directly observed therapy for treating tuberculosis. Cochrane Database Syst Rev 2, CD003343.
• 4. Noblit GW, Hare RD (1988) Meta-ethnography: Synthesizing qualitative studies. Newbury Park (CA): Sage. 88 p.
• 12. (1992) Deconstructing the qualitative–quantitative divide. In: Brannen J, editor. Mixing methods: Qualitative and quantitative research. Aldershot (United Kingdom): Avebury. pp. 39–55. editor.

• Open access
• Published: 13 December 2018

Using qualitative Health Research methods to improve patient and public involvement and engagement in research

• Danielle E. Rolfe 1 ,
• Vivian R. Ramsden 2 ,
• Davina Banner 3 &
• Ian D. Graham 1  

Research Involvement and Engagement volume  4 , Article number:  49 ( 2018 ) Cite this article

43k Accesses

58 Citations

23 Altmetric

Metrics details

Plain English summary

Patient engagement (or patient and public involvement) in health research is becoming a requirement for many health research funders, yet many researchers have little or no experience in engaging patients as partners as opposed to research subjects. Additionally, many patients have no experience providing input on the research design or acting as a decision-making partner on a research team. Several potential risks exist when patient engagement is done poorly, despite best intentions. Some of these risks are that: (1) patients’ involvement is merely tokenism (patients are involved but their suggestions have little influence on how research is conducted); (2) engaged patients do not represent the diversity of people affected by the research; and, (3) research outcomes lack relevance to patients’ lives and experiences.

Qualitative health research (the collection and systematic analysis of non-quantitative data about peoples’ experiences of health or illness and the healthcare system) offers several approaches that can help to mitigate these risks. Several qualitative health research methods, when done well, can help research teams to: (1) accurately incorporate patients’ perspectives and experiences into the design and conduct of research; (2) engage diverse patient perspectives; and, (3) treat patients as equal and ongoing partners on the research team.

This commentary presents several established qualitative health research methods that are relevant to patient engagement in research. The hope is that this paper will inspire readers to seek more information about qualitative health research, and consider how its established methods may help improve the quality and ethical conduct of patient engagement for health research.

Research funders in several countries have posited a new vision for research that involves patients and the public as co-applicants for the funding, and as collaborative partners in decision-making at various stages and/or throughout the research process. Patient engagement (or patient and public involvement) in health research is presented as a more democratic approach that leads to research that is relevant to the lives of the people affected by its outcomes. What is missing from the recent proliferation of resources and publications detailing the practical aspects of patient engagement is a recognition of how existing research methods can inform patient engagement initiatives. Qualitative health research, for example, has established methods of collecting and analyzing non-quantitative data about individuals’ and communities’ lived experiences with health, illness and/or the healthcare system. Included in the paradigm of qualitative health research is participatory health research, which offers approaches to partnering with individuals and communities to design and conduct research that addresses their needs and priorities.

The purpose of this commentary is to explore how qualitative health research methods can inform and support meaningful engagement with patients as partners. Specifically, this paper addresses issues of: rigour (how can patient engagement in research be done well?); representation (are the right patients being engaged?); and, reflexivity (is engagement being done in ways that are meaningful, ethical and equitable?). Various qualitative research methods are presented to increase the rigour found within patient engagement. Approaches to engage more diverse patient perspectives are presented to improve representation beyond the common practice of engaging only one or two patients. Reflexivity, or the practice of identifying and articulating how research processes and outcomes are constructed by the respective personal and professional experiences of researchers and patients, is presented to support the development of authentic, sustainable, equitable and meaningful engagement of patients as partners in health research.

Conclusions

Researchers will need to engage patients as stakeholders in order to satisfy the overlapping mandate in health policy, care and research for engaging patients as partners in decision-making. This paper presents several suggestions to ground patient engagement approaches in established research designs and methods.

Peer Review reports

Patient engagement (or patient and public involvement) in research involves partnering with ‘patients’ (a term more often used in Canada and the US, that is inclusive of individuals, caregivers, and/or members of the public) to facilitate research related to health or healthcare services. Rather than research subjects or participants, patients are engaged as partners in the research process. This partnership is intended to be meaningful and ongoing, from the outset of planning a research project, and/or at various stages throughout the research process. Engagement can include the involvement of patients in defining a research question, identifying appropriate outcomes and methods, collecting and interpreting data, and developing and delivering a knowledge translation strategy [ 1 ].

The concept of engaging non-researchers throughout the research process is not new to participatory health researchers, or integrated knowledge translation researchers, as the latter involves ongoing collaboration with clinicians, health planners and policy makers throughout the research process in order to generate new knowledge [ 2 , 3 ]. Patients, however, are less frequently included as partners on health research teams, or as knowledge users in integrated knowledge translation research teams compared to clinicians, healthcare managers and policy-makers, as these individuals are perceived as having “the authority to invoke change in the practice or policy setting.” (p.2) [ 2 ] Recent requirements for patient engagement by health research funders [ 4 , 5 , 6 ], ,and mandates by most healthcare planners and organizations to engage patients in healthcare improvement initiatives, suggest that it would be prudent for integrated knowledge translation (and indeed all) health researchers to begin engaging patients as knowledge users in many, if not all, of their research projects.

Training and tools for patient engagement are being developed and implemented in Canada via the Canadian Institutes for Health Research (CIHR) Strategy for Patient Oriented Research (SPOR) initiative, in the US via Patient Centered Outcomes Research Institute (PCORI), and very practical resources are already available from the UK’s more established INVOLVE Advisory Group [ 5 , 6 , 7 ]. What is seldom provided by these ‘get started’ guides, however, are rigorous methods and evidence-based approaches to engaging diverse patient perspectives, and ensuring that their experiences, values and advice are appropriately incorporated into the research process.

The purpose of this commentary is to stimulate readers’ further discussion and inquiry into qualitative health research methods as a means of fostering the more meaningfully engagement of patients as partners for research. Specifically, this paper will address issues of: rigour (how do we know that the interpretation of patients’ perspectives has been done well and is applicable to other patients?); representation (are multiple and diverse patient perspectives being sought?); and, reflexivity (is engagement being done ethically and equitably?). This commentary alone is insufficient to guide researchers and patient partners to use the methods presented as part of their patient engagement efforts. However, with increased understanding of these approaches and perhaps guidance from experienced qualitative health researchers, integrated knowledge translation and health researchers alike may be better prepared to engage patients in a meaningful way in research that has the potential to improve health and healthcare experiences and outcomes.

What can be learned from methods utilized in qualitative health research?

There is wide variation in researchers’ and healthcare providers’ openness to engaging patients [ 8 ]. Often, the patients that are engaged are a select group of individuals known to the research team, sometimes do not reflect the target population of the research, are involved at a consultative rather than a partnership level, and are more likely to be involved in the planning rather than the dissemination of research [ 9 , 10 , 11 ]. As a result, patient engagement can be seen as tokenistic and the antithesis of the intention of most patient engagement initiatives, which is to have patients’ diverse experiences and perspectives help to shape what and how research is done. The principles, values, and practices of qualitative health research (e.g., relativism, social equity, inductive reasoning) have rich epistemological traditions that align with the conceptual and practical spirit of patient engagement. It is beyond the scope of this commentary, however, to describe in detail the qualitative research paradigm, and readers are encouraged to gain greater knowledge of this topic via relevant courses and texts. Nevertheless, several qualitative research considerations and methods can be applied to the practice of patient engagement, and the following sections describe three of these: rigour, representation and reflexivity.

Rigour: Interpreting and incorporating patients’ experiences into the design and conduct of research

When patient engagement strategies go beyond the inclusion of a few patient partners on the research team, for example, by using focus groups, interviews, community forums, or other methods of seeking input from a broad range of patient perspectives, the diversity of patients’ experiences or perspectives may be a challenge to quickly draw conclusions from in order to make decisions about the study design. To make these decisions, members of the research team (which should include patient partners) may discuss what they heard about patients’ perspectives and suggestions, and then unsystematically incorporate these suggestions, or they may take a vote, try to achieve consensus, implement a Delphi technique [ 12 ], or use another approach designed specifically for patient engagement like the James Lind Alliance technique for priority setting [ 13 ]. Although the information gathered from patients is not data (and indeed would require ethical review to be used as such), a number of qualitative research practices designed to increase rigour can be employed to help ensure that the interpretation and incorporation of patients’ experiences and perspectives has been done systematically and could be reproduced [ 14 ]. These practices include member checking , dense description , and constant comparative analysis . To borrow key descriptors of rigour from qualitative research, these techniques improve “credibility” (i.e., accurate representations of patients’ experiences and preferences that are likely to be understood or recognized by other patients in similar situations – known in quantitative research as internal validity), and “transferability” (or the ability to apply what was found among a group of engaged patients to other patients in similar contexts – known in quantitative research as external validity) [ 15 ].

Member checking

Member checking in qualitative research involves “taking ideas back to the research participants for their confirmation” (p. 111) [ 16 ]. The objective of member checking is to ensure that a researcher’s interpretation of the data (whether a single interview with a participant, or after analyzing several interviews with participants) accurately reflects the participants’ intended meaning (in the case of a member check with a single participant about their interview), or their lived experience (in the case of sharing an overall finding about several individuals with one or more participants) [ 16 ]. For research involving patient engagement, member checking can be utilized to follow-up with patients who may have been engaged at one or only a few time points, or on an on-going basis with patient partners. A summary of what was understood and what decisions were made based on patients’ recommendations could be used to initiate this discussion and followed up with questions such as, “have I understood correctly what you intended to communicate to me?” or “do you see yourself or your experience(s) reflected in these findings or suggestions for the design of the study?”

Dense description

As with quantitative research, detailed information about qualitative research methods and study participants is needed to enable other researchers to understand the context and focus of the research and to establish how these findings relate more broadly. This helps researchers to not only potentially repeat the study, but to extend its findings to similar participants in similar contexts. Dense description provides details of the social, demographic and health profile of participants (e.g., gender, education, health conditions, etc.), as well as the setting and context of their experiences (i.e., where they live, what access to healthcare they have). In this way, dense description improves the transferability of study findings to similar individuals in similar situations [ 15 ]. To date, most studies involving patient engagement provide limited details about their engagement processes and who was engaged [ 17 ]. This omission may be done intentionally (e.g., to protect the privacy of engaged patients, particularly those with stigmatizing health conditions), or as a practical constraint such as publication word limits. Nonetheless, reporting of patient engagement using some aspects of dense description of participants (as appropriate), the ways that they were engaged, and recommendations that emanated from engaged patients can also contribute to greater transferability and understanding of how patient engagement influenced the design of a research study.

Constant comparative analysis

Constant comparative analysis is a method commonly used in grounded theory qualitative research [ 18 ]. Put simply, the understanding of a phenomenon or experience that a researcher acquires through engaging with participants is constantly redeveloped and refined based on subsequent participant interactions. This process of adapting to new information in order to make it more relevant is similar to processes used in rapid cycle evaluation during implementation research [ 19 ]. This method can be usefully adapted and applied to research involving ongoing collaboration and partnership with several engaged patient partners, and/or engagement strategies that seek the perspectives of many patients at various points in the research process. For example, if, in addition to having ongoing patient partners, a larger group of patients provides input and advice (e.g., a steering or advisory committee) at different stages in the research process, their input may result in multiple course corrections during the design and conduct of the research processes to incorporate their suggestions. These suggestions may result in refinement of earlier decisions made about study design or conduct, and as such, the research process becomes more iterative rather than linear. In this way, engaged patients and patient partners are able to provide their input and experience to improve each step of the research process from formulating an appropriate research question or objective, determining best approaches to conducting the research and sharing it with those most affected by the outcomes.

Representation: Gathering diverse perspectives to design relevant and appropriate research studies

The intention of engaging patients is to have their lived experience of health care or a health condition contribute to the optimization of a research project design [ 20 ]. Development of a meaningful and sustainable relationship with patient partners requires considerable time, a demonstrated commitment to partnership by both the patient partners and the researcher(s), resources to facilitate patient partners’ engagement, and often, an individual designated to support the development of this relationship [ 17 , 21 ]. This may lead some research teams to sustain this relationship with only one or two patients who are often previously known to the research team [ 17 ]. The limitation of this approach is that the experiences of these one or two individuals may not adequately reflect the diverse perspectives of patients that may be affected by the research or its outcomes. The notion of gaining ‘ the patient perspective’ from a single or only a few individuals has already been problematized [ 22 , 23 ]. To be sure, the engagement of a single patient is better than none at all, but the engagement of a broader and diverse population of patients should be considered to better inform the research design, and to help prevent further perpetuation of health disparities. Key issues to be considered include (1) how engagement can be made accessible to patients from diverse backgrounds, and (2) which engagement strategies (e.g., ranging from a community information forum to full partnership on the research team) are most appropriate to reach the target population [ 24 ].

Making engagement accessible

Expecting patient partner(s) to attend regular research team meetings held during working hours in a boardroom setting in a hospital, research institute or university limits the participation of many individuals. To support the participation and diversity of engaged patients, effort should be made to increase the accessibility and emotional safety of engagement initiatives [ 25 ]. A budget must be allocated for patient partners’ transportation, childcare or caregiving support, remuneration for time or time taken off work and, at the very least, covering expenses related to their engagement. Another consideration that is often made by qualitative health researchers is whether brief counselling support can be provided to patients should the sharing of their experiences result in emotional distress. There are some resources that can help with planning for costs [ 26 ], including an online cost calculator [ 27 ].

Engagement strategies

Patient partners can be coached to consider the needs and experiences of people unlike them, but there are other methods of engagement that can help to gain a more fulsome perspective of what is likely a diverse patient population that is the focus of the research study. In qualitative health research, this is known as purposeful or purposive sampling: finding people who can provide information-rich descriptions of the phenomenon under study [ 28 ]. Engagement may require different approaches (e.g., deliberative group processes, community forums, focus groups, and patient partners on the research team), at different times in the research process to reach different individuals or populations (e.g., marginalized patients, or patients or caregivers experiencing illnesses that inhibit their ability to maintain an ongoing relationship with the research team). Engagement strategies of different forms at different times may be required. For example, ongoing engagement may occur with patient partners who are members of the research team (e.g., co-applicants on a research grant), and intermittent engagement may be sought from other patients through other methods that may be more time-limited or accessible to a diverse population of patients (e.g., a one-time focus group, community forum, or ongoing online discussion) to address issues that may arise during various stages of the research or dissemination processes. The result of this approach is that patients are not only consulted or involved (one-time or low commitment methods), but are also members of the research team and have the ability to help make decisions about the research being undertaken.

Engagement can generate a wealth of information from very diverse perspectives. Each iteration of engagement may yield new information. Knowing when enough information has been gathered to make decisions with the research team (that includes patient partners) about how the research may be designed or conducted can be challenging. One approach from qualitative research that can be adapted for patient engagement initiatives is theoretical saturation [ 29 ], or “the point in analysis when…further data gathering and analysis add little new to the conceptualization, though variations can always be discovered.” (p. 263) [ 18 ]. That is, a one-time engagement strategy (e.g., a discussion with a single patient partner) may be insufficient to acquire the diverse perspectives of the individuals that will be affected by the research or its outcomes. Additional strategies (e.g., focus groups or interviews with several individuals) may be initiated until many patients identify similar issues or recommendations.

Engagement approaches should also consider: how patients are initially engaged (e.g., through known or new networks, posted notices, telephone or in-person recruitment) and whether involvement has been offered widely enough to garner multiple perspectives; how patients’ experiences are shared (e.g., community forums, formal meetings, individual or group discussions) and whether facilitation enables broad participation; and finally, how patients’ participation and experiences are incorporated into the research planning and design, with patients having equal decision-making capacity to other research team members. Several publications and tools are available that can help guide researchers who are new to processes of engaging patients in research [ 24 , 30 , 31 , 32 , 33 , 34 ], but unfortunately few address how to evaluate the effectiveness of engagement [ 35 ].

Reflexivity: Ensuring meaningful and authentic engagement

In qualitative research, reflexivity is an ongoing process of “the researcher’s scrutiny of his or her research experience, decisions, and interpretations in ways that bring the researcher into the process and allow the reader to assess how and to what extent the researcher’s interests, positions, and assumptions influenced inquiry. A reflexive stance informs how the researcher conducts his or her research, relates to the research participants, and represents them in written reports,” (p.188–189) [ 16 ]. The concept of reflexivity can be applied to research involving patient engagement by continually and explicitly considering how decisions about the research study were made. All members of the research team must consider (and perhaps discuss): (1) how patient partners are invited to participate in research planning and decision-making; (2) how their input is received relative to other team members (i.e., do their suggestions garner the same respect as researchers’ or providers’?); and, (3) whether engaged patients or patient partners feel sufficiently safe, able and respected to share their experiences, preferences and recommendations with the research team.

Ideally, reflexivity becomes a practice within the research team and may be operationalized through regular check-ins with patients and researchers about their comfort in sharing their views, and whether they feel that their views have been considered and taken onboard. Power dynamics should also be considered during patient engagement initiatives. For example, reflecting on how community forums, focus groups or interviews are to be facilitated, including a consideration of who is at the table/who is not, who speaks/who does not, whose suggestions are implemented/whose are not? Reflexivity can be practiced through informal discussions, or using methods that may allow more candid responses by engaged patients (e.g., anonymous online survey or feedback forms). At the very least, if these practices were not conducted throughout the research process, the research team (including patient partners) should endeavor to reflect upon team dynamics and consider how these may have contributed to the research design or outcomes. For example, were physicians and researchers seen as experts and patients felt less welcome or able to share their personal experiences? Were patients only engaged by telephone rather than in-person and did this influence their ability to easily engage in decision-making? Reflexive practices may be usefully supplemented by formal evaluation of the process of patient engagement from the perspective of patients and other research team members [ 36 , 37 ], and some tools are available to do this [ 35 ].

A note about language

One way to address the team dynamic between researchers, professional knowledge users (such as clinicians or health policy planners) and patients is to consider the language used to engage with patients in the planning of patient engagement strategies. That is, the term ‘patient engagement’ is a construction of an individual’s identity that exists only within the healthcare setting, and in the context of a patient-provider dynamic. This term does not consider how people make decisions about their health and healthcare within a broader context of their family, community, and culture [ 22 , 38 ]. This may be why research communities in some countries (e.g., the United Kingdom) use the term ‘patient and public involvement’. Additionally, research that involves communities defined by geography, shared experiences, cultural or ethnic identity, as is the case with participatory health research, may refer to ‘community engagement.’ Regardless of the term used, partnerships with patients, the public, or with communities need to be conceived instead as person-to-person interactions between researchers and individuals who are most affected by the research. Discussions with engaged patients should be conducted early on to determine how to best describe their role on the team or during engagement initiatives (e.g., as patient partners, community members, or people with lived experience).

Tokenism is the “difference between…the empty ritual of participation and having the real power needed to affect the outcome,” (p.2) [ 39 ]. Ongoing reflection on the power dynamic between researchers and engaged patients, a central tenet of critical qualitative health research [ 40 , 41 ], can increase the likelihood that engagement involves equitable processes and will result in meaningful engagement experiences by patients rather than tokenism [ 36 , 42 ]. Patient engagement initiatives should strive for “partnership” amongst all team members, and not just reflect a patient-clinician or researcher-subject dynamic [ 43 ]. To develop meaningful, authentic and sustainable relationships with engaged patients, methods used for participatory, action or community-based research (approaches that fall under the paradigm of qualitative inquiry) provide detailed experiential guidance [ 44 ]. For example, a realist review of community-based participatory research projects reported that gaining and maintaining trust with patient or community partners, although time-intensive, is foundational to equitable and sustainable partnerships that benefit communities and individuals [ 45 , 46 ]. Additionally, Chapter Nine of the Canadian Tri-Council Policy Statement on Research involving Humans, which has to date been applied to research involving First Nations, Inuit and, Métis Peoples in Canada [ 47 ], provides useful information and direction that can be applied to working with patient partners on research [ 48 ].

Authentic patient engagement should include their involvement at all stages of the research process [ 49 , 50 ], but this is often not the case [ 10 ]. .Since patient partners are not research subjects or participants, their engagement does not (usually) require ethics approval, and they can be engaged as partners as early as during the submission of grant applications [ 49 ]. This early engagement helps to incorporate patients’ perspectives into the proposed research before the project is wedded to particular objectives, outcomes and methods, and can also serve to allocate needed resources to support patient engagement (including remuneration for patient partners’ time). Training in research for patient partners can also support their meaningful engagement by increasing their ability to fully engage in decision-making with other members of the research team [ 51 , 52 ]. Patient partners may also thrive in co-leading the dissemination of findings to healthcare providers, researchers, patients or communities most affected by the research [ 53 ].

Patient engagement has gained increasing popularity, but many research organizations are still at the early stages of developing approaches and methods, many of which are based on experience rather than evidence. As health researchers and members of the public will increasingly need to partner for research to satisfy the overlapping mandate of patient engagement in health policy, healthcare and research, the qualitative research methods highlighted in this commentary provide some suggestions to foster rigorous, meaningful and sustained engagement initiatives while addressing broader issues of power and representation. By incorporating evidence-based methods of gathering and learning from multiple and diverse patient perspectives, we will hopefully conduct better patient engaged research, live out the democratic ideals of patient engagement, and ultimately contribute to research that is more relevant to the lives of patients; as well as, contribute to the improved delivery of healthcare services. In addition to the references provided in this paper, readers are encouraged to learn more about the meaningful engagement of patients in research from several key texts [ 54 , 55 , 56 ].

Abbreviations

Canadian Institutes for Health Research

Patient Centered Outcomes Research Institute

Strategy for Patient Oriented Research

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research. 2014. http://www.cihr-irsc.gc.ca/e/documents/spor_framework-en.pdf . Accessed 27 Sep 2017.

Google Scholar  

Kothari A, Wathen CN. Integrated knowledge translation: digging deeper, moving forward. J Epidemiol Community Health. 2017;71:619–23. https://doi.org/10.1136/jech-2016-208490 .

Article   PubMed   Google Scholar  

Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci. 2016;11:38. https://doi.org/10.1186/s13012-016-0399-1 .

Article   PubMed   PubMed Central   Google Scholar  

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research - CIHR. http://www.cihr-irsc.gc.ca/e/41204.html . Accessed 20 Sep 2017.

Patient-Centered Outcomes Research Institute. Patient-Centered Outcomes Research Institute website. https://www.pcori.org/ . Accessed 20 Sep 2017.

National Institute for Health Research. INVOLVE | INVOLVE Supporting public involvement in NHS, public health and social care research. http://www.invo.org.uk/ . Accessed 27 Sep 2017.

Canadian Institutes of Health Research. SPOR SUPPORT Units - CIHR. http://www.cihr-irsc.gc.ca/e/45859.html . Accessed 27 Sep 2017.

Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf. 2016;25:626–32.

Hearld KR, Hearld LR, Hall AG. Engaging patients as partners in research: factors associated with awareness, interest, and engagement as research partners. SAGE open Med. 2017;5:2050312116686709. https://doi.org/10.1177/2050312116686709.

Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89. https://doi.org/10.1186/1472-6963-14-89 .

Concannon TW, Fuster M, Saunders T, Patel K, Wong JB, Leslie LK, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29:1692–701.

Oostendorp LJM, Durand M-A, Lloyd A, Elwyn G. Measuring organisational readiness for patient engagement (MORE): an international online Delphi consensus study. BMC Health Serv Res. 2015;15:61. https://doi.org/10.1186/s12913-015-0717-3 .

Boddy K, Cowan K, Gibson A, Britten N. Does funded research reflect the priorities of people living with type 1 diabetes? A secondary analysis of research questions. BMJ Open. 2017;7:e016540. https://doi.org/10.1136/bmjopen-2017-016540 .

Mays N, Pope C. Rigour and qualitative research. BMJ Br Med J. 1995;311:109–12.

Article   CAS   Google Scholar  

Krefting L. Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther. 1991;45:214–22.

Article   CAS   PubMed   Google Scholar  

Charmaz K. Constructing grounded theory : a practical guide through qualitative analysis. London: Sage Publications; 2006.

Wilson P, Mathie E, Keenan J, McNeilly E, Goodman C, Howe A, et al. ReseArch with patient and public invOlvement: a RealisT evaluation – the RAPPORT study. Heal Serv Deliv Res. 2015;3:1–176. https://doi.org/10.3310/hsdr03380 .

Corbin JM, Strauss AL, Strauss AL. Basics of qualitative research : techniques and procedures for developing grounded theory. Thousand Oaks: Sage Publications; 2008.

Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the consolidated framework for implementation research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci. 2017;12:15. https://doi.org/10.1186/s13012-017-0550-7 .

Esmail L, Moore E. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015;4:133–45. https://doi.org/10.2217/cer.14.79 .

Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7:387–95.

Rowland P, McMillan S, McGillicuddy P, Richards J. What is “the patient perspective” in patient engagement programs? Implicit logics and parallels to feminist theories. Heal An Interdiscip J Soc Study Heal Illn Med. 2016. https://doi.org/10.1177/1363459316644494 .

Article   Google Scholar  

Martin GP. “Ordinary people only”: knowledge, representativeness, and the publics of public participation in healthcare. Sociol Health Illn. 2008;30:35–54. https://doi.org/10.1111/j.1467-9566.2007.01027.x .

Snow B, Tweedie K, Pederson A, Shrestha H, Bachmann L. Patient Engagement: Heard and valued. 2013. http://www.cfhi-fcass.ca/sf-docs/default-source/patient-engagement/awesome_workbook-fraserhealth.pdf . Accessed 24 Jan 2018.

Montesanti SR, Abelson J, Lavis JN, Dunn JR. Enabling the participation of marginalized populations: case studies from a health service organization in Ontario. Canada Health Promot Int. 2017;32(4):636–49. https://doi.org/10.1093/heapro/dav118.

The Change Foundation. Should Money Come into It? A Tool for Deciding Whether to Pay Patient- Engagement Participants. Toronto; 2015. http://www.changefoundation.ca/site/wp-content/uploads/2016/05/Should-money-come-into-it.pdf . Accessed 18 June 2018

INVOLVE. Involvement Cost Calculator | INVOLVE. http://www.invo.org.uk/resource-centre/payment-and-recognition-for-public-involvement/involvement-cost-calculator/ . Accessed 11 Oct 2017.

Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks: Sage Publications; 2002.

Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–81. https://doi.org/10.1586/erp.10.30 .

Canadian Institutes of Health Research. CIHR’S Citizen Engagement Handbook. http://www.cihr-irsc.gc.ca/e/documents/ce_handbook_e.pdf . Accessed 18 June 2018.

Elliott J, Heesterbeek S, Lukensmeyer CJ, Slocum N. Participatory and deliberative methods toolkit how to connect with citizens: a practitioner’s manual. King Baudouin Foundation; 2005. https://www.kbs-frb.be/en/Virtual-Library/2006/294864 . Accessed 28 Sep 2017.

Robbins M, Tufte J, Hsu C. Learning to “swim” with the experts: experiences of two patient co-investigators for a project funded by the Patient-Centered Outcomes Research Institute. Perm J. 2016;20:85–8.

PubMed   PubMed Central   Google Scholar  

Gauvin F-P, Abelson J, Lavis JN. Strengthening public and patient engagement in health technology assessment in Ontario. 2014. https://www.mcmasterforum.org/docs/default-source/product-documents/evidence-briefs/public-engagement-in-health-technology-assessement-in-ontario-eb.pdf?sfvrsn=2 . Accessed 18 June 2018.

Abelson J, Montesanti S, Li K, Gauvin F-P, Martin E. Effective strategies for interactive public engagement in the development of healthcare policies and Programs. 2010; https://www.cfhi-fcass.ca/sf-docs/default-source/commissionedresearch-reports/Abelson_EN_FINAL.pdf?sfvrsn=0. . Accessed 16 Nov 2018.

Abelson J. Public and patient engagement evaluation tool (version 1.0), vol. 0; 2015. https://www.fhs.mcmaster.ca/publicandpatientengagement/ppeet_request_form.html ! Accessed 18 June 2018

Johnson DS, Bush MT, Brandzel S, Wernli KJ. The patient voice in research—evolution of a role. Res Involv Engagem. 2016;2:6. https://doi.org/10.1186/s40900-016-0020-4 .

Crocker JC, Boylan AM, Bostock J, Locock L. Is it worth it? Patient and public views on the impact of their involvement in health research and its assessment: a UK-based qualitative interview study. Health Expect. 2017;20(3):519–28. https://doi.org/10.1111/hex.12479 .

Renedo A, Marston C. Healthcare professionals’ representations of “patient and public involvement” and creation of “public participant” identities: implications for the development of inclusive and bottom-up community participation initiatives. J Community Appl Soc Psychol. 2011;21:268–80. https://doi.org/10.1002/casp.1092 .

Hahn DL, Hoffmann AE, Felzien M, LeMaster JW, Xu J, Fagnan LJ. Tokenism in patient engagement. Fam Pract. 2017;34(3):290–5. https://doi.org/10.1093/fampra/cmw097.

Hart C, Poole JM, Facey ME, Parsons JA. Holding firm: power, push-Back, and opportunities in navigating the liminal space of critical qualitative Health Research. Qual Health Res. 2017;27:1765–74. https://doi.org/10.1177/1049732317715631 .

Eakin JM. Educating critical qualitative health researchers in the land of the randomized controlled trial. Qual Inq. 2016;22:107–18. https://doi.org/10.1177/1077800415617207 .

Safo S, Cunningham C, Beckman A, Haughton L, Starrels JL. “A place at the table:” a qualitative analysis of community board members’ experiences with academic HIV/AIDS research. BMC Med Res Methodol. 2016;16:80. https://doi.org/10.1186/s12874-016-0181-8 .

Thompson J, Bissell P, Cooper C, Armitage CJ, Barber R, Walker S, et al. Credibility and the “professionalized” lay expert: reflections on the dilemmas and opportunities of public involvement in health research. Heal Interdiscip J Soc Study Heal Illn Med. 2012;16:602–18. https://doi.org/10.1177/1363459312441008 .

Burke JG, Jones J, Yonas M, Guizzetti L, Virata MC, Costlow M, et al. PCOR, CER, and CBPR: alphabet soup or complementary fields of Health Research? Clin Transl Sci. 2013;6:493–6. https://doi.org/10.1111/cts.12064 .

Jagosh J, Bush PL, Salsberg J, Macaulay AC, Greenhalgh T, Wong G, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health. 2015;15:725. https://doi.org/10.1186/s12889-015-1949-1 .

Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012;90:311–46. https://doi.org/10.1111/j.1468-0009.2012.00665.x .

Government of Canada IAP on RE. TCPS 2 - Chapter 9 Research Involving the First Nations, Inuit and Métis Peoples of Canada. http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/chapter9-chapitre9/ . Accessed 16 Oct 2017.

Ramsden VR, Crowe J, Rabbitskin N, Rolfe D, Macaulay A. Authentic engagement, co-creation and action research. In: Goodyear-smith F, Mash B, editors. How to do primary care research. Boca Raton: CRC press, medicine, Taylor & Francis Group; 2019.

Ramsden VR, Salsberg J, Herbert CP, Westfall JM, LeMaster J, Macaulay AC. Patient- and community-oriented research: how is authentic engagement identified in grant applications? Can Fam Physician. 2017;63:74–6.

Woolf SH, Zimmerman E, Haley A, Krist AH. Authentic engagement of patients and communities can transform research, practice, and policy. Health Aff. 2016;35:590–4.

Parkes JH, Pyer M, Wray P, Taylor J. Partners in projects: preparing for public involvement in health and social care research. Health Policy. 2014;117:399–408. https://doi.org/10.1016/j.healthpol.2014.04.014 .

Norman N, Bennett C, Cowart S, Felzien M, Flores M, Flores R, et al. Boot camp translation: a method for building a community of solution. J Am Board Fam Med. 2013;26:254–63. https://doi.org/10.3122/jabfm.2013.03.120253 .

Ramsden VR, Rabbitskin N, Westfall JM, Felzien M, Braden J, Sand J. Is knowledge translation without patient or community engagement flawed. Fam Pract. 2016:1–3.

Denzin NK, Lincoln YS. SAGE Handbook of qualitative research. 3rd ed. Thousand Oaks: Sage Publications; 2005.

Pearce V, Baraitser P, Smith G, Greenhalgh T. Experience-based co-design. In: User involvement in health care. Oxford: Wiley-Blackwell; 2010. p. 28–51. https://doi.org/10.1002/9781444325164.ch3 .

Chapter   Google Scholar  

Hulatt I, Lowes L. Involving service users in health and social care research: Routledge; 2005. https://doi.org/10.4324/9780203865651.

Book   Google Scholar  

Download references

Acknowledgements

This paper was drafted in response to a call for concept papers related to integrated knowledge translation issued by the Integrated Knowledge Translation Research Network (CIHR FDN #143237).

This paper was commissioned by the Integrated Knowledge Translation Network (IKTRN). The IKTRN brings together knowledge users and researchers to advance the science and practice of integrated knowledge translation and train the next generation of integrated knowledge translation researchers. Honorariums were provided for completed papers. The IKTRN is funded by a Canadian Institutes of Health Research Foundation Grant (FDN #143247).

Availability of data and materials

Not applicable.

Author information

Authors and affiliations.

School of Epidemiology and Public Health, University of Ottawa, 307D- 600 Peter Morand Crescent, Ottawa, ON, K1G 5Z3, Canada

Danielle E. Rolfe & Ian D. Graham

Department of Academic Family Medicine, University of Saskatchewan, West Winds Primary Health Centre, 3311 Fairlight Drive, Saskatoon, SK, S7M 3Y5, Canada

Vivian R. Ramsden

School of Nursing, University of Northern British Columbia, 3333 University Way, Prince George, BC, V2K4C6, Canada

Davina Banner

You can also search for this author in PubMed   Google Scholar

Contributions

DR conceived and drafted this paper. All authors were involved in critiquing and revising the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Danielle E. Rolfe .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Rolfe, D.E., Ramsden, V.R., Banner, D. et al. Using qualitative Health Research methods to improve patient and public involvement and engagement in research. Res Involv Engagem 4 , 49 (2018). https://doi.org/10.1186/s40900-018-0129-8

Download citation

Received : 18 June 2018

Accepted : 09 November 2018

Published : 13 December 2018

DOI : https://doi.org/10.1186/s40900-018-0129-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative methods
• Qualitative health research
• Patient-oriented research
• Integrated knowledge translation
• Patient engagement
• Patient partners
• Patient and public involvement

Research Involvement and Engagement

ISSN: 2056-7529

• General enquiries: [email protected]

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Qualitative Research:...

Qualitative Research: Consensus methods for medical and health services research

• Related content
• Peer review
• Jeremy Jones , Lecturer in health economics a ,
• Duncan Hunter , research fellow b
• a Nuffield Community Care Studies Unit, Department of Epidemiology and Public Health, University of Leicester, Leicester LE1 7RH,
• b Health Services Research Unit, Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, London WC1E 7HT
• Correspondence to: Dr Jones.

Health providers face the problem of trying to make decisions in situations where there is insufficient information and also where there is an overload of (often contradictory) information. Statistical methods such as meta-analysis have been developed to summarise and to resolve inconsistencies in study findings—where information is available in an appropriate form. Consensus methods provide another means of synthesising information, but are liable to use a wider range of information than is common in statistical methods, and where published information is inadequate or non-existent these methods provide a means of harnessing the insights of appropriate experts to enable decisions to be made. Two consensus methods commonly adopted in medical, nursing, and health services research—the Delphi process and the nominal group technique (also known as the expert panel)—are described, together with the most appropriate situations for using them; an outline of the process involved in undertaking a study using each method is supplemented by illustrations of the authors' work. Key methodological issues in using the methods are discussed, along with the distinct contribution of consensus methods as aids to decision making, both in clinical practice and in health service development.

This is the sixth in a series of seven articles describing non-quantitative techniquesand showing their value in health research

• Download figure
• Open in new tab
• Download powerpoint

Defining consensus and consensus methods

Quantitative methods such as meta-analysis have been developed to provide statistical overviews of the results of clinical trials and to resolve inconsistencies in the results of published studies. Consensus methods are another means of dealing with conflicting scientific evidence. They allow a wider range of study types to be considered than is usual in statistical reviews. In addition they allow a greater role for the qualitative assessment of evidence ( box 1 ). These methods, unlike those described in the other papers in this series, are primarily concerned with deriving quantitative estimates through qualitative approaches.

Features of consensus methods

• View inline

The aim of consensus methods is to determine the extent to which experts or lay people agree about a given issue. They seek to overcome some of the disadvantages normally found with decision making in groups or committees, which are commonly dominated by one individual or by coalitions representing vested interests. In open committees individuals are often not ready to retract long held and publicly stated opinions, even when these have been proved to be false.

The term “agreement” takes two forms, which need to be distinguished: firstly, the extent to which each respondent agrees with the issue under consideration (typically rated on a numerical or categorical scale) and, secondly, the extent to which respondents agree with each other, the consensus element of these studies (typically assessed by statistical measures of average and dispersion).

Application

The focus of consensus methods lies where unanimity of opinion does not exist owing to a lack of scientific evidence or where there is contradictory evidence on an issue. The methods attempt to assess the extent of agreement (consensus measurement) and to resolve disagreement (consensus development).

The Delphi process takes its name from the Delphic oracle's skills of interpretation and foresight

The three best known consensus methods are the Delphi process, the nominal group technique (also known as the expert panel), and the consensus development conference. Each of these methods involves measuring consensus, and the last two methods are also concerned with developing consensus. The consensus development conference will not be covered in this paper because it requires resources beyond those at the disposal of most researchers (unlike the other two methods), is commonly organised within defined programmes (for example, by the King's Fund in Britain and the National Institutes of Health in the United States), and has been discussed at length elsewhere. 3 4 5 6

The methods described THE DELPHI PROCESS

The Delphi process takes its name from the Delphic oracle's skills of interpretation and foresight and proceeds in a series of rounds as follows:

Round 1: Either the relevant individuals are invited to provide opinions on a specific matter, based on their knowledge and experience, or the team undertaking the Delphi expresses opinions on a specific matter and selects suitable experts to participate in subsequent questionnaire rounds;

These opinions are grouped together under a limited number of headings and statements drafted for circulation to all participants on a questionnaire;

Round 2: Participants rank their agreement with each statement in the questionnaire;

The rankings are summarised and included in a repeat version of the questionnaire;

Round 3: Participants rerank their agreement with each statement in the questionnaire, with the opportunity to change their score in view of the group's response;

The rerankings are summarised and assessed for degree of consensus: if an acceptable degree of consensus is obtained the process may cease, with final results fed back to participants; if not, the third round is repeated.

Figure 1 shows an example of this process for a Delphi study undertaken by one of the authors (JJ). In addition to scoring agreement with statements, respondents are commonly asked to rate the confidence or certainty with which they express their opinions.

Example of Delphiprocess used in study by JJ

The Delphi technique has been used widely in health research within the fields of technology assessment, 7 8 9 10 education and training 11 12 13 14 and priorities and information, 15 16 17 and in developing nursing and clinical practice. 19 20 21 It enables a large group of experts to be contacted cheaply, usually by mail with a self administered questionnaire (though computer communications have also been used), with few geographical limitations on the sample. Some situations have included a round in which the participants meet to discuss the process and resolve uncertainty or any ambiguities in the wording of the questionnaire.

THE NOMINAL GROUP TECHNIQUE

The nominal group technique uses a highly structured meeting to gather information from relevant experts (usually 9-12 in number) about a given issue. It consists of two rounds in which panellists rate, discuss, and then rerate a series of items or questions. The method was developed in the United States in the 1960s and has been applied to problems in social services, education, government, and industry. 22 In the context of health care the method has most commonly been used to examine the appropriateness of clinical interventions 23 24 25 26 27 but has also been applied in education and training, 28 29 30 in practice development, 31 32 33 and for identifying measures for clinical trials. 34 35 36

A nominal group meeting is facilitated either by an expert on the topic 37 or a credible non-expert 38 and is structured as follows:

Participants spend several minutes writing down their views about the topic in question;

Each participant, in turn, contributes one idea to the facilitator, who records it on a flip chart;

Similar suggestions are grouped together, where appropriate. There is a group discussion to clarify and evaluate each idea;

Each participant privately ranks each idea (round 1);

The ranking is tabulated and presented;

The overall ranking is discussed and reranked (round 2);

The final rankings are tabulated and the results fed back to participants.

Figure 2 shows an example of a modified nominal group undertaken by one of the authors (DH).

Example of modified nominal group undertaken by DH

The method can be adapted and has been conducted as a single meeting or with the first stage conducted by post followed by a discussion and rerating at a face to face meeting. Some nominal group meetings have incorporated a detailed review of literature as background material for the topic under discussion.

Alongside the consensus process there may be a nonparticipant observer collecting qualitative data on the nominal group. This approach has some features in common with focus groups (see articl by Kitzinger 39 ). However, the nominal group technique focuses on a single goal (for example, the definition of criteria to assess the appropriateness of a surgical intervention) and is less concerned with eliciting a range of ideas or the qualitative analysis of the group process per se than is the case in focus groups.

ological issues WHO TO INCLUDE AS PARTICIPANTS

There can be few hard and fast rules about who to include as participants, except that each must be justifiable as in some way “expert” on the matter under discussion. Clearly, for studies concerned with defining criteria for clinical intervention, the most appropriate experts will be clinicians practising in the field under consideration. However, the inclusion of other clinicians such as general practitioners may be appropriate to provide an alternative clinical view, particularly when the study is expected to have an impact beyond a particular specialist field. When the discussion concerns matters of general interest, such as health service priorities, participants should include non-clinical health professionals and the expression of lay opinions should also be allowed for.

There is clearly a potential for bias in the selection of participants. Although it has been shown that doctors who are willing to participate in expert panels are representative of their colleagues, 40 the exact composition of the panel can affect the results obtained. 24 The results will also be affected by any “random” variation in panel behaviour. These problems can be overcome by using a different mixture of participants in further panels.

HOW TO MEASURE THE ACCURACY OF THE ANSWER OBTAINED

The existence of a consensus does not mean that the “correct” answer has been found—there is the danger of deriving collective ignorance rather than wisdom. The nominal group is not a replacement for rigorous scientific reviews of published reports or for original research, but rather a means of identifying current medical opinion and areas of disagreement. For Delphi surveys, Pill recommends that the results should, when possible, be matched to observable events. 1 Observers of the accuracy of opinion polls before the 1992 general election in Britain might well agree with this conclusion.

HOW TO FEED BACK THE RESULTS OF EACH ROUND

Agreement with statements is usually summarised by using the median and consensus assessed by using interquartile ranges for continuous numerical scales. These summary statistics may be fed back to participants at each round along with fuller indications of the distribution of responses to each statement in the form of tables of the proportions ranking at each point on the scale (see box 2), histograms, or other graphical representations of the range (see box 3 ). Feeding back the group's response enables participants to consider their initial ranking in relation to their colleagues' assessments. It should be made clear to each participant that they need not conform to the group view—though, in the nominal group technique, those with atypical opinions (compared with the rest of the group) may face critical questioning of their view from other panel members. In a Delphi exercise, the researcher undertaking the study may ask participants who they have defined as outliers (for example, those in the lower and upper quartiles) to provide written justification for their responses.

Box 2—Example of feedback of second round results in a Delphi 40

The following are possible adverse effects of lowering the number of junior medical staff in general medicine and its associated specialties. The star indicates the number you selected to indicate the extent to which you agreed or disagreed with each statement in response to the previous questionnaire. Each of the numbers below the scale represents the percentage of those responding to the questionnaire who selected that particular value. We would be grateful if you would read through the questionnaire and consider whether, in the light of your colleagues' assessments, you would like to alter your response. Please indicate the extent to which you agree or disagree with each statement by circling the appropriate number (0 indicates total disagreement and 9 total agreement): if your choice remains unchanged please circle the same number you selected on the previous questionnaire.

(i) Mortality rates in hospital will rise

disagree 0 1 2 3 4 5 6 7 8 9 agree

5-3-9-4-3-18-18-12-16-12

Example of feedback of first round results in a nominal group23

For nominal groups, rules have been developed to assess agreement when statements have been ranked on a 9 point scale (see box 3 ). In this example, the scale can be broken down so that scores 1-3 represent a region where participants feel intervention is not indicated; 4-6, a region where participants are equivocal; and 7-9, a region where participants feel intervention is indicated. The first rule is based on where the scores fall on the ranking scale ( box 4 ): if all ratings fall within one of these predefined regions there is said to be strict agreement (in the example, all participants agreed that transurethral resection of the prostate was not indicated). An alternative relaxed definition for agreement is that all ratings fall within any 3 point region. This may be treated as agreement, in that all ratings are within an acceptable range, but the group opinion is ambiguous as to whether intervention is indicated or not.

Examples of strict and relaxed rules for agreement in a nominal group

The second rule tests whether extreme rankings are having an undue influence on the final results and consists of assessing the strict and relaxed definitions by including all ratings for each statement and then by excluding one extreme high and one extreme low rating for each statement. The ranges indicated in box 3 include all ratings, and it is noticeable that several of these ranges are from 1 to 9. It may be that these ranges exaggerate the dispersion of the group's response.

Validity and applicability

There has been an active debate on the validity of the Delphi method. For example, Harold Sackman argued that the Delphi method fails to meet the standards normally set for scientific methods. 41 Many of his criticisms were aimed at past studies of poor quality rather than fundamental critiques of the method itself; he particularly criticised poor questionnaire design, inadequate testing of reliability and validity of methods, and the methods of defining and selecting experts. He also argued that the method forces consensus and is weakened by not allowing participants to discuss issues.

Reviews by Pill 1 and by Gerth and Smith (personal communication) showed no clear evidence in favour of meeting based methods over Delphi. Rowe et al, though, concluded that the Delphi technique is generally inferior to the nominal group technique, but state that the degree of inferiority is small, arising more from practical than from theoretical difficulties; they argue for further research aiming to improve the practice of Delphi studies—particularly a careful consideration of what constitutes expertise. 2

Consensus methods, in particular Delphi, have been described as methods of “last resort,” 42 with defenders warning against “overselling” the methods 43 and suggesting that they should be regarded more as methods for structuring group communication than as a means for providing answers. There is clearly a danger that since these approaches have a prescribed method and are often used to generate quantitative estimates, they may lead the casual observer to place greater reliance on their results than might be warranted. As we stated earlier, unless the findings can be tested against observed data, we can never be sure that the methods have produced the “correct” answer. This should be made clear in reporting study results.

The structures of Delphi and nominal groups (shown in box 1 ) aim to maximise the benefits from having informed panels consider a problem (often termed “process gain”) while minimising the disadvantages associated with collective decision making (“process loss”), particularly domination by individuals or professional interests. The extent to which these are realised depends on the ability of those running the studies to use the advantages of the methods. An important role of the facilitator in the nominal group is to ensure that all participants are able to express their views and to keep particular personal or professional views from dominating the discussion; participants in both Delphi and nominal group panels should be selected as to ensure that no particular interest or preconceived opinion is likely to dominate.

Consensus methods provide a useful way of identifying and measuring uncertainty in medical and health services research. Delphi and nominal group techniques have been used to clarify particular issues in health service organisation: to define professional roles, to aid design of educational programmes, to enable long term projections of need for care for particular client groups where there has been considerable uncertainty (for example, for cases of HIV and AIDS 9 ), and to develop criteria for appropriateness of interventions as part of technology assessment. In addition to forming studies in their own right, these techniques have been widely used as component parts of larger projects. 8 31 The two pieces of research from which materials have been presented in this paper each formed part of larger projects: the Delphi exercise 44 was concerned with defining possible adverse effects of reducing junior doctor staffing levels as part of a study of the adequacy of hospital medical staffing levels; the nominal group 23 was concerned with defining appropriate indications for surgical intervention as part of a population based assessment of need for prostate surgery within an NHS region.

Conclusions

The emphasis, when the findings of Delphi and nominal group studies are presented, should be on the justification in using such methods, the use of sound methodology (including selection of experts and the clear definition of target “acceptable” levels of consensus), appropriate presentation of findings (where proposed standards for presentation—as for clinical practice guidelines 45 —should be considered), and on the relevance and systematic use of the results. The output from consensus approaches (including consensus development conferences) is rarely an end in itself. Dissemination and implementation of such findings is the ultimate aim of consensus activities—for example, the publication of consensus statements intended to guide health policy, clinical practice, and research, such as the consensus statement on cancer of the colon and rectum. 46

• McGlynn EA ,
• Kosecoff J ,
• Kalberer JT
• Stocking B ,
• Ouslander JG ,
• Rollinger I ,
• Reuben DB ,
• Bellamy N ,
• Anastassiades TP ,
• Buchanan WW ,
• McCain GA ,
• Bernstein SJ ,
• Hilborne LH ,
• Fraser CE ,
• Elder OC Jr . ,
• Attala JM ,
• Gresley RS ,
• McSweeney N ,
• Moscovice I ,
• Armstrong P ,
• Oranga HM ,
• Rainhorn JD ,
• Brudon-Jakobowicz P ,
• Mobily PR ,
• Passannante MR ,
• Restifo RA ,
• Reichman LB
• Chassin M ,
• Hunter DJW ,
• Sanderson CFB ,
• Technology Subcommittee of the Working Group on Critical Care, Ontario Ministry of Health
• Ziemba SE ,
• Hubbell FA ,
• Battles JB ,
• Dowell DL ,
• Buchan MS ,
• Ginzler M ,
• Justice J ,
• Manning D ,
• Balson PM ,
• Barenberg N ,
• Neidlinger SH ,
• Ljutic PM ,
• Kowalczyk ME ,
• Gallagher M ,
• Spencer J ,
• Bradshaw C ,
• Phillips C ,
• Cats-Baril W
• Van de Ven A
• Kitzinger J
• Linstone HA
• Jones JMG ,
• Hayward RSA ,
• Wilson MC ,
• King's Fund Forum

Advertisement

• Previous Issue
• Previous Article
• Next Article
• Box 1. What to Look for in Research Using This Method

What Is Qualitative Research?

Qualitative versus quantitative research, conducting and appraising qualitative research, conclusions, research support, competing interests, qualitative research methods in medical education.

Submitted for publication January 5, 2018. Accepted for publication November 29, 2018.

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Open the PDF for in another window
• Cite Icon Cite
• Get Permissions
• Search Site

Adam P. Sawatsky , John T. Ratelle , Thomas J. Beckman; Qualitative Research Methods in Medical Education. Anesthesiology 2019; 131:14–22 doi: https://doi.org/10.1097/ALN.0000000000002728

Download citation file:

• Ris (Zotero)
• Reference Manager

Qualitative research was originally developed within the social sciences. Medical education is a field that comprises multiple disciplines, including the social sciences, and utilizes qualitative research to gain a broader understanding of key phenomena within the field. Many clinician educators are unfamiliar with qualitative research. This article provides a primer for clinician educators who want to appraise or conduct qualitative research in medical education. This article discusses a definition and the philosophical underpinnings for qualitative research. Using the Standards for Reporting Qualitative Research as a guide, this article provides a step-wise approach for conducting and evaluating qualitative research in medical education. This review will enable the reader to understand when to utilize qualitative research in medical education and how to interpret reports using qualitative approaches.

Image: J. P. Rathmell and Terri Navarette.

Qualitative research provides approaches to explore and characterize the education of future anesthesiologists. For example, the practice of anesthesiology is increasingly team-based; core members of the anesthesia care team include physicians, trainees, nurse anesthetists, anesthesiologist assistants, and other healthcare team members. 1   Understanding how to work within and how to teach learners about anesthesia care teams requires the ability to conceptualize the complexity of individual psychology and social interactions that occur within teams. Qualitative research is well suited to investigate complex issues like team-based care. For example, one qualitative study observed the interactions between members of the anesthesia care team during simulated stressful situations and conducted interviews of team members; they described limited understanding of each team member’s role and perceptions about appropriate roles and responsibilities, which provided insight for interprofessional team training. 2   Another qualitative study explored the hierarchy within the anesthesia care team, highlighting residents’ reluctance to challenge the established hierarchy and outlining the strategies they use to cope with fear and intimidation. 3   Key issues in medical education and anesthesiology, particularly when exploring human experience and social interactions, may be best studied using qualitative research methodologies and methods.

Medical education is a complex field, and medical education research and practice fittingly draws from many disciplines ( e.g. , medicine, psychology, sociology, education) and synthesizes multiple perspectives to explain how people learn and how medicine should be taught. 4 , 5   The concept of a field was well described by Cristancho and Varpio 5   in their tips for early career medical educators: “A discipline is usually guided by shared paradigms, assumptions, rules and methods to present their knowledge claims— i.e. , people from the same discipline speak the same language. A field brings people from multiple disciplines together.” Qualitative research draws from the perspectives of multiple disciplines and has provided methodologies to explore the complex research questions inherent to medical education.

When appraising qualitative research in medical education, do the authors:

Clearly state the study purpose and research question?

Describe the conceptual framework that inform the study and guide analysis?

Identify their qualitative methodology and research paradigm?

Demonstrate adequate reflexivity, conveying to the reader their values, assumptions and way of thinking, being explicit about the effects these ways of thinking have on the research process?

Choose data collection methods that are congruent with the research purpose and qualitative methodology?

Select an appropriate sampling strategy, choosing participants whose perspectives or experiences are relevant to the study question?

Define their method for determining saturation, how they decided to stop data collection?

Outline their process for data processing, including the management and coding of study data?

Conduct data analysis consistent with their chosen methodology?

Consider techniques to enhance trustworthiness of their study findings?

Synthesize and interpret their data with sufficient detail and supporting quotations to explain the phenomenon of study?

Current medical training is heavily influenced by the practice of evidence-based medicine. 6   Trainees are taught the “hierarchy of evidence” for evaluating studies of clinical interventions. 7   This hierarchy prioritizes knowledge gained through systematic reviews and meta-analyses, randomized controlled trials, and observational studies, but it does not include qualitative research methodologies. This means that because of their medical training and exposure to quantitative medical literature, clinician educators may be more familiar with quantitative research and feel more comfortable engaging in studies utilizing quantitative methodologies. However, many clinician educators are not familiar with the language and application of qualitative research and feel less comfortable engaging in studies using qualitative methodologies.

Because medical education is a diverse and complex field, qualitative research is a common approach in medical education research. Clinician educators who wish to understand the medical education literature need to be familiar with qualitative research. Clinician educators involved in research may also find themselves asking questions best answered by qualitative methodologies. Our goal is to provide a broad, practical overview of qualitative research in medical education. Our objectives are to:

1) Define qualitative research.

2) Compare and contrast qualitative and quantitative research.

3) Provide a framework for conducting and appraising qualitative research in medical education.

Qualitative research in medical education has a distinct vocabulary with terminology not commonly used in other biomedical research fields. Therefore, we have provided a glossary and definitions of the common terms that are used throughout this article ( table 1 ).

Glossary of Common Terms Used in Qualitative Research

Of the many attempts to provide a comprehensive definition of qualitative research, our favorite definition comes from Denzin and Lincoln:

“Qualitative research is a situated activity that locates the observer in the world. Qualitative research consists of a set of interpretive, material practices that make the world visible. These practices…turn the world into a series of representations, including field notes, interviews, conversations, photographs, recordings, and memos to the self. At this level, qualitative research involves an interpretive, naturalistic approach to the world. This means that qualitative researchers study things in their natural settings, attempting to make sense of or interpret phenomena in terms of the meanings people bring to them.” 12  

This definition reveals the following points: first, qualitative research is a “situated activity,” meaning that the research and observations are made in the real world, in this case a real life clinical or educational situation. Second, qualitative research “turns the world into a series of representations” by representing the observations, in this case of a clinical or educational situation, with qualitative data, usually taking the form of words, pictures, documents, and other symbols. Last, qualitative researchers seek to “make sense” of the meanings that research participants bring to different phenomena to allow for a greater understanding of those phenomena. Through qualitative research, observers comprehend participants’ beliefs and values and the way these beliefs and values are shaped by the context in which they are studied.

Because most clinician educators are familiar with quantitative methods, we will start by comparing qualitative and quantitative methods to gain a better understanding of qualitative research ( table 2 ). To illustrate the difference between qualitative and quantitative research in medical education, we pose the question: “What makes noon conference lectures effective for resident learning?” A qualitative approach might explore the learner perspective on learning in noon conference lectures during residency and conduct an exploratory thematic analysis to better understand what the learner thinks is effective. 13   A qualitative approach is useful to answer this question, especially if the phenomenon of interest is incompletely understood. If we wanted to compare types or attributes of conferences to assess the most effective methods of teaching in a noon conference setting, then a quantitative approach might be more appropriate, though a qualitative approach could be helpful as well. We could use qualitative data to inform the design of a survey 14   or even inform the design of a randomized control trial to compare two types of learning during noon conference. 15   Therefore, when discussing qualitative and quantitative research, the issue is not which research approach is stronger, because it is understood that each approach yields different types of knowledge when answering the research question.

Comparisons of Quantitative and Qualitative Research in Medical Education

Similarities

The first step of any research project, qualitative or quantitative, is to determine and refine the study question; this includes conducting a thorough literature review, crafting a problem statement, establishing a conceptual framework for the study, and declaring a statement of intent. 16   A common pitfall in medical education research is to start by identifying the desired methods ( e.g. , “I want to do a focus group study with medical students.”) without having a clearly refined research question, which is like putting the cart before the horse. In other words, the research question should guide the methodology and methods for both qualitative and quantitative research.

Acknowledging the conceptual framework for a study is equally important for both qualitative and quantitative research. In a systematic review of medical education research, only 55% of studies provided a conceptual framework, limiting the interpretation and meaning of the results. 17   Conceptual frameworks are often theories that represent a way of thinking about the phenomenon being studied. Conceptual frameworks guide the interpretation of data and situate the study within the larger body of literature on a specific topic. 9   Because qualitative research was developed within the social sciences, many qualitative research studies in medical education are framed by theories from social sciences. Theories from social science disciplines have the ability to “open up new ways of seeing the world and, in turn, new questions to ask, new assumptions to unearth, and new possibilities for change.” 18   Qualitative research in medical education has benefitted from these new perspectives to help understand fundamental and complex problems within medical education such as culture, power, identity, and meaning.

Differences

The fundamental difference between qualitative and quantitative methodologies centers on epistemology ( i.e. , differing views on truth and knowledge). Cleland 19   describes the differences between qualitative and quantitative philosophies of scientific inquiry: “quantitative and qualitative approaches make different assumptions about the world, about how science should be conducted and about what constitutes legitimate problems, solutions and criteria of ‘proof.’”

Quantitative research comes from objectivism , an epistemology asserting that there is an absolute truth that can be discovered; this way of thinking about knowledge leads researchers to conduct experimental study designs aimed to test hypotheses about cause and effect. 10   Qualitative research, on the other hand, comes from constructivism , an epistemology asserting that reality is constructed by our social, historical, and individual contexts, and leads researchers to utilize more naturalistic or exploratory study designs to provide explanations about phenomenon in the context that they are being studied. 10   This leads researchers to ask fundamentally different questions about a given phenomenon; quantitative research often asks questions of “What?” and “Why?” to understand causation, whereas qualitative research often asks the questions “Why?” and “How?” to understand explanations. Cook et al. 20   provide a framework for classifying the purpose of medical education research to reflect the steps in the scientific method—description (“What was done?”), justification (“Did it work?”), and clarification (“Why or how did it work?”). Qualitative research nicely fits into the categories of “description” and “clarification” by describing observations in natural settings and developing models or theories to help explain “how” and “why” educational methods work. 20  

Another difference between quantitative and qualitative research is the role of the researcher in the research process. Experimental studies have explicitly stated methods for creating an “unbiased” study in which the researcher is detached ( i.e. , “blinded”) from the analysis process so that their biases do not shape the outcome of the research. 21   The term “bias” comes from the positivist paradigm underpinning quantitative research. Assessing and addressing “bias” in qualitative research is incongruous. 22   Qualitative research, based largely on a constructivist paradigm, acknowledges the role of the researcher as a “coconstructer” of knowledge and utilizes the concept of “reflexivity.” Because researchers act as coconstructors of knowledge, they must be explicit about the perspectives they bring to the research process. A reflexive researcher is one who challenges their own values, assumptions, and way of thinking and who is explicit about the effects these ways of thinking have on the research process. 23   For example, when we conducted a study on self-directed learning in residency training, we were overt regarding our roles in the residency program as core faculty, our belief in the importance of self-directed learning, and our assumptions that residents actually engaged in self-directed learning. 24 , 25   We also needed to challenge these assumptions and open ourselves to alternative questions, methods of data collection, and interpretations of the data, to ultimately ensure that we created a research team with varied perspectives. Therefore, qualitative researchers do not strive for “unbiased” research but to understand their own roles in the coconstruction of knowledge. When assessing reflexivity, it is important for the authors to define their roles, explain how those roles may affect the collection and analysis of data, and how the researchers accounted for that effect and, if needed, challenged any assumptions during the research process. Because of the role of the researcher in qualitative research, it is vital to have a member of the research team with qualitative research experience.

A Word on Mixed Methods

In mixed methods research, the researcher collects and analyzes both qualitative and quantitative data rigorously and integrates both forms of data in the results of the study. 26   Medical education research often involves complex questions that may be best addressed through both quantitative and qualitative approaches. Combining methods can complement the strengths and limitations of each method and provide data from multiple sources to create a more detailed understanding of the phenomenon of interest. Examples of uses of mixed methods that would be applicable to medical education research include: collecting qualitative and quantitative data for more complete program evaluation, collecting qualitative data to inform the research design or instrument development of a quantitative study, or collecting qualitative data to explain the meaning behind the results of a quantitative study. 26   The keys to conducting mixed methods studies are to clearly articulate your research questions, explain your rationale for use of each approach, build an appropriate research team, and carefully follow guidelines for methodologic rigor for each approach. 27  

Toward Asking More “Why” Questions

We presented similarities and differences between qualitative and quantitative research to introduce the clinician educator to qualitative research but not to suggest the relative value of one these research methods over the other. Whether conducting qualitative or quantitative research in medical education, researchers should move toward asking more “why” questions to gain deeper understanding of the key phenomena and theories in medical education to move the field of medical education forward. 28   By understanding the theories and assumptions behind qualitative and quantitative research, clinicians can decide how to use these approaches to answer important questions in medical education.

There are substantial differences between qualitative and quantitative research with respect to the assessment of rigor; here we provide a framework for reading, understanding, and assessing the quality of qualitative research. O’Brien et al. 29   created a useful 21-item guide for reporting qualitative research in medical education, based upon a systematic review of reporting standards for qualitative research—the Standards for Reporting Qualitative Research. It should be noted, however, that just performing and reporting each step in these standards do not ensure research quality.

Using the Standards for Reporting Qualitative Research as a backdrop, we will highlight basic steps for clinician educators wanting to engage with qualitative research. If you use this framework to conduct qualitative research in medical education, then you should address these steps; if you are evaluating qualitative research in medical education, then you can assess whether the study investigators addressed these steps. Table 3 underscores each step and provides examples from our research in resident self-directed learning. 25  

Components of Qualitative Research: Examples from a Single Research Study

Refine the study question. As with any research project, investigators should clearly define the topic of research, describe what is already known about the phenomenon that is being studied, identify gaps in the literature, and clearly state how the study will fill that gap. Considering theoretical underpinnings of qualitative research in medical education often means searching for sources outside of the biomedical literature and utilizing theories from education, sociology, psychology, or other disciplines. This is also a critical time to engage people from other disciplines to identify theories or sources of information that can help define the problem and theoretical frameworks for data collection and analysis. When evaluating the introduction of a qualitative study, the researchers should demonstrate a clear understanding of the phenomenon being studied, the previous research on the phenomenon, and conceptual frameworks that contextualize the study. Last, the problem statement and purpose of the study should be clearly stated.

Identify the qualitative methodology and research paradigm. The qualitative methodology should be chosen based on the stated purpose of the research. The qualitative methodology represents the overarching philosophy guiding the collection and analysis of data and is distinct from the research methods ( i.e. , how the data will be collected). There are a number of qualitative methodologies; we have included a list of some of the most common methodologies in table 4 . Choosing a qualitative methodology involves examining the existing literature, involving colleagues with qualitative research expertise, and considering the goals of each approach. 32   For example, explaining the processes, relationships, and theoretical understanding of a phenomenon would point the researcher to grounded theory as an appropriate approach to conducting research. Alternatively, describing the lived experiences of participants may point the researcher to a phenomenological approach. Ultimately, qualitative research should explicitly state the qualitative methodology along with the supporting rationale. Qualitative research is challenging, and you should consult or collaborate with a qualitative research expert as you shape your research question and choose an appropriate methodology. 32  

Choose data collection methods. The choice of data collection methods is driven by the research question, methodology, and practical considerations. Sources of data for qualitative studies would include open-ended survey questions, interviews, focus groups, observations, and documents. Among the most important aspects of choosing the data collection method is alignment with the chosen methodology and study purpose. 33   For interviews and focus groups, there are specific methods for designing the instruments. 34 , 35   Remarkably, these instruments can change throughout the course of the study, because data analysis often informs future data collection in an iterative fashion.

Select a sampling strategy. After identifying the types of data to be collected, the next step is deciding how to sample the data sources to obtain a representative sample. Most qualitative methodologies utilize purposive sampling, which is choosing participants whose perspectives or experiences are relevant to the study question. 11   Although random sampling and convenience sampling may be simpler and less costly for the researcher than purposeful sampling, these approaches often do not provide sufficient information to answer the study question. 36   For example, in grounded theory, theoretical sampling means that the choice of subsequent participants is purposeful to aid in the building and refinement of developing theory. The criteria for selecting participants should be stated clearly. One key difference between qualitative and quantitative research is sample size: in qualitative research, sample size is usually determined during the data collection process, whereas in quantitative research, the sample size is determined a priori . Saturation is verified when the analysis of newly collected data no longer provides additional insights into the data analysis process. 10  

Plan and outline a strategy for data processing. Data processing refers to how the researcher organizes, manages, and dissects the study data. Although data processing serves data analysis, it is not the analysis itself. Data processing includes practical aspects of data management, like transcribing interviews, collecting field notes, and organizing data for analysis. The next step is coding the data, which begins with organizing the raw data into chunks to allow for the identification of themes and patterns. A code is a “word or short phrase that symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a portion of language-based or visual data.” 8   There is an artificial breakdown between data processing and analysis, because these steps may be conducted simultaneously; many consider coding as different from—yet a necessary step to facilitating—the analysis of data. 8   Qualitative software can support this process, by making it easier to organize, access, search, and code your data. However, it is noteworthy that these programs do not do the work for you, they are merely tools for supporting data processing and analysis.

Conduct the data analysis. When analyzing the data, there are several factors to consider. First, the process of data analysis begins with the initial data collection, which often informs future data collection. Researchers should be intentional when reading, reviewing, and analyzing data as it is collected, so that they can shape and enrich subsequent data collection ( e.g. , modify the interview questions). Second, data analysis is often conducted by a research team that should have the appropriate expertise and perspectives to bring to the analysis process. Therefore, when evaluating a qualitative study, you should consider the team’s composition and their reflexivity with respect to their potential biases and influences on their study subjects. Third, the overall goal is to move from the raw data to abstractions of the data that answer the research question. For example, in grounded theory, the research moves from the raw data, to the identification of themes, to categorization of themes, to identifying relationships between themes, and ultimately to the development of theoretical explanations of the phenomenon. 30   Consequently, the primary researcher or research team should be intimately involved with the data analysis, interrogating the data, writing analytic memos, and ultimately make meaning out of the data. There are differing opinions about the use of “counting” of codes or themes in qualitative research. In general, counting of themes is used during the analysis process to recognize patterns and themes; often these are not reported as numbers and percentages as in quantitative research, but may be represented by words like few , some , or many . 37  

Recognize techniques to enhance trustworthiness of your study findings. Ensuring consistency between the data and the results of data analysis, along with ensuring that the data and results accurately represent the perspectives and contexts related to the data source, are crucial to ensuring trustworthiness of study findings. Methods for enhancing trustworthiness include triangulation , which is comparing findings from different methods or perspectives, and member-checking , which is presenting research findings to study participants to provide opportunities to ensure that the analysis is representative. 10  

Synthesize and interpret your data. Synthesis of qualitative research is determined by the depth of the analysis and involves moving beyond description of the data to explaining the findings and situating the results within the larger body of literature on the phenomenon of interest. The reporting of data synthesis should match the research methodology. For instance, if the study is using grounded theory, does the study advance the theoretical understanding of the phenomenon being studied? It is also important to acknowledge that clarity and organization are paramount. 10   Qualitative data are rich and extensive; therefore, researchers must organize and tell a compelling story from the data. 38   This process includes the selection of representative data ( e.g. , quotations from interviews) to substantiate claims made by the research team.

Common Methodologies Used in Qualitative Research

For more information on qualitative research in medical education:

Qualitative Research and Evaluation Methods: Integrating Theory and Practice, by Michael Q. Patton (SAGE Publications, Inc., 2014)

Qualitative Inquiry and Research Design: Choosing Among Five Approaches, by John W. Cresswell (SAGE Publications, Inc. 2017)

Researching Medical Education, by Jennifer Cleland and Steven J. Durning (Wiley-Blackwell, 2015)

Qualitative Research in Medical Education, by Patricia McNally, in Oxford Textbook of Medical Education, edited by Kieren Walsh (Oxford University Press, 2013)

The Journal of Graduate Medical Education “Qualitative Rip Out Series” (Available at: http://www.jgme.org/page/ripouts )

The Standards for Reporting Qualitative Research (O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-51.)

The Wilson Centre Qualitative Atelier (For more information: http://thewilsoncentre.ca/atelier/ )

Qualitative research is commonly used in medical education but may be unfamiliar to many clinician educators. In this article, we provided a definition of qualitative research, explored the similarities and differences between qualitative and quantitative research, and outlined a framework for conducting or appraising qualitative research in medical education. Even with advanced training, it can be difficult for clinician educators to understand and conduct qualitative research. Leaders in medical education research have proposed the following advice to clinician educators wanting to engage in qualitative medical education research: (1) clinician educators should find collaborators with knowledge of theories from other disciplines ( e.g. , sociology, cognitive psychology) and experience in qualitative research to utilize their complementary knowledge and experience to conduct research—in this way, clinician educators can identify important research questions; collaborators can inform research methodology and theoretical perspectives; and (2) clinician educators should engage with a diverse range disciplines to generate new questions and perspectives on research. 4  

Support was provided solely from institutional and/or departmental sources.

The authors declare no competing interests.

Citing articles via

Most viewed, email alerts, related articles, social media, affiliations.

• ASA Practice Parameters
• Online First
• Author Resource Center
• About the Journal
• Editorial Board
• Rights & Permissions
• Online ISSN 1528-1175
• Print ISSN 0003-3022
• Anesthesiology
• ASA Monitor

• Terms & Conditions Privacy Policy
• Manage Cookie Preferences
• © Copyright 2024 American Society of Anesthesiologists

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Goldblatt Medical Library - Florida: Qualitative vs. Quantitative Research

• Goldblatt Library Home
• FRIC Services
• Indian River & Martin Health
• Goldblatt Rules
• Goldblatt Services
• CC Florida Publishing
• Citation Analysis & Journal Rankings This link opens in a new window
• Mendeley Citation Management
• Copyright & Fair Use This link opens in a new window
• Health Statistics
• LGBTQ+ Library Resources
• Library Presentations
• Qualitative vs. Quantitative Research
• Test Prep Books
• UpToDate Mobile
• Weston Nursing
• 2019 Novel Coronavirus - COVID-19

Qualitative vs Quantitative

• Qualitative Studies: General Search Strategies
• Quantitative research

From Eastern Kentucky University Library: https://libanswers.eku.edu/faq/350240

Introduction

Qualitative research is defined as  "research that derives data from observation, interviews, or verbal interactions and focuses on the meanings and interpretations of the participants ." (Holloway and Wheeler, 1995)

Qualitative research can be challenging to find as these methodologies are not always well-indexed in bibliographic databases. This help guide will provide some tips and information to guide you in your search for qualitative research articles through databases, such as  PubMed  and  CINAHL .

For Any Database

Strategy 1: Use Subject Headings 

Databases use controlled keywords (known as thesaurus or subject terms) to categorize each record stored. PubMed, for example, uses Medical Subject Headings (MeSH), a highly structured index of terminology. The subject headings vary for each database according to their indexing system.  The term  "qualitative research"  is indexed in PubMed as "Qualitative Research" or "Nursing Methodology Research", while in CINAHL the subject heading "Qualitative Studies" is complemented by more detailed terms, including "Phenomenological Research" and "Grounded Theory".

Strategy 2: Use Text Words

This strategy uses text or keywords that might specifically identify qualitative research and searches the titles, abstracts and keywords of records held in the databases. Some Text Words include: qualitative, ethnograph*, phenomenol*, ethnonurs*, grounded theor*, purposive sample, hermeneutic*, heuristic*, semiotics, lived experience*, narrative*, life experiences, cluster sample, action research, observational method, content analysis, thematic analysis, constant comparative method, field stud*, theoretical sample, discourse analysis, focus group*, ethnological research, ethnomethodolog*, interview*.

Strategy 3: Use Qualitative Research Filters

Qualitative Research Filters are pre-formulated search strategies that have been constructed by librarians to help you retrieve articles in databases that deal with qualitative research. You can use the filter and then combine the results with your subject.

Tips from University of Washington Health Sciences Library:  https://guides.lib.uw.edu/hsl/qualres/strategies

See below for database search strategies. 

The following are tips that can help you find quantitative research articles. These tips work best when your initial search produces a large amount of results. 

Tip:  Although CINAHL does not have a checkbox to limit your search to either quantitative or qualitative research, it is sometimes helpful to add vocabulary that describes quantitative research tools, methodology or assessments. In the advanced search screen page, you can add a search box that will include words that describe quantitative research.

For example, your quantitative research search statement may read:

quantitative  OR  t-test  OR “ Chi Square ” OR  covariance   OR  anova   OR  instrument*  OR  mean  OR  median  OR  "standard deviation"  OR  coefficient*

 Tip:  CINAHL allows you to filter by publication type.  If your initial search has returned lots of results, you might try using the Publication Type limiter to select only  Clinical Trial  and/or  Randomized Controlled Trial.   These are types of quantitative studies. 

Purdue University West Library:  https://guides.pnw.edu/NUR390/nur390selecting_research_article  

• CINAHL - Qualitative Search Strategies
• PubMed - Qualitative Search Strategies
• Explode the  Qualitative Studies  subject heading, and see what you get. Explode will include the following CINAHL Headings: Action Research, Ethnographic Research, Ethnological Research, Ethnonursing Research, Grounded Theory, Naturalistic Inquiry, Phenomenological Research.
• Experiment with  related CINAHL subject headings  such as Focus Groups, Narratives, Interviews (explode Interviews to include Semi-Structured Interview, Structured Interview, Unstructured Interview), Descriptive Research, Descriptive Statistics, Audiorecording, Videorecording, Diaries, Observational Methods (explode to include Non-Participant Observation, Participant Observation, Structured Categories, Unstructured Categories), Projective Techniques (explodes to include Pictorial Methods, Verbal Projective Techniques), Vignettes, etc.
• Refine/Limit the set to  Clinical Queries . Select within the drop-down Clinical Queries menu: Qualitative-High Sensitivity; Qualitative-High Specificity; Qualitative-Best Balance. High Sensitivity is the broadest search, to include ALL relevant material, but may also include less relevant materials. High Specificity is the most targeted search to include only the most relevant result set, but may miss some relevant materials. Best Balance retrieves the best balance between Sensitivity and Specificity.
• Refine/Limit the set to  Publication Types  that are qualitative by definition, for example Anecdote, Interview, Metasynthesis, etc.
• Use  Text Words  to find articles missed by CINAHL headings (see General Strategies #2)
• Use a  Qualitative Research Filter  for CINAHL. 

Combine these results with your search subject. The search below will automatically be executed for you (the search may be modified once you are in CINAHL):  

(MH “action research”) or (MH "Audiorecording") or (MH "cluster sample+") or (MH "constant comparative method") or (MH "content analysis") or (MH "discourse analysis") or (MH "ethnographic research") or (MH "ethnological research") or (MH "ethnography") or (MH "ethnonursing research") or (MH "field studies") or (MH "focus groups") or (MH "grounded theory") or (MH "Historical Records") or (MH "Interviews+") or (MH "Narratives") or (MH “naturalistic inquiry”) or (MH "observational methods+") or (MH "phenomenological research") or (MH "phenomenology") or (MH "purposive sample") or (MH "qualitative studies") or (MH "qualitative validity+") or (MH "questionnaires") or (MH "thematic analysis") or (MH "theoretical sample") or (MH "Videorecording+") or TX colaizzi* or TX constant comparative or TX constant comparison or TX cooperative inquir* or TX co-operative inquir* or TX co operative inquir* or TX Corbin* TX data saturat* or TX discourse* analysis or  TX emic or TX etic or TX ethnon* or TX field research or TX field stud* or TX focus group* or TX Foucault* or TX giorgi* or TX Glaser* or TX grounded analysis or TX grounded research or TX grounded studies or TX grounded study or TX grounded theor* or TX heidegger* or TX hermeneutic* or TX heuristic or TX human science or TX husserl* or TX life experiences or TX life stor* or TX lived experience* or TX merleau ponty* or TX narrative analysis or TX qualitative or TX participant observ* or TX phenomenol* or TX purpos* sampl* or TX questionnaire* or TX semiotics or TX spiegelberg* or TX Strauss* TX van kaam* or TX van manen*

Mixed Methods Research Design

CINAHL has the following Subject Headings that may be of use:  Multimethod Studies  or  Triangulation

You can also search your topic with the following suggested text words:

• mixed model* or mixed design* or multiple method* or multimethod* or triangulat*

Tips from University of Washington Health Sciences Library:  https://guides.lib.uw.edu/hsl/qualres/cin

• Qualitative Research  [research that derives data from observation, interviews, or verbal interactions and focuses on the meanings and interpretations of the participants. Year introduced 2003]
• Interviews as Topic  [conversations with an individual or individuals held in order to obtain information about their background and other personal biographical data, their attitudes and opinions, etc. It includes school admission or job interviews. Year introduced: 2008 (1980)]
• Focus Groups  [a method of data collection and a qualitative research tool in which a small group of individuals are brought together and allowed to interact in a discussion of their opinions about topics, issues, or questions. Year introduced: 1993]
• Grounded Theory  [The generation of theories from analysis of empirical data. Year introduced 2015]
• Nursing Methodology Research  [research carried out by nurses concerning techniques and methods to implement projects and to document information, including methods of interviewing patients, collecting data, and forming inferences. The concept includes exploration of methodological issues such as human subjectivity and human experience. Year introduced: 1991(1989)]
• Anecdotes as Topic  [brief accounts or narratives of an incident or event. Year introduced: 2008(1963)]
• Narration  [the act, process, or an instance of narrating, i.e., telling a story. In the context of MEDICINE or ETHICS, narration includes relating the particular and the personal in the life story of an individual. Year introduced: 2003]
• Video Recording  [the storing or preserving of video signals for television to be played back later via a transmitter or receiver. Recordings may be made on magnetic tape or discs (VIDEODISC RECORDING). Year introduced: 1984]
• Tape Recording  [recording of information on magnetic or punched paper tape. Year introduced: 1967(1964)]
• Personal Narratives as Topic  [works about accounts of individual experience in relation to a particular field or of participation in related activities. Year introduced: 2013]
• Observational Study as Topic  [A clinical study in which participants may receive diagnostic, therapeutic, or other types of interventions, but the investigator does not assign participants to specific interventions (as in an interventional study). Year introduced: 2014]

NOTE: Inconsistent indexing in PubMed. For example, grounded theory articles are not always indexed for qualitative research. Need to TextWord search for additional terms: “grounded theory”, “action research”, ethnograph* etc.

Additional MeSH terms that may be applicable to your topic include:  Attitude of Health Personnel ;  Attitude to Death ;  Attitude to Health ; or  Health Knowledge, Attitudes, Practice.

• Interview  [work consisting of a conversation with an individual regarding his or her background and other personal and professional details, opinions on specific subjects posed by the interviewer, etc. Year introduced: 2008(1993)]
• Diaries  [works consisting of records, usually private, of writers' experiences, observations, feelings, attitudes, etc. They may also be works marked in calendar order in which to note appointments and the like. (From Random House Unabridged Dictionary, 2d ed) Year introduced: 2008(1997)]
• Anecdotes  [works consisting of brief accounts or narratives of incidents or events. Year introduced: 2008(1999)]
• Personal Narratives  [works consisting of accounts of individual experience in relation to a particular field or of participation in related activities. Year introduced: 2013]
• Observational Study  [A clinical study in which participants may receive diagnostic, therapeutic, or other types of interventions, but the investigator does not assign participants to specific interventions (as in an interventional study).Year introduced: 2014]
• Use Text Words  to find articles missed by MeSH terms (see Strategy 2)
• Select Topic - Specific Queries from the PubMed home page and then Health Services Research Queries.
• This page provides a filter for specialized PubMed searches on healthcare quality and costs.
• Enter your search topic and select Qualitative Research under Category
• 2.   Qualitative Research search filter example  [copy and paste the following modified filter into PubMed and combine your subject terms with this search filter]

(((“semi-structured”[TIAB] OR semistructured[TIAB] OR unstructured[TIAB] OR informal[TIAB] OR “in-depth”[TIAB] OR indepth[TIAB] OR “face-to-face”[TIAB] OR structured[TIAB] OR guide[TIAB] OR guides[TIAB]) AND (interview*[TIAB] OR discussion*[TIAB] OR questionnaire*[TIAB])) OR (“focus group”[TIAB] OR “focus groups”[TIAB] OR qualitative[TIAB] OR ethnograph*[TIAB] OR fieldwork[TIAB] OR “field work”[TIAB] OR “key informant”[TIAB])) OR “interviews as topic”[Mesh] OR “focus groups”[Mesh] OR narration[Mesh] OR qualitative research[Mesh] OR "personal narratives as topic"[Mesh] OR (theme[TIAB] OR thematic[TIAB]) OR "ethnological research"[TIAB] OR phenomenol*[TIAB] OR "grounded theory" [TIAB]  OR "grounded study" [TIAB]  OR "grounded studies" [TIAB]  OR "grounded research" [TIAB]  OR "grounded analysis"[TIAB] OR "grounded analyses" [TIAB]  OR "life story" [TIAB]  OR "life stories"[TIAB] OR emic[TIAB] OR etic[TIAB] OR hermeneutics[TIAB] OR heuristic*[TIAB] OR semiotic[TIAB] OR "data saturation"[TIAB] OR "participant observation"[TIAB] OR "action research"[TIAB] OR "cooperative inquiry" [TIAB]  OR  "co-operative inquiry" [TIAB]  OR "field study" [TIAB] OR "field studies"[TIAB] OR "field  research"[TIAB] OR "theoretical sample"[TIAB] OR "theoretical samples" [TIAB] OR "theoretical sampling"[TIAB]  OR "purposive sampling"[TIAB] OR   "purposive sample"[TIAB] OR "purposive samples"[TIAB]   OR "lived experience"[TIAB] OR "lived experiences"[TIAB] OR  "purposive sampling"[TIAB]   OR "content analysis"[TIAB] OR discourse[TIAB] OR "narrative analysis"[TIAB] OR heidegger*[TIAB] OR colaizzi[TIAB] OR spiegelberg[TIAB] OR "van manen*"[TIAB] OR "van kaam"[TIAB] OR "merleau ponty"[TIAB] OR husserl*[TIAB] OR Foucault[TIAB] or Corbin[TIAB] OR Strauss[TIAB] OR Glaser[TIAB] 

PubMed does not have suitable MeSH terms for mixed methods research. Search your topic with the following suggested text words using the quotes and truncation symbol*:

• “mixed model*” OR “mixed design*” OR “multiple method*” OR multimethod* OR triangulat*

Tips from University of Washington Health Sciences Library:  https://guides.lib.uw.edu/hsl/qualres/pubmed

• << Previous: Library Presentations
• Next: Test Prep Books >>
• Last Updated: May 16, 2024 10:52 AM
• URL: https://my.clevelandclinic.libguides.com/floridaOLD

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Quantitative vs. Qualitative Research in Psychology

Anabelle Bernard Fournier is a researcher of sexual and reproductive health at the University of Victoria as well as a freelance writer on various health topics.

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

• Key Differences

Quantitative Research Methods

Qualitative research methods.

• How They Relate

In psychology and other social sciences, researchers are faced with an unresolved question: Can we measure concepts like love or racism the same way we can measure temperature or the weight of a star? Social phenomena⁠—things that happen because of and through human behavior⁠—are especially difficult to grasp with typical scientific models.

At a Glance

Psychologists rely on quantitative and quantitative research to better understand human thought and behavior.

• Qualitative research involves collecting and evaluating non-numerical data in order to understand concepts or subjective opinions.
• Quantitative research involves collecting and evaluating numerical data. 

This article discusses what qualitative and quantitative research are, how they are different, and how they are used in psychology research.

Qualitative Research vs. Quantitative Research

In order to understand qualitative and quantitative psychology research, it can be helpful to look at the methods that are used and when each type is most appropriate.

Psychologists rely on a few methods to measure behavior, attitudes, and feelings. These include:

• Self-reports , like surveys or questionnaires
• Observation (often used in experiments or fieldwork)
• Implicit attitude tests that measure timing in responding to prompts

Most of these are quantitative methods. The result is a number that can be used to assess differences between groups.

However, most of these methods are static, inflexible (you can't change a question because a participant doesn't understand it), and provide a "what" answer rather than a "why" answer.

Sometimes, researchers are more interested in the "why" and the "how." That's where qualitative methods come in.

Qualitative research is about speaking to people directly and hearing their words. It is grounded in the philosophy that the social world is ultimately unmeasurable, that no measure is truly ever "objective," and that how humans make meaning is just as important as how much they score on a standardized test.

Used to develop theories

Takes a broad, complex approach

Answers "why" and "how" questions

Explores patterns and themes

Used to test theories

Takes a narrow, specific approach

Answers "what" questions

Explores statistical relationships

Quantitative methods have existed ever since people have been able to count things. But it is only with the positivist philosophy of Auguste Comte (which maintains that factual knowledge obtained by observation is trustworthy) that it became a "scientific method."

The scientific method follows this general process. A researcher must:

• Generate a theory or hypothesis (i.e., predict what might happen in an experiment) and determine the variables needed to answer their question
• Develop instruments to measure the phenomenon (such as a survey, a thermometer, etc.)
• Develop experiments to manipulate the variables
• Collect empirical (measured) data
• Analyze data

Quantitative methods are about measuring phenomena, not explaining them.

Quantitative research compares two groups of people. There are all sorts of variables you could measure, and many kinds of experiments to run using quantitative methods.

These comparisons are generally explained using graphs, pie charts, and other visual representations that give the researcher a sense of how the various data points relate to one another.

Basic Assumptions

Quantitative methods assume:

• That the world is measurable
• That humans can observe objectively
• That we can know things for certain about the world from observation

In some fields, these assumptions hold true. Whether you measure the size of the sun 2000 years ago or now, it will always be the same. But when it comes to human behavior, it is not so simple.

As decades of cultural and social research have shown, people behave differently (and even think differently) based on historical context, cultural context, social context, and even identity-based contexts like gender , social class, or sexual orientation .

Therefore, quantitative methods applied to human behavior (as used in psychology and some areas of sociology) should always be rooted in their particular context. In other words: there are no, or very few, human universals.

Statistical information is the primary form of quantitative data used in human and social quantitative research. Statistics provide lots of information about tendencies across large groups of people, but they can never describe every case or every experience. In other words, there are always outliers.

Correlation and Causation

A basic principle of statistics is that correlation is not causation. Researchers can only claim a cause-and-effect relationship under certain conditions:

• The study was a true experiment.
• The independent variable can be manipulated (for example, researchers cannot manipulate gender, but they can change the primer a study subject sees, such as a picture of nature or of a building).
• The dependent variable can be measured through a ratio or a scale.

So when you read a report that "gender was linked to" something (like a behavior or an attitude), remember that gender is NOT a cause of the behavior or attitude. There is an apparent relationship, but the true cause of the difference is hidden.

Pitfalls of Quantitative Research

Quantitative methods are one way to approach the measurement and understanding of human and social phenomena. But what's missing from this picture?

As noted above, statistics do not tell us about personal, individual experiences and meanings. While surveys can give a general idea, respondents have to choose between only a few responses. This can make it difficult to understand the subtleties of different experiences.

Quantitative methods can be helpful when making objective comparisons between groups or when looking for relationships between variables. They can be analyzed statistically, which can be helpful when looking for patterns and relationships.

Qualitative data are not made out of numbers but rather of descriptions, metaphors, symbols, quotes, analysis, concepts, and characteristics. This approach uses interviews, written texts, art, photos, and other materials to make sense of human experiences and to understand what these experiences mean to people.

While quantitative methods ask "what" and "how much," qualitative methods ask "why" and "how."

Qualitative methods are about describing and analyzing phenomena from a human perspective. There are many different philosophical views on qualitative methods, but in general, they agree that some questions are too complex or impossible to answer with standardized instruments.

These methods also accept that it is impossible to be completely objective in observing phenomena. Researchers have their own thoughts, attitudes, experiences, and beliefs, and these always color how people interpret results.

Qualitative Approaches

There are many different approaches to qualitative research, with their own philosophical bases. Different approaches are best for different kinds of projects. For example:

• Case studies and narrative studies are best for single individuals. These involve studying every aspect of a person's life in great depth.
• Phenomenology aims to explain experiences. This type of work aims to describe and explore different events as they are consciously and subjectively experienced.
• Grounded theory develops models and describes processes. This approach allows researchers to construct a theory based on data that is collected, analyzed, and compared to reach new discoveries.
• Ethnography describes cultural groups. In this approach, researchers immerse themselves in a community or group in order to observe behavior.

Qualitative researchers must be aware of several different methods and know each thoroughly enough to produce valuable research.

Some researchers specialize in a single method, but others specialize in a topic or content area and use many different methods to explore the topic, providing different information and a variety of points of view.

There is not a single model or method that can be used for every qualitative project. Depending on the research question, the people participating, and the kind of information they want to produce, researchers will choose the appropriate approach.

Interpretation

Qualitative research does not look into causal relationships between variables, but rather into themes, values, interpretations, and meanings. As a rule, then, qualitative research is not generalizable (cannot be applied to people outside the research participants).

The insights gained from qualitative research can extend to other groups with proper attention to specific historical and social contexts.

Relationship Between Qualitative and Quantitative Research

It might sound like quantitative and qualitative research do not play well together. They have different philosophies, different data, and different outputs. However, this could not be further from the truth.

These two general methods complement each other. By using both, researchers can gain a fuller, more comprehensive understanding of a phenomenon.

For example, a psychologist wanting to develop a new survey instrument about sexuality might and ask a few dozen people questions about their sexual experiences (this is qualitative research). This gives the researcher some information to begin developing questions for their survey (which is a quantitative method).

After the survey, the same or other researchers might want to dig deeper into issues brought up by its data. Follow-up questions like "how does it feel when...?" or "what does this mean to you?" or "how did you experience this?" can only be answered by qualitative research.

By using both quantitative and qualitative data, researchers have a more holistic, well-rounded understanding of a particular topic or phenomenon.

Qualitative and quantitative methods both play an important role in psychology. Where quantitative methods can help answer questions about what is happening in a group and to what degree, qualitative methods can dig deeper into the reasons behind why it is happening. By using both strategies, psychology researchers can learn more about human thought and behavior.

Gough B, Madill A. Subjectivity in psychological science: From problem to prospect . Psychol Methods . 2012;17(3):374-384. doi:10.1037/a0029313

Pearce T. “Science organized”: Positivism and the metaphysical club, 1865–1875 . J Hist Ideas . 2015;76(3):441-465.

Adams G. Context in person, person in context: A cultural psychology approach to social-personality psychology . In: Deaux K, Snyder M, eds. The Oxford Handbook of Personality and Social Psychology . Oxford University Press; 2012:182-208.

Brady HE. Causation and explanation in social science . In: Goodin RE, ed. The Oxford Handbook of Political Science. Oxford University Press; 2011. doi:10.1093/oxfordhb/9780199604456.013.0049

Chun Tie Y, Birks M, Francis K. Grounded theory research: A design framework for novice researchers .  SAGE Open Med . 2019;7:2050312118822927. doi:10.1177/2050312118822927

Reeves S, Peller J, Goldman J, Kitto S. Ethnography in qualitative educational research: AMEE Guide No. 80 . Medical Teacher . 2013;35(8):e1365-e1379. doi:10.3109/0142159X.2013.804977

Salkind NJ, ed. Encyclopedia of Research Design . Sage Publishing.

Shaughnessy JJ, Zechmeister EB, Zechmeister JS.  Research Methods in Psychology . McGraw Hill Education.

By Anabelle Bernard Fournier Anabelle Bernard Fournier is a researcher of sexual and reproductive health at the University of Victoria as well as a freelance writer on various health topics.

• Open access
• Published: 16 May 2024

Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts

• Jeremy Segrott   ORCID: orcid.org/0000-0001-6215-0870 1 ,
• Sue Channon 2 ,
• Amy Lloyd 4 ,
• Eleni Glarou 2 , 3 ,
• Josie Henley 5 ,
• Jacqueline Hughes 2 ,
• Nina Jacob 2 ,
• Sarah Milosevic 2 ,
• Yvonne Moriarty 2 ,
• Bethan Pell 6 ,
• Mike Robling 2 ,
• Heather Strange 2 ,
• Julia Townson 2 ,
• Qualitative Research Group &
• Lucy Brookes-Howell 2  

Trials volume  25 , Article number:  323 ( 2024 ) Cite this article

Metrics details

Background/aims

The value of using qualitative methods within clinical trials is widely recognised. How qualitative research is integrated within trials units to achieve this is less clear. This paper describes the process through which qualitative research has been integrated within Cardiff University’s Centre for Trials Research (CTR) in Wales, UK. We highlight facilitators of, and challenges to, integration.

We held group discussions on the work of the Qualitative Research Group (QRG) within CTR. The content of these discussions, materials for a presentation in CTR, and documents relating to the development of the QRG were interpreted at a workshop attended by group members. Normalisation Process Theory (NPT) was used to structure analysis. A writing group prepared a document for input from members of CTR, forming the basis of this paper.

Actions to integrate qualitative research comprised: its inclusion in Centre strategies; formation of a QRG with dedicated funding/roles; embedding of qualitative research within operating systems; capacity building/training; monitoring opportunities to include qualitative methods in studies; maximising the quality of qualitative research and developing methodological innovation. Facilitators of these actions included: the influence of the broader methodological landscape within trial/study design and its promotion of the value of qualitative research; and close physical proximity of CTR qualitative staff/students allowing sharing of methodological approaches. Introduction of innovative qualitative methods generated interest among other staff groups. Challenges included: pressure to under-resource qualitative components of research, preference for a statistical stance historically in some research areas and funding structures, and difficulties faced by qualitative researchers carving out individual academic profiles when working across trials/studies.

Conclusions

Given that CTUs are pivotal to the design and conduct of RCTs and related study types across multiple disciplines, integrating qualitative research into trials units is crucial if its contribution is to be fully realised. We have made explicit one trials unit’s experience of embedding qualitative research and present this to open dialogue on ways to operationalise and optimise qualitative research in trials. NPT provides a valuable framework with which to theorise these processes, including the importance of sense-making and legitimisation when introducing new practices within organisations.

Peer Review reports

The value of using qualitative methods within randomised control trials (RCTs) is widely recognised [ 1 , 2 , 3 ]. Qualitative research generates important evidence on factors affecting trial recruitment/retention [ 4 ] and implementation, aiding interpretation of quantitative data [ 5 ]. Though RCTs have traditionally been viewed as sitting within a positivist paradigm, recent methodological innovations have developed new trial designs that draw explicitly on both quantitative and qualitative methods. For instance, in the field of complex public health interventions, realist RCTs seek to understand the mechanisms through which interventions generate hypothesised impacts, and how interactions across different implementation contexts form part of these mechanisms. Proponents of realist RCTs—which integrate experimental and realist paradigms—highlight the importance of using quantitative and qualitative methods to fully realise these aims and to generate an understanding of intervention mechanisms and how context shapes them [ 6 ].

A need for guidance on how to conduct good quality qualitative research is being addressed, particularly in relation to feasibility studies for RCTs [ 7 ] and process evaluations embedded within trials of complex interventions [ 5 ]. There is also guidance on the conduct of qualitative research within trials at different points in the research cycle, including development, conduct and reporting [ 8 , 9 ].

A high proportion of trials are based within or involve clinical trials units (CTUs). In the UK the UKCRC Registered CTU Network describes them as:

… specialist units which have been set up with a specific remit to design, conduct, analyse and publish clinical trials and other well-designed studies. They have the capability to provide specialist expert statistical, epidemiological, and other methodological advice and coordination to undertake successful clinical trials. In addition, most CTUs will have expertise in the coordination of trials involving investigational medicinal products which must be conducted in compliance with the UK Regulations governing the conduct of clinical trials resulting from the EU Directive for Clinical Trials.

Thus, CTUs provide the specialist methodological expertise needed for the conduct of trials, and in the case of trials of investigational medicinal products, their involvement may be mandated to ensure compliance with relevant regulations. As the definition above suggests, CTUs also conduct and support other types of study apart from RCTs, providing a range of methodological and subject-based expertise.

However, despite their central role in the conduct and design of trials, (and other evaluation designs) little has been written about how CTUs have integrated qualitative work within their organisation at a time when such methods are, as stated above, now recognised as an important aspect of RCTs and evaluation studies more generally. This is a significant gap, since integration at the organisational level arguably shapes how qualitative research is integrated within individual studies, and thus it is valuable to understand how CTUs have approached the task. There are different ways of involving qualitative work in trials units, such as partnering with other departments (e.g. social science) or employing qualitative researchers directly. Qualitative research can be imagined and configured in different ways—as a method that generates data to inform future trial and intervention design, as an embedded component within an RCT or other evaluation type, or as a parallel strand of research focusing on lived experiences of illness, for instance. Understanding how trials units have integrated qualitative research is valuable, as it can shed light on which strategies show promise, and in which contexts, and how qualitative research is positioned within the field of trials research, foregrounding the value of qualitative research. However, although much has been written about its use within trials, few accounts exist of how trials units have integrated qualitative research within their systems and structures.

This paper discusses the process of embedding qualitative research within the work of one CTU—Cardiff University’s Centre for Trials Research (CTR). It highlights facilitators of this process and identifies challenges to integration. We use the Normalisation Process Theory (NPT) as a framework to structure our experience and approach. The key gap addressed by this paper is the implementation of strategies to integrate qualitative research (a relatively newly adopted set of practices and processes) within CTU systems and structures. We acknowledge from the outset that there are multiple ways of approaching this task. What follows therefore is not a set of recommendations for a preferred or best way to integrate qualitative research, as this will comprise diverse actions according to specific contexts. Rather, we examine the processes through which integration occurred in our own setting and highlight the potential value of these insights for others engaged in the work of promoting qualitative research within trials units.

Background to the integration of qualitative research within CTR

The CTR was formed in 2015 [ 10 ]. It brought together three existing trials units at Cardiff University: the South East Wales Trials Unit, the Wales Cancer Trials Unit, and the Haematology Clinical Trials Unit. From its inception, the CTR had a stated aim of developing a programme of qualitative research and integrating it within trials and other studies. In the sections below, we map these approaches onto the framework offered by Normalisation Process Theory to understand the processes through which they helped achieve embedding and integration of qualitative research.

CTR’s aims (including those relating to the development of qualitative research) were included within its strategy documents and communicated to others through infrastructure funding applications, annual reports and its website. A Qualitative Research Group (QRG), which had previously existed within the South East Wales Trials Unit, with dedicated funding for methodological specialists and group lead academics, was a key mechanism through which the development of a qualitative portfolio was put into action. Integration of qualitative research within Centre systems and processes occurred through the inclusion of qualitative research in study adoption processes and representation on committees. The CTR’s study portfolio provided a basis to track qualitative methods in new and existing studies, identify opportunities to embed qualitative methods within recently adopted studies (at the funding application stage) and to manage staff resources. Capacity building and training were an important focus of the QRG’s work, including training courses, mentoring, creation of an academic network open to university staff and practitioners working in the field of healthcare, presentations at CTR staff meetings and securing of PhD studentships. Standard operating procedures and methodological guidance on the design and conduct of qualitative research (e.g. templates for developing analysis plans) aimed to create a shared understanding of how to undertake high-quality research, and a means to monitor the implementation of rigorous approaches. As the QRG expanded its expertise it sought to develop innovative approaches, including the use of visual [ 11 ] and ethnographic methods [ 12 ].

Understanding implementation—Normalisation Process Theory (NPT)

Normalisation Process Theory (NPT) provides a model with which to understand the implementation of new sets of practices and their normalisation within organisational settings. The term ‘normalisation’ refers to how new practices become routinised (part of the everyday work of an organisation) through embedding and integration [ 13 , 14 ]. NPT defines implementation as ‘the social organisation of work’ and is concerned with the social processes that take place as new practices are introduced. Embedding involves ‘making practices routine elements of everyday life’ within an organisation. Integration takes the form of ‘sustaining embedded practices in social contexts’, and how these processes lead to the practices becoming (or not becoming) ‘normal and routine’ [ 14 ]. NPT is concerned with the factors which promote or ‘inhibit’ attempts to embed and integrate the operationalisation of new practices [ 13 , 14 , 15 ].

Embedding new practices is therefore achieved through implementation—which takes the form of interactions in specific contexts. Implementation is operationalised through four ‘generative mechanisms’— coherence , cognitive participation , collective action and reflexive monitoring [ 14 ]. Each mechanism is characterised by components comprising immediate and organisational work, with actions of individuals and organisations (or groups of individuals) interdependent. The mechanisms operate partly through forms of investment (i.e. meaning, commitment, effort, and comprehension) [ 14 ].

Coherence refers to how individuals/groups make sense of, and give meaning to, new practices. Sense-making concerns the coherence of a practice—whether it ‘holds together’, and its differentiation from existing activities [ 15 ]. Communal and individual specification involve understanding new practices and their potential benefits for oneself or an organisation. Individuals consider what new practices mean for them in terms of tasks and responsibilities ( internalisation ) [ 14 ].

NPT frames the second mechanism, cognitive participation , as the building of a ‘community of practice’. For a new practice to be initiated, individuals and groups within an organisation must commit to it [ 14 , 15 ]. Cognitive participation occurs through enrolment —how people relate to the new practice; legitimation —the belief that it is right for them to be involved; and activation —defining which actions are necessary to sustain the practice and their involvement [ 14 ]. Making the new practices work may require changes to roles (new responsibilities, altered procedures) and reconfiguring how colleagues work together (changed relationships).

Third, Collective Action refers to ‘the operational work that people do to enact a set of practices’ [ 14 ]. Individuals engage with the new practices ( interactional workability ) reshaping how members of an organisation interact with each other, through creation of new roles and expectations ( relational interaction ) [ 15 ]. Skill set workability concerns how the work of implementing a new set of practices is distributed and the necessary roles and skillsets defined [ 14 ]. Contextual integration draws attention to the incorporation of a practice within social contexts, and the potential for aspects of these contexts, such as systems and procedures, to be modified as a result [ 15 ].

Reflexive monitoring is the final implementation mechanism. Collective and individual appraisal evaluate the value of a set of practices, which depends on the collection of information—formally and informally ( systematisation ). Appraisal may lead to reconfiguration in which procedures of the practice are redefined or reshaped [ 14 , 15 ].

We sought to map the following: (1) the strategies used to embed qualitative research within the Centre, (2) key facilitators, and (3) barriers to their implementation. Through focused group discussions during the monthly meetings of the CTR QRG and in discussion with the CTR senior management team throughout 2019–2020 we identified nine types of documents (22 individual documents in total) produced within the CTR which had relevant information about the integration of qualitative research within its work (Table  1 ). The QRG had an ‘open door’ policy to membership and welcomed all staff/students with an interest in qualitative research. It included researchers who were employed specifically to undertake qualitative research and other staff with a range of study roles, including trial managers, statisticians, and data managers. There was also diversity in terms of career stage, including PhD students, mid-career researchers and members of the Centre’s Executive team. Membership was therefore largely self-selected, and comprised of individuals with a role related to, or an interest in, embedding qualitative research within trials. However, the group brought together diverse methodological perspectives and was not solely comprised of methodological ‘champions’ whose job it was to promote the development of qualitative research within the centre. Thus whilst the group (and by extension, the authors of this paper) had a shared appreciation of the value of qualitative research within a trials centre, they also brought varied methodological perspectives and ways of engaging with it.

All members of the QRG ( n  = 26) were invited to take part in a face-to-face, day-long workshop in February 2019 on ‘How to optimise and operationalise qualitative research in trials: reflections on CTR structure’. The workshop was attended by 12 members of staff and PhD students, including members of the QRG and the CTR’s senior management team. Recruitment to the workshop was therefore inclusive, and to some extent opportunistic, but all members of the QRG were able to contribute to discussions during regular monthly group meetings and the drafting of the current paper.

The aim of the workshop was to bring together information from the documents in Table  1 to generate discussion around the key strategies (and their component activities) that had been adopted to integrate qualitative research into CTR, as well as barriers to, and facilitators of, their implementation. The agenda for the workshop involved four key areas: development and history of the CTR model; mapping the current model within CTR; discussing the structure of other CTUs; and exploring the advantages and disadvantages of the CTR model.

During the workshop, we discussed the use of NPT to conceptualise how qualitative research had been embedded within CTR’s systems and practices. The group produced spider diagrams to map strategies and actions on to the four key domains (or ‘generative mechanisms’ of NPT) summarised above, to aid the understanding of how they had functioned, and the utility of NPT as a framework. This is summarised in Table  2 .

Detailed notes were made during the workshop. A core writing group then used these notes and the documents in Table  1 to develop a draft of the current paper. This was circulated to all members of the CTR QRG ( n  = 26) and stored within a central repository accessible to them to allow involvement and incorporate the views of those who were not able to attend the workshop. This draft was again presented for comments in the monthly CTR QRG meeting in February 2021 attended by n  = 10. The Standards for QUality Improvement Reporting Excellence 2.0 (SQUIRE) guidelines were used to inform the structure and content of the paper (see supplementary material) [ 16 ].

In the following sections, we describe the strategies CTR adopted to integrate qualitative research. These are mapped against NPT’s four generative mechanisms to explore the processes through which the strategies promoted integration, and facilitators of and barriers to their implementation. A summary of the strategies and their functioning in terms of the generative mechanisms is provided in Table  2 .

Coherence—making sense of qualitative research

In CTR, many of the actions taken to build a portfolio of qualitative research were aimed at enabling colleagues, and external actors, to make sense of this set of methodologies. Centre-level strategies and grant applications for infrastructure funding highlighted the value of qualitative research, the added benefits it would bring, and positioned it as a legitimate set of practices alongside existing methods. For example, a 2014 application for renewal of trials unit infrastructure funding stated:

We are currently in the process of undertaking […] restructuring for our qualitative research team and are planning similar for trial management next year. The aim of this restructuring is to establish greater hierarchical management and opportunities for staff development and also provide a structure that can accommodate continuing growth.

Within the CTR, various forms of communication on the development of qualitative research were designed to enable staff and students to make sense of it, and to think through its potential value for them, and ways in which they might engage with it. These included presentations at staff meetings, informal meetings between project teams and the qualitative group lead, and the visibility of qualitative research on the public-facing Centre website and Centre committees and systems. For instance, qualitative methods were included (and framed as a distinct set of practices) within study adoption forms and committee agendas. Information for colleagues described how qualitative methods could be incorporated within funding applications for RCTs and other evaluation studies to generate new insights into questions research teams were already keen to answer, such as influences on intervention implementation fidelity. Where externally based chief investigators approached the Centre to be involved in new grant applications, the existence of the qualitative team and group lead enabled the inclusion of qualitative research to be actively promoted at an early stage, and such opportunities were highlighted in the Centre’s brochure for new collaborators. Monthly qualitative research network meetings—advertised across CTR and to external research collaborators, were also designed to create a shared understanding of qualitative research methods and their utility within trials and other study types (e.g. intervention development, feasibility studies, and observational studies). Training events (discussed in more detail below) also aided sense-making.

Several factors facilitated the promotion of qualitative research as a distinctive and valuable entity. Among these was the influence of the broader methodological landscape within trial design which was promoting the value of qualitative research, such as guidance on the evaluation of complex interventions by the Medical Research Council [ 17 ], and the growing emphasis placed on process evaluations within trials (with qualitative methods important in understanding participant experience and influences on implementation) [ 5 ]. The attention given to lived experience (both through process evaluations and the move to embed public involvement in trials) helped to frame qualitative research within the Centre as something that was appropriate, legitimate, and of value. Recognition by research funders of the value of qualitative research within studies was also helpful in normalising and legitimising its adoption within grant applications.

The inclusion of qualitative methods within influential methodological guidance helped CTR researchers to develop a ‘shared language’ around these methods, and a way that a common understanding of the role of qualitative research could be generated. One barrier to such sense-making work was the varying extent to which staff and teams had existing knowledge or experience of qualitative research. This varied across methodological and subject groups within the Centre and reflected the history of the individual trials units which had merged to form the Centre.

Cognitive participation—legitimising qualitative research

Senior CTR leaders promoted the value and legitimacy of qualitative research. Its inclusion in centre strategies, infrastructure funding applications, and in public-facing materials (e.g. website, investigator brochures), signalled that it was appropriate for individuals to conduct qualitative research within their roles, or to support others in doing so. Legitimisation also took place through informal channels, such as senior leadership support for qualitative research methods in staff meetings and participation in QRG seminars. Continued development of the QRG (with dedicated infrastructure funding) provided a visible identity and equivalence with other methodological groups (e.g. trial managers, statisticians).

Staff were asked to engage with qualitative research in two main ways. First, there was an expansion in the number of staff for whom qualitative research formed part of their formal role and responsibilities. One of the three trials units that merged to form CTR brought with it a qualitative team comprising methodological specialists and a group lead. CTR continued the expansion of this group with the creation of new roles and an enlarged nucleus of researchers for whom qualitative research was the sole focus of their work. In part, this was linked to the successful award of projects that included a large qualitative component, and that were coordinated by CTR (see Table  3 which describes the PUMA study).

Members of the QRG were encouraged to develop their own research ideas and to gain experience as principal investigators, and group seminars were used to explore new ideas and provide peer support. This was communicated through line management, appraisal, and informal peer interaction. Boundaries were not strictly demarcated (i.e. staff located outside the qualitative team were already using qualitative methods), but the new team became a central focus for developing a growing programme of work.

Second, individuals and studies were called upon to engage in new ways with qualitative research, and with the qualitative team. A key goal for the Centre was that groups developing new research ideas should give more consideration in general to the potential value and inclusion of qualitative research within their funding applications. Specifically, they were asked to do this by thinking about qualitative research at an early point in their application’s development (rather than ‘bolting it on’ after other elements had been designed) and to draw upon the expertise and input of the qualitative team. An example was the inclusion of questions on qualitative methods within the Centre’s study adoption form and representation from the qualitative team at the committee which reviewed new adoption requests. Where adoption requests indicated the inclusion of qualitative methods, colleagues were encouraged to liaise with the qualitative team, facilitating the integration of its expertise from an early stage. Qualitative seminars offered an informal and supportive space in which researchers could share initial ideas and refine their methodological approach. The benefits of this included the provision of sufficient time for methodological specialists to be involved in the design of the proposed qualitative component and ensuring adequate costings had been drawn up. At study adoption group meetings, scrutiny of new proposals included consideration of whether new research proposals might be strengthened through the use of qualitative methods where these had not initially been included. Meetings of the QRG—which reviewed the Centre’s portfolio of new studies and gathered intelligence on new ideas—also helped to identify, early on, opportunities to integrate qualitative methods. Communication across teams was useful in identifying new research ideas and embedding qualitative researchers within emerging study development groups.

Actions to promote greater use of qualitative methods in funding applications fed through into a growing number of studies with a qualitative component. This helped to increase the visibility and legitimacy of qualitative methods within the Centre. For example, the PUMA study [ 12 ], which brought together a large multidisciplinary team to develop and evaluate a Paediatric early warning system, drew heavily on qualitative methods, with the qualitative research located within the QRG. The project introduced an extensive network of collaborators and clinical colleagues to qualitative methods and how they could be used during intervention development and the generation of case studies. Further information about the PUMA study is provided in Table  3 .

Increasing the legitimacy of qualitative work across an extensive network of staff, students and collaborators was a complex process. Set within the continuing dominance of quantitative methods with clinical trials, there were variations in the extent to which clinicians and other collaborators embraced the value of qualitative methods. Research funding schemes, which often continued to emphasise the quantitative element of randomised controlled trials, inevitably fed through into the focus of new research proposals. Staff and external collaborators were sometimes uncertain about the added value that qualitative methods would bring to their trials. Across the CTR there were variations in the speed at which qualitative research methods gained legitimacy, partly based on disciplinary traditions and their influences. For instance, population health trials, often located within non-health settings such as schools or community settings, frequently involved collaboration with social scientists who brought with them experience in qualitative methods. Methodological guidance in this field, such as MRC guidance on process evaluations, highlighted the value of qualitative methods and alternatives to the positivist paradigm, such as the value of realist RCTs. In other, more clinical areas, positivist paradigms had greater dominance. Established practices and methodological traditions across different funders also influenced the ease of obtaining funding to include qualitative research within studies. For drugs trials (CTIMPs), the influence of regulatory frameworks on study design, data collection and the allocation of staff resources may have played a role. Over time, teams gained repeated experience of embedding qualitative research (and researchers) within their work and took this learning with them to subsequent studies. For example, the senior clinician quoted within the PUMA case study (Table  3 below) described how they had gained an appreciation of the rigour of qualitative research and an understanding of its language. Through these repeated interactions, embedding of qualitative research within studies started to become the norm rather than the exception.

Collective action—operationalising qualitative research

Collective action concerns the operationalisation of new practices within organisations—the allocation and management of the work, how individuals interact with each other, and the work itself. In CTR the formation of a Qualitative Research Group helped to allocate and organise the work of building a portfolio of studies. Researchers across the Centre were called upon to interact with qualitative research in new ways. Presentations at staff meetings and the inclusion of qualitative research methods in portfolio study adoption forms were examples of this ( interactive workability ). It was operationalised by encouraging study teams to liaise with the qualitative research lead. Development of standard operating procedures, templates for costing qualitative research and methodological guidance (e.g. on analysis plans) also helped encourage researchers to interact with these methods in new ways. For some qualitative researchers who had been trained in the social sciences, working within a trials unit meant that they needed to interact in new and sometimes unfamiliar ways with standard operating procedures, risk assessments, and other trial-based systems. Thus, training needs and capacity-building efforts were multidirectional.

Whereas there had been a tendency for qualitative research to be ‘bolted on’ to proposals for RCTs, the systems described above were designed to embed thinking about the value and design of the qualitative component from the outset. They were also intended to integrate members of the qualitative team with trial teams from an early stage to promote effective integration of qualitative methods within larger trials and build relationships over time.

Standard Operating Procedures (SOPs), formal and informal training, and interaction between the qualitative team and other researchers increased the relational workability of qualitative methods within the Centre—the confidence individuals felt in including these methods within their studies, and their accountability for doing so. For instance, study adoption forms prompted researchers to interact routinely with the qualitative team at an early stage, whilst guidance on costing grants provided clear expectations about the resources needed to deliver a proposed set of qualitative data collection.

Formation of the Qualitative Research Group—comprised of methodological specialists, created new roles and skillsets ( skill set workability ). Research teams were encouraged to draw on these when writing funding applications for projects that included a qualitative component. Capacity-building initiatives were used to increase the number of researchers with the skills needed to undertake qualitative research, and for these individuals to develop their expertise over time. This was achieved through formal training courses, academic seminars, mentoring from experienced colleagues, and informal knowledge exchange. Links with external collaborators and centres engaged in building qualitative research supported these efforts. Within the Centre, the co-location of qualitative researchers with other methodological and trial teams facilitated knowledge exchange and building of collaborative relationships, whilst grouping of the qualitative team within a dedicated office space supported a collective identity and opportunities for informal peer support.

Some aspects of the context in which qualitative research was being developed created challenges to operationalisation. Dependence on project grants to fund qualitative methodologists meant that there was a continuing need to write further grant applications whilst limiting the amount of time available to do so. Similarly, researchers within the team whose role was funded largely by specific research projects could sometimes find it hard to create sufficient time to develop their personal methodological interests. However, the cultivation of a methodologically varied portfolio of work enabled members of the team to build significant expertise in different approaches (e.g. ethnography, discourse analysis) that connected individual studies.

Reflexive monitoring—evaluating the impact of qualitative research

Inclusion of questions/fields relating to qualitative research within the Centre’s study portfolio database was a key way in which information was collected ( systematisation ). It captured numbers of funding applications and funded studies, research design, and income generation. Alongside this database, a qualitative resource planner spreadsheet was used to link individual members of the qualitative team with projects and facilitate resource planning, further reinforcing the core responsibilities and roles of qualitative researchers within CTR. As with all staff in the Centre, members of the qualitative team were placed on ongoing rather than fixed-term contracts, reflecting their core role within CTR. Planning and strategy meetings used the database and resource planner to assess the integration of qualitative research within Centre research, identify opportunities for increasing involvement, and manage staff recruitment and sustainability of researcher posts. Academic meetings and day-to-day interaction fulfilled informal appraisal of the development of the group, and its position within the Centre. Individual appraisal was also important, with members of the qualitative team given opportunities to shape their role, reflect on progress, identify training needs, and further develop their skillset, particularly through line management systems.

These forms of systematisation and appraisal were used to reconfigure the development of qualitative research and its integration within the Centre. For example, group strategies considered how to achieve long-term integration of qualitative research from its initial embedding through further promoting the belief that it formed a core part of the Centre’s business. The visibility and legitimacy of qualitative research were promoted through initiatives such as greater prominence on the Centre’s website. Ongoing review of the qualitative portfolio and discussion at academic meetings enabled the identification of areas where increased capacity would be helpful, both for qualitative staff, and more broadly within the Centre. This prompted the qualitative group to develop an introductory course to qualitative methods open to all Centre staff and PhD students, aimed at increasing understanding and awareness. As the qualitative team built its expertise and experience it also sought to develop new and innovative approaches to conducting qualitative research. This included the use of visual and diary-based methods [ 11 ] and the adoption of ethnography to evaluate system-level clinical interventions [ 12 ]. Restrictions on conventional face-to-face qualitative data collection due to the COVID-19 pandemic prompted rapid adoption of virtual/online methods for interviews, observation, and use of new internet platforms such as Padlet—a form of digital note board.

In this paper, we have described the work undertaken by one CTU to integrate qualitative research within its studies and organisational culture. The parallel efforts of many trials units to achieve these goals arguably come at an opportune time. The traditional designs of RCTs have been challenged and re-imagined by the increasing influence of realist evaluation [ 6 , 18 ] and the widespread acceptance that trials need to understand implementation and intervention theory as well as assess outcomes [ 17 ]. Hence the widespread adoption of embedded mixed methods process evaluations within RCTs. These broad shifts in methodological orthodoxies, the production of high-profile methodological guidance, and the expectations of research funders all create fertile ground for the continued expansion of qualitative methods within trials units. However, whilst much has been written about the importance of developing qualitative research and the possible approaches to integrating qualitative and quantitative methods within studies, much less has been published on how to operationalise this within trials units. Filling this lacuna is important. Our paper highlights how the integration of a new set of practices within an organisation can become embedded as part of its ‘normal’ everyday work whilst also shaping the practices being integrated. In the case of CTR, it could be argued that the integration of qualitative research helped shape how this work was done (e.g. systems to assess progress and innovation).

In our trials unit, the presence of a dedicated research group of methodological specialists was a key action that helped realise the development of a portfolio of qualitative research and was perhaps the most visible evidence of a commitment to do so. However, our experience demonstrates that to fully realise the goal of developing qualitative research, much work focuses on the interaction between this ‘new’ set of methods and the organisation into which it is introduced. Whilst the team of methodological specialists was tasked with, and ‘able’ to do the work, the ‘work’ itself needed to be integrated and embedded within the existing system. Thus, alongside the creation of a team and methodological capacity, promoting the legitimacy of qualitative research was important to communicate to others that it was both a distinctive and different entity, yet similar and equivalent to more established groups and practices (e.g. trial management, statistics, data management). The framing of qualitative research within strategies, the messages given out by senior leaders (formally and informally) and the general visibility of qualitative research within the system all helped to achieve this.

Normalisation Process Theory draws our attention to the concepts of embedding (making a new practice routine, normal within an organisation) and integration —the long-term sustaining of these processes. An important process through which embedding took place in our centre concerned the creation of messages and systems that called upon individuals and research teams to interact with qualitative research. Research teams were encouraged to think about qualitative research and consider its potential value for their studies. Critically, they were asked to do so at specific points, and in particular ways. Early consideration of qualitative methods to maximise and optimise their inclusion within studies was emphasised, with timely input from the qualitative team. Study adoption systems, centre-level processes for managing financial and human resources, creation of a qualitative resource planner, and awareness raising among staff, helped to reinforce this. These processes of embedding and integration were complex and they varied in intensity and speed across different areas of the Centre’s work. In part this depended on existing research traditions, the extent of prior experience of working with qualitative researchers and methods, and the priorities of subject areas and funders. Centre-wide systems, sometimes linked to CTR’s operation as a CTU, also helped to legitimise and embed qualitative research, lending it equivalence with other research activity. For example, like all CTUs, CTR was required to conform with the principles of Good Clinical Practice, necessitating the creation of a quality management system, operationalised through standard operating procedures for all areas of its work. Qualitative research was included, and became embedded, within these systems, with SOPs produced to guide activities such as qualitative analysis.

NPT provides a helpful way of understanding how trials units might integrate qualitative research within their work. It highlights how new practices interact with existing organisational systems and the work needed to promote effective interaction. That is, alongside the creation of a team or programme of qualitative research, much of the work concerns how members of an organisation understand it, engage with it, and create systems to sustain it. Embedding a new set of practices may be just as important as the quality or characteristics of the practices themselves. High-quality qualitative research is of little value if it is not recognised and drawn upon within new studies for instance. NPT also offers a helpful lens with which to understand how integration and embedding occur, and the mechanisms through which they operate. For example, promoting the legitimacy of a new set of practices, or creating systems that embed it, can help sustain these practices by creating an organisational ambition and encouraging (or requiring) individuals to interact with them in certain ways, redefining their roles accordingly. NPT highlights the ways in which integration of new practices involves bi-directional exchanges with the organisation’s existing practices, with each having the potential to re-shape the other as interaction takes place. For instance, in CTR, qualitative researchers needed to integrate and apply their methods within the quality management and other systems of a CTU, such as the formalisation of key processes within standard operating procedures, something less likely to occur outside trials units. Equally, project teams (including those led by externally based chief investigators) increased the integration of qualitative methods within their overall study design, providing opportunities for new insights on intervention theory, implementation and the experiences of practitioners and participants.

We note two aspects of the normalisation processes within CTR that are slightly less well conceptualised by NPT. The first concerns the emphasis within coherence on identifying the distinctiveness of new practices, and how they differ from existing activities. Whilst differentiation was an important aspect of the integration of qualitative research in CTR, such integration could be seen as operating partly through processes of de-differentiation, or at least equivalence. That is, part of the integration of qualitative research was to see it as similar in terms of rigour, coherence, and importance to other forms of research within the Centre. To be viewed as similar, or at least comparable to existing practices, was to be legitimised.

Second, whilst NPT focuses mainly on the interaction between a new set of practices and the organisational context into which it is introduced, our own experience of introducing qualitative research into a trials unit was shaped by broader organisational and methodological contexts. For example, the increasing emphasis placed upon understanding implementation processes and the experiences of research participants in the field of clinical trials (e.g. by funders), created an environment conducive to the development of qualitative research methods within our Centre. Attempts to integrate qualitative research within studies were also cross-organisational, given that many of the studies managed within the CTR drew together multi-institutional teams. This provided important opportunities to integrate qualitative research within a portfolio of studies that extended beyond CTR and build a network of collaborators who increasingly included qualitative methods within their funding proposals. The work of growing and integrating qualitative research within a trials unit is an ongoing one in which ever-shifting macro-level influences can help or hinder, and where the organisations within which we work are never static in terms of barriers and facilitators.

The importance of utilising qualitative methods within RCTs is now widely recognised. Increased emphasis on the evaluation of complex interventions, the influence of realist methods directing greater attention to complexity and the widespread adoption of mixed methods process evaluations are key drivers of this shift. The inclusion of qualitative methods within individual trials is important and previous research has explored approaches to their incorporation and some of the challenges encountered. Our paper highlights that the integration of qualitative methods at the organisational level of the CTU can shape how they are taken up by individual trials. Within CTR, it can be argued that qualitative research achieved high levels of integration, as conceptualised by Normalisation Process Theory. Thus, qualitative research became recognised as a coherent and valuable set of practices, secured legitimisation as an appropriate focus of individual and organisational activity and benefitted from forms of collective action which operationalised these organisational processes. Crucially, the routinisation of qualitative research appeared to be sustained, something which NPT suggests helps define integration (as opposed to initial embedding). However, our analysis suggested that the degree of integration varied by trial area. This variation reflected a complex mix of factors including disciplinary traditions, methodological guidance, existing (un)familiarity with qualitative research, and the influence of regulatory frameworks for certain clinical trials.

NPT provides a valuable framework with which to understand how these processes of embedding and integration occur. Our use of NPT draws attention to the importance of sense-making and legitimisation as important steps in introducing a new set of practices within the work of an organisation. Integration also depends, across each mechanism of NPT, on the building of effective relationships, which allow individuals and teams to work together in new ways. By reflecting on our experiences and the decisions taken within CTR we have made explicit one such process for embedding qualitative research within a trials unit, whilst acknowledging that approaches may differ across trials units. Mindful of this fact, and the focus of the current paper on one trials unit’s experience, we do not propose a set of recommendations for others who are working to achieve similar goals. Rather, we offer three overarching reflections (framed by NPT) which may act as a useful starting point for trials units (and other infrastructures) seeking to promote the adoption of qualitative research.

First, whilst research organisations such as trials units are highly heterogenous, processes of embedding and integration, which we have foregrounded in this paper, are likely to be important across different contexts in sustaining the use of qualitative research. Second, developing a plan for the integration of qualitative research will benefit from mapping out the characteristics of the extant system. For example, it is valuable to know how familiar staff are with qualitative research and any variations across teams within an organisation. Thirdly, NPT frames integration as a process of implementation which operates through key generative mechanisms— coherence , cognitive participation , collective action and reflexive monitoring . These mechanisms can help guide understanding of which actions help achieve embedding and integration. Importantly, they span multiple aspects of how organisations, and the individuals within them, work. The ways in which people make sense of a new set of practices ( coherence ), their commitment towards it ( cognitive participation ), how it is operationalised ( collective action ) and the evaluation of its introduction ( reflexive monitoring ) are all important. Thus, for example, qualitative research, even when well organised and operationalised within an organisation, is unlikely to be sustained if appreciation of its value is limited, or people are not committed to it.

We present our experience of engaging with the processes described above to open dialogue with other trials units on ways to operationalise and optimise qualitative research in trials. Understanding how best to integrate qualitative research within these settings may help to fully realise the significant contribution which it makes the design and conduct of trials.

Availability of data and materials

Some documents cited in this paper are either freely available from the Centre for Trials Research website or can be requested from the author for correspondence.

O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Hewison J. What can qualitative research do for randomised controlled trials? A systematic mapping review. BMJ Open. 2013;3(6):e002889.

Article   PubMed   PubMed Central   Google Scholar  

O’Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Goode J, Hewison J. Maximising the value of combining qualitative research and randomised controlled trials in health research: the QUAlitative Research in Trials (QUART) study – a mixed methods study. Health Technol Assess. 2014;18(38):1–197.

Clement C, Edwards SL, Rapport F, Russell IT, Hutchings HA. Exploring qualitative methods reported in registered trials and their yields (EQUITY): systematic review. Trials. 2018;19(1):589.

Hennessy M, Hunter A, Healy P, Galvin S, Houghton C. Improving trial recruitment processes: how qualitative methodologies can be used to address the top 10 research priorities identified within the PRioRiTy study. Trials. 2018;19:584.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350(mar19 6):h1258.

Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75(12):2299–306.

Article   PubMed   Google Scholar  

O’Cathain A, Hoddinott P, Lewin S, Thomas KJ, Young B, Adamson J, et al. Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot Feasibility Stud. 2015;1:32.

Cooper C, O’Cathain A, Hind D, Adamson J, Lawton J, Baird W. Conducting qualitative research within Clinical Trials Units: avoiding potential pitfalls. Contemp Clin Trials. 2014;38(2):338–43.

Rapport F, Storey M, Porter A, Snooks H, Jones K, Peconi J, et al. Qualitative research within trials: developing a standard operating procedure for a clinical trials unit. Trials. 2013;14:54.

Cardiff University. Centre for Trials Research. Available from: https://www.cardiff.ac.uk/centre-for-trials-research . Accessed 10 May 2024.

Pell B, Williams D, Phillips R, Sanders J, Edwards A, Choy E, et al. Using visual timelines in telephone interviews: reflections and lessons learned from the star family study. Int J Qual Methods. 2020;19:160940692091367.

Thomas-Jones E, Lloyd A, Roland D, Sefton G, Tume L, Hood K, et al. A prospective, mixed-methods, before and after study to identify the evidence base for the core components of an effective Paediatric Early Warning System and the development of an implementation package containing those core recommendations for use in th. BMC Pediatr. 2018;18:244.

May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 2007;7:148.

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54.

Article   Google Scholar  

May CR, Mair F, Finch T, Macfarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4:29.

Ogrinc G, Davies L, Goodman D, Batalden PB, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. BMJ Quality and Safety. 2016;25:986-92.

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

Jamal F, Fletcher A, Shackleton N, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example. Trials. 2015;16(1):466.

Download references

Acknowledgements

Members of the Centre for Trials Research (CTR) Qualitative Research Group were collaborating authors: C Drew (Senior Research Fellow—Senior Trial Manager, Brain Health and Mental Wellbeing Division), D Gillespie (Director, Infection, Inflammation and Immunity Trials, Principal Research Fellow), R Hale (now Research Associate, School of Social Sciences, Cardiff University), J Latchem-Hastings (now Lecturer and Postdoctoral Fellow, School of Healthcare Sciences, Cardiff University), R Milton (Research Associate—Trial Manager), B Pell (now PhD student, DECIPHer Centre, Cardiff University), H Prout (Research Associate—Qualitative), V Shepherd (Senior Research Fellow), K Smallman (Research Associate), H Stanton (Research Associate—Senior Data Manager). Thanks are due to Kerry Hood and Aimee Grant for their involvement in developing processes and systems for qualitative research within CTR.

No specific grant was received to support the writing of this paper.

Author information

Authors and affiliations.

Centre for Trials Research, DECIPHer Centre, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK

Jeremy Segrott

Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK

Sue Channon, Eleni Glarou, Jacqueline Hughes, Nina Jacob, Sarah Milosevic, Yvonne Moriarty, Mike Robling, Heather Strange, Julia Townson & Lucy Brookes-Howell

Division of Population Medicine, School of Medicine, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK

Eleni Glarou

Wales Centre for Public Policy, Cardiff University, Sbarc I Spark, Maindy Road, Cardiff, CF24 4HQ, UK

School of Social Sciences, Cardiff University, King Edward VII Avenue, Cardiff, CF10 3WA, UK

Josie Henley

DECIPHer Centre, School of Social Sciences, Cardiff University, Sbarc I Spark, Maindy Road, Cardiff, CF24 4HQ, UK

Bethan Pell

You can also search for this author in PubMed   Google Scholar

Qualitative Research Group

• , D. Gillespie
• , J. Latchem-Hastings
• , R. Milton
• , V. Shepherd
• , K. Smallman
•  & H. Stanton

Contributions

JS contributed to the design of the work and interpretation of data and was responsible for leading the drafting and revision of the paper. SC contributed to the design of the work, the acquisition of data and the drafting and revision of the paper. AL contributed to the design of the work, the acquisition of data and the drafting and revision of the paper. EG contributed to a critical review of the manuscript and provided additional relevant references. JH provided feedback on initial drafts of the paper and contributed to subsequent revisions. JHu provided feedback on initial drafts of the paper and contributed to subsequent revisions. NG provided feedback on initial drafts of the paper and contributed to subsequent revisions. SM was involved in the acquisition and analysis of data and provided a critical review of the manuscript. YM was involved in the acquisition and analysis of data and provided a critical review of the manuscript. MR was involved in the interpretation of data and critical review and revision of the paper. HS contributed to the conception and design of the work, the acquisition and analysis of data, and the revision of the manuscript. JT provided feedback on initial drafts of the paper and contributed to subsequent revisions. LB-H made a substantial contribution to the design and conception of the work, led the acquisition and analysis of data, and contributed to the drafting and revision of the paper.

Corresponding author

Correspondence to Jeremy Segrott .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was not sought as no personal or identifiable data was collected.

Consent for publication

Competing interests.

All authors are or were members of staff or students in the Centre for Trials Research. JS is an associate editor of Trials .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Segrott, J., Channon, S., Lloyd, A. et al. Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts. Trials 25 , 323 (2024). https://doi.org/10.1186/s13063-024-08124-7

Download citation

Received : 20 October 2023

Accepted : 17 April 2024

Published : 16 May 2024

DOI : https://doi.org/10.1186/s13063-024-08124-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Qualitative methods
• Trials units
• Normalisation Process Theory
• Randomised controlled trials

ISSN: 1745-6215

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 14 May 2024

Designing for student autonomy combining theory and clinical practice – a qualitative study with a faculty perspective

• Charlotte Silén 1 ,
• Katri Manninen 1 , 2 &
• Angelica Fredholm 1 , 3  

BMC Medical Education volume  24 , Article number:  532 ( 2024 ) Cite this article

41 Accesses

Metrics details

Although extensive research exists about students’ clinical learning, there is a lack of translation and integration of this knowledge into clinical educational practice. As a result, improvements may not be implemented and thus contribute to students’ learning. The present study aimed to explore the nature of clinical faculty members’ learning related to how they apply research about student autonomy.

A course, “Designing learning for students’ development of autonomy in clinical practice” was conducted for faculty responsible for students’ clinical education. Within the frame of the course the participants designed a project and planned how they would implement it in their clinical context. Fourteen clinical faculty members participated in the study. The participants’ interpretation of the educational intervention, which combines complex theory with the equally complex clinical practice, was explored by studying how the participants’ approaches and understanding of the facilitation of autonomy were manifested in their projects. The projects in the form of reports and oral presentations were analyzed using qualitative content analysis together with an abductive approach.

One identified domain was “Characteristics of the design and content of the projects”. This domain was signified by two themes with different foci: Preparing the soil for facilitating student autonomy ; and Cultivating opportunities for students to actively strive for autonomy. A second identified domain, “ Embracing the meaning of facilitating autonomy ” was connected to participants understanding of theories underlying how to support the development of autonomy. This domain contained two themes: Connection between activities and autonomy is self-evident and Certain factors can explain and facilitate development of autonomy.

Education directed to strategic clinical faculty members to develop evidence-based approaches to student learning can be productive. To succeed there is a need to emphasize faculty members individual understanding of actual research as well as learning theories in general. Faculty trying to reinforce changes are dependent on their own mandate, the structure in the clinic, and recognition of their work in the clinical context. To achieve a potential continuity and sustainability of implemented changes the implementation processes must be anchored throughout the actual organization.

Peer Review reports

Clinical education is a comprehensive part of health care education programs and therefore important and vital for health care students to become knowledgeable well-educated professionals. Research on students’ clinical learning has been extensively reported but there is still a lack of translation and integration of this knowledge into clinical educational practice. This is a problem when trying to improve clinical training and there is a need to understand more of this matter. Hence, this study explores clinical faculty members’ learning related to how they apply research.

Introduction

The clinical environment offers a rich and powerful setting for learning and professional development [ 1 , 2 , 3 ]. In the clinical environment, abstract knowledge becomes tangible through its application in patient care. Students’ encounters with patients and staff representing their own and other professional groups provide unique experiences to reflect on and integrate in their learning. They can train and test their skills, observe, and examine patients, and provide care and treatment. Generalizable knowledge about diseases and their impact on people’s lives can be realized in each patient encounter. The variation that every patient encounter and personal experience offer enhances this knowledge. Learning in every encounter is not only related to the clinic as a physical place per se, but to relationships and experiences relating to this encounter [ 4 , 5 , 6 ]. As such, the clinic and the activities that occur in the clinic offer an inherent space for learning. This learning space provides contact with the reality of future professions, provides challenges, motivational factors, and feedback on behavior and thoughts. In this way, knowledge and professional development increases [ 2 , 3 , 7 , 8 ].

Crucial aspects of creating and taking care of the rich opportunities for students’ learning are linked to the clinical environment, clinical supervisors and other stakeholders involved in the organization and implementation of clinical education. Responsible actors must engage, understand and be able to apply knowledge about how to support student learning to contribute to development. There is extensive research available regarding how to make the clinical learning environment fruitful for learning and to support students to reach their learning goals, while becoming well educated and well-functioning professionals (cf. [ 2 , 3 , 4 ], [ 8 , 9 , 10 , 11 ]).

However, several researchers point out that there is a lack of translation and integration of research-based knowledge into educational practice in the clinic [ 12 , 13 , 14 , 15 , 16 ]. This means that the main problem is not a lack of knowledge about student learning in the clinic, but the issue is why available research-based knowledge is not sufficiently applied. The complex nature of healthcare, the many faculty members involved, the lack of continuity related to education, and the fact that the education takes place in two different arenas – the university and the clinic – may explain some of the hindrances [ 14 , 15 , 16 , 17 ]. A major challenge in the development of health-care education is reaching and motivating faculty members to enhance their knowledge on how to improve student learning. This is especially true for faculty members involved in clinical education since their role mainly focuses on patient care and not on student education [ 12 , 13 ]. There is an expressed need for research illuminating how further professional development for faculty members can lead to improvement of clinical education [ 12 , 13 , 14 , 15 , 16 ]. In particular, studies are needed to examine the connection between the individual clinical faculty member and their application of knowledge in the clinical context. A more in-depth understanding of these connections is paramount to support clinical faculty members to integrate research and thereby enhance student learning.

In the present study, an educational intervention directed to faculty responsible for students’ clinical education was designed, executed, and studied. The intervention sought to improve how clinical faculty understand and apply research regarding what may influence and stimulate student autonomy in clinical education. In connection with the educational intervention, a study was carried out that aimed to examine faculty learning in terms of integration of theoretical knowledge in the clinic. The findings were meant to contribute to the understanding of how to support clinical faculty members to apply educational research on autonomy and thus enhance learning and consequently professional development for students in the clinic.

Autonomy and professional development

Development of autonomy is known to be crucial for student learning and professional development in the clinical setting. This concept was therefore chosen as the core of the intervention and this study. Nevertheless, autonomy is a complex concept, and its meaning is not possible to cover fully in the context of this study. The most fundamental aspects deemed important for this study will be presented below.

According to substantial research, autonomy is a fundamental need to experience self-governance and ownership of one’s actions [ 3 , 18 , 19 , 20 , 21 , 22 , 23 ]. Development of autonomy in learning is the foundation of life-long learning, meaning the ability to move on, constantly reevaluate your own knowledge, ability to obtain and use information, and understanding of your learning processes [ 18 , 19 , 24 ]. Studies about promoting self-directed and/or self-regulated learning have shown the importance of taking into account factors such as student motivation, experience of control, ability to seek and apply knowledge, ability to discern learning needs and ability to evaluate the outcome of learning [ 18 , 25 , 26 , 27 ]. Clinical practice needs autonomous health care providers, and here autonomy means something more than independence and control over your own learning. It has been shown that autonomy fosters personal identity and meaning, independent choices, responsibility, and critical thinking [ 19 , 20 , 21 , 22 , 23 ]. Important for professional competence is the ability to discern, assess, and pose new questions in unclear and incalculable situations [ 23 , 24 ]. Research shows how a curriculum designed to strengthen autonomy can create a qualitatively different understanding of a subject or professional field, as demonstrated in student’s ability to link theory and practice with abstract thinking [ 22 , 24 ].

In this study autonomy has been particularly connected to the meaning of authenticity and attachment. These concepts are identified as important parts of autonomy related to learning and professional development [ 2 , 3 , 22 , 28 , 29 ]. The rationale for this statement is outlined below. Autonomy is connected to authentic experiences in clinical training. It is also indicated that transformative learning processes that contribute to the development of a professional identity can be triggered by authentic experiences and the meaning-making of these processes [ 3 , 28 ]. Manninen et al. [ 29 ] showed how authenticity in clinical education functions as a driving force for learning by creating meaning and relevance. Furthermore, Manninen [ 2 , 29 ] has identified how authenticity can be both an external and an internal phenomenon, where external authenticity is produced by education and the surrounding environment – such as the interaction with patients in a clinical setting. Internal authenticity is experienced when students form mutual relationships with patients, feel a sense of belonging and perceive themselves as part of the team [ 2 , 29 ]. Levett-Jones & Lathlean [ 30 ], stress the positive effects on learning that occur when students experience a sense of belonging in their clinical practice. These experiences of relationships and sense of belonging are captured in the concept of attachment and linked to the development of autonomy [ 3 , 28 ]. Students need to be offered participation as well as actively strive to attach themselves to the actual clinical context in order to experience authenticity and autonomy in their learning. Prerequisites for students to experience and to seek attachment are based on mutual trust and respect [ 31 , 32 ]. Several studies [ 3 , 21 , 22 , 28 , 29 ] showed how both autonomy and authenticity are social phenomena having to do with the relationships that students can form in their clinical education and the clinical relevance of given tasks. Thus, students can develop as autonomous professionals when they experience both external and internal authenticity. This includes opportunities to experience attachment and gain responsibility for relevant parts of patient care, as well as the opportunity to follow up on administered care [ 2 , 3 ]. Students need to have access to and responsibility for entire processes, such as being able to evaluate the results of care and not just isolated actions or events. This reasoning applies to students, regardless of the clinical placement level since the complexity and length of processes can vary [ 33 , 34 ].

Designing for learning in the clinic

To further professional development, the design of learning in the clinic should offer students opportunities to experience through emotion and action what it means to be a professional nurse, doctor, or physiotherapist, etc [ 35 ]. By doing so, the risk for a narrow and static approach to knowledge decreases, thus making it easier to focus on knowledge application and the complexity of professional knowledge. A comprehensive review of the literature by Trede et al. [ 36 ] shows that the development of a professional identity is facilitated by learning based on cooperation and dialogue in practice and characterized by authentic experiences. Education should be designed to raise awareness of what autonomy means in clinical education to enhance student learning and the development of a professional identity. In turn this demands that clinical faculty members understand the concept and can integrate it in clinical education. The clinical application of evidence-based concepts means the ability to combine concepts and theory with a complex clinical practice [ 12 , 14 , 16 , 17 ]. This is regarded as a challenge for faculty responsible for developing clinical education and supporting students in their learning.

For the purpose of this study, an educational intervention was designed to present, explain, and illuminate theory and research related to supporting students’ development of autonomy. The study reached out to participants working in strategic positions in different clinical settings that enabled them to contribute to the design and development of clinical education in collaboration with different universities and educational programs. The participants designed and planned the implementation of a project aimed at enhancing student autonomy in their clinical context. The aim of this study was to explore the nature of clinical faculty members’ learning related to how they apply research about student autonomy in their projects .

The research approach was qualitative, interpreting participants’ experiences from a life-world perspective. The interpretation of meaning and lived experience was made possible through the tradition of phenomenological hermeneutics founded by Heidegger and further developed by Gadamer and Ricouer [ 37 ]. It is argued here that the lifeworld is mediated through narratives where individuals’ subjective understanding and sense-making of their lifeworld become visible [ 38 ]. Thus, the individual projects portrayed and studied here are viewed as narratives that manifest understanding of the phenomena under examination, the meaning of which is revealed through interpretation.

Pedagogical framework for the intervention

The pedagogical framework described in the background regarding the development of autonomy and professional identity formed an important part of the content of the designed educational intervention and the present study. Additionally, the educational intervention was based on constructivist learning theories that emphasize active, creative processing of information, including cognitive, emotional, and social aspects as well as testing and practical actions (cf. [ 35 , 39 , 40 , 41 , 42 ]). In the applied pedagogical framework, the lifeworld is seen as the total sum of the environment and everyday experiences that forms the individual’s world, thus forming the basis for the individual’s interpretations, thoughts, reactions, and actions [ 43 ]. Learning was seen as fundamentally situated in a physical as well as social and cultural context [ 39 , 40 , 43 ].

The intervention, a course, “Designing learning for students’ development of autonomy in clinical practice”, was designed for health-care professionals responsible for students’ clinical practice in Stockholm County Council, or other participants with similar overarching clinical pedagogical work assignments. The relevant faculty role for this in Sweden is often an adjunct clinical lecturer (ACL) and this term will be used in the following description of the participants. They have their main employment and activities in the health care sector outside of a higher education setting and provide the university with specific expertise not found within the organization. The ACL supports both clinical supervisors and students at the clinical workplace, has the possibility to influence prerequisites for clinical education and functions as a bridge between the university and the local clinical education organization.

The intervention – “Designing learning for students’ development of autonomy in clinical practice.”

The purpose of the course that constitutes the intervention in the present study was to strengthen the pedagogical competence of the ACL for her/him to understand the meaning of research-based knowledge about learning, and how to apply this knowledge in clinical supervision and teaching. The goal was that the ACL should be able to contribute to and support students’ opportunities to develop autonomy in learning. The intervention was designed aiming to help ACLs understand research about how to facilitate autonomy in clinical practice. The intervention design was built on the pedagogical framework described above. In one extensive and concluding learning activity, participants designed and implemented projects aiming to enhance student autonomy in their clinical context. These projects constitute the focus for analysis in this study.

The course was given online and included 5 weeks full-time study. The online design was believed to enhance accessibility and enable adaptation to individual clinical contexts. It was spread over 6 months to allow time for the participants to process the content of the course and to plan and implement their projects. The course consisted of both asynchronous parts and synchronous meetings using Zoom. However, the online design of the course and the analysis of outcomes related to this design is not within the scope of this study. Two of the authors, (CS, AF) were responsible for the course and acted as lectures and tutors. Other experts were invited to the synchronous meetings giving lectures and participating in discussions. A digital learning platform was created, and the participants were divided into groups of 4–5 participants and one tutor, who worked together mainly asynchronous online. The groups were mixed in terms of professional background and the nature of their clinical workplace to learn from each other and provide a range of perspectives while working with different learning activities. The content of the course was focused on the meaning of autonomy in learning and its application in clinical practice for students. The participants worked individually with written tasks and communicated with their group members and the tutor. They were asked to build on their previous knowledge and experiences and actively apply new knowledge and thoughts. The tutor facilitated communication in the group by posing questions and commenting on the written work and discussions. All learning activities were designed to allow participants to discern the relevance and implications of theory in their own individual clinical context and describe this with concrete examples. The core concepts of autonomy, authenticity and attachment were presented in lectures online and discussed synchronously. These lectures were also available on the digital platform.

In one extensive and concluding learning activity, participants designed and implemented projects aiming to enhance student autonomy in their clinical context. Participants worked on the project throughout the course, from a preliminary project plan to implementation, and evaluation. The projects were discussed in their groups as well as individually with the tutor. The projects were presented as written reports and final oral presentations synchronously in zoom. In the written reports, they described the design, theoretical background, implementation, and outcome of the projects. For the oral presentations, participants were asked to focus on what they perceived as most meaningful in their projects and how they applied pedagogical knowledge and reasoning.

Participants

The course was open to all ACLs in the region. A written invitation to the course was spread through the regional network where ACLs are registered. A pedagogical course within higher education comprising at least 5 weeks full time study was required to take the course. The participants of the course were informed about the study and could volunteer to take part or not. There were 15 ACLs that took part in the course and fourteen of them participated in the study: seven registered nurses (five with postgraduate specialist nursing education), two radiology nurses, two biomedical analysts, one physician, one speech therapist and one occupational therapist. All participants were women, aged 36 to 63, with ACL experience from 1 to 13 years.

The context in which the study participants were active as ACLs mirrored the variations of the health-care field. Variations came to the fore related to in-patient and out-patient care, medical specialty and whether the unit offered a specialized service, such as a laboratory, radiology, or anesthesia at an operating department. The responsibilities and tasks of the ACLs were different. Some of them were responsible for students from one profession at various sites and others responsible for one unit and all students at that site. Others coordinated both supervisors and students within one unit, while others mainly acted as supervisors with a special assignment to act as an ACL at a specific site. The number of and kinds of students placed at the different units varied. In most cases, the ACLs were responsible for students from one profession and one educational level – undergraduate or postgraduate – but there were also examples involving several professions and different educational levels. The organization of the students’ clinical placements governed the ability for the ACL to plan activities. There were variations in the length of the placement and whether students stayed in one place or rotated between different departments. What could be designed to stimulate student autonomy depended on what the students were supposed to learn for their profession and on their educational level.

Data collection

As described above the participants’ projects were chosen as objects for analysis. Data were collected using the written and oral accounts of the projects that constituted the concluding learning activity in the course. There were 11 projects included in the data. Three of them were collaborative projects where participants worked together; in two cases in the same clinic and specialty, and in one case from two different hospitals but in the same clinical specialty. Written accounts in the form of project reports were used together with video- and audio-recorded oral presentations of the projects.

Data analysis

Based on the learning theories presented in the pedagogical framework for the intervention, the point of departure for the analysis was that ACLs showed what they had learned by planning and implementing projects in their own clinical setting with the aim of contributing to and supporting the students’ opportunities to develop autonomy. The application of the theory they had studied, i.e., the discernment of the meaning of the theory in the clinical context and in everyday practice, was made by the ACLs. Thus, it was the participants who expressed how they would use what they had learned to bridge the gap between theory and practice.

The participants’ learning was analyzed based on the written projects together with the oral presentation of these projects. An interpretative content analysis of both the manifest and latent content was performed [ 44 , 45 ]. The manifest content refers to data close to the expressions used by the participants, in this case the written and oral descriptions of the projects. The latent content refers to the authors’ interpretation of the meaning of what is expressed related to the development of autonomy. An abductive approach was also applied, and thus data were analysed iteratively going back and forth between parts and wholes, both inductively, and deductively informed of theoretical perspectives during the research process [ 46 ]. The theoretical foundation for analysis was the above-described concepts furthering the development of autonomy. The inductive part of the analysis aimed to contribute to new perspectives and a development of how these concepts can be interpreted.

Condensed meaning units were extracted in the written projects and the video- and audio- recordings and subsequently coded. The codes were compared for differences and similarities and grouped into categories describing variations in focus and approaches to support development of student autonomy. In the next step, categories were interpreted and designated as latent themes. The identification of manifest and latent content of the participants understanding of theories underpinning autonomy, was built on revisiting the condensed meaning units and codes with focus on critical features, the relationship between concepts and practice, and how concepts were connected [ 44 , 45 ]. Thus, one project can be represented in both themes and several of the categories. Examples illustrating the steps of the data analysis process are described in Appendix 1. Two of the authors (CS, AF) performed the basis of the analysis iteratively independently and together. Emphasis was put on reflexivity concerning preconceived interpretations related to the researchers’ involvement in the course. The preliminary findings were critically reviewed by the third author (KM), less involved in the course, and then discussed and negotiated between all the authors to achieve consensus. All authors have extensive knowledge and experiences of clinical education as teachers, clinical supervisors, and as experienced qualitative researchers in medical education. The researchers’ collaborative analysis was meant to contribute richness and credibility to the findings.

The analysis of the content and implementation of the participants’ projects is described in two domains. A: Characteristics of the design and content of the projects, and B: Embracing the meaning of facilitating autonomy. Domain A was related to the description of the projects based on who was engaged and the focus of the content of the implementation. The categories and themes related to domain A illustrate the outcome of learning in terms of how they organize activities to implement student autonomy. The basis for the categories and themes in domain B was the analysis of the meaning of the projects and how the participants talked about how to achieve autonomy. The findings in Domain B relate to the participants’ learning in terms of their understanding of how to apply the theories underlying the support for the development of autonomy. An overview of the findings is displayed in Fig.  1 . Quotes are presented with numbers of the participants and marked with oral account (oa) and written account (wa).

- Overview of findings

Domain A: Characteristics of the design and content of the projects

The way the participants designed and described their projects on how to support students’ development of autonomy in clinical practice varied. Two themes with different foci were identified; Preparing the soil for facilitating student autonomy; and Cultivating opportunities for students to actively strive for autonomy . The first theme comprised two categories; Engaging supervisors to support student autonomy and Emphasizing organizational dimensions that have an impact on implementation. The second theme was characterized by the two categories Activities involving students during significant parts of their clinical placement and Specific activities focused on certain knowledge and skills. The content of Domain A is illustrated in Table  1 .

Preparing the soil for facilitating student autonomy

Several participants chose to prepare the soil , that is, they utilized their knowledge of developing student autonomy to prepare the clinical practice environment for the students. Within this theme two categories were identified: Engaging supervisors to support student autonomy and Emphasizing organizational dimensions that have an impact on implementation. The first category focused on the supervisors, and the second took a broader approach, including the structural factors of the clinical placement, the supervisors’ role, and managers on different levels.

Engaging supervisors to support student autonomy

To reach the goal of fostering independence in students, the supervisors in the clinic were engaged by the ACLs. One approach within this category was to focus on support for supervisors by offering written plans, advice and thoughts on factors that have an impact on student independence in the clinic.

The documents written by the ACL varied between concise information sheets, and comprehensive plans on activities. The documents contained hints and advice about how supervisors should act to facilitate student independence. The supervisors’ responsibilities in relation to the students were also pointed out. A document could be created by the ACL, presented, and handed over to the supervisors to use. Other documents were authored by the ACL, presented, discussed, and adapted to the supervisors’ comments before they were finalized and used.

“In the meeting with the supervisors, we used time to discuss the guide (advice to the supervisors to support student autonomy). Many supervisors attended the meeting …we believe we have created a great participation” (No 12 and 13, oa).

Another example was a comprehensive operational plan aiming to develop a pedagogical framework for supervision that would increase the supervisors’ own confidence, allowing the students to practice autonomy in their professional role.

“The plan is that the supervisor guides the student from Active Observations, involves the student to Work Together, then invites the student to Take a Lead and finally Work Independently “ (No 1, wa).

The ACLs instructions to the supervisors differed. In some cases, the content of the documents was left for the supervisors to use as they saw fit, while other documents contained prompts on how to use it and how to act.

Follow ups of the content of the created documents and usefulness of the initiated guidance for the supervisors varied. In some cases, there were no systematic plans for follow up, while others discussed and revised the content of the document.

Another approach within the category of preparing the soil was signified by activities that process the meaning of and facilitation of autonomy. The aim of these activities was to stimulate reflection on factors affecting the development of autonomy to ensure continuation.

“To make the implementation [new activities] work, there is a need for careful and long-term planning ….it takes even more communication [ between different stakeholders ] …and it must be adapted to the students’ actual clinical placement” (No 8 and No 14, wa).

An activity taking the form of a workshop was characterized by the ACLs presenting what autonomy and factors stimulating student development might mean for the students.

“I created a workshop and gave a lecture about the concepts we had studied in the course – about how to facilitate autonomy…. I talked about attachment, trust, and authenticity and how that relates to autonomy” (No 10, oa).

One activity consisted of recorded lectures by the ACL combined with prepared tasks for the supervisors to complete. In the lectures, the different factors presented in the course that the ACLs had attended were presented. Through the tasks, the supervisors were encouraged to reflect on what these factors might mean for their students in their own clinical context. It was up to the supervisors to decide when they wanted to study the recorded material.

“The recorded lectures combined with tasks were meant to provide those responsible for the students in clinical practice knowledge about factors influencing students’ possibilities to reach autonomy during their placement……It would provide them with tools and ways of thinking to be able to change the organization of the clinical practice towards the goal of increased autonomy for students” (No 11, wa).

Emphasizing organizational dimensions that have an impact on implementation

In this category, the importance of considering organizational structures of both the health-care unit and the students’ clinical education when implementing measures to stimulate student autonomy were pointed out. The dimensions brought up and considered in the participants’ projects were the structure of the clinical practice, such as how responsibility of supervision was distributed, and the total number of students at the unit, but also the spread of students over a semester. The ACLs considerations included the duration of the placement, how and whether the students were rotating between different sections at the unit or stayed in one place. To enhance development of student autonomy, the idea of continuity in supervision was emphasized, and to maintain sustainability, managers on several levels were engaged in the planning, as were supervisors.

” We involved managers and administrative assistants in our planning…. administrative assistants plan supervisors’ work schedules and thus influence their work…. a pedagogical encounter was set up to engage them [ managers and administrative assistants ] in a pedagogical discussion…. We also created a group with supervisors from different departments of the clinic to discuss the project and the purpose of the idea of achieving continuity” (No 8 and No 14, wa).

Communication and cooperation with the university was stressed as crucial for whether the supervisors succeeded in facilitating student autonomy. There was an agreement among the ACLs in this category on the importance of including managers from different levels to succeed with the planned project.

” We contacted our manager and presented the project to her, and we contacted our administrative assistants and informed them about who the main supervisors were so that their shift work would not be affected” (No 4, oa).

Cultivating opportunities for students to actively strive for autonomy

In this theme, cultivating opportunities for students , another significant approach to use the knowledge of autonomy was identified. The ACLs planned activities directly to students to stimulate their development of autonomy. The theme consists of two categories: Activities involving students during significant parts of their clinical placement and Specific activities focused on certain knowledge and skills.

Activities involving students during significant parts of their clinical placement

Activities in this category were planned to capture multiple competences and were integrated throughout most of the practice period. A characteristic activity planned by the ACLs included in this category involved assignments that were identified and described for the specific students to complete in pairs.

“Learning activity: The students listen and observe a professional encounter with a patient. Afterwards they attend a workshop about how to document data in a patient’s chart…. The students work together and give each other feedback …. they are asked to go on working like this, writing on their own, discussing with each other and after that consulting the supervisor” (No 10, oa).

The planned activities were related to students’ vocational training and involved ideas about progression regarding students’ possibilities to act autonomously and the complexity of the assignment itself. Students trained different skills on their own and did not only watch their supervisors. They had to make choices between different actions, such as how to proceed in a certain situation, as well as judge when an assignment was finished and how it should be reported. Similar activities were also planned for other students without emphasis on peer learning. Instead, the supervisor continuously identified learning tasks for one student at a time, and thus independence was gradually required.

Variations could be noted in the ACLs approach to stimulating autonomy in these activities. In some cases, the learning tasks were planned in detail by the ACL and the training was limited to certain skills and behaviors. Other approaches were planned to continuously encourage the student to take responsibility and perform independently.

…the supervisors were encouraged to give students increased responsibility, e.g. by allowing them to be supervised by colleagues/other professions/other students and receive more assignments to solve themselves…. such as that they can develop a sense of autonomy in parallel with a sense of belonging with the whole health-care team and the workplace.” (No 10, wa).

Students’ reflections on their own performance were emphasized as important to stimulating development of autonomy. Sometimes, the reflection sessions were mainly about how the tasks had been carried out. In other cases, the students’ own perceptions about their progression towards autonomy were also important to discuss.

“The ACL met the student and the supervisor every week to reflect, based on a certain model. The core concepts of autonomy, authenticity, attachment, trust, and professional identity were discussed to evaluate whether the students felt that they experienced autonomy at the clinical placement. The students were asked to write in their logbook and their questions were discussed during the weekly reflection time.” (No 7, wa).

Specific activities focused on certain knowledge and skills

In this category, ACL’s planned projects contained learning tasks that focused on single skills completed at a specific time during the placement. The aim was to stimulate the development of autonomy in different ways connected to this learning task. A typical kind of activity was characterized by the ACL creating conditions for the students’ training but leaving the implementation to the students. These kinds of activities could be about connecting well-planned written tasks to common clinical issues for the patients who were cared for on the ward. The students could choose when to review the written tasks and how to perform them.

” It is difficult for students to feel attachment and ‘to be nurses’ on the ward when they are there for a short period of time, the patients are very ill, and they don’t have the right knowledge and skills to independently take care of them. I created written learning activities for students to work with on their own or together with other students…it meant that they could select a patient to talk to, search for knowledge and consider questions about a patient’s status and appropriate care” (No 5, oa).

Another activity took the form of a room prepared with equipment, offering opportunities for students to independently train important professional skills. The aim of this activity was to facilitate autonomy and critical thinking within postgraduate nursing specialist training.

“The students can practice together to supervise the monitoring and treatment of a simulated patient based on an authentic scenario. This means that they themselves lead the activity and must make important decisions and reflect on the outcome after a presentation of a project (No 2, oa).

Most activities planned by the ACLs focused on one profession at a time but there was one example targeting interprofessional learning. Interprofessional seminars were implemented for students to learn about other professions from their peers, and these seminars were followed up with reflections on what professional teamwork meant for the development of professional autonomy.

Domain B: Embracing the meaning of facilitating autonomy

The participants’ choices of design and the ways they described and talked about their projects also reflected a dimension of their learning related to their theoretical understanding of development of students’ autonomy. The analysis of the participants’ descriptions of how different factors facilitate and relate to the development of autonomy resulted in the outcome of two qualitatively different perceptions, here designated as two themes: Connection between activities and autonomy is self-evident and Certain factors can explain and facilitate development of autonomy. The first theme consists of one category: Lack of reasoning about the meaning of autonomy . In the second theme, two categories emerged: The concept of autonomy as a core value and Various factors are linked to the development of autonomy. The content of the domain is illustrated in Table  2 .

Connection between activities and autonomy is self-evident

One category denoted this theme, namely the lack of reasoning. The activities were described as facilitating autonomy, but there was no explanation for the underlying ideas of why the activities facilitated autonomy. One example is that a project was meant to introduce peer learning and activities for the students were thus described. These activities could involve students training skills on their own and they were asked to make their own decisions and discuss with their peers. However, there was no elaboration on how and why these activities were supposed to result in students becoming more autonomous. The relationship between activities and the ability to make choices and more independent decisions seemed to be taken for granted. Another example is the notion that activities directed at interprofessional education led to autonomy, which also was never explained.

“They develop autonomy as they see their own responsibilities as they are reflected in what other professions perform and are responsible for. It promotes their own professional development.” (No 9, wa).

Certain factors can explain and facilitate development of autonomy

The main qualitative difference between this theme compared to the first is that the participants explained and reasoned about how and why certain activities stimulated and led to independence. However, which factors that were brought up varied, as did the complexity of related explanations and reasoning.

The concept of autonomy as a core value

The level of understanding in this category was characterized by explanations and reasoning linked to the use of autonomy as an overarching concept. Here, an activity such as being asked to independently use a skill or to handle an encounter with a patient was chosen because this training would lead to student autonomy. When the project was introduced to supervisors or managers, the planned activities were mainly motivated by claiming that if students were given opportunities to act independently, it would foster autonomy in them. Factors brought up in the course as influencing development of autonomy were not used by the ACLs to elaborate on how to facilitate autonomy. They didn’t elaborate on any other factors brought up in the course as influencing the development of autonomy.

“When we thought about how to work with student autonomy, we decided to use peer learning. The students’ assignments are described, they work together, and the students take responsibility to carry them out. The students can stand on their own two feet… they are trusted” (No 6, oa).

Various factors are linked to development of autonomy

This category was characterized by an elaborate understanding of the meaning of autonomy and factors that have an impact on the development of autonomy. The ACLs reasoning about autonomy and other factors influencing autonomy was complex to a varying degree. Some participants explained and related their activities to one or more factors.

“…the students meet new supervisors very often, generally speaking every day… this leads to obstacles for student learning, and it makes it difficult to develop autonomy and authenticity. Both the students and the supervisors become ambivalent when they must create new relationships almost every day” (No 8 and No 14, wa).

Others reasoned about how different factors were interdependent and related to facilitation of autonomy in a broader sense.

From a guide for supervisors: … we think this is about attachment, the students are invited, and they have got a place when they arrive….and this next guiding advice is connected to trust… we trust the student that they know a lot, but it takes time to learn this new specialty. It is about autonomy too – that the students take responsibility and think for themselves… Some supervisors are very controlling, so the students don’t have the possibility to practice how to really be critical care nurses, so they don’t experience authenticity” (No 12 and No 13, oa).

We argued in the background that a hindrance for development of a rich learning environment in clinical education is that available research-based knowledge is not sufficiently applied [ 12 , 13 , 14 , 15 , 16 , 17 ]. A way to face this problem is to enhance knowledge about how clinical faculty members understand and integrate theoretical knowledge in clinical practice. In this study we examined projects designed and implemented by clinical faculty members to find out how they, in this case, applied research about student autonomy in clinical education. The projects were the final part of a course introducing theories and research on the importance of students’ development of autonomy in clinical education. The purpose of describing and reasoning about these participants’ learning was to contribute to a deeper understanding of how to support clinical faculty to acquire and apply theoretical knowledge in clinical practice.

Two different domains mirroring the participants’ learning outcome were identified when their projects were analyzed. One domain concerned what they had decided to focus on to facilitate students’ development of autonomy and how they went about implementing their ideas in the clinic. The other identified domain involved the interpretation of the participants’ understanding of the theoretical framework underpinning autonomy as a concept. These findings expose different perspectives on how a course with a specific design directed at faculty members impacted their actions and understanding, i.e., the faculty members’ learning. The outcome of the course was encouraging in relation to the facilitation of student learning in clinical practice. All the participating ACLs projects contained activities and/or documents that involved some form of application of theories on how to facilitate student autonomy Some projects aimed to “prepare the soil”, such as educating supervisors and creating fertile ground for learning for the students. Another group of projects were planned directly for students, signified by “cultivating opportunities ” for them to practice autonomy through certain activities. Several studies have shown that the nature of clinical education is complex [ 12 , 14 , 16 , 17 ]. The students’ education takes place in two different arenas – the university and the clinic. Many faculty members are involved in clinical education, and their role mainly focuses on patient care and not on student education [ 12 , 13 ]. It became obvious that this complexity of clinical education influenced what the ACLs assessed possible to accomplish. This was mirrored in the choices that the ACLs made concerning the content and to whom they directed their projects. A comprehensive review of research on student learning in clinical practice found that issues about how to organize students’ learning were the most researched, indicating that organizational issues are an essential part of change [ 47 ]. The significant features of the health-care units in which the ACLs acted had a large impact on how they planned and implemented their projects. This underlines the importance of being familiar with the nature of context to introduce changes. The projects that were targeted supervisors, managers, and the organization presumably had an impact on a wider group of students compared to projects that were designed directly for a minor group of students or supervisors. This is important in relation to issues about sustainability. If faculty members on different levels in the clinic are engaged, ideas and knowledge about how to facilitate student learning can continue to develop and gain a foothold [ 14 , 48 , 49 , 50 ]. Projects involving managers and system levels are far more likely to become sustainable [ 13 , 15 , 17 , 48 ]. The activities planned directly for students, and where the ownership of the ideas was closely linked to the ACLs, run the risk of being dependent on a limited group of faculty members, and may cease as soon as the person in charge is not there.

The variety of planned activities and documents created by the ACLs displayed comprehensive understanding, challenges and shortcomings related to the meaning of autonomy. Two qualitatively different levels of understanding of autonomy and how different factors facilitate and relate to the development of autonomy emerged [ 3 , 22 , 24 ]. One level of understanding relied on a presumed self-evident relationship between an activity and student autonomy. The other level of understanding involved explanations for how different factors, such as authenticity, trust, and belonging, relate to the development of autonomy. These differences in learning outcomes are very important to consider in faculty development. We claim that a level of understanding that includes the ability to discern the meaning of theory in the clinical context and in everyday practice, is necessary to support others, such as supervisors and students, and implement sustainable change. The understanding of a situation and the understanding of the phenomenon that gives this situation meaning are connected. According to Marton and Booth [ 24 ] a situation is understood based on the phenomena involved – and the phenomena are perceived in light of the specific situation. When viewed in relation to the participants’ projects this meant that the understanding of the theory could be seen in the choices the participants made regarding their projects, what they perceived as important issues, how they proposed to solve these issues, etc. From a variation theory perspective, this is viewed as a matter of discrimination and differentiation, and learning is seen as the ability to discern these differences [ 24 , 35 ]. The space for learning, therefore, is the potential variation or difference provided by the situation [ 24 , 35 ]. Opportunities to participate in continuing professional courses and forums for discussions between ACLs may support the development of a deeper understanding of theory when it is linked to clinical practice.

In the design of the course emphasis was placed on participants creating and implementing a project. What did that mean for their learning? Some projects were quite limited, as they sometimes only comprised one document, or when an activity only reached a small group of students or a minor group of supervisors. This can be a shortcoming, but it is possible that further development and successful implementation is more dependent on the properties of the document or activity related to theoretical understanding and the ACLs ability to identify meaningful problems. If an initiative is well substantiated, there will be more opportunities to build on it [ 35 , 43 ]. Some projects were broader, where several activities were planned to be repeated and continued over time and they engaged both supervisors and students. These projects carry a high potential for successful implementation and impact on supporting student learning, since continuity and engagement increases opportunities for faculty members to learn [ 17 , 35 , 50 ].

Above, we have discussed how the ACLs’ learning manifested itself in the participants’ projects and how that reflected their understanding of theories about learning processes, connected to development of autonomy. The group of ACLs is particularly interesting as they support both supervisors and students and are responsible for bridging the education gap between the university and clinical practice. Understanding of the actual subject matter – in this case facilitating autonomy – turns out to be very important for the individual ACL to manage to drive development and change. It impacted problems that they discerned and identified, and the choices they made in their planning [ 24 , 35 ]. Successful implementation also seems to depend on the mandate of the change agents, in this case the ACLs and their awareness of the practices at their unit and on different management levels [ 13 , 17 , 48 , 49 , 50 ]. In addition to these requirements, we would argue that understanding how people learn in general is also critical to the implementation of new ideas. This statement is based on the application of the constructivist pedagogical framework underpinning this study [ 35 , 39 , 40 , 41 , 42 , 43 ]. If development is to occur, all stakeholders must process and understand the meaning of autonomy and be able to relate and link it to their practice [ 13 , 17 , 48 , 49 , 50 ]. The way the ACL communicated with and involved affected parties in their projects revealed their awareness of learning processes not only directed at facilitating autonomy. There were examples of projects in which an activity and/or a document was created by the ACL and the main strategy during implementation was to provide information about it to supervisors, managers, and students. This strategy essentially meant that no learning processes were initiated to facilitate understanding. In other projects, supervisors and/or students were involved to different extents in creating and making their own choices about how to perform suggested activities and review documents. These strategies encouraged the stakeholders to think about and react to practice, as well as reflect on what autonomy meant to them personally. Other factors conducive to active learning processes that were identified were planned follow-up opportunities, interactive feedback, and shared recurrent encounters to discuss documents and/or activities.

Strengths and limitations

The strength of this study lies in the theoretical and conceptual rigor applied throughout both the design process and the implementation of the educational intervention. There is also considerable procedural rigor due to the intervention being implemented with particularity and a firm epistemological stance. Limitations are connected to the sample of the study with only interested and ambitious learners who decided to take this course. However, this also contributes to rich data descriptions. There are notable challenges in studying an intervention that we as researchers have designed and the outcomes of this intervention. These challenges have been counteracted through constant reflexive discussions and questioning of assumptions.

This study shows that an educational intervention that emphasizes application of theoretical knowledge in clinical practice can enhance the development of evidence-based approaches to support students’ learning. Targeting a strategic group, such as the ACLs in this study, can be a successful way to strengthen faculty development. All participants in the intervention demonstrated the ability to use theoretical knowledge and create activities to support students’ learning. However, their applications differed in terms of underlying reasoning, reach and potential sustainability. To some extent these differences were due to a deeper understanding versus a more superficial understanding of the central concepts related to autonomy. Another critical factor affecting implementation was the ACLs understanding of learning processes in general. Lessons to learn for professional faculty development are that there is a need to stress individual understanding of actual theoretical concepts as well as learning theories in education addressing clinical faculty. The outcome of the ACLs planned projects turned out to be very dependent on their own mandate, the structure in the clinic, and acknowledgement of their work in the clinical context. This study also highlights that in order to achieve a potential continuity and sustainability of implemented changes in the clinic the implementation processes must be anchored throughout the actual organisation.

Data availability

The datasets generated and analyzed during the current study are not publicly available due to ethical reasons connected to the participant’s informed consent. The data generated during this study consists of written reports and transcribed audio recordings of participants who have been guaranteed confidential handling of data. On reasonable request, data can be made available from the corresponding author.

Dornan T, Tan N, Boshuizen H, Gick R, Isba R, Mann K, Scherpbier A, Spencer J, Timmins E. How and what do medical students learn in clerkships? Experience based learning (ExBL). Adv Health Sci Education: Theory Pract. 2014;19(5):721–49. https://doi.org/10.1007/s10459-014-9501-0

Article   Google Scholar  

Manninen K. (2014). Experiencing Authenticity- the core of student learning in clinical practice. Dissertation, Karolinska Institutet. Department of Learning, Informatics, Management and Ethics, Stockholm, Sweden.

Fredholm A. (2017). Understanding the meaning of autonomy. Creating a learning space for professional becoming in clinical education. Dissertation. Karolinska Institutet. Department of Learning, Informatics, Management and Ethics, Stockholm, Sweden.

van der Zwet J, Zwietering PJ, Teunissen PW, van der Vleuten CP, Scherpbier AJ. Workplace learning from a socio-cultural perspective: creating developmental space during the general practice clerkship. Adv Health Sci Education: Theory Pract. 2011;16(3):359–73. https://doi.org/10.1007/s10459-010-9268-x

Liljedahl M, Engqvist Boman L, Porthén Fält C, Bolander Laksov K. What students really learn: contrasting medical and nursing students’ experiences of the clinical learning environment. Adv Health Sci Educ Theory Pract. 2015;20:765–79.

Engqvist Boman L, Sundberg K, Petersson LM, Backman M, Silén C. A pedagogical model to enhance nurses’ ability to support patient learning: an educational design research study. Int J Med Educ. 2022;13:176–86. https://doi.org/10.5116/ijme.62c2.b9c4

Savin-Baden M, Learning Spaces. Creating opportunities for Knowledge Creation in Academic Life. The Society for Research into Higher Education & Open University; 2008.

Razack S, Philibert I. Inclusion in the clinical learning environment: building the conditions for diverse human flourishing. Med Teach. 2019;41(4):380–4. https://doi.org/10.1080/0142159X.2019.1566600

Warne T, Johansson UB, Papastavrou E, Tichelaar E, Tomietto M, Van den Bossche K, Moreno MF, Saarikoski M. An exploration of the clinical learning experience of nursing students in nine European countries. Nurse Educ Today. 2010;30(8):809–15. https://doi.org/10.1016/j.nedt.2010.03.003

Dilworth S, Higgins I, Parker V, Kelly B, Turner J. Finding a way forward: a literature review on the current debates around clinical supervision. Contemp Nurse. 2013;45(1):22–32. https://doi.org/10.5172/conu.2013.45.1.22

Chan M, Snell L, Philibert I. The education avenue of the clinical learning environment: a pragmatic approach. Med Teach. 2019;41(4):391–7.

Gruppen LD. Context and complexity in the clinical learning environment. Med Teach. 2019;41(4):373–4. https://doi.org/10.1080/0142159X.2019.1566599

Elmberger A, Björck E, Liljedahl M, Nieminen J, Bolander Laksov K. Contradictions in clinical teachers’ engagement in educational development: an activity theory analysis. Adv Health Sci Education: Theory Pract. 2019;24(1):125–40. https://doi.org/10.1007/s10459-018-9853-y

Elmberger A, Blitz J, Björck E, Nieminen J, Bolander Laksov K. Faculty development participants’ experiences of working with change in clinical settings. Med Educ. 2022. https://doi.org/10.1111/medu.14992 . Advance online publication.

Steinert Y, Mann K, Anderson B, Barnett BM, Centeno A, Naismith L, et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: a 10-year update: BEME Guide 40. Med Teach. 2016;38(8):769–86.

Miller-Young J, Poth CN. Complexifying’ our approach to evaluating educational development outcomes: bridging theoretical innovations with frontline practice. Int J Acad Dev Published Online Febr. 2021;25:1–14. https://doi.org/10.1080/1360144X.2021.1887876

Alsiö Å, Pettersson A, Silén C. Health Care leaders’ perspectives on how continuous Professional Development can be promoted in a Hospital Organization. J Contin Educ Health Prof. 2022;42(3):159–63. https://doi.org/10.1097/CEH.0000000000000451

Silén C. Self-directed learning – a learning issue for students and faculty. Teach High Educ. 2008;13(4):461–75.

Silén C. Responsibility and independence in learning – what is the role of the educators and the framework of the educational programme? In: Rust C, editor. Improving student learning – theory, research and practice. Volume I. Oxford: The Oxford Centre for Staff and Learning Development; 2003. pp. 249–62.

Google Scholar  

Clouder L. Being responsible’: students’ perspectives on trust, risk and work-based learning. Teach High Educ. 2009;14(3):289–301.

Eneau J. Educational Reciprocity and Developing Autonomy: The Social dimension of Becoming oneself. In: Schneider K, editor. Becoming oneself. Wiesbaden: Verlag für Sozialwissenschaften; 2012.

Fredholm A, Henningsohn L, Savin-Baden M, Silén C. Autonomy as both challenge and development in clinical education. Learn Cult Social Interact. 2015;5:20–7.

Clouder L, Jones M, Mackintosh S, Adefila A. Development of autonomy on placement: perceptions of physiotherapy students and educators in Australia and the United Kingdom. Physiother Theory Pract. 2022;38(12):2100–10.

Marton F, Booth S. (1997). Learning and awareness. Mahawa, N.J.: Lawrence Erlbaum Associates, Publishers; 1997.

Lee YM, Mann KV, Frank BW. What drives students’ self-directed learning in a hybrid PBL curriculum. Adv Health Sci Education: Theory Pract. 2010;15(3):425–37. https://doi.org/10.1007/s10459-009-9210-2

White CB, Fantone JC. Pass-fail grading: laying the foundation for self-regulated learning. Adv Health Sci Education: Theory Pract. 2010;15(4):469–77. https://doi.org/10.1007/s10459-009-9211-1

White CB. Smoothing out transitions: how pedagogy influences medical students’ achievement of self-regulated learning goals. Adv Health Sci Education: Theory Pract. 2007;12(3):279–97. https://doi.org/10.1007/s10459-006-9000-z

Fredholm A, Manninen K, Hjelmqvist H, Silén C. Authenticity made visible in medical students’ experiences of feeling like a doctor. Int J Med Educ. 2019;10:113–21. https://doi.org/10.5116/ijme.5cf7.d60c . PMID: 31203265; PMCID: PMC6766398.

Manninen K, Welin Henriksson E, Scheja M, Silén C. Authenticity in learning– nursing students’ experiences at a clinical education ward. Health Educ. 2013;113(2):32–143.

Levett-Jones T, Lathlean J. Belongingness: a prerequisite for nursing students’ clinical learning. Nurse Educ Pract. 2008;8(2):103–11. https://doi.org/10.1016/j.nepr.2007.04.003

Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ. 2013;19:435–56. https://doi.org/10.1007/s10459-013-9474-4

Bonnie LHA, Visser MRM, Kramer AWM, van Dijk N. Insight in the development of the mutual trust relationship between trainers and trainees in a workplace-based postgraduate medical training programme: a focus group study among trainers and trainees of the Dutch general practice training programme. BMJ Open. 2020;10e036593. https://doi.org/10.116/bmjopen-2019-036593.

Pront L, Gillham D, Schuwirth LWT. Competencies to enable learning-focused clinical supervision: a thematic analysis of the literature. Med Educ. 2016;50:485–95. https://doi.org/10.1111/medu.12854

Kjaer LB, Schimdt Nielsen KJ, Krogh Christiansen M, Strand P. Patient-centred learning in practice. A mixed methods study of supervision and learning in student clinics. Patient Educ Couns. 2023;112:107717. https://doi.org/10.1016/j.pec.2023.107717

Marton F. Necessary conditions of learning. London: Routledge; 2014.

Book   Google Scholar  

Trede F, Macklin R, Bridges D. Professional identity development: a review of the higher education literature. Stud High Educ. 2012;37(3):365–84.

Lindseth A, Norberg A. A phenomenological hermeneutical method for researching lived experience. Scand J Caring Sci. 2004;18(2):145–53. https://doi.org/10.1111/j.1471-6712.2004.00258.x

Ricoeur P. Interpretation theory: discourse and the surplus of meaning. Fort Worth, TX: Texas Christian University; 1976.

Dewey J. Democracy and education: an introduction to the philosophy of education. New York: Mac Millan; 1916.

Illeris K. A comprehensive understanding of human learning. Contemporary theories of learning: Routledge; 2018. pp. 1–14.

Kolb DA. Experiential learning: experience as the source of learning and development: FT press. London: Pearson Education LTD; 2014.

Kaufman DM. Teaching and learning in medical education: how theory can inform practice. Understanding medical education: evidence, theory, and practice. In Swanwick, T., Forrest, K., O’Brien, editors. Understanding Medical Education: Evidence, Theory, and Practice, Third Edition, West Sussex: Wiley- Blackwell. 2018:37–69.

Mezirow J. An overview on transformative learning. In: Illeris K, editor. Contemporary theories of learning: learning theorists in their own words. New York: Routledge; 2009. pp. 91–105.

Graneheim UH, Lindgren BM, Lundman B. Methodological challenges in qualitative content analysis: a discussion paper. Nurse Educ Today. 2017;56:29–34. https://doi.org/10.1016/j.nedt.2017.06.002

Krippendorff K. Content analysis: an introduction to its methodology. Sage; 2018.

Tavory I, Timmermans S. Abductive Analysis, theorizing qualitative research. Chicago: The University of Chicago; 2014. p. 172.

Silén C, Kalén S, Lundh P, Mattson J, Manninen K. Students’ learning in clinical practice – a scoping review of characteristics of research in the nordic countries. Med Educ Online. 2023;28(1). https://doi.org/10.1080/10872981.2023.2279347

Hannah ST, Lester PB. A multilevel approach to building and leading learning organizations. Leadersh Quart. 2009;20:34–48.

Caldwell R. Models of Change Agency: a fourfold classification. Br J Manag. 2003;14:131–42.

Mlambo M, Silén C, McGrath C. Lifelong learning and nurses’ continuing professional development, a metasynthesis of the literature. BMC Nurs. 2021;20(1):62. https://doi.org/10.1186/s12912-021-00579-2

Download references

Acknowledgements

We thank the adjunct clinical lecturers who kindly took part in the study.

Open access funding provided by Karolinska Institute. The study was funded by ALF, the Regional Agreement on Medical Training and Clinical Research between Region Stockholm and Karolinska Institutet.

Open access funding provided by Karolinska Institute.

Author information

Authors and affiliations.

Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden

Charlotte Silén, Katri Manninen & Angelica Fredholm

Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden

Katri Manninen

Centre for clinical research and education, County Council Värmland, Karlstad, Sweden

Angelica Fredholm

You can also search for this author in PubMed   Google Scholar

Contributions

All authors were active in designing and executing the intervention and the study. CS and AF performed the analysis and wrote the manuscript. KM critically reviewed and contributed to the description of the results, read, and revised the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Charlotte Silén .

Ethics declarations

Ethics approval and consent to participate.

The study was conducted in accordance with the Code of Ethics of the Declaration of Helsinki and adhered to the principles of confidentiality, integrity, right to self-determination, and privacy, as well as transparency and secure data processing. The study was given ethical approval by the Swedish Ethical Review Board (reference 2021–02178). Written information was presented before asking for consent to participate. The aim and context of the study were described, along with principles of confidentiality and their right to leave the study at any given moment. Written informed consent was obtained from all participants, and the confidential handling of data was guaranteed.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Silén, C., Manninen, K. & Fredholm, A. Designing for student autonomy combining theory and clinical practice – a qualitative study with a faculty perspective. BMC Med Educ 24 , 532 (2024). https://doi.org/10.1186/s12909-024-05514-y

Download citation

Received : 29 August 2023

Accepted : 03 May 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s12909-024-05514-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Educational intervention
• Implementation
• Faculty development
• Clinical practice

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 09 May 2024

Overlooked by the obstetric gaze – how women with persistent health problems due to severe perineal trauma experience encounters with healthcare services: a qualitative study

• Katharina Tjernström 1 ,
• Inger Lindberg 1 ,
• Maria Wiklund 2 &
• Margareta Persson 1  

BMC Health Services Research volume  24 , Article number:  610 ( 2024 ) Cite this article

124 Accesses

Metrics details

During the first year postpartum, about 25 per cent of Swedish women with severe perineal trauma (SPT), i.e., a third- or fourth-degree perineal laceration at childbirth, are unsatisfied with their healthcare contacts. Further, there is a lack of research on the more long-term experiences of healthcare encounters among women with persistent SPT-related health problems. This study explores how women with self-reported persistent SPT-related health problems experience their contact with healthcare services 18 months to five years after childbirth when the SPT occurred.

In this descriptive qualitative study, a purposive sample of twelve women with self-reported persistent health problems after SPT were individually interviewed from November 2020 – February 2022. The data was analysed using inductive qualitative content analysis.

Our results showed a paradoxical situation for women with persistent health problems due to SPT. They struggled with their traumatised body, but healthcare professionals rejected their health problems as postpartum normalities. This paradox highlighted the women’s difficulties in accessing postpartum healthcare, rehabilitation, and sick leave, which left them with neglected healthcare needs, diminished emotional well-being, and loss of financial and social status. Our results indicated that these health problems did not diminish over time. Consequently, the women had to search relentlessly for a ‘key person’ in healthcare who acknowledged their persistent problems as legitimate to access needed care, rehabilitation, and sick leave, thus feeling empowered.

Conclusions

Our study revealed that women with persistent SPT-related health problems experienced complex health challenges. Additionally, their needs for medical care, rehabilitation, and sick leave were largely neglected. Thus, the study highlights an inequitable provision of SPT-related healthcare services in Sweden, including regional disparities in access to care. Hence, the authors suggest that Swedish national guidelines for SPT-related care need to be developed and implemented, applying a woman-centered approach, to ensure equitable, effective, and accessible healthcare.

Peer Review reports

Intrapartum and postpartum healthcare should ideally be high-quality, evidence-based, and a positive experience stemming from woman-centred care with a holistic approach based on human rights [ 1 ]. This approach acknowledges each woman’s articulated needs and expectations in her social, emotional, physical, spiritual, and cultural context [ 2 ]. Nevertheless, during the first year postpartum, about one in four Swedish women with severe perineal trauma (SPT) [ 3 ], i.e., a third- or fourth-degree perineal laceration involving the anal sphincter muscle and anorectal mucosa at vaginal childbirth [ 4 ], are dissatisfied with their care and one in three women report ongoing health problems related to their SPT. Women with SPT may suffer from various physiological and psychological consequences such as pain [ 5 , 6 ] , incontinence [ 7 ], defecation problems [ 8 ], vaginal prolapse [ 5 ], sexual dysfunction [ 9 ] or depression and anxiety [ 10 , 11 , 12 ].

Reducing physical symptoms is essential to support emotional and social recovery after any perineal trauma [ 13 , 14 ]. Women with SPT emphasise that professional, competent, and respectful attitudes from healthcare professionals (HCPs), including individual and adapted information, facilitate, and promote their postpartum recovery. Thus, the HCPs’ competence and knowledge of treatment options is a prerequisite for women to access needed care [ 15 ]. An additional problem in the Swedish context is the lack of national recommendations or guidelines, which enables each of the 21 regions to develop own regional and local guidelines. An audit of the existing regional and local guidelines for prevention and care of SPT shows an unexpected diversity or lack of evidence-based recommendations [ 16 ]. However, dissatisfaction with access to healthcare has been expressed by women with persistent, i.e., beyond one year postpartum, SPT-related health problems [ 6 , 17 ]. Furthermore, women criticise inadequate or absent support [ 6 , 18 ], poor information and education [ 6 , 10 , 18 ], and lack of follow-up care regarding SPT and its potential psychological and social consequences [ 6 , 10 ]. Postpartum care focuses more on the baby than the mother’s well-being [ 18 , 19 ]. Also, the available treatment options are perceived as limited and outdated by those with access to needed care [ 17 , 18 ]. Moreover, women with SPT describe that some HCPs tend to normalise their SPT-related problems [ 10 , 17 , 18 , 19 , 20 , 21 , 22 ], and women are met in unprofessional and disrespectful ways [ 17 , 23 ], where HCPs are perceived as ignorant, nonchalant, and questioning women’s symptoms [ 10 , 17 ]. Previous research [ 24 ] indicates an institutional objectification of women with SPT by Swedish healthcare providers hindering access to healthcare, sick leave, and occupational rehabilitation after SPT. In contrast, women also report being acknowledged and liberated when HCPs have a professional and empathic approach and provide continuity of care that enables access to care for persistent SPT-related health problems [ 17 , 18 , 19 , 25 ]. Thus, several women who sustained an SPT during childbirth do not experience access to needed and necessary care, a fact that needs further exploration.

Globally, sexual and reproductive health and rights (SRHR) are crucial for individual health and gender equality [ 26 ]. Current issues within SRHR and midwifery are controlled by the institutional power in health institutions, i.e., medical power [ 27 ], connected to the still-existing economic and educational disadvantages of women globally, which are also feminist issues [ 26 , 28 ]. As midwife stands for ‘with woman’ [ 28 ], gender or feminist approaches are used in advancing midwifery theory [ 27 , 29 ] and various aspects of SRHR topics such as breastfeeding promotion [ 30 ], birth plans [ 31 ] and attitudes towards contraceptives [ 32 ]. In midwifery and feminist approaches, the biological material body and the socially constructed gendered body are viewed as intertwined [ 33 ]. Moreover, midwifery care is recommended to be woman-centred [ 1 , 2 ], focusing on the individual woman’s needs and transferring control from the institution to the woman herself. However, despite the different organisations of sexual and reproductive healthcare between countries, international research shows similar results regarding women’s diverging experiences with postpartum SPT-related healthcare [ 6 , 15 , 17 , 18 ].

In sum, there is growing evidence showing that many women with persisting health problems caused by SPT are often, but not always, met with mistrust and ignorance when seeking care for their problems. Even though there may be national, regional, or local protocols or guidelines for care after SPT, women with persistent SPT-related problems still raise their voices about the difficulty of getting access to competent quality care. This indicates a potential gender bias [ 34 ] and a need for gender theoretical perspectives in midwifery [ 28 ], as utilized in this study. Additionally, few studies explore the care-seeking experience among this group of women in a longer time perspective after childbirth when the SPT occurred.

The aim of this study is to explore how women with self-reported persistent SPT-related health problems experience their contact with healthcare services 18 months to five years after childbirth when the SPT occurred.

Study design and context

The present study is part of a larger research project investigating the long-term consequences of SPT on quality of life, working life, and healthcare contacts. This study had an inductive qualitative interview study design applying qualitative content analysis to analyse data [ 35 , 36 , 37 ]. This method searches for patterns, e.g., by identifying similarities and differences in the data. The researchers obtain an in-depth understanding of the studied phenomenon through abstraction and interpretation [ 36 ]; thus, an appropriate method to apply to capture women’s experiences of their healthcare encounters when seeking medical help and support. Throughout the research process, the recommendations for qualitative research according to ‘Consolidated criteria for reporting qualitative research’ (COREQ) were followed [ 38 ].

Sweden has 21 partly independent regions primarily responsible for providing healthcare services to the population. Healthcare services are tax-funded, and the regions have extensive autonomy to decide upon the healthcare services within each region based on the frameworks of the Health and Medical Service Act [ 39 ]. Additionally, within the Swedish social security system, 480 days of paid parental leave are allocated to each child in Sweden and can be utilised by their legal guardian(s) until the child is twelve years old. Of these 480 days, 60 days are specifically assigned to each parent, and the remaining days are split between parents as desired. The financial compensation is based on the parent’s income and is financed by taxes [ 40 ].

In Sweden, midwives are the primary care providers to women with normal pregnancies, births, and postpartum care. In case of complications to pregnancy and childbirth, midwives collaborate with other medical professionals, especially obstetricians. For example, midwives suture first- and second-degree perineal lacerations, while obstetricians are responsible for all SPT repairs [ 41 ]. Generally, in Sweden, women who sustain an SPT during childbirth are offered a check-up with the obstetrician responsible for the repair before discharge and should also have a follow-up with an obstetrician or sometimes a physiotherapist within the postnatal period. Thereafter, women with no mayor initial healing problems are advised to contact relevant healthcare services if any health issues related to the SPT should arise in the future. Women presenting with complicated healing are treated accordingly. Additionally, women with second- to fourth-degree perineal lacerations are assessed with questionnaires three times during the first year postpartum by the National Perineal Laceration Register. However, there are no recommendations in Sweden for prolonged check-ups for women with SPT after the postnatal period and no guidelines on organised check-ups for women with prolonged symptoms due to SPT exist [ 42 , 43 ].

Women with persistent SPT-related health problems and characteristics were purposively recruited to achieve a heterogeneous sample reflecting multiple experiences. An overview of inclusion and exclusion criteria can be found in Table 1 .

The closed Swedish Facebook community ‘Förlossningsskadad? Du är inte ensam!’ [‘Injured at childbirth? You are not alone!’] functioned as a recruitment platform for a national sample of women reporting persistent SPT-related health problems. The Facebook community is secluded to women with SPT and started in 2014. During the data collection period (Nov 2020 – Feb 2022), the group had over 7,600 members; today, the community has grown to include over 9,500 members [ 44 ].

In late November 2020, the administrators of the Facebook community pinned a digital poster with study information and a link to the study homepage in the group feed. The study homepage contained written information on the research project and contact details for the research group if any women wanted additional information about the study. Interested potential participants contacted the research group via a contact form on the homepage, and the first author (KT) confirmed that the potential participants met the inclusion criteria via telephone. Thirteen participants from different parts of Sweden showed interest in participating and left their contact information. One woman never responded to our efforts to reach her. The remaining twelve women fulfilled the inclusion criteria and were invited to an interview. Before the interview, the women answered a digital survey on background data (such as demographic data, education, employment, sick leave, and childbirth history) distributed via REDCap ® , a web-based application to create secure online questionnaires and research databases [ 45 ]. The interviews were finalised in February 2022.

Data collection

We collected data via individual open-ended interviews [ 46 ], supported by a semi-structured interview guide [see Additional file 1 ]. The interview guide, developed by KT and MP with input from MW and IL, was based on literature reviews, our awareness of gender as a social construct [ 33 ], and the clinical pre-understanding within the research group. After a pilot interview conducted by KT (not included in the data), minor adjustments were made to the interview guide. The final interview guide covered the topics of everyday life experiences, work, and general functioning. However, despite the mentioned interview topics, the emergent study design and the ability to speak freely about what was perceived as important for their daily functioning, the contacts with healthcare services was brought up in vivid and extensive narratives by all participants as part of their descriptions of their challenges in everyday life and their ability to function at work. Hence, the experiences the women made of the healthcare services played an important role for the women in their daily management of SPT-related health problems.

As data collection occurred during the COVID-19 pandemic, all participants were interviewed digitally via Zoom ® [ 47 , 48 ]. With the participant’s consent, the interviews were audio-recorded via Zoom ® and a separate digital recorder (as backup). Any Zoom video files automatically generated were deleted directly after the termination of the interview to protect participants’ identities. The first author interviewed all women; in two interviews, co-authors (IL or MW) also attended. The authors had no professional or personal affiliation with the enrolled participants . Detailed interviews ranging from 29 – 112 minutes (median: 61.5 minutes) gave extensive data. All interviews were performed in Swedish and transcribed verbatim. After that, the first author validated the transcripts for accuracy by reviewing the text while listening to the recordings.

Authors’ pre-understanding and theoretical positionality

The research group comprises three midwives (KT, IL, MP) and one physiotherapist (MW). We all have extensive professional experiences from clinical practice in primary and in-patient care, where three authors (KT, IL, MP) have specific professional experiences of caring for women with SPT. Additionally, we are women, feminists, and mothers with various birth experiences. Further, the group holds expertise in gender studies and qualitative research within midwifery science, such as perineal trauma and medical sociology. Hence, we stem from a social constructivist research standpoint and utilise ourselves as co-constructors in the analysis process. As feminist researchers, we apply a gender theoretical lens to the data.

Data analysis

The interviews were analysed using qualitative content analysis with an inductive and stepwise approach focusing on the manifest and latent content [ 35 , 36 , 37 ]. The interviews, transcripts, and analysis steps were performed in Swedish.

The analytical procedure started with reading the transcripts multiple times while highlighting text, meaning units, with content relevant to the aim of this study. Then, identified meaning units were condensed, focusing on preserving their core meaning and labelled with manifest codes [ 35 , 36 , 37 ]. Initially, KT coded one interview and triangulated those codes with the principal investigator (MP). KT then coded the rest of the interviews. In the next step, similar codes were clustered, forming subcategories based on the manifest content. Moving towards an interpretation of the content, categories were created by the abstraction of subcategories. This was done by KT and MP separately and then triangulated to identify significant concepts. Next, the preliminary categories and subcategories were triangulated with the whole research group until a consensus was obtained. To answer the question of ‘what?’ and ‘how?’ within the data, the latent content and thread of meaning were identified by clustering and abstracting the emerging findings to form subthemes and a theme [ 36 , 37 ]. The emerging findings were also peer-reviewed and discussed at a research seminar. The finalisation of the analysis resulted in an overarching theme and four subthemes. The translation of categories, subthemes, theme and inserted citations from Swedish into English was performed as a last step. The translation and choice of words were discussed between authors (all knowledgeable in English) to reach a consensus and minimise translation bias.

During the coding process, the researchers used MAXQDA ® [ 49 ], a software for organising, transcribing, analysing, and visualising qualitative research data, and Microsoft Excel ® [ 50 ] as aids to organise the codes.

Demographics of included participants

The background characteristics of the twelve participants in the final sample are presented in Table 2 .

The participants identified themselves as cis women, i.e., their gender identity matched their sex assigned at birth [ 51 ], and are thus referred to as ‘women’ in this paper. All women were in a partner relationship. The women reported a broad spectrum of physical and phycological health problems following the SPT at childbirth, e.g., urine or anal incontinence, pain in the lower abdomen, sexual dysfunction, and depression. Thirty per cent of the women had full-time employment, and the proportion of parental leave varied from 12% to 100% (three women had an ongoing parental leave with subsequent children at the interview). Further, 60% of the women had a sedentary occupation. Five women had been on sick leave after reconstructive surgery, and five reported sick leave for other reasons than their SPT.

The analysis resulted in one theme, ‘Overlooked by the obstetric gaze – living the paradox of a normalised but traumatised postpartum body’, with related subthemes ‘Questioning whether it’s all in my head’, ‘Fighting persistently for access and legitimacy in no (wo)man's-land’, ‘Facing multidimensional losses when no help in sight’, and ‘Depending on other’s advocacy to navigate an arbitrary system’. An overview of the findings is presented in Table 3 . The findings are presented as an overarching theme and thereafter, the related subthemes and categories. Citations from the participants illustrate the findings. All women have been allocated pseudonyms in the result presentation.

Overlooked by the obstetric gaze – living the paradox of a normalised but traumatised postpartum body

The latent theme ‘Overlooked by the obstetric gaze – living the paradox of a normalised but traumatised postpartum body’ represented the women’s experiences of healthcare encounters covering HCPs’ diminishing attitudes towards women’s persistent SPT-related health problems and the women’s difficulties accessing healthcare and sick leave. We interpreted that the women were assessed by the HCPs’ ‘obstetric gaze’, i.e., a medical gaze in postpartum healthcare normalising their persistent health problems and judging the women’s lower abdomen as ‘fine’ by their looks. The obstetric gaze put the women in a paradoxical situation where HCPs normalised tangible symptoms to be a natural part of childbirth. With no medical legitimacy of the health problems, the women also felt labelled as ‘hysterical’ (exaggerating health problems) by the HCPs. As a result, on the one hand, they had to continue facing persistent and tangible health problems such as incontinence, pain or prolapses. On the other hand, no acknowledgement by HCPs of their health problems led them to question whether their problems were merely a product of their imagination and, thus, only existed ‘in their head’. The theme also comprised women’s struggle for legitimacy in a gendered healthcare system - a no-(wo)man's land. They experienced that healthcare services and social insurance systems were challenging to access and demanded a tenacious and extensive fight to obtain legitimacy for their health problems. Consequently, the women had to put up with neglected healthcare needs, negatively impacting their physical and emotional well-being, and financial and social status when no medical help or rehabilitation was available. However, some women had encountered an HCP who was empathic and understanding, hence not guided by the obstetric gaze. Such encounters legitimised persistent problems and were crucial for accessing needed care, sick leave, and rehabilitation.

Questioning whether it’s all in my head

The subtheme ‘Questioning whether it’s all in my head’ focused on the women’s experiences of facing ignoration and no confirmation of perceived health problems and thus being labelled as a hysterical woman. The related categories referred to a normalisation process that the women experienced in their encounters with HCPs, which made them question their bodily perceptions. Furthermore, the women felt accused of exaggerating symptoms because their persistent SPT-related health problems did not match HPCs’ views of acceptable postpartum symptoms. Thus, it could be understood that the women found themselves in a paradox of suffering from tangible physical consequences after SPT, which were normalised by HCPs and their ‘obstetric gaze’.

Facing HCP's ignoration of perceived problems

The women experienced the HCPs defining their persistent health problems after the SPT as ‘normal’. The HCPs assured the women that their problems would disappear with time or that transient motherhood-related aspects, such as breastfeeding or fragile vaginal mucosa, were the cause of the problem. One woman expressed:

“Then I felt, ‘It should not feel like this; this is something wrong’, and I sought medical attention and was seen by multiple physicians […] They thought my vaginal mucous membrane was not ready for intercourse. I was still breastfeeding, so they thought I should stop breastfeeding. Then maybe the mucous membrane would be restored, which was causing me the pain. I was not listened to at all. I was treated very poorly by one physician in particular, and despite second opinions and so on, nobody… nobody took me seriously.” (Linda)

Consequently, the women perceived that their concerns were ignored. They also learned that the HCPs saw their prolonged physical problems after SPT as an inevitable part of childbirth, which the women should accept. One woman resigned:

“But then [the physician] says something like this: ’Well, that's completely normal’, but I felt like, ‘Yes, but it doesn't feel normal'.” (Emma)

After the genital and pelvic floor examinations, the HCPs often guaranteed the women that ‘everything looked fine’, i.e., reinforcing the normality of the genital area. Although the women described to the HCPs that they struggled with SPT-related problems, their concerns were met with a comment on the physical appearance rather than a comprehensive examination of the pelvic floor's functionality.

One woman responded:

“They think ‘everything looks fine’ and ‘everything looks good and repaired’. I still have problems. I was also referred to a surgeon, who did a rectoscopy, and ‘it looked so nice’. Then, I was referred to a urotherapist to learn how to pinch my muscles because ‘everything would be so good’. She helped me get a second opinion in XX [town], where they discovered that there was still damage." (Jin)

Another woman expressed:

”I couldn’t care less what it [genital area] looks like. Nobody will be down there watching. I only need it [genital area] to function as intended.” (Anna)

Consequently, the women felt ignored and unheard in their contact with healthcare services. They perceived that HCPs did not listen to them, leaving them feeling invisible, sometimes even having severe health problems.

“I was hospitalised with sepsis before someone listened to me.” (Josefin)

Being labelled as a hysterical woman

The women also experienced being labelled as the ‘hysterical woman’ who exaggerated their persistent symptoms and had mental health problems. The women described how the HCPs accused them of imagining their SPT-related health problems. One woman indignantly revealed that the HCP she encountered said, 'These problems only exist in your head’ (Joanna), i.e., suggesting that the perceived symptoms did not exist and rejecting the health concerns. Hence, this attitude made some women believe their problems were a product of their imagination and sometimes made them even question their sanity.

Moreover, the HCPs’ condescending attitudes towards the women made them feel dismissed and devalued. For example, the women shared that HCPs laughed at them or were rough or cold during the examination. Moreover, HCPs expressed that they had ‘seen worse’ (Amanda). Some women also conveyed that they were advised ‘to drink some wine to feel better’ (Elin) when discussing painful intercourses due to their SPT-related health problems.

“You are constantly dismissed, ‘No, but everything looks fine, you have no problems’. Then you start to think you’re imagining things. And then you may not dare to talk about the injuries.” (Jin)

Fighting persistently for access and legitimacy in no (wo)man's-land

The subtheme ‘Fighting persistently for access and legitimacy in no (wo)man's-land’ referred to the women’s experience of gender constructs related to inaccessible healthcare services and their often year-long struggles to access this gendered healthcare and linked social insurance systems. The difficulties in accessing care created negative attitudes towards the healthcare services, making the women wish for general improvements in women’s healthcare.

Struggling to access the gendered healthcare and social insurance systems

The women pointed out that after giving birth, they needed more extensive information on their injury, precautions, available help (follow-up care or re-operation), and sick leave. To overcome the lack of required information, they had to request or actively search for it on their own, which also led to uncertainty about where and when to seek further help if needed.

“I was sent home with a brochure and a pat on the shoulder.” (Amanda)

The women also experienced a lack of adequate healthcare services targeted at their SPT-related health problems. For example, many women did not have access to a pelvic floor clinic or had to travel long distances to see specialists. Hence, their place of residence decided the quality of care the women received. Moreover, some women problematised the organisation of postpartum care as they missed out on follow-up care and even, in some cases, were denied follow-up care or referrals to specialised care were lost. As a result, some women had no opportunity to talk to the operating physician or experienced no follow-up care, although they requested it.

“They said it can take up to a year to get better. So, when that year had passed, and before starting to work again, I called different places in the hospital and asked: What should I do now? […] It took several months before I got an appointment with the surgeon for an assessment. And then I had to get a second opinion. So, it took like seven months before I got an appointment at [a specialist clinic].” (Hawa)

For the women, access to healthcare services, sick leave certificates, and HCPs’ dismissive attitudes were perceived as gender-related, i.e., difficulties in obtaining help from women’s healthcare services would not exist if the services were more women-oriented. One woman illustrated this by expressing: ‘If men gave birth to babies, the situation would not be like this’ (Joanna). Moreover, they perceived that women’s healthcare services were not prioritised. They explicitly stated that the absence of sick leave certificates and benefits was related to their gender. The women were expected to cope without sick leave benefits because vaginal and perineal lacerations of any scope were viewed as a natural part of childbirth, a normal process of a woman’s body. Thus, sequelae thereof did not exist or were taboo in society.

“Everything that happens during and after childbirth and related injuries has been a taboo discussion topic, so it has been completely ‘normal’ to suffer from persistent pain.” (Anna)

“I have applied for compensation from the national patient insurance. I got rejection after rejection; nothing has gone wrong. I was told: 'You simply must expect these things in childbirth. And a caesarean section is not less risky'.” (Hawa)

Thus, the women argued that society and the government did not invest needed resources in women’s healthcare. In addition, those few women receiving a short period of sick cash benefits had it immediately after giving birth or after re-operation, but not for prolonged problems. Further, the women noted that they were not offered sick leave certificates due to persistent physical SPT-related health problems but instead due to mental issues, such as depression or anxiety.

“I've heard about women who have been mentally unwell and have hurt their children. So maybe physicians get cautious and put women on sick leave if they say, ‘I'm not feeling mentally well’. Then they act quickly because they think it's so important. But they don't think about the physical injuries because that's part of [childbirth].” (Jenny)

However, the women shared how they fought long and hard for acknowledgement and care and made demands; for many, this process had covered years. They had to repeatedly insist that something was wrong and felt pressure to prove their health problems to the HCPs. In some women, this led to their persistent problems being diagnosed and acknowledged after several years of delay. The struggle for care involved countless visits and referrals to different HCPs, demanding much strength and persistence, which exhausted them. Sometimes, the sequelae had to develop into an acute health situation, or some women decided to pay for private care to access the proper treatment and rehabilitation. Further, with time, they also became explicit about their demands for sick leave certificates and benefits.

“Well, it [short sick leave period because of birth traumas] just feels like scorn. To me, it is not a sufficient length of sick leave.” (Elin)

Wishing for improvement in women's healthcare

The perceived lack of adequate care and rehabilitation, access to sick leave benefits, and HCPs’ attitude negatively influenced the women’s opinions on healthcare services, especially postpartum healthcare. In addition, the women perceived many HCPs as unprofessional, indifferent, and unstructured. As a result, the women mistrusted the HCPs and lost hope in healthcare services. Thus, they were reluctant to seek further care and were anxious about receiving proper treatment or that HCPs would miss important things.

“I am not being listened to in women's healthcare. This is partly why I feel so disappointed.” (Linda) “You just don't trust the healthcare system. […] Some people have been struggling with their injuries for like 18 years. But the [specialist clinic] – I finally received fantastic treatment, and what if it could be available everywhere [in Sweden]?” (Hawa)

Moreover, the women described a struggle for their rights when deciding whether to report the HCPs to the authorities and pointed out the need to improve women’s healthcare. Reporting HCPs was perceived as complicated as the women did not want to blame specific individuals. The women saw that the major problem lay within the healthcare system and with individual HCPs.

“In the end, I met a fantastic person [healthcare professional]. She wanted me to report the mistreatment when I eventually had the strength. Because no one listened when I said I was ill. So, she has offered to help me if I want to, but I don't know if I have the strength to file a complaint.” (Josefin)

A wish to improve women’s healthcare services was articulated, especially regarding personal follow-up care beyond one year postpartum and the possibility of full-time or part-time sick leave certificates and benefits for persistent problems on equal terms. This wish also strengthened their decision not to give up searching for help and to raise their voices to help themselves and other women.

“I received physiotherapy and the follow-up surveys [the Perineal Laceration Register] during the first year, but thereafter I would have liked to have an annual follow-up for the next years to ensure the status and potential re-operations. […] I can google, but I want to have that information in dialog with a living person, but you do not get that.” (Jenny)

Partaking in developing educational material for HCPs or starting a career within women’s healthcare were some women’s ways to contribute and increase competency in persistent SPT-related health problems.

“One of my strategies since I got the injury is also to try to influence. Being able to be involved and influence what postpartum care should consist of.” (Jin)

Facing multidimensional losses when no help in sight

The subtheme ‘Facing multidimensional losses when no help in sight’ covered physical and mental health consequences and the financial and social losses the participating women faced when no support or access to needed care and rehabilitation was provided.

Being physically victimised by HCP's malpractice

The women’s experiences covered either being misdiagnosed during the suturing after birth or in the following years when seeking help for persistent SPT-related health problems. Further, they shared how physicians had incorrectly sutured vaginal and perineal muscles after childbirth, leading the women to live with incontinence, pain, prolapses, or sexual dysfunction if their vaginas were sutured too tight. They also described how they endured infections, wound ruptures, sepsis, necrosis, and re-operations. Additionally, the women perceived a general lack of competency regarding communication and persistent SPT-related health problems, including problems related to sex life and sexual functioning, besides a more specific lack regarding suturing techniques and ultrasound examinations.

“I was referred to a specialist clinic. And they found out that all the muscles were separated, the internal and external sphincters were torn, and my pinching ability was kind of weak. So, it was quite the opposite, really, quite the opposite. None of what the other physician had said was true [laughs]. Absolutely incredible. And she is supposed to be a specialist.” (Hawa)

Aching inside

Living with troubled postpartum bodies and the absence of HCPs’ legitimation of the women´s problems made them struggle mentally, feeling speechless and silenced. This neglect reinforced irritation, anger, distress, bitterness, and disappointment towards the HCPs and the healthcare services. One woman illustrated the emotional struggle in this way:

“It's just that the health services don’t believe you, which makes you feel terrible. It's a big deal that no one listens.” (Josefin)

Moreover, the women felt uncertain about their health status due to a default medical diagnosis with concerns for their future and which staff to trust. Consequently, some had to bite the bullet, put up with their situation, and try to think positively. Other women were denying or diminishing their SPT-related health problems, accepting that their symptoms would improve, even disappear or that their condition was ‘normal’ as they had been told. Further, the women described despair because their neglected health problems caused by their SPT made them feel exposed, unsure, and hopeless. In some, this desperation resulted in a mental breakdown, a fear of losing custody of their child due to mental illness or suicidal thoughts.

“Something broke inside of me that day. I felt entirely omitted; I was close to leaving my son and committing suicide. Nobody understood how bad everything was.” (Elin)

Additionally, the women suffered emotionally when motherhood was crushed. Their partner had to take the primary responsibility for the family, and the children had to come in second place as the mothers suffered from various physical and mental health problems. As a result, the women felt they missed their children’s development and could not use their parental social security benefits as desired.

“I feel devasted because people tell me, ‘You are on maternity leave’. I’m not on maternity leave; I’m sick. I should be on sick leave.” (Jaanika)

Suffering financial and societal losses

Moreover, the women suffered financially and societally due to persistent health problems. Some women were denied financial compensation from Patient Insurance (a national insurance system where patients can seek compensation for care injuries). The Social Security Agency and the HCPs were perceived as obstacles to receiving sick cash benefits. They noted that ‘extensive’ health problems were required to receive sick cash benefits and that their health problems paradoxically were not seen as extensive or even a problem per se by the HCPs; hence, no sick leave certificates were issued.

“He [the physician] tried to argue and clarify my pain situation in the sick leave certificate to meet the requirements for a sick leave benefit at the Social Security Agency. I was in so much pain and had to lie down to breastfeed. But, no, ‘If you can manage to hold the baby when breastfeeding, then you are on maternity leave, not sick leave benefit’ [mimicking the official at the Social Security Agency who rejected the certificate and consequently also the sick cash benefit]”. (Jaanika)

Furthermore, the women were set back financially and societally because they could not work full-time due to their persistent health problems. Therefore, some women chose to compensate for their work absence with part-time parental benefits to diminish their working hours and cover their inability to work due to persistent SPT-related health problems. Without a sick leave certificate, i.e., the physicians or the officials at the Social Security Agency’s acknowledgement of a ‘true’ health problem, partners or other relatives were obliged to adjust their work schedules to support or unburden the woman’s suffering and inability to work full-time. This reduction in working hours for the SPT-affected women and, in some cases, their partners was expressed to potentially negatively affect their upcoming careers and pensions. As a result, the women experienced being caught between stools in the social insurance systems:

“[…] You end up in a position where you are neither on sick leave nor unemployment benefits and at the same time cannot perform any offered work [due to persistent problems]. But multiple societal bodies demand and expect you to be a part of the working force, and nobody really listens.” (Elin)

Depending on other’s advocacy to navigate an arbitrary system

The last subtheme, ‘Depending on other’s advocacy to navigate an arbitrary system’, highlights the women’s experiences of, often by chance, finding a single devoted professional, i.e., a ‘key person’, to access needed care and rehabilitation. Such a ‘key person’ was vital to recognising persistent problems, legitimating symptoms, and enabling access to needed care, sick leave, and rehabilitation. The women who finally had legitimation for their health problems described that the medical diagnosis also came with a feeling of sanity and empowerment, relieving them of their paradoxical situation.

Encountering a ‘key person’ to receive needed care

A support system was a prerequisite for enduring their health problems and finding the strength to fight for access to care. This system could be a partner, other family members, or friends who gave the women power and courage, but most importantly – encountering a professional who saw their problems and provided referrals or other options to obtain the needed help and support. In most cases, women would search for years for competent HCPs, such as midwives, physicians, or physiotherapists, who would listen and acknowledge persistent problems. This ‘key person’ showed empathy and trustworthiness, creating relief and security. Further, the ‘key person’ was portrayed as competent, attentive, professional, and respectful. The ‘key persons’ also shared women’s outrage at the mistreatment and default healthcare they endured. Additionally, these ‘key persons’ were surprised that the women were not on sick cash benefits due to their symptoms and that they had to compensate for their financial situation with parental benefits or reduced working hours and lower salaries. Consequently, finding this ‘key person’, often by chance or word of mouth, was crucial for accessing care and marked a significant turning point in the women’s recovery.

“I sought help from another midwife, as I felt something was wrong. This midwife referred me to the physiotherapist, who referred me to a specialist, who then referred me to surgery and rehabilitation.” (Malin)

Some women received follow-up care for their persistent SPT-related health problems during the first year postpartum. If persistent problems occurred and were acknowledged, the women were offered different surgical approaches with various outcomes, consultations by colon specialists, physiotherapy, and psychiatric care. They were grateful for the help they received but felt more comprehensive care was needed.

Feeling sane and empowered

Confirmation of persistent SPT-related health problems was expressed as liberating, strengthening and, as one woman put it, a ‘win’ (Elin). Receiving a medical diagnosis and appurtenant treatment was relieving because the medical confirmation of the symptoms released a considerable burden. These women described being acknowledged, and the diagnosis proved that health problems existed, and the struggles were not in vain. Furthermore, it explicitly stated to everyone, including themselves, that they were not ‘crazy’, ‘imagining things’ or ‘hysterical’.

“So, my laceration has been classified as an injury caused by the healthcare services. This was somehow a confirmation. It's not just that it's in my head, but it has been established that it is a medical injury, and it could have been avoided.” (Jin)

Alongside feelings of sanity and being legitimised, the women experienced empowerment. The women felt supported and confident. Thus, finding an agency to address the taboo of their SPT by talking openly about it and helping others in the same situation was also seen as therapeutic. Further, the legitimation of the sequelae and access to appropriate care gave them time to heal and process their trauma. Receiving sick leave certificates and benefits was seen as a part of the empowerment and legitimacy of their persistent SPT-related health problems, reducing stress, and easing the financial burden. Furthermore, access to occupational rehabilitation and understanding at work became available. Thus, the women who had received the help they needed after a struggle to obtain it were hopeful about the future and possible recovery.

“I have regained my authority to speak up. It [SPT-related health problems] should be out in the open, not withheld.” (Jaanika)

Our main finding was that women with persistent health problems due to SPT at childbirth were caught in a paradox of living in a normalised but traumatised body, and their health problems were rejected as postpartum normalities. Furthermore, our results elucidated the difficulties in accessing postpartum healthcare, rehabilitation, and sick leave benefits. Therefore, the women struggled with neglected healthcare needs, diminished emotional well-being, and loss of financial and social status. Our study highlighted experiences up to 5 years after sustaining SPT, which showed that some women’s SPT-related health problems do not diminish with time. They faced challenges functioning in daily life, at work, and in society. In contrast, finding a ‘key person’, i.e., a professional who acknowledged the women’s persistent problems as legitimate, was a prerequisite for accessing all the needed care and sick leave and enhancing empowerment for the women. Thus, this ‘key person’ was not blinded by the obstetric gaze and instead used their agency and advocacy as support.

In the following, we will discuss our findings related to other empirical studies and problematise them with theoretical reflections.

The paradox of normalising the postpartum body

In our findings, the paradox arose when the HCPs dismissed physical health problems after SPT despite women’s perceived symptoms. Central in this context was a normalisation process where health problems were regarded as ‘normal’ by HCPs, a phenomenon also found in prior research on SPT [ 17 , 18 , 19 , 20 , 21 , 22 ]. The HCPs’ normalisation of women’s health problems can also be found regarding other medical conditions affecting women, such as pelvic organ prolapse [ 52 ], menstrual pain [ 53 ], endometriosis [ 54 ] or nausea and vomiting during pregnancy [ 55 ]. In light of the medicalisation of women’s healthcare, where the medical field has sought to pathologise natural bodily processes such as pregnancy and childbirth [ 33 ], actual medical conditions such as persistent SPT-related health problems are paradoxically normalised. Our findings, therefore, highlight the need to challenge HCPs’ views of what constitutes a ‘normal postpartum body’ or ‘normal postpartum symptoms’ after sustaining SPT.

The key to healthcare

In the context of denied legitimacy of health problems and neglected needs, it appeared that the women became dependent on the goodwill of a ‘key person’, personified as the respectful, competent, and empathetic HCP. Prior research on SPT has also found women struggling with accessing healthcare [ 6 , 17 ] and specific HCPs as enablers of care [ 12 ]. The dependency on a ‘key person’ to access adequate care might highlight a structural problem within the provision of postpartum SPT-related healthcare. Globally, there are a few national guidelines on SPT management and prevention [ 56 ]. Additionally, no national guidelines regarding postpartum care of SPT exist in Sweden, and pelvic floor teams are only available in some Swedish regions [ 16 ]. In our study, the women lacked information, and competent HCPs were hard to find or located far away. Other studies have shown poor patient information and education as a postpartum problem [ 6 , 10 , 18 ], indicating a need to develop targeted oral and written information on wound healing and recovery. Further, women in Australia describe similar challenges to accessing SPT-related healthcare when having persistent SPT-related health problems [ 18 ]. The absence of national Australian guidelines may have led to inconsistent care, failing to meet women’s healthcare needs. Further, women from rural areas have had additional difficulties accessing needed care. In 2021, a clinical standard for SPT was implemented in Australia, comprising care standards for follow-up [ 57 ]. Thus, to improve the national situation in Sweden, more research and resources must be allocated to develop evidence-based recommendations, preferably internationally accepted guidelines [ 56 ]. Moreover, the accessibility of SPT-related healthcare, such as pelvic floor clinics, needs to be expanded so that women can easily meet their ‘key person’ if required.

Woman-(de)centred care?

We found that HCPs were obstructed by their obstetric gaze when assessing women with persistent SPT-related health problems. Obstetric gaze derives from the medical gaze notions [ 58 ], suggesting a gaze that splits the individual from the body, constructing the care-seeker as a medical object or condition instead of an individual with a social context. This gaze blinded HCPs who normalised obvious health problems. Recent advances in women’s healthcare in industrial countries and midwifery research show development towards continuity of care models with a woman-centred approach in different caseload-midwifery projects and informed choice regarding place of childbirth [ 28 , 59 , 60 , 61 ]. Wom e n-centred care [ 2 ] is a widespread care philosophy within midwifery that advocates for providing individualised care to women. Further, wom a n-centred care emphasises the individual woman’s healthcare needs and situation, incorporating the concepts of choice, control, continuity of caregiver, and self-determination. It can be argued that the obstetric gaze obstructed HCPs in providing wom a n-centred care because they did not acknowledge the women’s healthcare needs. Consequently, the women did not have control over their health situation. Making women feel empowered [ 2 , 62 ] is crucial in woman-centred care. Hence, the ‘key persons’ in our study managed to provide wom a n-centred care where acknowledgement of problems as real medical problems and access to care made the women experience empowerment. Therefore, we argue that guidelines regarding follow-up care after SPT should ideally be developed with wom a n-centred care as its core.

Everything looks fine

The biomedical model has traditionally focused on normality and abnormality rather than health [ 63 ]. Theoretically, the ‘obstetric gaze’ is closely tied to the ‘medical gaze’ and the ‘male gaze’, referring to the biomedical paradigm and its power [ 27 , 58 ]. In our study, the obstetric gaze judged the women’s persistent health problems due to SPT as ‘normal’ and the appearance of their genital area as ‘fine’, which created a paradoxical situation regarding the legitimacy of their ongoing health problems after SPT. Generally, the healthcare sector is critiqued for reducing the body to only incorporating organs and tissue, i.e., focusing on physical symptoms [ 27 ].

The women in our study, of which most showed more than one significant symptom after SPT, noted that HCPs would comment on the physical appearance of the perineal area rather than its functionality by telling them that ‘everything looked fine’. The focus on looks rather than functionality regarding SPT-related health problems aligns with the findings presented by others [ 17 ]. Having women describe how their persistent physical pelvic floor problems after SPT during childbirth are trivialised, normalised, questioned, and labelled as mental health issues is of utmost concern. This implies the need for rapid improvements in HCPs’ knowledge and organisation of care but also raises the question of what is considered a normal status and recovery after any perineal laceration in the short- and long-term perspective. A similar discursive focus on women’s appearance instead of their health problems has also been found among HCPs when women seek care for chronic pain [ 64 ]. The sentence ‘Everything looks fine’ can be interpreted as an objectifying, gendered discourse in an obstetric context. This discourse may reinforce the obstetric gaze and, in the broader sense, the medical gaze [ 58 ]. The Swedish Health and Medical Care Act [ 39 ] advocates for the respectful treatment of patients. Hence, it is noteworthy that the women experienced being judged by the looks of their genital area in their medical encounters rather than HCPs addressing the functionality. Such treatment does not align with the legislation and calls for a discourse analysis of the attitudes of HCPs towards women with persistent SPT-related health problems and their experiences of providing care for affected women.

Being subjected to obstetric gaslighting

In light of the women’s perception of their dismissal as dramatic, illegitimate, and irrational patients, we argue that they faced so-called ‘gaslighting’ in an obstetric context [ 65 , 66 ]. Thus, the women experienced being offered sick leave for mental problems instead of their perceived physical health problems, depicting them as hysterical women who exaggerated their condition. Gaslighting is a concept used in medicine in general [ 66 ] and in obstetrics regarding traumatic childbirth experiences [ 65 ]. The concept of hysteria, i.e., a prior medical diagnosis and historical concept theoretically linked to femininity [ 67 , 68 ] and ‘obstetric gaslighting’ [ 65 ], has also been found in research on women’s chronic pain [ 64 ] and endometriosis [ 69 ]. Men with chronic pain are perceived as brave, and women in pain are hysterical, emotional, whining, malingering, or imagining pain [ 64 ]. Further, women with endometriosis are viewed as ‘reproductive bodies’ with a proneness for hysteria [ 69 ]. Obstetric gaslighting, enforced by the normalisation of SPT-related health problems and the gendered stereotype of women as hysterical patients, puts women with SPT in an inferior position towards HCPs and can, therefore, be interpreted as a demonstration of institutional power [ 65 ]. Hence, being overlooked by the obstetric gaze might constitute a form of obstetric gaslighting, a concept that has not been applied to SPT before.

Implications and significance

Our study indicated that women continue to have problems accessing healthcare for persistent SPT-related health problems several years postpartum. Additionally, women with persistent SPT-related health problems often depended on a ‘key person’ with the competence to open the doors to comprehensive care, as shown in our findings. The Swedish Government launched a multi-million project from 2015 to 2022 to improve and promote women’s health [ 70 ]. Despite this investment, the depicted experiences of the included women reflect upon remaining structural and clinical problems within Swedish healthcare, which need further attention, investigation, and actions. Additionally, there are considerable differences in reported satisfaction and prevalence of complications at the one-year follow-up between the regions [ 3 ], indicating that there are suboptimal healthcare services. With a significant variation in satisfaction and recovery at one year, there are reasons to believe that women with prolonged problems may experience problems getting access to needed care.

Our study also showed that SPT-related healthcare services are not available on equal terms to women with persistent SPT-related health problems. In general, many women within this group had problems accessing care and sick leave for years. However, depending on where the women reside, not all women have access to specialised care. This inequity may be explained by Sweden having 21 self-governing health regions, and in the absence of national guidelines regarding SPT care and follow-up, the healthcare provision for affected women varies. To secure access to postpartum care for women with SPT in general and those with different prerequisites within this group, implementation studies are needed to develop and evaluate the effect of national guidelines for follow-up care regarding SPT.

Strengths and limitations

This study has strengths and limitations that need to be addressed. A significant strength, enhancing credibility and transferability, was providing a clear context and thick descriptions of our results, where we thoroughly portrayed the women’s voices using quotations [ 35 ]. Further, our detailed account of the study context, data collection, and data analysis process facilitated the transferability of our study. Including three women born outside of Sweden added to the variety of the sample and thus improved credibility because qualitative research often overlooks immigrants' experiences. However, the migrant women spoke Swedish well enough to participate in an interview, indicating that they have been living in Sweden for some time and might be familiar with the healthcare system. Finally, the credibility and dependability of this study were also strengthened by the frequent use of interdisciplinary triangulation between the authors throughout data analysis and the writing process, as well as peer review at a research seminar.

A potential limitation was that this study may not have fully explored the situation of women with fourth-degree lacerations or those with lower education, as most participants had third-degree perineal lacerations and higher education. Further, we could not include non-binary persons and same-sex or single parents, which may be a weakness; consequently, future studies should focus on the under-represented participant groups and migrant women needing an interpreter. Additionally, all women responded voluntarily to the study invitation. Thus, our participants might be particularly outspoken about their problems or interested in raising their voices or experiences. However, they represented a variety of persistent SPT-related health problems of various severity, and some had been able to get access to medical help, whereas others had not. Additionally, our findings cohered to similar studies [ 12 , 17 ] covering shorter periods after the SPT, which may indicate that the experiences of the challenging search for needed help remain over time. Therefore, our findings may reflect other women’s experiences seeking care for SPT-related health problems and may be transferable to other women’s experiences with persistent health problems of a rare condition.

The data for this study was comprehensive and rich. Information power in qualitative research is an ongoing discussion, and the number of participants and their representativity can be seen as a limitation of credibility and transferability [ 71 , 72 ]. Graneheim, Lindgren and Lundman [ 36 ] argue that sample size should be determined by the study’s aim and the data’s quality so that variations in experiences can be captured. They do, therefore, not recommend a specific number of participants, but others do [ 71 ]. With this in mind, the authors believe that the women’s detailed descriptions of the included concepts and the extensive length of the conducted interviews enabled us to achieve sufficient information power based on the richness of the data [ 72 ].

By qualitatively exploring how women with persistent SPT-related health problems experienced their healthcare encounters, we interpreted that they faced a paradox of being reassured of normality by HCPs despite reporting sequelae symptoms. Thus, women’s needs for medical care, rehabilitation, and sick leave were largely neglected. Further, our study might indicate a structural problem within women’s postpartum healthcare, indicating that access to care depended on encountering a ‘key person’, a professional who acknowledged persistent problems as real symptoms. Access to quality care provided with a professional attitude was essential for the future well-being of women with persistent SPT-related health problems. Thus, it should not depend on meeting a single ‘key person’. Therefore, national guidelines for long-term postpartum care of persistent SPT-related health problems must be developed in Sweden. Additionally, to ensure that healthcare services meet the individual needs of women with persistent SPT-related health problems, it is crucial to consider arranging the organisation and availability of quality care for these women from a woman-centred perspective.

Availability of data and materials

The original recordings and transcripts from the current study are not publicly available due to securing the individual privacy and confidentiality of the participants. Data are available from the corresponding author upon reasonable request.

Abbreviations

Healthcare professionals

Interquartile range

Strategic Research Area Health Care Science

• Severe perineal trauma

Sexual and reproductive health and rights

World Health Organisation

World Health Organization (WHO). WHO recommendations: Intrapartum care for a positive childbirth experience. Geneva: WHO; 2018. Cited 2024 Feb 14. Available from: https://www.who.int/publications/i/item/9789241550215 .

Leap N. Woman-centred or women-centred care: does it matter? Br J Midwifery. 2009;17(1):12–6.

Article   Google Scholar  

Uustal E. Bristning vid förlossning grad 3 - 4. Årsrapport från GynOp-registret avseende operationer utförda år 2022. [Laceration at childbirth, grade 3 - 4, Annual Report of surgeries 2022]. Umeå: The Swedish National Quality Register of Gynecological Surgery; 2023. Cited 2024 Maj 8. Available from: https://www.gynop.se/rapporter/arsrapporter/ .

World Health Organization (WHO). International statistical classification of diseases and related health problems (ICD) 10th revision. Geneva: WHO; 2019. Cited 2024 Maj 8. Available from: https://icd.who.int/browse10/2019/en .

d’Almeida I. Women’s experiences following obstetric anal sphincter injury. J Pelvic Obstet Gynaecol Physiother. 2020;127:39–50.

Google Scholar  

Darmody E, Bradshaw C, Atkinson S. Women’s experience of obstetric anal sphincter injury following childbirth: An integrated review. Midwifery. 2020;91:102820.

Article   PubMed   Google Scholar  

LaCross A, Groff M, Smaldone A. Obstetric anal sphincter injury and anal incontinence following vaginal birth: a systematic review and meta-analysis. J Midwifery Women’s Health. 2015;60(1):37–47.

Samarasekera DN, Bekhit MT, Wright Y, Lowndes RH, Stanley KP, Preston JP, et al. Long-term anal continence and quality of life following postpartum anal sphincter injury. Colorectal Dis. 2008;10(8):793–9.

Article   CAS   PubMed   Google Scholar  

Andreucci CB, Bussadori JC, Pacagnella RC, Chou D, Filippi V, Say L, et al. Sexual life and dysfunction after maternal morbidity: a systematic review. BMC Pregnancy Childbirth. 2015;15:307.

Article   PubMed   PubMed Central   Google Scholar  

Beck CT. Effects of fourth-degree perineal lacerations on women’s physical and mental health. J Obstet Gynecol Neonatal Nurs. 2021;50(2):133–42.

Keighley MR, Perston Y, Bradshaw E, Hayes J, Keighley DM, Webb S. The social, psychological, emotional morbidity and adjustment techniques for women with anal incontinence following Obstetric Anal Sphincter Injury: use of a word picture to identify a hidden syndrome. BMC Pregnancy Childbirth. 2016;16(1):275.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lindqvist M, Lindberg I, Nilsson M, Uustal E, Persson M. “Struggling to settle with a damaged body” - A Swedish qualitative study of women’s experiences one year after obstetric anal sphincter muscle injury (OASIS) at childbirth. Sex Reprod Healthc. 2019;19:36–41.

Shoorab NJ, Taghipour A, Mirteimouri M, Roudsari RL. Social recovery: a neglected dimension of caring for women with perineal trauma in Iran. Iran J Nurs Midwifery Res. 2020;25(4):333–40.

Jahani Shoorab N, Mirteimouri M, Taghipour A, Latifnejad Roudsari R. Women’s experiences of emotional recovery from childbirth-related perineal trauma: a qualitative content analysis. Int J Community Based Nurs Midwifery. 2019;7(3):181–91.

PubMed   Google Scholar  

Swedish Agency for Health Technology Assessment and Assessment of Social Services. Practices to improve detection of perineal tears and women’s views and experiences of healthcare providers following sustained perineal tear. A systematic review. 2021. Report No.: 323.

Persson M, Lindberg I, Ohman A. Regional and clinical guidelines for prevention and care of obstetric anal sphincter injuries - A critical frame analysis. Midwifery. 2023;119:103608.

Huber M, Tunon K, Lindqvist M. “From hell to healed” - A qualitative study on women’s experience of recovery, relationships and sexuality after severe obstetric perineal injury. Sex Reprod Healthc. 2022;33:100736.

Priddis HS, Schmied V, Kettle C, Sneddon A, Dahlen HG. “A patchwork of services”–caring for women who sustain severe perineal trauma in New South Wales–from the perspective of women and midwives. BMC Pregnancy Childbirth. 2014;14:236.

Priddis H, Schmied V, Dahlen H. Women’s experiences following severe perineal trauma: a qualitative study. BMC Womens Health. 2014;14(1):32.

Williams A, Lavender T, Richmond DH, Tincello DG. Women’s experiences after a third-degree obstetric anal sphincter tear: a qualitative study. Birth. 2005;32(2):129–36.

Herron-Marx S, Williams A, Hicks C. A Q methodology study of women’s experience of enduring postnatal perineal and pelvic floor morbidity. Midwifery. 2007;23(3):322–34.

Crookall R, Fowler G, Wood C, Slade P. A systematic mixed studies review of women’s experiences of perineal trauma sustained during childbirth. J Adv Nurs. 2018;74(9):2038–52.

Priddis H, Dahlen H, Schmied V. Women’s experiences following severe perineal trauma: a meta-ethnographic synthesis. J Adv Nurs. 2013;69(4):748–59.

Tjernström K, Lindberg I, Wiklund M, Persson M. Negotiating the ambiguity of an (in)authentic working life: a grounded theory study into severe perineal trauma. BMC Women’s Health. 2023;23(1):47.

Lindqvist M, Persson M, Nilsson M, Uustal E, Lindberg I. “A worse nightmare than expected” - a Swedish qualitative study of women’s experiences two months after obstetric anal sphincter muscle injury. Midwifery. 2018;61:22–8.

United Nations. The Sustainable Development Goals Report 2022. New York City: United Nations; 2022. Cited 2024 Feb 24. Available from: https://unstats.un.org/sdgs/report/2022/ .

Fahy K, Foureur M, Hastie C. Birth territory and midwifery guardianship : theory for practice, education, and research. Edinburgh: Books for Midwives; 2008.

Walsh D, Christianson M, Stewart M. Why midwives should be feminists. Midirs Midwifery Digest. 2015;25(2):154–60.

Buchanan K, Newnham E, Geraghty S, Whitehead L. Navigating midwifery solidarity: a feminist participatory action research framework. Women Birth. 2023;36(1):e169–74.

Benoit B, Goldberg L, Campbell-Yeo M. Infant feeding and maternal guilt: The application of a feminist phenomenological framework to guide clinician practices in breast feeding promotion. Midwifery. 2016;34:58–65.

Westergren A, Edin K, Walsh D, Christianson M. Autonomous and dependent-The dichotomy of birth: a feminist analysis of birth plans in Sweden. Midwifery. 2019;68:56–64.

Alspaugh A, Barroso J, Reibel M, Phillips S. Women’s contraceptive perceptions, beliefs, and attitudes: an integrative review of qualitative research. J Midwifery Womens Health. 2020;65(1):64–84.

Davis DL, Walker K. Re-discovering the material body in midwifery through an exploration of theories of embodiment. Midwifery. 2010;26(4):457–62.

Govender V, Penn-Kekana L. Gender biases and discrimination: a review of health care interpersonal interactions. Global Public Health. 2008;3(sup1):90–103.

Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12.

Graneheim UH, Lindgren BM, Lundman B. Methodological challenges in qualitative content analysis: a discussion paper. Nurse Educ Today. 2017;56:29–34.

Lindgren BM, Lundman B, Graneheim UH. Abstraction and interpretation during the qualitative content analysis process. Int J Nurs Stud. 2020;108:103632.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Ministry of Health and Social Affairs. Hälso- och sjukvårdslag (2017:30) [Health and Medical Care Act]. Stockholm: The Swedish Riksdag; 2017. Cited 2024 Maj 8. Available from: https://www.riksdagen.se/sv/dokument-och-lagar/dokument/svensk-forfattningssamling/halso-och-sjukvardslag-201730_sfs-2017-30/ .

The Swedish Social Insurance Agency. Parental benefits. 2023. Cited 2024 Feb 20. Available from: https://www.forsakringskassan.se/english/parents/when-the-child-is-born/parental-benefit .

The Swedish Association of Midwives. Kompetensbeskrivning för legitimerad barnmorska [Description of competences for registered midwives]. Stockholm: The Swedish Association of Midwives; 2018. Updated 2019 Jan. Cited 2024 Feb 20. Available from: https://storage.googleapis.com/barnmorskeforbundet-se/uploads/2020/04/Kompetensbeskrivning-for-legitimerad-barnmorska.pdf .

Bäckenbottenutbildning [Pelvic floor education]. Utbildningsmaterial: Uppföljning [Educational material: Follow-up]. 2022. Cited 2024 Maj 8. Available from: https://backenbottenutbildning.se/index.php/utbildningsmaterial/uppfoljning .

GynOp [The Swedish National Quality Register of Gynecological Surgery]. About GynOp. 2023. Cited 2024 Feb 20. Available from: https://www.gynop.se/home/about-gynop/ .

Öhman H, Schesny S, Bearth L, Löwendahl J. Förlossningsskadad? Du är inte ensam! [Injured at childbirth? You are not alone!’]. 2014. Cited 2024 Feb 20. Available from: https://www.facebook.com/groups/forlossningsskadad/ .

Vanderbilt University. About REDCap. Nashville: Vanderbilt; 2023. Updated 2023. Cited 2024 Feb 20. Available from: https://projectredcap.org/about/ .

Kvale S, Brinkmann S. Interviews: learning the craft of qualitative research interviewing. 2nd ed. Los Angeles: Sage Publications; 2009.

Keen S, Lomeli-Rodriguez M, Joffe H. From challenge to opportunity: virtual qualitative research during COVID-19 and beyond. Int J Qual Methods. 2022;21:16094069221105076.

Thunberg S, Arnell L. Pioneering the use of technologies in qualitative research - A research review of the use of digital interviews. Int J Soc Res Methodol. 2022;25(6):757–68.

MAXQDA. Why MAXQDA? Berlin: VERBI Software GmbH; 2024. Cited 2024 Maj 8. Available from: https://www.maxqda.com/why-maxqda .

Microsoft. Microsoft Excel. Redmond: Microsoft; 2024. Cited 2024 Maj 8. Available from: https://www.microsoft.com/sv-se/microsoft-365/excel?market=se .

Griffin GA. Dictionary of Gender Studies. Oxford: Oxford University Press; 2017.

Book   Google Scholar  

Abhyankar P, Uny I, Semple K, Wane S, Hagen S, Wilkinson J, et al. Women’s experiences of receiving care for pelvic organ prolapse: a qualitative study. BMC Womens Health. 2019;19(1):45.

Scott KD, Hintz EA, Harris TM. “Having pain is normal”: How talk about chronic pelvic and genital pain reflects messages from menarche. Health Commun. 2022;37(3):296–306.

Pettersson A, Bertero CM. How women with endometriosis experience health care encounters. Womens Health Rep (New Rochelle). 2020;1(1):529–42.

Heitmann K, Svendsen HC, Sporsheim IH, Holst L. Nausea in pregnancy: attitudes among pregnant women and general practitioners on treatment and pregnancy care. Scand J Prim Health Care. 2016;34(1):13–20.

Roper JC, Amber N, Wan OYK, Sultan AH, Thakar R. Review of available national guidelines for obstetric anal sphincter injury. Int Urogynecol J. 2020;31(11):2247–59.

The Australian Commission on Safety and Quality in Health Care. Third and Fourth Degree Perineal Tears Clinical Care Standard. Sydney: The Australian Commission on Safety and Quality in Health Care; 2021. Cited 2024 Feb 20. Available from: https://www.safetyandquality.gov.au/standards/clinical-care-standards/third-and-fourth-degree-perineal-tears-clinical-care-standard .

Foucault M, Sheridan A. The birth of the clinic: an archaeology of medical perception. 3rd ed. London: Routledge; 2003.

Scarf VL, Rossiter C, Vedam S, Dahlen HG, Ellwood D, Forster D, et al. Maternal and perinatal outcomes by planned place of birth among women with low-risk pregnancies in high-income countries: a systematic review and meta-analysis. Midwifery. 2018;62:240–55.

Offerhaus P, Jans S, Hukkelhoven C, de Vries R, Nieuwenhuijze M. Women’s characteristics and care outcomes of caseload midwifery care in the Netherlands: a retrospective cohort study. BMC Pregnancy Childbirth. 2020;20(1):517.

Forster DA, McLachlan HL, Davey MA, Biro MA, Farrell T, Gold L, et al. Continuity of care by a primary midwife (caseload midwifery) increases women’s satisfaction with antenatal, intrapartum and postpartum care: results from the COSMOS randomised controlled trial. BMC Pregnancy Childbirth. 2016;16:28.

Brady S, Lee N, Gibbons K, Bogossian F. Woman-centred care: an integrative review of the empirical literature. Int J Nurs Stud. 2019;94:107–19.

Findlay D. The medical gaze: medical models, power, and women’s health. Atlantis. 1992;18(1&2):104–24.

Samulowitz A, Gremyr I, Eriksson E, Hensing G. “Brave Men” and “Emotional Women”: a theory-guided literature review on gender bias in health care and gendered norms towards patients with chronic pain. Pain Res Manag. 2018;2018:6358624.

Fielding-Singh P, Dmowska A. Obstetric gaslighting and the denial of mothers’ realities. Soc Sci Med (1982). 2022;301:114938.

Sebring JCH. Towards a sociological understanding of medical gaslighting in western health care. Sociol Health Ill. 2021;43(9):1951–64.

Jones CE. Wandering wombs and “female troubles”: the hysterical origins, symptoms, and treatments of endometriosis. Women Stud. 2015;44(8):1083–113.

Tasca C, Rapetti M, Carta MG, Fadda B. Women and hysteria in the history of mental health. Clin Pract Epidemiol Ment Health. 2012;8:110–9.

Young K, Fisher J, Kirkman M. “Do mad people get endo or does endo make you mad?”: Clinicians’ discursive constructions of Medicine and women with endometriosis. Fem Psychol. 2019;29(3):337–56.

The Swedish Ministry of Health and Social Affairs. Överenskommelse om en förbättrad förlossningsvård och insatser för kvinnors hälsa [Agreement on improving maternity care and women's health] [Internet]. Stockholm: The Swedish Government; 2015. S2015/07777/FS. [cited 2024 Maj 8]. Available from: https://www.regeringen.se/overenskommelser-och-avtal/2015/12/overenskommelse-om-en-forbattrad-forlossningsvard-och-insatser-for-kvinnors-halsa/ .

Boddy CR. Sample size for qualitative research. Qual Mark Res. 2016;19(4):426–32.

Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res. 2016;26(13):1753–60.

Download references

Acknowledgements

We want to thank the participating women for generously sharing their experiences.

Open access funding provided by Umea University. This work was supported by the Research Lift (SWE: Forskningslyftet) and Strategic Research Area Health Care Science (SFO-V), Umeå University. The funders had no specific role in the conceptualisation, design, data collection, analysis, publication decision, or manuscript preparation.

Author information

Authors and affiliations.

Department of Nursing, Umeå University, 901 87, Umeå, Sweden

Katharina Tjernström, Inger Lindberg & Margareta Persson

Department of Community Medicine and Rehabilitation, Section of Physiotherapy, Umeå University, 901 87, Umeå, Sweden

Maria Wiklund

You can also search for this author in PubMed   Google Scholar

Contributions

KT: conceptualisation; data curation; formal analysis; investigation; methodology; validation; visualisation; writing - original draft; writing - review & editing. IL: conceptualisation; methodology; supervision; visualisation; writing - review & editing. MW: conceptualisation; methodology; supervision; visualisation; writing - review & editing. MP: conceptualisation; data curation; funding acquisition; methodology; project administration; supervision; visualisation; writing - review & editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Katharina Tjernström .

Ethics declarations

Ethics approval and consent to participate.

The Swedish Ethical Review Authority approved the study. An amendment to the original ethical approval of the research project to further explore the experiences of encounters with healthcare services was obtained (Dnr: 2020-035410908 and Dnr: 2022-02784-02). The study was undertaken in compliance with research ethics guidelines. All participation was voluntary, and participants received oral and written information about the study and provided written informed consent before the interviews. No interview questions were mandatory, and the women decided if and how detailed they wanted to share their experiences.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12913_2024_11037_moesm1_esm.pdf.

Additional file 1. Semi-structured interview guide for individual interviews; contains interview questions aimed at highlighting the experience of everyday life and working life after suffering 3 rd or 4 th degree perineal laceration at childbirth (i.e., severe perineal trauma [SPT]).

Additional file 2. Consolidated criteria for reporting qualitative studies (COREQ): 32-item checklist.

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Tjernström, K., Lindberg, I., Wiklund, M. et al. Overlooked by the obstetric gaze – how women with persistent health problems due to severe perineal trauma experience encounters with healthcare services: a qualitative study. BMC Health Serv Res 24 , 610 (2024). https://doi.org/10.1186/s12913-024-11037-5

Download citation

Received : 11 September 2023

Accepted : 23 April 2024

Published : 09 May 2024

DOI : https://doi.org/10.1186/s12913-024-11037-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Persistent health problems
• Qualitative content analysis
• Healthcare encounters
• Postpartum healthcare
• Normalisation
• Access to care
• Empowerment

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Research Note
• Open access
• Published: 15 May 2024

Concepts of lines of therapy in cancer treatment: findings from an expert interview-based study

• Lisa Falchetto 1   na1 ,
• Bernd Bender 1 , 2   na1 ,
• Ian Erhard 1 , 2 ,
• Kim N. Zeiner 3 ,
• Jan A. Stratmann 11 ,
• Florestan J. Koll 4 ,
• Sebastian Wagner 11 ,
• Marcel Reiser 5 ,
• Khayal Gasimli 6 ,
• Angelika Stehle 7 ,
• Martin Voss 8 ,
• Olivier Ballo 11 ,
• Jörg Janne Vehreschild 1 , 9 , 10 &
• Daniel Maier 1 , 2  

BMC Research Notes volume  17 , Article number:  137 ( 2024 ) Cite this article

Metrics details

The concept of lines of therapy (LOT) in cancer treatment is often considered for decision making in tumor boards and clinical management, but lacks a common definition across medical specialties. The complexity and heterogeneity of malignancies and treatment modalities contribute to an inconsistent understanding of LOT among physicians. This study assesses the heterogeneity of understandings of the LOT concept, its major dimensions, and criteria from the perspective of physicians of different specialties with an oncological focus in Germany. Semi-structured expert interviews with nine physicians were conducted and evaluated using qualitative content analysis.

Most interviewees agreed that there is no single definition for LOT and found it difficult to explicate their understanding. A majority of experts stated that they had already encountered misunderstandings with colleagues regarding LOT and that they had problems with deciphering LOT from the medical records of their patients. Disagreement emerged about the roles of the following within the LOT concept: maintenance therapy, treatment intention, different therapy modalities, changing pharmaceutical agents, and therapy breaks. Respondents predominantly considered the same criteria as decisive for the definition of LOT as for a change in LOT (e.g., the occurrence of a progression event or tumor recurrence).

Peer Review reports

Introduction

While clinical oncology considers line of therapy (LOT) essential information for therapy planning, the field lacks a homogeneous understanding of the concept, as well as clear and consistent criteria for its classification [ 1 ]. Especially in real-world data-based research, it is often unclear whether a certain therapy is still part of an LOT; and often, conflicting interpretations lead to misunderstandings in information exchange about therapy progression [ 1 ]. Existing approaches, for standardizing the classification of LOT either focus on patterns proposed by guidelines (e.g., drug administration period, first-line termination) or on drug administration sequences [ 2 , 3 , 4 , 5 , 6 ]. However, other issues related to the LOT concept remain largely unclear. For example, the roles of maintenance therapies and local therapy modalities have not yet been discussed [ 1 ].

This expert-interview study aims to provide a better conceptual understanding of the defining criteria of LOT for solid and non-solid cancers. Therefore, it may contribute to identifying unclear aspects of the LOT concept and avoiding misunderstandings in communication about LOTs, especially between physicians of different medical disciplines. Concerning the rapidly developing field of real-world cancer research, data augmentation strategies and feature engineering require empirically validated concepts to obtain reliable evidence from observational data. More specifically, investigating the conceptual understanding of LOTs will help us build a rule-based framework for LOT classification within the Clinical Communication Platform of the German Cancer Consortium (DKTK).

The study’s target group was physicians from various specialties with an oncological focus, working in either university hospitals or private practice. Physicians from the University Hospital Frankfurt and private practices were contacted by e-mail. In total, nine were interviewed. Their varied specialties included neuro-oncology, pulmonology, hematology and medical oncology, urology, dermatology, and gynecological oncology, as well as one resident specialist in internal medicine with a focus on hematology and oncology. The interviewees’ professional experience ranged from 3.5 to 29 years and most had experience in treating both solid and non-solid malignancies.

Qualitative expert interviews [ 7 , 8 ] were conducted by posing open questions within a semi-structured framework [ 9 ]. An interview manual delineated this framework and was developed based on existing literature about oncological LOTs and associated concepts (see Additional File 1 ). Before the interviews, the interview manual was pre-tested with an experienced oncologist and adjusted accordingly. Each participant declared their consent before the interview. Confidentiality and anonymity of participants’ responses and information were assured. The first part of the interview manual asked about the interviewee’s underlying understanding of LOTs and the relevant criteria for their definition. Subsequently, questions concerning misunderstandings in interactions with colleagues were posed to determine whether there are frequent uncertainties in the use of the LOT concept and, if so, what reasons may underlie this situation. Next, the interviewer asked about how specific criteria, picked out of the literature, related to the definition of LOT. These included the influence of treatment intention, the role of maintenance therapy, and local therapies. Another focus of the interviews was how the interviewees judged the relationship of both changes in drug regimen and therapy breaks to the definition of LOT.

Data collection/conduct of interviews

The expert interviews were conducted between June 1 and July 17, 2022 via video conference and in German. They lasted between 10 and 25 minutes with an average duration of approximately 18 minutes. The interviews were recorded and transcribed using the ExpressScribe Pro software (Version 10.17).

Data analysis

The interviews were analyzed using methods of qualitative content analysis as described in Mayring [ 10 ] and the software MaxQDA Analytics Pro 2022 (release 22.2.0). A system for coding the interview material was developed based on literature research conducted before the interviews.

Since the interviews were conducted in German, we provide an English translation of selected quotes. Table  1 contains the main topics and sub-topics of the interview, as well as exemplary quotes from the interviewees.

LOT definition and misunderstandings

Most interviewees confirmed that there was no common understanding of LOT and that they had difficulties explicating their own understanding of the concept. Furthermore, four of the interviewees reported misunderstandings with colleagues regarding LOTs and seven reported that they experienced uncertainties in their clinical practice when defining an LOT. For instance, if care for a patient was delivered by multiple centers, misunderstandings concerning LOT progression frequently occurred, because involved persons lacked a common understanding:

“[…] when it comes to categorizing it somehow so that it is standardized and applicable across multiple centers, yes there existed discrepancies in the particular considerations.” (Expert interview (E)05).

Treatment intention

Six interviewees said that treatment intention (curative vs. palliative) is important in the choice of therapy. Consequently, treatment intention is also relevant to LOT planning. Three experts expressed that LOT is especially relevant and established in the palliative setting:

“With a curative therapy option, […] you shouldn’t have any progression under therapy, after all. So that’s why the definition [of the line of therapy] does differ somewhat – palliative versus curative.” (E03).

Maintenance therapy

Starting a maintenance therapy to control a tumor after chemotherapy was predominantly not considered an indicator for a change in LOT, since usually only part of the medication regimen is discontinued for maintenance, while the rest remains the same. However, interviewees also said that maintenance therapy can include an entirely new pharmaceutical agent, which would, in turn, complicate the delineation between LOT:

“Yes, that’s difficult, too. I would probably count maintenance therapy as part of that – if it’s sort of quasi-logically linked to the therapy that was administered before it. But if it’s a completely different type of substance now, then it becomes more difficult again.” (E03).

Local therapies vs. systemic therapies

Six of the physicians interviewed opined that a LOT can contain both local and systemic therapies. However, some participants stated that beginning a new local therapy would not lead to a change of LOT, in contrast to beginning a new systemic therapy. Meanwhile, in contrast to the other six, three physicians emphasized that only systemic therapies can constitute a LOT:

“In my opinion, the therapy line is primarily defined by the systemic therapies. The local therapies are rather something supplementary that is carried out additionally, or – as the case may be – primarily in addition to symptom relief. Local therapies can also be used to achieve a response, but are not usually mentioned as a line of therapy.” (E06).

Change of LOT

All interviewees said that the LOT must be changed if tumor progression or disease relapse occurs or if therapy response fails. Six interviewees considered the occurrence of adverse effects (e.g., severe toxicity) a significant criterion for the decision to change an LOT. Only three interviewees saw the addition of a new pharmaceutical agent as resulting in a change of LOT:

“Dropping an active substance, I would always see as being due to toxicity or at the patient’s request – so actually owed to toxicity. That is, I would never call that a new line of therapy, whereas the addition of a new agent – strictly speaking, it would have to be considered a new line of therapy, although it is also difficult in terms of definition.” (E09).

The other seven interviewees only considered the introduction of new pharmaceutical agents a change in LOT if the treatment intention changed as well, or if a recurrence or progression occurred. Only the replacement of one drug with another of the same class (e.g., cisplatin with carboplatin) was not considered a change of LOT by anyone.

Therapy breaks

There were also ambiguous opinions regarding the role of breaks in therapy for the classification of LOT. On the one hand, the length of the break was considered decisive, whereas on the other hand, it was said that the therapy following the break was more important. Additionally, some viewed breaks in therapy as important for the classification of LOT in the event of a relapse or progression:

“[…] In principle, if no recurrence has occurred and it is perhaps even the same substance […] then I would consider it one line of therapy, regardless of how long the break was.” (E01).

If the break was unplanned, it was considered a significantly more important criterion for a change in LOT than if it was part of the therapy concept.

The expert interviews in this study largely confirmed that there is no common understanding of the LOT concept or its defining criteria. The interview material suggests that individual backgrounds in differing medical disciplines may influence views on and understandings of LOT. This potential context dependency of the LOT concept also appears consistent with heterogeneous working definitions of LOT in different real-world studies of distinct cancer entities [ 1 , 11 , 12 ].

However, it appeared that a LOT was considered a therapeutic concept with start- and endpoints that is focused on systemic therapies, although it may also contain additional treatment modalities. If included in the LOT, such non-systemic modalities would be selected based on individual patient and disease characteristics, and terminated if certain events (e.g., tumor progression) occurred.

There was evident uncertainty about the role of adjuvant and maintenance therapy and whether they should be regarded as an LOT together with the preceding (systemic) therapy. Also, no prevailing opinion could be identified on the questions of whether treatment intention (curative vs. palliative) and therapy breaks were integral to defining LOTs. Furthermore, experts held differing opinions on which changes in the administered drug regimen would initiate a change in LOT.

In the literature, however, individual approaches for standardizing the criteria for a change in LOT exist in the following cases: the termination of a LOT is indicated in the event of treatment discontinuation, addition of a new, non-equivalent agent, interruption of treatment, clinical progression of the disease, or death of the patient [ 2 , 3 ]. The interviewees were also nearly unanimous on these criteria: all considered tumor progression and recurrence decisive for a change in LOT; six experts highlighted the occurrence of side effects or relevant toxicity; three mentioned the scheduled end of therapy; and one cited patients’ wishes. Only some of the interviewees considered a change in pharmaceutical regimen a factor in identifying a change in LOT, while replacement of one drug with another from the same class was not viewed as altering the LOT.

The interviews both identified tumor recurrence and progression as LOT-relevant events and raised questions about the nature of their role. Recurrence and progression during therapy breaks, as well as the length of the break and the treatment thereafter, were considered relevant factors for a change in LOT. In two interviews, although the participants initially identified recurrence and progression as indicators for a change in LOT, their further comments appeared to contradict this standpoint. This apparent inconsistency should be investigated in future research.

Seven interviewees considered treatment intention relevant to LOT. Predominantly, interviewees considered the adoption of maintenance therapy as a continuation of an ongoing LOT. However, it remains unclear whether changes in the dosage or interval of drug administration during maintenance therapy imply a change in LOT. Six interviewees said that both local and systemic therapy modalities should be included in characterizations of LOT, although previous research excluded local modalities [ 1 , 13 , 14 , 15 ].

While similar approaches to standardizing the duration of a LOT [ 2 ] and first-line therapy [ 2 , 3 ] exist, it is not clear whether the definition of LOT can be standardized across disciplines as well as tumor entities. Nevertheless, a cross-disciplinary standard definition of the LOT concept should be targeted.

Limitations

This study exhibits the following limitations:

Qualitative expert interviews were only feasible for a small sample ( n  = 9) of oncological experts, most of whom were located at a single center (eight out of nine). While the study delivers highly granular insights, this approach precludes generalization of the findings. Therefore, subsequent research must evaluate the qualitative insights leaned from this study in larger and more representative samples.

The interviewees had varying degrees of professional experience and different specialties, making direct comparisons of experience and assessments regarding oncological LOT difficult. However, this was intentional to obtain the widest possible range of assessments regarding the broad topic under investigation.

No triangulation in the form of using multiple and diverse data sources, perspectives, locations, or theories took place in conducting the study. Such methods can help to mitigate subjective bias resulting from the explicit focus on one’s own data [ 16 ].

Data availability

Details on the data and materials related to the study may be available upon reasonable request from Bernd Bender ([email protected]).

Abbreviations

German Cancer Consortium

• Expert interview

Line of therapy

Saini KS, Twelves C. Determining lines of therapy in patients with solid cancers: a proposed new systematic and comprehensive framework. Br J Canc. 2021;125(2):155–63.

Article   Google Scholar  

OPTUM. Determining lines of therapy (LOT) in oncology in claims databases. 2018. https://www.optum.com/content/dam/optum3/optum/en/resources/white-papers/guidelines-for-determining-lines-of-therapy-whitepaper.pdf . Accessed 28 March 2023.

Hess LM, Li X, Wu Y, Goodloe RJ, Cui ZL. Defining treatment regimens and lines of therapy using real-world data in oncology. Future Oncol. 2021;17(15):1865–77.

Article   CAS   PubMed   Google Scholar  

Rajkumar SV, Richardson P, San Miguel JF. Guidelines for determination of the number of prior lines of therapy in multiple myeloma. Blood. 2015;126(7):921–2.

Carroll NM, Burniece KM, Holzman J, McQuillan DB, Plata A, Ritzwoller DP. Algorithm to identify systemic cancer therapy treatment using structured electronic data. JCO Clin Cancer Inf. 2017;1:1–9.

Google Scholar  

Weymann D, Costa S, Regier DA. Validation of a cyclic algorithm to proxy number of lines of systemic cancer therapy using administrative data. JCO Clin Cancer Inf. 2019;3:1–10.

Döringer S. The problem-centred expert interview’. Combining qualitative interviewing approaches for investigating implicit expert knowledge. Int J Soc Res Methodol. 2021;24(3):265–78.

Cooke NM, McDonald JE. A formal methodology for acquiring and representing expert knowledge. Proc IEEE Inst Electr Electron Eng. 1986;74(10):1422–30.

Kallio H, Pietilä AM, Johnson M, Kangasniemi M. Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. J Adv Nurs. 2016;72(12):2954–65.

Article   PubMed   Google Scholar  

Mayring P. Qualitative content analysis: demarcation, varieties, developments. Forum Qual Social Res. 2019;20(3):1–26.

Abrams T, Hess LM, Zhu YE, Schelman W, Liepa AM, Fuchs C. Predictors of heterogeneity in the first-line treatment of patients with advanced/metastatic gastric cancer in the US. Gastric Cancer. 2018;21:738–44.

Meng W, Ou W, Chandwani S, Chen X, Black W, Cai Z. Temporal phenotyping by mining healthcare data to derive lines of therapy for cancer. J Biomed Inf. 2019;100:103335.

Shah S, Raskin L, Cohan D, Freeman M, Hamid O. Treatment patterns of malignant melanoma in the United States from 2011 to 2016: a retrospective cohort study. CMRO. 2020;36:63–72.

CAS   Google Scholar  

Hess GP, Wang PF, Quach D, Barber B, Zhao Z. Systemic therapy for metastatic colorectal cancer: patterns of chemotherapy and biologic therapy use in US medical oncology practice. J Oncol Pract. 2010;6(6):301–7.

Article   PubMed   PubMed Central   Google Scholar  

Fonseca R, Usmani SZ, Mehra M, Slavcev M, He J, Cote S, Lam A, Ukropec J, Maiese EM, Nair S, Potluri R, Voorhees PM. Frontline treatment patterns and attrition rates by subsequent lines of therapy in patients with newly diagnosed multiple myeloma. BMC Cancer. 2020;20:1087.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cho J, Lee EH. Reducing confusion about grounded theory and qualitative content analysis: similarities and differences. Qual Rep. 2014;19(64):1–20.

Download references

Acknowledgements

We would like to thank the expert physicians who participated in the interviews for their time and willingness to share their experiences and perspectives. Furthermore, we would like to thank the German Cancer Consortium’s Clinical Data Science Group for the support in realizing the study.

Open Access funding enabled and organized by Projekt DEAL. This research is partly funded by the German Cancer Consortium (DKTK).

Author information

Lisa Falchetto and Bernd Bender contributed equally to this work.

Authors and Affiliations

Institute for Digital Medicine and Clinical Data Science, Goethe University Frankfurt, Faculty of Medicine, Frankfurt, Germany

Lisa Falchetto, Bernd Bender, Ian Erhard, Jörg Janne Vehreschild & Daniel Maier

German Cancer Consortium (DKTK), partner site Frankfurt/Mainz and German Cancer Research Center (DKFZ), Heidelberg, Germany

Bernd Bender, Ian Erhard & Daniel Maier

Department for Dermatology, Venerology and Allergology, University Hospital Frankfurt, Frankfurt, Germany

Kim N. Zeiner

Department of Urology, University Hospital Frankfurt, Frankfurt, Germany

Florestan J. Koll

PIOH Praxis Internistischer Onkologie und Hämatologie, Cologne, Germany

Marcel Reiser

Clinic for Gynecology and Obstetrics, University Hospital Frankfurt, Frankfurt, Germany

Khayal Gasimli

Department for Internal Medicine 1, University Hospital Frankfurt, Frankfurt, Germany

Angelika Stehle

Department Neuro-Oncology, University Hospital Frankfurt, Frankfurt, Germany

Martin Voss

Department I of Internal Medicine, University Hospital of Cologne, Cologne, Germany

Jörg Janne Vehreschild

German Center for Infection Research (DZIF) partner site Bonn Cologne, Cologne, Germany

Medical Department 2 (Hematology/Oncology), Center for Internal Medicine, University Hospital Frankfurt, Goethe University Frankfurt, Frankfurt, Germany

Jan A. Stratmann, Sebastian Wagner & Olivier Ballo

You can also search for this author in PubMed   Google Scholar

Contributions

BB, LF, and DM contributed to the writing of this article. LF and DM created the interview manual. LF conducted the interviews with the oncological experts and analyzed the interview material collected. DM and JJV were substantially involved in the conception of the study and in the acquisition of the interviewed experts. JJV also supported the piloting of the interview manual. IE edited the manuscript. KNZ, JAS, FJK, SW, MR, KG, AS, MV and OB participated in the study and provided the substantive statements and findings.

Corresponding author

Correspondence to Bernd Bender .

Ethics declarations

Ethics approval and consent to participate.

All subjects provided written informed consent to participate and this study was conducted according to all relevant ethical and regulatory guidelines. The project was approved by the ethics committee of the department of medicine of the Goethe University Frankfurt (ethical code number: 274/18).

Consent for publication

All interviewees permitted the use of the interview material and consented to publication.

Competing interests

Kim N. Zeiner (KNZ) received an honorarium for presentation from Bristol-Myers Squibb. Jan A. Stratmann (JAS) has personal fees from Boehringer Ingelheim, AstraZeneca, Roche, BMS, Amgen, LEO pharma, Novartis and Takeda. Florestan J. Koll (FJK) received grants from the German Cancer Aid and the German Cancer Consortium (DKTK). Marcel Reiser (MR) received consulting fees from Amgen, Abbvie, Stemline, Novartis and honoria from Roche. Jörg Janne Vehreschild (JJV) has personal fees from Merck / MSD, Gilead, Pfizer, Astellas Pharma, Basilea, German Centre for Infection Research (DZIF), University Hospital Freiburg/ Congress and Communication, Academy for Infectious Medicine, University Manchester, German Society for Infectious Diseases (DGI), Ärztekammer Nordrhein, University Hospital Aachen, Back Bay Strategies, German Society for Internal Medicine (DGIM), Shionogi, Molecular Health, Netzwerk Universitätsmedizin, Janssen, NordForsk, Biontech, APOGEPHA and grants from Merck / MSD, Gilead, Pfizer, Astellas Pharma, Basilea, German Centre for Infection Research (DZIF), German Federal Ministry of Education and Research (BMBF), Deutsches Zentrum für Luft- und Raumfahrt (DLR), University of Bristol, Rigshospitalet Copenhagen. Daniel Maier (DM) received speaker honoraria from Free University Berlin and travel compensation from IQVIA. Lisa Falchetto (LF), Bernd Bender (BB), Ian Erhard (IE), Sebastian Wagner (SW), Khayal Gasimli (KG), Angelika Stehle (AS), Martin Voss (MV) and Olivier Ballo (OB) have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Additional file 1.

Interview manual with all instructions and questions.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Falchetto, L., Bender, B., Erhard, I. et al. Concepts of lines of therapy in cancer treatment: findings from an expert interview-based study. BMC Res Notes 17 , 137 (2024). https://doi.org/10.1186/s13104-024-06789-6

Download citation

Received : 28 July 2023

Accepted : 25 April 2024

Published : 15 May 2024

DOI : https://doi.org/10.1186/s13104-024-06789-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Lines of therapy
• cancer treatment
• Therapy planning

BMC Research Notes

ISSN: 1756-0500

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Exploring the usage of learning resources by medical students in the basic science stage and their effect on academic performance

• Open access
• Published: 15 May 2024
• Volume 24 , article number  543 , ( 2024 )

Cite this article

You have full access to this open access article

• Sabin Kumar Ranabhat   ORCID: orcid.org/0000-0002-0658-5068 1 ,
• Mohan Lal Kunjukrishnan 2 ,
• Muskan Dubey 3 ,
• Vernon Curran 4 ,
• Arun Kumar Dubey 5 &
• Neelam Dwivedi 6  

The United States Medical Licensing Examination (USMLE) step 1 is one of the two examinations written after completion of the first two years (basic science stage) of medical school to be eligible to apply for residency training in the USA. A huge number and types of study materials are available to prepare for the exam which might confuse students choosing a resource.

We investigated learning resources being used by the third and fifth-semester medical students and their association with academic performance. We also compared learning resources and exam scores of high-performing and low-performing students.

Data collection was done using structured (quantitative study) and semi-structured (qualitative study) questionnaires during a face-to-face interview. This article is about the quantitative part which was designed as a correlational study. Single factor one-way analysis of variance (ANOVA), Pearson correlation coefficient test, T-test, and Fisher’s exact test were used to analyze the data.

About half of all students used three or more commercial resources dealing with the same content. A weak negative correlation was observed between the number of commercial resources and the exam scores, especially when the number of these resources was three or more ( r  = -0.26). The mean exam score of textbook users was statistically significantly higher than the mean score of textbook non-users ( p  = 0.01). The usage of textbooks was statistically significantly higher in the cohort of top performers in comparison to the rest of the students ( p  = 0.006). In addition to less usage of textbooks, the mean number of review books was higher in the group of weakest students (2.84 versus 3.7; p  = 0.75).

Conclusions

Most students did not use professional textbooks and about half used too many commercial review resources. While the former fact was significantly associated with poor academic performance, the later fact had weak negative correlation with exam score.

Pedagogical interventions are urgently needed to make the right type of learning resources available by making professional textbooks more USMLE-oriented and helping the students choose the best and right number of resources for optimum academic performance. By fulfilling the observed needs of the students in this way, they might feel empowered because of self-determination which will motivate studies.

Introduction

The United States Medical Licensing Examination (USMLE) is written by the United States as well as international medical graduates at the end of the two-year basic science stage of medical school to enroll in three to seven more years of medical training (residency) in different branches of medical science in the USA. In 2022, that number stood at 53,881, which is much higher than the residency slots available which is 39,205 [ 1 ]. The successful residency match rate in 2022 for US medical graduates, US International Medical Graduates (US IMG), and non-US IMGs was 92.9%, 61.4%, and 58.1% respectively [ 2 ]. However, there is a silver lining in the cloud: according to a report from the Association of American Medical Colleges in 2017, there will be a shortage of up to 104,900 physicians in the USA by 2030 [ 3 ].

To maintain the competitive edge, a student needs to score higher than the others in Step 2 Clinical Knowledge (CK) even though USMLE Step 1 score reporting has been changed to a simple pass-fail from the previous three-digit numeric scoring on January 26, 2022 [ 4 ]. Passing step 1 in the first attempt is also not an easy job which is necessary to make oneself competitive in the residency match.

Numerous commercial organizations and individuals have developed a multitude of resources that are supposed to help students with their studies and preparation for the USMLE examinations. These resources include review books, question banks, and audio–video resources based on the fact that more than 70% of students are multimodal learners as described by the VARK model: Visual, Auditory, Reading and Writing, and Kinesthetic. These resources give students the flexibility to choose one according to their needs [ 5 , 6 , 7 ]. However, an important caveat is these resources are written concisely mainly to help students revise the already learned USMLE content in a short period (e.g., a few weeks) before USMLE step 1 and as supplementary resources to traditional textbooks during the basic science years [ 8 ]. Nevertheless, many medical students are using these resources as the primary learning platforms from an early period of the first year itself or in the second year well before the dedicated USMLE study period. Such a phenomenon is called “Step 1 climate” [ 9 ].

Several factors contribute to this phenomenon as explained hence onward. The influence of the USMLE test preparation industry and social media often promote a humanistic approach to learning, emphasizing student choice and control over resources. This resonates with students who desire the freedom to select materials they find suitable. Many concise review resources prioritize testable, high-yield information, presented in formats like bullet points and buzzwords. While seemingly efficient, this approach can have drawbacks. By focusing solely on “high-yield” material, students might miss out on broader foundational knowledge and the context required to truly understand tested concepts. These resources may not cover all testable topics, and low-yield information, although not explicitly tested, can be crucial for comprehending the bigger picture. While these concise resources might seem convenient, they may not equip students with the comprehensive understanding needed to excel in their studies and future careers. The result is a potential gap in students’ knowledge base due to lack of in-depth study [ 9 , 10 , 11 ]. Unlike review resources designed for rapid revision and recall, textbooks support in-depth exploration of a topic by delving into the historical context and theoretical foundations of each topic. They provide multifaceted explanations and diverse perspectives. Through this comprehensive approach, textbooks empower students to transcend rote memorization and cultivate a deep, nuanced understanding of the subject matter [ 12 ].

Jeyaraju et al. analyzed 201 studies on USMLE exams and found that the most important factor for success was a good foundational base of knowledge supported by regular and continuous habits of learning over time from comprehensive resources. Practice tests were also found to be helpful as the number of practice test items and practice exam scores positively correlated with higher USMLE scores [ 12 ]. Other researchers have also found that the foundation of knowledge developed through an accredited curriculum during the basic science years is the most important factor for success in USMLE steps [ 8 , 13 , 14 ]. Nevertheless, it does not mean commercial review resources have no place in medical education. Some researchers have demonstrated that the usage of such resources along with traditional learning material helps students score higher marks in USMLE exams [ 15 , 16 ]. Using question banks, which is called retrieval practice, has been shown to improve long-term retention and recall of previously learned information which can be explained by Cognitivist learning theory [ 14 , 17 , 18 ].

The National Board of Medical Examiners (NBME) and the Federation of State Medical Boards (FSMB), the organizations that provide oversight of the USMLE examinations, have changed the scoring system of Step 1 to a pass-and-fail from a 3-digit-scoring system to encourage students to pay more attention to the accredited school curriculum and discourage too much focus on Step 1 score and unwise usage of commercial test resources [ 19 ]. However, due to the following reasons, this might not be enough to bring students to the classroom environment from step 1 climate. The fact that principles and facts of basic sciences are the core basis of clinical sciences learning and thus success with the USMLE step 2, knowledge acquired during the first two years at medical school is vital to score better in the step 2 exam. This reason and the increment in the passing score of Step 1 from 194 to 196 and a similar change in Step 2 CK score from 209 to 214 will ensure the ever-increasing proliferation of commercial test preparation resources tailor-made for USMLE step 1 [ 20 ]. Program directors from now on will shift their focus to Step 2 score for the screening of potential candidates for interviews to screen thousands of candidates for a handful of residency positions [ 21 , 22 ]. Before the change in scoring system, program directors used step 1 score as a first-line screening tool that could compare applicants objectively [ 23 , 24 ].

Therefore, it would be better to maintain the status quo on the numeric scoring system of USMLE and find other ways to strike a balance between traditional teaching–learning methods and the usage of commercial test resources by students.

Statement of problem/rationale of the study

As educators guiding students through the foundational basic sciences in our medical school, we have witnessed firsthand the pervasive influence of the “step 1 climate.”

Students have become increasingly drawn to the perceived efficiency of these concise review resources. They often express frustration with the time commitment required to grasp complex concepts from textbooks. This preference for quick solutions has manifested in several concerning ways: reduced class attendance, textbook neglect, and utilization of too many concise resources.

Our research aims to dissect this problem from multiple angles. We want to determine if there’s a tipping point where relying on too many concise resources becomes counterproductive for learning and by analyzing how high-performing and low-performing students utilize learning resources, we want to identify potential strategies that contribute to academic success. Armed with objective research findings, we will be able to engage in data-driven discussions with students, faculty, and the USMLE preparation industry to create a learning environment that fosters a strong foundation in basic sciences while adequately preparing students for the USMLE.

While the current literature acknowledges student confusion regarding resource selection, a comprehensive investigation into the potential downsides of relying heavily on commercial review resources and neglecting textbooks is lacking.

This research has the potential to significantly impact medical education by providing much-needed clarity on the role of commercial review resources, textbooks, and Question Banks, and their impact on student learning outcomes. Additionally, by comparing resource utilization patterns between student groups, we can glean valuable insights into effective learning strategies.

The purpose of the study was

Explore the types of learning resources medical students are using in the basic science stage of medical school and examine their effect on academic performance.

The specific objectives of the research were

Explore what learning resources students use.

Examine whether the types of resources utilized influence exam scores.

Compare learning resources and exam scores of high-performing students and low-performing students.

Methodology

Research design and variables.

Research design: It was a mixed quantitative and qualitative study. For the quantitative part, a correlational study design was used to find out the association between independent and dependent variables (Table  1 ). This research paper is about the quantitative part. The qualitative part of the study will be submitted to a journal for publication in the second stage.

Study site and ethical approval

The study was conducted at Xavier University School of Medicine, Aruba after ethical approval was received from the Institutional Review Board (IRB) of Xavier University School of Medicine (Study ID number: XUSOM/IRB/2023/03/001). The University offers a four-year undergraduate medical program, Doctor of Medicine (MD). Students complete the first two years, the basic science stage, in Aruba. At the time of this research, students were studying in the first semester, the third semester, and the fifth semester.

Study participants

Twenty-nine ( n  = 29) out of 30 Doctor of Medicine students in the 3rd semester and fifteen students ( n  = 15) out of 19 in the 5th semester were included in the study.

The sample size was calculated with the below-mentioned formula with following parameters: confidence level (1-alpha) 95%, margin of error 5%, and population size 49.

The participation was voluntary. Participants were selected randomly by a simple random sampling technique using a table of random numbers. Written consent was taken before data collection. Participants were anonymized by giving each of them a unique code as described below: the first participant of the third semester was given the code “M3.1” and the first participant of the fifth semester was given the code “M5.1”. Subsequent participants were coded serially.

Top-performing students were identified as those who scored 65% or more in the most recent NBME exam. Those students who scored less than 35% were identified as the weakest students. The score thresholds were selected for the purpose arbitrarily.

Study resources

In this research, three types of resources have been studied: professional USMLE-oriented textbooks, commercial review resources, and question banks. Learning resources have been categorized as mentioned below based on a pilot study, relevant literature, and personal experience (Table  2 ).

Exam scores

NBME nervous system exam scores were taken as the reference for MD 3 semester students and NBME cardiovascular system exam scores were taken as the reference for MD 5 semester students in order to compare with the usage of learning resources.

Data collection

Data collection was undertaken using a questionnaire survey completed during a face-to-face interview by the lead author adhering to the interview protocol. Questionnaire items included questions regarding demographic variables, types of learning resources students were using, study behavior, and perceptions about the different types of learning resources, lectures, and exams.

A pilot study was conducted among 6 students to validate the questionnaire to check the internal consistency and reliability of survey questions. The pilot study was also used to assess question clarity, answer options, and the overall flow of the survey. These students were not included in the main study. Some questions were deleted, some were modified, and some were added to the questionnaire after gaining new insights from the data of the pilot study. The questionnaire also underwent expert review to enhance its content validity and ensure it appropriately captures the target construct.

Responses to the qualitative part of the study were audio-recorded.

Survey questions

There are in total 23 items in the questionnaire. Two items are open-ended questions intended to gain insight into students’ perceptions about learning resources and study behavior (qualitative study). The rest 21 items are devised to collect quantitative data. Thirteen of these 21 items are related to learning resources which is the main subject matter of the current manuscript. The questionnaire has been uploaded as a supplementary file.

Data analysis

Quantitative data were analyzed by descriptive methods: frequencies, percentages, mean, and significance tests where applicable.

Single factor one-way analysis of variance (ANOVA) was used to determine whether there was a statistically significant difference between mean exam scores among different age groups. The Pearson correlation coefficient test was completed to analyze the correlation between the number of review books used and the exam score.

An independent samples T-test was used to compare the mean score between the two groups of students with the following variables viz., review books, Q Banks, textbooks, and gender. F test was used before the T-test to find out whether the variance of the two populations was equal or not.

Fisher’s exact test was used for two pairs of nominal data when the sample size was not large enough to use the Chi-square test. An alpha level of 0.05 was used for all statistical tests of significance. The qualitative part of the study will be presented in another article.

Demographic profile of participants

A total of 44 medical students of the basic science stage were enrolled in the study. The largest number of participants were in the age group of 25 to 34 years. Age-related mean, median, and mode values are 27.18, 26.5, and 26 respectively. Table 3 contains the summary of demographic information of participants.

Table 4 consists of information about age-groups and mean score for each age-group, mean exam score for males and female students and score categories. P value was calculated for different categories wherever applicable.

Figure 1  shows the major types of learning resources and percentage of students using them.

Clustered column chart- proportion of students using learning resources

In Table  5 , the Pearson correlation coefficient was calculated for categories under ‘commercial review resources’. The mean exam score was calculated for three groups of students who differed by the number of review books they were using. A similar statistical analysis was done with question banks. The findings are shown in the table.

Usage of commercial review resources (third-party resources): Table  5

All students used commercial review resources. Some students used a greater number of these resources than others. The use of many review resources was found to be associated with decreased exam scores as tested by the Pearson correlation coefficient ( r  = -0.26). The mean score of students who used one or two review books was higher than the mean score of students who used more than three review books although the difference was not statistically significant when tested by ANOVA-single factor test.

The most commonly used review book was First Aid (88.6%) followed by Boards and Beyond (56.8%) and Ninja Nerd (27.3%). Other less frequently used review resources were Med School Bootcamp, BRS books, Sketchy, Anky cards, Pathoma, Dirty Medicine, and Kaplan books.

Figure 2 shows that as the number of review books increase, exam score of students decreases.

Scatter plot- correlation between the number of review books (x-axis) and exam score (y-axis)

Usage of questions banks: Table  5

Approximately ninety percent of 90% of students used question banks. The number of different Q Banks ranged from one to four. The mean number of Q Banks among users was 1.7. The most used Q Bank was UWorld (45.4%), the second was USMLE Rx (31.8%) and the third was Amboss (25%). The mean score of students who used three or four Q Banks (60.4) was higher than the mean score of students who used two or fewer (52.1) although the difference was not statistically significant when tested by ANOVA-single factor test ( P value: 0.48). There was a weak negative correlation between the usage of two or fewer Q Banks and the exam score ( r  = -0.19). However, there was a weak positive correlation between the usage of three or more Q Banks and the exam score which was not statistically significant.

Usage of textbooks: Table  5

Only six students (13.6%) said they used three or more textbooks daily as the main learning resource, while the rest did not use textbooks as the main learning resource. Independent samples t-test assuming unequal variance was used to analyze the statistical significance between the mean scores of these two populations. The mean exam score of textbook users was found to be statistically significantly higher than the mean score of textbook non-users: 72.6 versus 51.1 ( P value = 0.01).

Usage of lecture notes

All students used lecture notes for internal examinations.

Top-performing students versus the rest of the students (Table  6 )

There were 13 students in the cohort of top performers and 31 in the rest of the cohort.

The usage of textbooks was statistically significantly higher in the cohort of top performers in comparison to the rest of the students ( P value: 0.006): Table  6 .

Usage of first aid review book: Table  6

Twelve out of thirteen top-performing students used first aid while twenty-six out of thirty-one in the other cohort used this review book. The difference was not statistically significant as calculated by Fisher’s exact test ( P value = 0.6).

Usage of Q Bank: Table  6

Twelve out of thirteen top-performing students and twenty-eight out of thirty-one in the other cohort used at least one Question Bank book. The difference was not statistically significant as calculated by Fisher’s exact test. The difference in mean number of Q Bank books between the two groups was also not statistically significant ( P value = 0.28).

There was no statistically significant difference between the means of the two groups regarding study hours and the number of review books (P values 0.71 and 0.75).

Varied learning resources: what do students choose?

With growing mobile device usage and greater internet connectivity, there has been a radical change in the type and nature of learning resources used by medical students. In the past, students had to rely solely on didactic lectures and print books for learning purposes, however, this is simply not the case anymore. Students have a wide array of choices ranging from professional textbooks in digital format and paperback to concise commercial resources in paperback, audio, and video format, and question bank books. The practice of blended learning where students use both traditional and new digital learning tools has been well-established universally [ 25 , 26 , 27 , 28 ]. How has this change affected academic performance concerning the type and number of resources utilized? This is the question that has prompted this research.

The impact of commercial review resources usage on academic performance

It has been found that most students choose concise commercial resources over comprehensive professional textbooks as the source to acquire knowledge. In the current study, only 13.6% of students used professional textbooks along with commercial resources which reflected similar findings from a study (15%) in a German medical school [ 29 ]. Makus D et al. from the University of Ottawa reported students spend 63% of their time on commercial resources [ 30 ]. Snow CE et al. and Scott K et al. found that students are less motivated to use textbooks when commercial review resources are available and are perceived as more effective [ 27 , 31 ]. These findings are very important from an academic viewpoint because the mean score of the group of students who used professional textbooks along with other commercial review resources was statistically significantly higher than the mean score of textbook nonusers in this research.

Students are more likely to choose too many commercial review resources of the same scope due to several reasons: availability of numerous brands of commercial resources, digital advertisements, and exchange of information on social media and peer effect [ 6 ]. In this study, 54.5% of students used three or more commercial resources dealing with the same content. This trend might be causing harm to the students because a negative correlation, although weak, was observed between the number of commercial resources and the exam scores, especially when the number of these resources was three or more. The mean exam score for this group of students who used three or more of these resources was found to be lower than the mean exam score for students who used only one or two of these resources. However, the difference in mean scores between the two groups was not statistically significant. This suggests that the usage of a greater number of commercial review resources may not offer any advantage in academic success. Ikonne U et al. found a statistically significant positive correlation between usage of review resources and academic performance in the case of both first- and second-year medical students. However, the correlation was not studied in relation to the number of resources [ 16 ]. Bauzon, J et al. found that higher exam scores were moderately associated with less utilization of commercial review resources [ 32 ].

The impact of question bank usage on academic performance

Medical schools and licensing board examinations including USMLE in developed nations and some developing countries utilize multiple choice questions in the form of clinical vignettes in all types of theoretical examinations. Students use many types of question bank resources to prepare for these types of exams. Using question banks has been shown to improve long-term retention and recall of previously learned information [ 15 ]. However, we found no statistically significant difference in the mean exam score among several cohorts concerning the use of question banks: non-users, students using one Q bank, two Q banks, and three or more Q banks. We cannot explain the reason for this dissimilar finding in our study.

Because 61.3% of students were using more than two or more Q banks, it was not possible to study the effect of a particular type on the exam score. Multiple regression analysis completed at the University of Alabama School of Medicine found that exposure of students to USMLE-type questions throughout the preclinical stage resulted in improved academic performance [ 32 ]. In a study that involved two groups of students writing the Emergency Medical Services fellowship exam, the group that completed the prescribed question bank obtained an overall 12% higher pass rate than the group of students who did not. However, it is not clear from the study whether the same subject experts who created the question bank were also involved in formulating questions for the fellowship exam. The most important fact is these students used question banks along with regular learning resources [ 33 ].

High-performing students versus rest of the students

Our study has a total of 44 participants who had exam scores from 26 to 86. We arbitrarily separated the top 13 students who had exam scores more than 65 and analyzed different variables head-to-head with the rest of the students who had scores less than 65. The difference in means of exam scores of these two groups is statistically significant and the only factor which has a significant effect on the exam scores is the usage of textbooks. Approximately 38% of students in the top-performing cohort used textbooks in comparison to just 3.2% in the cohort of the rest of the students. This same variable was also examined among students within the top-performing cohort. Textbook users had higher mean exam scores, although the difference is not statistically significant. We did not observe differences concerning other variables (usage of Q Banks, review books, and study hours) between the top-performing cohort and the rest of the students.

High-performing students versus low-performing students

Similarly, the top-performing cohort was compared with the cohort of bottom-level students who scored less than 35 on the exam. The observation regarding textbook usage was the same as described above. In addition, the mean number of review books was higher in the bottom-level students than in the top-performing cohort although the difference was not statistically significant.

Usage of textbooks and their impact on academic performance

In a study done at the University of Michigan, Rafel JB et al. found that 77% of students who used commercial step 1 resources including concise review books and question banks early during the basic science stage along with preclinical curriculum, achieved higher step 1 scores [ 15 ]. In our study group, 13.7% of students used professional textbooks along with concise review books and question banks, and they had a significantly higher mean exam score. Parry S et al. have a slightly different finding than that of Rafel JB et al. In their study carried out in a Midwestern Medical School, the researchers found that preclinical grade was strongly associated with USMLE step 1 validating the usage of textbooks as the most important factor for success and interestingly, question bank usage but not the usage of review resources had a statistically significant positive impact on the scores [ 14 ].

According to the self-determination theory (SDT) proposed by Deci and Ryan, students feel motivated to learn when three psychological needs are fulfilled: autonomy to choose learning resources (feeling of self-governed), feeling competent to perform learning tasks, and the ability to engage with peers and teachers [ 34 ]. Professors at The University of Sydney School of Medicine opine that pedagogy should be aligned with self-determination theory and teachers should help students choose commercial resources of high quality to make learning more effective [ 25 ].

Embracing ‘step 1 climate change’ and ways to mitigate it

Step 1 climate is pervasive across all US and Canadian medical schools, and many parts of the world. Ikonne U et al. from Eastern Virginia Medical School found that students started using review resources from the first year. The authors recommend designing learning sessions and curricula to accommodate review resources to make students effective self-regulated learners [ 16 ]. Makus D et al. from the University of Ottawa suggest restructuring curriculum design and delivery in order to make room for nontraditional learning resources in the official curriculum [ 30 ]. Rather than denying and ignoring the “step 1 climate change”, it’s a good idea to embrace it and try to make it better.

In 2016, educators from the University of North Carolina School of Medicine undertook an audit of the basic science stage to evaluate content discrepancies between their curriculum and “high-yield” Step 1 topics. A highly popular review book among students “First Aid for USMLE Step 1 “and USMLE Step 1 score reports of students were used for this purpose. They also made customized exams administered by NBME, the same organization that oversees USMLE, mandatory to students. Many students-centered pedagogical interventions were also made: training students on test-taking skills, providing a subscription to USMLE-oriented resources namely USMLE-Rx for content and UWorld for multiple choice questions, and one-to-one counseling focused on the needs of a particular student. Because of the restructuring of the curriculum in line with USMLE step 1, the school achieved a resounding success with a first-time pass rate of 99.4% in 2018 which was 95% in 2016, and a numeric score of three points more than the national average [ 35 ].

The uniqueness of this study

The current study delves into the usage of learning resources by medical students in the basic science stage and its impact on their academic performance from multiple angles. While many studies have been done on this topic, the authors of this survey did not find any article in the literature regarding the comparison of high-performing students and low-performing students. Observations have been documented in the results section and described in the discussion section. These observations can help teachers and students make adjustments to enhance teaching–learning activities.

Apart from that, few studies have analyzed data regarding the number of commercial review resources and their effect on academic performance. We have divided students into four groups based on the number of review resources they are using, the data has been analyzed in detail.

Limitations of this study

We might get a clearer picture if research is done in many medical schools across all continents so that the findings can be generalized for all medical students in the basic science stage.

Mean exam score was found higher in the cohort of high-performing students than in the cohort of low-performing students because of the usage of textbooks in the former. However, to eliminate the confounding bias due to dissimilar levels of ‘intelligence’ among participants, this conclusion should be validated by a case–control study with a large sample size taking care to match the entry-time ‘Grade Point Average (GPA)’ of the participants in the two cohorts.

In this study, medical students in the basic science stage have provided themselves with the autonomy of choosing learning resources in line with Humanistic learning theory to fulfill unique needs demanded by the USMLE step 1 examination. However, it has been observed that the majority of students did not use professional medical textbooks and about half of the students used three or more commercial review resources. Both facts were significantly associated with poor academic performance.

It is therefore vitally necessary that medical schools, educators, and authors take the following pedagogical interventions to make the right type of learning resources available and help the students choose the best resources for optimum academic performance in the basic science stage of the MD program. In this way, students might feel empowered because of self-determination and stay on course because of the behaviorist paradigm from the subject experts.

Medical school curriculum should be restructured to incorporate balanced usage of commercial review books and NBME-styled multiple-choice questions apart from professional textbooks to fill the void of observed needs and felt needs of medical students.

Students should be discouraged from using too many parallel commercial review resources.

To draw students away from commercial review resources, authors should write textbooks in such a way that they are structured, aligned with the content of USMLE step 1, and concise without losing the depth and breadth of knowledge.

Availability of data and materials

All the data underlying the results are available which can be obtained through the corresponding author.

Early release of USMLE Step 1 2022 summary performance. Available from: https://www.usmle.org/early-release-usmle-step-1-2022-summary-performance . Cited 2024 Jan 8.

NRMP releases the 2022 main residency match results and data publication, the most comprehensive data resource for the Main Residency Match® 2022. Available from: https://www.nrmp.org/about/news/2022/06/nrmp-releases-the-2022-main-residency-match-results-and-data-publication-the-most-comprehensive-data-resource-for-the-main-residency-match/ . Cited 2024 Jan 8.

McKenzie L. Do medical schools still need books? Available from: https://www.insidehighered.com/news/2017/10/03/do-medical-schools-still-need-books . Cited 2024 Jan 8.

USMLE Step 1 transition to pass/fail only score reporting. Available from: https://www.usmle.org/usmle-step-1-transition-passfail-only-score-reporting . Cited 2024 Jan 8.

Samarakoon L, Fernando T, Rodrigo C, Rajapakse S. Learning styles and approaches to learning among medical undergraduates and postgraduates. BMC Med Educ. 2013;13(1). https://doi.org/10.1186/1472-6920-13-42 .

Finn E, Ayres F, Goldberg S, Hortsch M. Brave new e-world: medical students’ preferences for and usage of electronic learning resources during two different phases of their education. FASEB BioAdv. 2022;4(5):298–308. https://doi.org/10.1096/fba.2021-00124 .

Article   Google Scholar  

Hernandez JE, Vasan N, Huff S, Melovitz-Vasan C. Learning styles/preferences among medical students: Kinesthetic learner’s multimodal approach to learning anatomy. Med Sci Educ. 2020;30(4):1633–8. https://doi.org/10.1007/s40670-020-01049-1 .

Guilbault RWR, Lee SW, Lian B, Choi J. Predictors of USMLE step 1 outcomes: charting successful study habits. Med Sci Educ. 2020;30(1):103–6. https://doi.org/10.1007/s40670-019-00907-x .

Lee T, Bhushan V, Sochat M. First aid for the USMLE step 1 2021. 31st ed. New York: McGraw-Hill Education; 2021.

Google Scholar  

Gandhi MH, Mukherji P. Learning theories - statpearls - NCBI bookshelf. StatPearls Publishing; 2023. Available from: https://www.ncbi.nlm.nih.gov/books/NBK562189/ . Cited 2024 Jan 8.

Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student perspectives on the “step 1 climate” in preclinical medical education. Acad Med. 2019;94(3):302–4. https://doi.org/10.1097/acm.0000000000002565 .

Jeyaraju M, Linford H, Bosco Mendes T, Caufield-Noll C, Tackett S. Factors leading to successful performance on U.S. National Licensure Exams for medical students: a scoping review. Acad Med. 2022;98(1):136–48. https://doi.org/10.1097/acm.0000000000004877 .

Wu W, Garcia K, Chandrahas S, Siddiqui A, Baronia R, Ibrahim Y. Predictors of performance on USMLE Step 1. Southwest Respir Crit Care Chron. 2021;9(39):63–72. https://doi.org/10.12746/swrccc.v9i39.813 .

Parry S, Pachunka J, Beck Dallaghan GL. Factors predictive of performance on USMLE Step 1: do commercial study AIDS improve scores? Med Sci Educ. 2019;29(3):667–72. https://doi.org/10.1007/s40670-019-00722-4 .

Burk-Rafel J, Santen SA, Purkiss J. Study behaviors and USMLE Step 1 performance: implications of a student self-directed parallel curriculum. Acad Med. 2017;92(11S). https://doi.org/10.1097/acm.0000000000001916 .

Ikonne U, Brodie A, Bay C, Campbell A. Frequency of student resource use and academic performance in preclerkship education: a survey study. Med Sci Educ. 2022;32(6):1465–79. https://doi.org/10.1007/s40670-022-01674-y .

Little JL, Bjork EL, Bjork RA, Angello G. Multiple-choice tests exonerated, at least of some charges: fostering test-induced learning and avoiding test-induced forgetting. Psychol Sci. 2012;23(11):1337–44. https://doi.org/10.1177/0956797612443370 .

Liu Q, Wald N, Daskon C, Harland T. Multiple-choice questions (mcqs) for higher-order cognition: perspectives of university teachers. Innov Educ Teach Int. 2023:1–13. https://doi.org/10.1080/14703297.2023.2222715 .

Moon K. USMLE Step 1 is now pass/fail - who benefits from this big change?. Forbes; 2020. Available from: https://www.forbes.com/sites/kristenmoon/2020/04/07/usmle-step-1-is-now-passfailwho-benefits-from-this-big-change/ . Cited 2024 Jan 8.

Change to step 2 CK passing standard begins July 1, 2022. 2022. Available from: https://www.ecfmg.org/news/2022/04/12/change-to-step-2-ck-passing-standard-begins-july-1-2022/ . Cited 2024 Jan 8.

Murphy B. USMLE Step 1 score switch: answers to medical students’ top questions. 2022. Available from: https://www.ama-assn.org/medical-students/usmle-step-1-2/usmle-step-1-score-switch-answers-medical-students-top-questions . Cited 2024 Jan 8.

Makhoul AT, Pontell ME, Ganesh Kumar N, Drolet BC. Objective measures needed — program directors’ perspectives on a pass/fail USMLE Step 1. N Engl J Med. 2020;382(25):2389–92. https://doi.org/10.1056/nejmp2006148 .

Palmerton A. Is step 2 CK the new step 1? Objective facts and surveys. 2021. Available from: https://www.yousmle.com/step-2-ck-new-step-1/ . Cited 2024 Jan 8.

Jayakumar KL. Numerical USMLE step 1 scores are still important in selection of residency applicants. Acad Med. 2016;91(11):1470–1. https://doi.org/10.1097/acm.0000000000001402 .

Wynter L, Burgess A, Kalman E, Heron JE, Bleasel J. Medical students: what educational resources are they using? BMC Med Educ. 2019;19(1). https://doi.org/10.1186/s12909-019-1462-9 .

Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in medical education. Acad Med. 2006;81(3):207–12. https://doi.org/10.1097/00001888-200603000-00002 .

Scott K, Morris A, Marais B. Medical student use of digital learning resources. Clin Teac. 2017;15(1):29–33. https://doi.org/10.1111/tct.12630 .

Egarter S, Mutschler A, Tekian A, Norcini J, Brass K. Medical assessment in the age of digitalisation. BMC Med Educ. 2020;20(1). https://doi.org/10.1186/s12909-020-02014-7 .

Gutmann J, Kühbeck F, Berberat PO, Fischer MR, Engelhardt S, Sarikas A. Use of learning media by undergraduate medical students in pharmacology: a prospective cohort study. PLoS One. 2015;10(4). https://doi.org/10.1371/journal.pone.0122624 .

Makus D, Kashyap A, Labib M, Humphrey-Murto S. A curriculum ignored? the usage of unofficial commercial and peer learning resources in undergraduate medical education at a Canadian Medical School. Med Sci Educ. 2023;33(6):1379–88. https://doi.org/10.1007/s40670-023-01899-5 .

Snow CE, Torous J, Gordon-Elliott JS, Penzner JB, Meyer F, Boland R. Use of electronic resources for psychiatry clerkship learning: a medical student survey. Acad Psychiatry. 2016;41(5):656–60. https://doi.org/10.1007/s40596-016-0647-3 .

Bauzon J, Alver A, Ravikumar V, Devera A, Mikhael T, Nauman R, et al. The impact of educational resources and perceived preparedness on medical education performance. Med Sci Educ. 2021;31(4):1319–26. https://doi.org/10.1007/s40670-021-01306-x .

Clemency B, Martin-Gill C, Rall N, May P, Lubin J, Cooley C, et al. Association between EMS Question Bank completion and passing rates on the EMS certification examination. Prehosp Emerg Care. 2017;21(4):498–502. https://doi.org/10.1080/10903127.2017.1294225 .

Deci EL, Ryan RM. Conceptualizations of intrinsic motivation and self-determination. Intrinsic motivation and self-determination in human behavior. Psychol. 1985;11–40. https://doi.org/10.1007/978-1-4899-2271-7_2 .

Beck Dallaghan GL, Byerley JS, Howard N, Bennett WC, Gilliland KO. Medical school resourcing of USMLE Step 1 preparation: questioning the validity of step 1. Med Sci Educ. 2019;29(4):1141–5. https://doi.org/10.1007/s40670-019-00822-1 .

Download references

Acknowledgements

Xavier University School of Medicine management for funding the article processing charge.

Publication of this study was funded by Xavier University School of Medicine, Aruba as a part of the research grant to promote research.

Author information

Authors and affiliations.

Department of Pathology, Xavier University School of Medicine, Oranjestad, Aruba

Sabin Kumar Ranabhat

Department of Anatomy, Xavier University School of Medicine, Oranjestad, Aruba

Mohan Lal Kunjukrishnan

Xavier University School of Medicine, Oranjestad, Aruba

Muskan Dubey

Department of Medical Education, Memorial University of Newfoundland, Newfoundland, Canada

Vernon Curran

Department of Pharmacology, Xavier University School of Medicine, Oranjestad, Aruba

Arun Kumar Dubey

Department of Medicine, OSCE and SP Program, Xavier University School of Medicine, Oranjestad, Aruba

Neelam Dwivedi

You can also search for this author in PubMed   Google Scholar

Contributions

SKR collected data, designed study, analyzed data and prepared the manuscript. MLK and MD reviewed literature, contributed to manuscript drafting and revision. VC, AKD and ND contributed to study design, manuscript preparation and revision. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Sabin Kumar Ranabhat .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval to conduct this research was obtained from the Institutional Review Board (IRB) of Xavier University School of Medicine on March 4, 2023 (Study ID number: XUSOM/IRB/2023/03/001), in compliance with the “World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects (WMA 2008) and the Council for International Organizations of Medical Sciences’ International Guidelines for Ethical Review of Epidemiological Studies (CIOMS 1991). Informed written consent was taken from each participant.

Consent for publication

Not applicable.

Competing of interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Ranabhat, S.K., Kunjukrishnan, M.L., Dubey, M. et al. Exploring the usage of learning resources by medical students in the basic science stage and their effect on academic performance. BMC Med Educ 24 , 543 (2024). https://doi.org/10.1186/s12909-024-05511-1

Download citation

Received : 09 January 2024

Accepted : 02 May 2024

Published : 15 May 2024

DOI : https://doi.org/10.1186/s12909-024-05511-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• USMLE Step 1
• Learning resources
• Review books
• Medical students
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 16.5.2024 in Vol 26 (2024)

Public and Research Interest in Telemedicine From 2017 to 2022: Infodemiology Study of Google Trends Data and Bibliometric Analysis of Scientific Literature

Authors of this article:

Original Paper

• Andrea Maugeri 1 , PhD   ; 
• Martina Barchitta 1 , PhD   ; 
• Guido Basile 2 , MD   ; 
• Antonella Agodi 1 , PhD  

1 Department of Medical and Surgical Sciences and Advanced Technologies “GF Ingrassia”, University of Catania, Catania, Italy

2 Department of General Surgery and Medical-Surgical Specialties, University of Catania, Catania, Italy

Corresponding Author:

Antonella Agodi, PhD

Department of Medical and Surgical Sciences and Advanced Technologies “GF Ingrassia”

University of Catania

Via Santa Sofia 87

Catania, 95123

Phone: 39 0953792183

Email: [email protected]

Background: Telemedicine offers a multitude of potential advantages, such as enhanced health care accessibility, cost reduction, and improved patient outcomes. The significance of telemedicine has been underscored by the COVID-19 pandemic, as it plays a crucial role in maintaining uninterrupted care while minimizing the risk of viral exposure. However, the adoption and implementation of telemedicine have been relatively sluggish in certain areas. Assessing the level of interest in telemedicine can provide valuable insights into areas that require enhancement.

Objective: The aim of this study is to provide a comprehensive analysis of the level of public and research interest in telemedicine from 2017 to 2022 and also consider any potential impact of the COVID-19 pandemic.

Methods: Google Trends data were retrieved using the search topics “telemedicine” or “e-health” to assess public interest, geographic distribution, and trends through a joinpoint regression analysis. Bibliographic data from Scopus were used to chart publications referencing the terms “telemedicine” or “eHealth” (in the title, abstract, and keywords) in terms of scientific production, key countries, and prominent keywords, as well as collaboration and co-occurrence networks.

Results: Worldwide, telemedicine generated higher mean public interest (relative search volume=26.3%) compared to eHealth (relative search volume=17.6%). Interest in telemedicine remained stable until January 2020, experienced a sudden surge (monthly percent change=95.7%) peaking in April 2020, followed by a decline (monthly percent change=–22.7%) until August 2020, and then returned to stability. A similar trend was noted in the public interest regarding eHealth. Chile, Australia, Canada, and the United States had the greatest public interest in telemedicine. In these countries, moderate to strong correlations were evident between Google Trends and COVID-19 data (ie, new cases, new deaths, and hospitalized patients). Examining 19,539 original medical articles in the Scopus database unveiled a substantial rise in telemedicine-related publications, showing a total increase of 201.5% from 2017 to 2022 and an average annual growth rate of 24.7%. The most significant surge occurred between 2019 and 2020. Notably, the majority of the publications originated from a single country, with 20.8% involving international coauthorships. As the most productive country, the United States led a cluster that included Canada and Australia as well. European, Asian, and Latin American countries made up the remaining 3 clusters. The co-occurrence network categorized prevalent keywords into 2 clusters, the first cluster primarily focused on applying eHealth, mobile health (mHealth), or digital health to noncommunicable or chronic diseases; the second cluster was centered around the application of telemedicine and telehealth within the context of the COVID-19 pandemic.

Conclusions: Our analysis of search and bibliographic data over time and across regions allows us to gauge the interest in this topic, offer evidence regarding potential applications, and pinpoint areas for additional research and awareness-raising initiatives.

Introduction

The concept of telemedicine—defined as the use of technology to provide health care services remotely—has been around since the early 20th century, but it has gained significant attention globally only in recent years [ 1 ]. The development of technology such as videoconferencing, remote monitoring devices, and mobile health apps has made it possible to deliver health care services remotely, paving the way for the widespread adoption of telemedicine [ 2 - 4 ]. It is worth noting that the term telemedicine also covers health data analysis and the application of big data and artificial intelligence methods for epidemiological research and diagnosis support [ 3 , 5 - 7 ].

The use of technology allows patients to access care from anywhere, at any time and reduces the need for in-person visits, which can be particularly beneficial for individuals with mobility issues, those living in rural or remote areas, and individuals with chronic diseases [ 8 , 9 ]. The potential benefits of telemedicine are numerous, including increased access to health care services, reduced health care costs, and improved patient outcomes [ 8 - 11 ]. Telemedicine can also help to address workforce shortages in health care, particularly in rural and remote areas, by enabling health care providers to deliver care to patients in those regions without the need for travel [ 10 , 12 ].

The COVID-19 pandemic has underscored the significance of telemedicine in maintaining uninterrupted health care delivery while mitigating the risk of virus transmission [ 13 - 18 ]. As an illustration, in light of the imperative to reduce COVID-19 exposure among patients and health care providers, many elective surgical procedures were rescheduled, prompting surgeons to adopt telemedicine as an alternative for preoperative, follow-up, and urgent surgical care consultations [ 15 , 19 ]. The pandemic has therefore forced health care providers to adapt quickly to the new reality of delivering care remotely, leading to a significant increase in the adoption and use of telemedicine globally [ 3 , 20 ]. As demand grew exponentially during the pandemic period, even the telehealth market is expected to grow to US $218.5 billion by 2025 [ 3 ]. Nevertheless, even after the COVID-19 pandemic recedes, it is highly improbable that this mode of health care delivery will be disregarded.

Despite the potential benefits of telemedicine, its adoption and implementation have been slow in some regions. Barriers to adoption include regulatory challenges, technological limitations, and resistance from health care providers and patients. People living in poorer regions, women, the elderly, and those living in rural or remote areas are far less likely to be online than those in wealthier regions [ 1 , 11 ]. Of those connected, nearly 90% use mobile devices to access the internet, which might not be appropriate for delivering digital health services [ 3 ]. A digital divide also exists in terms of digital literacy or low skills, which is a concern for the poorest, elderly, and others with limited access to technology [ 3 ]. For these reasons, understanding the level of interest in telemedicine among the public and research community can help identify areas for improvement. Google Trends is a valuable tool for investigating the level of interest of the general public in a specific topic. As such, it has been used in previous research to analyze the public’s interest in various health and health care–related subjects [ 21 - 32 ]. The same applies to a topic such as telemedicine, for which there is little evidence. From a research standpoint, we have recently observed a significant increase in the number of publications related to telemedicine, indicating a growing interest in this field among researchers. An interest that needs to be mapped in terms of scientists and groups of researchers, countries that are contributing most to the research, and areas of applications.

Overall, this study aims to provide a comprehensive analysis of the level of public and research interest in telemedicine, also consider any potential impact of the COVID-19 pandemic. We therefore limit our analysis to the period between 2017 and 2022, a period of 6 years preceding and following the pandemic. We use 2 different approaches to analyze the level of interest in telemedicine. The first approach is an analysis of Google Trends data, which allows us to analyze the level of interest in telemedicine among the general public. The second approach is a bibliometric data analysis, which involves examining publications related to telemedicine over the past 6 years. Our analysis of search and bibliographic data over time and across regions helps us understand the level of interest in this topic, identify areas for further research and awareness-raising efforts, and inform policy decisions regarding telemedicine adoption and implementation.

Data Collection

Google trends data.

Google Trends provides open access to time-series data related to Google searches for specific terms and topics [ 33 ]. According to the framework proposed by Mavragani and Ochoa [ 34 ], we retrieved Google Trends data separately by using the search topics “telemedicine” or “e-health,” encompassing all search categories. We queried Google Trends on March 9, 2023, and Google Trends data were obtained at the global level, as well as by country, for the period between January 1, 2017, and December 31, 2022. It is worth mentioning that search topics are a group of terms that share the same concept across languages, covering an array of variations, typos, and related searches [ 34 , 35 ]. This precludes the need to enter a set of individual keywords, while maintaining the consistency of search queries across all regions and timeframes [ 34 , 35 ]. Furthermore, using specific search topics without any search category restrictions proves beneficial for capturing the general interest of diverse populations [ 34 , 35 ]. We also obtained data for the “top related topics” that are most frequently searched with the topics under investigation (ie, telemedicine” or “e-health”).

In general, Google Trends data are provided as a normalized measure (ie, relative search volume [RSV]), obtained by dividing the search volume for a given term or topic by the total number of searches. This normalization process resulted in a percentage scale, with 100% corresponding to the peak in search volume in any given time frame and location. The value 0% does not necessarily indicate no searches, but rather a very low search volume for a given term or topic [ 34 ].

Bibliometric Data

Before describing the collection of bibliometric data, it is necessary to distinguish bibliometric analysis from reviews and systematic reviews of scientific literature. The first primarily uses a mechanistic method to track the global research trends in a certain field based on the outputs of scientific literature databases. Reviews and systematic reviews are instead characterized by methodical and replicable methodologies to find, select, and synthesize all available evidence on a particular topic or clinical question.

In our study, we searched the Scopus database for all articles mentioning the terms “telemedicine” or “eHealth” in the title, abstract, and keywords. We have chosen Scopus because it is recognized as the largest scientific literature database of peer-reviewed articles covering a wide range of subjects [ 36 ]. The literature search was conducted on 9th March 2023 and was limited to original articles published in English and in the subject area of medicine from 2017 to 2022. The query string used for the search was: TITLE-ABS-KEY (“telemedicine” OR “eHealth” ) AND PUBYEAR > 2016 AND PUBYEAR < 2023 AND ( LIMIT-TO ( DOCTYPE , “ar” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) ) AND ( LIMIT-TO ( SUBJAREA , “MEDI” ) ). The Scopus search result was exported in the format of a CSV file with all data elements, including information on citation, bibliography, abstract, and keywords.

Data Analysis

Google trends data analysis.

We first conducted a univariate analysis and compared the public interest on the topics of telemedicine and eHealth at the global level. Next, a joinpoint regression analysis was carried out to identify possible time points at which public interest trends changed. This analysis was conducted on log-transformed RSV, with 5000 permutations and assuming uncorrelated errors. The grid search method was chosen to determine where to locate joinpoints on the timescale [ 37 ]. Results were reported as the monthly percent change (MPC), calculated as the average percentage change per month between different joinpoints. Joinpoint regression analysis was performed using the Joinpoint Regression Program (version 4.3.1.0; Statistical Research and Applications Branch, National Cancer Institute) provided by the Surveillance, Epidemiology, and End Results Program (National Cancer Institute) on the website. We next mapped the public interest in telemedicine and eHealth by country, selecting the top 5 countries with the greatest interest. In these countries, Google Trends data were correlated with the number of new confirmed COVID-19 cases, deaths, hospitalizations, and patients in the intensive care unit per million residents. These data were obtained from the Our World in Data website [ 38 ]. Results were reported as the Spearman rank correlation coefficient (ρ). All statistical analyses were 2-tailed and performed with a significance level of 0.05.

Descriptive Analysis of Bibliometric Data

Descriptive analysis of bibliometric data was performed using Bibliometrix (K-Synth), an open-source R-tool for automating the stages of data analysis and data visualization [ 39 ]. After loading and converting bibliometric data in R (R Core Team), the main descriptive results were summarized as the number of documents, authors, sources, keywords, timespan, and average number of citations. Accordingly, tables and visualizations were obtained for the annual scientific production, top articles per number of citations, most productive authors, most productive countries, total citations per country, most relevant journals, and most relevant keywords.

Network Analysis of Bibliometric Data

Next, the VOSviewer software (version 1.6.16; Centre for Science and Technology Studies, Leiden University) was used to construct networks projecting authors’ and countries’ collaborations, as well as trending research topics through the analysis of keywords. This mapping method is generally used to estimate the association strength between different bibliometric items (ie, the nodes of the network), which may for example be publications, authors, or keywords [ 40 ]. The relation between 2 items is represented by a link (ie, the edges of the network), which may be a bibliographic coupling link between publications, a coauthorship link between authors, or a co-occurrence link between keywords. Each link has a strength, indicated by a positive number with higher values associated with stronger links [ 40 ]. The association strength may for example be indicated as the number of cited references 2 publications have in common for the bibliographic coupling link; as the number of publications 2 authors have coauthored for the coauthorship links; or as the number of publications in which 2 keywords occur together for the co-occurrence link [ 40 ]. Accordingly, each item receives different attributes, such as the weight and score attributes. Weight is a nonnegative numerical attribute indicating the importance of the item in the network. From a graphical perspective, items with higher weights are shown more prominently than those with lower weights [ 40 ]. Score attributes instead indicate additional numerical properties of the item, which can be only visualized in the overlay visualization of a map. There are also 2 standard weight attributes that can be used for descriptive purposes and computed for each item or the entire network, the links attribute and the total link strength attribute [ 40 ]. By creating the network map, items can be grouped into clusters, which correspond to linked items, labeled with colors and numbers.

We first performed network analyses of coauthorship at the author and country level, using the fractional method to reduce the influence of documents with many authors. With this approach, the strength of a coauthorship between 2 authors is determined considering the number of documents coauthored normalized for the total number of authors of each coauthored document. At the author level, we included authors who have published at least 20 articles on the topic, with no restrictions on the number of citations. At the country level, we included the first 50 countries with the greatest number of articles, with no restrictions on the number of citations. To identify research areas of greatest interest and their connections, we also applied a co-occurrence analysis of author keywords occurring more than 50 times.

Public Interest Over Time

Globally, the mean public interest in telemedicine and eHealth—expressed as RSV—was 26.3% (SD 18.1%) and 17.6% (SD 8.8%), respectively. The greater interest in telemedicine rather than in eHealth is evident from Figure 1 , which shows RSVs from January 2017 to December 2022. According to the above figure, public interest in both topics was stable before the COVID-19 pandemic and then increased rapidly for telemedicine and more gradually for eHealth. Based on the joinpoint regression analysis (Figure S1 in Multimedia Appendix 1 ), public interest in telemedicine was stable until January 2020 (MPC=0.36%), which corresponded to the first joinpoint (37th month of the time series; 95% CI 36-38). Then, public interest suddenly increased (MPC=95.7%) to the highest peak reached in April 2020, which was the second joinpoint detected in the 40th month of the time series (95% CI 39-41). From that point on, public interest decreased (MPC=–22.7%) until August 2020 (ie, the third joinpoint on the 44th month; 95% CI 42-48) and then returned to being stable (MPC=–0.14%). Similarly, public interest in eHealth was stable until January 2020 (MPC=0.34%), which again was the first joinpoint of the time series (37th month; 95% CI 32-43). From that point on, public interest gradually increased (MPC=5.5%) to the highest peak reached in January 2022, which corresponded to the second joinpoint detected in the 61st month of the time series (95% CI 46-63). Then, public interest rapidly decreased (MPC=–20.1%) until May 2022 (ie, the third joinpoint on the 65th month; 95% CI 58-67) and then returned to being almost stable (MPC=1.8%).

Geographic Distribution of Public Interest and Correlations With COVID-19 Data

The map in Figure 2 shows the geographic distribution of public interest in telemedicine from 2017 to 2022. The top 5 countries were Chile, Australia, Canada, the United States, and Puerto Rico (Figure S2 in Multimedia Appendix 1 ). Public interest in eHealth was less widespread (Figure S3 in Multimedia Appendix 1 ), therefore we have not considered this topic for further analyses. For those countries included in the top 5, we evaluated correlations between the RSV for telemedicine and COVID-19 data per million residents. Public interest in Chile was strongly correlated with the number of new cases (ρ=0.718; P <.001), and weakly with the number of new deaths (ρ=0.321; P =.006). In Australia, public interest moderately correlated with the number of new cases (ρ=0.537; P <.001), new deaths (ρ=0.505; P <.001), hospitalized cases (ρ=0.559; P <.001), and patients in the intensive care unit (ρ=0.483; P <.001). Public interest in Canada was weakly correlated with the number of new cases (ρ=0.265; P =.001) and deaths (ρ=0.333; P <.001). In the United States, public interest was weakly correlated with the number of new deaths (ρ=0.295; P <.001). No correlations were evident for the public interest in Puerto Rico.

Common Topics Related to Telemedicine

Despite the considerable impact of the pandemic on the public interest in telemedicine, there were few COVID-19-related topics among those that were commonly searched with telemedicine (Figure S4 in Multimedia Appendix 1 ). In particular, Google users interested in telemedicine mainly searched for general topics including health, physician, health care, medicine, and therapy. Moreover, there were some common topics related to specific telemedicine providers (eg, “Telehealth Ontario,” “Medicare,” “Santa Catarina,” “Cigna,” and “CVS Pharmacy”).

Description of Bibliometric Data

We identified 19,539 original medical articles published in English and indexed in the Scopus database from 2017 to 2022. These articles were published by 2824 different sources (eg, journals and books), with a document average age of 2.8 years. Notably, the first 2 sources (JMIR and Telemedicine and eHealth) published a number of articles that were nearly 40% of the sum of the first 10 sources (Table S1 in Multimedia Appendix 1 ). The overall average number of citations per document was 11.2, while the average number of citations per document and per year was 2.6. The top 10 articles per citation are reported in Table S2 in Multimedia Appendix 1 , their total citations ranged from 506 to 999, while their total citations per year ranged from 121 to 221. A total of 553,913 references were found in the bibliometric analysis of all the articles. The top 10 cited references are reported in Table S3 in Multimedia Appendix 1 , with total citations ranging from 70 to 161.

Research Interest Over Time

Our analysis of the bibliometric data also revealed a significant increase in the number of publications related to telemedicine over the past 6 years ( Figure 3 A). The number of publications per year increased from 1615 in 2017 to 4870 in 2022, showing an overall increase of 201.5% and an annual average growth rate of 24.7%. The greatest increase has been reported between 2019 and 2020, with an annual growth rate of 89.0%. Regarding citations, the average total citations per year decreased from 2017 to 2022 ( Figure 3 B), probably as a consequence of different document ages. By contrast, the average article citations per year were stable from 2017 to 2019, followed by a peak in 2020, and then decreased until 2022 ( Figure 3 C).

Most Productive Authors and Coauthorship Network

Among the included articles, 93,394 authors were appearing 136,956 times overall. In particular, there were 690 single-authored documents written by 642 independent authors. Accordingly, the number of documents per author was 0.2, while the number of coauthors per document was 7.0. Table S4 in Multimedia Appendix 1 shows the list of the 10 most productive authors in terms of total and fractionalized articles. Their contribution to the field ranged from 48 to 80 articles (7.2 to 10.0 fractionalized articles). In Figure 4 it is shown that 71 of the authors who published at least 20 documents (n=80) were well connected. There were 488 links between these authors—which were grouped into 7 clusters—with a total link strength of 988.

Most Contributing Countries and International Collaboration Network

The majority of publications came from a single country, while 20.8% featured international coauthorships. Table 1 shows the top 10 corresponding author’s countries per document, also considering differences between single-country and multiple-country publications. One thing to note was that the United States alone published almost the same number of articles as the other 9 (6610 vs 6653), representing 33.8% of all articles analyzed. Countries with the highest proportion of multiple-country publications were Germany (31.5%), the United Kingdom (27.9%), and the Netherlands (26.8%); those with the lowest proportion were the United States (11.4%), India (20.3%), and Italy (22.5%). Similar results were evident in terms of citations, with average citations per document ranging from 8.9 for Spain to 15.2 for India (Table S5 in Multimedia Appendix 1 ).

The coauthorship network based on the top 50 most contributing countries is illustrated in Figure 5 . Overall, there were 1051 links with a total link strength of 14,751. Accordingly, countries were divided into 4 clusters. Cluster 1 consisted exclusively of European countries (n=21) with the United Kingdom, Germany, and Italy being the most interconnected; cluster 2 consisted of Asian countries (n=18) led by India and China; cluster 3 consisted of 6 countries, with the United States, Canada, and Australia being the most interconnected; and cluster 4 consisted of 5 countries of Latin America (ie, Brazil, Argentina, Chile, Colombia, and Mexico).

Most Relevant Keywords and Co-Occurrence Network

Overall, 22,798 author’s keywords were found among the included studies. Based on the analysis of the most common keywords, COVID-19-related terms appear in the top 10 (Table S6 in Multimedia Appendix 1 ). In particular, this ranking put “COVID-19” and “pandemic,” respectively, in the 2nd and 8th place. In addition to common terms related to telemedicine (eg, telehealth, eHealth, and mobile health [mHealth]), the analysis also revealed mental health as an area of relevant interest.

The co-occurrence network based on the top 50 most common keywords is illustrated in Figure 6 . Overall, there were 1030 links with a total link strength of 21,275. Accordingly, keywords were divided into 2 clusters. Cluster 1 seemed mostly related to the application of eHealth, mHealth, or digital health to noncommunicable or chronic diseases (eg, diabetes, hypertension, depression, and anxiety) and cluster 2 was instead related to the application of telemedicine and telehealth in the context of the COVID-19 pandemic.

Principal Results and Comparison With Prior Work

Our research highlights the increasing interest in telemedicine among the general public and researchers, particularly in response to the COVID-19 pandemic. Even prior to the outbreak, telemedicine had already demonstrated significant potential across various health care sectors. It offered a practical solution for delivering remote consultations, conducting preoperative evaluations, and facilitating postoperative follow-ups, particularly for patients residing in remote or underserved areas [ 9 , 19 ]. Although telemedicine comes with numerous advantages, it also presents certain drawbacks. For example, it may prove unsuitable for emergency situations requiring immediate hands-on medical attention, necessitating traditional, in-person emergency care [ 41 ]. Additionally, specific diagnostic procedures, like imaging or laboratory tests, may demand specialized equipment unavailable in a patient’s home, potentially impacting diagnostic accuracy [ 42 ]. Furthermore, not all individuals have access to the requisite technology for telemedicine consultations, such as a reliable internet connection, a suitable device, or the technical skills for virtual appointments [ 43 ]. This discrepancy in access can result in health care disparities. Finally, telemedicine involves transmitting sensitive health information over digital networks, posing challenges in ensuring the privacy and security of patient data and carrying a risk of data breaches or unauthorized access [ 43 ].

Despite these advantages and disadvantages, the pandemic served as a catalyst for the widespread adoption and acceptance of telemedicine [ 14 , 15 , 18 ]. Notably, telemedicine has garnered increased attention, especially in the aftermath of the COVID-19 pandemic, where its significance became evident due to the implementation of social distancing measures. However, limited studies have explored the population-level interest in telemedicine, as measured by tools such as Google Trends. Some of these studies are summarized in Table S7 in Multimedia Appendix 1 [ 44 - 51 ]. To the best of our knowledge, our study provides the most recent and comprehensive analysis in this field, shedding light on the level of interest among both the general public and researchers. An example of the use of infodemiological methods to explore global interest in telemedicine during the COVID-19 pandemic can be seen in the study conducted by Leochico et al [ 51 ] in 2020. Their study revealed a significant surge in online searches for telemedicine and related terms after the outbreak of the pandemic [ 51 ]. Extending the analysis period until the end of 2022, our study showed that while the level of public interest in telemedicine experienced a significant increase from January to April 2020, it later declined until August 2020 and eventually stabilized. Nevertheless, public interest has remained slightly higher compared to the pre-pandemic period. This finding is consistent with the results of a Google Trends analysis conducted by Wong et al [ 47 ] on the 50 countries most affected by the COVID-19 pandemic until July 2020. According to our findings, Chile, Australia, Canada, and the United States demonstrated the highest levels of public interest in telemedicine. In these countries, we observed moderate to strong correlations between Google Trends and COVID-19 data, including new cases and deaths, as well as hospitalizations. This finding is consistent with the results reported by Arshad Ali et al [ 45 ], who found a significant global correlation between the increase in COVID-19 cases and deaths and the interest in telemedicine.

With regard to research interest, there are examples of bibliometric analyses that existed even before the COVID-19 pandemic. For instance, Armfield et al [ 52 ] analyzed nearly 18,000 publication records, published between 2009 and 2013, to investigate the themes in telemedicine and telehealth literature. They found that the majority of studies focused on the clinical effectiveness of telemedicine. Other research questions include the adoption and implementation of telemedicine and eHealth technologies in health care systems [ 52 ]. Edirippulige et al [ 53 ] conducted a bibliometric analysis of telemedicine-related literature published until 2018 in highly ranked clinical journals and revealed that the acceptance of telemedicine research by these journals indicated a maturing of the telemedicine field. However, the pandemic has led to a surge of research interest in telemedicine and related fields. Our bibliometric analysis, in fact, revealed a considerable increase in the number of publications, particularly from 2020 onwards. Most of these publications came from a single country, with only 1 in 5 featuring international collaborations. Despite the importance of fostering collaborations among various stakeholders, including academics, health administrators, practitioners, policymakers, and communities, which involve reciprocal knowledge translation, such partnerships are often lacking [ 54 ]. The United States led the way as the most productive country, with Canada and Australia following in a cluster. Meanwhile, European, Asian, and Latin American countries comprised the other 3 clusters. Previous studies already demonstrated the predominant role of the United States, with Lan et al [ 55 ] providing a general overview, and Kumar et al [ 56 ] analyzing the trends in orthopedics and trauma-related telemedicine during the COVID-19 pandemic. Another aspect of our analysis focused on whether the different terms, such as telemedicine, telehealth, eHealth, mHealth, and so forth, can be used interchangeably or if each of them refers to a specific area of research. Fatehi and Wootton [ 57 ] conducted a bibliometric analysis in 2012 to examine the trends in the use of terms such as telemedicine, telehealth, and eHealth. They discovered that these terms were frequently used interchangeably, with a growing prevalence of the term “eHealth” in more recent years. On the contrary, our analysis revealed a specific focus on the use of eHealth, mHealth, or digital health for noncommunicable and chronic diseases, while telemedicine and telehealth were predominantly used in the context of the COVID-19 pandemic. The analysis by Lan et al [ 55 ], limited to the application of telemedicine to COVID-19, showed similar results. In particular, telemedicine was mainly used to provide mental health services, health care services delivery, and to control cross-infection. In contrast, the term “mobile apps” was closely associated with chronic illness entities such as diabetes, heart failure, and asthma, as well as health service entities such as patient education and self-care [ 55 ].

Limitations

While Google Trends provides a valuable tool for analyzing public interest, there are several limitations to this approach. First, Google Trends only provides information on internet searches and does not account for offline discussions, media coverage, or other forms of engagement with the topic. This means that our results may not be representative of the entire population or capture the full extent of public interest. Second, the data provided by Google Trends are aggregated and anonymous, making it difficult to determine the specific demographics, motivations, or intentions behind the searches. This can limit the ability to draw meaningful conclusions or make accurate predictions about public behavior or attitudes toward telemedicine. Third, Google Trends data may be subject to various biases, such as the effect of media coverage or search engine optimization strategies. Additionally, the results may be influenced by factors such as seasonality, news events, or changes in search algorithms, making it challenging to compare trends over time or across different regions. Some of these limitations also apply to bibliometric analysis. First, bibliometric data may not reflect the complete picture of research interest as not all research is published and indexed in databases. Some research may be unpublished or published in non-indexed sources. Second, bibliometric analysis may not capture changes in research interest in real time. It can take some time for research to be published, indexed, and reflected in bibliometric data, meaning that the data may not reflect the most current state of research interest. Third, bibliometric analysis may not capture the full range of research interest as it is limited to the keywords used in publications. For instance, to offer a more comprehensive overview, one might consider examining interest in other trending subjects, like generative artificial intelligence and large language models. Finally, bibliometric analysis does not provide insights into the reasons behind the trends observed. It is limited to providing quantitative data on the number and frequency of publications and citations, and cannot provide qualitative insights into the motivations or drivers behind the research interest. For all these reasons, while the analysis of Google Trends and bibliometric data can provide a useful starting point for understanding public and research interest in telemedicine, it is essential to supplement this analysis with other sources of data and to interpret the results with caution.

Conclusions

Our study offers a comprehensive picture of the evolving landscape of telemedicine and its growing importance in health care delivery. By analyzing search and bibliographic data across regions and over time, our study provides valuable insights into the level of interest in telemedicine. This information serves to pinpoint potential application fields, identify gaps in this research, and emphasize areas that warrant additional attention and efforts in raising awareness.

Data Availability

The data sets analyzed during this study are available from the corresponding author on reasonable request.

Authors' Contributions

AM, MB, and AA conceived and designed the study. AM and MB contributed to data collection. AM and MB contributed to the data analysis. AM, MB, GB, and AA contributed to the interpretation of results. AM, MB, GB, and AA contributed to writing the paper.

Conflicts of Interest

None declared.

Additional outputs from the examination of Google Trends and bibliometric data.

• A health telematics policy in support of WHO's health-for-all strategy for global health development: report of the WHO group consultation on health telematics. WHO. 1997. URL: https://pesquisa.bvsalud.org/portal/resource/pt/who-63857 [accessed 2024-02-03]
• Craig J, Patterson V. Introduction to the practice of telemedicine. J Telemed Telecare. 2005;11(1):3-9. [ CrossRef ] [ Medline ]
• The rise of digital health technologies during the pandemic. European Parliamentary Research Service. 2021. URL: https://www.europarl.europa.eu/thinktank/en/document/EPRS_BRI(2021)690548 [accessed 2024-02-03]
• Barchitta M, Maugeri A, Favara G, Lio RMS, Riela PM, Guarnera L, et al. Development of a web-app for the ecological momentary assessment of dietary habits among college students: the HEALTHY-UNICT project. Nutrients. 2022;14(2):330. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barchitta M, Maugeri A, Favara G, Riela PM, Gallo G, Mura I, et al. A machine learning approach to predict healthcare-associated infections at intensive care unit admission: findings from the SPIN-UTI project. J Hosp Infect. 2021;112:77-86. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barchitta M, Maugeri A, Favara G, Riela PM, La Mastra C, La Rosa MC, et al. Cluster analysis identifies patients at risk of catheter-associated urinary tract infections in intensive care units: findings from the SPIN-UTI network. J Hosp Infect. 2021;107:57-63. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barchitta M, Maugeri A, Favara G, Riela PM, Gallo G, Mura I, et al. Early prediction of seven-day mortality in intensive care unit using a machine learning model: results from the SPIN-UTI project. J Clin Med. 2021;10(5):992. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Flodgren G, Rachas A, Farmer AJ, Inzitari M, Shepperd S. Interactive telemedicine: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2015;2015(9):CD002098. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gonçalves-Bradley DC, Maria J, Ricci-Cabello I, Villanueva G, Fønhus MS, Glenton C, et al. Mobile technologies to support healthcare provider to healthcare provider communication and management of care. Cochrane Database Syst Rev. 2020;8(8):CD012927. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wootton R. Telemedicine support for the developing world. J Telemed Telecare. 2008;14(3):109-114. [ CrossRef ] [ Medline ]
• Jennett PA, Hall LA, Hailey D, Ohinmaa A, Anderson C, Thomas R, et al. The socio-economic impact of telehealth: a systematic review. J Telemed Telecare. 2003;9(6):311-320. [ CrossRef ] [ Medline ]
• Rao B, Lombardi A. Telemedicine: current status in developed and developing countries. J Drugs Dermatol. 2009;8(4):371-375. [ Medline ]
• Tilahun B, Gashu KD, Mekonnen ZA, Endehabtu BF, Angaw DA. Mapping the role of digital health technologies in prevention and control of COVID-19 pandemic: review of the literature. Yearb Med Inform. 2021;30(1):26-37. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Blue R, Yang AI, Zhou C, De Ravin E, Teng CW, Arguelles GR, et al. Telemedicine in the era of coronavirus disease 2019 (COVID-19): a neurosurgical perspective. World Neurosurg. 2020;139:549-557. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gachabayov M, Latifi LA, Parsikia A, Latifi R. The role of telemedicine in surgical specialties during the COVID-19 pandemic: a scoping review. World J Surg. 2022;46(1):10-18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Curcic IB, Berkovic MC, Kizivat T, Varzic SC, Smolic R, Smolic M. Effect of COVID-19 on management of type 1 diabetes: pushing the boundaries of telemedical healthcare. World J Diabetes. 2021;12(6):780-785. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Umano GR, Di Sessa A, Guarino S, Gaudino G, Marzuillo P, Del Giudice EM. Telemedicine in the COVID-19 era: taking care of children with obesity and diabetes mellitus. World J Diabetes. 2021;12(5):651-657. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kronenfeld JP, Penedo FJ. Novel coronavirus (COVID-19): telemedicine and remote care delivery in a time of medical crisis, implementation, and challenges. Transl Behav Med. 2021;11(2):659-663. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ghomrawi HMK, Holl JL, Abdullah F. Telemedicine in surgery-beyond a pandemic adaptation. JAMA Surg. Oct 01, 2021;156(10):901-902. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bouabida K, Lebouché B, Pomey M. Telehealth and COVID-19 pandemic: an overview of the telehealth use, advantages, challenges, and opportunities during COVID-19 pandemic. Healthcare (Basel). 2022;10(11):2293. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bragazzi NL, Alicino C, Trucchi C, Paganino C, Barberis I, Martini M, et al. Global reaction to the recent outbreaks of Zika virus: insights from a big data analysis. PLoS One. 2017;12(9):e0185263. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Galido A, Ecleo JJ, Husnayain A, Su ECY. Exploring online search behavior for COVID-19 preventive measures: the Philippine case. PLoS One. 2021;16(4):e0249810. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature. 2009;457(7232):1012-1014. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Greiner B, Ottwell R, Vassar M, Hartwell M. Public interest in preventive measures of coronavirus disease 2019 associated with timely issuance of statewide stay-at-home orders. Disaster Med Public Health Prep. 2020;14(6):765-768. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hamulka J, Jeruszka-Bielak M, Górnicka M, Drywień ME, Zielinska-Pukos MA. Dietary supplements during COVID-19 outbreak. Results of Google Trends analysis supported by PLifeCOVID-19 online studies. Nutrients. 2020;13(1):54. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Keitoku K, Nishimura Y, Hagiya H, Koyama T, Otsuka F. Impact of the world antimicrobial awareness week on public interest between 2015 and 2020: a Google Trends analysis. Int J Infect Dis. 2021;111:12-20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kurian SJ, Bhatti AUR, Alvi MA, Ting HH, Storlie C, Wilson PM, et al. Correlations between COVID-19 cases and Google Trends data in the United States: a state-by-state analysis. Mayo Clin Proc. 2020;95(11):2370-2381. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maugeri A, Barchitta M, Basile G, Agodi A. How COVID-19 has influenced public interest in antimicrobials, antimicrobial resistance and related preventive measures: a Google Trends analysis of Italian data. Antibiotics (Basel). 2022;11(3):379. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maugeri A, Barchitta M, Agodi A. Using Google Trends to predict COVID-19 vaccinations and monitor search behaviours about vaccines: a retrospective analysis of Italian data. Vaccines (Basel). 2022;10(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mavragani A, Gkillas K. COVID-19 predictability in the United States using Google Trends time series. Sci Rep. 2020;10(1):20693. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mayasari NR, Ho DKN, Lundy DJ, Skalny AV, Tinkov AA, Teng IC, et al. Impacts of the COVID-19 pandemic on food security and diet-related lifestyle behaviors: an analytical study of Google trends-based query volumes. Nutrients. 2020;12(10):3103. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maugeri A, Barchitta M, Perticone V, Agodi A. How COVID-19 pandemic has influenced public interest in foods: a Google Trends analysis of Italian data. Int J Environ Res Public Health. 2023;20(3):1976. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arora VS, McKee M, Stuckler D. Google Trends: opportunities and limitations in health and health policy research. Health Policy. 2019;123(3):338-341. [ CrossRef ] [ Medline ]
• Mavragani A, Ochoa G. Google Trends in infodemiology and infoveillance: methodology framework. JMIR Public Health Surveill. 2019;5(2):e13439. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mavragani A, Ochoa G, Tsagarakis KP. Assessing the methods, tools, and statistical approaches in Google Trends research: systematic review. J Med Internet Res. 2018;20(11):e270. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Khudzari JM, Kurian J, Tartakovsky B, Raghavan GSV. Bibliometric analysis of global research trends on microbial fuel cells using scopus database. Biochem Eng J. 2018;136:51-60. [ FREE Full text ] [ CrossRef ]
• Lerman PM. Fitting segmented regression models by grid search. J R Stat Soc. 1980;29(1):77-84. [ CrossRef ]
• Mathieu E, Ritchie H, Rodés-Guirao L, Appel C, Gavrilov D, Giattino C, et al. Coronavirus pandemic (COVID-19). Our World in Data. URL: https://ourworldindata.org/coronavirus [accessed 2024-02-03]
• Aria M, Cuccurullo C. bibliometrix: an R-tool for comprehensive science mapping analysis. J Informetr. 2017:959-975. [ FREE Full text ] [ CrossRef ]
• van Eck NJ, Waltman L. VOSviewer Manual. Manual for VOSviewer version 1 .6.19. 2023. URL: https://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.19.pdf [accessed 2024-02-03]
• Reed M, Huang J, Somers M, Hsueh L, Graetz I, Millman A, et al. Telemedicine versus in-person primary care: treatment and follow-up visits. Ann Intern Med. 2023;176(10):1349-1357. [ CrossRef ] [ Medline ]
• Calton BA, Nouri S, Davila C, Kotwal A, Zapata C, Bischoff KE. Strategies to make telemedicine a friend, not a foe, in the provision of accessible and equitable cancer care. Cancers (Basel). 2023;15(21):5121. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shaw RJ. Access to technology and digital literacy as determinants of health and health care. Creat Nurs. 2023;29(3):258-263. [ CrossRef ] [ Medline ]
• Hong YR, Lawrence J, Williams D, Mainous I. Population-level interest and telehealth capacity of US hospitals in response to COVID-19: cross-sectional analysis of Google search and national hospital survey data. JMIR Public Health Surveill. 2020;6(2):e18961. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arshad Ali S, Bin Arif T, Maab H, Baloch M, Manazir S, Jawed F, et al. Global interest in telehealth during COVID-19 pandemic: an analysis of Google Trends. Cureus. Sep 16, 2020;12(9):e10487. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jimenez AJ, Estevez-Reboredo RM, Santed MA, Ramos V. COVID-19 symptom-related Google searches and local COVID-19 incidence in Spain: correlational study. J Med Internet Res. 2020;22(12):e23518. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wong MYZ, Gunasekeran DV, Nusinovici S, Sabanayagam C, Yeo KK, Cheng CY, et al. Telehealth demand trends during the COVID-19 pandemic in the top 50 most affected countries: infodemiological evaluation. JMIR Public Health Surveill. 2021;7(2):e24445. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alonto AHD, Jamora RDG, Leochico CFD, Espiritu AI. Low online search interest in teleneurology before and during COVID-19 pandemic: an infodemiological study. Neurol Sci. 2022;43(5):2929-2934. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kinoshita T, Matsumoto T, Taura N, Usui T, Matsuya N, Nishiguchi M, et al. Public interest and accessibility of telehealth in Japan: retrospective analysis using Google Trends and national surveillance. JMIR Form Res. 2022;6(9):e36525. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van Kessel R, Kyriopoulos I, Wong BLH, Mossialos E. The effect of the COVID-19 pandemic on digital health-seeking behavior: big data interrupted time-series analysis of Google Trends. J Med Internet Res. 2023;25:e42401. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leochico CFD, Austria EMV, Espiritu AI. Global online interest in telehealth, telemedicine, telerehabilitation, and related search terms amid the COVID-19 pandemic: an infodemiological study. Acta Med Philipp. 2021;56(11):66-75. [ FREE Full text ] [ CrossRef ]
• Armfield NR, Edirippulige S, Caffery LJ, Bradford NK, Grey JW, Smith AC. Telemedicine--a bibliometric and content analysis of 17,932 publication records. Int J Med Inform. 2014;83(10):715-725. [ CrossRef ] [ Medline ]
• Edirippulige S, Senanayake B, Fatehi F, Hansen J, Bambling M, Smith AC, et al. Telemedicine: niche or mainstream? A bibliometric analysis and review of the output of highly ranked clinical journals. J Telemed Telecare. 2024;30(1):53-63. [ CrossRef ] [ Medline ]
• Gerber T, Olazabal V, Brown K, Pablos-Mendez A. An agenda for action on global e-Health. Health Aff (Millwood). 2010;29(2):233-236. [ CrossRef ] [ Medline ]
• Lan X, Yu H, Cui L. Application of telemedicine in COVID-19: a bibliometric analysis. Front Public Health. 2022;10:908756. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kumar A, Sinha S, Jameel J, Kumar S. Telemedicine trends in orthopaedics and trauma during the COVID-19 pandemic: a bibliometric analysis and review. J Taibah Univ Med Sci. 2022;17(2):203-213. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fatehi F, Wootton R. Telemedicine, telehealth or e-health? A bibliometric analysis of the trends in the use of these terms. J Telemed Telecare. Dec 2012;18(8):460-464. [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani; submitted 19.06.23; peer-reviewed by D Dinh, G Gallo, D Gunasekeran; comments to author 30.11.23; revised version received 01.12.23; accepted 03.01.24; published 16.05.24.

©Andrea Maugeri, Martina Barchitta, Guido Basile, Antonella Agodi. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 16.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Introduction
• Conclusions
• Article Information

Data Sharing Statement

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Kutscher E , Arshed A , Greene RE , Kladney M. Exploring Anabolic Androgenic Steroid Use Among Cisgender Gay, Bisexual, and Queer Men. JAMA Netw Open. 2024;7(5):e2411088. doi:10.1001/jamanetworkopen.2024.11088

Manage citations:

© 2024

• Permissions

Exploring Anabolic Androgenic Steroid Use Among Cisgender Gay, Bisexual, and Queer Men

• 1 Division of General Internal Medicine, Icahn School of Medicine at Mount Sinai, New York, New York
• 2 NYU Grossman School of Medicine, New York, New York
• 3 Bellevue Hospital Center, New York, New York

Question   How and why do cisgender gay, bisexual, and queer men use anabolic androgenic steroids (AAS), and what are their unique health care needs?

Findings   In this qualitative study of 12 men, AAS use was secondary to multifactorial motivators, including a likely AAS use disorder and muscle dysmorphia. Despite all participants experiencing harms from use, men seeking medical assistance found practitioners to be insistent on AAS cessation and, thus, developed their own harm reduction techniques.

Meaning   Cisgender gay, bisexual, and queer men using AAS reported insufficient medical support, suggesting that further research is warranted on the utility of practitioner education, the safety and efficacy of community-developed harm reduction methods, and the impact of AAS decriminalization on health care outcomes for this patient population.

Importance   Anabolic androgenic steroids (AAS) are disproportionately used by sexual minority men, with the physical and mental health implications of AAS use incompletely understood.

Objective   To understand the reasons for use and health care needs of gay, bisexual, and queer cisgender men using AAS.

Design, Setting, and Participants   This qualitative study was conducted from November 2021 to May 2023 using self-administered questionnaires and semistructured interviews that were transcribed and coded using reflexive thematic analysis. Participants were recruited through convenience and snowball sampling from lesbian, gay, bisexual, transgender, and queer clinical centers in New York, New York, as well as through online platforms. All patients self-identified as cisgender and gay, bisexual, or queer.

Exposures   History of nonprescribed AAS use for a minimum of 8 consecutive weeks was required.

Main Outcomes and Measures   The primary outcomes were reasons for and health implications of AAS use and interactions with health care practitioners, as determined through interviews. Interview transcripts were collected and analyzed.

Results   Thematic saturation was reached after interviews with 12 male participants (mean [SD] age, 44 [11] years), with the majority of participants identifying as gay (10 participants [83%]), White non-Hispanic (9 participants [75%]), being in their 30s and 40s (9 participants [75%]), holding a bachelor’s degree or higher (11 participants [92%]), and having used steroids for a mean (SD) of 7.5 (7.1) years. One participant (8%) self-identified as Black, and 2 (17%) identified as Hispanic. Seven men (58%) met the criteria for muscle dysmorphia on screening. Nine overarching themes were found, including internal and external motivators for initial use, continued use because of effectiveness or fear of losses, intensive personal research, physical and emotional harms experienced from use, using community-based harm reduction techniques, frustration with interactions with the medical community focused on AAS cessation, and concerns around the illegality of AAS.

Conclusions and Relevance   In this qualitative study, AAS use among cisgender gay, bisexual, and queer men was found to be associated with multifactorial motivators, including a likely AAS use disorder and muscle dysmorphia. Despite all participants experiencing harms from use, men seeking medical help found insufficient support with practitioners insistent on AAS cessation and, thus, developed their own harm reduction techniques. Further research is needed to assess the utility of practitioner education efforts, the safety and efficacy of community-developed harm reduction methods, and the impact of AAS decriminalization on health care outcomes for this patient population.

Anabolic androgenic steroids (AAS) are synthetic testosterones used by approximately 2.9 to 4.0 million US residents at least once, with 1 million developing AAS dependence. 1 AAS use disproportionately impacts gay, bisexual, and queer (GBQ) men, with 5.0% to 13.5% of gay and bisexual men reporting use, and up to 25.0% considering use. 2 , 3 Although many motivations exist for AAS use, improving physical appearance and gaining muscle and strength are the most common. 4 Among GBQ men, the history of HIV and prescriptions for AAS for HIV wasting syndrome, 5 views on masculinity, 6 gender role strain and internalized homophobia, 7 increased pressures to obtain an ideal male physique, and overemphasis on appearance and desirability 8 , 9 are cited as additional factors.

Despite widespread use, the health impacts of AAS remain incompletely understood. Although AAS is associated with increased risk of acquiring hepatitis B and C viruses, 10 its role in coronary artery disease, 11 , 12 stroke, 13 , 14 and the risk of developing prostate cancer remain poorly characterized. 15 Remarkably, gay men using AAS have a prevalence of HIV 39 times greater than that of the general population, although many acquire the virus before initiating AAS. 10 GBQ men using AAS have higher self-reported rates of male-male condomless anal sex in the past year, as well as increased use of ecstasy and methamphetamines. 16

The mental health implications of AAS are also unclear. Although a diagnosis of AAS use disorder was excluded in the Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) ( DSM-5 ), multiple studies have found an addiction model, including loss of control, social impairment, risky use, and pharmacologic dependence, in up to 24% of individuals using AAS. 17 - 23 AAS use has also been closely associated with muscle dysmorphia (a subtype of body dysmorphic disorder), including among cisgender sexual minority men. 24 , 25

Despite these concerns, most individuals using AAS have declined to disclose their use to a medical practitioner, and general practitioners themselves often feel underprepared when caring for patients using AAS. 26 The majority of individuals who disclose AAS use have felt discriminated against after disclosure. 4 Treatment for AAS dependence is controversial, with accepted practice ranging from psychotherapy to harm reduction to prescribed testosterone, with the unique treatment needs of GBQ men unknown. 27 - 29 Although facilitators to care for individuals using AAS have been investigated, the particular needs of GBQ men remain poorly defined. 30 This project seeks to understand the lived experiences of GBQ cisgender men using AAS in the US, with a particular focus on motivations and patterns of use, knowledge-seeking behaviors, perceived benefits and harms of use, interactions with health care practitioners, and community perspectives on how practitioners can best care for patients using AAS.

This qualitative study used semistructured interviews and self-administered questionnaires administered between November 2021 through May 2023. Inclusion criteria included cisgender adult men who self-identified as GBQ and had used AAS for at least 8 consecutive weeks at any point of their life. Exclusion criteria included active psychotic or manic symptoms, exclusive use of prescription AAS by a practitioner with use only as prescribed, and past or current prostate cancer. Participants were recruited through convenience and snowball sampling from lesbian, gay, bisexual, transgender, and queer clinical centers in New York, New York, using posters and social media platforms. Recruitment occurred until theoretical saturation was reached. 31

This study was approved by the NYU Langone Health and Bellevue Hospital Center Independent Review Boards. Participants gave written consent after reading a participant information form and were given a chance to ask questions before participation. In this article, identifying attributes are avoided. Data reporting followed Consolidated Criteria for Reporting Qualitative Research ( COREQ ) reporting guidelines. 32

Participants self-administered a demographics questionnaire and a screening questionnaire testing for muscle dysmorphia, the Muscle Dysmorphic Disorder Inventory, 33 , 34 using REDCap. 35 , 36 One of 2 authors with no prior patient relationship (E.K., a cisgender gay White man, or A.A., a cisgender gay South Asian man) conducted interviews either in person or through a Health Insurance Portability and Accountability Act–secured videoconferencing platform. Interviews were approximately 1 hour long, with 1 participant requesting a brief follow-up interview, and were audio recorded and transcribed using a computer-based transcription system (Otter.ai; Otter). Authors wrote field notes after each interview. All participants were compensated with $100 gift card. Participants’ race and ethnicity were self-identified and were included in this study to assess for the generalizability of our findings.

The authors developed an interview guide using concept elicitation questioning, highlighting topics found to be important in prior research. Two preliminary interviews were conducted with this guide, after which modifications were made to enhance the flow and clarity of interviews.

Interviews were evaluated through the reflexive thematic analysis approach, as described by Braun et al, 37 following inductive reasoning. Two reviewing authors (E.K. and M.K.) familiarized themselves with the transcripts and developed coding for all interviews independently using Atlas.ti version 22 (ATLAS.ti Scientific Software Development GmbH) to categorize responses, then harmonizing their categorization to create consensus themes. The 2 reviewing authors then developed and reviewed these themes, followed by refining, defining, and naming themes. Both authors acknowledged their identity as cisgender White gay men, critically evaluating their analyses to assess for bias. Finally, themes were woven together with data extracts and shared among the entire research team for consensus. Questionnaires were reviewed, with a Muscle Dysmorphic Disorder Inventory score of 39 points considered meeting the criteria for muscle dysmorphia. 33 , 38 , 39 Participants were not provided with interview transcripts or analysis before publication.

Twelve male participants (mean [SD] age, 44 [11] years) completed semistructured interviews until thematic saturation was reached, with the majority identifying as gay (10 participants [83%]) and White non-Hispanic (9 participants [75%]), being in their 30s and 40s (9 participants [75%]), and holding a bachelor’s degree or higher (11 participants [92%]). Of our sample, 1 participant (8%) self-identified as Black, and 2 (17%) identified as Hispanic. Participants had used steroids for a mean (SD) of 7.5 (7.1) years. During recruitment, an additional 2 participants declined to partake in the study after screening, 3 participants completed screening but were then lost to follow-up, 6 did not follow up after initially reaching out to study team, and 7 did not meet inclusion criteria; possible differences in demographics between cohort participants and those lost to follow-up are unknown. Table 1 outlines the demographics of participants. Seven of the 12 participants (58%) met the criteria for muscle dysmorphia ( Table 2 ). Nine overarching themes with additional subthemes were found ( Table 3 ).

Participants reported internal and external motivations for AAS initiation. As internal motivators, some men identified childhood obesity or thinness as reasons to pursue a more muscular body in adulthood. For others, underlying body dysmorphia predated and/or motivated AAS use. Some men noted the importance of unrealistic body ideals for GBQ men and a desire to try and achieve them. Insecurities around sexual orientation and an attempt to compensate for perceived deficits in masculinity were also reported.

As external motivators, partners and peers were frequently cited as instigators for AAS use. The pressure to obtain an ideal body was most frequently reported around preparing for event in which men anticipated being shirtless around other GBQ men (such as circuit parties and gay cruises or vacation destinations). Participants repeatedly described a community that normalized and sometimes explicitly encouraged AAS use. Multiple men highlighted their work in the adult entertainment industry as practically requiring them to use AAS to be successful.

After starting AAS, most participants noticed gains and changes in their physique along with increased energy and libido that positively reinforced continued use. These gains were described by both participants who felt that AAS were working as intended and those who felt that higher doses or more intense AAS regimens were needed to obtain their goals.

Some participants felt that AAS use improved self-image sufficiently to combat preexisting muscle dysmorphia and body dissatisfaction, whereas others felt that AAS use made the dysmorphia worse by placing the power of easily changing body shape within their control. Most participants found increased external validation to be an important motivator for continued use and felt that AAS use allowed them to climb the social ladder within the GBQ community.

Multiple men expressed a need to continue AAS out of fear of discontinuation. Some felt ready to stop AAS but were scared to lose muscle gains, whereas others continued using to avoid confronting what their body looked like when not enhanced with AAS. When considering cessation of AAS, many reported a fear of a crash or withdrawal when stopping and thus continued use, whereas others noticed an unpleasant and unmanageable period of decreased energy from secondary hypogonadism that ultimately motivated them to reinitiate use.

Access to evidence-based and reliable information was almost universally desired, with most men conducting substantial amounts of research on AAS. However, data sources were often informal with information sourced from nonmedical websites, social media postings, and blogs. Peers, coaches, and suppliers were often relied on for guidance on effective regimens and methods to mitigate adverse effects. Notably, multiple men searched medical journals and peer reviewed research, extrapolating from these materials information that felt relevant to their own lives and AAS practices. These interpretations were often insightful, with participants able to explain biological mechanisms of action. Participant research was occasionally made possible from working as a health care practitioner, or more frequently from being close to a health care practitioner who used AAS and would provide friends with guidance and support. Overall, men desired more streamlined and reputable methods for information gathering.

Every participant reported at least one instance of harm they attributed to AAS use. Most commonly noted harms were cardiovascular, including hyperlipidemia, hypertension, hypertrophic cardiomyopathy, and atrial fibrillation. One man reported needing a cardioversion, which he attributed to AAS use, whereas another reported developing symptoms concerning for heart failure. Two men reported tendon ruptures requiring surgery, with an additional 3 developing tendinitis requiring physical therapy. Two participants developed cellulitis at an injection site. Multiple men reported balding, acne, testicular shrinkage, and night sweats as adverse effects.

Aside from physical harms, emotional harms were reported frequently, including difficulty with anger management, emotional instability, and worsening muscle dysmorphia. One participant reported that steroids were the root cause of his recent divorce. Men reported a concern about AAS use being perceived as cheating to obtain results and insisted that hard work at the gym was the primary reason for muscle gains. Participants frequently kept their AAS use secret out of fear of judgment or embarrassment. A sense of moral distress and self-judgment was notable, with men wondering whether their use showed an overvaluing of aesthetics and superficiality.

Men universally attempted to avoid or minimize AAS-related harms through harm reduction methods. Participants prioritized obtaining AAS from seemingly reliable sources, inspecting labels for any evidence of tampering or low-quality manufacturing. Men learned sterile injection techniques, obtaining supplies for injecting from health care supply warehouses, online, or through needle exchange programs. Men regularly planned schedules with cycling, or having discrete periods of steroid use followed by abstinence. At the end of a cycle, men reported using postcycle therapy (often clomiphene) to induce endogenous testosterone production and minimize the risk of secondary hypogonadism. All men routinely checked bloodwork, monitoring blood counts, kidney and liver function, prostate-specific antigen levels, and cholesterol levels. A few reported daily checks of their blood pressure or blood glucose levels. Cardiovascular exercise was often considered protective. A minority of participants hired nonmedical coaches to monitor their cycles to optimize overall safety. Most participants noted that steroid use had increased their libido and the number of sexual encounters. However, participants used preexposure prophylaxis (PrEP) for HIV prevention and attributed an increase in condomless sexual activity and increased number of sexual partners more to PrEP than to AAS use.

Participants commonly used additional nonprescribed medications to mitigate the effects of steroid use. For example, some men reported taking aromatase inhibitors and selective estrogen receptor modulators to decrease gynecomastia. Aspirin was thought to decrease the risk of blood clots and strokes from polycythemia, with participants also opting to donate blood to decrease blood counts. Many took prescribed or nonprescribed statins to decrease cholesterol. One participant purchased nonprescribed telmisartan to decrease his risk of developing hypertrophic cardiomyopathy.

Study participants reported discontentment and distrust of the medical system, with real and perceived stigma preventing multiple participants from disclosing steroid use. For participants who did disclose, there was a sense of dissatisfaction with practitioner knowledge about the risks and benefits of steroid use and a discomfort with clinicians focusing exclusively on abstinence from AAS. For the participant who required a cardioversion, he withheld information on AAS use until moments before the procedure because of fear of judgment.

Facilitators to participant disclosure to health care practitioners were reported. First, men overall felt most comfortable talking about steroid use with doctors who themselves identified as GBQ or who had a large GBQ patient panel. Men perceived these clinicians as less judgmental and less likely to focus on AAS cessation or abstinence only. Participants also embraced online clinics where testosterone could be prescribed by a practitioner in a direct-to-consumer model.

When discussing how to optimize the care for individuals using AAS, the legal status of AAS as a Schedule III drug was perceived as one of the biggest barriers to care. Men were concerned that the illegality of their substance use required them to purchase from unregulated markets and to hide their use from their doctors. Participants often felt that there was an arbitrary distinction between use of AAS at physiologic vs supraphysiologic doses and that AAS use itself was more aligned with plastic surgery than with many other recreational drugs. The illegality of AAS resulted in a fear of documentation of use, with men concerned that disclosure could result in denial of disability and life insurance, refusal to prescribe AAS for secondary hypogonadism in the future, and employment consequences.

Overall, the themes from this qualitative study were similar to those found in previous research among cisgender men regardless of sexual orientation. 30 Individuals start and continue AAS for many reasons, not always for underlying body and/or muscle dysmorphia. The importance of masculinity discussed by participants overlaps with machismo views on masculinity and self-confidence, as described elsewhere. 40 - 42

Our work contributes to the literature describing the existence of an AAS use disorder, because many participants continued use despite harm owing to the efficacy of AAS and the fear of losing muscle mass if abstinent. 21 , 23 , 43 , 44 This bias toward an addictive nature may be the result of our inclusion criteria requiring a minimum of 8 weeks of use yet supports the inclusion of AAS use disorder within future iterations of the DSM .

Our study participants did not associate their AAS use with any increase in risk of acquiring HIV. Unlike Ip et al, 10 , 16 men in our study reported an increase in number of sexual partners from increased libido with steroid use, but reported consistent use of PrEP predating their AAS use, with no change in the likelihood of unprotected sex from AAS use.

Individuals using AAS desired accurate, scientifically reviewed, and comprehensive information on AAS use, with high-quality and evidence-based resources thought to be highly beneficial. The interim need to rely on peers for information instead of medical practitioners was similar to findings from other studies. 45 - 47

Participants perceived that disclosure to health care practitioners could be more harmful to their care than therapeutic given the lack of knowledge of most health care practitioners and the repercussions of medical record documentation, thus highlighting the urgent need for enhanced practitioner education and use of empathetic approaches to care for sexual minority men using AAS. 9 Abstinence-focused models of care for individuals using AAS did not appear to be efficacious for participants in our study, because all men experienced harm from AAS but continued to use nonetheless. Participants who disclosed AAS use often perceived messages focused on cessation as bias and discrimination, leading to decisions to forgo future medical care. Further research can assess the acceptability of more nuanced discussions about abstinence focused on the specific AAS most associated with harm, such as trenbolone. 48

In the absence of medical harm reduction guidance, participants described community-developed harm reduction techniques, with some techniques similar to those proposed as treatments for AAS dependence. 49 , 50 Community-led medical interventions have a historical role within the GBQ community, as demonstrated through the inventions of PrEP and doxycycline postexposure prophylaxis. Yet, many harm reduction techniques in our study are newly reported and require further research on their safety and efficacy.

Finally, illegality around AAS use complicates the ability for patients to obtain medical care, as similarly demonstrated by Piatkowski et al. 51 To optimize health care outcomes, future work should revisit the risks and benefits of the scheduling of AAS, as well as test methods to prevent AAS use disclosure from the electronic medical record to life and disability insurance companies.

This study has limitations that should be mentioned. We had great difficulty recruiting for our study. Although 30 individuals contacted our research team with interest in enrollment, 18 participants ultimately were not included in the study owing to ineligibility, loss to follow-up, or declining to participate after initial screening. The final cohort of participants was predominantly White, college-educated gay men, despite outreach and recruitment in areas where traditionally marginalized individuals seek care. This is consistent with prior research showing that men in the US using AAS were typically White, highly educated, and with an above-average income. 52 It is unknown whether this cohort reflects the demographics of the GBQ population using AAS given the substantial cost and knowledge that participants relied on to use AAS, or whether our results are subject to a sampling bias.

In this qualitative study, AAS use among GBQ men was found to be the result of multifactorial motivators, including a likely AAS use disorder and muscle dysmorphia. Despite all participants experiencing harms from use, men seeking help found insufficient health care support from practitioners insistent on AAS cessation and, thus, developed their own harm reduction techniques. Further research is warranted to assess for the utility of physician education efforts, the efficacy and safety of community-developed harm reduction methods, and the impact of AAS decriminalization on health care outcomes for this patient population.

Accepted for Publication: March 12, 2024.

Published: May 14, 2024. doi:10.1001/jamanetworkopen.2024.11088

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2024 Kutscher E et al. JAMA Network Open .

Corresponding Author: Eric Kutscher, MD, Division of General Internal Medicine, Icahn School of Medicine at Mount Sinai, 17 E 102nd St, 6th Floor, New York, NY 10029 ( [email protected] ).

Author Contributions: Drs Kutscher and Kladney had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Kutscher, Arshed.

Critical review of the manuscript for important intellectual content: All authors.

Statistical analysis: Arshed, Greene, Kladney.

Obtained funding: Kutscher.

Administrative, technical, or material support: Kutscher, Greene, Kladney.

Supervision: Arshed, Greene, Kladney.

Conflict of Interest Disclosures: None reported.

Funding/Support: This research was supported with funding from the TULA Health Research Fellowship. Additional support for the project was provided through mentorship in the Research in Addiction Medicine Scholars Program (National Institute on Drug Abuse grant R25DA033211), the NYU Clinical and Translational Sciences Institute (National Institutes of Health grant UL 1TR001445), and tuition support through the Health Resources and Services Administration (grant T25HP37605).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See the Supplement .

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts