mixed method research articles

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Nepal J Epidemiol
• v.12(1); 2022 Mar

The Growing Importance of Mixed-Methods Research in Health

Sharada prasad wasti.

1,2 School of Human and Health Sciences, University of Huddersfield, United Kingdom

Padam Simkhada

3 Centre for Midwifery, Maternal and Perinatal Health, Bournemouth University, Bournemouth, United Kingdom

Edwin R. van Teijlingen

Brijesh sathian.

4 Geriatrics and long term care Department, Rumailah Hospital, Hamad Medical Corporation, Doha, Qatar

Indrajit Banerjee

5 Sir Seewoosagur Ramgoolam Medical College, Belle Rive, Mauritius

All authors have made substantial contributions to all of the following: (1) the conception and design of the study (2) drafting the article or revising it critically for important intellectual content, (3) final approval of the version to be submitted

There is no conflict of interest for any author of this manuscript.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sector.

This paper illustrates the growing importance of mixed-methods research to many health disciplines ranging from nursing to epidemiology. Mixed-methods approaches requires not only the skills of the individual quantitative and qualitative methods but also a skill set to bring two methods/datasets/findings together in the most appropriate way. Health researchers need to pay careful attention to the ‘best’ approach to designing, implementing, analysing, integrating both quantitative (number) and qualitative (word) information and writing this up in a way offers greater insights and enhances its applicability. This paper highlights the strengths and weaknesses of mixed-methods approaches as well as some of the common mistakes made by researchers applying mixed-methods for the first time.

Quantitative and qualitative research methods each address different types of questions, collect different kinds of data and deliver different kinds of answers. Each set of methods has its own inherent strengths and weaknesses, and each offers a particular approach to address specific types of research questions (and agendas). Health disciplines such as dentistry, nursing, speech and language therapy, and physiotherapy often use either quantitative or qualitative research methods on their own. However, there is a steadily growing literature showing the advantages of mixed-methods research is used in the health care and health service field [ 1-2 ]. Although we have advocated the use of mixed-methods in this journal eight years ago [ 3 ], there is still not enough mixed-methods research training in the health research field, particularly for health care practitioners, such as nurses, physiotherapists, midwives, and doctors, wanting to do research. Mixed-methods research has been popular in the social sciences since the twentieth century [ 4 ], and it has been growing in popularity among healthcare professionals [ 5 ], although it is still underdeveloped in disciplines such nursing and midwifery [ 6 , 7 ].

Underpinning philosophies

To help understand that mixed-methods research is not simply employing two different methods in the same study, one needs to consider their underpinning research philosophies (also called paradigms). First, quantitative research is usually underpinned by positivism. This includes most epidemiological studies; such research is typically based on the assumption that there is one single real world out there that can be measured. For example, quantitative research would address the question “What proportion of the population of India drinks coffee?” Secondly, qualitative research is more likely to be based on interpretivism. This includes research based on interviews and focus groups, research which us is typically based on the assumption that we all experience the world differently. Since we all live in a slightly different world in our heads the task of qualitative research is to analyse the interpretations of the people in the sample. For example, qualitative research would address the question “How do people experience drinking coffee in India?”, and “What does drinking coffee mean to them?”

Mixed-methods research brings together questions from two different philosophies in what is being referred to as the third path [ 8 ], third research paradigm [ 9 , 10 ], the third methodology movement [ 11 , 12 ] and pragmatism [ 5 ]. The two paradigms differ in key underlying assumptions that ultimately lead to choices in research methodology and methods and often give a breadth by answering more complicated research questions [ 4 ]. The roles of mixed-methods are clear in an understanding of the situation (the what), meaning, norms, values (the why or how) within a single research question which combine the strength of two different method and offer multiple ways of looking at the research question [ 13 ]. Epidemiology sits strongly in the quantitative research corner, with a strong emphasis on large data sets and sophisticated statistical analysis. Although the use of mixed methods in health research has been discussed widely researchers raised concerns about the explanation of why and how mixed methods are used in a single research question [ 5 ].

The relevance of mixed-methods in health research

The overall goal of the mixed-methods research design is to provide a better and deeper understanding, by providing a fuller picture that can enhance description and understanding of the phenomena [ 4 ]. Mixed-methods research has become popular because it uses quantitative and qualitative data in one single study which provides stronger inference than using either approach on its own [ 4 ]. In other words, a mixed-methods paper helps to understand the holistic picture from meanings obtained from interviews or observation to the prevalence of traits in a population obtained from surveys, which add depth and breadth to the study. For example, a survey questionnaire will include a limited number of structured questions, adding qualitative methods can capture other unanticipated facets of the topic that may be relevant to the research problem and help in the interpretation of the quantitative data. A good example of a mixed-methods study, it one conducted in Australia to understand the nursing care in public hospitals and also explore what factors influence adherence to nursing care [ 14 ]. Another example is a mixed-methods study that explores the relationship between nursing care practices and patient satisfaction. This study started with a quantitative survey to understand the general nursing services followed by qualitative interviews. A logistic regression analysis was performed to quantify the associations between general nursing practice variables supplemented with a thematic analysis of the interviews [ 15 ]. These research questions could not be answered if the researchers had used either qualitative or quantitative alone. Overall, this fits well with the development of evidence-based practice.

Despite the strengths of mixed-methods research but there is not much of it in nursing and other fields [ 7 ]. A recent review paper shows that the prevalence of mixed-methods studies in nursing was only 1.9% [ 7 ]. Similarly, a systematic review synthesised a total of 20 papers [ 16 ], and 16 papers [ 17 ] on nursing-related research paper among these only one mixed-methods paper was identified. Worse, a further two mixed-methods review recently revealed that out of 48 [ 18 , 19 ] synthesised nursing research papers, not one single mixed-methods paper was identified. This clearly depicts that mixed-methods research is still in its infancy stage in nursing but we can say there is huge scope to implement it to understand research questions on both sides of coin [ 4 ]. Therefore, there is a great need for mixed-methods training to enhance the evidence-based decision making in health and nursing practices.

Strengths and weaknesses of mixed-methods

There are several challenges in identifying expertise of both methods and in working with a multidisciplinary, interdisciplinary, or transdisciplinary team [ 20 ]. It increases costs and resources, takes longer to complete as mixed-methods design often involves multiple stages of data collection and separate data analysis [ 4 , 5 ]. Moreover, conducting mixed-methods research does not necessarily guarantee an improvement in the quality of health research. Therefore, mixed-methods research is only appropriate when there are appropriate research questions [ 4 , 6 ].

Identifying an appropriate mixed-methods journal can also be challenging when writing mixed-methods papers [ 21 ]. Mixed-methods papers need considerably more words than single-methods papers as well as sympathetic editors who understand the underlying philosophy of a mixed-methods approach. Such papers, simply require more words. The mixed-methods researcher must be reporting two separate methods with their own characteristics, different samples, and ways of analysing, therefore needs more words to describe both methods as well as both sets of findings. Researcher needs to find a journal that accepts longer articles to help broaden existing evidence-based practice and promote its applicability in the nursing field [ 22 ].

Common mistakes in applying mixed-methods

Not all applied researchers have insight into the underlying philosophy and/or the skills to apply each set of methods appropriately. Younas and colleagues’ review identified that around one-third (29%) of mixed-methods studies did not provide an explicit label of the study design and 95% of studies did not identify the research paradigm [ 7 ]. Whilst several mixed-methods publications did not provide clear research questions covering both quantitative and qualitative approaches. Another common issue is how to collect data either concurrent or sequential and the priority is given to each approach within the study where equal or dominant which are not clearly stated in writing which is important to mention while writing in the methods section. Similarly, a commonly overlooked aspect is how to integrate both findings in a paper. The responsibility lies with the researcher to ensure that findings are sufficiently plausible and credible [ 4 ]. Therefore, intensive mixed-methods research training is required for nursing and other health practitioners to ensure its appropriate.

The way forward

Despite the recognised strengths and benefits of doing mixed-methods research, there is still only a limited number of nursing and related-health research publications using such this approach. Researchers need training in how to design, conduct, analyse, synthesise and disseminate mixed-methods research. Most importantly, they need to consider appropriate research questions that can be addressed using a mixed methods approach to add to our knowledge in evidence-based practice. In short, we need more training on mixed-methods research for a range of health researchers and health professionals.

Acknowledgement

Mixed methods research: what it is and what it could be

• Open access
• Published: 29 March 2019
• Volume 48 , pages 193–216, ( 2019 )

Cite this article

You have full access to this open access article

• Rob Timans 1 ,
• Paul Wouters 2 &
• Johan Heilbron 3  

112k Accesses

91 Citations

13 Altmetric

Explore all metrics

A Correction to this article was published on 06 May 2019

This article has been updated

Combining methods in social scientific research has recently gained momentum through a research strand called Mixed Methods Research (MMR). This approach, which explicitly aims to offer a framework for combining methods, has rapidly spread through the social and behavioural sciences, and this article offers an analysis of the approach from a field theoretical perspective. After a brief outline of the MMR program, we ask how its recent rise can be understood. We then delve deeper into some of the specific elements that constitute the MMR approach, and we engage critically with the assumptions that underlay this particular conception of using multiple methods. We conclude by offering an alternative view regarding methods and method use.

Similar content being viewed by others

Mixed Methods

Avoid common mistakes on your manuscript.

The interest in combining methods in social scientific research has a long history. Terms such as “triangulation,” “combining methods,” and “multiple methods” have been around for quite a while to designate using different methods of data analysis in empirical studies. However, this practice has gained new momentum through a research strand that has recently emerged and that explicitly aims to offer a framework for combining methods. This approach, which goes by the name of Mixed Methods Research (MMR), has rapidly become popular in the social and behavioural sciences. This can be seen, for instance, in Fig.  1 , where the number of publications mentioning “mixed methods” in the title or abstract in the Thomson Reuters Web of Science is depicted. The number increased rapidly over the past ten years, especially after 2006. Footnote 1

Fraction of the total of articles mentioning Mixed Method Research appearing in a given year, 1990–2017 (yearly values sum to 1). See footnote 1

The subject of mixed methods thus seems to have gained recognition among social scientists. The rapid rise of the number of articles mentioning the term raises various sociological questions. In this article, we address three of these questions. The first question concerns the degree to which the approach of MMR has become institutionalized within the field of the social sciences. Has MMR become a recognizable realm of knowledge production? Has its ascendance been accompanied by the production of textbooks, the founding of journals, and other indicators of institutionalization? The answer to this question provides an assessment of the current state of MMR. Once that is determined, the second question is how MMR’s rise can be understood. Where does the approach come from and how can its emergence and spread be understood? To answer this question, we use Pierre Bourdieu’s field analytical approach to science and academic institutions (Bourdieu 1975 , 1988 , 2004 , 2007 ; Bourdieu et al. 1991 ). We flesh out this approach in the next section. The third question concerns the substance of the MMR corpus seen in the light of the answers to the previous questions: how can we interpret the specific content of this approach in the context of its socio-historical genesis and institutionalization, and how can we understand its proposal for “mixing methods” in practice?

We proceed as follows. In the next section, we give an account of our theoretical approach. Then, in the third, we assess the degree of institutionalization of MMR, drawing on the indicators of academic institutionalization developed by Fleck et al. ( 2016 ). In the fourth section, we address the second question by examining the position of the academic entrepreneurs behind the rise of MMR. The aim is to understand these agents’ engagement in MMR, as well as its distinctive content as being informed by their position in this field. Viewing MMR as a position-taking of academic entrepreneurs, linked to their objective position in this field, allows us to reflect sociologically on the substance of the approach. We offer this reflection in the fifth section, where we indicate some problems with MMR. To get ahead of the discussion, these problems have to do with the framing of MMR as a distinct methodology and its specific conceptualization of data and methods of data analysis. We argue that these problems hinder fruitfully combining methods in a practical understanding of social scientific research. Finally, we conclude with some tentative proposals for an alternative view on combining methods.

A field approach

Our investigation of the rise and institutionalization of MMR relies on Bourdieu’s field approach. In general, field theory provides a model for the structural dimensions of practices. In fields, agents occupy a position relative to each other based on the differences in the volume and structure of their capital holdings. Capital can be seen as a resource that agents employ to exert power in the field. The distribution of the form of capital that is specific to the field serves as a principle of hierarchization in the field, differentiating those that hold more capital from those that hold less. This principle allows us to make a distinction between, respectively, the dominant and dominated factions in a field. However, in mature fields all agents—dominant and dominated—share an understanding of what is at stake in the field and tend to accept its principle of hierarchization. They are invested in the game, have an interest in it, and share the field’s illusio .

In the present case, we can interpret the various disciplines in the social sciences as more or less autonomous spaces that revolve around the shared stake in producing legitimate scientific knowledge by the standards of the field. What constitutes legitimate knowledge in these disciplinary fields, the production of which bestows scholars with prestige and an aura of competence, is in large part determined by the dominant agents in the field, who occupy positions in which most of the consecration of scientific work takes place. Scholars operating in a field are endowed with initial and accumulated field-specific capital, and are engaged in the struggle to gain additional capital (mainly scientific and intellectual prestige) in order to advance their position in the field. The main focus of these agents will generally be the disciplinary field in which they built their careers and invested their capital. These various disciplinary spaces are in turn part of a broader field of the social sciences in which the social status and prestige of the various disciplines is at stake. The ensuing disciplinary hierarchy is an important factor to take into account when analysing the circulation of new scientific products such as MMR. Furthermore, a distinction needs to be made between the academic and the scientific field. While the academic field revolves around universities and other degree-granting institutions, the stakes in the scientific field entail the production and valuation of knowledge. Of course, in modern science these fields are closely related, but they do not coincide (Gingras and Gemme 2006 ). For instance, part of the production of legitimate knowledge takes place outside of universities.

This framework makes it possible to contextualize the emergence of MMR in a socio-historical way. It also enables an assessment of some of the characteristics of MMR as a scientific product, since Bourdieu insists on the homology between the objective positions in a field and the position-takings of the agents who occupy these positions. As a new methodological approach, MMR is the result of the position-takings of its producers. The position-takings of the entrepreneurs at the core of MMR can therefore be seen as expressions in the struggles over the authority to define the proper methodology that underlies good scientific work regarding combining methods, and the potential rewards that come with being seen, by other agents, as authoritative on these matters. Possible rewards include a strengthened autonomy of the subfield of MMR and an improved position in the social-scientific field.

The role of these entrepreneurs or ‘intellectual leaders’ who can channel intellectual energy and can take the lead in institution building has been emphasised by sociologists of science as an important aspect of the production of knowledge that is visible and recognized as distinct in the larger scientific field (e.g., Mullins 1973 ; Collins 1998 ). According to Bourdieu, their position can, to a certain degree, explain the strategy they pursue and the options they perceive to be viable in the trade-off regarding the risks and potential rewards for their work.

We do not provide a full-fledged field analysis of MMR here. Rather, we use the concept as a heuristic device to account for the phenomenon of MMR in the social context in which it emerged and diffused. But first, we take stock of the current situation of MMR by focusing on the degree of institutionalization of MMR in the scientific field.

The institutionalization of mixed methods research

When discussing institutionalization, we have to be careful about what we mean by this term. More precisely, we need to be specific about the context and distinguish between institutionalization in the academic field and institutionalization within the scientific field (see Gingras and Gemme 2006 ; Sapiro et al. 2018 ). The first process refers to the establishment of degrees, curricula, faculties, etc., or to institutions tied to the academic bureaucracy and academic politics. The latter refers to the emergence of institutions that support the autonomization of scholarship such as scholarly associations and scientific journals. Since MMR is still a relatively young phenomenon and academic institutionalization tends to lag scientific institutionalization (e.g., for the case of sociology and psychology, see Sapiro et al. 2018 , p. 26), we mainly focus here on the latter dimension.

Drawing on criteria proposed by Fleck et al. ( 2016 ) for the institutionalization of academic disciplines, MMR seems to have achieved a significant degree of institutionalization within the scientific field. MMR quickly gained popularity in the first decade of the twenty-first century (e.g., Tashakkori and Teddlie 2010c , pp. 803–804). A distinct corpus of publications has been produced that aims to educate those interested in MMR and to function as a source of reference for researchers: there are a number of textbooks (e.g., Plowright 2010 ; Creswell and Plano Clark 2011 ; Teddlie and Tashakkori 2008 ); a handbook that is now in its second edition (Tashakkori and Teddlie 2003 , 2010a ); as well as a reader (Plano Clark and Creswell 2007 ). Furthermore, a journal (the Journal of Mixed Methods Research [ JMMR] ) was established in 2007. The JMMR was founded by the editors John Creswell and Abbas Tashakkori with the primary aim of “building an international and multidisciplinary community of mixed methods researchers.” Footnote 2 Contributions to the journal must “fit the definition of mixed methods research” Footnote 3 and explicitly integrate qualitative and quantitative aspects of research, either in an empirical study or in a more theoretical-methodologically oriented piece.

In addition, general textbooks on social research methods and methodology now increasingly devote sections to the issue of combining methods (e.g., Creswell 2008 ; Nagy Hesse-Biber and Leavy 2008 ; Bryman 2012 ), and MMR has been described as a “third paradigm” (Denscombe 2008 ), a “movement” (Bryman 2009 ), a “third methodology” (Tashakkori and Teddlie 2010b ), a “distinct approach” (Greene 2008 ) and an “emerging field” (Tashakkori and Teddlie 2011 ), defined by a common name (that sets it apart from other approaches to combining methods) and shared terminology (Tashakkori and Teddlie 2010b , p. 19). As a further indication of institutionalization, a research association (the Mixed Methods International Research Association—MMIRA) was founded in 2013 and its inaugural conference was held in 2014. Prior to this, there have been a number of conferences on MMR or occasions on which MMR was presented and discussed in other contexts. An example of the first is the conference on mixed method research design held in Basel in 2005. Starting also in 2005, the British Homerton School of Health Studies has organised a series of international conferences on mixed methods. Moreover, MMR was on the list of sessions in a number of conferences on qualitative research (see, e.g., Creswell 2012 ).

Another sign of institutionalization can be found in efforts to forge a common disciplinary identity by providing a narrative about its history. This involves the identification of precursors and pioneers as well as an interpretation of the process that gave rise to a distinctive set of ideas and practices. An explicit attempt to chart the early history of MMR is provided by Johnson and Gray ( 2010 ). They frame MMR as rooted in the philosophy of science, particularly as a way of thinking about science that has transcended some of the most salient historical oppositions in philosophy. Philosophers like Aristotle and Kant are portrayed as thinkers who sought to integrate opposing stances, forwarding “proto-mixed methods ideas” that exhibited the spirit of MMR (Johnson and Gray 2010 , p. 72, p. 86). In this capacity, they (as well as other philosophers like Vico and Montesquieu) are presented as part of MMR providing a philosophical validation of the project by presenting it as a continuation of ideas that have already been voiced by great thinkers in the past.

In the second edition of their textbook, Creswell and Plano Clark ( 2011 ) provide an overview of the history of MMR by identifying five historical stages: the first one being a precursor to the MMR approach, consisting of rather atomised attempts by different authors to combine methods in their research. For Creswell and Plano Clark, one of the earliest examples is Campbell and Fiske’s ( 1959 ) combination of quantitative methods to improve the validity of psychological scales that gave rise to the triangulation approach to research. However, they regard this and other studies that combined methods around that time, as “antecedents to (…) more systematic attempts to forge mixed methods into a complete research design” (Creswell and Plano Clark 2011 , p. 21), and hence label this stage as the “formative period” (ibid., p. 25). Their second stage consists of the emergence of MMR as an identifiable research strand, accompanied by a “paradigm debate” about the possibility of combining qualitative and quantitative data. They locate its beginnings in the late 1980s when researchers in various fields began to combine qualitative and quantitative methods (ibid., pp. 20–21). This provoked a discussion about the feasibility of combining data that were viewed as coming from very different philosophical points of view. The third stage, the “procedural development period,” saw an emphasis on developing more hands-on procedures for designing a mixed methods study, while stage four is identified as consisting of “advocacy and expansion” of MMR as a separate methodology, involving conferences, the establishment of a journal and the first edition of the aforementioned handbook (Tashakkori and Teddlie 2003 ). Finally, the fifth stage is seen as a “reflective period,” in which discussions about the unique philosophical underpinnings and the scientific position of MMR emerge.

Creswell and Plano Clark thus locate the emergence of “MMR proper” at the second stage, when researchers started to use both qualitative and quantitative methods within a single research effort. As reasons for the emergence of MMR at this stage they identify the growing complexity of research problems, the perception of qualitative research as a legitimate form of inquiry (also by quantitative researchers) and the increasing need qualitative researchers felt for generalising their findings. They therefore perceive the emergence of the practice of combining methods as a bottom up process that grew out of research practices, and at some point in time converged towards a more structural approach. Footnote 4 Historical accounts such as these add a cognitive dimension to the efforts to institutionalize MMR. They lay the groundwork for MMR as a separate subfield with its own identity, topics, problems and intellectual history. The use of terms such as “third paradigm” and “third methodology” also suggests that there is a tendency to perceive and promote MMR as a distinct and coherent way to do research.

In view of the brief exploration of the indicators of institutionalisation of MMR, it seems reasonable to conclude that MMR has become a recognizable and fairly institutionalized strand of research with its own identity and profile within the social scientific field. This can be seen both from the establishment of formal institutions (like associations and journals) and more informal ones that rely more on the tacit agreement between agents about “what MMR is” (an example of this, which we address later in the article, is the search for a common definition of MMR in order to fix the meaning of the term). The establishment of these institutions supports the autonomization of MMR and its emancipation from the field in which it originated, but in which it continues to be embedded. This way, it can be viewed as a semi-autonomous subfield within the larger field of the social sciences and as the result of a differentiation internal to this field (Steinmetz 2016 , p. 109). It is a space that is clearly embedded within this higher level field; for example, members of the subfield of MMR also qualify as members of the overarching field, and the allocation of the most valuable and current form of capital is determined there as well. Nevertheless, as a distinct subfield, it also has specific principles that govern the production of knowledge and the rewards of domination.

We return to the content and form of this specific knowledge later in the article. The next section addresses the question of the socio-genesis of MMR.

Where does mixed methods research come from?

The origins of the subfield of MMR lay in the broader field of social scientific disciplines. We interpret the positions of the scholars most involved in MMR (the “pioneers” or “scientific entrepreneurs”) as occupying particular positions within the larger academic and scientific field. Who, then, are the researchers at the heart of MMR? Leech ( 2010 ) interviewed 4 scholars (out of 6) that she identified as early developers of the field: Alan Bryman (UK; sociology), John Creswell (USA; educational psychology), Jennifer Greene (USA; educational psychology) and Janice Morse (USA; nursing and anthropology). Educated in the 1970s and early 1980s, all four of them indicated that they were initially trained in “quantitative methods” and later acquired skills in “qualitative methods.” For two of them (Bryman and Creswell) the impetus to learn qualitative methods was their involvement in writing on, and teaching of, research methods; for Greene and Morse the initial motivation was more instrumental and related to their concrete research activity at the time. Creswell describes himself as “a postpositivist in the 1970s, self-education as a constructivist through teaching qualitative courses in the 1980s, and advocacy for mixed methods (…) from the 1990s to the present” (Creswell 2011 , p. 269). Of this group, only Morse had the benefit of learning about qualitative methods as part of her educational training (in nursing and anthropology; Leech 2010 , p. 267). Independently, Creswell ( 2012 ) identified (in addition to Bryman, Greene and Morse) John Hunter, Allen Brewer (USA; Northwestern and Boston College) and Nigel Fielding (University of Surrey, UK) as important early movers in MMR.

The selections that Leech and Creswell make regarding the key actors are based on their close involvement with the “MMR movement.” It is corroborated by a simple analysis of the articles that appeared in the Journal of Mixed Methods Research ( JMMR ), founded in 2007 as an outlet for MMR.

Table 1 lists all the authors that have published in the issues of the journal since its first publication in 2007 and that have either received more than 14 (4%) of the citations allocated between the group of 343 authors (the TLCS score in Table 1 ), or have written more than 2 articles for the Journal (1.2% of all the articles that have appeared from 2007 until October 2013) together with their educational background (i.e., the discipline in which they completed their PhD).

All the members of Leech’s selection, except for Morse, and the members of Creswell’s selection (except Hunter, Brewer, and Fielding) are represented in the selection based on the entries in the JMMR . Footnote 5 The same holds for two of the three additional authors identified by Creswell. Hunter and Brewer have developed a somewhat different approach to combining methods that explicitly targets data gathering techniques and largely avoids epistemological discussions. In Brewer and Hunter ( 2006 ) they discuss the MMR approach very briefly and only include two references in their bibliography to the handbook of Tashakkori and Teddlie ( 2003 ), and at the end of 2013 they had not published in the JMMR . Fielding, meanwhile, has written two articles for the JMMR (Fielding and Cisneros-Puebla 2009 ; Fielding 2012 ). In general, it seems reasonable to assume that a publication in a journal that positions itself as part of a systematic attempt to build a research tradition, and can be viewed as part of a strategic effort to advance MMR as a distinct alternative to more “traditional” academic research—particularly in methods—at least signals a degree of adherence to the effort and acceptance of the rules of the game it lays out. This would locate Fielding closer to the MMR movement than the others.

The majority of the researchers listed in Table 1 have a background in psychology or social psychology (35%), and sociology (25%). Most of them work in the United States or are UK citizens, and the positions they occupied at the beginning of 2013 indicates that most of these are in applied research: educational research and educational psychology account for 50% of all the disciplinary occupations of the group that were still employed in academia. This is consistent with the view that MMR originated in applied disciplines and thematic studies like education and nursing, rather than “pure disciplines” like psychology and sociology (Tashakkori and Teddlie ( 2010b ), p. 32). Although most of the 20 individuals mentioned in Table 1 have taught methods courses in academic curricula (for 15 of them, we could determine that they were involved in the teaching of qualitative, quantitative, or mixed methods), there are few individuals with a background in statistics or a neighbouring discipline: only Amy Dellinger did her PhD in “research methodology.” In addition, as far as we could determine, only three individuals held a position in a methodological department at some time: Dellinger, Tony Onwuegbuzie, and Nancy Leech.

The pre-eminence of applied fields in MMR is supported when we turn our attention to the circulation of MMR. To assess this we proceeded as follows. We selected 10 categories in the Web of Science that form a rough representation of the space of social science disciplines, taking care to include the most important so-called “studies.” These thematically orientated, interdisciplinary research areas have progressively expanded since they emerged at the end of the 1960s as a critique of the traditional disciplines (Heilbron et al. 2017 ). For each category, we selected the 10 journals with the highest 5-year impact factor in their category in the period 2007–2015. The lists were compiled bi-annually over this period, resulting in 5 top ten lists for the following Web of Science categories: Economics, Psychology, Sociology, Anthropology, Political Science, Nursing, Education & Educational Research, Business, Cultural Studies, and Family Studies. After removing multiple occurring journals, we obtained a list of 164 journals.

We searched the titles and abstracts of the articles appearing in these journals over the period 1992–2016 for occurrences of the terms “mixed method” or “multiple methods” and variants thereof. We chose this particular period and combination of search terms to see if a shift from a more general use of the term “multiple methods” to “mixed methods” occurred following the institutionalization of MMR. In total, we found 797 articles (out of a total of 241,521 articles that appeared in these journals during that time), published in 95 different journals. Table 2 lists the 20 journals that contain at least 1% (8 articles) of the total amount of articles.

As is clear from Table 2 , the largest number of articles in the sample were published in journals in the field of nursing: 332 articles (42%) appeared in journals that can be assigned to this category. The next largest category is Education & Educational Research, to which 224 (28 percentage) of the articles can be allocated. By contrast, classical social science disciples are barely represented. In Table 2 only the journal Field Methods (Anthropology) and the Journal of Child Psychology and Psychiatry (Psychology) are related to classical disciplines. In Table 3 , the articles in the sample are categorized according to the disciplinary category of the journal in which they appeared. Overall, the traditional disciplines are clearly underrepresented: for the Economics category, for example, only the Journal of Economic Geography contains three articles that make a reference to mixed methods.

Focusing on the core MMR group, the top ten authors of the group together collect 458 citations from the 797 articles in the sample, locating them at the center of the citation network. Creswell is the most cited author (210 citations) and his work too receives most citations from journals in nursing and education studies.

The question whether a terminological shift has occurred from “multiple methods” to “mixed methods” must be answered affirmative for this sample. Prior to 2001 most articles (23 out of 31) refer to “multiple methods” or “multi-method” in their title or abstract, while the term “mixed methods” gains traction after 2001. This shift occurs first in journals in nursing studies, with journals in education studies following somewhat later. The same fields are also the first to cite the first textbooks and handbooks of MMR.

Taken together, these results corroborate the notion that MMR circulates mainly in nursing and education studies. How can this be understood from a field theoretical perspective? MMR can be seen as an innovation in the social scientific field, introducing a new methodology for combining existing methods in research. In general, innovation is a relatively risky strategy. Coming up with a truly rule-breaking innovation often involves a small probability of great success and a large probability of failure. However, it is important to add some nuance to this general observation. First, the risk an innovator faces depends on her position in the field. Agents occupying positions at the top of their field’s hierarchy are rich in specific capital and can more easily afford to undertake risky projects. In the scientific field, these are the agents richest in scientific capital. They have the knowledge, authority, and reputation (derived from recognition by their peers; Bourdieu 2004 , p. 34) that tends to decrease the risk they face and increase the chances of success. Moreover, the positions richest in scientific capital will, by definition, be the most consecrated ones. This consecration involves scientific rather than academic capital (cf. Wacquant 2013 , p. 20) and within disciplines these consecrated positions often are related to orthodox position-takings. This presents a paradox: although they have the capital to take more risks, they have also invested heavily in the orthodoxy of the field and will thus be reluctant to upset the status quo and risk destroying the value of their investment. This results in a tendency to take a more conservative stance, aimed at preserving the status quo in the field and defending their position. Footnote 6

For agents in dominated positions this logic is reversed. Possessing less scientific capital, they hold less consecrated positions and their chances of introducing successful innovations are much lower. This leaves them too with two possible strategies. One is to revert to a strategy of adaptation, accepting the established hierarchy in the field and embarking on a slow advancement to gain the necessary capital to make their mark from within the established order. However, Bourdieu notes that sometimes agents with a relatively marginal position in the field will engage in a “flight forward” and pursue higher risk strategies. Strategies promoting a heterodox approach challenge the orthodoxy and the principles of hierarchization of the field, and, if successful (which will be the case only with a small probability), can rake in significant profits by laying claim to a new orthodoxy (Bourdieu 1975 , p. 104; Bourdieu 1993 , pp. 116–117).

Thus, the coupling of innovative strategies to specific field positions based on the amount of scientific capital alone is not straightforward. It is therefore helpful to introduce a second differentiation in the field that, following Bourdieu ( 1975 , p. 103), is based on the differences between the expected profits from these strategies. Here a distinction can be made between an autonomous and a heteronomous pole of the field, i.e., between the purest, most “disinterested” positions and the most “temporal” positions that are more pervious to the heteronomous logic of social hierarchies outside the scientific field. Of course, this difference is a matter of degree, as even the works produced at the most heteronomous positions still have to adhere to the standards of the scientific field to be seen as legitimate. But within each discipline this dimension captures the difference between agents predominantly engaged in fundamental, scholarly work—“production solely for the producers”—and agents more involved in applied lines of research. The main component of the expected profit from innovation in the first case is scientific, whereas in the second case the balance tends to shift towards more temporal profits. This two-fold structuring of the field allows for a more nuanced conception of innovation than the dichotomy “conservative” versus “radical.” Holders of large amounts of scientific capital at the autonomous pole of the field are the producers and conservators of orthodoxy, producing and diffusing what can be called “orthodox innovations” through their control of relatively powerful networks of consecration and circulation. Innovations can be radical or revolutionary in a rational sense, but they tend to originate from questions raised by the orthodoxy of the field. Likewise, the strategy to innovate in this sense can be very risky in that success is in no way guaranteed, but the risk is mitigated by the assurance of peers that these are legitimate questions, tackled in a way that is consistent with orthodoxy and that does not threaten control of the consecration and circulation networks.

These producers are seen as intellectual leaders by most agents in the field, especially by those aspiring to become part of the specific networks of production and circulation they maintain. The exception are the agents located at the autonomous end of the field who possess less scientific capital and outright reject this orthodoxy produced by the field’s elite. Being strictly focused on the most autonomous principles of legitimacy, they are unable to accommodate and have no choice but to reject the orthodoxy. Their only hope is to engage in heterodox innovations that may one day become the new orthodoxy.

The issue is less antagonistic at the heteronomous side of the field, at least as far as the irreconcilable position-takings at the autonomous pole are concerned. The main battle here is also for scientific capital, but is complemented by the legitimacy it brings to gain access to those who are in power outside of the scientific field. At the dominant side, those with more scientific capital tend to have access to the field of power, agents who hold the most economic and cultural capital, for example by holding positions in policy advisory committees or company boards. The dominated groups at this side of the field will cater more to practitioners or professionals outside of the field of science.

Overall, there will be fewer innovations on this side. Moreover, innovative strategies will be less concerned with the intricacies of the pure discussions that prevail at the autonomous pole and be of a more practical nature, but pursued from different degrees of legitimacy according to the differences in scientific capital. This affects the form these more practical, process-orientated innovations take. At the dominant side of this pole, agents tend to accept the outcome of the struggles at the autonomous pole: they will accept the orthodoxy because mastery of this provides them with scientific capital and the legitimacy they need to gain access to those in power. In contrast, agents at the dominated side will be more interested in doing “what works,” neutralizing the points of conflict at the autonomous pole and deriving less value from strictly following the orthodoxy. This way, a four-fold classification of innovative strategies in the scientific field emerges (see Fig.  2 ) that helps to understand the context in which MMR was developed.

Scientific field and scientific innovation

In summary, the small group of researchers who have been identified as the core of MMR consist predominantly of users of methods, who were educated and have worked exclusively at US and British universities. The specific approach to combining methods that is proposed by MMR has been successful from an institutional point of view, achieving visibility through the foundation of a journal and association and a considerable output of core MMR scholars in terms of books, conference proceedings, and journal articles. Its origins and circulation in vocational studies rather than classical academic disciplines can be understood from the position these studies occupy in the scientific field and the kinds of position-taking and innovations these positions give rise to. This context allows a reflexive understanding of the content of MMR and the issues that are dominant in the approach. We turn to this in the next section.

Mixed methods research: Position-taking

The position of the subfield of MMR in the scientific field is related to the position-takings of agents that form the core of this subfield (Bourdieu 1993 , p. 35). The space of position takings, in turn, provides the framework to study the most salient issues that are debated within the subfield. Since we can consider MMR to be an emerging subfield, where positions and position takings are not as clearly defined as in more mature and settled fields, it comes as no surprise that there is a lively discussion of fundamental matters. Out of the various topics that are actively discussed, we have distilled three themes that are important for the way the subfield of MMR conveys its autonomy as a field and as a distinct approach to research. Footnote 7 In our view, these also represent the main problems with the way MMR approaches the issue of combining methods.

Methodology making and standardization

The first topic is that the approach is moving towards defining a unified MMR methodology. There are differences in opinion as to how this is best achieved, but there is widespread agreement that some kind of common methodological and conceptual foundation of MMR is needed. To this end, some propose a broad methodology that can serve as distinct marker of MMR research. For instance, in their introduction to the handbook, Tashakkori and Teddlie ( 2010b ) propose a definition of the methodology of mixed methods research as “the broad inquiry logic that guides the selection of specific methods and that is informed by conceptual positions common to mixed methods practitioners” (Tashakkori and Teddlie 2010b , p. 5). When they (later on in the text) provide two methodological principles that differentiate MMR from other communities of scholars, they state that they regard it as a “crucial mission” for the MMR community to generate distinct methodological principles (Tashakkori and Teddlie 2010b , pp. 16–17). They envision an MMR methodology that can function as a “guide” for selecting specific methods. Others are more in favour of finding a philosophical foundation that underlies MMR. For instance, Morgan ( 2007 ) and Hesse-Biber ( 2010 ) consider pragmatism as a philosophy that distinguishes MMR from qualitative (constructivism) and quantitative (positivist) research and that can provide a rationale for the paradigmatic pluralism typical of MMR.

Furthermore, there is wide agreement that some unified definition of MMR would be beneficial, but it is precisely here that there is a large variation in interpretations regarding the essentials of MMR. This can be seen in the plethora of definitions that have been proposed. Johnson et al. ( 2007 ) identified 19 alternative definitions of MMR at the time, out of which they condensed their own:

[MMR] is the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purpose of breath and depth of understanding and corroboration. Footnote 8

Four years later, the issue is not settled yet. Creswell and Plano Clark ( 2011 ) list a number of authors who have proposed a different definition of MMR, and conclude that there is a common trend in the content of these definitions over time. They take the view that earlier texts on mixing methods stressed a “disentanglement of methods and philosophy,” while later texts locate the practice of mixing methods in “all phases of the research process” (Creswell and Plano Clark 2011 , p. 2). It would seem, then, that according to these authors the definitions of MMR have become more abstract, further away from the practicality of “merely” combining methods. Specifically, researchers now seem to speak of mixing higher order concepts: some speak of mixing methodologies, others refer to mixing “research approaches,” or combining “types of research,” or engage in “multiple ways of seeing the social world” (Creswell and Plano Clark 2011 ).

This shift is in line with the direction in which MMR has developed and that emphasises practical ‘manuals’ and schemas for conducting research. A relatively large portion of the MMR literature is devoted to classifications of mixed methods designs. These classifications provide the basis for typologies that, in turn, provide guidelines to conduct MMR in a concrete research project. Tashakkori and Teddlie ( 2003 ) view these typologies as important elements of the organizational structure and legitimacy of the field. In addition, Leech and Onwuegbuzie ( 2009 ) see typologies as helpful guides for researchers and of pedagogical value (Leech and Onwuegbuzie 2009 , p. 272). Proposals for typologies can be found in textbooks, articles, and contributions to the handbook(s). For example, Creswell et al. ( 2003 , pp. 169-170) reviewed a number of studies and identified 8 different ways to classify MMR studies. This list was updated and extended by Creswell and Plano Clark ( 2011 , pp. 56-59) to 15 typologies. Leech and Onwuegbuzie ( 2009 ) identified 35 different research designs in the contributions to Teddlie and Tashakkori (2003) alone, and proposed their own three-dimensional typology that resulted in 8 different types of mixed methods studies. As another example of the ubiquity of these typologies, Nastasi et al. ( 2010 ) classified a large number of existing typologies in MMR into 7”meta-typologies” that each emphasize different aspects of the research process as important markers for MMR. According to the authors, these typologies have the same function in MMR as the more familiar names of “qualitative” or “quantitative” methods (e.g., “content analysis” or “structural equation modelling”) have: to signal readers of research what is going on, what procedures have been followed, how to interpret results, etc. (see also Creswell et al. 2003 , pp. 162–163). The criteria underlying these typologies mainly have to do with the degree of mixing (e.g., are methods mixed throughout the research project or not?), the timing (e.g., sequential or concurrent mixing of methods) and the emphasis (e.g., is one approach dominant, or do they have equal status?).

We find this strong drive to develop methodologies, definitions, and typologies of MMR as guides to valid mixed methods research problematic. What it amounts to in practice is a methodology that lays out the basic guidelines for doing MMR in a “proper way.” This entails the danger of straight-jacketing reflection about the use of methods, decoupling it from theoretical and empirical considerations, thus favouring the unreflexive use of a standard methodology. Researchers are asked to make a choice for a particular MMR design and adhere to the guidelines for a “proper” MMR study. Such methodological prescription diametrically opposes the initial critique of the mechanical and unreflexive use of methods. The insight offered by Bourdieu’s notion of reflexivity is, on the contrary, that the actual research practice is fundamentally open in terms of being guided by a logic of practice that cannot be captured by a preconceived and all-encompassing logic independent of that practice. Reflexivity in this view cannot be achieved by hiding behind the construct of a standardized methodology—of whatever signature—it can only be achieved by objectifying the process of objectification that goes on within the context of the field in which the researcher is embedded. This reflexivity, then, requires an analysis of the position of the researcher as a critical component of the research process, both as the embodiment of past choices that have consequences for the strategic position in the scientific field, and as predispositions regarding the choice for the subject and content of a research project. By adding the insight of STS researchers that the point of deconstructing science and technology is not so much to offer a new best way of doing science or technology, but to provide insights into the critical moments in research (for a take on such a debate, see, for example, Edge 1995 , pp. 16–20), this calls for a sociology of science that takes methods much more seriously as objects of study. Such a programme should be based on studying the process of codification and standardization of methods in their historical context of production, circulation, and use. It would provide a basis for a sociological understanding of methods that can illuminate the critical moments in research alluded to above, enabling a systematic reflection on the process of objectification. This, in turn, allows a more sophisticated validation of using—and combining—methods than relying on prescribed methodologies.

The role of epistemology

The second theme discussed in a large number of contributions is the role epistemology plays in MMR. In a sense, epistemology provides the lifeblood for MMR in that methods in MMR are mainly seen in epistemological terms. This interpretation of methods is at the core of the knowledge claim of MMR practitioners, i.e., that the mixing of methods means mixing broad, different ways of knowing, which leads to better knowledge of the research object. It is also part of the identity that MMR consciously assumes, and that serves to set it apart from previous, more practical attempts to combine methods. This can be seen in the historical overview that Creswell and Plano Clark ( 2011 ) presented and that was discussed above. This reading, in which combining methods has evolved from the rather unproblematic level (one could alternatively say “naïve” or “unaware”) of instrumental use of various tools and techniques into an act that requires deeper thinking on a methodological and epistemological level, provides the legitimacy of MMR.

At the core of the MMR approach we thus find that methods are seen as unproblematic representations of different epistemologies. But this leads to a paradox, since the epistemological frameworks need to be held flexible enough to allow researchers to integrate elements of each of them (in the shape of methods) into one MMR design. As a consequence, the issue becomes the following: methods need to be disengaged from too strict an interpretation of the epistemological context in which they were developed in order for them to be “mixable,”’, but, at the same time, they must keep the epistemology attributed to them firmly intact.

In the MMR discourse two epistemological positions are identified that matter most: a positivist approach that gives rise to quantitative methods and a constructivist approach that is home to qualitative methods. For MMR to be a feasible endeavour, the differences between both forms of research must be defined as reconcilable. This position necessitates an engagement with those who hold that the quantitative/qualitative dichotomy is unbridgeable. Within MMR an interesting way of doing so has emerged. In the first issue of the Journal of Mixed Methods Research, Morgan ( 2007 ) frames the debate about research methodology in the social sciences in terms of Kuhnian paradigms, and he argues that the pioneers of the emancipation of qualitative research methods used a particular interpretation of the paradigm-concept to state their case against the then dominant paradigm in the social sciences. According to Morgan, they interpreted a paradigm mainly in metaphysical terms, stressing the connections among the trinity of ontology, epistemology, and methodology as used in the philosophy of knowledge (Morgan 2007 , p. 57). This allowed these scholars to depict the line between research traditions in stark, contrasting terms, using Kuhn’s idea of “incommensurability” in the sense of its “early Kuhn” interpretation. This strategy fixed the contrast between the proposed alternative approach (a “constructivist paradigm”), and the traditional approach (constructed as “the positivist paradigm”) to research as a whole, and offered the alternative approach as a valid option rooted in the philosophy of knowledge. Morgan focuses especially on the work of Egon Guba and Yvonne Lincoln who developed what they initially termed a “naturalistic paradigm” as an alternative to their perception of positivism in the social sciences (e.g., Guba and Lincoln 1985 ). Footnote 9 MMR requires a more flexible or “a-paradigmatic stance” towards research, which would entail that “in real-world practice, methods can be separated from the epistemology out of which they emerged” (Patton 2002 , quoted in Tashakkori and Teddlie 2010b , p. 14).

This proposal of an ‘interpretative flexibility’ (Bijker 1987 , 1997 ) regarding paradigms is an interesting proposition. But it immediately raises the question: why stop there? Why not take a deeper look into the epistemological technology of methods themselves, to let the muted components speak up in order to look for alternative “mixing interfaces” that could potentially provide equally valid benefits in terms of the understanding of a research object? The answer, of course, was already seen above. It is that the MMR approach requires situating methods epistemologically in order to keep them intact as unproblematic mediators of specific epistemologies and, thus, make the methodological prescriptions work. There are several problems with this. First, seeing methods solely through an epistemological lens is problematic, but it would be less consequential if it were applied to multiple elements of methods separately. This would at least allow a look under the hood of a method, and new ways of mixing methods could be opened up that go beyond the crude “qualitative” versus “quantitative” dichotomy. Second, there is also the issue of the ontological dimension of methods that is disregarded in an exclusively epistemological framing of methods (e.g., Law 2004 ). Taking this ontological dimension seriously has at least two important facets. First, it draws attention to the ontological assumptions that are woven into methods in their respective fields of production and that are imported into fields of users. Second, it entails the ontological consequences of practising methods: using, applying, and referring to methods and the realities this produces. This latter facet brings the world-making and boundary-drawing capacities of methods to the fore. Both facets are ignored in MMR. We say more about the first facet in the next section. With regard to the second facet, a crucial element concerns the data that are generated, collected, and analysed in a research project. But rather than problematizing the link between the performativity of methods and the data that are enacted within the frame of a method, here too MMR relies on a dichotomy: that between quantitative and qualitative data. Methods are primarily viewed as ways of gathering data or as analytic techniques dealing with a specific kind of data. Methods and data are conceptualised intertwiningly: methods too are seen as either quantitative or qualitative (often written as QUANT and QUAL in the literature), and perform the role of linking epistemology and data. In the final analysis, the MMR approach is based on the epistemological legitimization of the dichotomy between qualitative and quantitative data in order to define and combine methods: data obtain epistemological currency through the supposed in-severable link to certain methods, and methods are reduced to the role of acting as neutral mediators between them.

In this way, methods are effectively reduced to, on the one hand, placeholders for epistemological paradigms and, on the other hand, mediators between one kind of data and the appropriate epistemology. To put it bluntly, the name “mixed methods research” is actually a misnomer, because what is mixed are paradigms or “approaches,” not methods. Thus, the act of mixing methods à la MMR has the paradoxical effect of encouraging a crude black box approach to methods. This is a third problematic characteristic of MMR, because it hinders a detailed study of methods that can lead to a much richer perspective on mixing methods.

Black boxed methods and how to open them

The third problem that we identified with the MMR approach, then, is that with the impetus to standardize the MMR methodology by fixing methods epistemologically, complemented by a dichotomous view of data, they are, in the words of philosopher Bruno Latour, “blackboxed.” This is a peculiar result of the prescription for mixing methods as proposed by MMR that thus not only denies practice and the ontological dimensions of methods and data, but also casts methods in the role of unyielding black boxes. Footnote 10 With this in mind, it will come as no surprise that most foundational contributions to the MMR literature do not explicitly define what a method is, nor that they do not provide an elaborative historical account of individual methods. The particular framing of methods in MMR results in a blind spot for the historical and social context of the production and circulation of methods as intellectual products. Instead it chooses to reify the boundaries that are drawn between “qualitative” and “quantitative” methods and reproduce them in the methodology it proposes. Footnote 11 This is an example of “circulation without context” (Bourdieu 2002 , p. 4): classifications that are constructed in the field of use or reception without taking the constellation within the field of production seriously.

Of course, this does not mean that the reality of the differences between quantitative and qualitative research must be denied. These labels are sticky and symbolically laden. They have come, in many ways, to represent “two cultures” (Goertz and Mahony 2012 ) of research, institutionalised in academia, and the effects of nominally “belonging” to (or being assigned to) one particular category have very real consequences in terms of, for instance, access to research grants and specific journals. However, if the goal of an approach such as MMR is to open up new pathways in social science research, (and why should that not be the case?) it is hard to see how that is accomplished by defining the act of combining methods solely in terms of reified differences between research using qualitative and quantitative data. In our view, methods are far richer and more interesting constructs than that, and a practice of combining methods in research should reflect that. Footnote 12

Addressing these problems entices a reflection on methods and using (multiple) methods that is missing in the MMR perspective. A fruitful way to open up the black boxes and take into account the epistemological and ontological facets of methods is to make them, and their use, the object of sociological-historical investigation. Methods are constituted through particular practices. In Bourdieusian terms, they are objectifications of the subjectively understood practices of scientists “in other fields.” Rather than basing a practice of combining methods on an uncritical acceptance of the historically grown classification of types of social research (and using these as the building stones of a methodology of mixing methods), we propose the development of a multifaceted approach that is based on a study of the different socio-historical contexts and practices in which methods developed and circulated.

A sociological understanding of methods based on these premises provides the tools to break with the dichotomously designed interface for combining methods in MMR. Instead, focusing on the historical and social contexts of production and use can reveal the traces that these contexts leave, both in the internal structure of methods, how they are perceived, how they are put into practice, and how this practice informs the ontological effects of methods. Seeing methods as complex technologies, with a history that entails the struggles among the different agents involved in their production, and use opens the way to identify multiple interfaces for combining them: the one-sided boxes become polyhedra. The critical study of methods as “objects of objectification” also entices analyses of the way in which methods intervene between subject (researcher) and object and the way in which different methods are employed in practice to draw this boundary differently. The reflexive position generated by such a systematic juxtaposition of methods is a fruitful basis to come to a richer perspective on combining methods.

We critically reviewed the emerging practice of combining methods under the label of MMR. MMR challenges the mono-method approaches that are still dominant in the social sciences, and this is both refreshing and important. Combining methods should indeed be taken much more seriously in the social sciences.

However, the direction that the practice of combining methods is taking under the MMR approach seems problematic to us. We identified three main concerns. First, MMR scholars seem to be committed to designing a standardized methodological framework for combining methods. This is unfortunate, since it amounts to enforcing an unnecessary codification of aspects of research practices that should not be formally standardized. Second, MMR constructs methods as unproblematic representations of an epistemology. Although methods must be separable from their native epistemology for MMR to work, at the same time they have to be nested within a qualitative or a quantitative research approach, which are characterized by the data they use. By this logic, combining quantitative methods with other quantitative methods, or qualitative methods with other qualitative methods, cannot offer the same benefits: they originate from the same way of viewing and knowing the world, so it would have the same effect as blending two gradations of the same colour paint. The importance attached to the epistemological grounding of methods and data in MMR also disregards the ontological aspects of methods. In this article, we are arguing that this one-sided perspective is problematic. Seeing combining methods as equivalent to combining epistemologies that are somehow pure and internally homogeneous because they can be placed in a qualitative or quantitative framework essentially amounts to reifying these categories.

It also leads to the third problem: the black boxing of methods as neutral mediators between these epistemologies and data. This not only constitutes a problem for trying to understand methods as intellectual products, but also for regarding the practice of combining methods, because it ignores the social-historical context of the development of individual methods and hinders a sociologically grounded notion of combining methods.

We proceed from a different perspective on methods. In our view, methods are complex constructions. They are world-making technologies that encapsulate different assumptions on causality, rely on different conceptual relations and categorizations, allow for different degrees of emergence, and employ different theories of the data that they internalise as objects of analysis. Even more importantly, their current form as intellectual products cannot be separated from the historical context of their production, circulation, and use.

A fully developed exposition of such an approach will have to await further work. Footnote 13 So far, the sociological study of methods has not (yet) developed into a consistent research programme, but important elements can be derived from existing contributions such as MacKenzie ( 1981 ), Chapoulie ( 1984 ), Platt ( 1996 ), Freeman ( 2004 ), and Desrosières ( 2008a , b ). The work on the “social life of methods” (e.g., Savage 2013 ) also contains important leads for the development of a systematic sociological approach to method production and circulation. Based on the discussion in this article and the contributions listed above, some tantalizing questions can be formulated. How are methods and their elements objectified? How are epistemology and ontology defined in different fields and how do those definitions feed into methods? How do they circulate and how are they translated and used in different contexts? What are the main controversies in fields of users and how are these related to the field of production? What are the homologies between these fields?

Setting out to answer these questions opens up the possibility of exploring other interesting combinations of methods that emerge from the combination of different practices, situated in different historical and epistemological contexts, and with their unique set of interpretations regarding their constituent elements. One of these must surely be the data-theoretical elements that different methods incorporate. The problematization of data has become all the more pressing now that the debate about the consequences of “big data” for social scientific practices has become prominent (Savage and Burrows 2007 ; Levallois et al. 2013 ; Burrows and Savage 2014 ). Whereas MMR emphasizes the dichotomy between qualitative and quantitative data, a historical analysis of the production and use of methods can explore the more subtle, different interpretations and enactments of the “same” data. These differences inform method construction, controversies surrounding methods and, hence, opportunities for combining methods. These could then be constructed based on alternative conceptualisations of data. Again, while in some contexts it might be enlightening to rely on the distinction between data as qualitative or quantitative, and to combine methods based on this categorization, it is an exciting possibility that in other research contexts other conceptualisations of data might be of more value to enhance a specific (contextual) form of knowledge.

Change history

06 may 2019.

Unfortunately, figure 2 was incorrectly published.

The search term used was “mixed method*” in the “topic” search field of SSCI, A&HCI, and CPCI-SSH as contained in the Web of Science. A Google NGram search (not shown) confirmed this pattern. The results of a search for “mixed methods” and “mixed methods research” showed a very steep increase after 1994: in the first case, the normalized share in the total corpus increased by 855% from 1994 till 2008. Also, Creswell ( 2012 ) reports an almost hundred-fold increase in the number of theses and dissertations with mixed methods’ in the citation and abstract (from 26 in 1990–1994 to 2524 in 2005–2009).

Retrieved from https://uk.sagepub.com/en-gb/eur/journal-of-mixed-methods-research/journal201775#aims-and-scope on 1/17/2019.

In terms of antecedents of mixed methods research, it is interesting to note that Bourdieu, whose sociology of science we draw on, was, from his earliest studies in Algeria onwards, a strong advocate of combining research methods. He made it into a central characteristic of his approach to social science in Bourdieu et al. ( 1991 [1968]). His approach, as we see below, was very different from the one now proposed under the banner of MMR. Significantly, there is no mention of Bourdieu’s take on combining methods in any of the sources we studied.

Morse’s example in particular warns us that restricting the analysis to the authors that have published in the JMMR runs the risk of missing some important contributors to the spread of MMR through the social sciences. On her website, Morse lists 11 publications (journal articles, book chapters, and books) that explicitly make reference to mixed methods (and a substantial number of other publications are about methodological aspects of research), so the fact that she has not (yet) published in the JMMR cannot, by itself, be taken as an indication of a lesser involvement with the practice of combining methods. See the website of Janice Morse at https://faculty.utah.edu/u0556920-Janice_Morse_RN,_PhD,_FAAN/hm/index.hml accessed 1/17/2019.

Bourdieu ( 1999 , p. 26) mentions that one has to be a scientific capitalist to be able to start a scientific revolution. But here he refers explicitly to the autonomy of the scientific field, making it virtually impossible for amateurs to stand up against the historically accumulated capital in the field and incite a revolution.

The themes summarize the key issues through which MMR as a group comes “into difference” (Bourdieu 1993 , p. 32). Of course, as in any (sub)field, the agents identified above often differ in their opinions on some of these key issues or disagree on the answer to the question if there should be a high degree of convergence of opinions at all. For instance, Bryman ( 2009 ) worried that MMR could become “a ghetto.” For him, the institutional landmarks of having a journal, conferences, and a handbook increase the risk of “not considering the whole range of possibilities.” He added: “I don’t regard it as a field, I kind of think of it as a way of thinking about how you go about research.” (Bryman, cited in Leech 2010 , p. 261). It is interesting to note that Bryman, like fellow sociologists Morgan and Denscombe, had published only one paper in the JMMR by the end of 2016 (Bryman passed away in June of 2017). Although these papers are among the most cited papers in the journal (see Table 1 ), this low number is consistent with the more eclectic approach that Bryman proposed.

Johnson, Onwuegbuzie, and Turner ( 2007 , p. 123).

Guba and Lincoln ( 1985 ) discuss the features of their version of a positivistic approach mainly in ontological and epistemological terms, but they are also careful to distinguish the opposition between naturalistic and positivist approaches from the difference between what they call the quantitative and the qualitative paradigms. Since they go on to state that, in principle, quantitative methods can be used within a naturalistic approach (although in practice, qualitative methods would be preferred by researchers embracing this paradigm), they seem to locate methods on a somewhat “lower,” i.e., less incommensurable level. However, in their later work (both together as well as with others or individually) and that of others in their wake, there seems to have been a shift towards a stricter interpretation of the qualitative/quantitative divide in metaphysical terms, enabling Teddlie and Tashakkori (2010b) to label this group “purists” (Tashakkori and Teddlie 2010b , p. 13).

See, for instance, Onwuegbuzie et al.’s ( 2011 ) classification of 58 qualitative data analysis techniques and 18 quantitative data analysis techniques.

This can also be seen in Morgan’s ( 2018 ) response to Sandelowski’s ( 2014 ) critique of the binary distinctions in MMR between qualitative and quantitative research approaches and methods. Morgan denounces the essentialist approach to categorizing qualitative and quantitative research in favor of a categorization based on “family resemblances,” in which he draws on Wittgenstein. However, this denies the fact that the essentialist way of categorizing is very common in the MMR corpus, particularly in textbooks and manuals (e.g., Plano Clark and Ivankova 2016 ). Moreover, and more importantly, he still does not extend this non-essentialist model of categorization to the level of methods, referring, for instance, to the different strengths of qualitative and quantitative methods in mixed methods studies (Morgan 2018 , p. 276).

While it goes beyond the scope of this article to delve into the history of the qualitative-quantitative divide in the social sciences, some broad observations can be made here. The history of method use in the social sciences can briefly be summarized as first, a rather fluid use of what can retrospectively be called different methods in large scale research projects—such as the Yankee City study of Lloyd Warner and his associates (see Platt 1996 , p. 102), the study on union democracy of Lipset et al. ( 1956 ), and the Marienthal study by Lazarsfeld and his associates (Jahoda et al. 1933 ); see Brewer and Hunter ( 2006 , p. xvi)—followed by an increasing emphasis on quantitative data and the objectification and standardization of methods. The rise of research using qualitative data can be understood as a reaction against this use and interpretation of method in the social sciences. However, out of the ensuing clash a new, still dominant classification of methods emerged, one that relies on the framing of methods as either “qualitative” or “quantitative.” Moreover, these labels have become synonymous with epistemological positions that are reproduced in MMR.

A proposal to come to such an approach can be found in Timans ( 2015 ).

Bijker, W. (1987). The social construction of bakelite: Toward a theory of invention. In W. Bijker, T. Hughes, T. Pinch, & D. Douglas (Eds.), The social construction of technological systems: New directions in the sociology and history of technology . Cambridge, MA: MIT press.

Google Scholar  

Bijker, W. (1997). Of bicycles, bakelites, and bulbs: Toward a theory of sociotechnical change . Cambridge, MA: MIT Press.

Bourdieu, P. (1975). La spécifité du champ scientifique et les conditions sociales du progrès de la raison. Sociologie et Sociétés, 7 (1), 91–118.

Article   Google Scholar  

Bourdieu, P. (1988). Homo academicus . Stanford, CA: Stanford University Press.

Bourdieu, P. (1993). The field of cultural production . Cambridge, UK: Polity Press.

Bourdieu, P. (1999). Les historiens et la sociologie de Pierre Bourdieu. Le Bulletin de la Société d'Histoire Moderne et Contemporaine/SHMC, 1999 (3&4), 4–27.

Bourdieu, P. (2002). Les conditions sociales de la circulation internationale des idées. Actes de la Recherche en Sciences Sociales, 145 (5), 3–8.

Bourdieu, P. (2004). Science of science and reflexivity . Cambridge, UK: Polity.

Bourdieu, P. (2007). Sketch for a self-analysis . Cambridge, UK: Polity.

Bourdieu, P., Chamboredon, J., & Passeron, J. (1991). The craft of sociology: Epistemological preliminaries . Berlin, Germany: De Gruyter.

Book   Google Scholar  

Brewer, J., & Hunter, A. (2006). Multimethod research: A synthesis of styles . London, UK: Sage.

Bryman, A. (2009). Sage Methodspace: Alan Bryman on research methods. Retrieved from http://www.youtube.com/watch?v=bHzM9RlO6j0 . Accessed 3/7/2019.

Bryman, A. (2012). Social research methods . Oxford, UK: Oxford University Press.

Burrows, R., & Savage, M. (2014). After the crisis? Big data and the methodological challenges of empirical sociology. Big Data & Society, 1 (1), 1–6.

Campbell, D., & Fiske, D. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56 (2), 81–105.

Chapoulie, J. (1984). Everett C. Hughes et le développement du travail de terrain en sociologie. Revue Française de Sociologie, 25 (4), 582–608.

Collins, R. (1998). The sociology of philosophies: A global theory of intellectual change . Cambridge, MA: Harvard University Press.

Creswell, J. (2008). Research design: Qualitative, quantitative, and mixed methods approaches . Thousand Oaks, CA: Sage.

Creswell, J. (2011). Controversies in mixed methods research. In N. Denzin & Y. Lincoln (Eds.), The Sage handbook of qualitative research . Thousand Oaks, CA: Sage.

Creswell, J. (2012). Qualitative inquiry and research design: Choosing among five approaches . Thousand Oaks, CA: Sage.

Creswell, J., & Plano Clark, V. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Creswell, J., Plano Clark, V., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research . Thousand Oaks, CA: Sage.

Denscombe, M. (2008). Communities of practice a research paradigm for the mixed methods approach. Journal of Mixed Methods Research, 2 (3), 270–283.

Desrosières, A. (2008a). Pour une sociologie historique de la quantification - L’Argument statistique I . Paris, France: Presses des Mines.

Desrosières, A. (2008b). Gouverner par les nombres - L’Argument statistique II . Paris, France: Presses des Mines.

Edge, D. (1995). Reinventing the wheel. In D. Edge, S. Jasanof, G. Markle, J. Petersen, & T. Pinch (Eds.), Handbook of science and technology studies . Thousand Oaks, CA: Sage.

Fielding, N. (2012). Triangulation and mixed methods designs data integration with new research technologies. Journal of Mixed Methods Research, 6 (2), 124–136.

Fielding, N., & Cisneros-Puebla, C. (2009). CAQDAS-GIS convergence: Toward a new integrated mixed method research practice? Journal of Mixed Methods Research, 3 (4), 349–370.

Fleck, C., Heilbron, J., Karady, V., & Sapiro, G. (2016). Handbook of indicators of institutionalization of academic disciplines in SSH. Serendipities, Journal for the Sociology and History of the Social Sciences, 1 (1) Retrieved from http://serendipities.uni-graz.at/index.php/serendipities/issue/view/1 . Accessed 10/10/2018.

Freeman, L. (2004). The development of social network analysis: A study in the sociology of science . Vancouver, Canada: Empirical Press.

Gingras, Y., & Gemme, B. (2006). L’Emprise du champ scientifique sur le champ universitaire et ses effets. Actes de la Recherche en Sciences Sociales, 164 , 51–60.

Goertz, G., & Mahony, J. (2012). A tale of two cultures: Qualitative and quantitative research in the social sciences . Princeton: Princeton University Press.

Greene, J. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 2 (1), 7–22.

Guba, E., & Lincoln, Y. (1985). Naturalistic inquiry . Thousand Oaks, CA: Sage.

Heilbron, J., Bedecarré, M., & Timans, R. (2017). European journals in the social sciences and humanities. Serendipities, Journal for the Sociology and History of the Social Sciences, 2 (1), 33–49 Retrieved from http://serendipities.uni-graz.at/index.php/serendipities/issue/view/5 . Accessed 10/10/2018.

Hesse-Biber, S. (2010). Mixed methods research: Merging theory with practice . New York, NY: Guilford Press.

Jahoda, M., Lazarsfeld, P., & Zeisel, H. (1933). Die Arbeitslosen von Marienthal. Psychologische Monographen, 5 .

Johnson, R., & Gray, R. (2010). A history of philosophical and theoretical issues for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Johnson, R., Onwuegbuzie, A., & Turner, L. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1 (2), 112–133.

Law, J. (2004). After method: Mess in social science research . London, UK: Routledge.

Leech, N. (2010). Interviews with the early developers of mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Leech, N., & Onwuegbuzie, T. (2009). A typology of mixed methods research designs. Quality & Quantity, 43 (2), 265–275.

Levallois, C., Steinmetz, S., & Wouters, P. (2013). Sloppy data floods or precise social science methodologies? In P. Wouters, A. Beaulieu, A. Scharnhorst, & S. Wyatt (Eds.), Virtual knowledge . Cambridge, MA: MIT Press.

Lipset, S., Trow, M., & Coleman, J. (1956). Union democracy: The internal politics of the international typographical union . Glencoe, UK: Free Press.

MacKenzie, D. (1981). Statistics in Britain: 1865–1930: The social construction of scientific knowledge . Edinburgh, UK: Edinburgh University Press.

Morgan, D. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1 (1), 48–76.

Morgan, D. (2018). Living with blurry boundaries: The values of distinguishing between qualitative and quantitative research. Journal of Mixed Methods Research, 12 (3), 268–276.

Mullins, N. (1973). Theories and theory groups in contemporary American sociology . New York, NY: Harper & Row.

Nagy Hesse-Biber, S., & Leavy, P. (Eds.). (2008). Handbook of emergent methods . New York, NY and London, UK: Guilford Press.

Nastasi, B., Hitchcock, J., & Brown, L. (2010). An inclusive framework for conceptualizing mixed method design typologies. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Onwuegbuzie, A., Leech, N., & Collins, K. (2011). Toward a new era for conducting mixed analyses: The role of quantitative dominant and qualitative dominant crossover mixed analyses. In M. Williams & P. Vogt (Eds.), Handbook of innovation in social research methods . London, UK: Sage.

Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.

Plano Clark, V., & Creswell, J. (Eds.). (2007). The mixed methods reader . Thousand Oaks, CA: Sage.

Plano Clark, V., & Ivankova, N. (2016). Mixed methods research: A guide to the field . Thousand Oaks, CA: Sage.

Platt, J. (1996). A history of sociological research methods in America: 1920–1960 . Cambridge, UK: Cambridge University Press.

Plowright, D. (2010). Using mixed methods – Frameworks for an integrated methodology . Thousand Oaks, CA: Sage.

Sandelowski, M. (2014). Unmixing mixed methods research. Research in Nursing & Health, 37 (1), 3–8.

Sapiro, G., Brun, E., & Fordant, C. (2018). The rise of the social sciences and humanities in France: Iinstitutionalization, professionalization and autonomization. In C. Fleck, M. Duller, & V. Karady (Eds.), Shaping human science disciplines: Institutional developments in Europe and beyond . Basingstoke, UK: Palgrave.

Savage, M. (2013). The ‘social life of methods’: A critical introduction. Theory, Culture and Society, 30 (4), 3–21.

Savage, M., & Burrows, R. (2007). The coming crisis of empirical sociology. Sociology, 41 (5), 885–899.

Steinmetz, G. (2016). Social fields, subfields and social spaces at the scale of empires: Explaining the colonial state and colonial sociology. The Sociological Review, 64(2_suppl). 98-123.

Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social and behavioral research . Thousand Oaks, CA: Sage.

Tashakkori, A., & Teddlie, C. (Eds.). (2010a). Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Tashakkori, A., & Teddlie, C. (2010b). Overview of contemporary issues in mixed methods. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Chapter   Google Scholar  

Tashakkori, A., & Teddlie, C. (2010c). Epilogue. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.

Tashakkori, A., & Teddlie, C. (2011). Mixed methods research: Contemporary issues in an emerging field. In N. Denzin & Y. Lincoln (Eds.), The SAGE handbook of qualitative research (4th ed.). Thousand Oaks, CA: Sage.

Teddlie, C., & Tashakkori, A. (2008). Foundations of mixed methods research – Integrating quantitative and qualitative approaches in the social and behavioral sciences . Thousand Oaks, CA: Sage.

Timans, R. (2015). Studying the Dutch business elite: Relational concepts and methods . Doctoral dissertation, Erasmus University Rotterdam, the Netherlands.

Wacquant, L. (2013). Bourdieu 1993: A case study in scientific consecration. Sociology, 47 (1), 15–29.

Download references

Acknowledgments

This research is part of the Interco-SSH project, funded by the European Union under the 7th Research Framework Programme (grant agreement no. 319974). Johan Heilbron would like to thank Louise and John Steffens, members of the Friends Founders’ Circle, who assisted his stay at the Princeton Institute for Advanced Study in 2017-18 during which he completed his part of the present article.

Author information

Authors and affiliations.

Erasmus Centre for Economic Sociology (ECES), Erasmus University Rotterdam, Rotterdam, Netherlands

Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, Netherlands

Paul Wouters

Erasmus Centre for Economic Sociology (ECES), Rotterdam and Centre européen de sociologie et de science politique de la Sorbonne (CESSP-CNRS-EHESS), Paris, France

Johan Heilbron

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rob Timans .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Timans, R., Wouters, P. & Heilbron, J. Mixed methods research: what it is and what it could be. Theor Soc 48 , 193–216 (2019). https://doi.org/10.1007/s11186-019-09345-5

Download citation

Published : 29 March 2019

Issue Date : 01 April 2019

DOI : https://doi.org/10.1007/s11186-019-09345-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Data, field analysis
• Mixed methods research
• Multiple methods
• Reflexivity
• Sociology of science
• Find a journal
• Publish with us
• Track your research

• Research article
• Open access
• Published: 24 September 2018

A mixed methods case study exploring the impact of membership of a multi-activity, multicentre community group on social wellbeing of older adults

• Gabrielle Lindsay-Smith   ORCID: orcid.org/0000-0003-3864-1412 1 ,
• Grant O’Sullivan 1 ,
• Rochelle Eime 1 , 2 ,
• Jack Harvey 1 , 2 &
• Jannique G. Z. van Uffelen 1 , 3  

BMC Geriatrics volume  18 , Article number:  226 ( 2018 ) Cite this article

70k Accesses

30 Citations

12 Altmetric

Metrics details

Social wellbeing factors such as loneliness and social support have a major impact on the health of older adults and can contribute to physical and mental wellbeing. However, with increasing age, social contacts and social support typically decrease and levels of loneliness increase. Group social engagement appears to have additional benefits for the health of older adults compared to socialising individually with friends and family, but further research is required to confirm whether group activities can be beneficial for the social wellbeing of older adults.

This one-year longitudinal mixed methods study investigated the effect of joining a community group, offering a range of social and physical activities, on social wellbeing of adults with a mean age of 70. The study combined a quantitative survey assessing loneliness and social support ( n  = 28; three time-points, analysed using linear mixed models) and a qualitative focus group study ( n  = 11, analysed using thematic analysis) of members from Life Activities Clubs Victoria, Australia.

There was a significant reduction in loneliness ( p  = 0.023) and a trend toward an increase in social support ( p  = 0.056) in the first year after joining. The focus group confirmed these observations and suggested that social support may take longer than 1 year to develop. Focus groups also identified that group membership provided important opportunities for developing new and diverse social connections through shared interest and experience. These connections were key in improving the social wellbeing of members, especially in their sense of feeling supported or connected and less lonely. Participants agreed that increasing connections was especially beneficial following significant life events such as retirement, moving to a new house or partners becoming unwell.

Conclusions

Becoming a member of a community group offering social and physical activities may improve social wellbeing in older adults, especially following significant life events such as retirement or moving-house, where social network changes. These results indicate that ageing policy and strategies would benefit from encouraging long-term participation in social groups to assist in adapting to changes that occur in later life and optimise healthy ageing.

Peer Review reports

Ageing population and the need to age well

Between 2015 and 2050 it is predicted that globally the number of adults over the age of 60 will more than double [ 1 ]. Increasing age is associated with a greater risk of chronic illnesses such as cardio vascular disease and cancer [ 2 ] and reduced functional capacity [ 3 , 4 ]. Consequently, an ageing population will continue to place considerable pressure on the health care systems.

However, it is also important to consider the individuals themselves and self-perceived good health is very important for the individual wellbeing and life-satisfaction of older adults [ 5 ]. The terms “successful ageing” [ 6 ] and “healthy ageing” [ 5 ] have been used to define a broader concept of ageing well, which not only includes factors relating to medically defined health but also wellbeing. Unfortunately, there is no agreed definition for what exactly constitutes healthy or successful ageing, with studies using a range of definitions. A review of 28 quantitative studies found that successful ageing was defined differently in each, with the majority only considering measures of disability or physical functioning. Social and wellbeing factors were included in only a few of the studies [ 7 ].

In contrast, qualitative studies of older adults’ opinions on successful ageing have found that while good physical and mental health and maintaining physical activity levels are agreed to assist successful ageing, being independent or doing something of value, acceptance of ageing, life satisfaction, social connectedness or keeping socially active were of greater importance [ 8 , 9 , 10 ].

In light of these findings, the definition that is most inclusive is “healthy ageing” defined by the World Health Organisation as “the process of developing and maintaining the functional ability (defined as a combination of intrinsic capacity and physical and social environmental characteristics), that enables well-being in older age” (p28) [ 5 ].This definition, and those provided in the research of older adults’ perceptions of successful ageing, highlight social engagement and social support as important factors contributing to successful ageing, in addition to being important social determinants of health [ 11 , 12 ].

Social determinants of health, including loneliness and social support, are important predictors of physical, cognitive and mental health and wellbeing in adults [ 12 ] and older adults [ 13 , 14 , 15 ]. Loneliness is defined as a perception of an inadequacy in the quality or quantity of one’s social relationships [ 16 ]. Social support, has various definitions but generally it relates to social relationships that are reciprocal, accessible and reliable and provide any or a combination of supportive resources (e.g. emotional, information, practical) and can be measured as perceived or received support [ 17 ]. These types of social determinants differ from those related to inequality (health gap social determinants) and are sometimes referred to as ‘social cure’ social determinants [ 11 ]. They will be referred to as ‘social wellbeing’ outcome measures in this study.

Unfortunately, with advancing age, there is often diminishing social support, leading to social isolation and loneliness [ 18 , 19 ]. Large nationally representative studies of adults and older adults reported that social activity predicted maintenance or improvement of life satisfaction as well as physical activity levels [ 20 ], however older adults spent less time in social activity than middle age adults.

Social wellbeing and health

A number of longitudinal studies have found that social isolation for older adults is a significant predictor of mortality and institutionalisation [ 21 , 22 , 23 ]. A meta-analysis by Holt-Lunstadt [ 12 ] reported that social determinants of health, including social integration and social support (including loneliness and lack of perceived social support) to be equal to, or a greater risk to mortality as common behavioural risk factors such as smoking, physical inactivity and obesity. Loneliness is independently associated with poor physical and mental health in the general population, and especially in older adults [ 13 , 14 , 15 ]. Adequate perceived social support has also been consistently associated with improved mental and physical health in both general and older adults [ 20 , 24 , 25 , 26 , 27 , 28 , 29 ]. The mechanism suggested for this association is that social support buffers the negative impacts of stressful situations and life events [ 30 ]. The above research demonstrates the benefit of social engagement for older adults; in turn this highlights the importance of strategies that reduce loneliness and improve social support and social connectedness for older adults.

Socialising in groups seems to be especially important for the health and wellbeing of older adults who may be adjusting to significant life events [ 26 , 31 , 32 , 33 ]. This is sometimes referred to as social engagement or social companionship [ 26 , 30 , 31 ]. It seems that the mechanism enabling such health benefits with group participation is through strengthening of social identification, which in turn increases social support [ 31 , 34 , 35 ]. Furthermore, involvement in community groups can be a sustainable strategy to reduce loneliness and increase social support in older adults, as they are generally low cost and run by volunteers [ 36 , 37 , 38 , 39 ].

Despite the demonstrated importance of social factors for successful ageing and the established risk associated with reduced social engagement as people age, few in-depth studies have longitudinally investigated the impact of community groups on social wellbeing. For example, a non-significant increase in social support and reduction in depression was found in a year-long randomised controlled trial conducted in senior centres in Norway with lonely older adults in poor physical and mental health [ 37 ]. Some qualitative studies have reported that community groups and senior centres can contribute to fun and socialisation for older adults, however social wellbeing was not the primary focus of the studies [ 38 , 40 , 41 ]. Given that social wellbeing is a broad and important area for the health and quality of life in older adults, an in-depth study is warranted to understand how it can be maximised in older adults. This mixed methods case study of an existing community aims to: i) examine whether loneliness and social support of new members of Life Activities Clubs (LACs) changes in the year after joining and ii) conduct an in-depth exploration of how social wellbeing changes in new and longer-term members of LACs.

A mixed methods study was chosen as the design for this research to enable an in-depth exploration of how loneliness and social support may change as a result of joining a community group. A case study was conducted using a concurrent mixed-methods design, with a qualitative component giving context to the quantitative results. Where the survey focused on the impact of group membership on social support and loneliness, the focus groups were an open discussion of the benefits in the lived context of LAC membership. The synthesis of the two sections of the study was undertaken at the time of interpretation of the results [ 42 ].

The two parts of our study were as follows:

a longitudinal survey (three time points over 1 year: baseline, 6 and 12 months). This part of the study formed the quantitative results;

a focus group study of members of the same organisation (qualitative).

Ethics approval to conduct this study was obtained from the Victoria University Human Research Ethics Committee (HRE14–071 [survey] and HRE15–291 [focus groups]) All participants provided informed consent to partake in the study prior to undertaking the first survey or focus group.

Setting and participants

Life activities clubs victoria.

Life Activities Clubs Victoria (LACVI) is a large not-for-profit group with 23 independently run Life Activities Clubs (LACs) based in both rural and metropolitan Victoria. It has approximately 4000 members. The organisation was established to assist in providing physical, social and recreational activities as well as education and motivational support to older adults managing significant change in their lives, especially retirement.

Eighteen out of 23 LAC clubs agreed to take part in the survey study. During the sampling period from May 2014 to December 2016, new members from the participating clubs were given information about the study and invited to take part. Invitations took place in the form of flyers distributed with new membership material.

Inclusion/ exclusion criteria

Community-dwelling older adults who self-reported that they could walk at least 100 m and who were new members to LACVI and able to complete a survey in English were eligible to participate. New members were defined as people who had never been members of LACVI or who had not been members in the last 2 years.

To ensure that the cohort of participants were of a similar functional level, people with significant health problems limiting them from being able to walk 100 m were excluded from participating in the study.

Once informed consent was received, the participants were invited to complete a self-report survey in either paper or online format (depending on preference). This first survey comprised the baseline data and the same survey was completed 6 months and 12 months after this initial time point. Participants were sent reminders if they had not completed each survey more than 2 weeks after each was delivered and then again 1 week later.

Focus groups

Two focus groups (FGs) were conducted with new and longer-term members of LACs. The first FG ( n  = 6) consisted of members who undertook physical activity in their LAC (e.g. walking groups, tennis, cycling). The second FG ( n  = 5) consisted of members who took part in activities with a non-physical activity (PA) focus (e.g. book groups, social groups, craft or cultural groups). LACs offer both social and physical activities and it was important to the study to capture both types of groups, but they were kept separate to assist participants in feeling a sense of commonality with other members and improving group dynamic and participation in the discussions [ 43 ]. Of the people who participated in the longitudinal survey study, seven also participated in the FGs.

The FG interviews were facilitated by one researcher (GLS) and notes around non-verbal communication, moments of divergence and convergence amongst group members, and other notable items were taken by a second researcher (GOS). Both researchers wrote additional notes after the focus groups and these were used in the analysis of themes. Focus groups were recorded and later transcribed verbatim by a professional transcriptionist, including identification of each participant speaking. One researcher (GLS) reviewed each transcription to check for any errors and made any required modifications before importing the transcriptions into NVivo for analysis. The transcriber identified each focus group participant so themes for individuals or other age or gender specific trends could be identified.

Dependent variables

• Social support

Social support was assessed using the Duke–UNC Functional Social support questionnaire [ 44 ]. This scale specifically measures participant perceived functional social support in two areas; i) confidant support (5 questions; e.g. chances to talk to others) and ii) affective support (3 questions; e.g. people who care about them). Participants rated each component of support on a 5-item likert scale between ‘much less than I would like’ (1 point) to ‘as much as I would like’ (5 points). The total score used for analysis was the mean of the eight scores (low social support = 1, maximum social support = 5). Construct validity, concurrent validity and discriminant validity are acceptable for confidant and affective support items in the survey in the general population [ 44 ].

Loneliness was measured using the de Jong Gierveld and UCLA-3 item loneliness scales developed for use in many populations including older adults [ 45 ]. The 11-item de Jong Gierveld loneliness scale (DJG loneliness) [ 46 ] is a multi-dimensional measure of loneliness and contains five positively worded and six negatively worded items. The items fall into four subscales; feelings of severe loneliness, feelings connected with specific problem situations, missing companionship, feelings of belongingness. The total score is the sum of the items scores (i.e. 11–55): 11 is low loneliness and 55 is severe loneliness. Self-administered versions of this scale have good internal consistency (> = 0.8) and inter-item homogeneity and person scalability that is as good or better than when conducted as face-to face interviews. The validity and reliability for the scale is adequate [ 47 ]. The UCLA 3-item loneliness scale consists of three questions about how often participants feel they lack companionship, feel left out and feel isolated. The responses are given on a three-point scale ranging from hardly ever (1) to often (3). The final score is the sum of these three items with the range being from lowest loneliness (3) to highest loneliness (9). Reliability of the scale is good, (alpha = 0.72) as are discriminant validity and internal consistency [ 48 ]. The scale is commonly used to measure loneliness with older adults ([ 49 ] – review), [ 50 , 51 ].

Sociodemographic variables

The following sociodemographic characteristics were collected in both the survey and the focus groups: age, sex, highest level of education, main life occupation [ 52 ], current employment, ability to manage on income available, present marital status, country of birth, area of residence [ 53 ]. They are categorised as indicated in Table  2 .

Health variables

The following health variables were collected: Self-rated general health (from SF-12) [ 54 ] and Functional health (ability to walk 100 m- formed part of the inclusion criteria) [ 55 ]. See Table 2 for details about the categories of these variables.

The effects of becoming a member on quantitative outcome variables (i.e. Social support, DJG loneliness and UCLA loneliness) were analysed using linear mixed models (LMM). LMM enabled testing for the presence of intra-subject random effects, or equivalently, correlation of subjects’ measures over time (baseline, 6-months and 12 months). Three correlation structures were examined: independence (no correlation), compound symmetry (constant correlation of each subjects’ measures over the three time points) and autoregressive (correlation diminishing with increase in spacing in time). The best fitting correlation structure was compound symmetry; this is equivalent to a random intercept component for each subject. The LMM incorporated longitudinal trends over time, with adjustment for age as a potential confounder. Statistical analyses were conducted using SPSS for windows (v24).

UCLA loneliness and social support residuals were not normally distributed and these scales were Log10 transformed for statistical analysis.

Analyses were all adjusted for age, group attendance (calculated as average attendance at 6 and 12 months) and employment status at baseline (Full-time, Part-time, not working).

Focus group transcripts were analysed using thematic analysis [ 56 , 57 ], a flexible qualitative methodology that can be used with a variety of epistemologies, approaches and analysis methods [ 56 ]. The transcribed data were analysed using a combination of theoretical and inductive thematic analysis [ 56 ]. It was theorised that membership in a LAC would assist with social factors relating to healthy ageing [ 5 ], possibly through a social identity pathway [ 58 ], although we wanted to explore this. Semantic themes were drawn from these codes in order to conduct a pragmatic evaluation of the LACVI programs [ 56 ]. Analytic rigour in the qualitative analysis was ensured through source and analyst triangulation. Transcriptions were compared to notes taken during the focus groups by the researchers (GOS and GLS). In addition, Initial coding and themes (by GLS) were checked by a second researcher (GOS) and any disagreements regarding coding and themes were discussed prior to finalisation of codes and themes [ 57 ].

Sociodemographic and health characteristics of the 28 participants who completed the survey study are reported in Table  1 . The mean age of the participants was 66.9 and 75% were female. These demographics are representative of the entire LACVI membership. Education levels varied, with 21% being university educated, and the remainder completing high school or technical certificates. Two thirds of participants were not married. Some sociodemographic characteristics changed slightly at 6 and 12 months, mainly employment (18% in paid employment at baseline and 11% at 12-months) and ability to manage on income (36% reporting trouble managing on their income at baseline and 46% at 12 months). Almost 90% of the participants described themselves as being in good-excellent health.

Types of activities

There were a variety of types of activities that participants took part in: physical activities such as walking groups ( n  = 7), table tennis ( n  = 5), dancing class ( n  = 2), exercise class ( n  = 1), bowls ( n  = 2), golf ( n  = 3), cycling groups ( n  = 1) and non-physical leisure activities such as art and literature groups ( n  = 5), craft groups ( n  = 5), entertainment groups ( n  = 12), food/dine out groups ( n  = 18) and other sedentary leisure activities (e.g. mah jong, cards),( n  = 4). A number of people took part in more than one activity.

Frequency of attendance at LACVI and changes in social wellbeing

At six and 12 months, participants indicated how many times in the last month they attended different types of activities at their LAC. Most participants maintained the same frequency of participation over both time points. Only four people participated more frequently at 12 than at 6 months and nine reduced participation levels. The latter group included predominantly those who reduced from more than two times per week at 6 months to 2×/week at 6 months to one to two times per week ( n  = 5) or less than one time per week ( n  = 2) at 12 months. Average weekly club attendance at six and 12 months was included as a covariate in the statistical model.

Outcome measures

Overall, participants reported moderate social support and loneliness levels at baseline (See Table 2 ). Loneliness, as measured by both scales, reduced significantly over time. There was a significant effect of time on the DJG loneliness scores (F (2, 52) = 3.83, p  = 0.028), with Post-Hoc analysis indicating a reduction in DJG loneliness between baseline and 12 months ( p  = 0.008). UCLA loneliness scores (transformed variable) also changed significantly over time (F (2, 52) = 4.08, p  = 0.023). Post hoc tests indicated a reduction in UCLA loneliness between baseline and 6 months ( p  = 0.007). There was a small non-significant increase in social support (F (2, 53) =2.88, p  = 0.065) during the first year of membership (see Table 2 and Figs. 1 and 2 ).

DJG loneliness for all participants over first year of membership at LAC club ( n  = 28).

*Represents significant difference compared to baseline ( p  < 0.01)

UCLA loneliness score for all participants over first year of membership at LAC club ( n  = 28).

*Indicates log values of the variable at 6-months were significantly different from baseline ( p  < 0.01)

In total, 11 participants attended the two focus groups, six people who participated in PA clubs (four women) and five who participated in social clubs (all women). All focus group participants were either retired ( n  = 9) or semi-retired ( n  = 2). The mean age of participants was 67 years (see Table 2 for further details). Most of the participants (82%) had been members of a LAC for less than 2 years and two females in the social group had been members of LAC clubs for 5 and 10 years respectively.

Analysis of the focus group transcripts identified two themes relating to social benefits of group participation; i) Social resources and ii) Social wellbeing (see Fig. 3 ). Group discussion suggested that membership of a LAC provides access to more social resources through greater and diverse social contact and opportunity. It is through this improvement in social resources that social wellbeing may improve.

Themes arising from focus group discussion around the benefits of LAC membership

Social resources

The social resources theme referred to an increase in the availability and variety of social connections that resulted from becoming a member of a LAC. The social nature of the groups enabled an expansion and diversification of members’ social network and improved their sense of social connectedness. There was widespread agreement in both the focus groups that significant life events, especially retirement, illness or death of spouse and moving house changes one’s social resources. Membership of the LAC had benefits especially at these times and these events were often motivators to join such a club. Most participants found that their social resources declined after retirement and even felt that they were grieving for the loss of their work.

“ I just saw work as a collection of, um, colleagues as opposed to friends. I had a few good friends there. Most were simply colleagues or acquaintances …. [interviewer- Mmm.] ..Okay, you’d talk to them every day. You’d chatter in the kitchen, oh, pass banter back and forth when things are busy or quiet, but... Um, in terms of a friendship with those people, like going to their home, getting to know them, doing other things with them, very few. But what I did miss was the interaction with other people. It had simply gone….. But, yeah, look, that, the, yeah, that intervening period was, oh, a couple of months. That was a bit tough…. But in that time the people in LAC and the people in U3A…. And the other dance group just drew me into more things. Got to know more people. So once again, yeah, reasonable group of acquaintances.” (Male, PAFG)

Group members indicated general agreement with these two responses, however one female found she had a greater social life following retirement due to the busy nature of her job.

Within the social resources theme, three subthemes were identified, i) Opportunity for social connectedness, ii) Opportunity for friendships, and iii) Opportunity for social responsibility/leadership . Interestingly, these subthemes were additional to the information gathered in the survey. This emphasises the power of the inductive nature of the qualitative exploration employed in the focus groups to broaden the knowledge in this area.

The most discussed and expanded subtheme in both focus groups was Opportunity for social connectedness , which arose through developing new connections, diversifying social connections, sharing interests and experiences with others and peer learning. Participants in both focus groups stated that being a member of LAC facilitated their socialising and connecting with others to share ideas, skills and to do activities with, which was especially important through times of significant life events. Furthermore, participants in each of the focus groups valued developing diverse connections:

“ Yeah, I think, as I said, I finished up work and I, and I had more time for wa-, walking. So I think a, in meeting, in going to this group which, I saw this group of women but then someone introduced me to them. They were just meeting, just meeting a new different set of people, you know? As I said, my work people and these were just a whole different group of women, mainly women. There’s not many men. [Interviewer: Yes.]….. Although our leader is a man, which is ironic and is about, this man out in front and there’s about 20 women behind him, but, um, so yeah, and people from different walks of life and different nationalities there which I never knew in my work life, so yeah. That’s been great. So from that goes on other things, you know, you might, uh, other activities and, yeah, people for coffee and go to the pictures or something, yeah. That’s great.” (Female, PAFG)

Simply making new connections was the most widely discussed aspect related to the opportunity for social connectedness subtheme, with all participants agreeing that this was an important benefit of participation in LAC groups.

“Well, my experience is very similar to everybody else’s…….: I, I went from having no social life to a social life once I joined a group.” (Female, PAFG)

There was agreement in both focus groups that these initial new connections made at a LAC are strengthened through development of deeper personal connections with others who have similar demographics and who are interested in the same activities. This concurs with the Social Identity Theory [ 58 ] discussed previously.

“and I was walking around the lake in Ballarat, like wandering on my own. I thought, This is ridiculous. I mean, you’ve met all those groups of women coming the opposite way, so I found out what it was all about, so I joined, yeah. So that’s how I got into that.[ Interviewer: Yeah.] Basically sick of walking round the lake on my own. [Interviewer: Yeah, yeah.] So that’s great. It’s very social and they have coffee afterwards which is good.” (female, PAFG)

The subtheme Opportunity for development of friendships describes how, for some people, a number of LAC members have progressed from being just initial social connections to an established friendship. This signifies the strength of the connections that may potentially develop through LAC membership. Some participants from each group mentioned friendships developing, with slightly more discussion of this seen in the social group.

“we all have a good old chat, you know, and, and it’s all about friendship as well.” (female, SocialFG)

The subtheme Opportunity for social responsibility or leadership was mentioned by two people in the active group, however it was not brought up in the social group. This opportunity for leadership is linked with the development of a group identity and desiring to contribute meaningfully to a valued group.

“with our riding group, um, you, a leader for probably two rides a year so you’ve gotta prepare for it, so some of them do reccie rides themselves, so, um, and also every, uh, so that’s something that’s, uh, a responsibility.” (male, PAFG)

Social wellbeing

The social resources described above seem to contribute to a number of social, wellbeing outcomes for participants. The sub themes identified for Social wellbeing were , i) Increased social support, ii) Reduced loneliness, iii) Improved home relationships and iv) Improved social skills.

Increased social support

Social support was measured quantitatively in the survey (no significant change over time for new members) and identified as a benefit of LAC membership during the focus group discussions. However, only one of the members of the active group mentioned social support directly.

‘it’s nice to be able to pick up the phone and share your problem with somebody else, and that’s come about through LAC. ……‘Cos before that it was through, with my family (female, PAFG)

There was some agreement amongst participants of the PA group that they felt this kind of support may develop in time but most of them had been members for less than 2 years.

“[Interviewer: Yeah. Does anyone else have that experience? (relating to above quote)]” There is one lady but she’s actually the one that I joined with anyway. [Interviewer: Okay.] But I, I feel there are others that are definitely getting towards that stage. It’s still going quite early days. (female1, PAFG) [Interviewer: I guess it’s quite early for some of you, yeah.] “yeah” (female 2, PAFG)

Social support through sharing of skills was mentioned by one participant in the social group also, with agreement indicated by most of the others in the social focus group.

Discussion in the focus groups also touched on the subthemes Reduced loneliness and Improved home relationships, which were each mentioned by one person. And focus groups also felt that group membership Improved social skills through opening up and becoming more approachable (male, PAFG) or enabling them to become more accepting of others’ who are different (general agreement in Social FG).

This case study integrated results from a one-year longitudinal survey study and focus group discussions to gather rich information regarding the potential changes in social wellbeing that older adults may experience when joining community organisations offering group activities. The findings from this study indicate that becoming a member of such a community organisation can be associated with a range of social benefits for older adults, particularly related to reducing loneliness and maintaining social connections.

Joining a LAC was associated with a reduction in loneliness over 1 year. This finding is in line with past group-intervention studies where social activity groups were found to assist in reducing loneliness and social isolation [ 49 ]. This systematic review highlighted that the majority of the literature explored the effectiveness of group activity interventions for reducing severe loneliness or loneliness in clinical populations [ 49 ]. The present study extends this research to the general older adult population who are not specifically lonely and reported to be of good general health, rather than a clinical focus. Our findings are in contrast to results from an evaluation of a community capacity-building program aimed at reducing social isolation in older adults in rural Australia [ 59 ]. That program did not successfully reduce loneliness or improve social support. The lack of change from pre- to post-program in that study was reasoned to be due to sampling error, unstandardised data collection, and changes in sample characteristics across the programs [ 59 ]. Qualitative assessment of the same program [ 59 ] did however suggest that participants felt it was successful in reducing social isolation, which does support our findings.

Changes in loneliness were not a main discussion point of the qualitative component of the current study, however some participants did express that they felt less lonely since joining LACVI and all felt they had become more connected with others. This is not so much of a contrast in results as a potential situational issue. The lack of discussion of loneliness may have been linked to the common social stigma around experiencing loneliness outside certain accepted circumstances (e.g. widowhood), which may lead to underreporting in front of others [ 45 ].

Overall, both components of the study suggest that becoming a member of an activity group may be associated with reductions in loneliness, or at least a greater sense of social connectedness. In addition to the social nature of the groups and increased opportunity for social connections, another possible link between group activity and reduced loneliness is an increased opportunity for time out of home. Previous research has found that more time away from home in an average day is associated with lower loneliness in older adults [ 60 ]. Given the significant health and social problems that are related to loneliness and social isolation [ 13 , 14 , 15 ], the importance of group involvement for newly retired adults to prevent loneliness should be advocated.

In line with a significant reduction in loneliness, there was also a trend ( p  = 0.056) toward an increase in social support from baseline to 12 months in the survey study. Whilst suggestive of a change, it is far less conclusive than the findings for loneliness. There are a number of possible explanations for the lack of statistically significant change in this variable over the course of the study. The first is the small sample size, which would reduce the statistical power of the study. It may be that larger studies are required to observe changes in social support, which are possibly only subtle over the course of 1 year. This idea is supported by a year-long randomised controlled trial with 90 mildly-depressed older adults who attended senior citizen’s club in Norway [ 37 ]. The study failed to see any change in general social support in the intervention group compared to the control over 1 year. Additional analysis in that study suggested that people who attended the intervention groups more often, tended to have greater increases in SS ( p  = 0.08). The researchers stated that the study suffered from significant drop-out rates and low power as a result. In this way, it was similar to our findings and suggests that social support studies require larger numbers than we were able to gain in this early exploratory study. Another possible reason for small changes in SS in the current study may be the type of SS measured. The scale used gathered information around functional support or support given to individuals in times of need. Maybe it is not this type of support that changes in such groups but more specific support such as task-specific support. It has been observed in other studies and reviews that task-specific support changes as a result of behavioural interventions (e.g. PA interventions) but general support does not seem to change in the time frames often studied [ 61 , 62 , 63 ].

There were many social wellbeing benefits such as increased social connectivity identified in focus group discussion, but the specific theme of social support was rarely mentioned. It may be that general social support through such community groups may take longer than 1 year to develop. There is evidence that strong group ties are sequentially positively associated between social identification and social support [ 34 ], suggesting that the connections formed through the groups may lead increased to social support from group members in the future. This is supported by results from the focus group discussions, where one new member felt she could call on colleagues she met in her new group. Other new members thought it was too soon for this support to be available, but they could see the bonds developing.

Other social wellbeing changes

In addition to social support and loneliness that were the focus of the quantitative study, the focus group discussions uncovered a number of other benefits of group membership that were related to social wellbeing (see Fig. 3 ). The social resources theme was of particular interest because it reflected some of the mechanisms that appeared enable social wellbeing changes as a result of being a member of a LAC but were not measured in the survey. The main social resources relating to group membership that were mentioned in the focus groups were social connectedness, development of friendships and opportunity for social responsibility or leadership. As mentioned above, there was wide-spread discussion within the focus groups of the development of social connections through the clubs. Social connectedness is defined as “the sense of belonging and subjective psychological bond that people feel in relation to individuals and groups of others.” ([ 25 ], pp1). As well as being an important predecessor of social support, greater social connectedness has been found to be highly important for the health of older adults, especially cognitive and mental health [ 26 , 32 , 34 , 35 , 64 ]. One suggested theory for this health benefit is that connections developed through groups that we strongly identify with are likely to be important for the development of social identity [ 34 ], defined by Taifel as: “knowledge that [we] belong to certain social groups together with some emotional and value significance to [us] of this group membership” (Tajfel, 1972, p. 31 in [ 58 ] p 2). These types of groups to which we identify may be a source of “personal security, social companionship, emotional bonding, intellectual stimulation, and collaborative learning and……allow us to achieve goals.” ([ 58 ] p2) and an overall sense of self-worth and wellbeing. There was a great deal of discussion relating to the opportunity for social connectedness derived through group membership being particularly pertinent following a significant life event such as moving to a new house or partners becoming unwell or dying and especially retirement. This change in their social circumstance is likely to have triggered the need to renew their social identity by joining a community group. Research with university students has shown that new group identification can assist in transition for university students who have lost their old groups of friends because of starting university [ 65 ]. In an example relevant to older adults, maintenance or increase in number of group memberships at the time of retirement reduced mortality risk 8 years later compared to people who reduce their number of group activities in a longitudinal cohort study [ 66 ]. This would fit with the original Activity Theory of ageing; whereby better ageing experience is achieved when levels of social participation are maintained, and role replacement occurs when old roles (such as working roles) must be relinquished [ 67 ]. These connections therefore appear to assist in maintaining resilience in older adults defined as “the ability to maintain or improve a level of functional ability (a combination of intrinsic physical and mental capacity and environment) in the face of adversity” (p29, [ 5 ]). Factors that were mentioned in the focus groups as assisting participants in forming connections with others were shared interest, learning from others, and a fun and accepting environment. It was not possible to assess all life events in the survey study. However, since the discussion from the focus groups suggested this to be an important motivator for joining clubs and potentially a beneficial time for joining them, it would be worth exploring in future studies.

Focus group discussion suggested that an especially valuable time for joining such clubs was around retirement, to assist with maintaining social connectivity. The social groups seem to provide social activity and new roles for these older adults at times of change. It is not necessarily important for all older adults but maybe these ones identify themselves as social beings and therefore this maintenance of social connection helps to continue their social role. Given the suggested importance of social connectivity gained through this organisation, especially at times of significant life events, it would valuable to investigate this further in future and consider encouragement of such through government policy and funding. The majority of these types of clubs exist for older adults in general, but this study emphasises the need for groups such as these to target newly retired individuals specifically and to ensure that they are not seen as ‘only for old people’.

Strengths and limitations

The use of mixed –methodologies, combining longitudinal survey study analysed quantitatively, with a qualitative exploration through focus group discussions and thematic analysis, was a strength of the current study. It allowed the researchers to not only examine the association between becoming a member of a community group on social support and loneliness over an extended period, but also obtain a deeper understanding of the underlying reasons behind any associations. Given the variability of social support definitions in research [ 17 ] and the broad area of social wellbeing, it allowed for open exploration of the topic, to understand associations that may exist but would have otherwise been missed. Embedding the research in an existing community organisation was a strength, although with this also came some difficulties with recruitment. Voluntary coordination of the community groups meant that informing new members about the study was not always feasible or a priority for the volunteers. In addition, calling for new members was innately challenging because they were not yet committed to the club fully. This meant that so some people did not want to commit to a year-long study if they were not sure how long they would be a member of the club. This resulted in slow recruitment and a resulting relatively low sample size and decreased power to show significant statistical differences, which is a limitation of the present study. However, the use of Linear Mixed Models for analysis of the survey data was a strength because it was able to include all data in the analyses and not remove participants if one time point of data was missing, as repeated measures ANOVAs would do. The length of the study (1 year) is another strength, especially compared to previous randomised controlled studies that are typically only 6–16 weeks in length. Drop-out rate in the current study is very low and probably attributable to the benefits of working with long-standing organisations.

The purpose of this study was to explore in detail whether there are any relationships between joining existing community groups for older adults and social wellbeing. The lack of existing evidence in the field meant that a small feasibility-type case study was a good sounding-board for future larger scale research on the topic, despite not being able to answer questions of causality. Owing to the particularistic nature of case studies, it can also be difficult to generalise to other types of organisations or groups unless there is a great deal of similarity between them [ 68 ]. There are however, other types of community organisations in existence that have a similar structure to LACVI (Seniors centres [ 36 , 40 ], Men’s Sheds [ 38 ], University of the Third Age [ 34 , 69 ], Japanese salons [ 70 , 71 ]) and it may be that the results from this study are transferable to these also. This study adds to the literature around the benefits of joining community organisations that offer social and physical activities for older adults and suggests that this engagement may assist with reducing loneliness and maintaining social connection, especially around the time of retirement.

Directions for future research

Given that social support trended toward a significant increase, it would be useful to repeat the study on a larger scale in future to confirm this. Either a case study on a similar but larger community group or combining a number of community organisations would enable recruitment of more participants. Such an approach would also assist in assessing the generalisability of our findings to other community groups. Given that discussions around social benefits of group membership in the focus groups was often raised in conjunction with the occurrence of significant life events, it would be beneficial to include a significant life event scale in any future studies in this area. The qualitative results also suggest that it would be useful to investigate whether people who join community groups in early years post retirement gain the same social benefits as those in later stages of retirement. Studies investigating additional health benefits of these community groups such as physical activity, depression and general wellbeing would also be warranted.

With an ageing population, it is important to investigate ways to enable older adults to age successfully to ensure optimal quality of life and minimisation of health care costs. Social determinants of health such as social support, loneliness and social contact are important contributors to successful ageing through improvements in cognitive health, quality of life, reduction in depression and reduction in mortality. Unfortunately, older adults are at risk of these social factors declining in older age and there is little research investigating how best to tackle this. Community groups offering a range of activities may assist by improving social connectedness and social support and reducing loneliness for older adults. Some factors that may assist with this are activities that encourage sharing interests, learning from others, and are conducted in a fun and accepting environment. Such groups may be particularly important in developing social contacts for newly retired individuals or around other significant life events such as moving or illness of loved ones. In conclusion, ageing policy and strategies should emphasise participation in community groups especially for those recently retired, as they may assist in reducing loneliness and increasing social connections for older adults.

Abbreviations

Focus group

Life Activities Club

Life Activities Clubs Victoria

Linear mixed model

Physical activity

World Health Organisation

United Nations. In: P.D. Department of Economic and Social Affairs, editor. World Population Ageing 2015. New York: United Nations; 2015.

Google Scholar  

World Health Organisation. Global Health and Ageing. 2011 [cited 2014 March 25]; Available from: http://www.who.int/ageing/publications/global_health/en/ .

Balogun JA, et al. Age-related changes in balance performance. Disabil Rehabil. 1994;16(2):58–62.

Article   CAS   Google Scholar  

Singh MAF. Exercise comes of age: rationale and recommendations for a geriatric exercise prescription. J Gerontol Ser A Biol Med Sci. 2002;57(5):M262–82.

Article   Google Scholar  

World Health Organisation. World report on ageing and health. Geneva: World Health Organisation; 2015.

Rowe JW, Kahn RL. Successful Aging1. The Gerontologist. 1997;37(4):433–40.

Depp CA, Jeste DV. Definitions and predictors of successful aging: a comprehensive review of larger quantitative studies. Am J Geriatr Psychiatry. 2006;14(1):6–20.

Song M, Kong E-H. Older adults’ definitions of health: a metasynthesis. Int J Nurs Stud. 2015;52(6):1097–106.

Tate RB, Lah L, Cuddy TE. Definition of successful aging by elderly Canadian males: the Manitoba follow-up study. The Gerontologist. 2003;43(5):735–44.

Phelan EA, et al. Older Adults’ views of “successful aging”—how do they compare with Researchers’ definitions? J Am Geriatr Soc. 2004;52(2):211–6.

Haslam SA, et al. Social cure, what social cure? The propensity to underestimate the importance of social factors for health. Soc Sci Med. 2018;198:14–21.

Holt-Lunstad J, Smith TB, Layton JB. Social relationships and mortality risk: a meta-analytic review. PLoS Med. 2010;7(7):e1000316.

Richard A, et al. Loneliness is adversely associated with physical and mental health and lifestyle factors: results from a Swiss national survey. PLoS One. 2017;12(7):1–18.

Luo Y, et al. Loneliness, health, and mortality in old age: a national longitudinal study. Soc Sci Med. 2012;74(6):907–14.

Luo Y, Waite LJ. Loneliness and mortality among older adults in China. J Gerontol B Psychol Sci Soc Sci. 2014;69(4):633–45.

de Jong Gierveld J, Van Tilburg T, Dykstra PA. Loneliness and social isolation. In: Vangelisti A, Perlman D, editors. Cambridge handbook of personal relationships. Cambridge: Cambridge University Press; 2006. p. 485-500.

Williams P, Barclay L, Schmied V. Defining social support in context: a necessary step in improving research, intervention, and practice. Qual Health Res. 2004;14(7):942–60.

Valtorta N, Hanratty B. Loneliness, isolation and the health of older adults: do we need a new research agenda? J R Soc Med. 2012;105(12):518–22.

Jylhä M. Old age and loneliness: cross-sectional and longitudinal analyses in the Tampere longitudinal study on aging. Can J Aging La Revue can du vieil. 2010;23(2):157–68.

Huxhold O, Miche M, Schüz B. Benefits of having friends in older ages: differential effects of informal social activities on well-being in middle-aged and older adults. J Gerontol B Psychol Sci Soc Sci. 2014;69(3):366–75.

Berkman L, Syme S. Social networks, host resistance and mortality: a nine year follow-up study of alameda county residents. Am J Epidemiol. 1979;185(11):1070–88.

House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540–5.

Pynnonen K, et al. Does social activity decrease risk for institutionalization and mortality in older people? J Gerontol B Psychol Sci Soc Sci. 2012;67(6):765–74.

Broadhead WE, et al. The epidemiologic evidence for a relationship between social support and health. Am J Epidemiol. 1983;117(5):521–37.

Haslam C, et al. Social connectedness and health. Encyclopedia of geropsychology. 2017:2174–82. https://doi.org/10.1007/978-981-287-080-3_46-1 .

Haslam C, Cruwys T, Haslam SA. “The we’s have it”: evidence for the distinctive benefits of group engagement in enhancing cognitive health in aging. Soc Sci Med. 2014;120:57–66.

Uebelacker LA, et al. Social support and physical activity as moderators of life stress in predicting baseline depression and change in depression over time in the Women’s Health Initiative. Soc Psychiatry Psychiatr Epidemiol. 2013;48(12):1971–82.

Tajvar M, et al. Social support and health of older people in middle eastern countries: a systematic review. Australas J Ageing. 2013;32(2):71–8.

Dalgard OS, Bjork S, Tambs K. Social support, negative life events and mental health. Br J Psychiatry. 1995;166(1):29–34.

Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychol Bull. 1985;98(2):310–57.

Gilmour H. Social participation and the health and well-being of Canadian seniors. Health Rep. 2012;23(4):1B.

Glei DA, et al. Participating in social activities helps preserve cognitive function: an analysis of a longitudinal, population-based study of the elderly. Int J Epidemiol. 2005;34(4):864–71.

Lee SH, Kim YB. Which type of social activities may reduce cognitive decline in the elderly?: a longitudinal population-based study. BMC Geriatr. 2016;16(1):165.

Haslam C, et al. Group ties protect cognitive health by promoting social identification and social support. J Aging Health. 2016;28(2):244–66.

Haslam C, et al. Groups 4 health: evidence that a social-identity intervention that builds and strengthens social group membership improves mental health. J Affect Disord. 2016;194:188–95.

Bøen H. Characteristics of senior Centre users -- and the impact of a group programme on social support and late-life depression. Norsk Epidemiologi. 2012;22(2):261–9.

Bøen H, et al. A randomized controlled trial of a senior Centre group programme for increasing social support and preventing depression in elderly people living at home in Norway. BMC Geriatr. 2012;12:20.

Golding BG. Social, local, and situated: recent findings about the effectiveness of older Men’s informal learning in community contexts. Adult Educ Q. 2011;61(2):103.

Life Activities Clubs. Life Activities Clubs. About Us. . 2014 [cited 2014 January 13, 2014]; Available from: http://www.life.org.au/aboutus .

Hutchinson SL, Gallant KA. Can senior Centres be contexts for aging in third places? J Leis Res. 2016;48(1):50–68.

Millard J. The health of older adults in community activities. Work Older People. 2017;21(2):90–9.

Creswell JW, Plano-Clark VL. Designing and conducting mixed methods research. Thousand Oaks, Calif: SAGE Publications; 2007. p. c2007.

Loeb S, Penrod J, Hupcey J. Focus groups and older adults: tactics for success. J Gerontol Nurs. 2006;32(3):32–8.

PubMed   Google Scholar  

Broadhead W, et al. The Duke-UNC functional social support questionnaire: measurement of social support in family medicine patients. Med Care. 1988;26(7):709–23.

De Jong Gierveld J, Van Tilburg T. Living arrangements of older adults in the Netherlands and Italy: Coresidence values and behaviour and their consequences for loneliness. J Cross Cult Gerontol. 1999;14(1):1–24.

de Jong-Gierveld J, Kamphuls F. The development of a Rasch-type loneliness scale. Appl Psychol Meas. 1985;9(3):289–99.

Tilburg Tv, Leeuw Ed. Stability of scale quality under various data collection procedures: a mode comparison on the ‘De Jong-Gierveld loneliness scale. Int J Public Opin Res. 1991;3(1):69–85.

Hughes ME, et al. A short scale for measuring loneliness in large surveys - results from two population-based studies. Res Aging. 2004;26(6):655–72.

Dickens AP, et al. Interventions targeting social isolation in older people: a systematic review. BMC Public Health. 2011;11:647.

Hawkley LC, et al. Loneliness predicts increased blood pressure: 5-year cross-lagged analyses in middle-aged and older adults. Psychol Aging. 2010;25(1):132.

Netz Y, et al. Loneliness is associated with an increased risk of sedentary life in older Israelis. Aging Ment Health. 2013;17(1):40–7.

Australian Bureau of Statistics. Australian and New Zealand Standard Classification of Occupations, 2013, Version 1.2. Canberra: Australian Bureau of Statistics; 2013.

Australian Bureau of Statistics, Australian Standard Geographical Classification (ASGC). Canberra: Australian Bureau of Statistics; 2011.

Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey - construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–33.

Sanson-Fisher RW, Perkins JJ. Adaptation and validation of the SF-36 health survey for use in Australia. J Clin Epidemiol. 51(11):961–7.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Patton MQ. Qualitative research & evaluation methods. 4 ed. Thousand Oaks, California: Sage Publications; 2015.

Haslam SA, et al. Social identity, health and well-being: an emerging agenda for applied psychology. Appl Psychol. 2009;58(1):1–23.

Bartlett H, et al. Preventing social isolation in later life: findings and insights from a pilot Queensland intervention study. Ageing Soc. 2013;33(07):1167–89.

Petersen J, et al. Time out-of-home and cognitive, physical, and emotional wellbeing of older adults: a longitudinal mixed effects model. PLoS One. 2015;10(10):e0139643.

Lindsay Smith G, et al. The association between social support and physical activity in older adults: a systematic review. Int J Behav Nutr Phys Act. 2017;14(1):56.

Sallis JF, et al. The development of scales to measure social support for diet and exercise behaviors. Prev Med. 1987;16(6):825–36.

Oka R, King A, Young DR. Sources of social support as predictors of exercise adherence in women and men ages 50 to 65 years. Womens Health Res Gender Behav Policy. 1995;1:161–75.

CAS   Google Scholar  

Greenaway KH, et al. From “we” to “me”: group identification enhances perceived personal control with consequences for health and well-being. J Pers Soc Psychol. 2015;109(1):53–74.

Iyer A, et al. The more (and the more compatible) the merrier: multiple group memberships and identity compatibility as predictors of adjustment after life transitions. Br J Soc Psychol. 2009;48(4):707–33.

Steffens NK, et al. Social group memberships in retirement are associated with reduced risk of premature death: evidence from a longitudinal cohort study. BMJ Open. 2016;6(2):e010164.

Havighurst RJ. Successful aging. Gerontol. 1961;1:8–13.

Yin R. Case study research: design and methods. Beverly Hills, CA: Sage publishing; 1994.

Merriam SB, Kee Y. Promoting community wellbeing: the case for lifelong learning for older adults. Adult Educ Q. 2014;64(2):128–44.

Hikichi H, et al. Social interaction and cognitive decline: results of a 7-year community intervention. Alzheimers Dement: Translat Res Clin Interv. 2017;3(1):23–32.

Hikichi H, et al. Effect of a community intervention programme promoting social interactions on functional disability prevention for older adults: propensity score matching and instrumental variable analyses, JAGES Taketoyo study. J Epidemiol Community Health. 2015;69(9):905–10.

Download references

The primary author contributing to this study (GLS) receives PhD scholarship funding from Victoria University. The other authors were funded through salaries at Victoria University.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due the ethics approval for this study not allowing open access to the individual participant data but are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

Institute for Health and Sport, Victoria University, Melbourne, Australia

Gabrielle Lindsay-Smith, Grant O’Sullivan, Rochelle Eime, Jack Harvey & Jannique G. Z. van Uffelen

School of Health and Life Sciences, Federation University, Ballarat, Australia

Rochelle Eime & Jack Harvey

Department of Movement Sciences, Physical Activity, Sports and Health Research Group, KU Leuven - University of Leuven, Leuven, Belgium

Jannique G. Z. van Uffelen

You can also search for this author in PubMed   Google Scholar

Contributions

GLS, RE and JVU made substantial contributions to the conception and design of the study. GLS and GOS supervised data collection for the surveys (GLS) and focus groups (GOS and GLS). GLS, GOS, RE, JH and JVU were involved in data analysis and interpretation. All authors were involved in drafting, the manuscript and approved the final version.

Corresponding author

Correspondence to Gabrielle Lindsay-Smith .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval to conduct this study was obtained from the Victoria University Human Research Ethics Committee (HRE14–071 [survey] and HRE15–291 [focus groups]). All survey participants provided informed consent to partake in the study prior to undertaking the first survey by clicking ‘agree’ in the survey software (online participants) or signing a written consent form (paper participants) after they had read all participant information relating to the study. Focus group participants also completed a written consent form after reading and understanding all participant information regarding the study.

Consent for publication

Focus group participants have given permission for their quotes to be used in de-identified form prior to undertaking the research.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Lindsay-Smith, G., O’Sullivan, G., Eime, R. et al. A mixed methods case study exploring the impact of membership of a multi-activity, multicentre community group on social wellbeing of older adults. BMC Geriatr 18 , 226 (2018). https://doi.org/10.1186/s12877-018-0913-1

Download citation

Received : 19 April 2018

Accepted : 10 September 2018

Published : 24 September 2018

DOI : https://doi.org/10.1186/s12877-018-0913-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social engagement
• Group activity

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 20, Issue 3
• Mixed methods research: expanding the evidence base
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Allison Shorten 1 ,
• Joanna Smith 2
• 1 School of Nursing , University of Alabama at Birmingham , USA
• 2 Children's Nursing, School of Healthcare , University of Leeds , UK
• Correspondence to Dr Allison Shorten, School of Nursing, University of Alabama at Birmingham, 1720 2nd Ave South, Birmingham, AL, 35294, USA; [email protected]; ashorten{at}uab.edu

https://doi.org/10.1136/eb-2017-102699

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

‘Mixed methods’ is a research approach whereby researchers collect and analyse both quantitative and qualitative data within the same study. 1 2 Growth of mixed methods research in nursing and healthcare has occurred at a time of internationally increasing complexity in healthcare delivery. Mixed methods research draws on potential strengths of both qualitative and quantitative methods, 3 allowing researchers to explore diverse perspectives and uncover relationships that exist between the intricate layers of our multifaceted research questions. As providers and policy makers strive to ensure quality and safety for patients and families, researchers can use mixed methods to explore contemporary healthcare trends and practices across increasingly diverse practice settings.

What is mixed methods research?

Mixed methods research requires a purposeful mixing of methods in data collection, data analysis and interpretation of the evidence. The key word is ‘mixed’, as an essential step in the mixed methods approach is data linkage, or integration at an appropriate stage in the research process. 4 Purposeful data integration enables researchers to seek a more panoramic view of their research landscape, viewing phenomena from different viewpoints and through diverse research lenses. For example, in a randomised controlled trial (RCT) evaluating a decision aid for women making choices about birth after caesarean, quantitative data were collected to assess knowledge change, levels of decisional conflict, birth choices and outcomes. 5 Qualitative narrative data were collected to gain insight into women’s decision-making experiences and factors that influenced their choices for mode of birth. 5

In contrast, multimethod research uses a single research paradigm, either quantitative or qualitative. Data are collected and analysed using different methods within the same paradigm. 6 7 For example, in a multimethods qualitative study investigating parent–professional shared decision-making regarding diagnosis of suspected shunt malfunction in children, data collection included audio recordings of admission consultations and interviews 1 week post consultation, with interactions analysed using conversational analysis and the framework approach for the interview data. 8

What are the strengths and challenges in using mixed methods?

Selecting the right research method starts with identifying the research question and study aims. A mixed methods design is appropriate for answering research questions that neither quantitative nor qualitative methods could answer alone. 4 9–11 Mixed methods can be used to gain a better understanding of connections or contradictions between qualitative and quantitative data; they can provide opportunities for participants to have a strong voice and share their experiences across the research process, and they can facilitate different avenues of exploration that enrich the evidence and enable questions to be answered more deeply. 11 Mixed methods can facilitate greater scholarly interaction and enrich the experiences of researchers as different perspectives illuminate the issues being studied. 11

The process of mixing methods within one study, however, can add to the complexity of conducting research. It often requires more resources (time and personnel) and additional research training, as multidisciplinary research teams need to become conversant with alternative research paradigms and different approaches to sample selection, data collection, data analysis and data synthesis or integration. 11

What are the different types of mixed methods designs?

Mixed methods research comprises different types of design categories, including explanatory, exploratory, parallel and nested (embedded) designs. 2   Table 1 summarises the characteristics of each design, the process used and models of connecting or integrating data. For each type of research, an example was created to illustrate how each study design might be applied to address similar but different nursing research aims within the same general nursing research area.

• View inline

Types of mixed methods designs*

What should be considered when evaluating mixed methods research?

When reading mixed methods research or writing a proposal using mixed methods to answer a research question, the six questions below are a useful guide 12 :

Does the research question justify the use of mixed methods?

Is the method sequence clearly described, logical in flow and well aligned with study aims?

Is data collection and analysis clearly described and well aligned with study aims?

Does one method dominate the other or are they equally important?

Did the use of one method limit or confound the other method?

When, how and by whom is data integration (mixing) achieved?

For more detail of the evaluation guide, refer to the McMaster University Mixed Methods Appraisal Tool. 12 The quality checklist for appraising published mixed methods research could also be used as a design checklist when planning mixed methods studies.

• Elliot AE , et al
• Creswell JW ,
• Plano ClarkV L
• Greene JC ,
• Caracelli VJ ,
• Ivankova NV
• Shorten A ,
• Shorten B ,
• Halcomb E ,
• Cheater F ,
• Bekker H , et al
• Tashakkori A ,
• Creswell JW
• 12. ↵ National Collaborating Centre for Methods and Tools . Appraising qualitative, quantitative, and mixed methods studies included in mixed studies reviews: the MMAT . Hamilton, ON : BMJ Publishing Group , 2015 . http://www.nccmt.ca/resources/search/232 (accessed May 2017) .

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

• Open access
• Published: 11 April 2018

Towards the use of mixed methods inquiry as best practice in health outcomes research

• Antoine Regnault   ORCID: orcid.org/0000-0001-8234-2896 1 ,
• Tom Willgoss 2 &
• Skye Barbic 3

On behalf of the International Society for Quality of Life Research (ISOQOL) Mixed Methods Special Interest Group (SIG)

Journal of Patient-Reported Outcomes volume  2 , Article number:  19 ( 2018 ) Cite this article

45k Accesses

106 Citations

28 Altmetric

Metrics details

Mixed methods research (MMR) has found an increased interest in the field of health outcomes research. Consideration for both qualitative and quantitative perspectives has become key to contextualising patient experiences in a clinically meaningful measurement framework. The purpose of this paper is to outline a process for incorporating MMR in health outcomes research to guide stakeholders in their understanding of the essence of mixed methods inquiry. In addition, this paper will outline the benefits and challenges of MMR and describe the types of support needed for designing and conducting robust MMR measurement studies. MMR involves the application of a well-defined and pre-specified research design that articulates purposely and prospectively, qualitative and quantitative components to generate an integrated set of evidence addressing a single research question. Various methodological design options are possible depending on the research question. MMR designs allow a research question to be studied thoroughly from different perspectives. When applied, it allows the strengths of one approach to complement the restrictions of another. Among other applications, MMR can be used to enhance the creation of conceptual models and development of new instruments, to interpret the meaningfulness of outcomes in a clinical study from the patient perspective, and inform health care policy. Robust MMR requires research teams with experience in both qualitative and quantitative research. Moreover, a thorough understanding of the underlying principles of MMR is recommended at the point of study conception all the way through to implementation and knowledge dissemination. The framework outlined in this paper is designed to encourage health outcomes researchers to apply MMR to their research and to facilitate innovative, patient-centred methodological solutions to address the complex challenges of the field.

“We need a moral and methodological community that honors and celebrates paradigm and methodological diversity.” Denzin, [ 1 ]; (pp.425).

Mixed methods research (MMR) has been established for more than 50 years as a methodological approach in the social and behavioural sciences and is now well accepted and commonly used in health sciences [ 2 , 3 , 4 ]. In line with the call to “measure what matters” to patients, patient reported outcomes are increasingly being used in clinical care and research. However, a recent review of studies documenting the development of patient-reported outcomes (PRO) measures highlights that only 11% of PROs have been developed by actually asking patients which outcomes are important to them [ 5 ]. This highlights a clear application for MMR to combining qualitative and quantitative methods in health research to ensure a focus on patient-identified priorities, scientific rigour, and improved patient outcomes.

In 2012, a special section of Quality of Life Research was dedicated to applied MMR [ 6 ]. In this issue, MMR was described as an approach to inform the content validity of PROs within the early development phase. The Food and Drug Administration (FDA) also outlined a clear role for MMR in their roadmap to patient focused measurement [ 7 ]. Shortly after, a Special Interest Group (SIG) was created within the International Society for Quality of Life Research (ISOQOL) to promote the use of MMR and encourage health outcomes researchers to embrace the MMR paradigm. As a SIG, we believe that there is a need for guiding principles for researchers who wish to undertake MMR. With this position paper, we aim to provide a framework for MMR in health outcomes by outlining the characteristics of this methodology, what can be expected and where caution should be exercised.

Defining features of mixed methods research

The application of the MMR paradigm in the health outcomes field can be rooted in the widely accepted definition by Tashakkori and Creswell: “Mixed Methods Research is a research in which the investigator collects and analyses data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or program of inquiry” [ 8 ] (pp.2).

In the spirit of pragmatism which underlies this methodology, we assert that utilizing MMR in the health outcomes field should not be limited to the application of a closed list of possible methodological options, but should be viewed as a framework characterized by three key defining features described in Table  1 .

In this framework, not only should both qualitative and quantitative strands be used by the researcher, but they should be complemented in a relevant research design that is set a priori. They may address distinct, specific research questions but they contribute to the same overall end purpose of the MMR. Once the overall purpose, and the specific research questions of the qualitative and quantitative strands, are well-defined and procedures are outlined, a clear plan for interaction between the qualitative and quantitative research components is needed. This point relates to the notion of meta-inference that requires qualitative and quantitative evidence not be considered independently, but interpreted together as a single body of evidence [ 3 ]. Importantly, the importance of the specification relates to the research design, and especially the articulation of the qualitative and quantitative strands, but it obviously does not necessary apply within the research strands, as, in many instances, in particular for qualitative research, a full prespecification may not be appropriate, the research being of exploratory nature.

The characterization of MMR is driven neither by the data collection process, nor by the analysis technique. In MMR, it is not necessary that the qualitative and quantitative streams involve data collected with the same respondents. A well-designed MMR study may combine qualitative and quantitative data from different samples of individuals to address a single research question, combining rigorous qualitative and quantitative evidence. Conversely, the collection of qualitative and quantitative data for the same individuals does not necessarily allow for a proper MMR solution as it may be done to address different research questions or without considering both data sets in an integrated approach.

Many options are available to the health outcomes researcher looking to utilize MMR, depending on the research question and design. We assert that MMR should not be restricted to any specific research design or methodology, but rather the design that is best suited to answer the research question posed. The articulation of the qualitative and quantitative elements can be performed in various designs that are well described in the methodological literature (e.g. convergent, or parallel or concurrent designs; sequential designs; embedded designs) [ 4 ]. The choice of the appropriate design and analysis technique (qualitative and quantitative) remains the responsibility of the researcher who should be guided by the principles outlined above.

Benefits and challenges of mixed methods research

MMR allows a research question to be studied from different perspectives. For example, one can combine the rich, subjective insights on complex realities from qualitative inquiry, with the standardized, generalizable data generated through quantitative research. When applied, MMR allows respective strengths and weaknesses of each approach to complement each other.

Since its conception in 2015, the Mixed Methods SIG of ISOQOL has identified and discussed many different applications of MMR in the health outcomes field: exploration of patient experiences to support the development of conceptual models with group concept mapping [ 9 , 10 , 11 ]; development of new clinical outcome assessment instruments with integrated qualitative and early quantitative analyses with Rasch model [ 12 , 13 , 14 , 15 ]; quantitatization of qualitative data to support conceptual saturation analyses [ 16 , 17 ]; use of qualitative information to support the interpretation of quantitative patient-reported outcomes results [ 18 , 19 ], to name but a few.

These examples show that MMR can help us to address common questions in the health outcomes field. This is typified by the inductive and iterative process characteristic of the development of new PROs. It also allows for flexibility and the ability to make the most of small samples. MMR enables a pragmatic path forward to conduct health outcomes measurement research in rare disease populations [ 20 ] or populations that are often difficult to recruit for research purposes (e.g., paediatrics, acute mental health, palliative). In a clinical research context, the versatility of MMR makes it a method of choice for hypothesis generation on PRO endpoints, especially in phase II trials.

Finally, a critical strength of MMR approaches is that they typically capitalize on data reflecting individual lived experiences (in the qualitative strand). This ensures that the results are considered from the patient-perspective. Incorporating the patient voice in MMR helps ensure that the research is focused on the needs and priorities of patients. Moreover, MMR can facilitate the involvement of other key stakeholders, such as partners, family members, and/or other knowledge users, in the process of developing the research question(s) and outlining the research designs. In this context, it appears clearly that MMR is a strong option to leverage effective patient engagement and support ongoing research focused on patient-identified priorities and the improvement of patient outcomes [ 21 ].

Despite some clear benefits, the application of MMR in the health outcomes research does not come without challenges. One major hurdle is that MMR is demanding in terms of methodological skillsets. MMR requires a team of researchers who are experienced in both qualitative and quantitative research, and in MMR designs. Indeed, as with traditional qualitative or quantitative methodologies, best practices should be applied rigorously across the multiple methods, but also in the way the quantitative and qualitative strands are articulated. A particularly critical issue in this context is that of meta-inference, in which the qualitative and quantitative strands connect. Meta-inference should be carefully specified, and researchers should be aware of the challenges of interpreting conflicting results.

The application of MMR can also raise practical considerations, particularly as the integration of both qualitative and quantitative data can require additional resources and time. However, it should be noted that this additional burden can often be offset against the potential benefits of MMR, particularly where multiple insights support the investigation of complex research questions or small populations.

Finally, we acknowledge that some theoretical debate still exists on how - or even whether - quantitative and qualitative paradigms can be mixed, a debate typified by the ‘paradigm wars’ of the second half of the twentieth century [ 1 ]. Such challenges stem from differences in the underlying ontological and epistemological positions of positivism (that a single objective reality exists) and constructivism (that reality is a subjective construct and therefore multiple realities exist). Even though the MMR paradigm goes beyond simply mixing the quantitative and qualitative paradigms and builds a third path, some purists continue to question this third paradigm, considering the very nature of the qualitative and quantitative paradigms irreconcilable. However, as Maxwell and Mittapalli argue [ 22 ], there is an alternative position to positivism and constructivism – critical realism. Critical realists deny that we have any objective or certain knowledge of the world, and accept the possibility of alternative valid accounts of any phenomenon. They argue that all theories about the world are grounded in a particular perspective and worldview, and all knowledge is partial, incomplete, and fallible. As such, critical realism provides a philosophical stance that is compatible with MMR in that it acknowledges the methodological characteristics of both qualitative and quantitative research, and can facilitate communication and cooperation between the two. Against the background of this ongoing debate, it is clear from the growing literature that the acceptance of MMR is increasing as the health outcomes research community continues to promote and celebrate methodological diversity.

Conclusions and recommended reading

Two conditions appear critical for the continued development of credible and robust MMR. First, health outcomes researchers have the potential to learn about the different MMR methodologies and outline how MMR can be used to more thoroughly answer health outcomes research questions. This may include increasing knowledge about the underlying philosophy and history of MMR, examples of MMR research designs and principles, and the pros and cons of this approach above a purely qualitative or quantitative inquiry. To support this journey we provide a list of recommended texts which can form a starting point for the curious researcher.

Second, an open dialogue and collaboration between health outcome researchers with positivist or interpretivist leanings should be encouraged to prepare the ground for robust MMR. In this context, it will be possible to design health outcomes research studies in which the whole is greater than the sum of its parts and allow the research community to further the science through providing innovative solutions to our research challenges.

Abbreviations

Food and Drug Administration

International Society for Quality of Life Research

Mixed Methods Research

Patient-Reported Outcomes

Special Interest Group

Key: *Recommended reading for researchers interested in MMR

Denzin D. Moments, Mixed Methods, and Paradigm Dialogs. Qualitative Inquiry. 2010;16(6):419–427.*

Article   Google Scholar  

Creswell, J. W., & Designing, P. C. V. L. (2007). Conducting mixed methods research . Thousand Oaks, CA: Sage Publications *.

Google Scholar  

Tashakkori A, Teddlie C. Handbook of mixed Methods in social behavioral research. Thousand Oakc,CA: Sage Publications; 2003.*.

Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research . Thousand Oaks, CA: Sage Publications *.

Wiering, B., de Boer, D., & Delnoij, D. (2017). Patient involvement in the development of patient-reported outcome measures: A scoping review. Health Expect, 20 (1), 11–23.

Special Section on Mixed Methods Research. (2012). Qual Life Research, 21 (3), 1573–2649.

Roadmap to Patient-Focused Outcome Measurement in Clinical Trials: US Food and Drug Administration; 2015. Available from: https://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugDevelopmentToolsQualificationProgram/ucm370177.htm .

Tashakkori, A., & Editorial, C. J. W. (2007). The new era of mixed methods. Journal of Mixed Methods Research, 1 , 1–6.

Busija L, Nicholson G, Toombs M, Cinelli R, Easton C, Sanders K, et al. Developing Conceptual Model of the Role of Elders in the Wellbeing of Australian Indigenous Communities: Group Concept Mapping Study 23rd ISOQOL Annual conference; Copenhagen, Denmark2016.

Rosas, S. R., & Ridings, J. W. (2017). The use of concept mapping in measurement development and evaluation: Application and future directions. Eval Program Plann, 60 , 265–276.

Willgoss T. A Novel, Patient-Centered, Mixed Methods Approach to Identifying Relevant Concepts for Patient-Reported Outcome Measures and Facilitating Participant Engagement. 21st ISOQOL Annual Conference; Berlin, Germany2014.

Barbic S. Development and testing of the Personal Recovery Outcome Measure (PROM) for people with mental illness: Application of Mixed methods. 23rd ISOQOL Annual conference; Copenhagen, Denmark2016.

Blum SI, Bushnell DM, McCarrier KP, Martin ML, Cano S, Liedgens H, et al. The Pain Assessment for Lower Back Symptoms (PAL-S) and Impacts (PAL-I): A Case Example of the Application of Mixed-Methods in PRO Instrument Development. 23rd ISOQOL Annual Conference; Copenhagen, Denmark2016.

Cleanthous S. Why we should move Mixed Methods in Psychometric Research from a three-step to a “two-step”. 23rd ISOQOL Annual Conference; Copenhagen, Denmark2016.

Hudgens S. Application of mixed models for clinician reported outcome development 23rd ISOQOL annual conference; Copenhagen, Denmark2016.

Fofana F, Bonnaud-Antignac A, Regnault A. A mixed method approach to help demonstrate saturation in qualitative research: applying Partial Least Square regression to qualitative data. 1st Mixed Methods International Research Association (MMIRA) Annual Conference; Boston, MA2014.

Onwuegbuzie, A. J., Bustamante, R. M., & Nelson, J. A. (2010). Mixed research as a tool for developing quantitative instruments. Journal of Mixed Methods Research., 4 (1), 56–78 *.

Gelhorn, H. L., Kulke, M. H., O'Dorisio, T., Yang, Q. M., Jackson, J., Jackson, S., et al. (2016). Patient-reported symptom experiences in patients with carcinoid syndrome after participation in a study of Telotristat Etiprate: A qualitative interview approach. Clin Ther, 38 (4), 759–768.

Article   CAS   Google Scholar  

Marrel A, Fofana F, Guillemin I. Increasing the interpretability of patient-reported outcomes questionnaire findings using a mixed methods design: An example in a rare cardiac clinical trial. 19th ISPOR Annual European Congress; Vienna, Autria2016.

Patient-Centered Outcome Measures Initiatives in the Field of Rare Diseases. International Rare Diseases Research Consortium; 2016.

Kirwan, J. R., de Wit, M., Frank, L., Haywood, K. L., Salek, S., Brace-McDonnell, S., et al. (2017). Emerging guidelines for patient engagement in research. Value Health, 20 (3), 481–486.

Maxwell, J., & Mittapalli, K. (2010). Realism as a stance for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), SAGE handbook of mixed methods in social & behavioral research (pp. 145–168). Thousand Oaks, CA: SAGE Publications Ltd. https://doi.org/10.4135/9781506335193 .

Chapter   Google Scholar  

Download references

Acknowledgements

This paper was reviewed and endorsed by the membership of the International Society for Quality of Life Research (ISOQOL) Mixed Methods Research Special Interest Group. As part of this review, insightful comments from Benoit Arnould, Sophie Cleanthous, Chinmay Deshpande, and Claudia Marcela Vélez were implemented.

This paper was reviewed and endorsed by the International Society for Quality of Life Research (ISOQOL) Board of Directors as an ISOQOL publication and does not reflect an endorsement of the ISOQOL membership.

Author information

Authors and affiliations.

Modus Outcomes, 61, Cours de la Liberté, 69003, Lyon, France

Antoine Regnault

Roche Products Ltd., Welwyn Garden City, UK

Tom Willgoss

The University of British Columbia, Vancouver, Canada

Skye Barbic

You can also search for this author in PubMed   Google Scholar

Contributions

AR, TW and SB led the preparation of this manuscript on behalf of the International Society for Quality of Life Research (ISOQOL) Mixed Methods Research Special Interest Group (MMR SIG). AR and TW submitted the original idea to the MMR SIG. AR, TW and SB drafted the manuscript and coordinated the review by the MMR SIG membership and by the ISOQOL Board of Directors. All reviewed and approved the final version.

Corresponding author

Correspondence to Antoine Regnault .

Ethics declarations

Ethics approval and consent to participate, competing interests.

AR is an employee of Modus Outcomes. TW and SB declare that they have no conflict of interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Regnault, A., Willgoss, T., Barbic, S. et al. Towards the use of mixed methods inquiry as best practice in health outcomes research. J Patient Rep Outcomes 2 , 19 (2018). https://doi.org/10.1186/s41687-018-0043-8

Download citation

Received : 08 December 2017

Accepted : 03 April 2018

Published : 11 April 2018

DOI : https://doi.org/10.1186/s41687-018-0043-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

ORIGINAL RESEARCH article

A mixed methods research study of parental perception of physical activity and quality of life of children under home lock down in the covid-19 pandemic.

• 1 Transdisciplinary Research Center in Psychology, Autonomous University of the State of Morelos, Cuernavaca, Mexico
• 2 Institute of Psychology and Special Education, Department of Applied Psychology, University Center for Health Sciences, University of Guadalajara, Guadalajara, Mexico
• 3 Facultad de Ciencias Sociales, Universidad Europea de Canarias, La Orotava, Spain
• 4 Faculty of Psychology and Speech Therapy, Department of Clinical Psychology, Psychobiology and Methodology, University of La Laguna, Santa Cruz de Tenerife, Spain

Household confinement due to the rapid spread of the pandemic caused by COVID-19 has brought very significant changes, such as the forced stay-at-home of children due to the closure of schools. This has meant drastic changes in the organization of daily life and restrictions on their activities, including exercise, which could affect the quality of life of the children due to its importance. In order to study the relationship between physical activity and psychological well-being of minors, a study has been carried out with Mixed Methods Research, combining survey methodology with transversal design with qualitative methodology using discourse analysis. A total of 234 parents of minors in Spain and several Spanish-speaking countries in America participated. The instrument was a questionnaire in Google Forms, which included the Kidscreen-27 quality of life scale. The results show significant differences in both the type of physical activity and its frequency due to age, and differences in parents’ perception of whether their children’s physical activity levels were sufficient or not, both on the health, mood and school subscales, and in the categorization of open responses referring to concerns due to the pandemic, analyzed with the ALCESTE technique. The relationship between physical activity of children and adolescents and quality of life is clearly concluded.

Introduction

At the end of December 2019, the first evidence appeared in Wuhan, China, that a new lethal viral disease had emerged, for which no vaccine or specific medication was available. In March the disease became a pandemic and a large majority of countries, either with specific regulations or through recommendations to the population, established confinement and social distance as the possible solution to prevent further spread of the disease, to avoid saturation of hospitals and curb the lethality of the virus. On March 10th, the global situation with regard to COVID-19 was 113,702 confirmed cases (4,125 new) and 4,012 deaths (203 new) ( World Health Organization [WHO], 2020a ). On December 29, the number of confirmed cases worldwide was to over 79 million, with a cumulative death toll of over 1.7 million ( World Health Organization [WHO], 2020b ).

In the field of Psychology, theoretical formulations have been made to explain the reasons why COVID-19 evolved so rapidly and was so widely spread. Urzúa et al. (2020) point out three factors: (a) illusory optimism; (b) inadequate perception of absence of contingencies produced by the population’s behavior; (c) optimistic risk perception. Vera-Villarroel (2020) has stated that physical and mental health are closely linked, and explains the expansion of the pandemic based on three psychological processes: cognitive, with the population having irrational beliefs about the disease and illusory optimism; emotional, with feelings such as fear, stress and anger; behavioral, with exposure and risk behaviors. The author points out that these factors must be considered in the intervention to save lives.

Several studies have shown the risk that social isolation caused by the pandemic implies not only for the most exposed groups (health workers), but also for the mental health of the general population. Problems of anxiety ( Chew et al., 2020 ; Holmes et al., 2020 ; Wang and Zhao, 2020 ), stress and psychological distress have been reported, both during and even after the biodisaster ( Liu D. et al., 2020 ). Along the same lines, the narrative review conducted by Huarcaya-Victoria (2020) points out three types of problems for the general population: health anxiety, depression and stress. Rajkumar (2020) groups the problems derived from the pandemic into four sections: general population, health workers, vulnerable people and therapeutic strategies and interventions. The author emphasizes the need to study the effect of the situation generated by the pandemic on children and adolescents.

A particularly vulnerable group in this whole situation is children and adolescents. Although results in children for Coronavirus-19 disease are still inconsistent. Changes produced in their environment since COVID-19, such as the restrictions that home isolation and not being able to access the main areas of socialization ( Socías et al., 2020 ), with risks such as stress from both them and their parents, since COVID-19 can cause psychological alterations in children such as those caused by other stressors ( Espada et al., 2020 ; Socías et al., 2020 ).

Certain factors can have effects not only during confinement but also afterward, such as the disappearance of healthy habits like attending classes, which have been replaced by unhealthy behaviors, such as sedentary lifestyles, inappropriate diets, excessive use of screens which can produce, in addition to weight gain, physical problems ( Brazendale et al., 2017 ; Wu et al., 2017 ). From this follows the importance of understanding the effects that a wide variety of personal and contextual factors ( Holgado-Tello et al., 2010 ) can have on children and adolescents and their interaction in the way they experience physical activity and sports during the pandemic situation. Other risks that have been highlighted, depending on age, include substance abuse, accommodation issues and overcrowding and change and disruption of social networks ( Holmes et al., 2020 ). It is expected that, after confinement, in most cases these problems will disappear ( Barlett et al., 2020 ), although some may persist after the situation generated by the pandemic has passed ( Espada et al., 2020 ). Space restrictions and not being able to go outside are especially important in childhood for the proper development of playing, which is essential for its maturation process ( García-Serrano and García-Fernández, 2015 ).

In view of the difficult situation experienced, the population has been provided with recommendations, some of which have been generated by institutions to support their citizens ( Socías et al., 2020 ). These guidelines have many points in common: maintaining routines, being active, supporting minors, carrying out social activities, in short, maintaining a normal life in safety ( Liu J. J. et al., 2020 ). The support of parents is important, who can strengthen family ties and meet the needs of children through appropriate parenting styles ( Wang et al., 2020 ). The need for physical exercise is also stressed ( Holmes et al., 2020 ; Mera-Mamián et al., 2020 ; Romero et al., 2020 ). Physical exercise plays a relevant role both on a physical level ( Vidarte Claros et al., 2011 ) and in mental processes ( Ramírez et al., 2004 ; Zhou et al., 2020 ) as well as on a psychological level ( Berger and Motl, 2000 ; Biddle and Mutrie, 2001 ; Tessier et al., 2007 ; Anderson and Brice, 2011 ). In particular, there is clear evidence of the contribution of physical activity to psychological well-being ( Molina-García et al., 2007 ; Jiménez et al., 2008 ; Romero et al., 2009 ; Fernández Ozcorta et al., 2015 ).

The relationship between physical activity and well-being linked to the quality of life has been the subject of multiple investigations in recent years, which have also emphasized its influence on the general health of the various sectors of the population ( Schwartzmann, 2003 ; Bize et al., 2007 ; Anokye et al., 2012 ). In particular, different studies have highlighted the association between high levels of physical activity, or the practice of sports, and the quality of life in children and adolescents ( Anokye et al., 2012 ; Marker et al., 2018 ; Luna et al., 2019 ).

Likewise, recent reviews of studies on interventions focused on the promotion of sports practices and their impact on issues such as mental health, self-esteem, anxiety levels, and perception of well-being in children and adolescents, underline the benefits of this kind of activities for the general health of this population in particular, showing that physical-sport education pilot programs might promoted significant improvements in specific indicators of subjective well-being and emotional intelligence of participating adolescents’ groups ( Bermejo-Cantarero et al., 2017 ; Luna et al., 2019 ).

The lack of physical activity is a widely reported public health problem ( Bermejo-Cantarero et al., 2017 ). For this reason, evaluation that focuses on the relationships between physical activity and health-related quality of life is an important focus of research in this field. On the other hand, there is little research aimed at exploring parents’ knowledge and perceptions of their children’s physical activity, their ideas about its importance and impact on the way they experience diverse dimensions of a stressful life ( Gallego-Méndez et al., 2020 ; Spinelli et al., 2020 ; Yarımkaya and Esentürk, 2020 ) particularly during the Coronavirus outbreak. Exploring these issues, including the different perspectives of persons involved in families’ life ( Izquierdo-Sotorrío et al., 2016 ), could help provide recommendations and support programs for parents to guide their children’s physical activity.

In the case of children and adolescents, physical activity has important benefits: it promotes growth and enhances both physical development ( Rosa et al., 2018 ) and psychomotor, cognitive and social development, and generally favors all body systems: metabolism of carbohydrates and lipids, control of blood pressure, decreases the risk of type 2 diabetes and improves body composition ( Camargo Lemos and Ortiz Dallos, 2010 ).

Physical activity also favors psychological factors: it helps to build a balanced self-concept and improves self-perception, mood, self-image, physical self-concept, perception of health and life satisfaction, and intellectual function ( Camargo Lemos and Ortiz Dallos, 2010 ; Reigal-Garrido et al., 2012 ; Rosa, 2015 ).

The home quarantine imposed by the COVID-19 may make physical activity more difficult, and as we have seen in the studies reviewed, this leads to a decline in the quality of life of children and adolescents. Quality of life (QoL) is understood as personal satisfaction (or dissatisfaction) with the cultural or intellectual conditions in which an individual lives. Health is one of the domains of quality of life, this domain comprises not only physical health but also psychological health, as well as the interaction that people have with others and with the community ( Ravens-Sieberer et al., 2005 ). For this research, we are interested in reviewing the quality of life, based on the assessment of the well-being perceived by parents.

Given that the collection of high quality data is a priority in order to understand the psychological effects that the quarantine may have produced in the population, and that there is an urgent need to discover, analyze and evaluate the psychological interventions that could alleviate the problems generated and minimize the risks that could occur in the mental health of society ( Holmes et al., 2020 ), the aim of this research is to analyze parents’ perceptions of their children’s quality of life in relation to observed physical activity in the conditions of staying in the housing due to the pandemic situation due to the COVID-19. It hypothesizes the existence of greater quality of life perceived by parents who consider their children to be sufficiently physically active.

In this sense, we try to find out if there is any difference in quality of life between children of different ages and sex in the conditions of staying in the housing due to the pandemic situation due to the COVID-19 as perceived by mothers and fathers. In addition, it is investigated whether the characteristics of the housing (the space) conditioned the perception of the parents about their children doing more or less physical activity, and whether there are differences between the age and the type of physical exercise done. It is also interesting to know the relationship between the level of physical activity and psychological well-being.

Materials and Methods

Methodology and design.

This is a non-experimental design. Mixed methodology was used (Mixed Methods Research, MMR; Johnson and Onwuegbuzie, 2004 ; Denscombe, 2008 ). The data was collected through a cross-sectional design with survey methodology, using an ex post facto design, and there are open questions that allow a qualitative analysis.

To determine the differences in physical activity, three independent variables were considered: age (children, adolescents), sex (male, female), as well as a third variable, grouping parents according to their opinion about the physical activity developed by their children in confinement (sufficient, insufficient). The dependent variables used has been the different scales that make up the KIDSCREEN test, which therefore requires multivariate analysis.

Participants

A total of 234 participants responded to the survey. The average age was 42.82 (SD = 7.10), with a range between 24 and 65. More mothers (203) than fathers (30) participated, and only one of the informants was guardian of the minors, relative in charge of the child. Table 1 presents the data regarding age (values corresponding to the percentiles 25, 50, 75, and over 75) and educational level. The procedure for selecting the sample was one of convenience.

Table 1. Parents’ age and educational level.

Parents and caregivers were asked to think of one of their children when answering the questionnaires. In this way, for the data analysis, they were grouped by the ages of the children, the largest group being children between 8 and 11 years old, 125 (52 female) and 109 adolescents between 12 and 17 (54 female).

The countries of origin of the participants were mainly Spain (134, 57.3%), and Mexico (86, 36.9%) and others American countries (Panamá, Colombia, Argentina, and Chile; 13, 5.8%).

Most families (230, 98.3%) reported not having been victims of the coronavirus. Only four families had a confirmed patient in the family unit, and in four other cases there was a suspicion that a family member had the disease.

In the questionnaire, a question was included about family and housing conditions. Most of the sample lived in the same dwelling with up to four family members (167, 71.4%), while it was less frequent for the family size to be greater than four (67, 28.6%). The average number of rooms, discounting common services, such as kitchen, living room and bathroom, was 3 (113 of the participants, 48.3%), with a range between 1 and 10 rooms. Most of the dwellings have at least one exterior space (177 of the participants, 75.6%).

Instruments

A questionnaire was designed to obtain data on parents’ perceptions of their children’s physical activity, some specific data on the type of housing during their child’s confinement. This questionnaire consists of 18 questions (15 closed, 3 open-ended) distributed in the following categories: (1) descriptive data of the participants (6 items); (2) family and housing conditions (5 items); (3) issues related to the situation produced by the COVID-19 pandemic (3 items); (4) complaints and needs caused by the situation produced by the COVID-19 pandemic (4 items) (see Supplementary Data Sheet 1 ). At the end of the questionnaire it was mentioned that if they wanted to ask for the results of the research they could leave their e-mail. All questions were marked as mandatory in the Google form, so there was no room for incomplete or missing data.

For the HRQoL measure, the Kidscreen-27 Parent Questionnaire ( Ravens-Sieberer et al., 2005 ). Spanish version was used, once the authorization for its use in this study was requested and obtained. This is a questionnaire that assesses health-related quality of life. This questionnaire was used because it provides a parameter to contrast the perception of psychological and health well-being in the child population with the physical activity observed by the parents. It consists of 27 items, which are answered in a Likert-type scale of five alternatives (from nothing to very much), structured in five scales: physical activity (4 items), mood (7 items), family life (7 items), friends (4 items), and school (4 items), and a single question about your child’s general state of health in the last week. The test is filled in by parents, for children and adolescents between the ages of 8 and 18. The original authors ( Ravens-Sieberer et al., 2005 ) offer evidence of the factorial validity of the test and its reliability in all the subscales of the test, in terms of internal consistency, with the total Cronbach’s Alpha value equal to 0.82. With our data, a similar Alpha of 0.831 has been obtained.

The questionnaires were assembled in electronic format with the Google Forms application. It was sent out by email and through social networks (Whatsapp, Facebook, and Twitter) to contacts in different educational associations, using the snowball technique. It was sent during the month of May 2020 (it can be defined as the first period of confinement). Only one of both parents was asked to answer the questionnaire with one of their children in mind (in case they have two or more), and who was in the age range of 8–17 years. The time required to fill in the questionnaire was 15 to 20 min.

At the time of data collection, all participants (regardless of country) were in the same conditions of confinement, leaving the home only for essential activities, with restrictions on going to school, physical activities or recreation outside the home.

As far as ethical aspects are concerned, the Commission on Ethics in Research and Animal Welfare of the University of La Laguna (CEIBA) was asked to authorize the study, which was granted (Registration Number: CEIBA2020-0396). In the questionnaire, the corresponding information for the participants was set out in the Organic Law 3/2018, of December 5th, on Personal Data Protection and guarantee of digital rights ( BOE, 2018 ), guaranteeing the anonymity and confidentiality of the data.

Data Analysis

The relationship between parental consideration of physical activity sufficiency and having or not having outdoor space in the home was calculated using the V of Cramer.

To check the absence of univariate outliers, we used Tukey’s test that takes as reference the difference in interquartile range, considering a slight outlier at 1.5 times this distance, and extreme when it is at three times that distance. To determine the existence of multivariate outliers, the Mahalanobis distance was calculated.

Regarding quality of life, it was analyzed in two ways taking three independent variables: age, sex and parents’ assessment. Since the quality of life variable, measured by Kidscreen, is split into several scales, it requires a multivariate approach, so three MANOVAs were carried out, one according to each independent variable studied. All quantitative analyses were conducted with the SPPS program, v.21.

For the qualitative analysis, the phenomenological discourse analysis method was used, which identifies the meanings of language, through lexical analysis using the ALCESTE software (in French: Analyse des Lexèmes Coocurrents dans les Enoncés Simples d’un Texte ) ( Reinert, 2003 ). This program facilitates the analysis of linguistic materials that generally arise in social research, such as answers to open-ended questions in questionnaires, in-depth interviews or answers based on projective techniques ( De Alba, 2004 ). The ALCESTE methodology consists of three stages: the construction of the data matrix, the classification of the context units (statements) and the description of the classes ( Gil et al., 1994 ). The methodology focuses on the statistical distribution of word succession, taking into account only the simultaneous presence of several words in the same statement. In this way, classes are identified as semantic fields, represented in trees or dendograms. In the ALCESTE method, the initial text is broken down into elementary contextual units (ECUs), which approximately match the size of a sentence.

The statistical analysis, although limited to explain in detail the meaning of a text, allows the elaboration of a “cartography” of the lexical worlds chosen by the speaker to express himself and, therefore, of the reference systems from which he constructs his way of seeing reality ( Gil et al., 1994 ; Reinert, 2003 ).

Quantitative Analysis

Physical activity.

In order to know if there is a relation between the participant’s perception of the sufficiency or not of the physical activity developed by his or her child and the space dedicated to exercises, these variables were analyzed, considering in the household conditions whether there was no outdoor space to carry out activities or if, on the contrary, there was. The results are shown in Table 2 . There is significant dependence between both variables (V of Cramer = 0.146; p = 0.026).

Table 2. Perception of adequacy of physical activity and space for it.

Elimination of Outsiders

Eleven extreme univariate cases were eliminated and none multivariate by Mahalanobis distance, with the criterion of probabilities less than 0.001.

Psychological Well-Being by Age and Sex

Most parents consider their child’s health to be excellent (88, 39.5%) or very good (114, 51.1%), while only 21 (9.4%) rate it as “fair.”

The group was divided into two ages: from 8 to 11 (children) and 12 and older (adolescents). Table 3 shows the descriptive statistics.

Table 3. Age and sex level descriptive statistics.

To know if there are differences by sex and age, a MANOVA was calculated, which was for sex (Wilk’s λ = 0.949, F 5 . 215 = 2.3, p = 0.046, Partial η 2 = 0.051), and for age (Wilk’s λ = 0.843, F 5 . 215 = 8.034, p = 0.001, Partial η 2 = 0.157) nor for interaction (Wilk’s λ = 0.982, F 5 . 215 = 0.796, p = 0.554, Partial η 2 = 0.018). Individual ANOVA results are only significant for the variable age in the health scale ( F 1 , 219 = 7.692, p = 0.006, Partial η 2 = 0.034), with a small effect size and in the friend one ( F 1 , 219 = 28.421, p < 0.001, Partial η 2 = 0.115), with a large effect size.

Physical Activity and Well-Being

In order to assess whether the children developed adequate physical activity, the parents were asked whether they considered it sufficient or insufficient. A total of 146 considered it to be insufficient and 77 sufficient. The informants were divided into two groups according to this variable and it was analyzed whether there were significant differences in their assessment of the psychological well-being of the children. Table 4 presents the mean values and standard deviations of each welfare scale.

Table 4. Descriptive statistics of physical activity and well-being.

The result of the MANOVA was significant (Wilk’s λ = 0.743, F 5 , 217 = 15.001, p < 0.001, Partial η 2 = 0.257). Individual ANOVA results are only significant for the health scale ( F 1 , 223 = 64.821, p < 0.001, Partial η 2 = 0.227), with a large effect size.

Qualitative Analysis

In order to find out the perceptions that families have regarding different aspects of stay-at-home confinement, both required by law and recommended, four open-ended questions were analyzed by ALCESTE, separating into two samples parents who considered that their children were getting enough exercise and those who thought it was insufficient: (a) Explain why you say you have sufficient or insufficient physical activity; (b) How did your child live it?; and (c) What or who does your child miss?

Analysis of the Question “Explain Why You Have Sufficient or Insufficient Physical Activity”

The analysis of ALCESTE, for the group of parents who consider that their children have sufficient physical activity (see Figure 1 ), the results are grouped into three classes, which explain 66% of textual units. The first class is linked to the link between the second and third classes. The most representative class is 1, as it groups the largest number of EUs. The details of the analysis, in terms of class name, UCEs grouped and percentage involved, most representative word and examples, are presented in Table 5 .

Figure 1. Dendogram corresponding to the question “Explain why you have sufficient physical activity.”

Table 5. Analysis of the question “Explain why you have sufficient or insufficient physical activity.”

The reasons given by parents for considering that their children could not get enough physical activity are more dispersed, as they have been grouped into six clases (see Figure 2 ). In this case, there are two groupings: on the one hand, class 2 connects with the union of classes 5 and 6, while class 1 connects with the link between classes 3 and 4. Classes 1, 5, and 6 are related to the impossibility of doing either exercise or sports that they did before the pandemic, while the difficulties of the other set of classes go in the direction of lack of space and the need to go outside.

Figure 2. Dendogram corresponding to the question “Explain why you have insufficient physical activity.”

Analysis of the Question “How Did Your Child Live Not Being Able to go Out on the Street?”

The analysis of the group that considers that their son or daughter has had enough activity explains 51% of the text corpus. The dendogram is shown in Figure 3 .

Figure 3. Dendogram of the question “How did your child live not being able to go outside?” Sufficient physical activity.

On the other hand, in the group of parents who consider the activity performed by their children insufficient, although it explains only 27% of the corpus, extracting only two classes ( Figure 4 ).

Figure 4. Dendogram of the question: “How did your child live not being able to go outside?” Insufficient physical activity.

Table 6 shows the detail of the classes, in terms of their name, number of UCEs they group, percentage of the corpus they explain and the most representative word, as well as representative examples of each class. In both groups, a distinction is made between positive aspects, of being at home, or pointing out some kind of problem.

Table 6. Information from the analysis to the question “How did your child live not being able to go out on the street?”

Analysis of the Question “What or Who Does Your Child Miss?”

The analysis of this question, for the group of parents who consider that the physical activity developed by their son or daughter is sufficient, gives two classes, which explain 65% of the textual units, that is, an average relevance of the treatment (see Figure 5 ). These are two antagonistic classes: the second is the one that groups the most textual units (71.70%), where it is clear that the child misses both the extended family and the people in his or her school environment. The first class includes those who responded that they have not missed anything or anyone and is quite homogeneous: they do not miss anyone (see Table 7 ).

Figure 5. Dendogram of the answers: What or who does your child miss? Sufficient physical activity.

Table 7. Information from the analysis to the question “What or who does your child miss?”

In the case of parents who feel that their son or daughter does not get enough physical activity, there are six classes, with a grouping of classes on a ladder: from class 1 to 4 are connected individually, linking class 5 with 6.

It explains 72% of the textual units, which means that the relevance of the treatment is high. They are presented in Figure 6 .

Figure 6. Dendogram of the answers: “What or who does your child miss?” Insufficient physical activity.

The first thing to note is that the data collection was done in the months of April and early May, at the time of the most severe confinement, as in Spain, Mexico, Panama, and Argentina ( BBC, 2020 ). It is true that the regulatory conditions regarding confinement have differed in the countries where the participants in this research live, in some cases, such as Spain, being obligatory by the State of Alarm, while in other countries governments strongly recommended avoiding going out and staying at home. This has meant 24-hour family life, with parents having to telework and children being taught online. The possibilities of exercising under these conditions have been very limited, which can have important effects on the psychological well-being of the minors.

As far as the health of their children is concerned, a large majority consider it to be good or very good. Furthermore, taking into account the five scales of quality of life in relation to age levels (children and adolescents), parents value the health of their children more the younger they are. In contrast, differences in contact with friends score higher for adolescents.

Parents’ perceptions of their children’s quality of life significant differences are observed with respect to sex at the global level, which is not maintained in the scales separately, but they do differ by age on two of the instrument’s scales: health, where younger children score higher, and friends, with the opposite result, as would be expected: adolescents score significantly higher on this scale.

Physical activity is conditioned by the type of housing. The results show that when there is no outdoor space to develop physical activity, parents find that exercise performed by their children is insufficient more often.

The objective of this study, to establish whether there is a relationship between physical activity and psychological quality of life in the conditions of confinement at home from the parents’ perspective, has been clearly corroborated, both in quantitative and qualitative analyses, finding differences between the two established groups of participants: those who considered that their children could develop sufficient physical exercise versus those who thought it was insufficient. Divergences are shown in both groups at the quantitative and qualitative levels.

With respect to the quality of life instrument, there are significant differences between the overall scores of the two groups; however, significant differences are only found in the health scale; when parents consider that the physical activity developed by their children is insufficient, lower scores are obtained in that scale. These results support the hypothesis of a positive relationship between quality of life and physical activity.

The differences found between the two groups of parents (those who consider their children’s physical activity sufficient and those who do not) in the quantitative analyses are also verified in the qualitative ones. The second group of parents shows more dispersion in the open responses given, as well as greater concern.

Thus, in the first open question analyzed qualitatively, “ Explain why you have sufficient or insufficient physical activity ,” the discourse of some parents differs significantly, as it is obvious, since the reasons they give for the physical activity done by their children being sufficient must be differentiated from those who consider it to be insufficient. In the latter, two perspectives are clearly distinguished in the two branches that appear in the dendogram: lack of space or impossibility of doing the exercise they would like to do. Moreover, it also confirms what has already been commented, that is, how there is a relationship between physical space and the facilities of households to exercise is related to the satisfaction or dissatisfaction with the physical activity performed by their children.

The second question, centered on their child’s experiences of not being able to go out, parents who feel their children have enough physical activity, report that their children experienced the lock down positively. On the other hand, in the other group there is a division of opinions: one part considers that their children lived the lock down without problems, but others think that their children lived it with stress, being this last one the most representative class. It confirms again a greater decline in the quality of life of their children for this group.

Finally, in the question relating to whether their child misses something or someone, there is greater variability among the children whose parents consider they do not have enough activity, since the answers are grouped together in one more class, where there is content where school life is missing.

The limitations of this work are about convenience samples, since there is no guarantee of absence of selection bias. However, having included several countries, all of them with a significant restriction on going out of the house, it gives indications of cross validity. This unusual development of the pandemic has evened out the differences between nations in a common struggle against an unprecedented biological crisis.

As far as the uncertainty of living under what has come to be called the new normality, together with the certainty that the threat of the pandemic is not over and that outbreaks, more or less virulent, may occur, it is particularly relevant to carry out research on mental health and psychological well-being, in order to be able to foresee more precisely the actions to be taken, knowing the dangers involved. Holmes et al. (2020) point out how important it is to accumulate experience based on the evidence that has provided the lessons learned so that those in power can coordinate measures that will damage the lives of citizens as little as possible, especially those who are most vulnerable. In this regard, since children are a vulnerable sector of the population, knowledge of their reactions and how they have been affected is particularly relevant. For future research, this could also include children’s self-report, comparing their perception with their mothers and fathers’s ( Izquierdo-Sotorrío et al., 2016 ). As a general recommendation in the light of the data collected, emphasizing the importance of exercise in guaranteeing the psychological well-being of minors is vital and must be conveyed to the population.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by The University of La Laguna’s Ethics Committee of Research and Animal Welfare has approved this research (Registration Number: CEIBA2020-0396). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

ÁB, GL-A, MV, and DC-S had participated in theoretical review. ÁB, ER-N, GL-A, DC-S, and MV had participated in research design and instrument. ÁB had participated in the data analysis. ÁB, ER-N, GL-A, DC-S, and MV had participated in discussion. ÁB, ER-N, GL-A, DC-S, MV, and TA had participated in the study planning, writing, and revision of the article. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank all the families that have participated in this research.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2021.649481/full#supplementary-material

Anderson, R. J., and Brice, S. (2011). The mood-enhancing benefits of exercise: memory biases augment the effect. Psychol. Sport Exerc. 12, 79–82. doi: 10.1016/j.psychsport.2010.08.003

CrossRef Full Text | Google Scholar

Anokye, N. K., Trueman, P., Green, C., Pavey, T. G., and Taylor, R. S. (2012). Physical activity and health related quality of life. BMC Public Health 12:624. doi: 10.1186/1471-2458-12-624

PubMed Abstract | CrossRef Full Text | Google Scholar

Barlett, J. D., Griffin, J., and Thomson, D. (2020). Resources for Supporting Children’s Emotional Well-Being During the COVID-19 Pandemic. Available online at: https://www.childtrends.org/publications/resources-for-supporting-childrens-emotional-well-being-during-the-covid-19-pandemic [Accessed March 19, 2020]

Google Scholar

BBC (2020). Coronavirus: Qué tan Rápidamente se Expande el Covid-19 Por América Latina (y Cómo se Compara con Otras Regiones Del Mundo) [Coronavirus: How Fast Covid-19 is Spreading in Latin America (and How it Compares to Other Regions of the World)]. Available online at: https://www.bbc.com/mundo/noticias-america-latina-51802906 [Accessed June 20, 2020]

Berger, B. G., and Motl, R. W. (2000). Exercise and mood: a selective review and synthesis of research employing the Profile of mood states. J. Appl. Sport Psychol. 12, 69–92. doi: 10.1080/10413200008404214

Bermejo-Cantarero, A., Álvarez-Bueno, C., Martínez-Vizcaino, V., García-Hermoso, A., Torres-Costoso, A. I., and Sánchez-López, M. (2017). Association between physical activity, sedentary behavior, and fitness with health related quality of life in healthy children and adolescents: a protocol for a systematic review and meta-analysis. Medicine 96:e6407. doi: 10.1097/MD.0000000000006407

Biddle, S. J. H., and Mutrie, N. (2001). Psychology of Physical Activity: Determinants, Well-Being and Interventions. London: Routledge.

Bize, R., Johnson, J. A., and Plotnikoff, R. C. (2007). Physical activity level and health-related quality of life in the general adult population: a systematic review. Prev. Med. 45, 401–415. doi: 10.1016/j.ypmed.2007.07.017

BOE (2018). BOE (2018, n° 294 of December 6th) Boletín Oficial del Estado. Available online at: https://www.boe.es/boe/dias/2018/12/06/pdfs/BOE-S-2018-294.pdf [Accessed June 20, 2020]

Brazendale, K., Beets, M. W., Weaver, R. G., Pate, R. R., Turner-McGrievy, G. M., Kaczynski, A. T., et al. (2017). Understanding differences between summer vs. school obesogenic behaviors of children: the structured days hypothesis. Int. J. Behav. Nutr. Phys. Act. 14:100.

Camargo Lemos, D. M., and Ortiz Dallos, C. J. (2010). Actividad física en niños y adolescentes: determinantes y medición. Rev. Univ. Ind. Santander. Salud. 42, 153–165.

Chew, Q. H., Wei, K. C., Vasoo, S., Chua, H. C., and Sim, K. (2020). Narrative synthesis of psychological and coping responses towards emerging infectious disease outbreaks in the general population: practical considerations for the COVID-19 pandemic. Singapore Med. J. 61, 1–31. doi: 10.11622/smedj.2020046

De Alba, M. (2004). El método ALCESTE y su aplicación al estudio de las representaciones sociales del espacio urbano: el caso de la ciudad de México. Papers Soc. Representations 13, 1.1–1.20.

Denscombe, M. (2008). A Research Paradigm for the Mixed Methods Approach. J. Mix. Methods Res. 2, 270–283. doi: 10.1177/1558689808316807

Espada, J. P., Orgilés, M., Piqueras, J. A., and Morales, A. (2020). Las buenas prácticas en la atención psicológica infanto-juvenil ante el COVID-19. Clín. Salud. 31, 109–113. doi: 10.5093/clysa2020a14

Fernández Ozcorta, E. J., Almagro Torres, B. J., and Sáenz-López Buñuel, P. (2015). Inteligencia emocional percibida y el bienestar psicológico de estudiantes universitarios en función del nivel de actividad física. Cult. Cienc. Deporte 10, 31–39. doi: 10.12800/ccd.v10i28.513

Gallego-Méndez, J., Pérez-Gómez, J., Calzada-Rodríguez, J. I., Denche-Zamorano, ÁM., Mendoza-Muñoz, M., Carlos-Vivas, J., et al. (2020). Relationship between health-related quality of life and physical activity in children with hyperactivity. Int. J. Environ. Res. Public Health 17:2804. doi: 10.3390/ijerph17082804

García-Serrano, P., and García-Fernández, J. A. (2015). “La rueda del hámster. Crítica a la especialización y confinamiento infantil en el espacio público urbano,” in Proceedings of the 2nd Conference on Urban Processes. Facultad de CC Políticas y Sociología , (Madrid: Universidad Complutense de Madrid).

Gil, J., García, E., and Rodríguez, E. (1994). Una metodología para el análisis estadístico de datos textuales: el programa Alceste. Rev. Investig. Educ. 24, 107–124.

Holgado-Tello, F., Navas, L., López, M., and García, T. (2010). A structural model of goal orientation in sports: personal and contextual variables. Span. J. Psychol. 13, 257–266. doi: 10.1017/S1138741600003838

Holmes, E. A., O’Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., et al. (2020). Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry 7, 547–560. doi: 10.1016/S2215-0366(20)30168-1

Huarcaya-Victoria, J. (2020). Consideraciones sobre la salud mental en la pandemia de COVID-19. Rev. Peru. Med. Exp. Salud Pública 37, 327–334. doi: 10.17843/rpmesp.2020.372.5419

Izquierdo-Sotorrío, E., Holgado-Tello, F. P., and Carrasco, M. Á (2016). Incremental validity and informant effect from a multi-method perspective: assessing relations between parental acceptance and children’s behavioral problems. Front. Psychol. 7:664. doi: 10.3389/fpsyg.2016.00664

Jiménez, M. G., Martínez, P., Miró, E., and Sánchez, A. I. (2008). Bienestar psicológico y hábitos saludables: ¿están asociados a la práctica de ejercicio físico? Inter. J. Clin. Health Psychol. 8, 185–202.

Johnson, R. B., and Onwuegbuzie, A. J. (2004). Mixed methods research: a research paradigm whose time has come. Educ. Res. 33, 14–26. doi: 10.3102/0013189X033007014

Liu, D., Ren, Y., Yan, F., Li, Y., Xu, X., Yu, X., et al. (2020). Psychological impact and predisposing factors of the coronavirus disease 2019 (COVID-19) pandemic on general public in China. Lancet Psychiatry doi: 10.2139/ssrn.3551415 [Epub ahead of print].

Liu, J. J., Bao, Y., Huang, X., Shi, J., and Lu, L. (2020). Mental health considerations for children quarantined because of COVID-19. Lancet Child Adolesc. Health 4, 347–349.

Luna, P., Guerrero, J., and Cejudo, J. (2019). Improving adolescents’ subjective well-being, trait emotional intelligence and social anxiety through a programme based on the sport education model. Int. J. Environ. Res. Public Health 16:1821. doi: 10.3390/ijerph16101821

Marker, A. M., Steele, R. G., and Noser, A. E. (2018). Physical activity and health-related quality of life in children and adolescents: a systematic review and meta-analysis. Health Psychol. 37, 893–903. doi: 10.1037/hea0000653

Mera-Mamián, A. Y., Tabares González, E., Montoya-González, S., Muñoz Rodríguez, D., and Monsalve, F. (2020). Recomendaciones prácticas para evitar el desacondicionamiento físico durante el confinamiento por la pandemia asociada a COVID-19. Universidad Salud. 22, 166–177. doi: 10.22267/rus.202202.188

Molina-García, J., Castillo, I., and Pablos, C. (2007). Bienestar psicológico y práctica deportiva en universitarios. Motricidad. Eur. J. Hum. Mov. 18, 79–91.

Rajkumar, R. P. (2020). COVID-19 and mental health: a review of the existing literature. Asian J. Psychiatry 52:102066. doi: 10.1016/j.ajp.2020.102066

Ramírez, R., Vinaccia, S., and Suárez, G. R. (2004). El impacto de la actividad física y el deporte sobre la salud, la cognición, la socialización y el rendimiento académico: una revisión teórica. Rev. Estudios Soc. 18, 67–75. doi: 10.7440/res18.2004.06

Ravens-Sieberer, U., Gosch, A., Rajmil, L., Erhart, M., Bruil, J., Dür, W., et al. (2005). KIDSCREEN-52 Medida de la calidad de vida de los niños y adolescentes. Rev. Exp. Farmacoeconomía Resultados Invest. 5, 353–364.

Reigal-Garrido, R., Videra García, A., Parra Flores, J. L., and Juárez Ruiz de Mier, R. (2012). Actividad Físico Deportiva, Autoconcepto Físico y Bienestar Psicológico en la Adolescencia. Retos: Nuevas tendencias en Educación Física, Deporte y Recreación. 22, 19–23.

Reinert, M. (2003). Alceste Users’ Manual.(English Version). Touluse: Image.

Romero, A., García-Mas, A., and Brustad, R. (2009). Estado del arte y perspectiva actual del concepto de bienestar psicológico en psicología del deporte. Rev. Latinoam. Psicol. 41, 335–347.

Romero, C. C., del Mazo Fuente, A., Besada, M. D., and Hernández, M. D. L. O. R. (2020). Algunas aportaciones de la orientación educativa en red durante el confinamiento por la pandemia de COVID-19. Rev. AOSMA, N° Extra 28, 33–49. Available online at: https://dialnet.unirioja.es/descarga/articulo/7381631.pdf

Rosa, A. (2015). Actividad Física y Autoconcepto: Una Revisión Teórica Aplicada al Ámbito Escolar. Available online at: https://www.efdeportes.com/efd202/actividad-fisica-y-autoconcepto.htm (accessed December 31, 2020).

Rosa, A., García-Cantó, E., and Carrillo, P. J. (2018). Percepción de salud, actividad física y condición física en escolares. Cuad. Psicol. Deport. 18, 179–189.

Schwartzmann, L. (2003). Calidad de vida relacionada con la salud: aspectos conceptuales. Cienc. Enferm. 9, 09–21. doi: 10.4067/S0717-95532003000200002

Socías, C. O., Ballester Brage, L. L., and Nevot-Caldentey, L. L. (2020). Apoyo Familiar Ante el Covid-19 en España [Family Support Against Covid-19 in Spain]. Available online at: https://doi.org/10.1590/SciELOPreprints.297 [Accessed March 3, 2020].

Spinelli, M., Lionetti, F., Pastore, M., and Fasolo, M. (2020). Parents’ stress and children’s psychological problems in families facing the COVID-19 outbreak in Italy. Front. Psychol. 11:1713. doi: 10.3389/fpsyg.2020.01713

Tessier, S., Vuillemin, A., Bertrais, S., Boini, S., Le Bihan, E., Oppert, J. M., et al. (2007). Association between leisure-time physical activity and health-related quality of life changes over time. Prev. Med. 44, 202–208. doi: 10.1016/j.ypmed.2006.11.012

Urzúa, A., Vera-Villarroel, P., Caqueo-Urízar, A., and Polanco-Carrasco, R. (2020). La Psicología en la prevención y manejo del COVID-19. Aportes desde la evidencia inicial. Terapia Psicol. 38, 103–118.

Vera-Villarroel, P. (2020). Psicología y covid-19: un análisis desde los procesos psicológicos básicos. Panam. J. Neuropsychol. 14, 10–18.

Vidarte Claros, J. A., Vélez, C., Sandoval, C., and Alfonso, M. L. (2011). Actividad física: estrategia de promoción de la salud. Rev. Hacia Promoción Salud. 16, 202–218.

Wang, C., and Zhao, H. (2020). The impact of COVID-19 on anxiety in Chinese university students. Front. Psychol. 11:1168. doi: 10.3389/fpsyg.2020.01168

Wang, G., Zhang, Y., Zhao, J., Zhang, F., and Jiang, F. (2020). Mitigate the effects of home confinement on children during the COVID-19 outbreak. Lancet 395, 945–947. doi: 10.1016/S0140-6736(20)30547-X

World Health Organization [WHO] (2020a). Coronavirus Disease 2019 (COVID-19) Situation Report – 50. Available online at: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200310-sitrep-50-covid-19.pdf?sfvrsn=55e904fb_2 . [Accessed Mars 10, 2020]

World Health Organization [WHO] (2020b). Weekly Epidemiological Update - 29 December 2020. Available online at: https://www.who.int/publications/m/item/weekly-epidemiological-update—29-december-2020 [Accessed December 31, 2020]

Wu, X. Y., Han, L. H., Zhang, J. H., Luo, S., Hu, J. W., and Sun, K. (2017). The influence of physical activity, sedentary behavior on health-related quality of life among the general population of children and adolescents: a systematic review. PLoS One 12:e0187668. doi: 10.1371/journal.pone.0187668

Yarımkaya, E., and Esentürk, O. K. (2020). Promoting physical activity for children with autism spectrum disorders during Coronavirus outbreak: benefits, strategies, and examples. Int. J. Dev. Disabil. 1–6. doi: 10.1080/20473869.2020.1756115

Zhou, F., Xi, X., and Qin, C. H. (2020). Regular open-skill exercise generally enhances attentional resources related to perceptual processing in young males. Front. Psychol. 11:941. doi: 10.3389/fpsyg.2020.00941

Keywords : home lock down, physical activity, quality of life, pandemic, mixed methods research

Citation: López-Aymes G, Valadez MD, Rodríguez-Naveiras E, Castellanos-Simons D, Aguirre T and Borges Á (2021) A Mixed Methods Research Study of Parental Perception of Physical Activity and Quality of Life of Children Under Home Lock Down in the COVID-19 Pandemic. Front. Psychol. 12:649481. doi: 10.3389/fpsyg.2021.649481

Received: 04 January 2021; Accepted: 16 February 2021; Published: 15 March 2021.

Reviewed by:

Copyright © 2021 López-Aymes, Valadez, Rodríguez-Naveiras, Castellanos-Simons, Aguirre and Borges. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Elena Rodríguez-Naveiras, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Mixed Methods Research | Definition, Guide & Examples

Mixed Methods Research | Definition, Guide & Examples

Published on August 13, 2021 by Tegan George . Revised on June 22, 2023.

Mixed methods research combines elements of quantitative research and qualitative research in order to answer your research question . Mixed methods can help you gain a more complete picture than a standalone quantitative or qualitative study, as it integrates benefits of both methods.

Mixed methods research is often used in the behavioral, health, and social sciences, especially in multidisciplinary settings and complex situational or societal research.

• To what extent does the frequency of traffic accidents ( quantitative ) reflect cyclist perceptions of road safety ( qualitative ) in Amsterdam?
• How do student perceptions of their school environment ( qualitative ) relate to differences in test scores ( quantitative ) ?
• How do interviews about job satisfaction at Company X ( qualitative ) help explain year-over-year sales performance and other KPIs ( quantitative ) ?
• How can voter and non-voter beliefs about democracy ( qualitative ) help explain election turnout patterns ( quantitative ) in Town X?
• How do average hospital salary measurements over time (quantitative) help to explain nurse testimonials about job satisfaction (qualitative) ?

Table of contents

When to use mixed methods research, mixed methods research designs, advantages of mixed methods research, disadvantages of mixed methods research, other interesting articles, frequently asked questions.

Mixed methods research may be the right choice if your research process suggests that quantitative or qualitative data alone will not sufficiently answer your research question. There are several common reasons for using mixed methods research:

• Generalizability : Qualitative research usually has a smaller sample size , and thus is not generalizable. In mixed methods research, this comparative weakness is mitigated by the comparative strength of “large N,” externally valid quantitative research.
• Contextualization: Mixing methods allows you to put findings in context and add richer detail to your conclusions. Using qualitative data to illustrate quantitative findings can help “put meat on the bones” of your analysis.
• Credibility: Using different methods to collect data on the same subject can make your results more credible. If the qualitative and quantitative data converge, this strengthens the validity of your conclusions. This process is called triangulation .

As you formulate your research question , try to directly address how qualitative and quantitative methods will be combined in your study. If your research question can be sufficiently answered via standalone quantitative or qualitative analysis, a mixed methods approach may not be the right fit.

But mixed methods might be a good choice if you want to meaningfully integrate both of these questions in one research study.

Keep in mind that mixed methods research doesn’t just mean collecting both types of data; you need to carefully consider the relationship between the two and how you’ll integrate them into coherent conclusions.

Mixed methods can be very challenging to put into practice, and comes with the same risk of research biases as standalone studies, so it’s a less common choice than standalone qualitative or qualitative research.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

There are different types of mixed methods research designs . The differences between them relate to the aim of the research, the timing of the data collection , and the importance given to each data type.

As you design your mixed methods study, also keep in mind:

• Your research approach ( inductive vs deductive )
• Your research questions
• What kind of data is already available for you to use
• What kind of data you’re able to collect yourself.

Here are a few of the most common mixed methods designs.

Convergent parallel

In a convergent parallel design, you collect quantitative and qualitative data at the same time and analyze them separately. After both analyses are complete, compare your results to draw overall conclusions.

• On the qualitative side, you analyze cyclist complaints via the city’s database and on social media to find out which areas are perceived as dangerous and why.
• On the quantitative side, you analyze accident reports in the city’s database to find out how frequently accidents occur in different areas of the city.

In an embedded design, you collect and analyze both types of data at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.

This is a good approach to take if you have limited time or resources. You can use an embedded design to strengthen or supplement your conclusions from the primary type of research design.

Explanatory sequential

In an explanatory sequential design, your quantitative data collection and analysis occurs first, followed by qualitative data collection and analysis.

You should use this design if you think your qualitative data will explain and contextualize your quantitative findings.

Exploratory sequential

In an exploratory sequential design, qualitative data collection and analysis occurs first, followed by quantitative data collection and analysis.

You can use this design to first explore initial questions and develop hypotheses . Then you can use the quantitative data to test or confirm your qualitative findings.

“Best of both worlds” analysis

Combining the two types of data means you benefit from both the detailed, contextualized insights of qualitative data and the generalizable , externally valid insights of quantitative data. The strengths of one type of data often mitigate the weaknesses of the other.

For example, solely quantitative studies often struggle to incorporate the lived experiences of your participants, so adding qualitative data deepens and enriches your quantitative results.

Solely qualitative studies are often not very generalizable, only reflecting the experiences of your participants, so adding quantitative data can validate your qualitative findings.

Method flexibility

Mixed methods are less tied to disciplines and established research paradigms. They offer more flexibility in designing your research, allowing you to combine aspects of different types of studies to distill the most informative results.

Mixed methods research can also combine theory generation and hypothesis testing within a single study, which is unusual for standalone qualitative or quantitative studies.

Mixed methods research is very labor-intensive. Collecting, analyzing, and synthesizing two types of data into one research product takes a lot of time and effort, and often involves interdisciplinary teams of researchers rather than individuals. For this reason, mixed methods research has the potential to cost much more than standalone studies.

Differing or conflicting results

If your analysis yields conflicting results, it can be very challenging to know how to interpret them in a mixed methods study. If the quantitative and qualitative results do not agree or you are concerned you may have confounding variables , it can be unclear how to proceed.

Due to the fact that quantitative and qualitative data take two vastly different forms, it can also be difficult to find ways to systematically compare the results, putting your data at risk for bias in the interpretation stage.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

Triangulation in research means using multiple datasets, methods, theories and/or investigators to address a research question. It’s a research strategy that can help you enhance the validity and credibility of your findings.

Triangulation is mainly used in qualitative research , but it’s also commonly applied in quantitative research . Mixed methods research always uses triangulation.

These are four of the most common mixed methods designs :

• Convergent parallel: Quantitative and qualitative data are collected at the same time and analyzed separately. After both analyses are complete, compare your results to draw overall conclusions. 
• Embedded: Quantitative and qualitative data are collected at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.
• Explanatory sequential: Quantitative data is collected and analyzed first, followed by qualitative data. You can use this design if you think your qualitative data will explain and contextualize your quantitative findings.
• Exploratory sequential: Qualitative data is collected and analyzed first, followed by quantitative data. You can use this design if you think the quantitative data will confirm or validate your qualitative findings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). Mixed Methods Research | Definition, Guide & Examples. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/mixed-methods-research/

Is this article helpful?

Tegan George

Other students also liked, writing strong research questions | criteria & examples, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, what is your plagiarism score.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Taking a critical stance towards mixed methods research: A cross-disciplinary qualitative secondary analysis of researchers’ views

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing

* E-mail: [email protected] , [email protected]

Affiliation Department of Psychology and Education, Universitat Oberta de Catalunya, Barcelona, Spain

Roles Conceptualization, Methodology, Validation, Writing – review & editing

Affiliation Department of Education, Universidad del Norte, Barranquilla, Colombia

Roles Conceptualization, Formal analysis, Methodology, Validation, Writing – review & editing

Affiliation Department of Management, Universidad de Alicante, Alicante, Spain

Roles Conceptualization, Validation, Writing – review & editing

Affiliation Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), University College London, London, United Kingdom

Roles Conceptualization, Supervision, Writing – review & editing

Affiliation Centre d’Estudis Sociologics sobre la Vida Quotidiana i el Treball (Sociological Research Centre on Everyday Life and Work—QUIT), Universitat Autonoma de Barcelona, Barcelona, Spain

• Sergi Fàbregues, 
• Elsa Lucia Escalante-Barrios, 
• José Francisco Molina-Azorin, 
• Quan Nha Hong, 
• Joan Miquel Verd

• Published: July 9, 2021
• https://doi.org/10.1371/journal.pone.0252014
• Peer Review
• Reader Comments

Recent growth and institutionalization in the field of mixed methods research has provided fertile ground for a wide range of thoughtful criticism of how this research approach has been developed and conceptualized by some members of the mixed methods community. This criticism reflects the increasing maturity of the field as well as the different theoretical perspectives and methodological practices of researchers in different disciplines. While debates addressing these criticisms are likely to lead to valuable insights, no empirical studies have been carried out to date that have investigated researchers’ critical views on the development and conceptualization of mixed methods research. This study examines the criticisms of the mixed methods field raised by a cross-national sample of researchers in education, nursing, psychology, and sociology. We carried out a secondary analysis of semi-structured interviews with 42 researchers and identified 11 different criticisms, which we classified in four domains: essence of mixed methods, philosophy, procedures, and politics. The criticisms related to the procedures domain were equally distributed among the four disciplines, while those related to the essence, philosophy and politics domains were more common among sociologists. Based on our findings, we argue that the divergence of views on foundational issues in this field reflects researchers’ affiliation to different communities of practice, each having its own principles, values, and interests. We suggest that a greater awareness of this divergence of perspectives could help researchers establish effective collaboration and anticipate potential challenges when working with researchers having different methodological approaches.

Citation: Fàbregues S, Escalante-Barrios EL, Molina-Azorin JF, Hong QN, Verd JM (2021) Taking a critical stance towards mixed methods research: A cross-disciplinary qualitative secondary analysis of researchers’ views. PLoS ONE 16(7): e0252014. https://doi.org/10.1371/journal.pone.0252014

Editor: Adam T. Perzynski, The MetroHealth System and Case Western Reserve University, UNITED STATES

Received: July 24, 2020; Accepted: May 7, 2021; Published: July 9, 2021

Copyright: © 2021 Fàbregues et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Data cannot be shared publicly as participants did not give consent for their transcripts to be shared in this manner. Since the consent statement approved by the Institutional Review Board (IRB) of the Universitat Autònoma de Barcelona, and signed by the participants, did not include the provision that data would be made publicly available, we do not have participant consent to share this data. Also, the content is sensitive, and participants could be identified. Requests for anonymized data can be made to the Principal Investigator of the study, Sergi Fàbregues ( [email protected] ) or the Institutional Review Board (IRB) of the Universitat Autònoma de Barcelona ( [email protected] ).

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Since the 1990s, mixed methods research as a distinct methodology has seen vigorous institutionalization [ 1 , 2 ] with the launch of a specialized journal ( Journal of Mixed Methods Research ) in 2007, the establishment of the Mixed Methods International Research Association (MMIRA) in 2013, and the publication of a stream of handbooks, textbooks, and articles on the philosophy and practice of mixed methods. Mixed methods research is increasingly used in a wide range of disciplines, particularly in the social, educational, behavioral and health sciences [ 3 – 6 ]. Several scholars have argued that mixed methods inquiry can help researchers arrive at a more complete understanding of research problems, develop more robust quantitative instruments, and integrate several worldviews in a single research study [ 7 , 8 ].

A clear indication of the institutionalization of mixed methods research as a field is the expansion of the mixed methods community. Tashakkori, Johnson and Teddlie [ 9 ] have characterized this community as a group of scholars who share similar backgrounds, methodological orientations, philosophical assumptions, and views on research and practice. As distinct from researchers using only qualitative or quantitative approaches, these scholars often share similarities in training, research background, and professional affiliation. Those authors’ view of the mixed methods community is consistent with Thomas Kuhn’s preferred definition of paradigms, as cited by Morgan [ 10 ]. In disagreement with the view of paradigms as incompatible epistemological stances, Morgan [ 10 ], following Kuhn [ 11 ], advanced a more integrative notion of paradigms focused on shared beliefs and joint actions in a community of researchers. Denscombe [ 12 ] took this perspective one step further by proposing the notion of communities of practice , an idea originally developed by the educationalist Étienne Wenger. This notion brings us closer to a definition of paradigms as shared beliefs able to accommodate the diversity of perspectives and approaches that currently exists within the mixed methods community [ 12 , 13 ].

While the institutionalization of mixed methods as a field has helped to formalize and clarify research practices [ 14 ], this trend has also led to some criticism of the ways in which this approach has been developed and conceptualized by some members of the mixed methods community [ 15 , 16 ]. The critiques of the mixed methods field have already been summarized in two overviews published in the early 2010s. In the second edition of the Handbook , Tashakkori and Teddlie [ 17 ] outlined four frequent criticisms raised in the mixed methods literature, including overreliance on typologies and the higher status of quantitative versus qualitative research. One year later, Creswell [ 18 ] described some of the same criticisms in a summary of 11 key controversies. The emergence of this criticism testifies to the increasing maturity of the field and its progress towards what Creswell and Plano Clark [ 7 ] define as the period of reflection and refinement in mixed methods research. In their view, the mixed methods community should be honored that it has attracted critical attention and it should stimulate debate around the issues raised. Therefore, it is crucially important to address these criticisms in greater detail because such engagement will lead to valuable insights that could lay the basis for further discussion needed to ensure the healthy development of the field. Furthermore, addressing those criticisms is essential to enhance researchers’ understanding of the complexity of the mixed methods field and to provide them with the awareness needed to deal with tensions that might emerge when working in teams with researchers subscribing to different methodological viewpoints [ 19 , 20 ].

Most of the criticisms of mixed methods so far have been formulated in the literature by several scholars specialized in theoretical aspects of methodology. However, it would be useful to find out whether other researchers share these criticisms and whether they may have formulated others. Scholars with an interest in mixed methods research come from different academic disciplines that embody different theoretical and methodological perspectives. As Plano Clark and Ivankova [ 3 ] argue, these differences are highly likely to influence the ways in which those scholars view mixed methods as well as the questions they might raise regarding current ideas in the field. Therefore, it would be useful to examine the ways in which researchers’ criticisms differ according to discipline.

The aim of the present study is to examine the criticisms of the mixed methods field raised by a cross-national sample of 42 researchers working in the disciplines of education, nursing, psychology, and sociology. We report a secondary analysis of semi-structured interviews originally conducted to describe how researchers operationalize and conceptualize the quality of mixed methods research [ 14 ]. The contribution of this article is twofold: (a) it is, to our knowledge, the first study based on an empirical approach to examining researchers’ critical views on the development and conceptualization of mixed methods research, and (b) it enhances our understanding of the ways in which these critical views may be associated with different academic disciplines.

This article reports a secondary analysis of data originally collected in a multiple-case study of the quality of mixed methods based on semi-structured interviews with researchers in the disciplines of education, nursing, psychology, and sociology. In line with Heaton’s [ 21 ] definition of secondary analysis — which he calls supplementary analysis — as an “in-depth investigation of an emergent issue or aspect of the data which was not considered or fully addressed in the primary study” [ 21 ], in this article we re-analyze the original interview data in order to address the following two research questions (RQs): (RQ1) What criticisms of the mixed methods field are made by researchers in education, nursing, psychology, and sociology? and (RQ2) What differences and similarities can be identified in the criticisms reported by researchers working in different disciplines? In the following subsections, we provide a brief description of the sampling and data collection methods used in the original study, and of the procedures used in the secondary analysis of data. A more detailed explanation of procedures followed in the original study can be found in Fàbregues, Paré, and Meneses [ 22 ].

The original study

Sampling and recruitment of participants..

The disciplines of education, nursing, psychology, and sociology were selected for four main reasons: (1) professionals working in these disciplines contribute a relatively high proportion of mixed methods empirical articles and other methodological publications, (2) a considerable number of prevalence studies and methodological reviews on the use of mixed methods in subfields of these disciplines have been published, and (3) these disciplines are characterized by their clear disciplinary boundaries, and this characteristic offers the possibility of gaining useful comparative insights [ 6 ]. Criterion and maximum variation sampling were used to select the researchers who participated in the study [ 23 ]. In the criterion sampling, participants fulfilled two inclusion criteria: (1) they had carried out research primarily in one of the four disciplines mentioned above, and (2) they had contributed to at least one methodological publication on mixed methods research. Participant identification started with a systematic search for methodological publications on mixed methods published in English during or after 2003. Selected publications fulfilled the definitions of mixed methods suggested by either Creswell and Tashakkori [ 24 ] or Johnson, Onwuegbuzie, and Turner [ 25 ]. A number of characteristics of the first authors’ profiles were extracted, including the field of expertise, the country of affiliation, and the job title. To ensure heterogeneity of the sample, maximum variation sampling was applied to authors meeting the two inclusion criteria. An iterative approach was used to recruit 11 participants for each discipline. Sample size was based on recommendations found in the literature [ 26 ]. Potential participants were contacted using a prioritized list until a total of 44 participants had been recruited.

Data collection.

Data collection involved semi-structured interviews. Questions from the interview guide were focused on the following topics: (a) participants’ research background and methodological expertise, (b) participants’ conceptualization of mixed methods research, and (c) how participants perceived the quality of the mixed methods approach in practice. Interviews were conducted using Skype, telephone and, in two cases, e-mail correspondence. Interviews were audio-recorded and transcribed, and average interview length was 49 minutes. We carried out a member-checking process by sending back to participants the transcriptions and summaries of key points of the interviews to confirm that the data accurately represented their views. At this stage, we also gave participants an opportunity to clarify or expand the statements they made during the interview.

Trustworthiness.

Four strategies were used to enhance the trustworthiness of the original study. First, as explained above, all participants member-checked their transcribed interviews and summaries to confirm accuracy. Second, peer-debriefing was carried out during data collection by one researcher working together with another researcher familiar with mixed methods research who was not included in the sample. Third, an audit trail was used to record the decisions made during the study and to help researchers to reflect on the influence on the study findings of their own assumptions and disciplines. Fourth, the decisions taken during the analysis and interpretation of the data, as well as the disagreements arising during this stage, were discussed by the researchers until a consensus was reached.

The secondary data analysis

While the original study aimed to examine researchers’ views on the conceptualization and operationalization of the quality of mixed methods research, the aim of this secondary analysis of the same data is to examine researchers’ critical views of commonly accepted concepts and practices in the mixed methods field. Ethical approval for secondary data analysis was included in the ethics application for the original study, which was approved by the Institutional Review Board of the Universitat Autònoma de Barcelona. Participants signed an informed consent before the interview. Of the 44 researchers who participated in the original study, two did not consent to the subsequent use of their interview data for a secondary analysis. Therefore, the information provided by these two researchers was not used in the present study. Table 1 shows the characteristics of the 42 participants finally included in this secondary analysis.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0252014.t001

Qualitative content analysis as described by Graneheim & Lundman [ 27 ] was used to carry out the secondary analysis of the interview data. This form of analysis is especially appropriate when, as in this study, researchers are interested in systematically describing only the topics of interest indicated by the research questions, and not in obtaining a holistic overview of all of the data [ 28 ]. The data analysis was carried out in three stages using NVivo 12 for Mac (QSR International Pty Ltd, Victoria, Australia). In the first stage, the interview transcripts were read thoroughly in order to extract the sections of text in which the participants raised criticisms of the mixed methods field. These sections of text constituted the unit of analysis. In the second stage, the extracted sections were divided into meaning units, which were subsequently condensed, abstracted and labelled with codes. Each code included a description of the meaning of the code, an indicator to identify its presence in the data, and an example of a passage coded as belonging to that code. In the third stage, the codes were compared for similarities and differences, and clustered into several categories. The underlying meaning of these categories was then examined and formulated into themes. These themes represented the study participants’ criticisms of mixed methods as a field, which were the focus of RQ1. Decisions made in this phase of the study, along with any disagreements, were discussed by the researchers until a consensus was reached.

In order to answer RQ2, a multiple correspondence analysis [ 29 – 31 ] was carried out. This technique is a non-inferential form of statistical analysis designed to analyze the multivariate association of categorical variables by generating a representation of the underlying structure of a dataset. Since the statistical requirements of multiple correspondence analysis (i.e., sampling, linearity, and normality) are highly flexible, this method is especially suited for examining qualitative interview data transformed into quantitative data [ 32 ]. The output of the multiple correspondence analysis is a scatterplot representing the spatial grouping of categories and participants. The distances between plotted points represent the degree of similarity in the patterns of participants’ responses. Multiple correspondence analysis was used to examine the relationship between the participants’ discipline and the themes relating to criticisms. To perform this analysis, we used the NVivo “matrix coding query” function to generate a matrix in which binary codes related to criticisms identified in the qualitative content analysis (the mention or failure to mention the criticism) were displayed in the columns, while the 42 participants were displayed in the rows. The matrix output was exported to XLSTAT Version 2018.1 (Addinsoft, Paris, France), which was used to perform the analysis, using the binary codes for the criticisms as active variables and the participants’ discipline as supplementary variables. Following the recommendations of Bazeley [ 32 ], after carrying out the multiple correspondence analysis, we checked the results against the qualitative data to verify the interpretation of the statistical analysis.

RQ1; What criticisms of the mixed methods field are made by researchers in education, nursing, psychology, and sociology?

Eleven criticisms of how some members of the mixed methods community have developed and conceptualized this research approach were identified in 27 of the 42 interviews included in this secondary analysis. These criticisms were then grouped into the four domains used by Creswell [ 33 ] to map the landscape of mixed methods research: (1) the essence of mixed methods research (definitions and nomenclature), (2) philosophy (philosophical assumptions and paradigmatic stances), (3) procedures ( methods and techniques for carrying out mixed methods research), and (4) politics (justification of the use of mixed methods research). Each of these four domains and the corresponding criticisms are discussed in the following sections with the support of verbatim quotes from the interviews. Table 2 shows the criticisms for each domain and the number of participants making each criticism.

https://doi.org/10.1371/journal.pone.0252014.t002

Domain 1: The essence of mixed methods research.

Criticism 1 : The accepted definition of mixed methods research takes into account only the mixing of both quantitative and qualitative methods . Some participants objected that the most common definition of mixed methods research that usually prevails in the literature conceives the approach as being limited to the use of quantitative and qualitative methods. These participants believe that the field should adopt a broader definition that would also include the mixing of methods within the same tradition in a single design, that is, the combination of two or more quantitative methods or qualitative methods:

“(…) for me mixed methods is not only mixing qualitative and quantitative methods , but it could also be qualitative + qualitative or quantitative + quantitative methods” (Sociologist #4).

These participants argued that the current definition of mixed methods implies that mixing the two distinct families of methods is often the only appropriate approach while in fact this definition obviates the contingent nature of research. Certain research questions might be better answered by using a combination of methods from a single tradition. Furthermore, combining two methods from the same tradition can be as valuable and as challenging as combining two methods from different traditions. One participant used the term “pressure” to describe the feeling that he was obliged to mix quantitative and qualitative methods even when this approach was not the most appropriate one:

“ … combining methods isn’t just a matter of combining quantitative and qualitative methods . You can combine different methods that are both qualitative or both quantitative and that’s , that’s valuable in it- itself , and I am worried about the kind of pressure to combine quantitative and qualitative as if that would always be appropriate” (Sociologist #2).

Criticism 2 : The terminology used in mixed methods reflects a lack of agreement among its proponents . Several participants noted the lack of clear agreement on the terminology generally used to describe the concepts and procedures that pertain to mixed methods research. They also cited a tendency to use multiple definitions for the same term and different terms to refer to similar notions. One participant cited as problematic the use of several different terms (e.g., legitimation, validity, rigor) to refer to the quality of mixed methods research:

“I would like to see a word that’s used by as many people as possible to describe that [quality]… . But , you know , I , I just think if we , everybody continues to use different terms , that could be problematic” (Educationalist #8).

According to this participant, while synonymous terms might add some precision when used to describe the complexities of implementing mixed methods research, their use can also generate confusion, especially among reviewers, editors and researchers who are trying to familiarize themselves with the field:

“…it just gets to the point where if everyone has a different definition , then how useful is that ? And that gets confusing for those who review manuscripts , or editors , when people are using in different ways that exact same term” (Educationalist #8).

Participants suggested two possible reasons for this lack of agreement. First, the tendency among some scholars to consider that mixed methods researchers should be able to use whatever terminology they may find convenient. Second, the desire of some authors to claim priority for the terminology that defines a particular method or typology. In order to resolve this lack of agreement, participants suggested that members of the mixed methods community should work towards building a greater consensus on terminology:

“There needs to be a common language” (Educationalist #9).

Criticism 3 : Mixed methods research is not a new type of methods practice . Some participants noted the tendency in the literature to present mixed methods as a new type of research practice that emerged during the past three decades. They pointed out that the use of mixed methods has a prior history that considerably predates the time when it became formalized as a research field. These participants cited examples of studies in sociology by Jahoda and Zeisel (Marienthal study of unemployment) and fieldwork in anthropology by Margaret Mead, both dating from the early 20th century. While these studies had an influence on methodology in the social sciences on account of the ways in which they creatively combined multiple quantitative and qualitative data sources, they have been generally overlooked in the mixed methods literature:

“I don’t particularly think that [mixed methods research has allowed us to answer research questions which were left unanswered in the past] but what I do think is that , you know , do remember as well that mixed methods research does actually have a long history in Sociology” (Sociologist #7).

Domain 2: Philosophy.

Criticism 4 : Mixed methods research is not a third paradigm . A considerable number of participants argued against the idea of characterizing mixed methods research as a third paradigm. They found two major faults with this characterization. First, it relies on the idea of mixed methods research as an approach that is distinct from quantitative and qualitative methodologies. In the view of these participants, mixed methods approaches do not rely on singular elements that are distinct in their nature, philosophy, or procedures:

“So no , I think , ultimately , I’m probably , I’m not really convinced that is a distinct methodology (…) So I worry when , when the idea of something that’s very special about mixed methods is given a lot , is given too much primacy” (Sociologist #3). “I don’t think it’s helpful to see it as a separate approach in terms of actually conducting , you know , planning and conducting , the research…I certainly think it’s stretching it to see it as a different , as a separate paradigm…I think the whole idea of ‘paradigm’ is a little bit difficult” (Sociologist #5).

Second, the conceptualization of mixed methods as a paradigm presupposes a strong link between epistemology and method, that is, the identification of the use of mixed methods with a particular epistemological or ontological view, whereas, in fact, these are separate entities. Attaching epistemological and ontological assumptions to mixed methods research would weaken its functionality and creative potential:

“…if we restrict mixed methods to only one paradigm then we’re bottlenecking mixed methods into a certain area , and we restrict the functionality of it” (Nurse, #10).

Criticism 5 : Current discussions of mixed methods research conceive quantitative and qualitative research as separate paradigms . Related to the previous criticism, a number of participants noted that current conceptualizations of mixed methods take for granted the nature of the quantitative and qualitative approaches, conceiving them as separate paradigms based on particular philosophical assumptions, thus reinforcing the conventional divide between them and accentuating their differences:

“…the whole purpose , of course , of mixed methods is that it’s , that’s a paradigm , but I’m not convinced it is because it still draws on those conventional traditional paradigms…I find that’s likely less helpful because again it starts from the assumption that there is a strong division between qualitative and quantitative research” (Educationalist #2).

These participants stated that the mixed methods literature may have uncritically incorporated the methodological “rules” (conventions) that were dominant in the 1980s by associating qualitative research with the constructivist paradigm and quantitative research with the positivist paradigm. This linkage between philosophy and method may have been a result of the process of formalization of the methodology carried out by the “second generation of mixed methods researchers” (from 1980s to present), while the “first generation” (i.e., from the 1900s to 1980s) might not have had a philosophical problem:

“I think the biggest problem that mixed methods research is in right now is having adopted , without reflection the rules that were established in the mid 80’s on , on paradigms in quali and quanti…we have these pillars , these quali-quanti pillars and we’re working on these rooms…All these classical studies [from the first generation] had no problems in doing quali-quanti , it was only the attempt to formalize it which has actually created these , these problems” (Sociologist #6).

According to this participant, the association of the quantitative and qualitative approaches with particular epistemological stances contradicts the very nature of the mixed methods approach: if such philosophical and methodological differences between quantitative and qualitative research really existed, then the integration of the two approaches would not be possible:

“…the big problem with having adopted this [association] , on the one hand , it actually makes mixed methods impossible . So , it is not possible within one single design to argue that your da- , that there is a single and objectifiable reality out there , on the other hand , and there are multiple or no reality , there’s no reality” (Sociologist #6).

In the view of another participant, part of the mixed methods literature may have accentuated the differences between the two methodologies by representing their characteristics in different columns in a table, while ignoring the existence of methods that incorporate features of both approaches (e.g., survey containing both open and closed ended questions or qualitative studies that include descriptive statistics):

“I know what’s been recently suggested in the literature (…) I’m not even sure that I would say that we should have drawn a line between qualitative and quantitative as firmly as we have . A lot of the qualitative work that I do includes descriptive statistics” (Educationalist #7).

Criticism 6 : Superficiality of pragmatism . Some participants argued that authors in the mixed methods community sometimes characterize the notion of pragmatism in a superficial way by reducing it to merely eclecticism and confusing it with “practicalism”. In this way, these authors advocate a “what works” approach which may be useful when justifying the integration of the quantitative and qualitative methods, but this attitude distorts the nature of pragmatism by failing to consider its underlying theoretical and philosophical assumptions:

“…they tend to think that pragmatism is just the practicalities , and it’s just the technicalities…” (Educationalist #2).

One participant noted that the feebly argued debates on pragmatism to date may have led the mixed methods community to undervalue the important contribution this paradigm has made to the philosophical basis of empirical inquiry:

“I’ve probably never in my life seen such weak debates on pragmatism as I have in mix- , in the mixed methods debate . I mean if I think of this fabulous contribution that , that pragmatism as a philosophical discipline has made” (Sociologist #6).

Moreover, another participant observed that many researchers in the field have acquired their knowledge of pragmatism mainly from the descriptions of the mixed methods paradigm found in the literature, whereas a sound basis for pragmatism in mixed methods research practice would require consulting the seminal papers on pragmatism, such as those by John Dewey, Charles Sanders Pierce or William James:

“… from what I’ve read anything about pragmatism that’s in a mixed methods paper does tend to be superficial…you have to go right back to the original authors of pragmatism and I think sometimes when we speak about pragmatism in mixed methods research , students particularly ten- tend , maybe just read some articles in pragmatism and think they know about this , but I think it is important to go right back to Dewey and James and Pierce” (Nurse, #6).

Criticism 7 : Mixed methods research aligns with positivism . A few participants noted that some members of the mixed methods community tend to accord a higher status to the quantitative component because they consider that it is more objective and more closely embodies the scientific method. In their view, some researchers regard the qualitative component as mainly a supplement to the quantitative component. Consequently, researchers may fail to appreciate the added value that may be gained by using mixed methods research:

“qualitative research [is often used] to almost to kind of flesh out the , the , the quantitative aspects , so it’s a kind of embellishment rather than seeing it as something that might challenge some of the quantitative findings or might contribute to , to ultimately rephrasing the research question or to reanalyzing the , the quantitative data” (Sociologist #5).

Domain 3: Procedures.

Criticism 8 : Limitations of typologies . A number of participants criticized the tendency of some authors to present mixed methods designs and procedures from a typological perspective. Typologies are used in the mixed methods literature as classifications of methodological features, such as the timing and priority of the quantitative and qualitative components and the stage at which integration is carried out [ 7 , 9 ]. In the view of those participants, typologies are presented in the literature in a way that is excessively mechanical and prescriptive, unnecessarily simplifying the process of carrying out a mixed methods study by suggesting that a successful implementation of a mixed methods design can be carried out only by following a predefined set of steps:

“I’m arguing against approaches that I think are too sort of mechanical in the sense of laying out : ‘Ok , here’s categories A , B , C , D and E , and here are the rules for applying them . And if you just follow the rules , then you’ll be ok’” (Educationalist #1).

Participants noted that this approach entails four problems. First, in order to adapt their approach to the research questions that they need to answer, researchers may need to modify the guidelines suggested in the literature. Therefore, guidelines for the use of mixed methods designs should be only “guiding principles” that are adaptable to varying circumstances and able to take into account the interactions between the different elements of the design. One of these participants stated the following:

“I think , I mean , I started off by using…the sort of prescription…and it’s only when you start to get delve more into mixed methods… So , it’s , it’s really , I think [they should be] just guiding principles” (Nurse, #5).

Second, the typologies may curtail the creativity of researchers by restricting them to a series of predefined models that are considered the “correct” ways of combining quantitative and qualitative methods. As expressed by the following participant:

“…researchers are using mixed methods in such creative ways , it’s like , it’s just , when you read these designs and they can be just so , so different and they just don’t fit into , you know , the typologies” (Nurse, #3).

Third, rather than being empirically generated by examining how mixed methods research is actually carried out in practice, these typologies are the highly formalized result of a list of ideal designs formulated by mixed methods theorists, as noted by this participant:

“There were basically two different approaches [to the development of typologies] and the one that was most common was the sort of develop very formal systems…The opposite of that was Bryman who went out and interviewed qualitative researchers about what they did…he talked to people about what they really did rather than coming up with formal systems” (Sociologist #1).

Fourth, the existing typologies are too extensive, which makes them difficult for inexperienced researchers to apply, as we can see in the following quote:

“…there must be forty or fifty different designs associated with mixed methods and I think it’s , you know , I think that’s confusing to people and it’s … in some way I think it becomes irrelevant” (Psychologist #9).

Criticism 9 : Procedures described in the literature are not aligned with mixed methods practice . Some participants mentioned occasional discrepancies between the procedures explained in textbooks and articles and the implementation and reporting of mixed methods in practice, which may not always conform to published guidelines and typologies.

“you , you open any textbook…and the rules that are proposed there are broken every day very successfully by researchers who , who actually conduct the research…the practice and the debate need to run parallel and they probably , right now I think they are a bit too separate from each other” (Sociologist #3).

Participants attributed this disjuncture to the fact that a few influential authors probably lacked sufficient practical experience in using mixed methods. These authors may have tended to suggest methodological guidelines “from their desk” without testing them in practice or reviewing the empirical work of other researchers:

“…there might be a gap that , that a lot of researchers talk about using mixed methods , but I’m not sure if they actually do it in practice” (Psychologist #2).

Furthermore, participants also noted that in some cases those authors may have placed greater emphasis on the philosophical and theoretical basis of mixed methods than on describing the techniques involved in implementing mixed methods research:

“…people spend far too much time talking about epistemology , most of those discussions are actually very simple , but people make them very complicate…I think…technical questions about how you work with the data and what it means [are more important]” (Educationalist #6).

Domain 4: Politics.

Criticism 10 : Mixed methods research is not better than monomethod research . Some participants pointed out a tendency among some members of the mixed methods community to consider this type of research to be inherently superior to monomethod research:

“…thinking about the , the papers that I’ve reviewed have been for the kind of applied end journals…I think the main issue for me has been in terms of , you know , the purpose of using mixed methods; that I think there’s a tendency to slip into thinking that more is necessarily better” (Sociologist #9).

Participants noted that to attribute a higher status to mixed methods research is wrong because this view could lead to the oversimplification of other approaches, which would undermine their prestige. Furthermore, participants argued that a mixed methods approach is not always the best research option and a fully integrated design may not be the most appropriate. What really determines the suitability of an approach or a design is the research question of a study, so that a monomethod design is sometimes the most appropriate.

Criticism 11 : Homogenization of mixed methods research . A few participants criticized a tendency in the mixed methods field to homogenize terminology and procedures. In their view, some members of the community have tried to develop a “mixed methods way of doing things” which would be acceptable to all researchers and would require them to write in a particular way using particular terminologies and strategies:

“They’re , they’re trying to develop a language , they’re trying to develop an approach , a strategy that , that is going to be acceptable by all mixed methods researchers , which really is , is unacceptable” (Educationalist #2).

This attitude towards homogenization of mixed methods research could hinder the advancement of the field since it promotes a uniform approach, suppresses intellectual disputes and ignores the diversity of approaches and attitudes regarding mixed methods found in the literature. As one participant argued, to find space for legitimate difference in the field is very difficult due to the protectionist attitude of some prominent authors who are interested in propagating their own ideas rather than incorporating the ideas of other authors:

“I fear that there’s , among , among those who have some prominence , there are some who would be very eager to protect their own turf and not wanting to come together for some kind of joint effort” (Educationalist #3).

RQ2: What differences and similarities can be identified in the criticisms reported by researchers working in different disciplines?

Of the 27 participants who raised criticisms, ten were sociologists, eight were educationalists, five were nurses, and four were psychologists. Multiple correspondence analysis was used to analyze the differences, depending on their discipline, in the types of criticisms the participants raised. Fig 1 shows the multiple correspondence analysis map for the first two axes. Highly associated categories are plotted near to one another on the basis of their loading to the corresponding axes, while the least associated categories are plotted far from one another. Therefore, the axes should be interpreted based on the grouping seen on the map of the relative positions of the categories, which are expressed by the magnitude of the coordinates. The measures known as eigenvalues indicate how much of the categorical information is explained by each dimension. Higher eigenvalues indicate a greater amount of variance of the variables in that dimension [ 29 , 30 ].

Note: The Yes label indicates that the criticism was mentioned, while the No label indicates that the criticism was not mentioned.

https://doi.org/10.1371/journal.pone.0252014.g001

On Axis 1, which accounts for most of the variation in the data (eigenvalue of 92.4%), the yes categories of the criticisms associated with the domains of philosophy, politics and the essence of mixed methods are plotted on the right-hand side of the map, while the no categories of the same domains are on the left-hand side of the map. As shown in the figure, sociologists were more likely to formulate criticisms associated with the three abovementioned domains while psychologists and nurses were less likely to formulate criticisms associated with any of these three domains. Educationalists were less inclined than sociologists to formulate criticisms associated with those domains, as indicated by the proximity of the education category to the centroid (the center of the axis). Finally, the fact that the yes and no categories of the procedures domain are plotted on Axis 2, which has very low explanatory power (eigenvalue of 7.6%), reveals that participants in the four disciplines showed no relevant differences in formulating this criticism.

Summary of findings

The aim of this study, based on a secondary analysis of interview data, was to describe the criticisms of mixed methods as a field raised by a sample of researchers in the disciplines of education, nursing, psychology, and sociology. Overall, the findings revealed a number of criticisms related to several different issues. These criticisms were initially identified by the first author in the data analysis phase of the original study, which was focused on the conceptualization and operationalization of quality in mixed methods research. The criticisms were particularly relevant since they were unprompted, that is, they were spontaneously given by the participants in response to questions about another subject, rather than to questions about criticisms. Furthermore, the original study was based on a broad and diverse sample of participants; it included a few researchers from the disciplines of sociology and education known for their criticisms as well as a considerable number of researchers whose publications seemed to be neutral on the subject of criticisms of the mixed methods field. To this latter group of researchers, the interviews afforded an opportunity to express their disagreement with some predominant notions in the mixed methods field which they otherwise might not have published. Another key element of this study is the examination of the relationship between the discipline of the participants and the frequency and type of criticisms they made. This has been possible since the sample is relatively balanced in terms of the number of participants from each of the four disciplines included in the study.

In response to RQ1, participants raised a total of eleven unprompted critical remarks, categorized in the following four domains: the essence of mixed methods (three criticisms), philosophy (four criticisms), procedures (two criticisms), and politics (two criticisms). Nine of the eleven critical comments have been previously mentioned in the literature. For instance, on the essence of mixed methods domain, some authors have alluded to problems such as the narrow definition of mixed methods research [ 15 , 34 – 36 ], the lack of agreement on the terminology used [ 16 , 37 ] and the inappropriateness of considering mixed methods a new methodology [ 35 , 38 , 39 ]. In the philosophical domain, several authors have criticized the dominance of a positivist approach to mixed methods research in some disciplines [ 37 , 40 , 41 ] while some authors have pointed out that considering as a separate or distinct paradigm can lead to an artificial separation of the quantitative and qualitative approaches [ 42 – 45 ]. In the procedures domain, a number of authors cited the problems inherent in conceptualizing mixed methods designs typologically, since such a view is restrictive and unable to reflect the variety of mixed methods designs used in practice [ 38 , 46 , 47 ]. Finally, in the politics domain, a few authors have criticized a tendency, in some of the literature, to homogenize the field [ 48 ], while others have critically noted the occasional adoption of a universalist position based on the idea that the mixed methods approach is inherently superior to monomethod research [ 44 , 47 ]. However, we also identified two criticisms not previously mentioned in the literature: the excessively superficial characterizations of pragmatism (criticism 6) found occasionally in the mixed methods literature and the description of procedures that are not necessarily in line with research practice (criticism 9).

Regarding criticism 6, some participants noted a tendency on the part of some researchers in the field to cite, when writing about pragmatism, what other mixed methods researchers had written about this paradigm rather than citing foundational writings, such as those by John Dewey, William James, or Charles Sanders Peirce. In fact, none of the most influential and most frequently cited textbooks on mixed methods research in the four disciplines we studied cites any work by key authors in the pragmatist tradition. Therefore, it could be useful to learn whether these authors’ highly synthetic explanation of foundational knowledge leads inexperienced researchers to only weakly engage with this paradigm, or, on the other hand, whether this simplification might help them to grasp the basic principles of pragmatism more quickly while leading them to consult first-hand the foundational writings.

With respect to criticism 9, participants reported that researchers do not always follow in practice all of the procedures described in mixed methods textbooks. This disjuncture between textbook guidance and research practice has been described in several methodological reviews of the use of mixed methods in the four disciplines we included in our study. Features such as explicitly stating the mixed methods design used, reporting mixed methods research questions, or explicitly stating the limitations associated with the use of a mixed methods design are regarded by some authors as key characteristics of mixed methods studies (Creswell and Plano Clark, 2018; Plano Clark and Ivankova, 2016; Onwuegbuzie and Corrigan, 2014; O’Cathain et al, 2008). However, Bartholomew & Lockard (2018) reported that very few of the studies included in their review of the use of mixed methods in psychotherapy explicitly stated the mixed methods design used (13%) or reported mixed methods research questions (29%). Additionally, Bressan et al. (2017) and Irvine et al. (2021), in their reviews on mixed methods in nursing, found that most of the studies they included failed to report the limitations associated with the use of a mixed methods design. Therefore, it could be of great interest to study whether the omission of these characteristic features of mixed methods studies reflects the researchers’ view that these features are unimportant, or whether they are unfamiliar with reporting standards. Identifying this latter criticism is a particularly relevant finding of this study, since the intimate context of the interview might have led to the expression of subjective judgments that otherwise might not have come to light (i.e., the participant’s perception that some authors may not habitually carry out empirical research).

With respect to RQ2, we found relevant differences in the type of criticisms raised across disciplines. In fact, one of our key findings is that criticisms in the procedures domain were equally distributed across the four disciplines, while criticisms in the essence of mixed methods, philosophy and politics domains were clearly more common in sociology. First, these findings are consistent with statements made by Plano Clark and Ivankova [ 3 ] regarding the ways in which the sociocultural context of researchers — including the discipline in which they work — can shape their beliefs, knowledge and even experiences with regards to mixed methods. Indeed, the greater number of criticisms made by sociologists categorized in three of the four domains shows how disciplinary conventions might affect how researchers think about mixed methods and judge the acceptability of certain predominant conceptualizations. According to several authors [ 49 – 51 ], critique is a foundational and distinct feature of the discipline of sociology. Therefore, the generalized tendency among sociologists to question traditional assumptions about the order of the world and to detach themselves from predominant belief systems and ideologies might help to explain why many sociologists in our sample criticized ideas such as the conceptualization of quantitative and qualitative research as separate entities and the consideration of mixed methods as inherently better than monomethod research.

Theoretical implications of the study

The findings of our study highlight several differences in opinion in the mixed methods field previously identified by authors such as Greene [ 52 ], Tashakkori and Teddlie [ 17 ], Leech [ 53 ] and Maxwell, Chmiel, and Rogers [ 54 ], among others. Those authors showed that, owing to differences in philosophical and theoretical stances and their adherence to different research cultures, researchers in this field sometimes disagree on foundational issues such as nomenclature, the need for consensus, and the definition of mixed methods research. In our study, the participants, particularly those in the field of sociology, made several criticisms about how some foundational and philosophical aspects of mixed methods research have been conceptualized by the mixed methods community, including how mixed methods has been defined and accorded status as a third paradigm. Furthermore, our findings showed contradictory criticisms formulated by the participants as a group: while some researchers criticized the lack of a consensus in the field on the terminology used to describe mixed methods research (criticism 2), others criticized a tendency by some authors to homogenize terminology (criticism 11).

This divergence of views is consonant with the notion of communities of practice suggested by Denscombe [ 12 ]. Departing from Kuhn’s notion of paradigms as “shared beliefs among the members of a specialty area” (as cited by Morgan [ 10 ]), in Denscombe’s view, the broader mixed methods community is a paradigm encompassing a conglomerate of multiple research communities shaped by the principles, values and interests prevailing in their disciplines and research orientations. In line with that author’s view that methodological decisions and viewpoints “will be shaped by a socialization process involving the influence of peers” [ 12 ], our findings suggest that the disciplinary community of our participants is likely to have informed their criticisms of the mixed methods field. Although this divergence of views suggests that complete agreement and unhindered communication among researchers is not possible [ 55 ], Ghiara [ 56 ] argued that, in Kuhn’s view, some form of communication is always possible; and furthermore, conflicting viewpoints can be reconciled to a certain extent. Similarly, Johnson [ 57 ] has argued that this diversity of critical voices, rather than being a problem, merely indicates that “reality is likely plural”—there is no single set of ontological assumptions underlying mixed methods—and, furthermore, knowledge is articulated on the basis of “multiple standpoints and strategies for learning about our world” that can be reconciled to some degree. An example of the healthy coexistence of divergent viewpoints within the mixed methods community can be found in the Journal of Mixed Methods Research , the leading journal in the field, described in its webpage as a “primary forum for the growing community of international and multidisciplinary scholars of mixed methods research”. The journal publishes a wide range of manuscripts, including articles revealing approaches to mixed methods research that rest on divergent foundational and philosophical perspectives.

The desire for inclusion of divergent viewpoints should not lead researchers to ignore the challenges posed by this divergence. Due to the interdisciplinary nature of mixed methods research, Curry et al. [ 58 ] argue that mixed methods teams often include researchers with different methodological backgrounds and propensities. Occasionally, these differences may pose challenges for establishing effective collaboration and for efficiently integrating research methods. A greater awareness of multiple perspectives on mixed methods research, including divergent critical views like those reported in our study, could help researchers better anticipate difficulties that might present themselves in the course of working with researchers who hold differing viewpoints. Furthermore, as Maxwell, Chmiel, and Rogers [ 54 ] have suggested, a better understanding on the part of mixed methods researchers of the perspectives of others in the field who embrace a differing approach should facilitate the process of integrating quantitative and qualitative methods in studies where different ontological positions coincide. In a similar vein, such an understanding could also help overcome a form of methodological tokenism described by Hancock, Sykes and Verma [ 59 ]. This can occur when mixed methods researchers fail to attend to, and therefore align, the distinct epistemological and ontological premises that underlie the methodological orientations that are integrated in a mixed methods design.

Furthermore, the recognition of mixed methods researchers’ divergence of views should be an integral part in any effort to design and implement a curriculum for mixed methods research. Plano Clark and Ivankova [ 13 ] have pointed out that any lack of clarity concerning the existing disagreements about foundational elements of the mixed methods approach could be confusing to researchers inexperienced in this field. Therefore, it is essential that courses and workshops on mixed methods research take note of these criticisms. This last point is particularly important since the topic of critical viewpoints is not included in any of the mixed methods syllabus exemplars published in the literature, including those by Earley [ 60 ], Christ [ 61 ], and Ivankova and Plano Clark [ 13 ].

Limitations, strengths, and possibilities for future research

Our findings are subject to a few limitations. First of all, the interviews were carried out by Skype and telephone. While these two forms of communication allowed us to interview participants residing in various locations around the world, they limited the possibilities for building the sort of rapport that might have encouraged some participants to elaborate more in their responses. To minimize this limitation, participants were given the opportunity in the member-checking phase to add additional insights to their initial statements. Second, the transferability of our findings is limited by our decision to include only four disciplines while excluding the views of researchers working in other disciplines that also have a high prevalence of mixed methods studies, such as medicine, business, and information science. From our findings alone, it is not possible to infer how frequent these criticisms are and what types of criticisms may be more prevalent in each of the disciplines. Third, multiple correspondence analysis is an exploratory method not appropriate for testing hypotheses or statistical significance. In other words, the method is designed to describe associations between categorical variables rather than to make predictions about a population [ 31 ]. Therefore, in light of the limitations of multiple correspondence analysis for drawing deeper inferences, the findings regarding RQ2 should be considered provisional and subject to further investigation. Finally, since this study is based on a secondary analysis, the interview questions did not specifically prompt the participants to bring up criticisms since the questions were focused on participants’ views related to the quality of mixed methods research. If we had specifically prompted participants to report their own criticisms, it is likely that more critical opinions would have been gathered.

While the use of secondary data entailed certain limitations, it also conferred some advantages. Participants’ critical statements were entirely spontaneous since they were not explicitly solicited. This spontaneity probably helped to reduce the social desirability bias, that is, a presumed tendency for respondents to dissimulate their own critical views in a way that might seem professionally and socially more acceptable. Furthermore, despite the limitations of multiple correspondence analysis, this method allowed us to generate a parsimonious visual representation of the underlying patterns of relationships among the criticisms and the disciplines. This representation helped us to improve our interpretation of the qualitative findings and to identify useful leads for carrying our further analysis. A further strength is that the study included a broad sample of participants in terms of their geographic location, academic position, seniority, and methodological expertise. This diversity probably also afforded us access to a wider range of views. Finally, a key strength of our study is that, to our knowledge, it is the first empirical study that has addressed the topic of criticisms of mixed methods as a field.

This study represents a step towards a better understanding of some current criticisms of mixed methods research. However, further research will be needed to confirm and expand our findings. Future research based on a larger and more diverse sample of mixed methods researchers could extend the scope of our research questions and help researchers generalize from our findings. Such studies might help us discover whether researchers from other disciplines share the criticisms made by the participants in our study and whether those researchers harbor other criticisms of their own. Other analytical tools could be used to examine in greater detail how the circumstances and attributes of researchers — including their disciplinary background, methodological expertise and paradigmatic viewpoints — influence the way these scholars formulate their criticisms of the mixed methods field. Finally, future research is needed to examine the critical accounts of researchers less experienced in methodological writing and probably less exposed to current theoretical debates and developmental issues in the field.

Acknowledgments

The authors would like to acknowledge the help of Dick Edelstein in editing the final manuscript. The first draft of this paper was written while the first author was a visiting scholar in the Mixed Methods Program of the University of Michigan, directed by Michael D. Fetters, John W. Creswell, and Timothy D. Guetterman. This author would like to thank these three scholars for their support and, particularly Michael D. Fetters for his invaluable mentorship over the past few years.

• 1. Hesse-Biber S. Navigating a turbulent research landscape: Working the boundaries, tensions, diversity, and contradictions of multimethod and mixed methods inquiry. In: S Hesse-Biber, Johnson RB, editors. The Oxford handbook of multimethod and mixed methods research inquiry. New York, NY: Oxford University Press; 2015. p. xxxiii–liii.
• View Article
• Google Scholar
• 3. Plano Clark VL, Ivankova NV. Mixed methods research: A guide to the field. Thousand Oaks, CA: Sage; 2016.
• PubMed/NCBI
• 6. Ivankova NV, Kawamura Y. Emerging trends in the utilization of integrated designs in the social, behavioral, and health sciences. In: Tashakkori A, Teddlie C, editors. The SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks, CA: Sage; 2010. p. 581–612.
• 7. Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. Thousand Oaks, CA: Sage; 2018.
• 8. Fetters MD. The mixed methods research workbook: Activities for designing, implementing, and publishing projects. Thousand Oaks, CA: Sage; 2020.
• 9. Tashakkori A, Johnson RB, Teddlie C. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. 2nd ed. Thousand Oaks, CA: Sage; 2021.
• 11. Kuhn TS. The structure of scientific revolutions. 4th ed. Chicago: The University of Chicago Press; 2012.
• 17. Tashakkori A, Teddlie C. Epilogue: Current developments and emerging trends in integrated research methodology. In: Tashakkori A, Teddlie C, editors. The SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks, CA: Sage; 2010. p. 803–26.
• 18. Creswell JW. Controversies in mixed methods research. In: Denzin NK, Lincoln Y, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011. p. 269–83.
• 21. Heaton J. Reworking qualitative data. London: Sage; 2004.
• 23. Patton MQ. Qualitative research & evaluation methods. 4th ed. Thousand Oaks, CA: Sage; 2014.
• 28. Schreier M. Qualitative content analysis in practice. London: Sage; 2012.
• 29. Greenacre M. Correspondence analysis in practice. 3rd ed. Boca Raton, FL: Chapman & Hall/CRC; 2017.
• 30. Le Roux B, Rouanet H. Multiple correspondence analysis. Thousand Oaks, CA: Sage; 2010.
• 31. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis: A global perspective. 7th ed. Upper Saddle River, NJ: Pearson Education; 2010.
• 32. Bazeley P. Integrating analyses in mixed methods research. London: Sage; 2018.
• 33. Creswell JW. Mapping the developing landscape of mixed methods research. In: Tashakkori A, Teddlie C, editors. The SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks, CA: Sage; 2010. p. 45–68.
• 39. Pelto PJ. Mixed methods in ethnographic research: Historical perspectives. New York, NY: Routledge; 2017.
• 43. Niglas K. The multidimensional model of research methodology: An integrated set of continua. In: Tashakkori A, Teddlie C, editors. The SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks, CA: Sage; 2010. p. 215–36.
• 52. Greene JC. Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass; 2007.
• 53. Leech NL. Interviews with the early developers of mixed methods research. In: Tashakkori A, Teddlie C, editors. The SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks: Sage; 2010. p. 253–72.
• 54. Maxwell JA, Chmiel M, Rogers SE. Designing integration in multimethod and mixed methods research. In: Hesse-Biber S, Johnson RB, editors. The Oxford handbook of multimethod and mixed methods research inquiry. New York: Oxford University Press; 2015. p. 223–39.

This paper is in the following e-collection/theme issue:

Published on 11.4.2024 in Vol 26 (2024)

Evaluating the Digital Health Experience for Patients in Primary Care: Mixed Methods Study

Authors of this article:

Original Paper

• Melinda Ada Choy 1, 2 , BMed, MMed, DCH, MD   ; 
• Kathleen O'Brien 1 , BSc, GDipStats, MBBS, DCH   ; 
• Katelyn Barnes 1, 2 , BAPSC, MND, PhD   ; 
• Elizabeth Ann Sturgiss 3 , BMed, MPH, MForensMed, PhD   ; 
• Elizabeth Rieger 1 , BA, MClinPsych, PhD   ; 
• Kirsty Douglas 1, 2 , MBBS, DipRACOG, Grad Cert HE, MD  

1 School of Medicine and Psychology, College of Health and Medicine, The Australian National University, Canberra, Australia

2 Academic Unit of General Practice, Office of Professional Leadership and Education, ACT Health Directorate, Canberra, Australia

3 School of Primary and Allied Health Care, Monash University, Melbourne, Australia

Corresponding Author:

Melinda Ada Choy, BMed, MMed, DCH, MD

School of Medicine and Psychology

College of Health and Medicine

The Australian National University

Phone: 61 51244947

Email: [email protected]

Background: The digital health divide for socioeconomic disadvantage describes a pattern in which patients considered socioeconomically disadvantaged, who are already marginalized through reduced access to face-to-face health care, are additionally hindered through less access to patient-initiated digital health. A comprehensive understanding of how patients with socioeconomic disadvantage access and experience digital health is essential for improving the digital health divide. Primary care patients, especially those with chronic disease, have experience of the stages of initial help seeking and self-management of their health, which renders them a key demographic for research on patient-initiated digital health access.

Objective: This study aims to provide comprehensive primary mixed methods data on the patient experience of barriers to digital health access, with a focus on the digital health divide.

Methods: We applied an exploratory mixed methods design to ensure that our survey was primarily shaped by the experiences of our interviewees. First, we qualitatively explored the experience of digital health for 19 patients with socioeconomic disadvantage and chronic disease and second, we quantitatively measured some of these findings by designing and administering a survey to 487 Australian general practice patients from 24 general practices.

Results: In our qualitative first phase, the key barriers found to accessing digital health included (1) strong patient preference for human-based health services; (2) low trust in digital health services; (3) high financial costs of necessary tools, maintenance, and repairs; (4) poor publicly available internet access options; (5) reduced capacity to engage due to increased life pressures; and (6) low self-efficacy and confidence in using digital health. In our quantitative second phase, 31% (151/487) of the survey participants were found to have never used a form of digital health, while 10.7% (52/487) were low- to medium-frequency users and 48.5% (236/487) were high-frequency users. High-frequency users were more likely to be interested in digital health and had higher self-efficacy. Low-frequency users were more likely to report difficulty affording the financial costs needed for digital access.

Conclusions: While general digital interest, financial cost, and digital health literacy and empowerment are clear factors in digital health access in a broad primary care population, the digital health divide is also facilitated in part by a stepped series of complex and cumulative barriers. Genuinely improving digital health access for 1 cohort or even 1 person requires a series of multiple different interventions tailored to specific sequential barriers. Within primary care, patient-centered care that continues to recognize the complex individual needs of, and barriers facing, each patient should be part of addressing the digital health divide.

Introduction

The promise of ehealth.

The rapid growth of digital health, sped up by the COVID-19 pandemic and associated lockdowns, brings the promise of improved health care efficiency, empowerment of consumers, and health care equity [ 1 ]. Digital health is the use of information and communication technology to improve health [ 2 ]. eHealth, which is a type of digital health, refers to the use of internet-based technology for health care and can be used by systems, providers, and patients [ 2 ]. At the time of this study (before COVID-19), examples of eHealth used by patients in Australia included searching for web-based health information, booking appointments on the web, participating in online peer-support health forums, using mobile phone health apps (mobile health), emailing health care providers, and patient portals for electronic health records.

Digital health is expected to improve chronic disease management and has already shown great potential in improving chronic disease health outcomes [ 3 , 4 ]. Just under half of the Australian population (47.3%) has at least 1 chronic disease [ 5 ]. Rates of chronic disease and complications from chronic disease are overrepresented among those with socioeconomic disadvantage [ 6 ]. Therefore, patients with chronic disease and socioeconomic disadvantage have a greater need for the potential benefits of digital health, such as an improvement in their health outcomes. However, there is a risk that those who could benefit most from digital health services are the least likely to receive them, exemplifying the inverse care law in the digital age by Hart [ 7 ].

Our Current Understanding of the Digital Health Divide

While the rapid growth of digital health brings the promise of health care equity, it may also intensify existing inequities [ 8 ]. The digital health divide for socioeconomic disadvantage describes a pattern in which patients considered socioeconomically disadvantaged who are already marginalized through poor access to traditional health care are additionally hindered through poor access to digital health [ 9 ]. In Australia, only 67.4% of households in the lowest household income quintile have home internet access, compared to 86% of the general population and 96.9% of households in the highest household income quintile [ 10 ]. Survey-based studies have also shown that even with internet access, effective eHealth use is lower in populations considered disadvantaged, which speaks to broader barriers to digital health access [ 11 ].

The ongoing COVID-19 global pandemic has sped up digital health transitions with the rapid uptake of telephone and video consultations, e-prescription, and the ongoing rollout of e-mental health in Australia. These have supported the continuation of health care delivery while limiting physical contact and the pandemic spread; however, the early evidence shows that the digital health divide remains problematic. A rapid review identified challenges with reduced digital access and digital literacy among the older adults and racial and ethnic minority groups, which are both groups at greater health risk from COVID-19 infections [ 12 ]. An Australian population study showed that the rapid uptake of telehealth during peak pandemic was not uniform, with the older adults, very young, and those with limited English language proficiency having a lower uptake of general practitioner (GP) telehealth services [ 13 ].

To ensure that digital health improves health care outcome gaps, it is essential to better understand the nature and nuance of the digital health divide for socioeconomic disadvantage. The nature of the digital health divide for socioeconomic disadvantage has been explored primarily through quantitative survey data, some qualitative papers, a few mixed methods papers, and systematic reviews [ 11 , 14 - 16 ]. Identified barriers include a lack of physical hardware and adequate internet bandwidth, a reduced inclination to seek out digital health, and a low ability and confidence to use digital health effectively [ 16 ]. The few mixed methods studies that exist on the digital health divide generally triangulate quantitative and qualitative data on a specific disease type or population subgroup to draw a combined conclusion [ 17 , 18 ]. These studies have found digital health access to be associated with education, ethnicity, and gender as well as trust, complementary face-to-face services, and the desire for alternative sources of information [ 17 , 19 ].

What This Work Adds

This project sought to extend previous research by using an exploratory mixed methods design to ensure that the first step and driver of our survey of a larger population was primarily shaped by the experiences of our interviewees within primary care. This differs from the triangulation method, which places the qualitative and quantitative data as equal first contributors to the findings and does not allow one type of data to determine the direction of the other [ 18 ]. We qualitatively explored the experience of digital health for patients with socioeconomic disadvantage and chronic disease and then quantitatively measured some of the qualitative findings via a survey of the Australian general practice patient population. Our key objective was to provide comprehensive primary mixed methods data, describing the experience and extent of barriers to accessing digital health and its benefits, with a focus on the digital health divide. We completed this research in a primary care context to investigate a diverse community-based population with conceivable reasons to seek digital help in managing their health. Findings from this mixed methods study were intended to provide health care providers and policy makers with a more detailed understanding of how specific barriers affect different aspects or steps of accessing digital health. Ultimately, understanding digital health access can influence the future design and implementation of digital health services by more effectively avoiding certain barriers or building in enablers to achieve improved digital health access not only for everyone but also especially for those in need.

Study Design

We conducted a sequential exploratory mixed methods study to explore a complex phenomenon in depth and then measure its prevalence. We qualitatively explored the experience of digital health for patients with chronic disease and socioeconomic disadvantage in the first phase. Data from the first phase informed a quantitative survey of the phenomenon across a wider population in the second phase [ 18 ]. Both stages of research were conducted before the COVID-19 pandemic in Australia.

Recruitment

Qualitative phase participants.

The eligibility criteria for the qualitative phase were as follows: English-speaking adults aged ≥18 years with at least 1 self-reported chronic disease and 1 marker of socioeconomic disadvantage (indicated by ownership of a Health Care Card or receiving a disability pension, unemployment, or a user of public housing). A chronic disease was defined to potential participants as a diagnosed long-term health condition that had lasted at least 6 months (or is expected to last for at least 6 months; examples are listed in Multimedia Appendix 1 ). The markers of socioeconomic disadvantage we used to identify potential participants were based on criteria typically used by local general practices to determine which patients can have lower or no out-of-pocket expenses. Apart from unemployment, the 3 other criteria to identify socioeconomic disadvantage are means-tested government-allocated public social services [ 20 ]. Qualitative phase participants were recruited from May to July 2019 through 3 general practices and 1 service organization that serve populations considered socioeconomically disadvantaged across urban, regional, and rural regions in the Australian Capital Territory and South Eastern New South Wales. A total of 2 recruitment methods were used in consultation with and as per the choice of the participating organizations. Potential participants were either provided with an opportunity to engage with researchers (KB and MAC) in the general practice waiting room or identified by the practice or organization as suitable for an interview. Interested participants were given a detailed verbal and written description of the project in a private space before providing written consent to be interviewed. All interview participants received an Aus $50 (US $32.68) grocery shopping voucher in acknowledgment of their time.

Quantitative Phase Participants

Eligibility for the quantitative phase was English-speaking adults aged ≥18 years. The eligibility criteria for the quantitative phase were deliberately broader than those for the qualitative phase to achieve a larger sample size within the limitations of recruitment and with the intention that the factors of socioeconomic disadvantage and having a chronic disease could be compared to the digital health access of a more general population. The quantitative phase participants were recruited from November 2019 to February 2020. Study information and paper-based surveys were distributed and collected through 24 general practices across the Australian Capital Territory and South Eastern New South Wales regions, with an option for web-based completion.

Ethical Considerations

Qualitative and quantitative phase research protocols, including the participant information sheet, were approved by the Australian Capital Territory Health Human Research Ethics Committee (2019/ETH/00013) and the Australian National University Human Research Ethics Committee (2019/ETH00003). Qualitative phase participants were given a verbal and written explanation of the study, including how and when they could opt out, before they provided written consent. All interview participants received an Aus $50 (US $32.68) grocery shopping voucher in acknowledgment of their time. Quantitative participants were given a written explanation and their informed consent was implied by return of a completed survey. Participants in both phases of the study were told that all their data was deidentified. Consent was implied through the return of a completed survey.

Qualitative Data Collection and Analysis

Participants were purposively sampled to represent a range in age, gender, degree of socioeconomic disadvantage, and experience of digital health. The sampling and sample size were reviewed regularly by the research team as the interviews were being completed to identify potential thematic saturation.

The interview guide was developed by the research team based on a review of the literature and the patient dimensions of the framework of access by Levesque et al [ 21 ]. The framework by Levesque et al [ 21 ] is a conceptualization of health care access comprising 5 service and patient dimensions of accessibility and ability. The patient dimensions are as follows: (1) ability to perceive, (2) ability to seek, (3) ability to reach, (4) ability to pay, and (5) ability to engage [ 21 ]. The key interview topics included (1) digital health use and access, including facilitators and barriers; (2) attitudes toward digital health; and (3) self-perception of digital health skills and potential training. The interview guide was reviewed for face and content validity by the whole research team, a patient advocate, a digital inclusion charity representative, and the general practices where recruitment occurred. The questions and guide were iteratively refined by the research team to ensure relevance and support reaching data saturation. The interview guide has been provided as Multimedia Appendix 1 . The interviews, which took 45 minutes on average, were taped and transcribed. An interview summary sheet and reflective journal were completed by the interviewer after each interview to also capture nonverbal cues and tone.

Interview transcriptions were coded and processed by inductive thematic analysis. Data collection and analysis were completed in parallel to support the identification of data saturation. Data saturation was defined as no significant new information arising from new interviews and was identified by discussion with the research team [ 22 ]. The 2 interviewers (MAC and KB) independently coded the first 5 transcripts and reflected on them with another researcher (EAS) to ensure intercoder validity and reliability. The rest of the interviews were coded independently by the 2 interviewers, who regularly met to reflect on emerging themes and thematic saturation. Data saturation was initially indicated after 15 interviews and subsequently confirmed with a total of 19 interviews. Coding disagreements and theme development were discussed with at least 1 other researcher (EAS, ER, and KD). Thematic saturation and the final themes were agreed upon by the entire research team.

Quantitative Survey Development

The final themes derived in the qualitative phase of the project guided the specific quantitative phase research questions. The final themes were a list of ordered cumulative barriers experienced by participants in accessing digital health and its benefits ( Figure 1 ). The quantitative survey was designed to test the association between barriers to access and the frequency of use of digital health as a proxy measure for digital health access.

In the survey, the participants were asked about their demographic details, health and chronic diseases, knowledge, use and experience of digital health tools, internet access, perception of digital resource affordability, trust in digital health and traditional health services, perceived capability, health care empowerment, eHealth literacy, and relationship with their GP.

Existing scales and questions from the literature and standardized Australian-based surveys were used whenever possible. We used selected questions and scales from the Australian Bureau of Statistics standards, the eHealth Literacy Scale (eHEALS), the eHealth Literacy Questionnaire, and the Southgate Institute for Health Society and Equity [ 17 , 23 - 26 ]. We adapted other scales from the ICEpop Capability Measure for Adults, the Health Care Empowerment Inventory (HCEI), the Patient-Doctor Relationship Questionnaire, and the Chao continuity questionnaire [ 23 , 27 - 29 ]. Where an existing scale to measure a barrier or theme did not exist, the research team designed the questions based on the literature. Our questions around the frequency of digital health use were informed by multiple existing Australian-based surveys on general technology use [ 30 , 31 ]. Most of the questions used a Likert scale. Every choice regarding the design, adaptation, or copy of questions for the survey was influenced by the qualitative findings and decided on by full agreement among the 2 researchers who completed and coded the interviews. A complete copy of the survey is provided in Multimedia Appendix 2 .

Pilot-testing of the survey was completed with 5 patients, 2 experts on digital inclusion, and 3 local GPs for both the paper surveys and web-based surveys via Qualtrics Core XM (Qualtrics LLC). The resulting feedback on face and content validity, functionality of the survey logic, and feasibility of questionnaire completion was incorporated into the final version of the survey.

The survey was offered on paper with a participant information sheet, which gave the patients the option to complete the web-based survey. The survey was handed out to every patient on paper to avoid sampling bias through the exclusion of participants who could not complete the web-based survey [ 32 ].

Quantitative Data Treatment and Analysis

Data were exported from Qualtrics Core XM to an SPSS (version 26; IBM Corp) data set. Data cleaning and screening were undertaken (KB and KO).

Descriptive statistics (number and percentage) were used to summarize participant characteristics, preference measures, and frequency of eHealth use. Significance testing was conducted using chi-square tests, with a threshold of P <.05; effect sizes were measured by the φ coefficient for 2×2 comparisons and Cramer V statistic for all others. Where the cells sizes were too small, the categories were collapsed for the purposes of significance testing. The interpretation of effect sizes was as per the study by Cohen [ 33 ]. The analysis was conducted in SPSS and SAS (version 9.4; SAS Institute).

Participant Characteristics

Participants’ self-reported characteristics included gender, indigenous status, income category, highest level of education, marital status, and language spoken at home.

Age was derived from participant-reported year of birth and year of survey completion as of 2019 and stratified into age groups. The state or territory of residence was derived from the participant-reported postcode. The remoteness area was derived using the postcode reported by the participants and mapped to a modified concordance from the Australian Bureau of Statistics. Occupation-free text responses were coded using the Australian Bureau of Statistics Census statistics level 1 and 2 descriptors. The country of birth was mapped to Australia, other Organisation for Economic Cooperation and Development countries, and non–Organisation for Economic Cooperation and Development countries.

Frequency of eHealth Use

A summary measure of the frequency of eHealth use was derived from the questions on the use of different types of eHealth.

Specifically, respondents were asked if they had ever used any form of web-based health (“eHealth“) and, if so, to rate how often (never, at least once, every now and then, and most days) against 6 types of “eHealth” (searching for health information online, booking appointments online, emailing health care providers, using health-related mobile phone apps, accessing My Health Record, and accessing online health forums). The frequency of eHealth use was then classified as follows:

• High user: answered “most days” to at least 1 question on eHealth use OR answered “every now and then” to at least 2 questions on eHealth use
• Never user: answered “no” to having ever used any form of eHealth OR “never” to all 6 questions on eHealth use
• Low or medium user: all other respondents.

The frequency of eHealth use was reported as unweighted descriptive statistics (counts and percentages) against demographic characteristics and for the elements of each of the themes identified in phase 1.

Overview of Key Themes

Data were reported against the 6 themes from the phase 1 results of preference, trust, cost, structural access, capacity to engage, and self-efficacy. Where the components of trust, cost, capacity to engage, and self-efficacy had missing data (for less than half of the components only), mean imputation was used to minimize data loss. For each theme, the analysis excluded those for whom the frequency of eHealth use was unknown.

Preference measures (survey section D1 parts 1 to 3) asked participants to report against measures with a 4-point Likert scale (strongly disagree, disagree, agree, and strongly agree). Chi-square tests were conducted after the categories were condensed into 2 by combining strongly disagree and as well as combining strongly agree and agree.

Summary measures for trust were created in 4 domains: trust from the eHealth Literacy Questionnaire (survey section D1 parts 4 to 8), trust from Southgate—GPs, specialists, or allied health (survey section D2 parts 1 to 5), trust from Southgate—digital health (survey section D2 parts 6, 7, 9, and 10), and trust from Southgate—books or pamphlets (survey section D2 part 8). The data were grouped as low, moderate, and high trust based on the assigned scores from the component data. Chi-square tests were conducted comparing low-to-moderate trust against high trust for GP, specialists, or allied health and comparing low trust against moderate-to-high trust for book or pamphlet.

Summary measures for cost were created from survey item C10. To measure cost, participants were asked about whether they considered certain items or services to be affordable. These included cost items mentioned in the qualitative phase interviews relating to mobile phones (1 that connects to the internet, 1 with enough memory space to download apps, downloads or apps requiring payment, repairs, and maintenance costs), having an iPad or tablet with internet connectivity, a home computer or laptop (owning, repairs, and maintenance), home fixed internet access, and an adequate monthly data allowance. These 9 items were scored as “yes definitely”=1 or 0 otherwise. Chi-square tests were conducted with never and low or medium eHealth users combined.

Structural Access

Structural access included asking where the internet is used by participants (survey section C8) and factors relating to internet access (survey section C8 parts 1-3) reporting against a 4-point Likert scale (strongly disagree, disagree, agree, and strongly agree). Chi-square tests were conducted with strongly disagree, disagree, agree, or strongly agree, and never, low, or medium eHealth use combined.

Capacity to Engage

Summary measures for capacity to engage were created from survey section E1. To measure the capacity to engage, participants were asked about feeling “settled and secure,” “being independent,” and “achievement and progress” as an adaptation of the ICEpop Capability Measure for Adults [ 27 ], reporting against a 4-point Likert-like scale. Responses were scored from 1 (“I am unable to feel settled and secure in any areas of my life”) to 4 (“I am able to feel settled and secure in all areas of my life”).

The summary capacity measure was derived by the summation of responses across the 3 questions, which were classified into 4 groups, A to D, based on these scores. Where fewer than half of the responses were missing, mean imputation was used; otherwise, the record was excluded. Groups A and B were combined for significance testing.

Self-Efficacy

Summary measures for self-efficacy were adapted from the eHEALS (E3) and the HCEI (E2) [ 23 , 24 ].

Survey section E3—eHEALS—comprised 8 questions, with participants reporting against a 5-point Likert scale for each (strongly disagree, disagree, neither, agree, and strongly agree). These responses were assigned 1 to 5 points, respectively. The summary eHEALS measure was derived by the summation of responses across the 8 questions, which were classified into 5 groups, A to E, based on these scores. Where fewer than half of the responses were missing, mean imputation was used; otherwise, the record was excluded. Groups A to C and D to E were combined for significance testing.

Survey section E2—HCEI—comprised 5 questions, with participants reporting against a 5-point Likert scale for each (strongly disagree, disagree, neither, agree, and strongly agree). Strongly disagree and disagree and neither were combined, and similarly agree and strongly agree were combined for significance testing.

Qualitative Results

The demographic characteristics of the patients that we interviewed are presented in Table 1 .

The key barriers found to accessing digital health included (1) strong patient preference for human-based health services; (2) low trust in digital health services; (3) high financial costs of necessary tools, maintenance, and repairs; (4) poor publicly available internet access options; (5) reduced capacity to engage due to increased life pressures; and (6) low self-efficacy and confidence in using digital health.

Rather than being an equal list of factors, our interviewees described these barriers as a stepped series of cumulative hurdles, which is illustrated in Figure 1 . Initial issues of preference and trust were foundational to a person even when considering the option of digital health, while digital health confidence and literacy were barriers to full engagement with and optimal use of digital health. Alternatively, interviewees who did use digital health had been enabled by the same factors that were barriers to others.

a GP: general practitioner.

b Multiple answers per respondent.

Strong Patient Preference for Human-Based Health Services

Some patients expressed a strong preference for human-based health services rather than digital health services. In answer to a question about how digital health services could be improved, a patient said the following:

Well, having an option where you can actually bypass actually having to go through the app and actually talk directly to someone. [Participant #10]

For some patients, this preference for human-based health services appeared to be related to a lack of exposure to eHealth. These patients were not at all interested in or had never thought about digital health options. A participant responded the following to the interviewer’s questions:

Interviewer: So when...something feels not right, how do you find out what’s going on?

Respondent: I talk to Doctor XX.

Interviewer: Do you ever Google your symptoms or look online for information?

Respondent: No, I have never even thought of doing that actually. [Participant #11]

For other patients, their preference for human-based health care stemmed from negative experiences with technology. These patients reported actively disliking computers and technology in general and were generally frustrated with what they saw as the pitfalls of technology. A patient stated the following:

If computers and internet weren’t so frigging slow because everything is on like the slowest speed network ever and there’s ads blocking everything. Ads, (expletive) ads. [Participant #9]

A patient felt that he was pushed out of the workforce due his inability to keep up with technology-based changes and thus made a decision to never own a computer:

But, you know, in those days when I was a lot younger those sorts of things weren’t about and they’re just going ahead in leaps and bounds and that’s one of the reasons why I retired early. I retired at 63 because it was just moving too fast and it’s all computers and all those sorts of things and I just couldn’t keep up. [Participant #17]

Low Trust in Digital Health Services

Several patients described low trust levels for digital and internet-based technology in general. Their low trust was generally based on stories they had heard of other people’s negative experiences. A patient said the following:

I don’t trust the internet to be quite honest. You hear all these stories about people getting ripped off and I’ve worked too hard to get what I’ve got rather than let some clown get it on the internet for me. [Participant #11]

Some of this distrust was specific to eHealth. For example, some patients were highly suspicious of the government’s motives with regard to digital health and were concerned about the privacy of their health information, which made them hesitant about the concept of a universal electronic health record. In response to the interviewer’s question, a participant said the following:

Interviewer: Are there any other ways you think that eHealth might help you?

Respondent: I’m sorry but it just keeps coming back to me, Big Brother. [Participant #7]

Another participant said the following:

I just would run a mile from it because I just wouldn’t trust it. It wouldn’t be used to, as I said, for insurance or job information. [Participant #16]

High Financial Costs of the Necessary Tools, Maintenance, and Repairs

A wide variety of patients described affordability issues across several different aspects of the costs involved in digital health. They expressed difficulty in paying for the following items: a mobile phone that could connect to the internet, a mobile phone with enough memory space to download apps, mobile phone apps requiring extra payment without advertisements, mobile phone repair costs such as a broken screen, a computer or laptop, home internet access, and adequate monthly data allowance and speeds to functionally use the internet. Current popular payment systems, such as plans, were not feasible for some patients. A participant stated the following:

I don’t have a computer...I’m not in the income bracket to own a computer really. Like I could, if I got one on a plan kind of thing or if I saved up for x-amount of time. But then like if I was going on the plan I’d be paying interest for having it on like lay-buy kind of thing, paying it off, and if it ever got lost or stolen I would still have to repay that off, which is always a hassle. And yeah. Yeah, I’m like financially not in the state where I’m able to...own a computer right now as I’m kind of paying off a number of debts. [Participant #9]

Poor Publicly Available Internet Access Options

Some patients described struggling without home internet access. While they noted some cost-free public internet access points, such as libraries, hotel bars, and restaurants, they often found these to be inconvenient, lacking in privacy, and constituting low-quality options for digital health. A patient stated the following:

...it’s incredibly slow at the library. And I know why...a friend I went to school with used to belong to the council and the way they set it up, they just got the raw end of the stick and it is really, really slow. It’s bizarre but you can go to the X Hotel and it’s heaps quicker. [Participant #15]

In response to the interviewer's question, a participant said the following:

Interviewer: And do you feel comfortable doing private stuff on computers at the library...?

Respondent: Not really, no, but I don’t have any other choice, so, yeah. [Participant #9]

Reduced Capacity to Engage Due to Increased Life Pressures

When discussing why they were not using digital health or why they had stopped using digital health, patients often described significant competing priorities and life pressures that affected their capacity to engage. An unemployed patient mentioned that his time and energy on the internet were focused primarily on finding work and that he barely had time to focus on his health in general, let alone engage in digital health.

Other patients reported that they often felt that their ability to learn about and spend time on digital health was taken up by caring for sick family members, paying basic bills, or learning English. Some patients said that the time they would have spent learning digital skills when they were growing up had been lost to adverse life circumstances such as being in jail:

So we didn’t have computers in the house when I was growing up. And I didn’t know I’ve never...I’ve been in and out of jail for 28 odd years so it sort of takes away from learning from this cause it’s a whole different… it’s a whole different way of using a telephone from a prison. [Participant #11]

Low Self-Efficacy and Confidence in Starting the Digital Health Process

Some patients had a pervasive self-perception of being slow learners and being unable to use technology. Their stories of being unconfident learners seemed to stem from the fact that they had been told throughout their lives that they were intellectually behind. A patient said the following:

The computer people...wouldn’t take my calls because I’ve always been dumb with that sort of stuff. Like I only found out this later on in life, but I’m actually severely numerically dyslexic. Like I have to triple-check everything with numbers. [Participant #7]

Another patient stated the following:

I like went to two English classes like a normal English class with all the kids and then another English class with about seven kids in there because I just couldn’t I don’t know maybe because I spoke another language at home and they sort of like know I was a bit backward. [Participant #6]

These patients and others had multiple missing pieces of information that they felt made it harder to engage in digital health compared to “easier” human-based services. A patient said the following:

Yeah I’ve heard of booking online but I just I don’t know I find it easier just to ring up. And I’ll answer an email from a health care provider but I wouldn’t know where to start to look for their email address. [Participant #11]

In contrast, the patients who did connect with digital health described themselves as independent question askers and proactive people. Even when they did not know how to use a specific digital health tool, they were confident in attempting to and asking for help when they needed it. A patient said the following:

I’m a “I will find my way through this, no matter how long it takes me” kind of person. So maybe it’s more my personality...If I have to ask for help from somewhere, wherever it is, I will definitely do that. [Participant #3]

Quantitative Results

A total of 487 valid survey responses were received from participants across 24 general practices. The participant characteristics are presented in detail in Table S1 in Multimedia Appendix 3 .

The mean age of the participants was approximately 50 years (females 48.9, SD 19.4 years; males 52.8, SD 20.0 years), and 68.2% (332/487) of the participants identified as female. Overall, 34.3% (151/439) of respondents reported never using eHealth, and 53.8% (236/439) reported high eHealth use.

There were statistically significant ( P <.05) differences in the frequency of eHealth use in terms of age group, gender, state, remoteness, highest level of education, employment status, occupation group, marital status, and language spoken at home, with effect sizes being small to medium. Specifically, high eHealth characteristics were associated with younger age, being female, living in an urban area, and being employed.

Table 2 presents the frequency of eHealth use against 3 internet preference questions.

Preference for using the internet and technology in general and for health needs in particular were significantly related to the frequency of eHealth use ( P <.05 for each), with the effect sizes being small to medium.

a Excludes those for whom frequency of eHealth use is unknown.

b Chi-square tests conducted with strongly disagree and disagree combined, and agree and strongly agree combined.

Table 3 presents the frequency of eHealth use against 4 measures of trust.

The degree of trust was not statistically significantly different for the frequency of eHealth use for any of the domains.

b eHLQ: eHealth Literacy Questionnaire.

c Derived from survey question D1, parts 4 to 8. Mean imputation used where ≤2 responses were missing. If >2 responses were missing, the records were excluded.

d Derived from survey question D2, parts 1 to 5. Mean imputation used where ≤2 responses were missing. If >2 responses were missing, the records were excluded.

e Chi-square test conducted comparing low-to-moderate trust against high trust.

f Derived from survey question D2, parts 6, 7, 9, and 10. Mean imputation used where ≤2 responses were missing. If >2 responses were missing, the records were excluded.

g Derived from survey question D2 part 8.

h Chi-square test conducted comparing low trust against moderate-to-high trust.

Affordability of items and services was reported as No cost difficulty or Cost difficulty. eHealth frequency of use responses were available for 273 participants; among those with no cost difficulty , 1% (2/204) were never users, 14.2% (29/204) were low or medium users, and 84.8% (173/204) were high users of eHealth; among those with cost difficulty , 1% (1/69) were never users, 26% (18/69) were low or medium users, and 73% (50/69) were high users. There was a statistically significant difference in the presence of cost as a barrier between never and low or medium eHealth users compared to high users ( χ 2 1 =5.25; P =.02), although the effect size was small.

Table 4 presents the frequency of eHealth use for elements of structural access.

Quality of internet access and feeling limited in access to the internet were significantly associated with frequency of eHealth use ( P <.05), although the effect sizes were small.

b N/A: not applicable (cell sizes insufficient for chi-square test).

c Chi-square tests conducted with strongly disagree and disagree combined, agree and strongly agree combined, and never and low or medium categories combined.

Table 5 presents the frequency of eHealth use against respondents’ capacity to engage.

Capacity to engage was not significantly different for the frequency of eHealth use ( P =.54). 

b Derived from survey item E1. Where 1 response was missing, the mean imputation was used. If >1 response was missing, the record was excluded.

c Chi-square tests conducted with groups A and B combined.

Table 6 presents the frequency of eHealth use for elements of self-efficacy.

Statistically significant results were observed for the relationship between self-efficacy by eHEALS (moderate effect size) and frequency of eHealth use as well as for some of the questions from the HCEI (reliance on health professionals or others to access and explain information; small effect size; P <.05).

b eHEALS: eHealth Literacy Scale.

c eHEALS derived from item E3 (8 parts). Where ≤ 4 responses were missing, mean imputation was used. If >4 responses were missing, the records were excluded. Groups A to C as well as groups D to E were combined for the chi-square test.

d Strongly disagree, disagree, neither, and agree or strongly agree combined for significance testing.

Principal Findings

This paper reports on the findings of a sequential exploratory mixed methods study on the barriers to digital health access for a group of patients in Australian family medicine, with a particular focus on chronic disease and socioeconomic disadvantage.

In the qualitative first phase, the patients with socioeconomic disadvantage and chronic disease described 6 cumulative barriers, as demonstrated in Figure 1 . Many nonusers of digital health preferred human-based services and were not interested in technology, while others were highly suspicious of the technology in general. Some digitally interested patients could not afford quality hardware and internet connectivity, a barrier that was doubled by low quality and privacy when accessing publicly available internet connections. Furthermore, although some digitally interested patients had internet access, their urgent life circumstances left scarce opportunity to access digital health and develop digital health skills and confidence.

In our quantitative second phase, 31% (151/487) of the survey participants from Australian general practices were found to have never used a form of digital health. Survey participants were more likely to use digital health tools frequently when they also had a general digital interest and a digital health interest. Those who did not frequently access digital health were more likely to report difficulty affording the financial costs needed for digital access. The survey participants who frequently accessed digital health were more likely to have high eHealth literacy and high levels of patient empowerment.

Comparison With Prior Work

In terms of general digital health access, the finding that 31% (151/487) of the survey participants had never used one of the described forms of eHealth is in keeping with an Australian-based general digital participation study that found that approximately 9% of the participants were nonusers and 17% rarely engaged with the internet at all [ 34 ]. With regard to the digital health divide, another Australian-based digital health divide study found that increased age, living in a lower socioeconomic area, being Aboriginal or Torres Strait Islander, being male, and having no tertiary education were factors negatively associated with access to digital health services [ 17 ]. Their findings correspond to our findings that higher-frequency users of eHealth were associated with younger age, being female, living in an urban area, and being employed. Both studies reinforce the evidence of the digital health divide based on gender, age, and socioeconomic disadvantage in Australia.

With regard to digital health barriers, our findings provide expanded details on the range of digital health items and services that present a cost barrier to consumers. Affordability is a known factor in digital access and digital health access, and it is measured often by general self-report or relative expenditure on internet access to income [ 30 ]. Our study revealed the comprehensive list of relevant costs for patients. Our study also demonstrated factors of cost affordability beyond the dollar value of an item, as interviewees described the struggle of using slow public internet access without privacy features and the risks involved in buying a computer in installments. When we reflected on the complexity and detail of the cost barrier in our survey, participants demonstrated a clear association between cost and the frequency of digital health use. This suggests that a way to improve digital health access for some people is to improve the quality, security, and accessibility of public internet access options as well as to provide free or subsidized hardware, internet connection, and maintenance options for those in need, work that is being done by at least 1 digital inclusion charity in the United Kingdom [ 35 ].

Many studies recognize the factors of eHealth literacy and digital confidence for beneficial digital health access [ 36 ]. Our interviews demonstrated that some patients with socioeconomic disadvantage have low digital confidence, but that this is often underlined by a socially reinforced lifelong low self-confidence in their intellectual ability. In contrast, active users, regardless of other demographic factors, described themselves as innately proactive question askers. This was reinforced by our finding of a relationship between health care empowerment and the frequency of eHealth use. This suggests that while digital health education and eHealth literacy programs can improve access for some patients, broader and deeper long-term solutions addressing socioeconomic drivers of digital exclusion are needed to improve digital health access for some patients with socioeconomic disadvantage [ 8 ]. The deep permeation of socially enforced low self-confidence and lifelong poverty experienced by some interviewees demonstrate that the provision of free hardware and a class on digital health skills can be, for some, a superficial offering when the key underlying factor is persistent general socioeconomic inequality.

The digital health divide literature tends to identify the digital health divide, the factors and barriers that contribute to it, and the potential for it to widen if not specifically addressed [ 16 ]. Our findings have also identified the divide and the barriers, but what this study adds through our qualitative phase in particular is a description of the complex interaction of those barriers and the stepped nature of some of those barriers as part of the individual’s experience in trying to access digital health.

Strengths and Limitations

A key strength of this study is the use of a sequential exploratory mixed methods design. The initial qualitative phase guided a phenomenological exploration of digital health access experiences for patients with chronic disease and socioeconomic disadvantage. Our results in both study phases stem from the patients’ real-life experiences of digital health access. While some of our results echo the findings of other survey-based studies on general digital and digital health participation, our method revealed a greater depth and detail of some of these barriers, as demonstrated in how our findings compare to prior work.

As mentioned previously, the emphasis of this study on the qualitative first phase is a strength that helped describe the interactions between different barriers. The interviewees described their experiences as cumulative unequal stepped barriers rather than as producing a nonordered list of equal barriers. These findings expand on the known complexity of the issue of digital exclusion and add weight to the understanding that improving digital health access needs diverse, complex solutions [ 17 ]. There is no panacea for every individual’s digital health access, and thus, patient-centered digital health services, often guided by health professionals within the continuity of primary care, are also required to address the digital health divide [ 37 ].

While the sequential exploratory design is a strength of the study, it also created some limitations for the second quantitative phase. Our commitment to using the qualitative interview findings to inform the survey questions meant that we were unable to use previously validated scales for every question and that our results were less likely to lead to a normal distribution. This likely affected our ability to demonstrate significant associations for some barriers. We expect that further modeling is required to control for baseline characteristics and determine barrier patterns for different types of users.

One strength of this study is that the survey was administered to a broad population of Australian family medicine patients with diverse patterns of health via both paper-based and digital options. Many other digital health studies use solely digital surveys, which can affect the sample. However, we cannot draw conclusions from our survey about patients with chronic disease due to the limitations of the sample size for these subgroups.

Another sample-based limitation of this study was that our qualitative population did not include anyone aged from 18 to 24 years, despite multiple efforts to recruit. Future research will hopefully address this demographic more specifically.

While not strictly a limitation, we recognize that because this research was before COVID-19, it did not include questions about telehealth, which has become much more mainstream in recent years. The patients may also have changed their frequency of eHealth use because of COVID-19 and an increased reliance on digital services in general. Future work in this area or future versions of this survey should include telehealth and acknowledge the impact of COVID-19. However, the larger concept of the digital health divide exists before and after COVID-19, and in fact, our widespread increased reliance on digital services makes the digital divide an even more pressing issue [ 12 ].

Conclusions

The experience of digital health access across Australian primary care is highly variable and more difficult to access for those with socioeconomic disadvantage. While general digital interest, financial cost, and digital health literacy and empowerment are clear factors in digital health access in a broad primary care population, the digital health divide is also facilitated in part by a stepped series of complex and cumulative barriers.

Genuinely improving digital health access for 1 cohort or even 1 person requires a series of multiple different interventions tailored to specific sequential barriers. Given the rapid expansion of digital health during the global COVID-19 pandemic, attention to these issues is necessary if we are to avoid entrenching inequities in access to health care. Within primary care, patient-centered care that continues to recognize the complex individual needs of, and barriers facing, each patient should be a part of addressing the digital health divide.

Acknowledgments

The authors are thankful to the patients who shared their experiences with them via interview and survey completion. The authors are also very grateful to the general practices in the Australian Capital Territory and New South Wales who kindly gave their time and effort to help organize interviews, administer, and post surveys in the midst of the stress of day-to-day practice life and the bushfires of 2018-2019. The authors thank and acknowledge the creators of the eHealth Literacy Scale, the eHealth Literacy Questionnaire, the ICEpop Capability Measure for Adults, the Health Care Empowerment Inventory, the Patient-Doctor Relationship Questionnaire, the Chao continuity questionnaire, and the Southgate Institute for Health Society and Equity for their generosity in sharing their work with the authors [ 17 , 19 - 25 ]. This study would not have been possible without the support of the administrative team of the Academic Unit of General Practice. This project was funded by the Royal Australian College of General Practitioners (RACGP) through the RACGP Foundation IPN Medical Centres Grant, and the authors gratefully acknowledge their support.

Data Availability

The data sets generated during this study are not publicly available due to the nature of our original ethics approval but are available from the corresponding author on reasonable request.

Authors' Contributions

MAC acquired the funding, conceptualized the project, and organized interview recruitment. MAC and KB conducted interviews and analyzed the qualitative data. EAS, ER, and KD contributed to project planning, supervision and qualitative data analysis. MAC, KB and KO wrote the survey and planned quantitative data analysis. MAC and KB recruited practices for survey administration. KO and KB conducted the quantitative data analysis. MAC and KO, with KB drafted the paper. EAS, ER, and KD helped with reviewing and editing the paper.

Conflicts of Interest

None declared.

Phase 1 interview guide.

Phase 2 survey: eHealth and digital divide.

Phase 2 participant characteristics by frequency of eHealth use.

• Eysenbach G. What is e-health? J Med Internet Res. 2001;3(2):E20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Iyawa GE, Herselman M, Botha A. Digital health innovation ecosystems: from systematic literature review to conceptual framework. Procedia Comput Sci. 2016;100:244-252. [ FREE Full text ] [ CrossRef ]
• Berrouiguet S, Baca-García E, Brandt S, Walter M, Courtet P. Fundamentals for future mobile-health (mHealth): a systematic review of mobile phone and web-based text messaging in mental health. J Med Internet Res. Jun 10, 2016;18(6):e135. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shen H, van der Kleij RM, van der Boog PJ, Chang X, Chavannes NH. Electronic health self-management interventions for patients with chronic kidney disease: systematic review of quantitative and qualitative evidence. J Med Internet Res. Nov 05, 2019;21(11):e12384. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Australia's health 2018. Australian Institute of Health and Welfare. 2018. URL: https://www.aihw.gov.au/reports/australias-health/australias-health-2018/contents/table-of-contents [accessed 2024-04-04]
• Australian Institute of Health and Welfare. Chronic Diseases and Associated Risk Factors in Australia, 2006. Canberra, Australia. Australian Institute of Health and Welfare; 2006.
• Hart JT. The inverse care law. The Lancet. Feb 27, 1971;297(7696):405-412. [ CrossRef ]
• Davies AR, Honeyman M, Gann B. Addressing the digital inverse care law in the time of COVID-19: potential for digital technology to exacerbate or mitigate health inequalities. J Med Internet Res. Apr 07, 2021;23(4):e21726. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: internet use patterns, eHealth literacy, and attitudes toward computer/internet use. J Med Internet Res. May 02, 2013;15(5):e93. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Household use of information technology. Australian Bureau of Statistics. 2018. URL: https://tinyurl.com/4efm6u92 [accessed 2024-03-24]
• Kontos E, Blake KD, Chou WY, Prestin A. Predictors of eHealth usage: insights on the digital divide from the health information national trends survey 2012. J Med Internet Res. Jul 16, 2014;16(7):e172. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Litchfield I, Shukla D, Greenfield S. Impact of COVID-19 on the digital divide: a rapid review. BMJ Open. Oct 12, 2021;11(10):e053440. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Butler DC, Joshy G, Douglas KA, Sayeed MS, Welsh J, Douglas A, et al. Changes in general practice use and costs with COVID-19 and telehealth initiatives: analysis of Australian whole-population linked data. Br J Gen Pract. Apr 27, 2023;73(730):e364-e373. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arsenijevic J, Tummers L, Bosma N. Adherence to electronic health tools among vulnerable groups: systematic literature review and meta-analysis. J Med Internet Res. Feb 06, 2020;22(2):e11613. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kontos EZ, Bennett GG, Viswanath K. Barriers and facilitators to home computer and internet use among urban novice computer users of low socioeconomic position. J Med Internet Res. Oct 22, 2007;9(4):e31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Latulippe K, Hamel C, Giroux D. Social health inequalities and eHealth: a literature review with qualitative synthesis of theoretical and empirical studies. J Med Internet Res. Apr 27, 2017;19(4):e136. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Foley K, Freeman T, Ward P, Lawler A, Osborne R, Fisher M. Exploring access to, use of and benefits from population-oriented digital health services in Australia. Health Promot Int. Aug 30, 2021;36(4):1105-1115. [ CrossRef ] [ Medline ]
• Cresswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA. SAGE Publications; 2007.
• Tappen RM, Cooley ME, Luckmann R, Panday S. Digital health information disparities in older adults: a mixed methods study. J Racial Ethn Health Disparities. Feb 2022;9(1):82-92. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Who can get a card. Services Australia. URL: https://www.servicesaustralia.gov.au/who-can-get-health-care-card?context=21981 [accessed 2023-11-03]
• Levesque JF, Harris MF, Russell G. Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. Mar 11, 2013;12:18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bryant A, Charmaz K. The SAGE Handbook of Grounded Theory, Paperback Edition. Thousand Oaks, CA. SAGE Publications; 2010.
• Johnson MO, Rose CD, Dilworth SE, Neilands TB. Advances in the conceptualization and measurement of health care empowerment: development and validation of the health care empowerment inventory. PLoS One. 2012;7(9):e45692. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Norman CD, Skinner HA. eHEALS: the eHealth literacy scale. J Med Internet Res. Nov 14, 2006;8(4):e27. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kayser L, Karnoe A, Furstrand D, Batterham R, Christensen KB, Elsworth G, et al. A multidimensional tool based on the eHealth literacy framework: development and initial validity testing of the eHealth Literacy Questionnaire (eHLQ). J Med Internet Res. Feb 12, 2018;20(2):e36. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Standards. Australian Bureau of Statistics. URL: https://www.abs.gov.au/statistics/standards [accessed 2024-04-04]
• Al-Janabi H, Flynn TN, Coast J. Development of a self-report measure of capability wellbeing for adults: the ICECAP-A. Qual Life Res. Feb 2012;21(1):167-176. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Van der Feltz-Cornelis CM, Van Oppen P, Van Marwijk HW, De Beurs E, Van Dyck R. A patient-doctor relationship questionnaire (PDRQ-9) in primary care: development and psychometric evaluation. Gen Hosp Psychiatry. 2004;26(2):115-120. [ CrossRef ] [ Medline ]
• Chao J. Continuity of care: incorporating patient perceptions. Fam Med. 1988;20(5):333-337. [ Medline ]
• Wilson CK, Thomas J, Barraket J. Measuring digital inequality in Australia: the Australian digital inclusion index. JTDE. Jun 30, 2019;7(2):102-120. [ CrossRef ]
• Digital participation: a view of Australia's online behaviours. Australia Post. Jul 2017. URL: https://auspost.com.au/content/dam/auspost_corp/media/documents/white-paper-digital-inclusion.pdf [accessed 2024-04-04]
• Poli A, Kelfve S, Motel-Klingebiel A. A research tool for measuring non-participation of older people in research on digital health. BMC Public Health. Nov 08, 2019;19(1):1487. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cohen J. Statistical Power Analysis for the Behavioral Sciences Second Edition. London, UK. Routledge; 1988.
• Borg K, Smith L. Digital inclusion and online behaviour: five typologies of Australian internet users. Behav Inf Technol. Feb 15, 2018;37(4):367-380. [ CrossRef ]
• Mathers A, Richardson J, Vincent S, Joseph C, Stone E. Good Things Foundation COVID-19 response report. Good Things Foundation. 2020. URL: https://tinyurl.com/2peu3kak [accessed 2024-04-04]
• Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. Jun 16, 2006;8(2):e9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Neves AL, Burgers J. Digital technologies in primary care: implications for patient care and future research. Eur J Gen Pract. Dec 11, 2022;28(1):203-208. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung; submitted 03.07.23; peer-reviewed by T Freeman, H Shen; comments to author 16.08.23; revised version received 30.11.23; accepted 31.01.24; published 11.04.24.

©Melinda Ada Choy, Kathleen O'Brien, Katelyn Barnes, Elizabeth Ann Sturgiss, Elizabeth Rieger, Kirsty Douglas. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Open access
• Published: 10 April 2024

The burden of anxiety, depression, and stress, along with the prevalence of symptoms of PTSD, and perceptions of the drivers of psychological harms, as perceived by doctors and nurses working in ICUs in Nepal during the COVID-19 pandemic; a mixed method evaluation

• Shirish KC 1 ,
• Tiffany E. Gooden 2 ,
• Diptesh Aryal 1 ,
• Kanchan Koirala 1 ,
• Subekshya Luitel 1 ,
• Rashan Haniffa 3 , 4 ,
• Abi Beane 3 , 4 on behalf of

Collaboration for Research, Implementation, and Training in Critical Care in Asia and Africa (CCAA)

BMC Health Services Research volume  24 , Article number:  450 ( 2024 ) Cite this article

2317 Accesses

4 Altmetric

Metrics details

The COVID-19 pandemic resulted in significant physical and psychological impacts for survivors, and for the healthcare professionals caring for patients. Nurses and doctors in critical care faced longer working hours, increased burden of patients, and limited resources, all in the context of personal social isolation and uncertainties regarding cross-infection. We evaluated the burden of anxiety, depression, stress, post-traumatic stress disorder (PTSD), and alcohol dependence among doctors and nurses working in intensive care units (ICUs) in Nepal and explored the individual and social drivers for these impacts.

We conducted a mixed-methods study in Nepal, using an online survey to assess psychological well-being and semi-structured interviews to explore perceptions as to the drivers of anxiety, stress, and depression. Participants were recruited from existing national critical care professional organisations in Nepal and using a snowball technique. The online survey comprised of validated assessment tools for anxiety, depression, stress, PTSD, and alcohol dependence; all tools were analysed using published guidelines. Interviews were analysed using rapid appraisal techniques, and themes regarding the drivers for psychological distress were explored.

134 respondents (113 nurses, 21 doctors) completed the online survey. Twenty-eight (21%) participants experienced moderate to severe symptoms of depression; 67 (50%) experienced moderate or severe symptoms of anxiety; 114 (85%) had scores indicative of moderate to high levels of stress; 46 out of 100 reported symptoms of PTSD. Compared to doctors, nurses experienced more severe symptoms of depression, anxiety, and PTSD, whereas doctors experienced higher levels of stress than nurses. Most (95%) participants had scores indicative of low risk of alcohol dependence. Twenty participants were followed up in interviews. Social stigmatism, physical and emotional safety, enforced role change and the absence of organisational support were perceived drivers for poor psychological well-being.

Nurses and doctors working in ICU during the COVID-19 pandemic sustained psychological impacts, manifesting as stress, anxiety, and for some, symptoms of PTSD. Nurses were more vulnerable. Individual characteristics and professional inequalities in healthcare may be potential modifiable factors for policy makers seeking to mitigate risks for healthcare providers.

Peer Review reports

Introduction

Between January 2020 and December 2021, the COVID-19 pandemic led to an estimated 18.2 million deaths [ 1 ]. Globally, healthcare systems were overwhelmed during the pandemic, with intensive care units (ICUs) receiving an unprecedented burden of patients [ 2 ]. In Nepal, the government first declared a lockdown on March 24, 2020, that lasted until July 21, 2020, and the second lockdown was announced on April 29, 2021, which was fully lifted on September 1, 2021 [ 3 ]. The first wave of the COVID-19 pandemic reached a peak of over 5000 cases a day in October 2020, and the second wave reached a peak of more than 9000 cases a day in May 2021, which was almost double [ 4 ]. Prior to the pandemic, Nepal reported a capacity of 1595 ICU beds across 194 hospitals and around 840 ventilators, equating to 2.8 ventilator-equipped ICU beds per 100,000 people [ 5 ]. To cope with the influx of COVID-19 patients, several existing postoperative wards and other high-dependency units of the hospitals were converted into improvised critical care units [ 6 ]. Globally, healthcare professionals (HCPs) and specifically those working in ICU and critical care services, arguably were at the frontline of the healthcare response. These HCPs faced the uncertainty of managing this new condition, extended working hours, limited personal protective equipment (PPE), and an increased risk of infection as they provide essential lifesaving interventions, including intubation and non-invasive respiratory management [ 7 , 8 ].

The impacts of the COVID-19 pandemic on the mental health and well-being of HCPs who worked during and after this global emergency are slowly becoming apparent. Research emerging from China, the USA, and Europe [ 9 ] describes a significant burden of psychological distress and symptoms synonymous with mental health conditions in HCPs. This is also evident from the limited studies that have been conducted in Nepal. For instance, one study conducted among 150 HCPs from outpatient clinics and inpatient wards caring for COVID-19 patients in Nepal reported that 38% of participants suffered from anxiety and/or depression [ 10 ]. Another Nepali study revealed that the prevalence of anxiety and depression among HCPs, including health assistants and support staff was 47% and 41%, respectively [ 11 ]. A larger online survey of 475 HCPs including pharmacists, paramedics and public health practitioners reported similar findings (42% had anxiety) and noted that nurses had a higher proportion of symptoms compared to other HCPs [ 12 ].. Whilst these studies, in conjunction with a meta-analysis, indicate that depression, anxiety, and post-traumatic disorder (PTSD) are highly prevalent among HCPs during the pandemic [ 9 , 10 , 11 , 12 , 13 ], fewer studies have explored the disparities between professionals’ roles, specifically among ICU workers, a group exposed to more advanced cases of COVID-19. Indeed a small study in Nepal comprising 96 nurses revealed that nurses who worked directly with COVID-19 patients experienced more severe symptoms of depression and anxiety [ 13 ]. The nature and characteristics of mental health symptoms appear to vary geographically, the HCPs’ role, their individual characteristics (age, gender) along with health system’s pre-existing resource capacity and ability to respond to increasing demand placed by events such as a pandemic. Understanding the mental health impact of ICU workers, any disparities between professional roles and drivers behind poor mental health in Nepal will help to identify what support is needed for ICU workers for pandemic preparedness; thus, providing important directions for investment in health systems strengthening.

We aimed to investigate the burden of anxiety, depression, stress, PTSD, and alcohol dependence among doctors and nurses in Nepal that worked in the ICU during the COVID-19 pandemic. We further sought to identify the factors driving the self-reported burden of psychological distress by exploring the lived experiences of these two different professional groups, and how these experiences impacted their psychological health and well-being.

Study design

We undertook a mixed-methods cross-sectional study [ 14 ] in Nepal with ICU doctors and nurses, combining an online questionnaire consisting of validated self-assessment tools combined with semi-structured interviews. The following self-reporting psychological assessment tools were used, given they have been used in previous studies in other settings and their widely validated in a variety of settings: Beck Anxiety Inventory (BAI) [ 15 ], Beck Depression Inventory (BDI) [ 16 ], Perceived Stress Scale (PSS) [ 17 ], PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders-5 (PCL-5) [ 18 ] and Alcohol Use Disorder identification Tool (AUDIT) [ 19 ]. BDI, BAI, and AUDIT have been validated in Nepal [ 20 , 21 , 22 ] and the PSS has been tested for reliability and correlation in Nepal [ 23 ]. Whilst the PCL-5 has not been validated in a Nepali setting, it was piloted (along with all other assessment tools used) with 20 people before the study commenced. Participants were given the flexibility to complete the questionnaire in either Nepali or English language. Despite this option, all participants opted to respond in English.

Ethics approval

was granted from the Nepal Health Research Council (approval number: 176/2021 P). All participants provided informed consent electronically before completing the online questionnaire. Participants from the qualitative component provided further informed verbal consent before the interview commenced.

In 2020, Nepal reported a capacity of 1595 ICU beds across 194 hospitals and around 840 ventilators, equating to 2.8 ventilator-equipped ICU beds per 100,000 people [ 5 ]. A year later, Nepal was under a state of health emergency, with patients being turned down due to a lack of ICU beds, oxygen, and ventilators [ 24 ].

Participants and recruitment

Doctors and nurses with experience in caring for COVID-19 patients in Nepalese ICUs were eligible for participation. Initially doctors registered with the Nepalese Society of Critical Care Medicine (NSCCM) [ 25 ] and nurses registered with the Critical Care Nurses Association of Nepal (CCNAN) [ 26 ] were contacted and invited to participate. Both organisations consist of voluntary memberships and represent the doctors and nurses working in a critical care setting in Nepal. At the time of recruitment, there were 187 doctors and 104 nurses registered at these organisations. This initial purposive sampling was augmented by snowballing techniques, whereby respondents were invited to forward the questionnaire link to other doctors or nurses working in ICUs [ 27 ]. Following completion of the questionnaire, respondents were invited to participate in a virtual interview. A convenience sample of 20 participants (a number which, based on the literature, was likely to provide saturation of findings [ 28 ]) was subsequently scheduled for an interview.

Study materials and data collection

The questionnaire was developed using an online survey platform (Google Forms) [ 29 ]. The questionnaire was piloted for readability and responder reliability with twenty HCPs based in Nepal, prior to roll out, who did not participate in the final analysis. Questionnaire content included socio-demographic information; age, sex, professional role and experience, degree of schooling, and home living arrangements; factors which had been identified as being important in the burden of psychological distress and impact on family life in similar research conducted during the previous SARS pandemic as well as the current COVID-19 event [ 30 ]. Participants could opt out of the study at any time. Participants could only complete the questionnaire once, and all survey responses were anonymous. Participants were signposted to healthcare services available to them should they be suffering from any distressing, mild, moderate or severe mental health symptoms. Invitations to participate in the questionnaire were sent out from 20th May 2021, and the questionnaire was closed to responses on 2nd October 2021.

The semi-structured interview topic guide was co-developed between doctors and nurses working in ICUs in Kathmandu. Co-design was used to ensure the sensitivity and appropriateness of the questions. None of the doctors and nurses involved in the codesign of the topic guide participated in the study proper. The qualitative component was aimed to augment the quantitative findings by providing an understanding of what social, organisational, and environmental factors were related to HCPs’ mental health. Topic guide questions focused on HCPs’ perceptions of their experiences of working during the pandemic and explored social, organisational, and environmental factors that may have influenced their self-reported burden and symptoms of psychological distress. These factors were selected from a review of the findings of the previously published meta-analysis and other studies conducted in Nepal [ 9 , 10 , 11 , 12 , 13 ]. The interview questions were piloted with five HCPs for interpretability and interviewer consistency. All interviews were conducted via video conferencing (Zoom) [ 31 ] between September 2021 and March 2022. Five ICU nurses with experience in conducting interviews and mixed methods research led the data collection following training on the topic guide. To ensure there was no prior relationship between the interviewer and the participant, interviewers were assigned to participants that worked in different ICUs than themselves and were not known to the interviewee. No one other than the interviewer and the participant was present for each interview, and interviews were conducted at the time chosen by the interviewee. Rapid assessment procedure (RAP) sheets were used for note-taking during the interviews [ 32 ]. Commonly used in rapid evaluations - designed to improve the rapidity and replicability of research during public health emergencies - RAP sheets help reduce the need for long-form transcription and encourage reflexivity for both interviewers and researchers, reduce interviewer bias, and enable validation of internal consistency with coding [ 33 ]. The RAP sheet contained the summary of questions from the topic guide, and the interviewers took notes of what the participants said regarding each question during the interview.

Data analysis

Descriptive statistics were used to describe participants’ demographics and professional profiles. Psychological health and well-being assessment tools from the questionnaire were analysed using published guidelines. For the BDI, each of the 21 items corresponding to a symptom of depression was summed for each participant to give a single total score [ 16 ]. With each item ranging from 0 to 3 points, a total score of 13 or less was considered minimal to no depression, 14 to 19 as mild depression, 20 to 28 as moderate depression, and 29 to 63 as severe depression [ 16 ]. Data is also presented separately for suicidality (question 9 from the BDI) whereby anyone that said they have thoughts about or plans to kill themselves is said to have experienced suicidality. The BAI scores reported included the 21 symptoms of anxiety that ranged between 0 and 63 points [ 15 ]. The values for each symptom were summed, and a total score of 0 to 7 was interpreted as a minimal level of anxiety, 8 to 15 as mild, 16 to 25 as moderate, and 26 to 63 as severe anxiety [ 15 ]. Scores on the PSS ranged from 0 to 40, with higher scores indicating higher perceptions of stress [ 17 ]: scores ranging from 0 to 13 were considered low descriptors of stress; 14 to 26 moderate; and 27 to 40 were considered higher levels of perceived stress. For alcohol use disorder reported using AUDIT [ 19 ], a score of 0 indicated no previous or current alcohol use; a score of 1 to 7 suggested low-risk consumption; 8 to 14 hazardous or harmful alcohol consumption; 15 or higher indicated the likelihood of alcohol dependence (moderate to severe alcohol use disorder). The PCL-5 included 20 items with a score range of 0 to 80 and a score of 33 or higher, indicating the presence of PTSD [ 18 ]. A sensitivity analysis was conducted for the BDI, BAI and AUDIT scores based on local validation studies whereby a score of 15 or lower from the BDI indicated no depression [ 20 ], 12 or lower from the BAI indicated no anxiety [ 21 ], and a score of 11 or above from the AUDIT indicated discriminate dependent drinkers [ 22 ].

RAP sheets, along with interviewer notes, were reviewed by the research team before analysis to ensure information was complete. SK, KK and AB used a constant comparative method, coding data following each round of interviews and then reflecting back on the summary of the codes together with the interviewers to promote the accuracy of findings and reduce recall and interviewer bias. In addition, emerging themes identified following each round of coding were used to guide subsequent interviews [ 34 ]. The broader research team met following each coding round to review the findings and reflexivity [ 35 ]. Categories and the subsequent themes (‘drivers’) were developed through the iterative process of interviewing, coding, analysing, and reviewing.

We invited 120 doctors and 341 nurses to participate. A total of 21 doctors and 113 nurses responded, all of which completed the BDI, BAI, PSS, and AUDIT questions; 100 completed the PCL-5 (16 doctors and 84 nurses). Nearly all nurses were female (99%, n  = 112), whereas most doctors were male (81%, n  = 17). The characteristics of respondents are described in Table  1 .

50% ( n  = 67) of respondents reported experiencing symptoms associated with moderate to severe anxiety, and a further 27% ( n  = 36) scored for mild anxiety as a result of working in the ICU during the COVID-19 pandemic (Table  2 ). Anxiety levels (and associated symptoms) were more pronounced in nurses than doctors, with 55% ( n  = 62) of the former scoring moderate to severe on the anxiety scale, compared to 24% ( n  = 6) of the latter. 21% ( n  = 28) of respondents described symptoms associated with moderate to severe depression, with a near-even split between nurses and doctors. Three-quarters of respondents ( n  = 114; 85%) had scores indicative of moderate to high levels of stress; this proportion was higher among doctors ( n  = 19; 91%) compared to nurses ( n  = 95; 84%). Of the 100 individuals that completed the PCL-5 assessment (16 doctors and 84 nurses), 45% ( n  = 46) reported a constellation of symptoms closely associated with PTSD, with a higher prevalence among nurses ( n  = 40; 47%) compared to doctors ( n  = 6; 38%).

Using cut-off scores from Nepali validation studies, 45 (34%) participants were experiencing mild, moderate or severe depressive symptoms, 80 (60%) were experiencing mild, moderate or severe anxiety symptoms, and 3 (2%) were considered discriminate dependent drinkers. These results are in line with our main analysis, including that a greater proportion of nurses were still found to suffer from depression and anxiety symptoms (supplementary Table 1 ).

Forty-six respondents to the online questionnaire volunteered to participate in the subsequent semi-structured interviews. Twenty participants were approached and consented to an interview: 16 were nurses (all female), and 4 were doctors (1 female, 3 male). On average, each interview resulted in 45 to 60 min of qualitative data. Saturation was met within the first 15 interviews, and findings were consistent between the coders and the research team. Analysis and synthesis of the interviews revealed nine themes, which, when codified, can be described as three key drivers of the psychological symptoms and impacts on mental well-being experienced by the interviewees: social stigmatism, physical and emotional safety, and organisational support. (Fig.  1 ). During the interviews, HCPs further described some of the coping strategies that they found helpful in mitigating the impacts experienced and may provide insights for future pandemic preparedness. These three themes, the drivers, and coping strategies, are explored below, along with quotes from the respondents.

Coding tree for the four main drivers for psychological distress

Social stigmatism

Interviewees described experiencing feelings of social stigmatisation as a result of interactions with their families, peers, as well as from the wider public. Examples of stigmatism experienced included physical avoidance from neighbours and community members when the HCP travelled to and from and around their home, especially when dwellings were in shared buildings and common areas.

“My house owner avoided talking and meeting me because I worked with COVID patients.” [N]. “I have an elderly family member, and I was afraid and worried [for them] when I came back from duty.” [N].

Interviewees described how rumours would spread within the community, notably related to concerns of risk of co-infection or cross-infection, either directly from parent to child or indirectly via friends and extended family. Some HCPs were asked or elected to stay away from their home so as to reduce the stigma to them and their family and in an attempt to reduce the risk of co-infection, particularly when they had vulnerable family members. Interviewees described how this self-selected or enforced separation and isolation resulted in feelings of rejection, physically and emotionally heightened feelings of stress and anxiety, alongside the threat to physical and emotional safety.

Physical and emotional safety

Increased workload and an enforced change in working pattern/ shift structures were experienced by all the HCPs interviewed. These longer overall working hours, increased duration of shift patterns, and enforced working rotas were perceived as resulting in a loss of physical and emotional safety by the interviewees. Feelings of loss of control, insomnia, or disruption to sleep patterns, alongside physical discomfort through sustained working in personal protective equipment, often in hot and humid temperatures. This physical and mental endurance contributed to feelings of emotional stress and anxiety.

“Shift frequency was increased, and I only got one night off in a week. Sometimes I had to work extra hours, which was very stressful.” [N]. “My sleep pattern had changed, I felt restless and was afraid about COVID” [D].

The change in shift structure and in working patterns meant for some HCPs enforced separation from family and friends whereby HCPs sought accommodation away from family or in temporary lodgings. This again resulted in isolation and additional strain on other family members so as to provide care for HCP’s dependents.

“I had to involve other family members to arrange for the medication and care of my grandmother” [N].

Increased working hours and changes in working patterns further had physical impacts; participants described skipping meals or having limited time to eat. The need to wear personal protective equipment (PPE), and indeed the risks to safety when PPE was not available, associated risks of non-availability of equipment, brought with it a risk to physical and emotional safety. HCPs interviewed reported skin lacerations, irritation, and discomfort whilst wearing equipment in hot, humid working environments.

“We had to frequently change the PPE and masks, which has caused skin problems that still exist.” [N].

Organisational support

Interviewees found the COVID-19 pandemic brought new and often enforced work responsibilities, some of which were associated with high levels of professional anxiety, stress, and uncertainty. A professionally challenging situation, even for those with many years of ICU working experience. HCPs faced emotionally challenging tasks such as dealing with end-of-life situations (particularly without relatives of the patient present) and having to comfort relatives over the phone, of which they received limited to no training or support on handling such situations.

“I went through an emotional breakdown while dealing with the end of the life situation of patients without the presence of family members in the COVID ICU… I felt sad when a young patient lost their lives” [D]. “Accommodation or isolation facilities should be provided by the hospital” [D]. “If incentives were provided in time and staff were provided with health insurance it would motivate us” [N].

Ever-changing role and responsibilities created anxiety for HCPs as to what care to deliver, and the rapidity and uncertainty of care were associated with feelings of vulnerability. Interviewees expressed how they wished there was a need for greater organisational support to better cope with the frequent updates and changes to practice. Furthermore, HCPs expressed concerns regarding a shortage of staff and the lack of mental health counselling and support, accommodation on-site at the hospital, and transportation to and from work.

“Mental health support or counselling facilities were not provided. It should be there… seniors and hospital staff should also talk to the staff to know the situation.” [N]. “Safety of healthcare workers should be the priority and nurse-patient ratio should be maintained to provide quality care to the patients… hospital should have recruited more staff.” [N].

Coping strategies

Participants described various ways in which they coped with the emotional, physical, social, and professional impacts of working through the pandemic. This included speaking with family and friends about the pressures they were under, taking up activities in their off time, such as gardening and reading, and using media entertainment such as music, movies, and shows. A few participants also mentioned that comparing the situation in Nepal to other countries (i.e., keeping up-to-date with the news) also helped them cope. Others mentioned that detachment from social media and more self-awareness through meditation helped.

“I ventilated my feelings with friends and family. Listening to soothing music also helped me cope with the stress.” [N]. “I coped by gardening with my sister in my home.” [N]. “I… watched the news that compared the death rates, which was low compared to others.” [D].

The COVID-19 pandemic’s impact on healthcare services and population health internationally is unprecedented in recent times. As healthcare professionals, policymakers, and researchers work to strengthen services in preparation for future pandemics now and mitigate the long-term impacts on individual and population health, understanding the impact on and perspectives of doctors and nurses at the frontline of care can provide important learning regarding the individuals characteristics and professional, social and economic drivers which may increase the risk of psychological impacts.

Mandated and enforced changes in role, specifically in working hours and shift patterns, were a key driver of psychological anxiety and distress. Within hospitals in Nepal, many departments were closed, and stay-at-home orders meant that outpatient or clinical services all but ceased. This resulted in an increased role and scope for critical care trained staff, and in contrast to other health systems (such as the UK) where healthcare staff were redeployed to ICU, there was a separation for ICU staff even from their professional peers working in other specialties. The increased scope and uncertainty of the HCP’s role, along with limited choice in redeployment in the ICU was another driver of poor mental health- and dominated nursing participants’ experiences. Interviewees described how these changes impacted not only themselves but the multigenerational families for whom many cared for. This enforcement of role change, and the related descriptions of the drivers for these impacts as experienced by participants in this study point not only to the differences in roles between nurses and doctors; but also highlights disparities in autonomy, advocacy for role change during international emergencies, and the implications of work on home and family life [ 36 ].

Giving staff choice to select shift patterns and ensuring the opportunity to have periods of rest to reconnect with family and have self-care is needed. Consultation and shared decision-making, even in times of restricted choice, are associated with improved perceptions of work from staff and may result in reducing psychological distress and promoting emotional safety, which is, in turn, associated with better outcomes for patients [ 37 , 38 ]. However, nurses in Nepal, as with many health systems, may have less opportunity for strategic and organisational decision making in response to public health emergencies. The impact of ongoing disparities between professionals and their agency to advocate for wellbeing and safety warrants further research.

Nurses were disproportionately burdened by both occurrence and severity of symptoms of anxiety and depression as a result of their work during the pandemic when compared to doctors.

Nearly half of all respondents had symptoms of anxiety and PTSD (again more prevalent in nurses), and the burden of anxiety symptoms was higher than the reported 22–33% from a recent umbrella review [ 39 ]. The burden of stress we report was also higher than a smaller study conducted in Nepal during the pandemic, which reported stress among 53.2% of healthcare professionals working in hospitals, primary health centres, pharmacies, and health posts in Nepal [ 40 ]; it was also higher than a meta-analysis of published studies exploring the incidence of both stress (57%) and PTSD (22%) among all cadres of healthcare workers [ 41 ]. One reason for the higher reported symptoms in our study may be the focus on ICU workers and their role in the management of end-of-life care. Indeed, our results for depression and anxiety are comparable to a study involving nurses working directly with COVID-19 in Nepal [ 13 ]. Studies conducted elsewhere in Asia have highlighted this positive relationship between ICU experiences and poor mental health [ 42 ].

Nurses in Nepal, as with many other countries, are more likely to be female, younger in age, and have less opportunity for graduate study; and have lower earning potential than physician colleagues [ 43 ]; all characteristics associated with increased risk of poorer mental health outcomes [ 44 ]. Exploration into the disparities of the psychological and health impacts of COVID-19 on different cadres of healthcare workers is emerging. A systematic review conducted in 2020, identified 27 studies which sought to explore the disparity in impacts of the pandemic on HCP’s psychological well-being. The findings from the review are in line with ours, indicating that the burden of symptoms for anxiety, depression, and PTSD is higher in nurses compared to doctors [ 45 ]. Notably only a few of these studies used validated tools for assessment of specific symptoms of anxiety, depression, or substance misuse [ 45 ]. Our study serves to strengthen the evidence of the vulnerability of nurses.

Nepal, like many other lower and middle-income countries in South and Southeast Asia, enforced large-scale lockdowns and restrictions of movement for all but essential healthcare and municipal staff [ 46 ]. As such, social stigmatism, physical and emotional safety, and organisational support were key drivers behind the elevated symptoms of psychological distress in ICU HCPs and may be a key determinant of differences between health systems internationally. Furthermore, the family responsibilities and social circumstances for nurses, contributed to their experiences of isolation, rejection, vulnerability, physical discomfort, and strain. These drivers mirrored those reported from Europe; and may reflect differences experienced by nurses as a result of their gender, and role norms of primary family carers within society [ 44 ].

Interviewees from both professional groups expressed concern at the absence of preparedness and support they felt from their employing institutions. This is notable given the ongoing investment in pandemic preparedness and the potential to make changes now to prepare for the next pandemic or public health emergency. Interventions such as resilience training, scenario-based simulation training, and group exercises based on psychoeducation and cognitive behavioural therapy (CBT) principles have proved effective in reducing anxiety, depression, stress, and PTSD among doctors and nurses while simultaneously improving their ability to work in unprecedented situations in other sectors [ 47 ]. Similar provisions may be valuable for ICU-based healthcare professionals and are deliverable online, making rollout potentially more feasible.

Strengths and limitations

A strength of this study is the exploration of participants’ perspectives on the drivers behind the burden of poor mental health described in ICU HCPs. This mixed methods approach offers insights into doctors’ and nurses’ unique individual, social and professional characteristics that may be associated with increased risk of distress. These differences and their potential for disparity in impacts on health and wellbeing should be of interest to policymakers and healthcare facility managers involved in future pandemic preparedness. However, the study has some limitations to acknowledge. Given the use of the snowball technique, we were able to ensure a high number of respondents, but as a consequence, we were unable to track the number of respondents that came from using this technique compared to those initially invited from the NSCCM and CCNAN. Therefore, a response rate and, subsequently, a non-response rate could not be reported. We did not collect information on the level of training in critical care that participants received; trained health professionals are likely to have additional skills in how to handle the potential stressful environment in critical care settings. Also, due to the lack of validation of the PCL-5 in Nepal, the results of this assessment tool should be interpreted with caution. The survey tools used for this study have not been validated in an online format. However, given these tools were self-reporting, and were piloted and administered in English, the online format is thought to have minimal impact on the results. Additionally, participants for the qualitative component were recruited based on convenience sampling; therefore, the diversity of the sample may not be optimised. We acknowledge that recall bias may be present in the participants during the interview, given they were recalling their experiences throughout the pandemic for up to 24 months prior to the interview; however, we hope the piloting of the interviews, the use of multiple researchers to code the data, and the constant comparative nature of the evaluation will mitigate this potential.

The COVID-19 pandemic negatively impacted the mental health of HCPs worldwide. This study strengthens existing evidence that nurses were (and may remain) at increased risk of both cross infection and may also be more vulnerable to psychological impacts including anxiety, depression and PTSD than their professional colleagues. In addition, critical care staff may be at even greater risk, due to the uniqueness of their role which includes prolonged periods of time with infected patients, frontline role in managing end of life care, and as described here, limited ability to advocate for changing role and working patterns during an emergency. Professional hierarchies, and social-economic and gender profiles unique to nurses, may be potential drivers for these disparities, and warrants further research. Learning from the ICU HCPs’ experiences during the COVID-19 pandemic may inform future preparedness strategies e to mitigate short and long-term mental illness among ICU HCPs in future pandemics.

Data availability

The interview guide is available in the Figshare repository,

https://doi.org/10.6084/m9.figshare.24247384.v1 .

The data supporting the conclusions of this article are available in the Figshare repository, https://doi.org/10.6084/m9.figshare.23999790.v1 .

Abbreviations

Coronavirus disease 2019

Intensive care unit

Healthcare professional

Personal protective equipment

Post-traumatic stress disorder

Nepalese Society of Critical Care Medicine

Critical Care Nurses Association of Nepal

Beck Anxiety Inventory

Beck Depression Inventory

Perceived Stress Scale

PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders-5

Alcohol Use Disorder Identification Tool

Rapid assessment procedure

Cognitive behavioural therapy

Wang H, Paulson KR, Pease SA, et al. Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21. Lancet. 2022;399(10334):1513–36.

Article   CAS   Google Scholar  

Greco M, De Corte T, Ercole A et al. Clinical and organizational factors associated with mortality during the peak of first COVID-19 wave: the global UNITE-COVID study. Intensive Care Med. 2022:1–16.

All Covid restrictions lifted in Valley [Internet]. kathmandupost.com. [Accessed 7 Feb 2024]. Available from: https://kathmandupost.com/valley/2022/03/05/all-covid-restrictions-lifted-in-valley .

Kharel P. Nepal’s fight against the second wave of COVID-19 pandemic. repositoryunescaporg [Internet]. 2021 [cited 2024 Feb 7]; Available from: https://hdl.handle.net/20.500.12870/5102 .

Neupane HC, Gauli B, Adhikari S, et al. Contextualizing critical care medicine in the face of COVID-19 pandemic. JNMA J Nepal Med Assoc. 2020;58(226):447.

PubMed   PubMed Central   Google Scholar  

Shrestha GS, Lamsal R, Tiwari P, Acharya SP. Anesthesiology and critical care response to COVID-19 in Resource-Limited settings. Anesthesiol Clin. 2021;39(2):285–92.

Article   PubMed   PubMed Central   Google Scholar  

Hölscher AH. Patient, surgeon, and health care worker safety during the COVID-19 pandemic. Ann Surg. 2021;274(5):681.

Article   PubMed   Google Scholar  

Gholami M, Fawad I, Shadan S, et al. COVID-19 and healthcare workers: a systematic review and meta-analysis. Int J Infect Dis. 2021;104:335–46.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ghahramani S, Kasraei H, Hayati R et al. Health care workers’ mental health in the face of COVID-19: a systematic review and meta-analysis. Int J Psychiatry Clin Pract. 2022:1–10.

Gupta AK, Mehra A, Niraula A, et al. Prevalence of anxiety and depression among the healthcare workers in Nepal during the COVID-19 pandemic. Asian J Psychiatr. 2020;54:102260.

Adhikari SP, Rawal N, Shrestha DB et al. Prevalence of anxiety, depression, and perceived stigma in healthcare workers in Nepal during later phase of first wave of covid-19 pandemic: a web-based cross-sectional survey. Cureus. 2021, 13(6).

Khanal P, Devkota N, Dahal M, et al. Mental health impacts among health workers during COVID-19 in a low resource setting: a cross-sectional survey from Nepal. Global Health. 2020;16(1):1–12.

Google Scholar  

Tamrakar P, Pant SB, Acharya SP. Anxiety and depression among nurses in COVID and non-COVID intensive care units. Nurs Crit Care. 2021 Sep 28.

Fitzpatrick R, Boulton M. Qualitative methods for assessing health care. Qual Health Care. 1994;3(2):107.

Beck AT, Epstein N, Brown G, et al. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893.

Article   CAS   PubMed   Google Scholar  

Richter P, Werner J, Heerlein A, et al. On the validity of the Beck Depression Inventory. Psychopathology. 1998;31(3):160–8.

Cohen S, Kamarck T, Mermelstein R. Perceived stress scale. Measuring Stress: Guide Health Social Scientists. 1994;10(2):1–2.

Blevins CA, Weathers FW, Davis MT, et al. The posttraumatic stress disorder checklist for DSM-5 (PCL‐5): development and initial psychometric evaluation. J Trauma Stress. 2015;28(6):489–98.

Saunders JB, Aasland OG, Babor TF, et al. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction. 1993;88(6):791–804.

Kohrt B, Kunz RD, Koirala NR, Sharma VD, Nepal, Mahendra. Validation of a Nepali version of the Beck Depression Inventory. Nepal J Psychiatry. 2002;2:123–30.

Kohrt B, Kunz R, Koirala N, Campus M, Nepal. Validation of the Nepali version of beck anxiety inventory. Journal of Institute of Medicine [Internet]. 2007 Jan 21 [cited 2024 Feb 7]; Available from: https://www.semanticscholar.org/paper/Validation-of-the-Nepali-version-of-beck-anxiety-Kohrt-Kunz/148d 01852884f54d28967e105d57c177a3f0be36.

Pradhan B, Chappuis F, Baral D, Karki P, Rijal S, Hadengue A et al. The alcohol use disorders identification test (AUDIT): validation of a Nepali version for the detection of alcohol use disorders and hazardous drinking in medical settings. Substance Abuse Treatment, Prevention, and Policy [Internet]. 2012;7(1). Available from: https://substanceabusepolicy.biomedcentral.com/articles/ https://doi.org/10.1186/1747-597X-7-42 .

Sharma P, Devkota G. Mental health screening questionnaire: a study on reliability and correlation with perceived stress score. J Psychiatrists’ Association Nepal. 2019;8(2):4–8.

Article   Google Scholar  

Pandey BD, Morita K, Costello A. Twin crises in Nepal: covid-19 and climate change. 2022, 377.

Nepalese Society of Critical Care Medicine. NSCCM. 2022. [Accessed 16 August 2023]. Available from http://www.nsccm.org.np/ .

Critical Care Nurses Association of Nepal. Critical Care Nurses Association of Nepal. 2022. [Accessed 16 August 2023]. Available from https://ccnan.org.np/ .

Etikan I, Alkassim R, Abubakar S. Comparision of snowball sampling and sequential sampling technique. Biom Biostat Int J. 2016;3(1):55.

Saunders B, Sim J, Kingstone T, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893–907.

Google Inc. Google Forms. 2022. [Accessed 16 August 2023]. Available from https://www.google.co.uk/forms/about/ .

Zerbini G, Ebigbo A, Reicherts P et al. Psychosocial burden of healthcare professionals in times of COVID-19–a survey conducted at the University Hospital Augsburg. Ger Med Sci. 2020, 18.

Zoom Z. One platform to connect. 2022. [Accessed 16 August 2023]. Available from https://zoom.us/ .

Vindrola-Padros C, Chisnall G, Cooper S, et al. Carrying out rapid qualitative research during a pandemic: emerging lessons from COVID-19. Qual Health Res. 2020;30(14):2192–204.

Beebe J. Basic concepts and techniques of rapid appraisal. Hum Organ. 1995;54(1):42–51.

Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc. 2016:211–7.

Macbeth D. On reflexivity in qualitative research: two readings, and a third. Qual Inq. 2001;7(1):35–68.

Jackson D, Anders R, Padula WV, et al. Vulnerability of nurse and physicians with COVID-19: monitoring and surveillance needed. J Clin Nurs. 2020;29(19–20):3584.

Bae SH. Intensive care nurse staffing and nurse outcomes: a systematic review. Nurs Crit Care. 2021;26(6):457–66.

Ejebu O-Z, Dall’Ora C, Griffiths P. Nurses’ experiences and preferences around shift patterns: a scoping review. PLoS ONE. 2021;16(8):e0256300.

Fernandez R, Sikhosana N, Green H, et al. Anxiety and depression among healthcare workers during the COVID-19 pandemic: a systematic umbrella review of the global evidence. BMJ Open. 2021;11(9):e054528.

Kafle K, Shrestha DB, Baniya A, et al. Psychological distress among health service providers during COVID-19 pandemic in Nepal. PLoS ONE. 2021;16(2):e0246784.

Marvaldi M, Mallet J, Dubertret C, et al. Anxiety, depression, trauma-related, and sleep disorders among healthcare workers during the COVID-19 pandemic: a systematic review and meta-analysis. Neurosci Biobehav Rev. 2021;126:252–64.

Thatrimontrichai A, Weber DJ, Apisarnthanarak A. Mental health among healthcare personnel during COVID-19 in Asia: a systematic review. J Formos Med Assoc. 2021;120(6):1296–304.

Prakash S, Yadav P, Yadav K. Perspectives of developing nursing education in Nepal. Nurs Care Open Access J. 2018;5(4):214–20.

Tiete J, Guatteri M, Lachaux A, et al. Mental health outcomes in healthcare workers in COVID-19 and non-COVID-19 care units: a cross-sectional survey in Belgium. Front Psychol. 2021;11:612241.

Kunz M, Strasser M, Hasan A. Impact of the coronavirus disease 2019 pandemic on healthcare workers: systematic comparison between nurses and medical doctors. Curr Opin Psychiatry. 2021;34(4):413.

Adhikari NK, Beane A, Devaprasad D et al. Impact of COVID-19 on non-COVID intensive care unit service utilization, case mix and outcomes: a registry-based analysis from India. Wellcome Open Res. 2021, 6.

Zaçe D, Hoxhaj I, Orfino A, et al. Interventions to address mental health issues in healthcare workers during infectious disease outbreaks: a systematic review. J Psychiatr Res. 2021;136:319–33.

Download references

Acknowledgements

We thank the volunteers who took the time to interview the participants: Radhika Maharjan, Dipika Khadka, Anita Bashyal, Samina Amatya, and Roshani Kafle. We also want to thank Dr. Rohini Nepal and Jugmaya Chaudhary of Rhythm Neuropsychiatry Hospital and Research Centre for their contribution to advising and reviewing the self-reporting psychological assessment tools used in the questionnaire. We would also like to thank Transcultural Psychosocial Organisation (TPO) Nepal and Dr. Nabaraj Koirala for the permission to use the Nepali-validated version of BDI I and BAI for the study. We additionally thank Nilu Dullewe, who helped in coding the qualitative data. For the ongoing mutual support for improvements in ICU care, we would also like to acknowledge and thank members of the CCAA.

CCAA members

Diptesh Aryal, Shirish KC, Kanchan Koirala, Subekshya Luitel, Rohini Nepal, Sushil Khanal, Hem R Paneru, Subha K Shreshta, Sanjay Lakhey, Samina Amatya, Kaveri Thapa, Radhika Maharjan, Roshani Kafle, Anita Bashyal, Reema Shrestha, Dipika Khadka and Nilu Dullewe.

This study was funded by a Wellcome Innovations Flagship Programme grant (Wellcome grant number: 215522/Z/19/Z). They had no role in the design, analysis, or reporting of this protocol.

Author information

Authors and affiliations.

Nepal Intensive Care Research Foundation, Kathmandu, Nepal

Shirish KC, Diptesh Aryal, Kanchan Koirala & Subekshya Luitel

Institute of Applied Health Research, University of Birmingham, Birmingham, UK

Tiffany E. Gooden

Centre for Inflammation Research, University of Edinburgh, Edinburgh, UK

Rashan Haniffa & Abi Beane

Mahidol Oxford Tropical Medicine Research Unit, Bangkok, Thailand

You can also search for this author in PubMed   Google Scholar

• Diptesh Aryal
• , Shirish KC
• , Kanchan Koirala
• , Subekshya Luitel
• , Rohini Nepal
• , Sushil Khanal
• , Hem R Paneru
• , Subha K Shreshta
• , Sanjay Lakhey
• , Samina Amatya
• , Kaveri Thapa
• , Radhika Maharjan
• , Roshani Kafle
• , Anita Bashyal
• , Reema Shrestha
• , Dipika Khadka
•  & Nilu Dullewe

Contributions

All authors conceptualised this study. SK, DA, AB, RH, and SL developed the protocol, study methods, and materials. KK and SL facilitated the data collection, supervised by SK and DA. Data were analysed by SK, AB, KK, and TEG. SK and TEG wrote the drafts of the manuscript, and all authors reviewed the manuscript and consented to it being submitted. AB is the senior author.

Corresponding author

Correspondence to Diptesh Aryal .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was granted from the Nepal Health Research Council (approval number: 176/2021 P). All participants provided informed consent electronically before completing the online questionnaire. Participants from the qualitative component provided further informed verbal consent before the interview commenced.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Members of the CCAA are listed in the acknowledgments

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

KC, S., Gooden, T.E., Aryal, D. et al. The burden of anxiety, depression, and stress, along with the prevalence of symptoms of PTSD, and perceptions of the drivers of psychological harms, as perceived by doctors and nurses working in ICUs in Nepal during the COVID-19 pandemic; a mixed method evaluation. BMC Health Serv Res 24 , 450 (2024). https://doi.org/10.1186/s12913-024-10724-7

Download citation

Received : 05 September 2023

Accepted : 14 February 2024

Published : 10 April 2024

DOI : https://doi.org/10.1186/s12913-024-10724-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Pandemic preparedness, psychological distress
• Healthcare professionals
• Critical care

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 12 April 2024

Building public engagement and access to palliative care and advance care planning: a qualitative study

• Rachel Black   ORCID: orcid.org/0000-0001-8952-0501 1 ,
• Felicity Hasson   ORCID: orcid.org/0000-0002-8200-9732 2 ,
• Paul Slater   ORCID: orcid.org/0000-0003-2318-0705 3 ,
• Esther Beck   ORCID: orcid.org/0000-0002-8783-7625 4 &
• Sonja McIlfatrick   ORCID: orcid.org/0000-0002-1010-4300 5  

BMC Palliative Care volume  23 , Article number:  98 ( 2024 ) Cite this article

33 Accesses

2 Altmetric

Metrics details

Research evidence suggests that a lack of engagement with palliative care and advance care planning could be attributed to a lack of knowledge, presence of misconceptions and stigma within the general public. However, the importance of how death, dying and bereavement are viewed and experienced has been highlighted as an important aspect in enabling public health approaches to palliative care. Therefore, research which explores the public views on strategies to facilitate engagement with palliative care and advance care planning is required.

Exploratory, qualitative design, utilising purposive random sampling from a database of participants involved in a larger mixed methods study. Online semi-structured interviews were conducted ( n  = 28) and analysed using reflexive thematic analysis. Thematic findings were mapped to the social-ecological model framework to provide a holistic understanding of public behaviours in relation to palliative care and advance care planning engagement.

Three themes were generated from the data: “Visibility and relatability”; “Embedding opportunities for engagement into everyday life”; “Societal and cultural barriers to open discussion”. Evidence of interaction across all five social ecological model levels was identified across the themes, suggesting a multi-level public health approach incorporating individual, social, structural and cultural aspects is required for effective public engagement.

Conclusions

Public views around potential strategies for effective engagement in palliative care and advance care planning services were found to be multifaceted. Participants suggested an increase in visibility within the public domain to be a significant area of consideration. Additionally, enhancing opportunities for the public to engage in palliative care and advance care planning within everyday life, such as education within schools, is suggested to improve death literacy and reduce stigma. For effective communication, socio-cultural aspects need to be explored when developing strategies for engagement with all members of society.

Peer Review reports

It is estimated that globally only 14% of patients who require palliative support receive it [ 1 ]. The World Health Organisation (WHO) advocates for palliative care (PC) to be considered a public health issue and suggests earlier integration of PC services within the wider healthcare system is required [ 2 ]. However, research has shown that a lack of public knowledge and misconceptions about PC may deter people from accessing integrative PC services early in a disease trajectory [ 3 ]. Integral to good PC is the facilitation of choice and decision-making, which can be facilitated via advance care planning (ACP). Evidence suggests that ACP can positively impact the quality of end of life care and increase the uptake of palliative care services [ 4 ]. While ACP is commonly associated with end of life (EOL) care, it provides the opportunity for adults of any age to consider their wishes for future care and other financial and personal planning. However, there is evidence of a lack of active engagement in advance care planning (ACP) [ 5 ]. Recent research exploring knowledge and public attitudes towards ACP found just 28.5% of participants had heard the term and only 7% had engaged in ACP [ 6 ]. Barriers to engagement in ACP discussions have been found to include topics such as death and dying are considered a social taboo, posing an increased risk of distress for loved ones; and [ 6 ] a misconception that ACP is only for those at the end of life rather than future planning [ 7 ]. Therefore, there is a need for a public health approach to ACP, to enable and support individuals to engage in conversations about their wishes and make decisions surrounding their future care.

The need for a public health approach to PC, to tackle the challenges of equity and access for diverse populations, was noted in a recent Lancet paper [ 8 ]. This is further supported in a recent review, exploring inequalities in hospice care in the UK, Australia, New Zealand, and Canada which reported that disadvantaged groups such as those with non-cancer illnesses, people living in rural locations and homeless individuals had unequal access to palliative care [ 9 ]. They postulated that differing levels of public awareness in what hospice care provides, and to whom, was an influencing factor with variations in health literacy and knowledge of health services being present in both minority and socioeconomic groups [ 9 ].

Changes in how we experience death and dying have resulted in a shift away from family and community settings into healthcare settings. The Lancet commission exploring the ‘Value of Death’, suggests it has created an imbalance where the value of death is no longer recognised [ 10 ]. The commission’s report posits the need to rebalance death, dying and grieving, where changes across all death systems are required. This needs to consider how the social, cultural, economic, religious, and political factors that determine how death, dying, and bereavement are understood, experienced, and managed [ 10 ].

New public health approaches that aim to strengthen community action and improve death literacy, through increased community responsibility are reflected in initiatives, such as ‘Compassionate Communities’ and ‘Last Aid’ [ 11 , 12 ]. However, a suggested challenge is the management of potential tensions that are present when attempting to conceptualise death in a way that mobilises a whole community [ 13 ]. Whilst palliative care education (PCE) can be effective in improving knowledge and reducing misconceptions, many PCE intervention studies, have focused on carers and healthcare professionals [ 14 ]. Initiatives such as ‘Last Aid’ attempt to bridge this gap by focusing on delivering PCE to the public, however, they are not embedded into the wider social networks of communities. It can be argued that public health campaigns, such as these are falling short by neglecting to use the full range of mass media to suit different ages, cultures, genders and religious beliefs [ 15 ]. Consequently, to understand what is required to engage the public successfully, the voice of the public must lead this conversation. Therefore, this study sought to explore public views on strategies and approaches to enable engagement with palliative care and advance care planning to help share future debate and decision making.

Within the last decades the delivery of PC and ACP have been increasingly medicalised and viewed as a specialist territory, however in reality, the care of those with life-limiting conditions occurs not only within clinical settings but within a social structure that affects the family and an entire community [ 16 ]. Therefore, death, dying and bereavement involve a combination of social, physical, psychological and spiritual events, therefore, to frame PC and ACP within a public health approach the response requires a shift from the individual to understanding the systems and culture within which we live. The Social Ecological Model (SEM) recognises the complex interplay between individual behaviours, and organisational, community, and societal factors that shape our acceptance and engagement. SEM provides a framework to understand the influences affecting engagement with PC and ACP and has been utilised as a lens through which the data in this study is explored.

Qualitative research, using semi-structured interviews were adopted as this enabled an in-depth understanding of public views on strategies to enable engagement with PC. This research was part of a larger mixed-methods study [ 17 ]. Comprehensive Consolidated Criteria for Reporting Qualitative research (COREQ) were used [ 18 ](See Supplementary file 1 ).

A purposive random sampling method, using a random number generator, was adopted to recruit participants who consented to be contacted during data collection of a larger mixed methods study. Selected individuals were contacted by telephone and email to invite them to participate. Inclusion and exclusion criteria are outlined in Table  1 . Interested individuals were provided with a participant information sheet detailing the aims of the study and asked to complete a consent form and demographic questionnaire.

A total of 159 participants were contacted, 105 did not respond, 21 declined and three were ineligible to participate. A total of thirty participants consented, however, two subsequently opted to withdraw prior to the interview.

Data collection

Data was collected from December 2022 to March 2023 by RB. The qualitative interview schedule comprised four broad topic areas: (1) participants’ knowledge of PC and ACP; (2) sources of information on PC and ACP and current awareness of local initiatives for public awareness; (3) knowledge of accessibility to PC and ACP and (4) future strategies for promoting public awareness of PC and ACP, with a consideration of supporting and inhibiting factors. The interview schedule was adapted from a previous study on palliative care to incorporate the topic of ACP [ 3 ] (See Supplementary file 2 ). This paper reports on future strategies.

Participants were asked to complete a short demographic questionnaire prior to the interview to enable the research team to describe the characteristics of those who participated. These questions included variables such as age, gender, religion, marital status, behaviour relating to ACP and experience of PC.

Data was collected via online interviews conducted using the videoconferencing platform Microsoft Teams. Interviews lasted between 20 and 60 min and were recorded with participant consent. Data were stored on a secure server and managed through NVivo 12 Software.

Data analysis

Qualitative data were transcribed verbatim automatically by Microsoft Teams and the transcripts were reviewed and mistakes corrected by the interviewer. All identifying information was removed. Transcripts were analysed using reflexive thematic analysis which involved a six-step process: familiarisation, coding, generating initial themes, developing and reviewing themes, refining, defining and naming themes, and writing up [ 19 ]. Themes were derived by exploring patterns, similarities and differences within and across the data in relation to participant’s views on the promotion of PC and ACP and the best ways to engage the public in open discussions.

The study explored the data through a SEM lens to provide a holistic framework for understanding the influences surrounding health behaviour change in relation to palliative care and advance care planning by mapping the findings to each of the SEM constructs.

The SEM for public health was conceptualised by McLeroy et al. [ 20 ]., and was based on previous work by Bronfenbrenner’s ecological systems theory [ 21 ]. The SEM looks to identify social-level determinants of health behaviours [ 22 ]. Five factor levels have been identified within the SEM; (1) Intrapersonal factors (2) Interpersonal processes (3) Institutional factors (4) Community factors and (5) Public policy [ 20 ]. In short, the SEM suggests that the social factors that influence health behaviours on an individual level are nestled within a wider complex system of higher levels. Current research literature has explored SEM as a model for understanding barriers and facilitators to the delivery of PC, adults’ preferences for EOL care and older adults’ knowledge and attitudes of ACP within differing socioeconomic backgrounds [ 23 , 24 , 25 ]. It has demonstrated the importance of a multilevel approach within these populations. However, there is a scarcity of research exploring strategies for public engagement with PC and/or ACP which are underpinned by SEM theory.

To ensure rigour in the analysis four members of the research team (RB, SM, FH, EB) independently reviewed the transcripts and were involved in the analysis and development of themes as a method of confirmability [ 26 ].

Ethical approval was gained from the University Research Ethics Filter Committee prior to commencing data collection. Participants provided written informed consent prior to the commencement of the interviews. They were advised of their right to withdraw, and the confidentiality and anonymity of all data were confirmed. All data was kept in accordance with the Data Protection Act (2018) [ 27 ].

All participants were white; 70% were female (n-19) and 70% were either married or cohabiting (n-19). The largest proportion of the sample 44% was aged under 50 years (n-12), with 22% aged between 50 and 59 (n-6) and 33% (n-9) aged between 60 and 84. Over half of the sample was employed (n-15), 15% were self-employed [ 4 ] whilst 26% were retired (n-7). Demographic data were missing for one of the included participants (see Table  2 ).

Responses to questions relating to ACP knowledge and behaviours found just 12 participants had heard of the term ACP prior to completing the Northern Ireland Life and Times Survey. Furthermore, none of the participants had been offered the opportunity to discuss ACP and none had prepared a plan of their wishes and preferences.

Main findings

Three overarching themes were generated from the data: ‘Visibility and relatability’; ‘Embedding opportunities for engagement into everyday life’; ‘Societal and cultural barriers. These findings were then mapped to the five social ecological model (SEM) levels ( individual; interpersonal; institutional; community; and policy ) to demonstrate the importance of a multilevel approach when seeking to engage the public around PC and ACP. See Fig.  1 for SEM construct mapping.

Theme 1: visibility and relatability

This theme relates to the suggestion that social taboo was a barrier to awareness and the mechanism to ameliorate this was visibility – in turn promoting reduction of stereotypes and promoting understanding and engagement. This posits the idea that the lack of understanding of PC is the root cause of much of the stigma surrounding it. The SEM construct mapping suggests a multilevel approach is required with intrapersonal (increased individual understanding), interpersonal (openness in discussion with friends and family through media normalisation) and institutional (health service policies for promotion and support) levels being identified.

Participants discussed how there is a lack of knowledge on what PC is, with many assuming that it was for people in the latter stages of life or facing end of life care. This highlighted the lack of individual education with participants suggesting that there should be more visibility and promotion on PC and ACP so that individuals are better informed.

“So, it’s really um there needs to be more education, maybe, I think around it. So that people can view it maybe differently or you know talk about it a bit more. Yeah, probably demystifying what it is. This is this is what it is. This isn’t what it is. You know, this isn’t about um, ending your life for you, you know. And this is about giving you choices and ensuring that you know, you know people are here looking after you”(P37538F45) .

However, there was a recognition that individual differences play a part in whether people engage in discussions. A number of participants explored the idea that some people just don’t want to talk about death and that for some it was not a subject that they want to approach. Despite this, there was a sense that increasing visibility was considered important as there will still be many people who are willing to increase their knowledge and understanding of PC and ACP.

“I can talk about it, for example, with one of my sisters, but not with my mom and not with my other sister or my brothers. They just refuse point blank to talk about it…. some of them have done and the others have started crying and just shut me. Shut me off. And just. No, we don’t want to talk about that. So, it just depends on the personality, I suppose” (P14876F59) .

The lack of knowledge and awareness of PC and ACP was suggested to be the attributed to the scarcity of information being made available at a more institutional level. For some participant’s, this was felt to be the responsibility of the health service to ensure the knowledge is out there and being promoted.

“I think people are naive and they know they’re not at that stage and they don’t know what palliative care is, you know. It’s all like it’s ignorance. But our health service is not promoting this. Well in my eyes, they’re not promoting it whatsoever. And they should, they should, because it would help a hell of a lot of people ” (P37172M61). “I and I think it needs to be promoted by the point of contact, whether it’s a GP, National Health, whatever it might be, I think when they’re there, there needs to be a bit more encouragement to have that conversation” (P26495M43) .

The lack of visibility within the general practice was discussed by several participants who said that leaflets and posters would be helpful in increasing visibility. One participant went as far as to say that a member of staff within a GP surgery would be beneficial.

“I suppose the palliative care because it is a bit more personal. There should be even maybe a professional that you could talk to in your GP practice, or you know, like they have mental health practitioners now in GP practices. Maybe there are I don’t know if there is or not, but there should be maybe a palliative health practitioner that talks to people when they are at that stage of their life” (P21647F29).

Participants also noted how there were generational differences in how people accessed health information. Many of the participants suggested that they would turn to the internet and ‘google’ for information, however, the suggestion was made that care should be taken to target awareness campaigns to different age groups via different methods to reduce disparities in technology skills such as those with less computer literacy.

“ I think a certain proportion of society need the visibility because they’re not always going to be self-sufficient enough to jump on the Internet even though you know we’re getting to the point now where the generational thing is. The generation have been brought up with the Internet and they’re obviously they go to it as the first point of call. But we still got the generation at the moment that don’t”. (P25046F-)

One of the participants talked about ways to increase visibility via the use of the media, including social media, and the utilisation of famous faces.

“Yeah, I think you know, they need to discuss it on Loose Women. You know, morning TV need to get on the bandwagon….But you know. It only takes like that one celebrity to mention it and then the whole media is jumping on the bandwagon.” (P19874F-) .

The UK media coverage of other successful campaigns such as those highlighting mental health and bowel cancer were noted to have been particularly helpful in raising awareness.

“if I think myself about the whole exposure that we have and as a society at the minute about mental health in general, you know a lot of the work on that has been done via social media. You know, celebrities hash tagging and talked about their experience. It’s OK to not be OK etcetera. And I feel like that is responsible for a lot of people who are now discussing their mental health” (P21647F29) .

The sentiment expressed in the above quotes regarding increased visibility in the media also suggests that unless a topic seems relevant to an individual then they won’t engage with health promotion. This concept of relatability pertains to those aspects of human empathy where they can place themselves within a situation leading it to become more relevant to them.

Many participants discussed that using real-life stories on television and in campaigns would be an effective way to connect with the public and it would make PC and ACP more relatable and highlight the importance of thinking about it.

“I think always what tends to be most effective is when it’s somebody that we could all relate to telling their own story (…)……I’m not too sure I know enough about what it involves, but really, like the consequences, that the consequences that people have suffered from not having done that. Not knowing what the wishes were, not having planned for it”. (P32288F62)

Several participants discussed how the topics of PC and ACP were not something they would identify with as being relevant to them. The suggestion was that without it being an immediate concern, for example, if they were not approaching a certain age, then they would assume that they did not need to increase their knowledge about what PC or ACP involves.

“You have to be able to relate to it in order to think, oh, yeah, that applies to me. You need to have something in which you identify with”. (P13697F51)

Theme 2: embedding opportunities for engagement into everyday life

Throughout the discussions, there was evidence that participants felt that death literacy could be increased by providing more opportunities to gain knowledge about planning for future care and what PC involves. Education was highlighted as a potential pathway to engaging the public by targeting appropriate age groups and professions with relevant knowledge and skills. For some, this was thought to be best achieved through educating the youth and for others, the importance of educating those who are working in the healthcare profession was particularly salient. In addition, almost half of the participants suggested they would approach charity organisations for information, with participants advocating for education within secondary and tertiary levels and within community organisations. This data reflects an institutional-level construct within the SEM framework.

Educating younger generations on the topics of PC and ACP through open discussions in schools, and providing skills on how to have difficult conversations with loved ones were seen to be a valuable strategies.

“ young people don’t have that ability to accept and admit and bring it out into the open and I think they need to be perhaps encouraged more to do that through some kind of teaching in the school environment when they’re at a young and impressionable age ”. (P25046F-)

Due to the difficulties around having conversations about death, it was suggested that different healthcare professionals should be trained to have conversations with their patients.

“Yeah, it’s like you think the discussions are difficult to broach for maybe health care professionals that you know, a difficult topic even for them to bring up. Well, if you’re working with someone who’s you know with a family and where things are quite distressed and very often it can be either, it can be the stress can cause a lot of friction and you know, decision making can be very difficult for people…., but just at every level, there’s, you know, possibility to be having conversations like that with people.”(P37538F45) . “I suppose you could think about training some care professionals (…) there may be some way that as a second part of the person’s job or whatever that they’re trained so that people could go along and discuss ” (P19265F76) .

Further to education for young people and healthcare professionals, there was the recognition that community organisations are perfectly positioned to educate the public in PC and ACP. One participant highlighted the missed opportunity to educate family members and carers through an existing programme on dementia. This is particularly pertinent to ACP due to the impact of cognitive decline on decision-making.

“I went to zoom meetings for four weeks in a row with the Alzheimer’s Society. Umm regarding things to do with dementia. And you know, there was a week about your finances and things like that. I suppose. They never really talked about end of life care you know that sort of thing. Um I suppose it would have been useful had they you know, broached that subject as well. But it wasn’t, you know, there was more about looking after yourself, looking after the person. The symptoms of dementia and all this sort of thing, Alzheimer’s and then you know, um the financial and the help available to you. You know, but they didn’t mention about the end-of-life care and like the end result of dementia, I guess, is death. So, you know that that subject, you know, I was, I suppose to just those organizations that deal with um the issues of, like dementia or, as you say, all the rest for, you know, the rest of the diseases and that. You know to be up front and honest and say you know where this can lead and to make people you know, make people aware that there is a palliative care process that can be gone through.” (P19874F-) .

Furthermore, the option of a helpline was suggested with reference made to other successful charity helplines such as the National Society for the Prevention of Cruelty to Children (NSPCC) and The Samaritans. Whilst it was acknowledged there were specialist support services available for people living with a terminal diagnosis, there was a sense that a generic information support helpline would be helpful.

“You know, people need to be (aware)… they’re not alone. There is help out there. You know, you see your helpline, your children, NSPCC, your Samaritans. All on all these helplines, I have to think, I’ve never seen a helpline for palliative care or who you can contact. You don’t see things like that.” (P22964F52) .

In addition to education, the suggestion that embedding ACP discussions into other more common aspects of future planning such as will making, and organ donation was postulated as a potential way to engage the public. This demonstrated clear links to the policy and institutional level constructs within SEM. Changes within organisational policies and public law to support individuals to consider future planning would promote better engagement on a wider societal level.

Participants suggested that they would like to see some of the conversations surrounding ACP introduced into workplace policies and guidelines, as well as through other legal discussions. They noted that conversations surrounding future planning already occur when discussing legal wills and workplace pensions. The potential to expand these discussions to include ACP with solicitors and in workplaces was seen to be a missed opportunity.

“So maybe um around people who are making their wills and you know, you get will making services advertised and things like that. And I think once you get into your 30s and 40s, people start thinking about a will and things like that. So maybe aligned to something like that, you know would get younger people.” (P13790F67) . “when people talk about the pension, you know so retirement, you know, to make people aware about this as well. You know, I would say that’s probably good ways to reach people” (P21263M54) .

Current legislation and promotion regarding organ donation were discussed as being successful in engaging the public and therefore implied that the government should take a more active role in the promotion of ACP.

“If you look at the way, sort of, government have been promoting like organ donation and that. You know that sort of thing. And then people have really bought into it and you know, and there’s a lot of positivity around it. So, I think that’s sort of similar approach would be good”. (P29453F40) “You know, people talking about pension, pension plans and so on and it’s part of the natural life circle, you know. So, if it’s in connection with this, you know, so think about your future, make your plans …so probably in connection with organ donation and so on, you know, so I think they could be trigger points, you know,, people talking about this.” (P21263M54) .

Theme 3: societal and cultural barriers to open discussion

In conversations surrounding why there was a lack of openness in discussion, participants postulated that a potential factor was the influence of cultural and societal norms. This was found to overlap in the SEM levels of intrapersonal, interpersonal and community.

Rural farming communities were highlighted as potentially being more isolated and traditional in their views around death and dying, whilst those with strong religious beliefs were seen to be less likely to engage in discussions.

“You know, and there is a, I’m out in the countryside. Well, it’s (place name). So it’s a relatively rural sort of conservative place and it and it’s that… you know you’re tied to the land, you’re tied to the farm. This is your home and sending you away from it early it is seen as a bit of a shameful thing. So yeah, to try and educate folk and to try and speak into that I think would be really helpful because again, I’ve seen situations where probably the individual’s life, the end of life, has been made tougher because the family have fought to keep that person at home.…. We’re going to try and do it ourselves” (P26495M43) . “I think it’s probably a lot to do with religion more than likely because people just like to hear, because we still are very much, you know, a lot is dictated by religion. So, people just want to leave stuff in God’s hands so they don’t want to have like…that would be the kind of where my mom and dad are coming from, you know, don’t interfere with it, blah blah blah. So, it’s a very gentle like kind of reminding them, you know, well, I think we do need to think about it. And I think people are afraid that they’ll make again that um it’s kind of euthanasia you know what I mean” (P37538F45) .

Cultural differences between countries were also considered with some seeing other global communities approaching death as a celebration rather than something to shy away from.

“I think in Northern Ireland we are, and the UK and probably the world in general. We are really poor at talking about, about death, and it has to be a positive thing to be able to talk more about death. You look at other cultures where you know. Death is treated differently, you know, even in Africa things where, you know, it’s a real celebration, whereas it’s seen so differently in in Northern Ireland” (P31154F35) .

Furthermore, regions and countries that have experienced war and conflict were perceived by one participant as a potential barrier to engaging in subject matters which involve death.

“I don’t I don’t know about other countries, but those that grew up maybe during the conflict here, maybe it’s just something that, you know it’s it’s a completely different mindset to them” (P23609M39) .

Whilst it is unclear from the data why the participant felt the conflict might inhibit engagement with the subject of dying, it could be interpreted that the participant was suggesting that death is too morbid to engage with following a conflict period which saw numerous deaths. An alternative interpretation could be that people are desensitised to death and do not see death as something that an individual has autonomy over.

It was noted that in many modern societies communities are changing. People no longer interact with their neighbours in the way they used to which results in a reduced sense of community responsibility.

“I don’t know that everyone is as neighbourly as they used to be. Northern Ireland I always perceived as being open door policy and everybody looked out for everyone else. But I think as a society has changed and it has changed in Northern Ireland, and it’s become becoming more closed (…) So I think we need to try and encourage that community um experience back again so that people are mindful of their neighbours and share that responsibility and making sure that everyone’s OK” (P25046F) .

This sense of social isolation was discussed by one participant who referred to homeless people. They reflected on how current social structures may not be providing the information and support to this minority group and therefore should be considered when developing public health approaches to engaging them in PC and ACP education.

“And if you look at homeless people on the streets, where do they get the information from? Where do they get the care, the information, the attention. I know there are Street workers that work, but I don’t know again to what extent in Northern Ireland compared with the likes of London” (P25046F) .

Mapping the findings to a social ecological model framework

Following thematic analysis each of the resulting themes were mapped to the five socioecological levels identified by McLeroy et al. (1988) for health promotion programmes. Construct mapping can be found in Fig.  1 below.

Thematic interaction within the Social Ecological Model levels

The findings from this study highlight the complexity of current public perceptions of palliative care and their views on effective engagement with PC and ACP. Within medicalised western culture there is a tendency to focus on the preservation of life, with conversations about death avoided. This has resulted in death becoming a taboo, raising fear and stigma where death is equated with failure. These social taboos that exist around death, dying and bereavement are posited to stem from the lack of awareness and understanding of PC and ACP and the resulting stigma of approaching these discussions. There was evidence of influencing factors on all SEM levels, which demonstrates the need for a multifaceted public health approach that uses not only behaviour change communication but also social change communication, social mobilisation and advocacy. It can be argued this reflects the key aspects outlined in Lancet Commission report on ‘Valuing Death’, which advocated for a ‘systems approach’ [ 10 ]. This systems approach is aligned to differing levels within the SEM and the different approaches the public have identified when seeking to build public engagement and access to palliative care. Three key aspects were noted: visibility, embedding opportunities for engagement in everyday life and societal and cultural influences.

It was clear from the analysis that a major factor associated with poor public engagement was the lack of visibility within the public domain, which was hindering both the normalisation of death and understanding that PC was more than just end of life care. The findings demonstrated different ways to address the lack of visibility, such as the use of targeted social media and wider publicity campaigns. Research to date has demonstrated that palliative care education is a useful tool in improving knowledge of, confidence in and attitudes towards palliative care amongst healthcare professionals and carers [ 14 ]. Similar results have been noted for the public when exploring the potential to promote palliative care through various media challenges such as YouTube and social media [ 28 ]. This does, however, raise questions around the quality and accuracy of information offered via the media, taking cognisance of whether some of the messaging may inadvertently be adding to misunderstanding, and thus a lack of public engagement.

Secondly, the findings indicated that experience at the individual level within a social context was noted as an important element when seeking ways to increase public engagement with PC and ACP. The experience of illness, dying and loss is often overlooked, therefore, this points to the potential value of community-based education approaches, with peers enabling experience-based exchange. Such interventions have been noted in the literature on the role of volunteers and education [ 29 ]. This reflects the need for an overall public health palliative care approach that seeks to empower individuals, families and communities to draw on their own resources and community supports to adapt and cope with death and dying [ 6 , 30 ].

Thirdly, the findings from this study indicated the need for enhancing opportunities for engagement in PC and ACP within everyday life. Research indicates there is an appetite for people to talk about death, for example, in the UK, a recent YouGov ‘daily question’ survey reported 67% of adults who responded think the subject of death and dying should be talked about in schools [ 31 ]. This speaks to the need to consider schools, workplaces and key trigger points in life as times to consider engagement with PC and ACP. This reflects the overall need for death literacy in society to improve experiences at the end of life [ 10 ].

Finally, the importance of socio-cultural aspects for the public cannot be underestimated. Therefore, effective communication strategies need to be tailored to individuals, and communities and be culturally appropriate. This has been noted as an important aspect for specific communities, such as the Chinese diaspora, for example, but nuances around this for specific ethnic, political, religious, and geographical aspects need further consideration [ 32 ]. Cultural competence, defined as an understanding of how culture affects an individual’s beliefs, values and behaviour, is an important consideration [ 33 ]. A meta-analysis of 19 review articles, concluded that interventions to increase cultural competence in healthcare were effective in enhancing the knowledge, skills and attitudes of healthcare providers, leading to clinical benefits for patients/clients through improved access and utilization of healthcare [ 34 ]. The translation of such reviews for public engagement in PC and ACP warrants further exploration. It has been advocated that elements of cultural systems should be analysed with a socio-ecological framework [ 35 ]. Such consideration and integration of salient contextual cultural factors could assist public messaging and cultural communication, which would enhance more effective and sustainable public engagement in PC and ACP.

Limitations

When considering potential limitations, it is pertinent to note that due to the sensitive nature of the topic the exclusion criteria restricted the sample to those who had not experienced a recent bereavement. This may have limited the ability to gain a wider perspective, as the views of the recently bereaved may have provided further nuanced insights into how best to engage the public. Furthermore, the participant sample was limited to those involved in a larger mixed-methods study. This may have introduced bias in relation to true knowledge and attitudes due to the participants having completed the survey questionnaire prior to the interviews.

In conclusion, this qualitative study has provided insights into how the public would like to be engaged in PC and ACP. The findings highlighted that to build public engagement and access to palliative care and advance care planning a multifaceted public health approach is required. Discussions of death and dying remain difficult for many members of society, therefore, an increase in death literacy across all systems to reduce misperceptions surrounding PC and APC is needed, by increasing visibility and providing opportunities for the public to engage with PC and ACP within everyday life. Finally, socio-cultural aspects need consideration when developing strategies to ensure effective communication and engagement with all members of the community.

Data availability

The datasets analysed are not publicly available but are available from the corresponding author upon reasonable request.

Abbreviations

Advance care plan

• Palliative care

Palliative care education

Social ecological model

World Health Organization. Newsroom Palliative Care Fact Sheet. 2020. https://www.who.int/news-room/fact-sheets/detail/palliative-care .

World Health Assembly 67. Strengthening of palliative care as a component of comprehensive care throughout the life course [Internet]. 2014. https://apps.who.int/iris/handle/10665/162863 .

McIlfatrick S, Slater P, Beck E, Bamidele O, McCloskey S, Carr K, et al. Examining public knowledge, attitudes and perceptions towards palliative care: a mixed method sequential study. BMC Palliat Care. 2021;20(1):44.

Article   PubMed   PubMed Central   Google Scholar  

Brinkman-Stoppelenburg A, Rietjens JA, Van Der Heide A. The effects of advance care planning on end-of-life care: a systematic review. Palliat Med. 2014;28(8):1000–25.

Article   PubMed   Google Scholar  

Grant MS, Back AL, Dettmar NS. Public perceptions of Advance Care Planning, Palliative Care, and Hospice: a scoping review. J Palliat Med. 2021;24(1):46–52.

McIlfatrick S, Slater P, Bamidele O, Muldrew D, Beck E, Hasson F. ‘ It’s almost superstition: If I don’t think about it, it won’t happen ’. Public knowledge and attitudes towards advance care planning: A sequential mixed methods study. Palliat Med. 2021;35(7):1356–65.

Canny A, Mason B, Boyd K. Public perceptions of advance care planning (ACP) from an international perspective: a scoping review. BMC Palliat Care. 2023;22(1):107.

Mills J, Abel J, Kellehear A, Patel M. Access to palliative care: the primacy of public health partnerships and community participation. Lancet Public Health. 2021;6(11):e791–2.

Tobin J, Rogers A, Winterburn I, Tullie S, Kalyanasundaram A, Kuhn I, et al. Hospice care access inequalities: a systematic review and narrative synthesis. BMJ Support Palliat Care. 2022;12(2):142–51.

Sallnow L, Smith R, Ahmedzai SH, Bhadelia A, Chamberlain C, Cong Y, et al. Report of the Lancet Commission on the value of death: bringing death back into life. Lancet. 2022;399(10327):837–84.

Kellehear A. Compassionate communities: end-of-life care as everyone’s responsibility. QJM. 2013;106(12):1071–5.

Last Aid. Last Aid [Internet]. 2023. https://www.letztehilfe.info/project/ .

Sawyer JM, Higgs P, Porter JDH, Sampson EL. New public health approaches to palliative care, a brave new horizon or an impractical ideal? An integrative literature review with thematic synthesis. Palliat Care Soc Pract. 2021;15:263235242110329.

Article   Google Scholar  

Li WW, Chhabra J, Singh S. Palliative care education and its effectiveness: a systematic review. Public Health. 2021;194:96–108.

Article   CAS   PubMed   Google Scholar  

Seymour J. The Impact of Public Health Awareness Campaigns on the awareness and quality of Palliative Care. J Palliat Med. 2018;21(S1):S–30.

Seymour JE. Revisiting medicalisation and ‘natural’ death. Soc Sci Med. 1999;49(5):691–704.

McIlfatrick S, Slater PF, Beck E, Hasson F, Black R. NI Life and Times Survey: palliative care and Advance Care Planning. Ulster University; 2023.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Braun V, Clarke V. Thematic analysis: a practical guide. 1st ed. SAGE; 2022. p. 338.

McLeroy KR, Bibeau D, Steckler A, Glanz K. An Ecological Perspective on Health Promotion Programs. Health Educ Q. 1988;15(4):351–77.

Bronfenbrenner U. Toward an experimental ecology of human development. Am Psychol. 1977;32(7):513–31.

Sutton S. Determinants of Health-Related Behaviours: Theoretical and Methodological Issues. In: The SAGE Handbook of Health Psychology [Internet]. 1 Oliver’s Yard, 55 City Road, London EC1Y 1SP United Kingdom: SAGE Publications Ltd; 2008 [cited 2023 Jun 21]. pp. 94–126. https://sk.sagepub.com/reference/hdbk_sagehealthpsych/n4.xml .

Tripken JL, Elrod C, Bills S. Factors influencing Advance Care Planning among older adults in two socioeconomically diverse Living communities. Am J Hosp Palliat Med. 2018;35(1):69–74.

Chen Y, Zhang R, Lou Y, Li W, Yang H. Facilitators and barriers to the delivery of palliative care to patients with Parkinson’s disease: a qualitative study of the perceptions and experiences of stakeholders using the socio-ecological model. BMC Health Serv Res. 2023;23(1):215.

Huang Y, Yates P, Thorberg FA, Wu C (Jo), editors. Influence of social interactions, professional supports and fear of death on adults’ preferences for life-sustaining treatments and palliative care. Int J Nurs Pract [Internet]. 2022 Aug [cited 2023 Jun 21];28(4). https://onlinelibrary.wiley.com/doi/ https://doi.org/10.1111/ijn.12940 .

Korstjens I, Moser A, Series. Practical guidance to qualitative research. Part 4: trustworthiness and publishing. Eur J Gen Pract. 2018;24(1):120–4.

Data Protection Act. (2018) [Internet]. https://www.legislation.gov.uk/ukpga/2018/12/contents/enacted .

Liu M, Cardenas V, Zhu Y, Enguidanos S. YouTube Videos as a Source of Palliative Care Education: a review. J Palliat Med. 2019;22(12):1568–73.

Buonocore S, Sussman-Skalka C, Project, insights: An evaluation of a community vision education project for older adults. Educ Gerontol. 2002;28(4):289–99.

McIlfatrick S, Hasson F, Slater PF, Beck E, Hanna-Trainor L, Muldrew D. Where are we now? - Examining public knowedge and attitudes towards palliative care and advance care planning in Northern Ireland [Internet]. 2021. https://www.ulster.ac.uk/__data/assets/pdf_file/0012/819678/Where-are-we-now-Examining-public-knowledge-and-attitudes-towards-palliative-care-and-advance-care-planning-in-NI.pdf .

YouGov. Daily question. Do you think the subject of death, and issues around dying, should or should not be taught in schools? https://yougov.co.uk/topics/philosophy/survey-results/daily/2021/03/16/d5536/2 .

Li Z, Beck ER, McIlfatrick S, Hasson F. Chinese Diaspora communities’ knowledge of and Engagement with Advance Care Planning: a systematic integrative review. J Palliat Care. 2023;38(3):381–97.

Balcazar FE, Suarez-Balcazar Y, Taylor-Ritzler T. Cultural competence: development of a conceptual framework. Disabil Rehabil. 2009;31(14):1153–60.

Truong M, Paradies Y, Priest N. Interventions to improve cultural competency in healthcare: a systematic review of reviews. BMC Health Serv Res. 2014;14(1):99.

Cain CL, Surbone A, Elk R, Kagawa-Singer M. Culture and Palliative Care: preferences, communication, meaning, and mutual decision making. J Pain Symptom Manage. 2018;55(5):1408–19.

Download references

Acknowledgements

The authors would like to thank all interviewees for their participation in the research.

This study was funded by HSC R&D Division of Public Health Agency in Northern Ireland.

Author information

Authors and affiliations.

Institute of Nursing and Health Research, Ulster University, Belfast, BT15 1AD, Northern Ireland

Rachel Black

Felicity Hasson

Institute of Nursing and Health Research, Ulster University, Belfast, BT15 1ED, Northern Ireland

Paul Slater

Esther Beck

Sonja McIlfatrick

You can also search for this author in PubMed   Google Scholar

Contributions

SMcI is principal investigator responsible for study design, project management, analysis and manuscript review and dissemination. FH, PS, EB were responsible for study design, analysis, manuscript review and dissemination. RB was responsible for qualitative data collection, analysis, manuscript drafting and dissemination. All authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Sonja McIlfatrick .

Ethics declarations

Conflicts of interest.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics approval and consent to participate

Ethical approval was granted from the Ulster University Nursing and Health Sciences Ethics Filter Committee (FCNUR-22-063 Sept 2022). Participants provided written informed consent prior to the commencement of the interviews. They were advised of their right to withdraw, and the confidentiality and anonymity of all data was confirmed. All data was kept in accordance with the Data protection act (2018).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Black, R., Hasson, F., Slater, P. et al. Building public engagement and access to palliative care and advance care planning: a qualitative study. BMC Palliat Care 23 , 98 (2024). https://doi.org/10.1186/s12904-024-01420-8

Download citation

Received : 08 September 2023

Accepted : 25 March 2024

Published : 12 April 2024

DOI : https://doi.org/10.1186/s12904-024-01420-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Advance care planning
• Social Ecological Model
• Death literacy
• Public engagement
• Public health
• Death and dying.

BMC Palliative Care

ISSN: 1472-684X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 06 April 2024

Delayed discharge in inpatient psychiatric care: a systematic review

• Ashley-Louise Teale   ORCID: orcid.org/0000-0002-1756-7711 1 ,
• Ceri Morgan   ORCID: orcid.org/0000-0002-2417-8677 1 ,
• Tom A. Jenkins   ORCID: orcid.org/0000-0001-7875-4417 1 &
• Pamela Jacobsen   ORCID: orcid.org/0000-0001-8847-7775 1  

International Journal of Mental Health Systems volume  18 , Article number:  14 ( 2024 ) Cite this article

291 Accesses

8 Altmetric

Metrics details

Delayed discharge is problematic. It is financially costly and can create barriers to delivering best patient care, by preventing return to usual functioning and delaying admissions of others in need. This systematic review aimed to collate existing evidence on delayed discharge in psychiatric inpatient settings and to develop understanding of factors and outcomes of delays in these services.

A search of relevant literature published between 2002 and 2022 was conducted on Pubmed, PsycInfo and Embase. Studies of any design, which published data on delayed discharge from psychiatric inpatient care in high income countries were included. Studies examining child and adolescent, general medical or forensic settings were excluded. A narrative synthesis method was utilised. Quality of research was appraised using the Mixed Methods Appraisal Tool (MMAT).

Eighteen studies from England, Canada, Australia, Ireland, and Norway met the inclusion criteria. Six main reasons for delayed discharge were identified: (1) accommodation needs, (2) challenges securing community or rehabilitation support, (3) funding difficulties, (4) family/carer factors, (5) forensic considerations and (6) person being out of area. Some demographic and clinical factors were also found to relate to delays, such as having a diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, and increased service input prior to admission. Being unemployed and socially isolated were also linked to delays. Only one study commented on consequences of delays for patients, finding they experienced feelings of lack of choice and control. Four studies examined consequences on services, identifying high financial costs.

Overall, the findings suggest there are multiple interlinked factors relevant in delayed discharge that should be considered in practice and policy. Suggestions for future research are discussed, including investigating delayed discharge in other high-income countries, examining delayed discharge from child and forensic psychiatric settings, and exploring consequences of delays on patients and staff. We suggest that future research be consistent in terms used to define delayed discharge, to enhance the clarity of the evidence base.

Review registration number on PROSPERO

Date of registration.

9th December 2021.

Delayed discharge, also termed ‘bed blocking’ and ‘delayed transfer of care,’ refers to when patients remain in hospital beyond the time they are determined to be clinically fit to leave [ 1 , 2 ]. It is an international challenge, costly to individuals, health services and governments [ 3 , 4 ], impacting physical health settings, and also psychiatric inpatient services [ 5 ].

Psychiatric inpatient stays are one of the most expensive forms of treatment for mental health conditions, particularly when compared to care delivered in community settings [ 6 ]. Prolonged stays in mental health hospitals likely increase resource use and as such financial expenditure. This is particularly concerning in instances of delayed discharge when stays are determined to not be of clinical benefit. Delayed discharge also could prevent admission of new patients, contributing to bed crises, where there are not enough beds for all who require admission [ 7 ]. This can have consequences on the course of recovery for newly referred patients, either delaying admission, contributing to inappropriate placements, or leading to individuals being placed out of area [ 7 , 8 ]. Extended hospital stay could also detrimentally impact the delayed patient themselves, preventing their return to usual day-to-day functioning and make returning to the community increasingly difficult [ 9 , 10 ].

Existing reviews have examined predictors of longer stays in psychiatric inpatient settings, finding substance use and being employed are associated with shorter length of stay; while being female, having a diagnoses of mood or psychotic disorders and use of Electroconvulsive Therapy are associated with longer stay [ 11 ]. However, there is not to our knowledge a systematic review collating evidence examining delayed discharge in psychiatric settings. As delayed discharge is a unique experience, distinct from long stay driven by clinical need, it requires separate focus to further understand this specific experience.

Furthermore, a large body of evidence has examined delayed discharge in physical health settings with several systematic reviews, examining causes and outcomes. Such reviews have found that delayed discharges were linked to problems in discharge planning, transfer of care difficulties and patient age [ 12 , 13 ]. Outcomes for services included overcrowding and financial costs, whereas outcomes for patients included infections, depression, reduction in activities and mortality. There may be both overlapping and non-overlapping factors associated with delayed discharge between physical and psychiatric inpatient settings. For example, inpatient psychiatric services may differ in organisational structure, daily workings, and treatment focus from general medical services. The clinical population might also differ in psychiatric and physical health settings, for example in age, socio-economic status, and other demographic, plus clinical factors. As such, it is vital that separate attention be given to the area of psychiatric care.

This systematic review aims to fill the current research gap and synthesise existing literature on psychiatric delayed discharges. We aimed to synthesise the available international data from high-income countries, as the prevalence and underlying reasons for delayed discharge are likely to be highly sensitive to context and heterogeneous across countries. This is due to factors such as different models of healthcare funding, and the varying social role of the family in providing care, for example. Developing in-depth understanding of the causes and consequences of delays in a psychiatric inpatient context is important in informing practice and policies at a service, organisational, societal, and government level. This could help develop ways to reduce occurrence of delays and mitigate any negative impacts.

The aim of this review was to increase understanding of what is known about factors influencing delayed discharge in adult psychiatric inpatient settings. Secondary aims were to examine outcomes of delayed discharge for patients and compare findings across different psychiatric settings and age groups.

The systematic review protocol was pre-registered on PROSPERO before the review was started and the searches were run (PROSPERO: 292515). The review is reported in line with PRISMA guidelines [ 14 , 15 ]. The primary research question of this review is: What is known about factors associated with delayed discharge from inpatient psychiatric care settings?

Secondary research questions were:

What are the outcomes for those who have experienced delayed discharge from inpatient psychiatric settings, for example, in mental health outcomes, health outcomes, readmissions and quality of life?

What are the outcomes on services in terms of resources and costs from delayed psychiatric inpatient discharge?

What are the experiences of staff and patients of delayed discharge from inpatient mental health wards?

Are there differences between types of inpatient services, including acute, rehabilitation or specialist inpatient wards, in factors and costs, are there differences between working age adults and older adults, in experiences of delayed discharge, search strategy.

Initial searches were conducted on the 15th of January 2022, and updated on the 5th of August 2022. Pubmed, PsycInfo and Embase were searched.

Search terms (Appendix B in supplementary materials) were developed through examining key words of published studies on the topic, reviewing the terms used in comparative reviews based in physical health settings and thesaurus mapping. Terms included: “delayed discharge,” “bed blocking” and “long stays.” Search terms were piloted on each database prior to running the final search.

The search included studies published from 2002. A 20-year search timeframe was selected, as psychiatric inpatient care has adapted in response to changing need and updated knowledge over time. As such, studies published before 2002 are likely to be less relevant to current practice.

Following database searches, reference lists of included papers were examined, to identify any relevant studies missed in the search. A forward citation search was also conducted, to identify any relevant studies that were cited in the included papers.

Inclusion and exclusion criteria

Studies were included if they reported data related to delayed discharge or associated outcomes, in adult psychiatric inpatient wards. Specialist and rehabilitation psychiatric inpatient settings were included. Studies of any design were included, providing they were published in a peer-reviewed journal. Both quantitative and qualitative studies were included.

Studies exploring delayed discharge in child or adolescent units and/or forensic units were excluded. This was because the causes and outcomes of delays in such settings are likely unique, given the specialist context. For example, there is likely different systemic involvement from families and different governing legislation in these contexts. As such, it was determined that such settings were too disparate, and synthesising studies from these settings together with adult psychiatric settings could lead to inaccurate conclusions. Physical health settings were also excluded, given the different processes, procedures and treatment focus involved in such settings. In addition, reviews have already been conducted examining delayed discharge from such settings. Studies not conducted in high-income countries were also excluded. In this review, we included high-income countries as defined by World Bank criteria, accessed in January 2022 [ 16 ] (see Appendix C in supplementary material for the list of included countries). Globally, countries differ in the conceptualisation of mental health and provisions offered, therefore, limiting this review to only high-income countries would enable comparisons to be made.

Study selection and data extraction

Screening was conducted using Covidence Systematic Review Software [ 17 ]. All records were independently double-screened by two reviewers at both title/abstract and full-text stage. Conflicts were resolved by discussion to reach consensus, with referral to the senior author (PJ) when needed.

A standardised template was used for data extraction, with all included studies being independently double extracted by two reviewers, with consensus achieved by discussion where needed.

A narrative synthesis method was used. For data examining reasons for delayed discharge, a deductive approach was taken initially. Authors identified possible reasons for delays based on existing literature and organised data under these categories/themes. Any data that did not fit into the pre-defined categories was pooled as ‘other’. All categories were then reviewed, with particular attention placed on the ‘other’ categories, to determine if additional categories need to be added or existing categories adapted. Sub-categories were identified when appropriate through coding. Once categories were established, the number of papers which reported each reason/factor were tabulated and data was reviewed to examine relationships, exploring both links and disparities within and between studies. The final synthesis was checked by three authors (AT, TJ, and CM), to achieve final agreement.

Data relating to outcomes/consequences of delayed discharge was synthesised in a similar way, with data initially organised into three categories: (1) consequences for patients, (2) consequences for service, (3) consequences for staff. Categories were reviewed by the authors following synthesis. Financial costs were converted to US dollars by the authors to support comparison.

Quality assessment of the included studies formed part of the synthesis with the appraisal of quality considered in the interpretation of results.

Quality Assessment

Quality assessment of studies was completed during the synthesis stage. In the protocol, we initially outlined that the Quality Assessment Tool for Studies of Diverse Designs (QATSDD) would be utilised [ 18 ]. However, following a trial of this tool with the included papers, we noted disparities in interpretations between authors. Therefore, the Mixed Methods Appraisal Tool (MMAT) was established to be a more suitable appraisal of quality for the included studies. The MMAT was developed for assessing and comparing the quality of studies using quantitative, qualitative and mixed-methods design, in one tool [ 19 ]. This tool was selected as studies of different designs were included in the review and this tool allows for quality appraisal across five different study types, distinguishing between methodology.

Two initial screening questions were answered to determine appropriateness of using the MMAT to assess quality of the study (are there clear research questions and do the collected data address the research questions). If screening questions are not passed, this tool is deemed inappropriate. Providing the screening questions were passed, quality was assessed on five questions within one of five categories. The category in which questions were answered was determined by study design. The MMAT discourages from scoring and assigning qualitative labels to describe quality, instead advises a more detailed evaluation of quality [ 19 ]. This approach has therefore been taken in this paper.

To achieve reliable and accurate quality ratings, every study was quality rated by two members of the research team and conflicts were discussed to reach consensus.

Identification of studies

Figure  1 (PRISMA flowchart) shows the study selection process. After removing duplicates, a total of 4891 papers were identified for screening. 4397 papers were excluded at title and abstract stage. Full texts were then obtained for 492 papers. Two full texts could not be obtained via the library service and the authors did not respond to a request for the paper. There were four papers obtained that were erratum’s, all of which related to excluded studies that were not examining delayed discharge and as such, were not linked to the included studies. Following full text screening 18 papers were eligible for inclusion. Each paper represented a different study.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flowchart

Study characteristics

Table  1 shows the characteristics of the 18 included studies. Twelve of these studies examined delayed discharge as a primary outcome, with three of these studies specifically examining Housing Related Delayed Discharge (HRDD). HRDD is defined as instances where delayed discharge is attributed to housing issues. The remaining studies ( n  = 6) reported delayed discharge as secondary outcomes. Fifteen studies were of quantitative observational design, two studies used mixed methodologies and one was qualitative.

In the included studies, there was a range of psychiatric inpatient settings: psychiatric/general mental health units ( n  = 11), Psychiatric Intensive Care Units (PICUs) ( n  = 2), older adult psychiatric units ( n  = 3) and Mental Health Trusts ( n  = 1). One study looked across three inpatient settings: acute psychiatric, PICU and older adult. Studies were conducted in five high income countries (England = 10, Ireland = 1, Australia = 3, Canada = 3, and Norway = 1). There were no studies from any other high-income countries, identified in the search.

The MMAT quality scores are shown (Table  2 ). One included study [ 20 ] did not meet initial criteria to be assessed using this tool, as the research questions were unclear.

All studies were of fairly good quality, with all studies meeting at least three out of five of the quality assessment criteria. Quality was highest in Australian and Canadian studies, with included papers in these countries meeting all five quality assessment criteria [ 21 , 22 , 23 , 24 , 25 , 26 ]. Quality assessment ratings indicate that three quantitative descriptive studies included, did not clearly report use of a representative sample or appropriate measures. Ratings per question are shown in Table two.

Research Q1

What is known about factors associated with delayed discharge.

Thirteen studies identified reasons for delayed discharge (Table  1 ). The results showed that there are many complex reasons for delays with often overlapping contributing factors. We categorised reasons for delay into six categories: (1) accommodation needs, (2) difficulty securing rehabilitation or community support, (3) finance/funding challenges, (4) family/carer factors, (5) forensic factors, (6) patient being out of area.

The most common reason for delays was due to accommodation and placement factors. This was identified as a contributing reason for delay in twelve studies and a further two studies assessed Housing-Related Delayed Discharge (HRDD), suggesting accommodation factors contributing to delay in these cases. Accommodation/placement factors included limited availability of placements ( n  = 7), difficulty finding appropriate placements ( n  = 5), awaiting or undergoing placement assessment ( n  = 3), challenges in person returning to accommodation ( n  = 3), e.g., awaiting repairs or adaptations to their home, individuals being rejected from placement ( n  = 2), patients/family rejecting placement ( n  = 2) and awaiting transfer ( n  = 1). It should be noted that one of the studies which examined specific accommodation factors was unable to be quality assessed due to not having clear research questions and therefore did not meet the screening criteria for assessment with the MMAT [ 20 ], and two studies only met three of the five quality assessment criteria, with queries regarding the quality of measures used and analysis technique for one study [ 27 ], and some difficulties integrating and meeting the full quality criteria for the mixed methods approaches used in the second [ 28 ]. The second reason identified for delays was difficulty sourcing support for the person to enable discharge, such as community, rehabilitation, and homecare support. This contributed to delays in twelve studies. Eight of these studies met four to five of the quality assessment criteria, one was not able to be assessed [ 20 ], and three only met three of the five quality assessment criteria [ 27 , 28 , 29 ]. A third reason for delay was finance/funding challenges identified in nine studies. These included challenges obtaining funding, patients/families’ refusal to pay for placements and funding applications being rejected. Six studies identified family/carers factors in creating delays, such as family conflict, family not wanting the person to live with them and ongoing family discussion. The quality of two of the studies identifying family and finance factors should be considered, as one of these studies was unable to be quality assessed due to a lack of clear research questions [ 20 ] and a second met only three of the five quality assessment criteria [ 28 ]. The fifth reason identified in this review as contributing to delay was forensic factors, which accounted for delays in three studies, all of good methodological quality. Forensic delays incorporated delay by Ministry of Justice and awaiting forensic assessment. Person being out of area was highlighted as a reason for delay in only one study and it was not possible to quality assess this study due to no specific research questions identified [ 20 ], suggesting limited exploration or evidence for out of areas contributing to delays.

Fourteen studies included in this review examined the demographic and clinical factors relevant in delays, with eight conducting significance testing to establish associations. Significant associations with delay were having a diagnosis of schizophrenia or other psychotic disorder ( n  = 4), cognitive impairment ( n  = 3) and type/amount of service input prior to admission ( n  = 3). All studies reporting these significant results were of a good methodological quality, achieving at least four of the five MMAT quality criteria. Results were mainly consistent across those studies which examined significance, however, there was one study of good quality that did not find significant association with schizophrenia diagnosis [ 22 ]. The impact of physical health differed between Australia and England, where in one English study having fair-excellent health was more associated with delays [ 30 ], though two Australian studies found poorer physical health linked to delays [ 24 , 25 ]. Findings related to demographic characteristics, including gender, age, ethnicity, socio-economic status, were inconsistent across studies. The only consistent finding was that a smaller proportion of the delayed group were employed ( n  = 3). One of these studies found significant association between being unemployed and delayed discharge. The two other studies found only one member of the delayed group was employed, less than non-delayed groups, though this was not significance tested. There was some indication that being not being married and lacking a support network, was higher in delayed groups. One study found significant relationships to being unmarried and another finding that the delayed group were visited significantly less often by relatives. The other studies did not conduct significance testing. However, there was no significant relationship related to marriage between delayed and non-delayed groups in two studies [ 22 , 31 ]. One of these studies only clearly met three of the quality assessment criteria [ 31 ], though the other met all five quality assessment criteria. Being male was significantly associated with delays in two Canadian studies [ 21 , 22 ]. No significant association with gender was found in other studies.

The supplementary materials provide additional analysis of results for research question one, further describing each study’s findings. Additional materials also include tables showing tabulation of which study examined each variable.

Research Q2

What are the outcomes for those who have experienced delayed discharge from inpatient psychiatric settings for example, in mental health outcomes, health outcomes, readmissions and quality of life.

Only one study examined individual outcomes of delayed discharge for patients [ 26 ]. As such, there is limited data to draw conclusions to answer this research question. The study that evaluated patient outcomes was of qualitative design and good quality. The study explored Housing-Related Delayed Discharge (HRDD) in Australia for 10 patients using semi-structured interviews. They found consequences of lack of choice and control for patients, which impacted mental wellbeing, physical health and created a sense of anticipation for transition to community. Some participants highlighted a positive outcome of delayed discharge in preventing homelessness.

Research Q3

What is the outcome on services in terms of resources and costs from delayed psychiatric inpatient discharge.

Four studies assessed financial costs of delayed discharge for services, providing limited evidence in terms of financial outcomes. Each study focused on a different country. At an old age psychiatry unit in England, delayed discharges were estimated to cost over $855,820 for the year [ 20 ]. Notably, this study was not quality assessed due to the omission of research questions. In a high-quality paper from Australia, HRDD cost the health district $2,828,174 over one year [ 25 ]. While both papers present yearly costs, there is disparity in area covered, contributing to difficulty making comparisons regarding financial expenditure. Two studies calculated financial expenditure and did not present the cost per year. In a Canadian study, using the median number of delayed days ( M  = 17), it was calculated that the average cost incurred by one episode of delayed days was approximately $5,746 [ 21 ]. Furthermore, in Norway, $491,406 was allocated to delays on the acute ward included in the study, though methodological quality might be queried, due to lack of clarity on whether the sample was representative and the appropriateness of measures utilised [ 29 ]. The information necessary to calculate costs per year or costs per delayed day, to enable comparisons to be made across studies, has not included in the studies.

Aside from financial costs, no other type of outcome for services were assessed.

Research Q4

None of the included studies explored specific experiences of delayed discharge for staff. Some information on experiences for patients is detailed in question two.

Research Q5

This systematic review identified studies in acute psychiatric, older adult and Psychiatric Intensive Care Unit (PICU) settings. Only one study included Learning Disability inpatient care settings [ 28 ]. This study was of mixed-method design and met three quality assessment criteria. No studies reported data from rehabilitation units. There were few differences identified between types of setting. Prevalence of delayed discharge was highest in older adult settings (56.9%) [ 30 ] and PICU settings (51.1%) [ 32 ], compared to working age adult settings (18–32%) [ 31 , 33 ]. However, the highest proportion of delayed days was found in acute psychiatric settings in Norway acute psychiatric units (54.8%) [ 29 ]. More information on prevalence is provided in supplementary materials.

Reasons for delay did not vary much across type of setting. There is a potential service difference in the impact of physical health in delays, as having fair-excellent health was more associated with delays in an English older adult study [ 30 ], while in a working age adult sample in Australian studies [ 24 , 25 ], having poor health was more associated with delays. However, this could represent a disparity in country. There were some other differences across countries found. Forensic reasons for delay were only found in the UK ( n  = 2), as was due to patient being out of area ( n  = 1). In UK settings, there was no significant difference found in gender between those delayed and those not [ 30 , 34 ], though there was in Canada [ 21 ]. England and Australia were the only countries identifying funding issues as contributing to delay. Each country will have its own respective funding system, which could impact delays. For example, two Australian studies identified difficulties with their own National Disability Insurance Scheme [ 24 , 25 ].

Research Q6

Only five of the included studies looked specifically at older adult settings, all of which were in the UK. A further five studies, from the UK and Canada, included older adults within their sample, despite not examining a specific older adult setting.

The highest proportion of inpatients experiencing delayed discharge were from older adult settings, with one study identifying 56.9% [ 30 ] of inpatients experiencing delays. There were lower rates of delayed patients in working age adult psychiatric inpatient settings in comparison, with 3.5% [ 21 , 25 ] to 39.1% [ 29 ] of patients experiencing delay. Similarly, two studies in Canada identified that a higher proportion of older adults made up the delayed group compared to the non-delayed group, suggesting that older adult inpatients are more likely to experience delay [ 21 , 22 ]. However, two English studies found delayed discharge was not associated with age [ 31 , 35 ]. One of these studies met only three quality assessment criteria, with lack of clarity regarding the quality of sampling and representativeness of the sample [ 31 ].

In terms of reasons for delay, no clear differences were found across age groups. Although when limiting comparisons to studies conducted in the UK, family/carer factors was identified as a reason for delay more frequently in older adult samples ( n  = 3) compared to studies looking at working age adults ( n  = 1). To support this finding, one study in England found that eight older adult trusts identified patient/carer exercising choice as a reason for delay, whereas the same was true for only four working age adult trusts [ 28 ]. However, this finding cannot be generalised across all countries. There is also some indication that cognitive impairment/dementia might increase likelihood of delay in older adult samples, as two studies identified the role of dementia and greater cognitive impairment in the delayed older adult groups [ 20 , 30 ]. A further two studies examined the impact of cognitive impairment, finding association with delay [ 21 , 22 ]. However, these studies included working age samples, so it is unclear who in the sample this impacted. In addition, physical health status could cause delays differently in older adult populations. In an older adult UK sample having fair-excellent health was more associated with delays [ 30 ], whereas two Australian studies in working age adult inpatient settings found poorer physical health increased delays [ 24 , 25 ]. This difference could however be attributed to country or setting. Funding was identified as a reason for delay in all studies in older adult settings ( n  = 5), but the same was not true for the other setting types. Forensic factors were not found to be a reason for delay in any of the studies with older adult inpatients, conversely patient being out of area was only identified as a reason for delay in an older adult sample [ 20 ].

This systematic review aimed to fill a research gap and examine factors contributing to delayed discharge in adult psychiatric inpatient settings and explore associated consequences. This adds a unique contribution to the evidence base, which predominantly has focused on delayed discharge from physical health settings. Eighteen studies were included for synthesis.

The findings suggest that there are varying inter-related reasons for delay, including accommodation or placement needs, difficulties securing the required support services, funding and finance challenges, family/carer factors, forensic factors and the person being out of area. There were mixed findings regarding demographic and clinical characteristics associated with delays. However, this review showed that delays could be associated with the person having diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, being unemployed and receiving increased service input prior to admission.

There were only a few studies that commented on outcomes of delays. Only one study examined outcomes for patients, identifying feelings of lack of choice and control, while four studies looked at financial outcomes for services, finding large costs associated with delays. This points to a lack of evidence examining the outcomes and experiences of psychiatric delayed discharge, and therefore requires further attention in research.

This review adds to and expands on existing findings, identifying similarities and differences between longer stay generally. For example, one review [ 11 ] found that long stay was associated with mood and psychotic disorders, use of Electroconvulsive Therapy, and being female. Being married, employed, and using substances were associated with a shorter stay [ 11 ]. Our review found that psychiatric delayed discharge was also associated with diagnosis of schizophrenia or other psychotic disorder and being unemployed. However, we found delayed discharge to be associated with cognitive impairment and increased service input prior to admission, but not gender or treatment. This could suggest some important differences in those at risk of delays or those requiring longer inpatient treatment. It is important to note however, the review by Gopalakrishina and colleagues did not distinguish between those patients with long stay clinically warranted and delayed discharge patients [ 11 ]. It would be of benefit for future research on long stay patients to better define their sample based on those who clinically needed treatment or longer stay patients in the context of delayed discharge, allowing similarities and differences to be better explored. This will support policy makers and service managers to better identify those at risk of delays that are not clinically necessary, and those who might need additional clinical input. The findings in this review provide some suggestion that there could be benefit in considering a person’s social context when they are admitted to psychiatric inpatient care, including their living situation at admission, employment status and cognitive functioning. Identifying patients at higher risk of delays earlier in admission might be useful, to ensure more time be given to organise and find appropriate accommodations, placements and service support and facilitate discharge. Wider policy and structural changes are needed, such as improving the availability of appropriate accommodation placements.

It is important to highlight that there were discrepancies across studies in language used to term delayed discharge, e.g., ‘alternate level of care,’ ‘waiting days’ and ‘prolonged stay.’ Due to such discrepancies in definitions and terminology, during the screening process it was at times difficult to determine if studies were focused on delayed discharge or longer lengths of stay clinically required. In this review, studies were excluded if the focus was unclear to prevent incorrect conclusions being drawn related to the unique experience of delayed discharge. However, this means other relevant findings might have been missed. It would therefore be useful for future research on psychiatric inpatient care to ensure clarity in the terminology and definitions used in reports. There were also discrepancies in the way financial costs related to delays were reported, i.e., whether reported as cost per day, cost per year. This made comparing the costs across countries challenging and prevented clear conclusions being drawn. Future research should therefore aim to ensure clarity when reporting financial expenditure, for example, by calculating the daily cost of delays. It is important to highlight that only eighteen studies were identified over the 20-year search period, suggesting this area has not yet been subject to much research focus. All high-income countries met inclusion, but the final sample included studies from only five countries. It might have been expected that studies in other high-income countries be identified, particularly given the expensive nature of inpatient stays and as such delayed discharge. It might be beneficial for future research to further examine delayed discharge in psychiatric settings across other countries, particularly in the USA and EU. For the purposes of this review, studies not conducted in high-income countries were excluded. This was because lower-income countries might experience different factors contributing to delays due to differences in healthcare funding and social factors. As such, separate attention should be given to these settings, to understand similarities or differences in reasons for delays across low- and mid- income countries. Studies on forensic psychiatric settings and child and adolescent settings were also excluded in this instance, so again, there might be benefit in future research examining these areas.

Furthermore, future research could look not only at factors creating delays, but those causing longer delays. Some of the studies in this review began examining this, but more research in this area could be of interest. Finally, while the quality of included studies was relatively high, the studies were primarily of quantitative audit design and infrequently conducted significance testing. As such, further exploration of associations using significance testing would strengthen the evidence base.

In conclusion, 18 studies identified reasons for delayed discharge, including accommodation and placement related factors, challenges securing appropriate support, funding difficulties, family/carer factors, forensic factors and person being out of area. Delay was associated with having a diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, increased service involvement prior to admission, and being unemployed. Service, societal and policy changes might be indicated, to improve accommodation and care provisions following discharge. Future research should continue to examine prolonged inpatient psychiatric stays, ensuring to distinguish between long stays and delayed discharge and improve clarity in terminology used.

Data availability

The data on which this review is based will be made publicly available on publication. A link to data for anonymous peer-review is here: https://osf.io/j4kng/?view_only=1fbf2558d9d044bbb1778fccd5fd6f51 .

Bryan K. Policies for reducing delayed discharge from hospital. Br Med Bull. 2010;95(1):33–46.

Article   PubMed   Google Scholar  

NHS England. Monthly delayed transfers of care situation reports: definitions and guidance. London: NHS England; 2015.

Google Scholar  

House of Commons Health Committee. Delayed Discharges (third report). In: Committee H, editor. 2002.

Rojas-García A, Turner S, Pizzo E, Hudson E, Thomas J, Raine R. Impact and experiences of delayed discharge: a mixed‐studies systematic review. Health Expect. 2018;21(1):41–56.

Christ W. Factors delaying discharge of psychiatric patients. Health Soc Work. 1984;9(3):178–87.

Article   CAS   Google Scholar  

Department of Health. NHS reference costs 2013 to 2014. Department of Health London; 2013.

Glasby J, Lester H. Delayed hospital discharge and mental health: the policy implications of recent research. Social Policy Adm. 2004;38(7):744–57.

Article   Google Scholar  

National Institute of Health and Care Excellence [NICE]. Transition between inpatient mental health settings and community or care home settings. National Institue for Health and Care Excellence; 2016.

Katsakou C, Rose D, Amos T, Bowers L, McCabe R, Oliver D, et al. Psychiatric patients’ views on why their involuntary hospitalisation was right or wrong: a qualitative study. Soc Psychiatry Psychiatr Epidemiol. 2012;47:1169–79.

Csipke E, Williams P, Rose D, Koeser L, McCrone P, Wykes T, et al. Following the Francis report: investigating patient experience of mental health in-patient care. Br J Psychiatry. 2016;209(1):35–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gopalakrishna G, Ithman M, Malwitz K. Predictors of length of stay in a psychiatric hospital. Int J Psychiatry Clin Pract. 2015;19(4):238–44.

Micallef A, Buttigieg SC, Tomaselli G, Garg L. Defining delayed discharges of inpatients and their impact in acute hospital care: a scoping review. Int J Health Policy Manage. 2022;11(2):103.

Cadel L, Guilcher SJ, Kokorelias KM, Sutherland J, Glasby J, Kiran T, et al. Initiatives for improving delayed discharge from a hospital setting: a scoping review. BMJ open. 2021;11(2):e044291.

Article   PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group* t. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. 2021;134:103–12.

World Bank. World Bank Country and Lending Groups 2022 [ https://datahelpdesk.worldbank.org/knowledgebase/articles/906519.]

Covidence systematic review software. Veritas health innovation Melbourne, Australia. 2020. Available at: www.covidence.org.

Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. 2012;18(4):746–52.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inform. 2018;34(4):285–91.

Hanif I, Rathod B. Delays in discharging elderly psychiatric in-patients. Psychiatr Bull. 2008;32(6):211–3.

Little J, Hirdes JP, Daniel I, editors. ALC status in in-patient mental health settings: evidence based on the Ontario Mental Health Reporting System. Healthcare Management Forum. Los Angeles, CA: SAGE Publications Sage CA; 2015.

Little J, Hirdes JP, Perlman CM, Meyer SB. Clinical predictors of delayed discharges in inpatient mental health settings across Ontario. Adm Policy Mental Health Mental Health Serv Res. 2019;46:105–14.

Aflalo M, Soucy N, Xue X, Colacone A, Jourdenais E, Boivin J-F. Characteristics and needs of psychiatric patients with prolonged hospital stay. Can J Psychiatry. 2015;60(4):181–8.

Honey A, Arblaster K, Nguyen J, Heard R. Predicting Housing Related delayed discharge from Mental Health Inpatient units: a Case Control Study. Adm Policy Mental Health Mental Health Serv Res. 2022;49(6):962–72.

Nguyen J, Honey A, Arblaster K, Heard R. Housing-related delayed discharge from inpatient mental health units: Magnitude and contributors in a metropolitan mental health service. Australian J Social Issues. 2022;57(1):144–63.

Chuah CPT, Honey A, Arblaster K. I’m institutionalised… there’s not much I can do’: lived experience of housing related delayed discharge. Aust Occup Ther J. 2022;69(5):574–84.

Cowman J, Whitty P. Prevalence of housing needs among inpatients: a 1 year audit of housing needs in the acute mental health unit in Tallaght Hospital. Ir J Psychol Med. 2016;33(3):159–64.

Article   CAS   PubMed   Google Scholar  

Lewis R, Glasby J. Delayed discharge from mental health hospitals: results of an English postal survey. Health Soc Care Commun. 2006;14(3):225–30.

Berg JE, Restan A. Duration of bed occupancy as calculated at a random chosen day in an acute care ward. Implications for the use of scarce resources in psychiatric care. Ann Gen Psychiatry. 2005;4(1):1–6.

Tucker S, Hargreaves C, Wilberforce M, Brand C, Challis D. What becomes of people admitted to acute old age psychiatry wards? An exploration of factors affecting length of stay, delayed discharge and discharge destination. Int J Geriatr Psychiatry. 2017;32(9):1027–36.

Tyrer P, Suryanarayan G, Rao B, Cicchetti D, Fulop N, Roberts F, et al. The bed requirement inventory: a simple measure to estimate the need for a psychiatric bed. Int J Soc Psychiatry. 2006;52(3):267–77.

Onyon R, Khan S, George M. Delayed discharges from a psychiatric intensive care unit–are we detaining patients unlawfully? J Psychiatric Intensive Care. 2006;2(2):59–64.

Impey M, Milner E. Delayed discharge from mental health inpatient care in the UK. Mental Health Pract. 2013;16(9).

Haw C, Otuwehinmi O, Kotterbova E. Out of area admissions to two independent sector PICUs: patient characteristics, length of stay and delayed discharges. J Psychiatric Intensive Care. 2017;13(1):27–36.

Poole R, Pearsall A, Ryan T. Delayed discharges in an urban in-patient mental health service in England. Psychiatric Bull. 2014;38(2):66–70.

Commander M, Rooprai D. Survey of long-stay patients on acute psychiatric wards. Psychiatr Bull. 2008;32(10):380–3.

Paton JM, Fahy MA, Livingston GA. Delayed discharge—a solvable problem? The place of intermediate care in mental health care of older people. Aging Ment Health. 2004;8(1):34–9.

Download references

No funding to declare.

Author information

Authors and affiliations.

Department of Psychology, University of Bath, Bath, BA2 7AY, UK

Ashley-Louise Teale, Ceri Morgan, Tom A. Jenkins & Pamela Jacobsen

You can also search for this author in PubMed   Google Scholar

Contributions

AT and PJ formulated the initial research questions and developed the systematic review protocol. AT ran the searches on databases. AT, CM and TJ conducted the screening, data extraction and quality assessment. PJ acted as senior reviewer to resolve any conflicts. AT synthesised the results. All authors contributed to data synthesis and interpretation. AT wrote the paper. All authors read and approved the final version of manuscript for submission.

Corresponding author

Correspondence to Ashley-Louise Teale .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Teale, AL., Morgan, C., Jenkins, T.A. et al. Delayed discharge in inpatient psychiatric care: a systematic review. Int J Ment Health Syst 18 , 14 (2024). https://doi.org/10.1186/s13033-024-00635-9

Download citation

Received : 28 July 2023

Accepted : 25 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1186/s13033-024-00635-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Delayed discharge
• Bed blocking
• Delayed transfer
• Psychiatric inpatient
• Inpatient treatment
• Prolonged stays
• Length of stay

International Journal of Mental Health Systems

ISSN: 1752-4458

• Submission enquiries: [email protected]