qualitative research a literature review

Archer Library

Qualitative research: literature review .

Archer Library This link opens in a new window
Schedule a Reference Appointment This link opens in a new window
Qualitative Research Handout This link opens in a new window
Locating Books
ebook Collections This link opens in a new window
A to Z Database List This link opens in a new window
Research & Stats
Literature Review Resources
Citation & Reference

Exploring the literature review

Literature review model: 6 steps.

Adapted from The Literature Review , Machi & McEvoy (2009, p. 13).

Your Literature Review

Step 2: search, boolean search strategies, search limiters, ★ ebsco & google drive.

1. Select a Topic

"All research begins with curiosity" (Machi & McEvoy, 2009, p. 14)

Selection of a topic, and fully defined research interest and question, is supervised (and approved) by your professor. Tips for crafting your topic include:

Be specific. Take time to define your interest.
Topic Focus. Fully describe and sufficiently narrow the focus for research.
Academic Discipline. Learn more about your area of research & refine the scope.
Avoid Bias. Be aware of bias that you (as a researcher) may have.
Document your research. Use Google Docs to track your research process.
Research apps. Consider using Evernote or Zotero to track your research.

Consider Purpose

What will your topic and research address?

In The Literature Review: A Step-by-Step Guide for Students , Ridley presents that literature reviews serve several purposes (2008, p. 16-17). Included are the following points:

Historical background for the research;
Overview of current field provided by "contemporary debates, issues, and questions;"
Theories and concepts related to your research;
Introduce "relevant terminology" - or academic language - being used it the field;
Connect to existing research - does your work "extend or challenge [this] or address a gap;"
Provide "supporting evidence for a practical problem or issue" that your research addresses.

★ Schedule a research appointment

At this point in your literature review, take time to meet with a librarian. Why? Understanding the subject terminology used in databases can be challenging. Archer Librarians can help you structure a search, preparing you for step two. How? Contact a librarian directly or use the online form to schedule an appointment. Details are provided in the adjacent Schedule an Appointment box.

2. Search the Literature

Collect & Select Data: Preview, select, and organize

AU Library is your go-to resource for this step in your literature review process. The literature search will include books and ebooks, scholarly and practitioner journals, theses and dissertations, and indexes. You may also choose to include web sites, blogs, open access resources, and newspapers. This library guide provides access to resources needed to complete a literature review.

Books & eBooks: Archer Library & OhioLINK

Databases: scholarly & practitioner journals.

Review the Library Databases tab on this library guide, it provides links to recommended databases for Education & Psychology, Business, and General & Social Sciences.

Expand your journal search; a complete listing of available AU Library and OhioLINK databases is available on the Databases A to Z list . Search the database by subject, type, name, or do use the search box for a general title search. The A to Z list also includes open access resources and select internet sites.

Databases: Theses & Dissertations

Review the Library Databases tab on this guide, it includes Theses & Dissertation resources. AU library also has AU student authored theses and dissertations available in print, search the library catalog for these titles.

Did you know? If you are looking for particular chapters within a dissertation that is not fully available online, it is possible to submit an ILL article request . Do this instead of requesting the entire dissertation.

Newspapers: Databases & Internet

Consider current literature in your academic field. AU Library's database collection includes The Chronicle of Higher Education and The Wall Street Journal . The Internet Resources tab in this guide provides links to newspapers and online journals such as Inside Higher Ed , COABE Journal , and Education Week .

Search Strategies & Boolean Operators

There are three basic boolean operators: AND, OR, and NOT.

Used with your search terms, boolean operators will either expand or limit results. What purpose do they serve? They help to define the relationship between your search terms. For example, using the operator AND will combine the terms expanding the search. When searching some databases, and Google, the operator AND may be implied.

Overview of boolean terms

About the example: Boolean searches were conducted on November 4, 2019; result numbers may vary at a later date. No additional database limiters were set to further narrow search returns.

Database Search Limiters

Database strategies for targeted search results.

Most databases include limiters, or additional parameters, you may use to strategically focus search results. EBSCO databases, such as Education Research Complete & Academic Search Complete provide options to:

Limit results to full text;
Limit results to scholarly journals, and reference available;
Select results source type to journals, magazines, conference papers, reviews, and newspapers
Publication date

Keep in mind that these tools are defined as limiters for a reason; adding them to a search will limit the number of results returned. This can be a double-edged sword. How?

If limiting results to full-text only, you may miss an important piece of research that could change the direction of your research. Interlibrary loan is available to students, free of charge. Request articles that are not available in full-text; they will be sent to you via email.
If narrowing publication date, you may eliminate significant historical - or recent - research conducted on your topic.
Limiting resource type to a specific type of material may cause bias in the research results.

Use limiters with care. When starting a search, consider opting out of limiters until the initial literature screening is complete. The second or third time through your research may be the ideal time to focus on specific time periods or material (scholarly vs newspaper).

★ Truncating Search Terms

Expanding your search term at the root.

Truncating is often referred to as 'wildcard' searching. Databases may have their own specific wildcard elements however, the most commonly used are the asterisk (*) or question mark (?). When used within your search. they will expand returned results.

Asterisk (*) Wildcard

Using the asterisk wildcard will return varied spellings of the truncated word. In the following example, the search term education was truncated after the letter "t."

Explore these database help pages for additional information on crafting search terms.

EBSCO Connect: Searching with Wildcards and Truncation Symbols
EBSCO Connect: Searching with Boolean Operators
EBSCO Connect: EBSCOhost Search Tips
EBSCO Connect: Basic Searching with EBSCO
ProQuest Help: Search Tips
ERIC: How does ERIC search work?

★ EBSCO Databases & Google Drive

Tips for saving research directly to Google drive.

Researching in an EBSCO database?

It is possible to save articles (PDF and HTML) and abstracts in EBSCOhost databases directly to Google drive. Select the Google Drive icon, authenticate using a Google account, and an EBSCO folder will be created in your account. This is a great option for managing your research. If documenting your research in a Google Doc, consider linking the information to actual articles saved in drive.

EBSCO Databases & Google Drive

EBSCOHost Databases & Google Drive: Managing your Research

This video features an overview of how to use Google Drive with EBSCO databases to help manage your research. It presents information for connecting an active Google account to EBSCO and steps needed to provide permission for EBSCO to manage a folder in Drive.

About the Video: Closed captioning is available, select CC from the video menu. If you need to review a specific area on the video, view on YouTube and expand the video description for access to topic time stamps. A video transcript is provided below.

EBSCOhost Databases & Google Scholar

Defining Literature Review

What is a literature review.

A definition from the Online Dictionary for Library and Information Sciences .

A literature review is "a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works" (Reitz, 2014).

A systemic review is "a literature review focused on a specific research question, which uses explicit methods to minimize bias in the identification, appraisal, selection, and synthesis of all the high-quality evidence pertinent to the question" (Reitz, 2014).

About this page

EBSCO Connect [Discovery and Search]. (2022). Searching with boolean operators. Retrieved May, 3, 2022 from https://connect.ebsco.com/s/?language=en_US

EBSCO Connect [Discover and Search]. (2022). Searching with wildcards and truncation symbols. Retrieved May 3, 2022; https://connect.ebsco.com/s/?language=en_US

Machi, L.A. & McEvoy, B.T. (2009). The literature review . Thousand Oaks, CA: Corwin Press:

Reitz, J.M. (2014). Online dictionary for library and information science. ABC-CLIO, Libraries Unlimited . Retrieved from https://www.abc-clio.com/ODLIS/odlis_A.aspx

Ridley, D. (2008). The literature review: A step-by-step guide for students . Thousand Oaks, CA: Sage Publications, Inc.

Archer Librarians

Schedule an appointment.

Contact a librarian directly (email), or submit a request form. If you have worked with someone before, you can request them on the form.

★ Archer Library Help • Online Reqest Form
Carrie Halquist • Reference & Instruction
Jessica Byers • Reference & Curation
Don Reams • Corrections Education & Reference
Diane Schrecker • Education & Head of the IRC
Tanaya Silcox • Technical Services & Business
Sarah Thomas • Acquisitions & ATS Librarian
<< Previous: Research & Stats
Next: Literature Review Resources >>
Last Updated: Nov 17, 2023 7:46 AM
URL: https://libguides.ashland.edu/qualitative

Archer Library • Ashland University © Copyright 2023. An Equal Opportunity/Equal Access Institution.

Chapter 9. Reviewing the Literature

What is a “literature review”.

No researcher ever comes up with a research question that is wholly novel. Someone, somewhere, has asked the same thing. Academic research is part of a larger community of researchers, and it is your responsibility, as a member of this community, to acknowledge others who have asked similar questions and to put your particular research into this greater context. It is not simply a convention or custom to begin your study with a review of previous literature (the “ lit review ”) but an important responsibility you owe the scholarly community.

Too often, new researchers pursue a topic to study and then write something like, “No one has ever studied this before” or “This area is underresearched.” It may be that no one has studied this particular group or setting, but it is highly unlikely no one has studied the foundational phenomenon of interest. And that comment about an area being underresearched? Be careful. The statement may simply signal to others that you haven’t done your homework. Rubin ( 2021 ) refers to this as “free soloing,” and it is not appreciated in academic work:

The truth of the matter is, academics don’t really like when people free solo. It’s really bad form to omit talking about the other people who are doing or have done research in your area. Partly, I mean we need to cite their work, but I also mean we need to respond to it—agree or disagree, clarify for extend. It’s also really bad form to talk about your research in a way that does not make it understandable to other academics.…You have to explain to your readers what your story is really about in terms they care about . This means using certain terminology, referencing debates in the literature, and citing relevant works—that is, in connecting your work to something else. ( 51–52 )

A literature review is a comprehensive summary of previous research on a topic. It includes both articles and books—and in some cases reports—relevant to a particular area of research. Ideally, one’s research question follows from the reading of what has already been produced. For example, you are interested in studying sports injuries related to female gymnasts. You read everything you can find on sports injuries related to female gymnasts, and you begin to get a sense of what questions remain open. You find that there is a lot of research on how coaches manage sports injuries and much about cultures of silence around treating injuries, but you don’t know what the gymnasts themselves are thinking about these issues. You look specifically for studies about this and find several, which then pushes you to narrow the question further. Your literature review then provides the road map of how you came to your very specific question, and it puts your study in the context of studies of sports injuries. What you eventually find can “speak to” all the related questions as well as your particular one.

In practice, the process is often a bit messier. Many researchers, and not simply those starting out, begin with a particular question and have a clear idea of who they want to study and where they want to conduct their study but don’t really know much about other studies at all. Although backward, we need to recognize this is pretty common. Telling students to “find literature” after the fact can seem like a purposeless task or just another hurdle for completing a thesis or dissertation. It is not! Even if you were not motivated by the literature in the first place, acknowledging similar studies and connecting your own research to those studies are important parts of building knowledge. Acknowledgment of past research is a responsibility you owe the discipline to which you belong.

Literature reviews can also signal theoretical approaches and particular concepts that you will incorporate into your own study. For example, let us say you are doing a study of how people find their first jobs after college, and you want to use the concept of social capital . There are competing definitions of social capital out there (e.g., Bourdieu vs. Burt vs. Putnam). Bourdieu’s notion is of one form of capital, or durable asset, of a “network of more or less institutionalized relationships of mutual acquaintance or recognition” ( 1984:248 ). Burt emphasizes the “brokerage opportunities” in a social network as social capital ( 1997:355 ). Putnam’s social capital is all about “facilitating coordination and cooperation for mutual benefit” ( 2001:67 ). Your literature review can adjudicate among these three approaches, or it can simply refer to the one that is animating your own research. If you include Bourdieu in your literature review, readers will know “what kind” of social capital you are talking about as well as what kind of social scientist you yourself are. They will likely understand that you are interested more in how some people are advantaged by their social capital relative to others rather than being interested in the mechanics of how social networks operate.

The literature review thus does two important things for you: firstly, it allows you to acknowledge previous research in your area of interest, thereby situating you within a discipline or body of scholars, and, secondly, it demonstrates that you know what you are talking about. If you present the findings of your research study without including a literature review, it can be like singing into the wind. It sounds nice, but no one really hears it, or if they do catch snippets, they don’t know where it is coming from.

Examples of Literature Reviews

To help you get a grasp of what a good literature review looks like and how it can advance your study, let’s take a look at a few examples.

Reader-Friendly Example: The Power of Peers

The first is by Janice McCabe ( 2016 ) and is from an article on peer networks in the journal Contexts . Contexts presents articles in a relatively reader-friendly format, with the goal of reaching a large audience for interesting sociological research. Read this example carefully and note how easily McCabe is able to convey the relevance of her own work by situating it in the context of previous studies:

Scholars who study education have long acknowledged the importance of peers for students’ well-being and academic achievement. For example, in 1961, James Coleman argued that peer culture within high schools shapes students’ social and academic aspirations and successes. More recently, Judith Rich Harris has drawn on research in a range of areas—from sociological studies of preschool children to primatologists’ studies of chimpanzees and criminologists’ studies of neighborhoods—to argue that peers matter much more than parents in how children “turn out.” Researchers have explored students’ social lives in rich detail, as in Murray Milner’s book about high school students, Freaks, Geeks, and Cool Kids , and Elizabeth Armstrong and Laura Hamilton’s look at college students, Paying for the Party . These works consistently show that peers play a very important role in most students’ lives. They tend, however, to prioritize social over academic influence and to use a fuzzy conception of peers rather than focusing directly on friends—the relationships that should matter most for student success. Social scientists have also studied the power of peers through network analysis, which is based on uncovering the web of connections between people. Network analysis involves visually mapping networks and mathematically comparing their structures (such as the density of ties) and the positions of individuals within them (such as how central a given person is within the network). As Nicholas Christakis and James Fowler point out in their book Connected , network structure influences a range of outcomes, including health, happiness, wealth, weight, and emotions. Given that sociologists have long considered network explanations for social phenomena, it’s surprising that we know little about how college students’ friends impact their experiences. In line with this network tradition, I focus on the structure of friendship networks, constructing network maps so that the differences we see across participants are due to the underlying structure, including each participant’s centrality in their friendship group and the density of ties among their friends. ( 23 )

What did you notice? In her very second sentence, McCabe uses “for example” to introduce a study by Coleman, thereby indicating that she is not going to tell you every single study in this area but is going to tell you that (1) there is a lot of research in this area, (2) it has been going on since at least 1961, and (3) it is still relevant (i.e., recent studies are still being done now). She ends her first paragraph by summarizing the body of literature in this area (after giving you a few examples) and then telling you what may have been (so far) left out of this research. In the second paragraph, she shifts to a separate interesting focus that is related to the first but is also quite distinct. Lit reviews very often include two (or three) distinct strands of literature, the combination of which nicely backgrounds this particular study . In the case of our female gymnast study (above), those two strands might be (1) cultures of silence around sports injuries and (2) the importance of coaches. McCabe concludes her short and sweet literature review with one sentence explaining how she is drawing from both strands of the literature she has succinctly presented for her particular study. This example should show you that literature reviews can be readable, helpful, and powerful additions to your final presentation.

Authoritative Academic Journal Example: Working Class Students’ College Expectations

The second example is more typical of academic journal writing. It is an article published in the British Journal of Sociology of Education by Wolfgang Lehmann ( 2009 ):

Although this increase in post-secondary enrolment and the push for university is evident across gender, race, ethnicity, and social class categories, access to university in Canada continues to be significantly constrained for those from lower socio-economic backgrounds (Finnie, Lascelles, and Sweetman 2005). Rising tuition fees coupled with an overestimation of the cost and an underestimation of the benefits of higher education has put university out of reach for many young people from low-income families (Usher 2005). Financial constraints aside, empirical studies in Canada have shown that the most important predictor of university access is parental educational attainment. Having at least one parent with a university degree significantly increases the likelihood of a young person to attend academic-track courses in high school, have high educational and career aspirations, and ultimately attend university (Andres et al. 1999, 2000; Lehmann 2007a). Drawing on Bourdieu’s various writing on habitus and class-based dispositions (see, for example, Bourdieu 1977, 1990), Hodkinson and Sparkes (1997) explain career decisions as neither determined nor completely rational. Instead, they are based on personal experiences (e.g., through employment or other exposure to occupations) and advice from others. Furthermore, they argue that we have to understand these decisions as pragmatic, rather than rational. They are pragmatic in that they are based on incomplete and filtered information, because of the social context in which the information is obtained and processed. New experiences and information can, however, also be allowed into one’s world, where they gradually or radically transform habitus, which in turn creates the possibility for the formation of new and different dispositions. Encountering a supportive teacher in elementary or secondary school, having ambitious friends, or chance encounters can spark such transformations. Transformations can be confirming or contradictory, they can be evolutionary or dislocating. Working-class students who enter university most certainly encounter such potentially transformative situations. Granfield (1991) has shown how initially dislocating feelings of inadequacy and inferiority of working-class students at an elite US law school were eventually replaced by an evolutionary transformation, in which the students came to dress, speak and act more like their middle-class and upper-class peers. In contrast, Lehmann (2007b) showed how persistent habitus dislocation led working-class university students to drop out of university. Foskett and Hemsley-Brown (1999) argue that young people’s perceptions of careers are a complex mix of their own experiences, images conveyed through adults, and derived images conveyed by the media. Media images of careers, perhaps, are even more important for working-class youth with high ambitions as they offer (generally distorted) windows into a world of professional employment to which they have few other sources of access. It has also been argued that working-class youth who do continue to university still face unique, class-specific challenges, evident in higher levels of uncertainty (Baxter and Britton 2001; Lehmann 2004, 2007a; Quinn 2004), their higher education choices (Ball et al. 2002; Brooks 2003; Reay et al. 2001) and fears of inadequacy because of their cultural outsider status (Aries and Seider 2005; Granfield 1991). Although the number of working-class university students in Canada has slowly increased, that of middle-class students at university has risen far more steeply (Knighton and Mizra 2002). These different enrolment trajectories have actually widened the participation gap, which in tum explains our continued concerns with the potential outsider status Indeed, in a study comparing first-generation working-class and traditional students who left university without graduating, Lehmann (2007b) found that first-generation working-class students were more likely to leave university very early in some cases within the first two months of enrollment. They were also more likely to leave university despite solid academic performance. Not “fitting in,” not “feeling university,” and not being able to “relate to these people” were key reasons for eventually withdrawing from university. From the preceding review of the literature, a number of key research questions arise: How do working-class university students frame their decision to attend university? How do they defy the considerable odds documented in the literature to attend university? What are the sources of information and various images that create dispositions to study at university? What role does their social-class background- or habitus play in their transition dispositions and how does this translate into expectations for university? ( 139 )

What did you notice here? How is this different from (and similar to) the first example? Note that rather than provide you with one or two illustrative examples of similar types of research, Lehmann provides abundant source citations throughout. He includes theory and concepts too. Like McCabe, Lehmann is weaving through multiple literature strands: the class gap in higher education participation in Canada, class-based dispositions, and obstacles facing working-class college students. Note how he concludes the literature review by placing his research questions in context.

Find other articles of interest and read their literature reviews carefully. I’ve included two more for you at the end of this chapter . As you learned how to diagram a sentence in elementary school (hopefully!), try diagramming the literature reviews. What are the “different strands” of research being discussed? How does the author connect these strands to their own research questions? Where is theory in the lit review, and how is it incorporated (e.g., Is it a separate strand of its own or is it inextricably linked with previous research in this area)?

One model of how to structure your literature review can be found in table 9.1. More tips, hints, and practices will be discussed later in the chapter.

Table 9.1. Model of Literature Review, Adopted from Calarco (2020:166)

Embracing Theory

A good research study will, in some form or another, use theory. Depending on your particular study (and possibly the preferences of the members of your committee), theory may be built into your literature review. Or it may form its own section in your research proposal/design (e.g., “literature review” followed by “theoretical framework”). In my own experience, I see a lot of graduate students grappling with the requirement to “include theory” in their research proposals. Things get a little squiggly here because there are different ways of incorporating theory into a study (Are you testing a theory? Are you generating a theory?), and based on these differences, your literature review proper may include works that describe, explain, and otherwise set forth theories, concepts, or frameworks you are interested in, or it may not do this at all. Sometimes a literature review sets forth what we know about a particular group or culture totally independent of what kinds of theoretical framework or particular concepts you want to explore. Indeed, the big point of your study might be to bring together a body of work with a theory that has never been applied to it previously. All this is to say that there is no one correct way to approach the use of theory and the writing about theory in your research proposal.

Students are often scared of embracing theory because they do not exactly understand what it is. Sometimes, it seems like an arbitrary requirement. You’re interested in a topic; maybe you’ve even done some research in the area and you have findings you want to report. And then a committee member reads over what you have and asks, “So what?” This question is a good clue that you are missing theory, the part that connects what you have done to what other researchers have done and are doing. You might stumble upon this rather accidentally and not know you are embracing theory, as in a case where you seek to replicate a prior study under new circumstances and end up finding that a particular correlation between behaviors only happens when mediated by something else. There’s theory in there, if you can pull it out and articulate it. Or it might be that you are motivated to do more research on racial microaggressions because you want to document their frequency in a particular setting, taking for granted the kind of critical race theoretical framework that has done the hard work of defining and conceptualizing “microaggressions” in the first place. In that case, your literature review could be a review of Critical Race Theory, specifically related to this one important concept. That’s the way to bring your study into a broader conversation while also acknowledging (and honoring) the hard work that has preceded you.

Rubin ( 2021 ) classifies ways of incorporating theory into case study research into four categories, each of which might be discussed somewhat differently in a literature review or theoretical framework section. The first, the least theoretical, is where you set out to study a “configurative idiographic case” ( 70 ) This is where you set out to describe a particular case, leaving yourself pretty much open to whatever you find. You are not expecting anything based on previous literature. This is actually pretty weak as far as research design goes, but it is probably the default for novice researchers. Your committee members should probably help you situate this in previous literature in some way or another. If they cannot, and it really does appear you are looking at something fairly new that no one else has bothered to research before, and you really are completely open to discovery, you might try using a Grounded Theory approach, which is a methodological approach that foregrounds the generation of theory. In that case, your “theory” section can be a discussion of “Grounded Theory” methodology (confusing, yes, but if you take some time to ponder, you will see how this works). You will still need a literature review, though. Ideally one that describes other studies that have ever looked at anything remotely like what you are looking at—parallel cases that have been researched.

The second approach is the “disciplined configurative case,” in which theory is applied to explain a particular case or topic. You are not trying to test the theory but rather assuming the theory is correct, as in the case of exploring microaggressions in a particular setting. In this case, you really do need to have a separate theory section in addition to the literature review, one in which you clearly define the theoretical framework, including any of its important concepts. You can use this section to discuss how other researchers have used the concepts and note any discrepancies in definitions or operationalization of those concepts. This way you will be sure to design your study so that it speaks to and with other researchers. If everyone who is writing about microaggressions has a different definition of them, it is hard for others to compare findings or make any judgments about their prevalence (or any number of other important characteristics). Your literature review section may then stand alone and describe previous research in the particular area or setting, irrespective of the kinds of theory underlying those studies.

The third approach is “heuristic,” one in which you seek to identify new variables, hypotheses, mechanisms, or paths not yet explained by a theory or theoretical framework. In a way, you are generating new theory, but it is probably more accurate to say that you are extending or deepening preexisting theory. In this case, having a single literature review that is focused on the theory and the ways the theory has been applied and understood (with all its various mechanisms and pathways) is probably your best option. The focus of the literature reviewed is less on the case and more on the theory you are seeking to extend.

The final approach is “theory testing,” which is much rarer in qualitative studies than in quantitative, where this is the default approach. Theory-testing cases are those where a particular case is used to see if an existing theory is accurate or accurate under particular circumstances. As with the heuristic approach, your literature review will probably draw heavily on previous uses of the theory, but you may end up having a special section specifically about cases very close to your own . In other words, the more your study approaches theory testing, the more likely there is to be a set of similar studies to draw on or even one important key study that you are setting your own study up in parallel to in order to find out if the theory generated there operates here.

If we wanted to get very technical, it might be useful to distinguish theoretical frameworks properly from conceptual frameworks. The latter are a bit looser and, given the nature of qualitative research, often fit exploratory studies. Theoretical frameworks rely on specific theories and are essential for theory-testing studies. Conceptual frameworks can pull in specific concepts or ideas that may or may not be linked to particular theories. Think about it this way: A theory is a story of how the world works. Concepts don’t presume to explain the whole world but instead are ways to approach phenomena to help make sense of them. Microaggressions are concepts that are linked to Critical Race Theory. One could contextualize one’s study within Critical Race Theory and then draw various concepts, such as that of microaggressions from the overall theoretical framework. Or one could bracket out the master theory or framework and employ the concept of microaggression more opportunistically as a phenomenon of interest. If you are unsure of what theory you are using, you might want to frame a more practical conceptual framework in your review of the literature.

Helpful Tips

How to maintain good notes for what your read.

Over the years, I have developed various ways of organizing notes on what I read. At first, I used a single sheet of full-size paper with a preprinted list of questions and points clearly addressed on the front side, leaving the second side for more reflective comments and free-form musings about what I read, why it mattered, and how it might be useful for my research. Later, I developed a system in which I use a single 4″ × 6″ note card for each book I read. I try only to use the front side (and write very small), leaving the back for comments that are about not just this reading but things to do or examine or consider based on the reading. These notes often mean nothing to anyone else picking up the card, but they make sense to me. I encourage you to find an organizing system that works for you. Then when you set out to compose a literature review, instead of staring at five to ten books or a dozen articles, you will have ten neatly printed pages or notecards or files that have distilled what is important to know about your reading.

It is also a good idea to store this data digitally, perhaps through a reference manager. I use RefWorks, but I also recommend EndNote or any other system that allows you to search institutional databases. Your campus library will probably provide access to one of these or another system. Most systems will allow you to export references from another manager if and when you decide to move to another system. Reference managers allow you to sort through all your literature by descriptor, author, year, and so on. Even so, I personally like to have the ability to manually sort through my index cards, recategorizing things I have read as I go. I use RefWorks to keep a record of what I have read, with proper citations, so I can create bibliographies more easily, and I do add in a few “notes” there, but the bulk of my notes are kept in longhand.

What kinds of information should you include from your reading? Here are some bulleted suggestions from Calarco ( 2020:113–114 ), with my own emendations:

Citation . If you are using a reference manager, you can import the citation and then, when you are ready to create a bibliography, you can use a provided menu of citation styles, which saves a lot of time. If you’ve originally formatted in Chicago Style but the journal you are writing for wants APA style, you can change your entire bibliography in less than a minute. When using a notecard for a book, I include author, title, date as well as the library call number (since most of what I read I pull from the library). This is something RefWorks is not able to do, and it helps when I categorize.

I begin each notecard with an “intro” section, where I record the aims, goals, and general point of the book/article as explained in the introductory sections (which might be the preface, the acknowledgments, or the first two chapters). I then draw a bold line underneath this part of the notecard. Everything after that should be chapter specific. Included in this intro section are things such as the following, recommended by Calarco ( 2020 ):

Key background . “Two to three short bullet points identifying the theory/prior research on which the authors are building and defining key terms.”
Data/methods . “One or two short bullet points with information about the source of the data and the method of analysis, with a note if this is a novel or particularly effective example of that method.” I use [M] to signal methodology on my notecard, which might read, “[M] Int[erview]s (n-35), B[lack]/W[hite] voters” (I need shorthand to fit on my notecard!).
Research question . “Stated as briefly as possible.” I always provide page numbers so I can go back and see exactly how this was stated (sometimes, in qualitative research, there are multiple research questions, and they cannot be stated simply).
Argument/contributions . “Two to three short bullet points briefly describing the authors’ answer to the central research question and its implication for research, theory, and practice.” I use [ARG] for argument to signify the argument, and I make sure this is prominently visible on my notecard. I also provide page numbers here.

For me, all of this fits in the “intro” section, which, if this is a theoretically rich, methodologically sound book, might take up a third or even half of the front page of my notecard. Beneath the bold underline, I report specific findings or particulars of the book as they emerge chapter by chapter. Calarco’s ( 2020 ) next step is the following:

Key findings . “Three to four short bullet points identifying key patterns in the data that support the authors’ argument.”

All that remains is writing down thoughts that occur upon finishing the article/book. I use the back of the notecard for these kinds of notes. Often, they reach out to other things I have read (e.g., “Robinson reminds me of Crusoe here in that both are looking at the effects of social isolation, but I think Robinson makes a stronger argument”). Calarco ( 2020 ) concludes similarly with the following:

Unanswered questions . “Two to three short bullet points that identify key limitations of the research and/or questions the research did not answer that could be answered in future research.”

As I mentioned, when I first began taking notes like this, I preprinted pages with prompts for “research question,” “argument,” and so on. This was a great way to remind myself to look for these things in particular. You can do the same, adding whatever preprinted sections make sense to you, given what you are studying and the important aspects of your discipline. The other nice thing about the preprinted forms is that it keeps your writing to a minimum—you cannot write more than the allotted space, even if you might want to, preventing your notes from spiraling out of control. This can be helpful when we are new to a subject and everything seems worth recording!

After years of discipline, I have finally settled on my notecard approach. I have thousands of notecards, organized in several index card filing boxes stacked in my office. On the top right of each card is a note of the month/day I finished reading the item. I can remind myself what I read in the summer of 2010 if the need or desire ever arose to do so…those invaluable notecards are like a memento of what my brain has been up to!

Where to Start Looking for Literature

Your university library should provide access to one of several searchable databases for academic books and articles. My own preference is JSTOR, a service of ITHAKA, a not-for-profit organization that works to advance and preserve knowledge and to improve teaching and learning through the use of digital technologies. JSTOR allows you to search by several keywords and to narrow your search by type of material (articles or books). For many disciplines, the “literature” of the literature review is expected to be peer-reviewed “articles,” but some disciplines will also value books and book chapters. JSTOR is particularly useful for article searching. You can submit several keywords and see what is returned, and you can also narrow your search by a particular journal or discipline. If your discipline has one or two key journals (e.g., the American Journal of Sociology and the American Sociological Review are key for sociology), you might want to go directly to those journals’ websites and search for your topic area. There is an art to when to cast your net widely and when to refine your search, and you may have to tack back and forth to ensure that you are getting all that is relevant but not getting bogged down in all studies that might have some marginal relevance.

Some articles will carry more weight than others, and you can use applications like Google Scholar to see which articles have made and are continuing to make larger impacts on your discipline. Find these articles and read them carefully; use their literature review and the sources cited in those articles to make sure you are capturing what is relevant. This is actually a really good way of finding relevant books—only the most impactful will make it into the citations of journals. Over time, you will notice that a handful of articles (or books) are cited so often that when you see, say, Armstrong and Hamilton ( 2015 ), you know exactly what book this is without looking at the full cite. This is when you know you are in the conversation.

You might also approach a professor whose work is broadly in the area of your interest and ask them to recommend one or two “important” foundational articles or books. You can then use the references cited in those recommendations to build up your literature. Just be careful: some older professors’ knowledge of the literature (and I reluctantly add myself here) may be a bit outdated! It is best that the article or book whose references and sources you use to build your body of literature be relatively current.

Keep a List of Your Keywords

When using searchable databases, it is a good idea to keep a list of all the keywords you use as you go along so that (1) you do not needlessly duplicate your efforts and (2) you can more easily adjust your search as you get a better sense of what you are looking for. I suggest you keep a separate file or even a small notebook for this and you date your search efforts.

Here’s an example:

Table 9.2. Keep a List of Your Keywords

Think Laterally

How to find the various strands of literature to combine? Don’t get stuck on finding the exact same research topic you think you are interested in. In the female gymnast example, I recommended that my student consider looking for studies of ballerinas, who also suffer sports injuries and around whom there is a similar culture of silence. It turned out that there was in fact research about my student’s particular questions, just not about the subjects she was interested in. You might do something similar. Don’t get stuck looking for too direct literature but think about the broader phenomenon of interest or analogous cases.

Read Outside the Canon

Some scholars’ work gets cited by everyone all the time. To some extent, this is a very good thing, as it helps establish the discipline. For example, there are a lot of “Bourdieu scholars” out there (myself included) who draw ideas, concepts, and quoted passages from Bourdieu. This makes us recognizable to one another and is a way of sharing a common language (e.g., where “cultural capital” has a particular meaning to those versed in Bourdieusian theory). There are empirical studies that get cited over and over again because they are excellent studies but also because there is an “echo chamber effect” going on, where knowing to cite this study marks you as part of the club, in the know, and so on. But here’s the problem with this: there are hundreds if not thousands of excellent studies out there that fail to get appreciated because they are crowded out by the canon. Sometimes this happens because they are published in “lower-ranked” journals and are never read by a lot of scholars who don’t have time to read anything other than the “big three” in their field. Other times this happens because the author falls outside of the dominant social networks in the field and thus is unmentored and fails to get noticed by those who publish a lot in those highly ranked and visible spaces. Scholars who fall outside the dominant social networks and who publish outside of the top-ranked journals are in no way less insightful than their peers, and their studies may be just as rigorous and relevant to your work, so it is important for you to take some time to read outside the canon. Due to how a person’s race, gender, and class operate in the academy, there is also a matter of social justice and ethical responsibility involved here: “When you focus on the most-cited research, you’re more likely to miss relevant research by women and especially women of color, whose research tends to be under-cited in most fields. You’re also more likely to miss new research, research by junior scholars, and research in other disciplines that could inform your work. Essentially, it is important to read and cite responsibly, which means checking that you’re not just reading and citing the same white men and the same old studies that everyone has cited before you” ( Calarco 2020:112 ).

Consider Multiple Uses for Literature

Throughout this chapter, I’ve referred to the literature of interest in a rather abstract way, as what is relevant to your study. But there are many different ways previous research can be relevant to your study. The most basic use of the literature is the “findings”—for example, “So-and-so found that Canadian working-class students were concerned about ‘fitting in’ to the culture of college, and I am going to look at a similar question here in the US.” But the literature may be of interest not for its findings but theoretically—for example, employing concepts that you want to employ in your own study. Bourdieu’s definition of social capital may have emerged in a study of French professors, but it can still be relevant in a study of, say, how parents make choices about what preschools to send their kids to (also a good example of lateral thinking!).

If you are engaged in some novel methodological form of data collection or analysis, you might look for previous literature that has attempted that. I would not recommend this for undergraduate research projects, but for graduate students who are considering “breaking the mold,” find out if anyone has been there before you. Even if their study has absolutely nothing else in common with yours, it is important to acknowledge that previous work.

Describing Gaps in the Literature

First, be careful! Although it is common to explain how your research adds to, builds upon, and fills in gaps in the previous research (see all four literature review examples in this chapter for this), there is a fine line between describing the gaps and misrepresenting previous literature by failing to conduct a thorough review of the literature. A little humility can make a big difference in your presentation. Instead of “This is the first study that has looked at how firefighters juggle childcare during forest fire season,” say, “I use the previous literature on how working parents juggling childcare and the previous ethnographic studies of firefighters to explore how firefighters juggle childcare during forest fire season.” You can even add, “To my knowledge, no one has conducted an ethnographic study in this specific area, although what we have learned from X about childcare and from Y about firefighters would lead us to expect Z here.” Read more literature review sections to see how others have described the “gaps” they are filling.

Use Concept Mapping

Concept mapping is a helpful tool for getting your thoughts in order and is particularly helpful when thinking about the “literature” foundational to your particular study. Concept maps are also known as mind maps, which is a delightful way to think about them. Your brain is probably abuzz with competing ideas in the early stages of your research design. Write/draw them on paper, and then try to categorize and move the pieces around into “clusters” that make sense to you. Going back to the gymnasts example, my student might have begun by jotting down random words of interest: gymnasts * sports * coaches * female gymnasts * stress * injury * don’t complain * women in sports * bad coaching * anxiety/stress * careers in sports * pain. She could then have begun clustering these into relational categories (bad coaching, don’t complain culture) and simple “event” categories (injury, stress). This might have led her to think about reviewing literature in these two separate aspects and then literature that put them together. There is no correct way to draw a concept map, as they are wonderfully specific to your mind. There are many examples you can find online.

Ask Yourself, “How Is This Sociology (or Political Science or Public Policy, Etc.)?”

Rubin ( 2021:82 ) offers this suggestion instead of asking yourself the “So what?” question to get you thinking about what bridges there are between your study and the body of research in your particular discipline. This is particularly helpful for thinking about theory. Rubin further suggests that if you are really stumped, ask yourself, “What is the really big question that all [fill in your discipline here] care about?” For sociology, it might be “inequality,” which would then help you think about theories of inequality that might be helpful in framing your study on whatever it is you are studying—OnlyFans? Childcare during COVID? Aging in America? I can think of some interesting ways to frame questions about inequality for any of those topics. You can further narrow it by focusing on particular aspects of inequality (Gender oppression? Racial exclusion? Heteronormativity?). If your discipline is public policy, the big questions there might be, How does policy get enacted, and what makes a policy effective? You can then take whatever your particular policy interest is—tax reform, student debt relief, cap-and-trade regulations—and apply those big questions. Doing so would give you a handle on what is otherwise an intolerably vague subject (e.g., What about student debt relief?).

Sometimes finding you are in new territory means you’ve hit the jackpot, and sometimes it means you’ve traveled out of bounds for your discipline. The jackpot scenario is wonderful. You are doing truly innovative research that is combining multiple literatures or is addressing a new or under-examined phenomenon of interest, and your research has the potential to be groundbreaking. Congrats! But that’s really hard to do, and it might be more likely that you’ve traveled out of bounds, by which I mean, you are no longer in your discipline . It might be that no one has written about this thing—at least within your field— because no one in your field actually cares about this topic . ( Rubin 2021:83 ; emphases added)

Don’t Treat This as a Chore

Don’t treat the literature review as a chore that has to be completed, but see it for what it really is—you are building connections to other researchers out there. You want to represent your discipline or area of study fairly and adequately. Demonstrate humility and your knowledge of previous research. Be part of the conversation.

Supplement: Two More Literature Review Examples

Elites by harvey ( 2011 ).

In the last two decades, there has been a small but growing literature on elites. In part, this has been a result of the resurgence of ethnographic research such as interviews, focus groups, case studies, and participant observation but also because scholars have become increasingly interested in understanding the perspectives and behaviors of leaders in business, politics, and society as a whole. Yet until recently, our understanding of some of the methodological challenges of researching elites has lagged behind our rush to interview them.

There is no clear-cut definition of the term elite, and given its broad understanding across the social sciences, scholars have tended to adopt different approaches. Zuckerman (1972) uses the term ultraelites to describe individuals who hold a significant amount of power within a group that is already considered elite. She argues, for example, that US senators constitute part of the country’s political elite but that among them are the ultraelites: a “subset of particularly powerful or prestigious influentials” (160). She suggests that there is a hierarchy of status within elite groups. McDowell (1998) analyses a broader group of “professional elites” who are employees working at different levels for merchant and investment banks in London. She classifies this group as elite because they are “highly skilled, professionally competent, and class-specific” (2135). Parry (1998:2148) uses the term hybrid elites in the context of the international trade of genetic material because she argues that critical knowledge exists not in traditional institutions “but rather as increasingly informal, hybridised, spatially fragmented, and hence largely ‘invisible,’ networks of elite actors.” Given the undertheorization of the term elite, Smith (2006) recognizes why scholars have shaped their definitions to match their respondents . However, she is rightly critical of the underlying assumption that those who hold professional positions necessarily exert as much influence as initially perceived. Indeed, job titles can entirely misrepresent the role of workers and therefore are by no means an indicator of elite status (Harvey 2010).

Many scholars have used the term elite in a relational sense, defining them either in terms of their social position compared to the researcher or compared to the average person in society (Stephens 2007). The problem with this definition is there is no guarantee that an elite subject will necessarily translate this power and authority in an interview setting. Indeed, Smith (2006) found that on the few occasions she experienced respondents wanting to exert their authority over her, it was not from elites but from relatively less senior workers. Furthermore, although business and political elites often receive extensive media training, they are often scrutinized by television and radio journalists and therefore can also feel threatened in an interview, particularly in contexts that are less straightforward to prepare for such as academic interviews. On several occasions, for instance, I have been asked by elite respondents or their personal assistants what they need to prepare for before the interview, which suggests that they consider the interview as some form of challenge or justification for what they do.

In many cases, it is not necessarily the figureheads or leaders of organizations and institutions who have the greatest claim to elite status but those who hold important social networks, social capital, and strategic positions within social structures because they are better able to exert influence (Burt 1992; Parry 1998; Smith 2005; Woods 1998). An elite status can also change, with people both gaining and losing theirs over time. In addition, it is geographically specific, with people holding elite status in some but not all locations. In short, it is clear that the term elite can mean many things in different contexts, which explains the range of definitions. The purpose here is not to critique these other definitions but rather to highlight the variety of perspectives.

When referring to my research, I define elites as those who occupy senior-management- and board-level positions within organizations. This is a similar scope of definition to Zuckerman’s (1972) but focuses on a level immediately below her ultraelite subjects. My definition is narrower than McDowell’s (1998) because it is clear in the context of my research that these people have significant decision-making influence within and outside of the firm and therefore present a unique challenge to interview. I deliberately use the term elite more broadly when drawing on examples from the theoretical literature in order to compare my experiences with those who have researched similar groups.

”Changing Dispositions among the Upwardly Mobile” by Curl, Lareau, and Wu ( 2018 )

There is growing interest in the role of cultural practices in undergirding the social stratification system. For example, Lamont et al. (2014) critically assess the preoccupation with economic dimensions of social stratification and call for more developed cultural models of the transmission of inequality. The importance of cultural factors in the maintenance of social inequality has also received empirical attention from some younger scholars, including Calarco (2011, 2014) and Streib (2015). Yet questions remain regarding the degree to which economic position is tied to cultural sensibilities and the ways in which these cultural sensibilities are imprinted on the self or are subject to change. Although habitus is a core concept in Bourdieu’s theory of social reproduction, there is limited empirical attention to the precise areas of the habitus that can be subject to change during upward mobility as well as the ramifications of these changes for family life.

In Bourdieu’s (1984) highly influential work on the importance of class-based cultural dispositions, habitus is defined as a “durable system of dispositions” created in childhood. The habitus provides a “matrix of perceptions” that seems natural while also structuring future actions and pathways. In many of his writings, Bourdieu emphasized the durability of cultural tastes and dispositions and did not consider empirically whether these dispositions might be changed or altered throughout one’s life (Swartz 1997). His theoretical work does permit the possibility of upward mobility and transformation, however, through the ability of the habitus to “improvise” or “change” due to “new experiences” (Friedman 2016:131). Researchers have differed in opinion on the durability of the habitus and its ability to change (King 2000). Based on marital conflict in cross-class marriages, for instance, Streib (2015) argues that cultural dispositions of individuals raised in working-class families are deeply embedded and largely unchanging. In a somewhat different vein, Horvat and Davis (2011:152) argue that young adults enrolled in an alternative educational program undergo important shifts in their self-perception, such as “self-esteem” and their “ability to accomplish something of value.” Others argue there is variability in the degree to which habitus changes dependent on life experience and personality (Christodoulou and Spyridakis 2016). Recently, additional studies have investigated the habitus as it intersects with lifestyle through the lens of meaning making (Ambrasat et al. 2016). There is, therefore, ample discussion of class-based cultural practices in self-perception (Horvat and Davis 2011), lifestyle (Ambrasat et al. 2016), and other forms of taste (Andrews 2012; Bourdieu 1984), yet researchers have not sufficiently delineated which aspects of the habitus might change through upward mobility or which specific dimensions of life prompt moments of class-based conflict.

Bourdieu (1999:511; 2004) acknowledged simmering tensions between the durable aspects of habitus and those aspects that have been transformed—that is, a “fractured” or “cleft” habitus. Others have explored these tensions as a “divided” or “fragmented” habitus (Baxter and Britton 2001; Lee and Kramer 2013). Each of these conceptions of the habitus implies that changes in cultural dispositions are possible but come with costs. Exploration of the specific aspects of one’s habitus that can change and generate conflict contributes to this literature.

Scholars have also studied the costs associated with academic success for working-class undergraduates (Hurst 2010; Lee and Kramer 2013; London 1989; Reay 2017; Rondini 2016; Stuber 2011), but we know little about the lasting effects on adults. For instance, Lee and Kramer (2013) point to cross-class tensions as family and friends criticize upwardly mobile individuals for their newly acquired cultural dispositions. Documenting the tension many working-class students experience with their friends and families of origin, they find that the source of their pain or struggle is “shaped not only by their interactions with non-mobile family and friends but also within their own minds, by their own assessments of their social positions, and by how those positions are interpreted by others” (Lee and Kramer 2013:29). Hurst (2010) also explores the experiences of undergraduates who have been academically successful and the costs associated with that success. She finds that decisions about “class allegiance and identity” are required aspects of what it means to “becom[e] educated” (4) and that working-class students deal with these cultural changes differently. Jack (2014, 2016) also argues that there is diversity among lower-income students, which yields varied college experiences. Naming two groups, the “doubly disadvantaged” and the “privileged poor,” he argues that previous experience with “elite environments” (2014:456) prior to college informs students’ ability to take on dominant cultural practices, particularly around engagement, such as help seeking or meeting with professors (2016). These studies shed light on the role college might play as a “lever for mobility” (2016:15) and discuss the pain and difficulty associated with upward mobility among undergraduates, but the studies do not illuminate how these tensions unfold in adulthood. Neither have they sufficiently addressed potential enduring tensions with extended family members as well as the specific nature of the difficulties.

Some scholars point to the positive outcomes upwardly mobile youth (Lehmann 2009) and adults (Stuber 2005) experience when they maintain a different habitus than their newly acquired class position, although, as Jack (2014, 2016) shows, those experiences may vary depending on one’s experience with elite environments in their youth. Researchers have not sufficiently explored the specific aspects of the habitus that upwardly mobile adults change or the conflicts that emerge with family and childhood friends as they reach adulthood and experience colliding social worlds. We contribute to this scholarship with clear examples of self-reported changes to one’s cultural dispositions in three specific areas: “horizons,” food and health, and communication. We link these changes to enduring tension with family members, friends, and colleagues and explore varied responses to this tension based on race.

The Ultimate Guide to Qualitative Research - Part 1: The Basics

qualitative research a literature review

Introduction and overview
What is qualitative research?
What is qualitative data?
Examples of qualitative data
Qualitative vs. quantitative research
Mixed methods
Qualitative research preparation
Theoretical perspective
Theoretical framework
Introduction

What is a literature review?

What does a literature review look like, how can i conduct a literature review, how do i analyze a literature review, how do i write a literature review.

Research question
Conceptual framework
Conceptual vs. theoretical framework
Data collection
Qualitative research methods
Focus groups
Observational research
Case studies
Ethnographical research
Ethical considerations
Confidentiality and privacy
Power dynamics
Reflexivity

Literature reviews

The literature review might be among the least romantic parts of research. However, the importance of understanding and synthesizing relevant literature in helping establish the research question for your study cannot be overstated.

To emphasize the importance of a literature review, writing a literature review, and incorporating literature reviews in research, let's examine its place in scholarly inquiry.

Foundations of scholarly literature

In mainstream research, a researcher shares their research in a paper that appears in a journal article, book chapter, thesis, or dissertation. Novel and interesting research is more likely to be published as peer-reviewed articles .

A researcher needs to be able to write a literature review that persuades scholars that their work is original and important. Put differently, a literature review is how researchers familiarize themselves with current conversation concerning their research topic and demonstrate that they are contributing something new and interesting to this conversation.

What is the purpose of a literature review?

A good literature review can help you critically evaluate current knowledge and key concepts around current knowledge and key concepts within a particular topic. A researcher should provide an overview of relevant research and demonstrate they understand the knowledge surrounding their inquiry to generate new understanding.

Identifying a research gap

Literature reviews can identify which developments occurred and where a gap or puzzle remains that can be informed by further inquiry. For example, current research may assert that a sufficient amount of sleep benefits athletes in sports like basketball and soccer. At the same time, other researchers have reached the same conclusion in tennis and golf. However, if a researcher believes rugby players might have different sleep needs, there is a gap in the literature if no one has conducted a study specific to rugby.

Various gaps can be uncovered in a literature review. Examples include:

A lack of research in a particular context that cannot be explained by existing theory, such as an unexamined geographical area or group of people
A focus on certain theories or methodologies while overlooking others that can explain an unresolved puzzle
An absence of studies after a certain time period or the occurrence of a significant event that challenges existing assumptions or understanding

Summarizing themes and patterns

Literature reviews can help develop theory by synthesizing and building on existing knowledge. This is an approach to knowledge called synthesis. A literature review helps draw connections across various studies and brings insights that individual papers may not be able to provide.

Writing for your audience

To publish in a peer-reviewed journal, you should also consider how reviewers and readers will see your work to ensure your research gets accepted for publication and is read widely by your target audience.

Reviewers for an academic journal want to know if the author of a paper understands the theories and work in his or her field. Thus, a researcher must summarize the key ideas, theories, and knowledge relevant to the theories or knowledge under discussion. A reviewer may not find the author credible if they do not demonstrate a comprehensive understanding of certain developments central to the research area.

Students may also be asked to report on relevant literature to demonstrate knowledge of the field and critical analysis skills. Well-structured paragraphs and sections convey that you understand your topic and that your approach makes sense. These skills are fundamental in a thesis or dissertation. Students are often expected to write an entire dissertation chapter providing a summary of the existing research and different aspects that can inform further research.

ATLAS.ti helps you make sense of research.

Download a free trial of our qualitative data analysis software to see how it can work for you.

The shape of your literature review will depend on your research objectives. A written literature review can either be part of a larger research paper or stand alone as its own study.

A literature review section

A typical, original research paper begins with reviewing the literature to justify the novelty and importance of the inquiry being undertaken. For example, imagine that the author wants to examine the study habits of university students heavily engaged in social media. How does the author know other researchers haven't already explored the topic? And if they did, how can the author contribute new and interesting insights to move understanding further?

A background section in a larger paper should summarize the relevant, existing research to answer these questions. By examining studies on study habits and social media use, the author can demonstrate they have thoroughly surveyed the field to determine if and how further research can meaningfully expand understanding.

A stand-alone literature review

A review can also be a full-length paper. As stand-alone papers, they present a secondary analysis of a series of discrete but related studies in which the researcher provides their own interpretations on the overarching topic.

Remember that writing this type of paper is similar to writing a full paper on your own research project, and peer reviewers will evaluate it as such. A full research paper typically requires a description of the research methodology and the author's specific theoretical approach to analyzing the studies . A clear account of how the author conducted the analysis makes the key findings more credible.

You need not read all of the available theoretical and empirical scholarship to identify recurring central themes or important trends. A review merely samples the most relevant studies to generate key insights after summarizing sources.

Put simply, the researcher needs to collect written studies. There is no minimum number of required studies that constitute a good literature search, as it varies depending on the topic, the research question , and the amount of discussion that already exists on your subject.

In general, relevant sources include academic journals and monographs. Journals are easily searchable via library databases and online search engines like Google Scholar, which may even lead you to downloadable PDFs of full articles. Scholars may also make their papers available on websites like Academia.edu and ResearchGate.

Look for scholars who are well-known in their field. One quick way to identify key experts in your area is to search for highly cited papers and researchers. They can provide fundamental theories and findings critical to your review and link to previous, useful literature.

It's also important to emphasize recent studies in your collection. Especially in the social sciences, developments within the last five years may have opened up new insights or approaches that are important to consider. Later studies may have already challenged or refuted older papers, diminishing their potential contribution to your inquiry.

Other sources of research can include conference proceedings, university bulletins, unpublished theses, and any other publication available for other scholars to read. You may also incorporate records from cultural sources, theoretical discussions in presentations, or newspaper articles.

Keep in mind, however, that scholars take peer-reviewed publications like journal articles more seriously. On the other hand, if you are studying an underdeveloped topic with few published studies, you can consider other information sources to demonstrate a need for further research and discussion on your specific topic.

Organize your data all in one place.

Text files, PDFs, videos, and pictures. ATLAS.ti helps you analyze them all. See how with a free trial.

A good analysis should have a deliberate methodological approach. In other words, a review is more than just a report of relevant research from scholarly articles. A researcher writing a literature review needs to synthesize the existing knowledge and key debates from the collected studies.

Thematic review

Simply put, a thematic literature review aims to identify key themes or concepts that frequently appear in the collected studies. If your inquiry revolves around study tools, for example, what tools appear to be studied more often than others? Perhaps technology like computers and tablets appear often in studies, while newer devices such as virtual reality headsets and other wearable technology are given less focus.

Understanding what topics have already been studied in abundance can help you make an argument about what researchers should explore next. Prominent themes provide a helpful guide for identifying important trends and turning points in research.

As a result, your literature review can benefit from an annotated bibliography , which is a systematic organization of the research papers in your analysis. Many bibliographies include summaries of each study and a checklist of data points (e.g., does the study mention computers, handheld devices, or wearable technology?) to provide a visualization of your analysis.

Quantitative analysis

As mentioned earlier, identifying gaps or unresolved puzzles is a common rationale for writing a literature review. Examples of knowledge gaps include underexplored aspects of theory, lack of discussion in particular contexts, and underutilized research methods.

A quantitative analysis of the literature can help illustrate these gaps. For example, you can argue for a methodological gap among your collected studies. In a set of 40 papers, if 25 involve studies based on interviews and another 10 are based on surveys , you can argue that further observational research is necessary.

Writing a literature review involves discussing your collected studies by describing prominent themes, theoretical or empirical aspects that are missing or underexplored, or a combination of both. Remember that a key goal is to demonstrate your knowledge of the most recent and important developments in the field.

Examine the thematic codes in your project to determine which themes are most apparent and which elements remain underexamined. You can use these code frequencies in your written report to explain the extent or absence of theoretical development in the field (e.g., "Bus transportation is mentioned in only 7% (14 times) of the collected literature, demonstrating a lack of research on what people think about using buses for commuting.").

The goal of the writing process is not to exhaustively detail every study but to describe the most apparent trends or missing information. Select 3-5 themes that arise from the codes and describe each in detail, citing studies from the collected literature as key examples.

From a presentation of these themes, you can make an assertion about the need for novel research inquiry. Using the identified themes to represent a coherent whole, analyze and identify the overlooked areas of research that your own work can address.

Ultimately, most scholarly research is published based on whether that research contributes something novel and interesting to the current conversation around the topic. As a result, a rigorous literature review presents multiple sources of significant studies revolving around the same research topic. It presents a compelling argument about what such studies have yet to explore.

From research question to findings, make it happen with ATLAS.ti.

All the tools you need at every step of the research process. Try ATLAS.ti with a free trial.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

An introductory paragraph that explains what your working topic and thesis is
A forecast of key topics or texts that will appear in the review
Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
Critically Evaluate: Mention the strengths and weaknesses of your sources
Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

Summarize the key findings you have taken from the literature and emphasize their significance
Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
Qualitative versus quantitative research
Empirical versus theoretical scholarship
Divide the research by sociological, historical, or cultural sources
Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Dissertations and research projects

Book a session
Planning your research

Developing a theoretical framework

Reflecting on your position, extended literature reviews, presenting qualitative data.

Quantitative research
Writing up your research project
e-learning and books
SkillsCheck This link opens in a new window
⬅ Back to Skills Centre This link opens in a new window
Review this resource

What is a theoretical framework?

Developing a theoretical framework for your dissertation is one of the key elements of a qualitative research project. Through writing your literature review, you are likely to have identified either a problem that need ‘fixing’ or a gap that your research may begin to fill.

The theoretical framework is your toolbox . In the toolbox are your handy tools: a set of theories, concepts, ideas and hypotheses that you will use to build a solution to the research problem or gap you have identified.

The methodology is the instruction manual: the procedure and steps you have taken, using your chosen tools, to tackle the research problem.

Why do I need a theoretical framework?

Developing a theoretical framework shows that you have thought critically about the different ways to approach your topic, and that you have made a well-reasoned and evidenced decision about which approach will work best. theoretical frameworks are also necessary for solving complex problems or issues from the literature, showing that you have the skills to think creatively and improvise to answer your research questions. they also allow researchers to establish new theories and approaches, that future research may go on to develop., how do i create a theoretical framework for my dissertation.

First, select your tools. You are likely to need a variety of tools in qualitative research – different theories, models or concepts – to help you tackle different parts of your research question.

An overview of what to include in a theoretical framework: theories, models, ideologies, concepts, assumptions and perspectives.

When deciding what tools would be best for the job of answering your research questions or problem, explore what existing research in your area has used. You may find that there is a ‘standard toolbox’ for qualitative research in your field that you can borrow from or apply to your own research.

You will need to justify why your chosen tools are best for the job of answering your research questions, at what stage they are most relevant, and how they relate to each other. Some theories or models will neatly fit together and appear in the toolboxes of other researchers. However, you may wish to incorporate a model or idea that is not typical for your research area – the ‘odd one out’ in your toolbox. If this is the case, make sure you justify and account for why it is useful to you, and look for ways that it can be used in partnership with the other tools you are using.

You should also be honest about limitations, or where you need to improvise (for example, if the ‘right’ tool or approach doesn’t exist in your area).

This video from the Skills Centre includes an overview and example of how you might create a theoretical framework for your dissertation:

How do I choose the 'right' approach?

When designing your framework and choosing what to include, it can often be difficult to know if you’ve chosen the ‘right’ approach for your research questions. One way to check this is to look for consistency between your objectives, the literature in your framework, and your overall ethos for the research. This means ensuring that the literature you have used not only contributes to answering your research objectives, but that you also use theories and models that are true to your beliefs as a researcher.

Reflecting on your values and your overall ambition for the project can be a helpful step in making these decisions, as it can help you to fully connect your methodology and methods to your research aims.

Should I reflect on my position as a researcher?

If you feel your position as a researcher has influenced your choice of methods or procedure in any way, the methodology is a good place to reflect on this. Positionality acknowledges that no researcher is entirely objective: we are all, to some extent, influenced by prior learning, experiences, knowledge, and personal biases. This is particularly true in qualitative research or practice-based research, where the student is acting as a researcher in their own workplace, where they are otherwise considered a practitioner/professional. It's also important to reflect on your positionality if you belong to the same community as your participants where this is the grounds for their involvement in the research (ie. you are a mature student interviewing other mature learners about their experences in higher education).

The following questions can help you to reflect on your positionality and gauge whether this is an important section to include in your dissertation (for some people, this section isn’t necessary or relevant):

How might my personal history influence how I approach the topic?
How am I positioned in relation to this knowledge? Am I being influenced by prior learning or knowledge from outside of this course?
How does my gender/social class/ ethnicity/ culture influence my positioning in relation to this topic?
Do I share any attributes with my participants? Are we part of a s hared community? How might this have influenced our relationship and my role in interviews/observations?
Am I invested in the outcomes on a personal level? Who is this research for and who will feel the benefits?

One option for qualitative projects is to write an extended literature review. This type of project does not require you to collect any new data. Instead, you should focus on synthesising a broad range of literature to offer a new perspective on a research problem or question.

The main difference between an extended literature review and a dissertation where primary data is collected, is in the presentation of the methodology, results and discussion sections. This is because extended literature reviews do not actively involve participants or primary data collection, so there is no need to outline a procedure for data collection (the methodology) or to present and interpret ‘data’ (in the form of interview transcripts, numerical data, observations etc.) You will have much more freedom to decide which sections of the dissertation should be combined, and whether new chapters or sections should be added.

Here is an overview of a common structure for an extended literature review:

A structure for the extended literature review, showing the results divided into multiple themed chapters.

Introduction

Provide background information and context to set the ‘backdrop’ for your project.
Explain the value and relevance of your research in this context. Outline what do you hope to contribute with your dissertation.
Clarify a specific area of focus.
Introduce your research aims (or problem) and objectives.

Literature review

You will need to write a short, overview literature review to introduce the main theories, concepts and key research areas that you will explore in your dissertation. This set of texts – which may be theoretical, research-based, practice-based or policies – form your theoretical framework. In other words, by bringing these texts together in the literature review, you are creating a lens that you can then apply to more focused examples or scenarios in your discussion chapters.

Methodology

As you will not be collecting primary data, your methodology will be quite different from a typical dissertation. You will need to set out the process and procedure you used to find and narrow down your literature. This is also known as a search strategy.

Including your search strategy

A search strategy explains how you have narrowed down your literature to identify key studies and areas of focus. This often takes the form of a search strategy table, included as an appendix at the end of the dissertation. If included, this section takes the place of the traditional 'methodology' section.

If you choose to include a search strategy table, you should also give an overview of your reading process in the main body of the dissertation. Think of this as a chronology of the practical steps you took and your justification for doing so at each stage, such as:

Your key terms, alternatives and synonyms, and any terms that you chose to exclude.
Your choice and combination of databases;
Your inclusion/exclusion criteria, when they were applied and why. This includes filters such as language of publication, date, and country of origin;
You should also explain which terms you combined to form search phrases and your use of Boolean searching (AND, OR, NOT);
Your use of citation searching (selecting articles from the bibliography of a chosen journal article to further your search).
Your use of any search models, such as PICO and SPIDER to help shape your approach.
Search strategy template A simple template for recording your literature searching. This can be included as an appendix to show your search strategy.

The discussion section of an extended literature review is the most flexible in terms of structure. Think of this section as a series of short case studies or ‘windows’ on your research. In this section you will apply the theoretical framework you formed in the literature review – a combination of theories, models and ideas that explain your approach to the topic – to a series of different examples and scenarios. These are usually presented as separate discussion ‘chapters’ in the dissertation, in an order that you feel best fits your argument.

Think about an order for these discussion sections or chapters that helps to tell the story of your research. One common approach is to structure these sections by common themes or concepts that help to draw your sources together. You might also opt for a chronological structure if your dissertation aims to show change or development over time. Another option is to deliberately show where there is a lack of chronology or narrative across your case studies, by ordering them in a fragmentary order! You will be able to reflect upon the structure of these chapters elsewhere in the dissertation, explaining and defending your decision in the methodology and conclusion.

A summary of your key findings – what you have concluded from your research, and how far you have been able to successfully answer your research questions.

Recommendations – for improvements to your own study, for future research in the area, and for your field more widely.
Emphasise your contributions to knowledge and what you have achieved.

Alternative structure

Depending on your research aims, and whether you are working with a case-study type approach (where each section of the dissertation considers a different example or concept through the lens established in your literature review), you might opt for one of the following structures:

Splitting the literature review across different chapters:

This structure allows you to pull apart the traditional literature review, introducing it little by little with each of your themed chapters. This approach works well for dissertations that attempt to show change or difference over time, as the relevant literature for that section or period can be introduced gradually to the reader.

Whichever structure you opt for, remember to explain and justify your approach. A marker will be interested in why you decided on your chosen structure, what it allows you to achieve/brings to the project and what alternatives you considered and rejected in the planning process. Here are some example sentence starters:

In qualitative studies, your results are often presented alongside the discussion, as it is difficult to include this data in a meaningful way without explanation and interpretation. In the dsicussion section, aim to structure your work thematically, moving through the key concepts or ideas that have emerged from your qualitative data. Use extracts from your data collection - interviews, focus groups, observations - to illustrate where these themes are most prominent, and refer back to the sources from your literature review to help draw conclusions.

Here's an example of how your data could be presented in paragraph format in this section:

Example from 'Reporting and discussing your findings ', Monash University .

<< Previous: Planning your research
Next: Quantitative research >>
Last Updated: Apr 5, 2024 11:10 AM
URL: https://libguides.shu.ac.uk/researchprojects

USC Libraries
Research Guides

Organizing Your Social Sciences Research Paper

5. The Literature Review
Purpose of Guide
Design Flaws to Avoid
Independent and Dependent Variables
Glossary of Research Terms
Reading Research Effectively
Narrowing a Topic Idea
Broadening a Topic Idea
Extending the Timeliness of a Topic Idea
Academic Writing Style
Applying Critical Thinking
Choosing a Title
Making an Outline
Paragraph Development
Research Process Video Series
Executive Summary
The C.A.R.S. Model
Background Information
The Research Problem/Question
Theoretical Framework
Citation Tracking
Content Alert Services
Evaluating Sources
Primary Sources
Secondary Sources
Tiertiary Sources
Scholarly vs. Popular Publications
Qualitative Methods
Quantitative Methods
Insiderness
Using Non-Textual Elements
Limitations of the Study
Common Grammar Mistakes
Writing Concisely
Avoiding Plagiarism
Footnotes or Endnotes?
Further Readings
Generative AI and Writing
USC Libraries Tutorials and Other Guides
Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

Give a new interpretation of old material or combine new with old interpretations,
Trace the intellectual progression of the field, including major debates,
Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

Place each work in the context of its contribution to understanding the research problem being studied.
Describe the relationship of each work to the others under consideration.
Identify new ways to interpret prior research.
Reveal any gaps that exist in the literature.
Resolve conflicts amongst seemingly contradictory previous studies.
Identify areas of prior scholarship to prevent duplication of effort.
Point the way in fulfilling a need for additional research.
Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews." Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I. Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
An explanation of how each work is similar to and how it varies from the others,
Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
Persuasiveness -- which of the author's theses are most convincing or least convincing?
Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II. Development of the Literature Review

Four Basic Stages of Writing 1. Problem formulation -- which topic or field is being examined and what are its component issues? 2. Literature search -- finding materials relevant to the subject being explored. 3. Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4. Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1. Roughly how many sources would be appropriate to include? 2. What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3. Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4. Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5. Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III. Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

Current Situation : Information necessary to understand the current topic or focus of the literature review.
Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
Standards : Description of the way in which you present your information.
Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV. Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V. Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

Sources in your literature review do not clearly relate to the research problem;
You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
Does not describe the search procedures that were used in identifying the literature to review;
Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

How are they organizing their ideas?
What methods have they used to study the problem?
What theories have been used to explain, predict, or understand their research problem?
What sources have they cited to support their conclusions?
How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research? Does it forge a new path? Or, does is merely add more of the same thing being said?
Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

<< Previous: Theoretical Framework
Next: Citation Tracking >>
Last Updated: Apr 11, 2024 1:27 PM
URL: https://libguides.usc.edu/writingguide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on June 19, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analyzing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

How does social media shape body image in teenagers?
How do children and adults interpret healthy eating in the UK?
What factors influence employee retention in a large organization?
How is anxiety experienced around the world?
How can teachers integrate social issues into science curriculums?

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, other interesting articles, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography , action research , phenomenological research, and narrative research. They share some similarities, but emphasize different aims and perspectives.

Note that qualitative research is at risk for certain research biases including the Hawthorne effect , observer bias , recall bias , and social desirability bias . While not always totally avoidable, awareness of potential biases as you collect and analyze your data can prevent them from impacting your work too much.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

Academic style
Vague sentences
Style consistency

See an example

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

Observations: recording what you have seen, heard, or encountered in detailed field notes.
Interviews: personally asking people questions in one-on-one conversations.
Focus groups: asking questions and generating discussion among a group of people.
Surveys : distributing questionnaires with open-ended questions.
Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
You take field notes with observations and reflect on your own experiences of the company culture.
You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves “instruments” in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analyzing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

Prepare and organize your data. This may mean transcribing interviews or typing up fieldnotes.
Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorize your data.
Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analyzing qualitative data. Although these methods share similar processes, they emphasize different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Researchers must consider practical and theoretical limitations in analyzing and interpreting their data. Qualitative research suffers from:

Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

Subjectivity

Due to the researcher’s primary role in analyzing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

Limited generalizability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalizable conclusions because the data may be biased and unrepresentative of the wider population .

Labor-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Chi square goodness of fit test
Degrees of freedom
Null hypothesis
Discourse analysis
Control groups
Mixed methods research
Non-probability sampling
Quantitative research
Inclusion and exclusion criteria

Research bias

Rosenthal effect
Implicit bias
Cognitive bias
Selection bias
Negativity bias
Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

There are five common approaches to qualitative research :

Grounded theory involves collecting data in order to develop new theories.
Ethnography involves immersing yourself in a group or organization to understand its culture.
Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
Phenomenological research involves investigating phenomena through people’s lived experiences.
Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

Prepare and organize your data.
Review and explore your data.
Develop a data coding system.
Assign codes to the data.
Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved April 13, 2024, from https://www.scribbr.com/methodology/qualitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, how to do thematic analysis | step-by-step guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Literature Review

Literature review is important because it:

Provides ideas about what should be studied;
Helps us conduct inquires that have not already been done
Connects our research to existing studies

But…doing a literature review is not simply summarizing (or copying) what you think is related and useful to your work. BEING CRITICAL AND CAREFUL IS A MUST !

In reviewing existing literature, you may try to look for gaps in the field and rework your study in a different setting or with different people. Nonetheless, literature review is a continuous sense-making process -- you need to review the literature continuously in order to organize your thoughts and refine your analysis.

A good literature review should be able to:

Connect to your research questions
Connect to your choice of methods and research design
Support your data analysis
Help you draw conclusions and make claims about your research.

Selecting your literature with a purpose

It is impossible to read everything, so when selecting literature for reviewing, consider these:

Is it relevant to your topic/field of study?
Is it a primary source from the researcher(s) or secondary source (e.g. a summary you read in a book about someone’s research)?
Is it updated?

Nature of literatures:

Your literature review can be of different dimensions. Each has its foci and purposes

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care

Affiliations.

1 PhD candidate, School of Nursing and Midwifey, Monash University, and Clinical Nurse Specialist, Adult and Pediatric Intensive Care Unit, Monash Health, Melbourne, Victoria, Australia.
2 Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
3 Senior Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
PMID: 26790142
DOI: 10.1111/wvn.12134

Background: The qualitative systematic review is a rapidly developing area of nursing research. In order to present trustworthy, high-quality recommendations, such reviews should be based on a review protocol to minimize bias and enhance transparency and reproducibility. Although there are a number of resources available to guide researchers in developing a quantitative review protocol, very few resources exist for qualitative reviews.

Aims: To guide researchers through the process of developing a qualitative systematic review protocol, using an example review question.

Methodology: The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction techniques; and data synthesis. The paper highlights important considerations during the protocol development process, and uses a previously developed review question as a working example.

Implications for research: This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings.

Linking evidence to action: Qualitative systematic reviews should be based on well planned, peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in clinical practice. Protocols should outline, in detail, the processes which will be used to undertake the review, including key search terms, inclusion and exclusion criteria, and the methods used for critical appraisal, data extraction and data analysis to facilitate transparency of the review process. Additionally, journals should encourage and support the publication of review protocols, and should require reference to a protocol prior to publication of the review results.

Keywords: guidelines; meta synthesis; qualitative; systematic review protocol.

Evidence-Based Practice / standards*
Information Seeking Behavior
Nursing / methods
Qualitative Research*
Research Design / standards*
Systematic Reviews as Topic*
Writing / standards*

Search Menu
Advance Articles
Editor's Choice
Author Guidelines
Submission Site
Open Access
About Journal of Public Administration Research and Theory
About the Public Management Research Association
Editorial Board
Advertising and Corporate Services
Journals Career Network
Self-Archiving Policy
Dispatch Dates
Journals on Oxford Academic
Books on Oxford Academic

Article Contents

Introduction, key claims in administrative burden research, characteristics of studies on administrative burden, qualitative analysis of key causal relationships, setting an agenda for future research, supplementary material, acknowledgment, data availability.

< Previous

Administrative Burden in Citizen–State Interactions: A Systematic Literature Review

Article contents
Figures & tables
Supplementary Data

Aske Halling, Martin Baekgaard, Administrative Burden in Citizen–State Interactions: A Systematic Literature Review, Journal of Public Administration Research and Theory , Volume 34, Issue 2, April 2024, Pages 180–195, https://doi.org/10.1093/jopart/muad023

Permissions Icon Permissions

Based on a systematic review of 119 articles and working papers, we provide an overview of how administrative burdens in citizen–state interactions have been studied since the inception of the research agenda in 2012. We develop a new and comprehensive model of how key concepts in the framework are related, assess the evidence of the causal relationships proposed by the model, and discuss where more evidence is needed. Empirical research supports conventional claims that burdens are consequential, distributive, and constructed. However, the literature has moved further by (1) demonstrating that factors such as frontline service delivery and government communication influence experiences of burdens; (2) highlighting how factors beyond ideology influence constructions of burdens; (3) introducing the burden tolerance concept; (4) illustrating that experiences of burden influence policymakers’ and members of the publics’ burden tolerance. Based on the review, we propose an agenda for future administrative burden research. We call for studies linking experiences of burden to outcomes such as democratic behavior and take-up, and for studies connecting policymakers’ burden tolerance to actual state actions. Moreover, we argue that future studies should use qualitative methods to further explore the nature of burdens from the perspective of citizens, rely on experimental methods to establish causal links between state actions and experiences of burden, and compare burdens across contexts. Further, empirical studies should examine the tradeoffs between legitimacy and experiences of burden, and how actors outside the citizen–state interaction may influence experiences of administrative burden.

Administrative burden is defined as an individual’s experiences of policy implementation as onerous ( Burden et al. 2012 ). The concept thus emphasizes the experiences of individuals and how state actions, in the form of policies and how they are implemented in practice, influence said experiences ( Baekgaard and Tankink 2022 ). In principle, the definition applies to any individual subject to policy implementation ( Madsen, Mikkelsen, and Moynihan 2022 , 7–8), but the concept has particularly been used in the context of citizen–state interactions ( Jakobsen et al. 2016 ).

Building on research traditions on, among others, take-up of policies and benefits ( Bhargava and Manoli 2015 ; Currie 2006 ), policy feedback ( Moynihan and Soss 2014 ; Soss 1999 ), street-level bureaucracy ( Brodkin and Majmundar 2010 ; Lipsky 1980 ), and red tape ( Bozeman and Youtie 2020 ) that all draw attention to onerous experiences with the state, administrative burden has been showcased as an important concept to create an overarching framework to understand such experiences.

However, we lack a comprehensive overview of how the field has studied administrative burden since the introduction of the concept in the seminal articles by Burden et al. (2012) and Moynihan, Herd, and Harvey (2015) , and how various research questions relate to one another. Even though the standard definition of administrative burden points to individual experiences, scholars in practice refer to different phenomena when studying administrative burden. Some focus on actions made by the state (i.e., “objective” burdens), some focus on individuals’ subjective perceptions, and some focus on individual outcomes, such as take-up of benefits or health ( Baekgaard and Tankink 2022 ). Moreover, research foci differ. Some studies focus on understanding individual experiences and outcomes and how negative experiences and outcomes can be reduced, while others focus on why policies and practices associated with burdensome experiences are enacted by policymakers or how they are implemented at the frontline.

To take stock of the current state of administrative burden research and to better connect empirical knowledge and research questions in current research, we conduct a systematic review of 119 published articles and working papers focusing on administrative burdens in citizen–state interactions. We limit our sample to papers specifically claiming to draw on this framework, that is, studies published between the inception of the concept and framework in Burden et al. (2012) and Moynihan, Herd, and Harvey (2015) and the beginning of 2023. To ensure reproducibility and transparency, we follow the widely used PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines ( Page et al. 2021 ).

There have been a number of theoretical and conceptual articles and literature reviews about administrative burdens ( Baekgaard and Tankink 2022 ; Campbell, Pandey, and Arnesen 2022 ; Madsen, Mikkelsen, and Moynihan 2022 ; Peeters 2020 ). However, none of these articles have systematically covered all studies on the topic or taken up the task of connecting different streams of administrative burden research in a theoretical model. Our systematic review therefore makes two contributions to administrative burden research.

The first contribution is theoretical. Using a qualitative coding of the articles and papers included in our review, we build a theoretical model of how key concepts in the administrative burden causally relate to each other. This model is based partly on theoretical arguments in the literature, partly on empirical evidence, and seeks to connect studies of very different research questions within administrative burden research to create a coherent theoretical framework. The aim is not to make a parsimonious theoretical claim but rather to construct a model of the many antecedents, moderators, and potential consequences of administrative burden experiences identified in this literature.

The second contribution is an overview of how administrative burdens in citizen–state interactions have been studied to date. We describe the methodological and contextual characteristics of the studies included in the review and connect key concepts in the administrative burden framework to identify areas of inquiry where substantial progress has been made and to point to areas where future studies could best be directed.

The next section briefly discusses the concept of administrative burden and key causal claims in the administrative burden literature. In the methods section, we describe our literature search, criteria for including studies in the review, and how studies were coded and analyzed. This is followed by our qualitative analysis of the literature. We start out by presenting a model based on the review and then classify the evidence for seven causal claims in the model. The article concludes with a discussion of limitations and proposals for a future research agenda.

Administrative burden refers to the learning, compliance, and psychological costs experienced by citizens when interacting with the state ( Herd and Moynihan 2018 ). Learning costs are the costs of learning about rights, rules, and demands associated with interacting with the state ( Barnes and Riel 2022 ). For instance, an individual eligible for the TANF program in the United States has to be aware that the program exists and how to apply for the benefits. Compliance costs are the costs of complying with specific rules in interactions with the state. In the TANF example, the applicant has to fill out an application form and demonstrate eligibility. For the unemployed, compliance costs may manifest as the costs of having to show up for meetings at public offices to demonstrate an active search for work and of updating CVs on a regular basis ( Baekgaard et al. 2021 ; Madsen nda ). Finally, psychological costs have to do with the mental discomfort of interacting with the government ( Hattke, Hensel, and Kalucza 2020 ). For instance, interactions associated with uncertainty may lead to experiences of stress, loss of autonomy, or even stigma ( Cecchini nd ).

By emphasizing the subjective costs experienced by citizens and explicitly referring to individual experiences in the definition of the concept ( Burden et al. 2012 ), the administrative burden literature makes a key distinction between what the state does (sometimes called barriers, frictions, state actions, or state constructions of burdens) and what the individual experiences. However, this does not mean that the literature is only interested in the impact of experiences. Rather, the literature makes three key claims about burdens ( Herd and Moynihan 2018 ; Moynihan, Herd, and Harvey 2015 ) regarding what the state does, how citizens experience the actions of the state, individual differences in said experiences, and the consequences of burdensome experiences. Figure 1 summarizes the claims in a simplified model (see Baekgaard and Tankink 2022 ; Christensen et al. 2020 ; Herd and Moynihan 2018 for similar models).

Key Claims in the Administrative Burden Literature.

First, burdens are consequential. The impact of burdens is likely to extend beyond people’s experiences and influence outcomes such as civic and electoral participation, health, and take-up of benefits. This claim is rooted very much in literatures on policy feedback, benefit take-up, and applied economics. These research traditions empirically demonstrate that aspects of what the state does may have important impacts for the mobilization of citizens (e.g., Bruch, Ferree, and Soss 2010 ; Soss 1999 ), the extent to which target groups take up services and benefits for which they are eligible (e.g., Currie 2006 ), and the long-run health of citizens enrolled in welfare programs (e.g., Hoynes, Schanzenbach, and Almond 2016 ). However, they do less to explore people’s subjective experiences of burden. In this respect, the administrative burden framework contributes to previous streams of research by creating a language for the mechanisms linking state actions to outcomes.

Second, burdens have distributive consequences and are likely to fall harder on those with fewer resources in the form of human and administrative capital ( Christensen et al. 2020 ; Masood and Nisar 2021 ), administrative literacy ( Döring 2021 ), and bureaucratic self-efficacy ( Bisgaard 2023 ). Third, while some burdensome state actions are likely unintended ( Peeters and Widlak 2023 ), or the result of unconscious biases ( Olsen, Kyhse-Andersen, and Moynihan 2020 ), other state actions are constructed and the result of deliberate political and administrative decisions, that is, politicians or bureaucrats prefer to introduce burdens to, for instance, limit fraud in public programs ( Moynihan, Herd, and Ribgy 2016 ).

Thus, the literature relies on a broad understanding of the relevant domain of inquiry. This domain is not limited to the study of experiences and outcomes among citizens and target groups. As per the third causal claim above, it also encompasses decisions and rationales for decisions made by elected politicians, administrators, and frontline personnel. In our review, we therefore rely on a broad understanding of the domain of administrative burden in citizen–state interactions where studies are relevant whenever the subject matter has to do with a state arrangement introducing burden for citizens. The studies may focus either on decisions made by politicians or bureaucrats or on the consequences of such decisions for citizens. In our analysis of the literature, we seek to develop the model presented in figure 1 further by reviewing the empirical findings in existing studies.

We adhere to the PRISMA guidelines when conducting our systematic literature review ( Page et al. 2021 ). These guidelines were developed to ensure that literature reviews are comprehensive, transparent, and well documented to minimize reporting biases and ensure reproducibility. The PRISMA checklist is available in the appendix . Below we describe the eligibility criteria for inclusion in the review as well as our search and coding strategy.

Eligibility Criteria

Our focus is on administrative burdens in citizen–state interactions. The main inclusion criterion is that studies use the conceptual framework formulated by Burden et al. (2012) , Moynihan, Herd, and Harvey (2015) , and Herd and Moynihan (2018) , that is, refer explicitly to administrative burden and/or learning, compliance, and psychological costs. Other streams of literature in economics, sociology, political science, and public administration also deal with frictions in interactions between citizens and government. This includes, but is not limited to, literatures on red tape, sludge, ordeals, take-up of government benefits, street-level bureaucracy, and policy feedback ( Baekgaard and Tankink 2022 ; Madsen, Mikkelsen, and Moynihan 2022 ). However, as administrative burden has developed into a sizeable subfield of its own, which in several aspects differs from related research in other disciplines ( Madsen, Mikkelsen, and Moynihan 2022 ), it is important to take stock of the current state of this particular field and explore what the literature has taught us so far.

The second inclusion criterion is that studies focus on administrative burdens in citizen–state interactions. This means that we exclude studies that use the administrative burden framework but focus either on companies ( Petersen, Hansen, and Houlberg 2022 ), third-party organizations ( Carey et al. 2020 ), or on the costs experienced by public employees in their interactions with the organization where they are employed ( Bozeman and Youtie 2020 ; Linos and Riesch 2020 ; Sievert, Vogel, and Feeney 2020 ). We make this decision because our goal is to understand how, why, and when citizens experience burdens in their interactions with the state. In comparison, studies on public employees burdened by work routines focus on internal organizational affairs rather than a bureaucratic relationship between the state and individual outside the formal organizational hierarchy. Also, burdens among public employees have been studied extensively in the red tape literature ( George et al. 2020 ). Nevertheless, the review still includes studies where elected politicians and frontline personnel were asked about the imposition of burdens on individuals outside the organization. Thus, the review applies a broad understanding of citizens as individuals and organizations outside the formal organizational hierarchy of the state in a given case.

The remaining inclusion criteria are more straightforward. We are interested in all English-language peer-reviewed publications and working papers from 2012 until our data collection closed in February 2023. 1 We set the start to 2012, because this is when Burden et al. (2012) wrote their seminal article that introduced and defined the term “administrative burdens.” Table 1 gives an overview of the eligibility criteria.

Overview of Eligibility Criteria

Literature Search

To identify peer-reviewed journal articles, we searched all journals in the Social Sciences Citation Index using Web of Science. We searched titles, abstracts, and keywords for “administrative burden,” “psychological cost,” “compliance cost,” “learning cost,” and derivatives of these terms. We limited our search to English-language articles. We also searched 12 leading public administration journals (see list of journals in appendix table A1 ) using the same terms. We then screened titles and abstracts and then full papers to identify all papers that passed our eligibility criteria. Finally, we screened the literature list of all eligible journal articles for missing records. In total, we identified 100 peer-reviewed journal articles for the systematic literature review.

To obtain a comprehensive pool of working papers, we created a list of all authors who contributed at least two articles to the literature review (see appendix table A2 ). We then contacted all authors on the list and asked them to provide any unpublished, full-length papers on administrative burdens that they had (co-)authored. We also encouraged them to let us know if they knew of other working papers on the topic. Almost all authors replied within a few days and most sent one or more working papers. Moreover, we made a call for working papers through our Twitter accounts and a similar call through a listserv for scholars interested in administrative burden research managed by Professor Donald Moynihan. Based on these steps, we collected 19 eligible working papers. 2 In total, 119 papers are included in the review (see the full list of papers in appendix table A7 ). Figure 2 summarizes the selection process.

Selection Process.

Coding Strategy

We relied on two strategies for coding the articles. First, we systematically coded several facts about the articles (year of publication, whether empirical material was collected, methods used, country covered by empirical analysis, policy area, and type of subjects) using a closed coding strategy (see appendix table A3 for a full description of coding criteria). We present this information in the first part of the results section to give an overview of the field and the types of studies conducted.

Second, we used an open coding where we focused on core concepts covered in the articles and types of causal relations covered in the papers. This is a demanding task that requires that coders have in-depth knowledge of the literature. We therefore handled all coding ourselves and met several times during the coding process to ensure consistency in the categorization of relationships and concepts. We use the qualitative coding to summarize current knowledge about the different relationships shown in figure 1 and to extend the causal model based on the findings and arguments in extant research.

Citation Analysis

This first part of the analysis covers key characteristics of the articles on administrative burdens in citizen–state interactions. Related to the discussion of eligibility criteria, we initially explore whether studies frequently cited by our 119 eligible studies are missing in the review. Table 2 shows that among the top 10 most cited papers and publications in the review, three publications do not rely on the administrative burden framework and therefore do not meet the eligibility criteria. Two of these publications ( Brodkin and Majmundar 2010 ; Lipsky 1980 ) concern street-level bureaucracy, and the third ( Bhargava and Manoli 2015 ) focuses on take-up of benefits. Thus, while there certainly are some widely cited works outside the narrow domain of administrative burden research, the field is generally dominated by internal references, suggesting that administrative burden research indeed constitutes a distinct field of its own.

Top 10 Most Cited Publications by the 119 Papers Included in the Systematic Review

A related question is how well studies with different foci, research questions, and methodologies speak to one another. We conducted a bibliographical network analysis ( Perianes-Rodriguez, Waltman, and van Eck 2016 ) in which we explored citation patterns between articles. As shown in appendix table A4 , assortativity scores are generally low, suggesting that articles tend to cite each other to an almost equal extent despite different methodologies and research questions ( Newman 2003 ). Overall, the analysis suggests that the field is coherent in the sense that even the most different parts of the field tend to rely on each other’s work.

Methodological Characteristics

Of the 119 collected articles, 75% are empirical papers using qualitative or quantitative analysis of data, while 25% are theoretical papers, literature reviews, or case studies. Articles are published in 35 different journals. Most are published in public administration journals, but some are published in either health, economics, or political science journals. The most frequent appearances are in Public Administration Review with 17, Journal of Public Administration Research and Theory with 16, and Journal of Behavioral Public Administration with 11 articles (see appendix table A5 for full details). Figure 3 shows a timeline of all published papers on the topic. Only nine were published between 2012 and 2017, but the publication trend changed significantly in 2018. From 2018 to 2021, the number of yearly published papers almost doubled each year from 5 in 2018 to 36 in 2021. While 2022 saw a decline in publications to 20, the overall trend still indicates that the study of administrative burden has established itself as a sizeable subfield within public administration research.

Publication Timeline.

Note: n = 100. The figure shows the year studies were made available online and does not include working papers.

Figure 4 graphs methodological characteristics of the studies. Panel A shows that more than half the empirical studies use quantitative methods. However, a substantial number of articles employ qualitative methods or case studies, meaning the field is characterized by some methodological diversity. This is also evident from panel D, where we divide the quantitative and qualitative categories into more specific subcategories. We see that studies on administrative burdens use a great variety of methods, and that studies utilize both observational and experimental data to a high extent. Studies are also relatively diverse when it comes to the origin of data, as our review includes studies from all six inhabited continents. However, studies from Western countries dominate the literature, as 82% of all studies were conducted in either the United States, Europe, or Australia (see panel B). We also coded whether papers used data from more than one country. Only three papers used data do so, and neither of them used a comparative approach where they compared burdens across contexts. Panel D shows that almost half of the studies focus on target group members. This aligns well with the fact that one purpose of the administrative burden framework is to draw attention to individuals’ experiences of policy implementation ( Moynihan, Herd, and Harvey 2015 ). Finally, panel E shows that around 50% of all studies focused on means-tested welfare benefits. This may reflect that means-tested programs are often where citizens encounter the most requirements and therefore are likely to experience various burdens when interacting with the state.

Methodological Characteristics of Empirical Studies.

Note: Figures A-E display various charatersitics of empirical studies. Articles that fit into more than one category are coded into all relevant categories. Purely theoretical articles are not included in any of the figures.

This section presents the results of our qualitative analysis of the literature. Figure 5 provides an overview of our main findings. This model extends the theoretical model in figure 1 in four important respects. First, it proposes a more nuanced understanding of what state actions are. In line with Baekgaard and Tankink (2022 , 17), we understand state actions broadly to cover what the state does “including laws, rules, requirements, and how such are implemented by public officials and street-level bureaucrats.” This leads us to distinguish between formal (arrow 1) and informal policy designs (arrow 2). While formal policy design refers to the laws and rules enacted by politicians, that is, the rules that people will have to abide to get access to services and benefits, informal policy design concerns how these rules are implemented at the frontline and communicated more broadly. This allows us to discuss how different aspects of policies lead to experiences of administrative burdens. Second, the model extends the number of factors explaining state actions beyond political ideology by introducing the concepts of burden support and burden tolerance, that is, “the willingness of policymakers and people more generally to passively allow or actively impose state actions that result in others experiencing administrative burdens” ( Baekgaard, Moynihan, and Thomsen 2021 , 184). As shown, support and tolerance for burdens may sometimes be influenced by the content of state actions when people become aware of actual rules and implementation (arrow 6b).

Extended Model of Causal Claims.

Third, the model proposes that other factors than political ideology and beliefs may influence burden tolerance and state actions. In particular, the model highlights the importance of target group deservingness, personal experience, and bureaucratic processes (arrow 7). Fourth, the model proposes feedback effects of citizens’ experiences of burden on how burdens are constructed by the state and how tolerant policymakers and others are of burdens to begin with (arrows 5a and 5b).

Table 3 lists the number of studies that cover each relationship. Below, we discuss each of the seven arrows in figure 5 . Our aim is not to mention all studies discussing each specific arrow but rather to summarize current knowledge about each relationship. Our discussion therefore only covers selected articles that provide knowledge on the relationship under discussion. Appendix table A6 is an extended version of table 3 and shows the articles that provide knowledge on each relationship.

Number of Papers Studying Each Causal Relationship

Arrow 1: Formal Policy Design → Experiences of Burden

With few exceptions, studies find that state barriers are associated with experiences of learning and compliance costs. Learning costs, for instance, arise when being subject to requirements ( Cook 2021 ), misinformation ( Chudnovsky and Peeters 2021a ), and having to deal with vouchers ( Barnes 2021 ), while compliance costs arise because of transportation time to vaccinator camps ( Ali and Altaf 2021 ) and completing forms ( Yates et al. 2022 ). Some studies find that learning and compliance costs arise as a consequence of (eligibility) requirements in means-tested welfare programs ( Holler and Tarshish 2022 ) and insurance programs ( Yates et al. 2022 ). Other studies find that learning and compliance costs also arise in settings such as the restoration of voting rights ( Selin 2019 ), digital government services ( Madsen, Lindgren, and Melin 2022 ), and accessing vaccinations ( Ali and Altaf 2021 ).

Studies are conducted in diverse contexts such as Pakistan, Denmark, the United States, and Argentina, suggesting there is some universality to the claim that interacting with the state is associated with experiences of learning and compliance costs. However, one paper finds that having a scheduled compulsory meeting with frontline workers causes no changes in compliance costs and is associated with experiences of less learning costs ( Baekgaard and Madsen 2023 ). Another study finds that digital self-service solutions have the potential to both increase and reduce learning and compliance costs ( Madsen, Lindgren, and Melin 2022 ).

This suggests that more research is needed on how different types of state actions reduce and impose experiences of learning and compliance costs. Such studies could build on more qualitative approaches to obtain a better understanding of the mechanisms linking state actions to experiences. Also, when it comes to understanding the costs of dealing with different state actions, qualitative methods have major advantages over other methods. With a few exceptions ( Ali and Altaf 2021 ; Baekgaard and Madsen 2023 ), most papers indeed use qualitative methods to study the relationship between barriers and learning and compliance costs, while no papers use experimental methods. This is not surprising, as it is often hard to manipulate barriers or state actions. However, in addition to more qualitative research, the literature would benefit from studies that are able to causally link state actions to experiences of learning and compliance costs. As mentioned in the next section, a few studies document how state actions causally influence experiences of psychological costs, showing that it is possible to causally study the link between state actions and experiences of administrative burdens.

There are 50% more studies on the relationship between formal policy designs and psychological costs than on the comparable relationship with learning and compliance costs discussed above, illustrating that this relationship has received high scholarly attention. The general finding from the 16 studies discussing this topic is that state actions are associated with various forms of psychological costs. Examples of psychological costs arising from state actions are autonomy loss and stress ( Baekgaard et al. 2021 ), frustration ( Cook 2021 ), stigma ( Selin 2019 ; Thomsen, Baekgaard, and Jensen 2020 ), externalization of locus of control ( Madsen and Mikkelsen 2022 ), uncertainty ( Cecchini nd ) and confusion, anger, and frustration ( Hattke, Hensel, and Kalucza 2020 ).

Studies fall in two methodological categories: qualitative studies and experiments. Qualitative studies provide in-depth knowledge about how state actions may lead to psychological costs. One example is Yates et al.’s (2022) study of burdens in Australia’s National Disability Insurance Scheme. One interviewee mentions that it was “wearing” and “soul destroying” “to be constantly questioned about, are you disabled enough” (p. 5), showing how eligibility requirements can create psychological costs.

Experimental studies establish causal links between barriers and costs. Baekgaard et al. (2021) use survey- and field-experimental evidence to show that reductions in state compliance demands reduce stress and increase the sense of autonomy among target group members. Hattke, Hensel, and Kalucza (2020) and Hattke et al. (nd) rely on laboratory experiments to show how redundant documentation requirements and simple administrative processes can cause confusion, frustration, and anger.

In general, the link between state actions and psychological costs is relatively well covered in the literature. However, studies so far have generally examined only one or a few state actions. There is a lack of studies that compare effects of different actions on psychological costs. Such studies could provide valuable knowledge on which state actions translate into psychological costs.

Arrow 2: Informal Policy Design → Experiences of Burden

Informal policy design has to do with the actions by the state that do not directly refer to the formal rules and requirements as decided by policymakers but rather how these are processed and communicated to citizens. Two aspects of informal policy design are particularly prevalent in research on administrative burden: frontline service delivery and government communication.

Frontline Service Delivery

It is no surprise that the delivery of services at the frontline of public organizations matters for experiences of burden. Lipsky (1980) alluded to this, and subsequent work has explored this question without explicitly using the concept of administrative burden (e.g., Brodkin and Majmundar 2010 ; Soss, Fording, and Schram 2011 ). Studies applying the administrative burden framework show that workload matters for experiences of administrative burden. For instance, Bell and Meyer (nd) use administrative data from college financial aid programs to show that decreases in workload lead to an increase in program access for low-income students and that the increase is highest among students who have been subject to discrimination based on their race. Ali and Altaf (2021) show that citizens experience more burdens in areas with lower administrative capacity, while others find that stress and burnout ( Mikkelsen, Madsen, and Baekgaard 2023 ) and red tape ( Madsen ndb ) among frontline workers are associated with experiences of burden among their clients.

The behavior of frontline workers also matters for citizens’ experiences. Bell and Smith (2022) show that frontline workers who adopt a support role rather than a role as “compliance officer” are more likely to use their discretionary power to help students overcome administrative burdens. In a similar vein, Halling’s (nd) results suggest that frontline workers help citizens overcome burdens by circumventing rules. Finally, Barnes and Henly’s (2018) qualitative analysis shows that clients tend to blame their experiences of administrative burden on frontline employees.

Government Communication

Another part of informal policy design that has received considerable attention is how communication from the state affects individuals’ experiences of administrative burden. All these papers rely on field experiments with randomized exposure to different forms of government communication. Linos et al. (2022) show that disadvantaged groups prefer postcards over a telephone hotline to seek information about free dental care. They use focus groups to show that this is likely explained by lower psychological costs associated with postcards as participants fear uncomfortable interactions with bureaucrats. Moynihan et al. (2022) show how the framing of state categories matters for selecting into the right categories and that a more intuitive presentation of information increased the number of claimants providing adequate documentation. Simplified communication ( Linos, Reddy, and Rothstein 2022 ), destigmatizing language ( Lasky-Fink and Linos 2023 ), early communication ( Linos, Quan, and Kirkman 2020 ), postcards ( Hock et al. 2021 ), letters ( Bhanot 2021 ), and text messages ( Lopoo, Heflin, and Boskovski 2020 ) can also improve take-up.

Altogether, these field experiments show that different forms of nudges can be effective in increasing take-up of benefits among eligible individuals. Apart from the two first-mentioned studies, the studies do not measure experiences of burden directly. Instead, they measure different outcomes while theorizing that the link between communication and outcomes has to do with experiences of burden. Hence, there is a need for studies that show that reduction of administrative burdens is the process through which these nudges work.

Arrow 3: Distributive Effects

The argument that administrative burdens are distributive and can foster inequality is at the core of the administrative burden framework ( Christensen et al. 2020 ; Herd and Moynihan 2018 ). Thirty-one papers contribute knowledge on the distributional consequences of state actions. Differences in resources, attitudes, and expectations between citizens constitute one main type of distributive effects identified in the literature ( Christensen et al. 2020 ; Heinrich 2018 ; Nisar 2018 ). The other type, which has received less attention, focuses on how characteristics of the state may contribute to different experiences of burden among different parts of the population ( Griffiths 2021 ; Peeters, Renteria, and Cejudo nd ). We discuss both types next.

Citizen Factors

Studies show that possessing administrative literacy ( Döring 2021 ; Döring and Madsen 2022 ), self-efficacy ( Thomsen, Baekgaard, and Jensen 2020 ), habitus and different forms of capital ( Carey, Malbon, and Blackwell 2021 ; Masood and Nisar 2021 ) all make state barriers easier to handle, resulting in fewer experiences of burdens. All these contributions are important in documenting that possessing the necessary capital and skills is key when dealing with onerous state demands.

However, there is a considerable overlap between the different concepts. Apart from self-efficacy, all focus on a type of capital (or literacy) that makes state encounters easier to handle. Some are specific to encounters with the state (administrative literacy and capital), while others are more general forms of capital (human capital and Bourdieu’s capital concepts). Discussing differences and similarities between the concepts is beyond the scope of this article, but we note that using fewer concepts would strengthen the comparative potential across studies.

Other studies focus on how experiences of burdens are distributed across demographic and non-demographic characteristics. The general finding is that individuals from marginalized or low-resource groups tend to struggle more with state barriers. So far, studies have shown that individuals with low income or who are experiencing scarce financial resources ( Chudnovsky and Peeters 2021b ; Heinrich et al. 2022 ; Larsson 2021 ; Madsen, Baekgaard and Kvist 2022 ), ethnic minorities ( Heinrich 2018 ; Olsen, Kyhse-Andersen, and Moynihan 2020 ), women ( Kyle and Frakt 2021 ; Yates et al. 2022 ), individuals with low or no education ( Chudnovsky and Peeters 2021b ; Collie et al. 2021 ; Kyle and Frakt 2021 ), and those suffering from sickness and disabilities ( Bell et al. 2022 ; Collie et al. 2021 ; Kyle and Frakt 2021 ) experience more administrative burdens as a result of state actions.

Relatedly, a few studies discuss how citizens’ attitudes and expectations might influence how citizens engage with the state and hence lead to different impacts of state actions on experiences of burden. These attitudes and expectations may themselves stem from a variety of sources including prior interactions with the state ( Chudnovsky and Peters 2021b , 531), thus suggesting a potential feedback effect from outcomes on attitudes and expectations (see also Moynihan and Soss 2014 ). 3

Finally, a last stream of studies considers how individuals’ access to relevant third parties, actors outside the citizen–state interaction that provide help to citizens or otherwise influence interactions ( Moynihan, Herd, and Harvey 2015 ), may affect their experiences of administrative burden. A few papers explore the role of such actors. Barnes (2021, nd ) shows that retailers play a crucial role in shaping compliance costs in voucher programs such as WIC. Because citizens must redeem their vouchers in retail stores, retailers play a huge role in shaping how easy redemption is. Concrete examples are the degree to which eligible food is marked and displayed and whether store personnel are trained in handling vouchers. NGOs may also contribute to reduced learning and compliance costs by helping citizens overcome burdens ( Nisar 2018 ; Nisar and Masood nd ). Finally, (ex-)family members may influence experiences of administrative burden ( Nisar 2018 ). Cook (2021) illustrates how ex-partners may directly impose burdens on mothers in the child support benefit system in Australia. As an example, some fathers limit their child support liabilities or claim that they have already provided payments to mothers. Each time fathers make such changes or claims, mothers are required to respond, which can be associated with substantial compliance costs.

State Characteristics

Another possible source of distributive effects is the state itself. A key insight from this stream of research is that variations in administrative capacities to reach out to vulnerable populations may contribute to inequality in the experience of burdens. Some studies investigate how individuals may experience different burdens in states with different characteristics. The most prominent characteristic examined so far is the extent to which the state is automated and digitalized. Peeters, Renteria, and Cejudo (nd) illustrate how governments with higher information capacity are better able to “absorb” burdens, which means that citizens face fewer administrative burdens. Digital government may also create unintentional errors that contribute to considerable experiences of administrative burden. Griffiths (2021) shows how automation of benefit calculation can create burdensome experiences. For example, people with irregular pay dates risk missing out on benefits for which they are eligible because automation processes do not account for irregular cases. Likewise, Widlak and Peeters (2020) show that citizens face various administrative burdens in correcting errors made by the state, while Compton et al. (2022) show that blacks and Hispanics are disproportionally hit by administrative errors.

Other state characteristics that may influence experiences of administrative burdens are material and artificial artifacts present in physical and virtual government arenas ( Nisar and Masood nd ) and consistent application of rules ( Kaufmann, Ingrams, and Jacobs 2021 ). Finally, Johnson and Kroll (2021) theorize but find no supporting empirical evidence that representative government and shared identities between frontline employees and citizens may decrease experiences of burden.

Arrow 4: Experiences of Burden → Outcomes

According to Moynihan, Herd, and Harvey (2015) , administrative burdens are an important part of governance, “since they affect whether citizens succeed in accessing services (did I get what I want), whether public polices succeed (did a program reach the targeted group?), and the perceptions of government (was I treated fairly and with respect?)” (p. 43). However, despite the obvious importance of studying the link between experiences of burden and various outcomes, only Daigneault and Macé (2020) have done so among published papers. Based on interviews with target group members, they show that individuals experiencing compliance and learning costs are less likely to take up Quebec’s Supplement to the Work Premium program. Other papers study the link between state actions and outcomes but without subjective measures of people’s experiences of administrative burden. Notable examples are Heinrich (2016) and Jenkins and Nguyen (2022) , who convincingly, and with strong causal traction, show that various state actions influence take-up of welfare programs and might even impact long-term outcomes such as risky behaviors in adolescence ( Heinrich 2016 ; Heinrich and Brill 2015 ). These studies contribute important knowledge on how state actions influence take-up of welfare benefits but not on the relationship between subjective experiences of burden and outcomes.

Several working papers show that experiences of burden are associated with behaviors that can lead to reduced program take-up, such as compliance and autonomous motivation ( Madsen nda ), making errors on forms ( Hattke et al. nd ), and filing complaints ( Bell et al. 2022 ). While these papers make valuable contributions, none of them study actual outcomes but rather behaviors that are likely to influence take-up of benefits. The final working paper by Lasky-Fink and Linos (2023) offers a promising approach to dealing with some of the shortcomings of other research on this relationship. Contrary to the other working papers, the authors study actual take-up of welfare benefits and show that destigmatized language leads to substantially higher take-up rates. Moreover, contrary to studies linking state actions and take-up, the authors go one step further and use three survey experiments to make it probable that the mechanism linking state actions and take-up is psychological costs in the form of perceived stigma. In doing so, the working paper studies the whole causal chain from barriers over subjective experiences of administrative burdens to outcomes. This is a model for future studies to pursue because such studies will be able to show not only whether individuals experience burdens as a result of state actions, but also the extent to which these burdens subsequently influence service use or other relevant outcomes.

There is also a lack of studies that look beyond take-up and focus on other types of outcomes. In some instances, burdens may not discourage people from taking up public services, but they may still affect the adequacy and quality of services provided—in particular when citizens interact with the same public agency for a prolonged period of time ( Peeters and Campos 2021 ). Furthermore, inspired by the policy feedback literature, it has been suggested that experiences of burden may affect civic capacities such as political efficacy, trust in institutions, and civic engagement ( Christensen et al. 2020 ). However, no studies have so far examined these questions systematically.

Arrow 5: Feedback Effects: Experiences → Burden Tolerance and State Actions

While state actions are expected to trigger experiences of burdens in the original theoretical model, a few studies suggest a feedback effect, that is, experiences may influence burden tolerance and state actions. The argument is that knowledge about experiences may make policymakers and others understand the detrimental effects of state actions and hence induce less burden. This proposition finds mixed support in the three studies dealing with the question. In a survey-experimental study of Danish local politicians using a treatment cue about psychological costs experienced by target group members, Baekgaard, Moynihan, and Thomsen (2021) find no evidence of a feedback effect. Conversely, in a survey experiment, Halling and Petersen (nd) find that Danish frontline employees are more likely to reduce compliance demands in the implementation process and to help citizens who communicate psychological costs. Sievert and Bruder (2023) find mixed support in their study of the feedback effects of treatments increasing awareness of learning and compliance, costs among German citizens. While there is some evidence of feedback effects of compliance costs, exposing participants to information about learning costs does not affect burden tolerance. Finally, Gilad and Assouline (2022) do not study feedback effects directly, but rather a prerequisite of their existence, namely citizens voicing their experiences of burden. They find that citizens indeed voice their experiences to authorities but also that disadvantaged groups are less inclined to do so.

On balance, there is a need for much more research to establish the relevance of feedback effects. Such studies could investigate differences between groups of respondents (policymakers, frontline workers, citizens). They may also focus on the way in which information about experiences of burden is provided. Here, a distinction could be made between statistical and episodic information. Previous research has identified stronger effects of episodic data in other contexts ( Olsen 2017 ). Finally, studies could examine feedback effects from citizen outcomes.

Arrow 6: The Relationship Between Burden Tolerance and State Actions

The literature on burden tolerance presumes that tolerance among political decision-makers and the mass public influences the extent to which the state constructs burdens (e.g., Aarøe et al. 2021 ; Baekgaard, Moynihan, and Thomsen 2021 ; Keiser and Miller 2020 ; Nicholson-Crotty, Miller, and Keiser 2021 ). However, none of the studies in the review study the causal influence of burden tolerance on state actions, likely due to challenges obtaining causal estimates. Nevertheless, we indicate this relationship in figure 5 with a dashed line (arrow 6a) due to the strong theoretical expectation that burden tolerance influences the extent to which the state introduces burdens in public policies.

Alternatively, it is possible that knowledge about existing barriers influences the extent to which people are supportive of burdensome barriers (arrow 6b). Two empirical studies examine this question using survey experiments among the mass public. Keiser and Miller (2020) find that information about the presence of barriers increases support for welfare programs and their recipients, in particular among conservative voters. Nicholson-Crotty, Miller, and Keiser (2021) show that information about barriers has heterogeneous effects on program approval depending on whether the target group is perceived as deserving (information about more barriers reduces approval) or undeserving (information about barriers has no significant effect). While the two studies support the idea that information about state actions may influence burden tolerance, there is certainly room for more research about how state actions may influence burden tolerance in the mass public and among decision-makers. Such studies may for instance investigate how state actions are constructed in popular debates.

Arrow 7: Factors Shaping Burden Tolerance and State Actions

This section looks into other factors that shape burden tolerance and state actions. A total of seven studies examine factors shaping burden support, while 13 studies investigate factors shaping state actions. We deal with the questions jointly, because many of the key explanations are similar for burden tolerance and state actions. Overall, explanations can be divided into four broad categories.

First, a series of studies present evidence that burdens are constructed and that political ideological beliefs influence the extent to which barriers are introduced. For instance, the studies by Moynihan, Herd, and Harvey (2015) , Moynihan, Herd, and Ribgy (2016) , and Heinrich (2018) find that more barriers are introduced in states governed by conservatives than in states governed by liberals. Likewise, a series of cross-sectional studies find strong correlations between the ideological beliefs of politicians ( Baekgaard, Moynihan, and Thomsen 2021 ), street-level bureaucrats ( Bell et al. 2020 ), and the mass public ( Haeder, Sylvester, and Callaghan 2021 ; Halling, Herd, and Moynihan 2022 ) and their support for administrative burden policies.

Second, in accordance with the claim by Schneider and Ingram (1993) that target group construction matters to the benefits and burdens assigned to each group, target group deservingness and minority status appear to be of major importance to both burden tolerance ( Baekgaard, Moynihan, and Thomsen 2021 ; Haeder, Sylvester, and Callaghan 2021 ) and barriers ( Jilke, Van Dooren, and Rys 2018 ).

Third, a series of individual-specific explanations of burden tolerance have been investigated in the literature. Most factors have not been theorized very clearly, however, and have only been the subject in few empirical studies. Personal experience with benefits has been shown to be associated with less tolerance for burdensome state actions among Danish local politicians ( Baekgaard, Moynihan, and Thomse 2021 ) and a representative sample of US citizens ( Halling, Herd, and Moynihan 2022 ), while big five personality traits in the form of conscientiousness and openness to experiences have been shown to correlate with burden tolerance in the study of Aarøe et al. (2021) .

Fourth, studies of factors explaining variation in barriers find bureaucratic processes are likely to shape the barriers that citizens meet when interacting with the state. These studies are primarily based on discussions of specific exemplary cases. Peeters (2020) points out that barriers are likely to be unintentional in many cases. They can, for instance, be a result of very complex cases that make it impossible to ease application processes for citizens by means of automation ( Larsson 2021 ), or they can be unintended results of large-scale digitalization and automated decision-making processes where citizens who do not fit into predefined boxes face barriers in the implementation process ( Peeters and Widlak 2018 , 2023 ). Other studies show that bureaucratic low-trust culture and inertia may increase barriers that citizens face when interacting with government ( Bashir and Nisar 2020 ; Peeters et al. 2018 ).

Before we move on to the discussion of next steps to be taken, we note three limitations of our study. The first is publication bias. While we approached the field to include unpublished research, it is possible that some unpublished null findings have not been included or that published null findings did not show up in our literature search because publications with null findings on administrative burden hypotheses have been framed into other literatures. While we consider this a lesser concern given our extensive strategy for collecting studies, publication bias may have made evidence appear stronger than it is. The second limitation has to do with the qualitative coding of studies. While we adhere to stringent coding criteria and have conducted multiple rounds of cross-validating the coding, categorizing studies based on the kind of relationships they study is—at least for some studies—a matter of nuance and assessment. Third, the quality of the included studies is likely to vary, meaning that our review may not give an accurate picture of the strength of evidence for the many propositions studied in administrative burden research. While we have confidence in the general pattern of how different relationships have been covered, others may disagree with our coding of some studies and with the strength of evidence presented in these studies.

Limitations aside, our review points out where evidence is missing and suggests steps to be taken in future research. Next, we discuss which parts of our theoretical model warrant more empirical evidence before finishing with a discussion of new questions for future research to pursue.

More Evidence Needed

Our review points to several issues that should get more attention in future research. First, our understanding of people’s experiences is very much based on the deductive categorization of experiences as learning, compliance, and psychological costs developed in Moynihan, Herd, and Harvey (2015) . While this has laid the foundation for important research, future research could do more to supplement it with bottom-up qualitative research of what burdens are from the perspectives of those interacting with the state. Such research could also aid our understanding of what constitutes more important types of burdensome experiences and under what circumstances they arise. A good example of this kind of research is the work of Barnes (2021, nd ).

Second, it is a core claim of the administrative burden framework that what the state does is consequential for citizens’ experiences. Providing solid causal evidence about this relationship is therefore a key point for future research. Future studies could for instance rely on laboratory experiments inspired by the studies by Hattke, Hensel, and Kalucza 2020 and Hattke et al. (nd) . Another way forward may be to embed surveys and in-depth interviews as part of randomized field experiments to explore how changes in state action influence experiences and in turn outcomes. Here, the study by Lasky-Fink and Linos (2023) may also serve as an example to follow, as the authors combined their field experiment with survey experimental evidence to explore whether the impact of destigmatized language on take-up indeed was mediated by reduced perceived stigma as hypothesized by the authors.

Third, most studies examining this link are conducted among recipients of various social welfare benefits. However, experiences of burden are likely to arise in other types of interactions with the state as is evident from studies of, among others, digital government services ( Madsen, Lindgren, and Melin 2022 ) and voting rights ( Herd and Moynihan 2018 , 43–70; Selin 2019 ). To better understand the scope and importance of administrative burden, there is a need for studies that move beyond social welfare to investigate experiences of burdens in areas such as law enforcement, taxation, and regulation.

Fourth, research on how experiences of administrative burden affect outcomes such as welfare take-up, trust in government, health, and voting behavior is scarce. Most of the articles that study outcomes (primarily take-up) examine how they relate to state actions and not to experiences of burden. To get a more comprehensive picture of how burdensome encounters influence citizens’ lives, we encourage future studies to examine the link between experiences of burden and outcomes.

Fifth, the advancement of the burden tolerance concept allows researchers to examine the extent to which individuals support barriers. An important assumption is that the burden tolerance of policymakers and bureaucrats shapes the actual design of state actions, but it has never been empirically examined. Doing so would help ascertain whether burden tolerance is consequential for the actual design of polices.

Sixth, the administrative burden literature is diverse in terms of methods, policy areas, and subjects. Most studies are conducted in Western countries, but there are studies of burdens from other contexts such as Pakistan and Latin America. However, there is a general lack of comparative studies of burdens across countries and across policies, which would be valuable in terms of providing knowledge on the extent to which context matters for experiences of burden. Likewise, comparative studies of barriers or across policy areas could elucidate which types of state actions are most likely to produce experiences of burdens.

New Questions to Pursue

While we have presented a quite extensive model based on current administrative burden studies, there are still important questions that have received little to no attention in the literature. An important part of the framework formulated by Herd and Moynihan (2018) is that burdens are not inherently bad, and that they often serve legitimate purposes of protecting program integrity and avoiding fraud. While the issue of burden legitimacy has received some theoretical attention ( Doughty and Baehler 2020 ), empirical scholarship has yet to engage with it. One important question is how policymakers and citizens form preferences regarding program integrity vis à vis target group members’ onerous experiences. Studies on burden tolerance touch upon this question, but do not tackle it directly. Another question is how policymakers legitimize the existence of administrative burdens. Do they emphasize fraud protection, budget concerns, targeting the most deserving individuals, or something else? A third question that should get more attention is how actors outside the citizen–state interaction shape experiences of administrative burdens. A few studies show that various third parties such as NGOs and family members can influence experiences of burden, but the roles of these actors still warrant more attention. Further, civil society and the media may influence citizens’ experiences. For example, target group members are often negatively portrayed in the media ( Baekgaard, Herd, and Moynihan 2022 ; Schneider and Ingram 1993 ), which could increase their experiences of burden.

The administrative burden literature, while surprisingly clearly demarcated from other fields of research, has developed into a thematically and methodologically diverse research field within few years. Overall, our systematic review demonstrates that empirical research in the field generally supports the original three-fold claim made by Moynihan, Herd, and Harvey (2015) that burdens are consequential, constructed, and fall harder on groups with few resources. Yet, the review also demonstrates that the literature has moved past these claims in important ways. Based on our reading and coding of 119 articles and working papers, we build a comprehensive model of causal claims in the literature. The model illustrates different relationships that have been explored in the still nascent literature on administrative burdens, and it highlights several new theoretical insights gained since the founding work of Moynihan, Herd, and Harvey (2015) . First, experiences of administrative burdens are sometimes unrelated to how burdens are constructed by the state and instead rely on other factors such as frontline service delivery, government communication, unintended actions, and third parties. Second, the model highlights that factors beyond political ideology may affect the construction of state actions by introducing the concept of burden tolerance. Third, the model shows that factors such as personal experience with programs, personality traits, and the structure of bureaucratic processes affect individuals’ burden tolerance. Finally, the model illustrates a potential feedback effect of citizens’ experiences of administrative burden on policymakers’ burden tolerance.

Our systematic coverage of the administrative burden literature offers promising avenues for new research. First, we call for studies that causally link state actions and experiences of administrative burden, for studies that link experiences of burden to outcomes such as democratic behavior and take-up, and for studies that connect policymakers’ burden tolerance to actual state actions. Methodologically, we call for in-depth qualitative studies of how burdens are experienced by people taking part in citizen–state interactions and comparative studies. Last, we argue that important questions remain unexplored. One topic that future research should address is how policymakers, bureaucrats, and members of the public balance the legitimacy of public policies against target group members’ experiences of administrative burden. Is it acceptable to enhance experiences of administrative burdens to avoid fraud or to target the right populations? Another topic that warrants more attention is how actors outside the citizen–state interaction shape experiences of administrative burden. For example, we know that welfare recipients are often negatively constructed in the media and society ( Baekgaard, Herd, and Moynihan 2022 ; Schneider and Ingram 1993 ), yet we have limited knowledge about whether this leads to them experiencing administrative burdens to a larger extent when interacting with the state.

Supplementary data is available at the Journal of Public Administration Research and Theory online.

We thank Arne Hørlück Høeg for providing excellent research assistance. We are also thankful for the great comments we received from participants at the Administrative Burden pre-conference workshop at the 2022 PMRC.

This work was supported by funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 802244).

No new data were generated or analyzed in support of this research.

Aarøe , Lene , Martin Baekgaard , Julian Christensen , and Donald P. Moynihan . 2021 . Personality and public administration: Policymaker tolerance of administrative burdens in welfare services . Public Administration Review 81 ( 4 ): 652 – 63 .

Google Scholar

Ali , Sameen A. Mohsin , and Samia W. Altaf . 2021 . Citizen trust, administrative capacity and administrative burden in Pakistan’s immunization program . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 17 .

Baekgaard , Martin , Pamela Herd , and Donald P. Moynihan . 2022 . Of “Welfare Queens” and “Poor Carinas” social constructions, deservingness messaging, and the mental health of welfare clients . British Journal of Political Science 53 ( 2 ): 594 – 612 .

Baekgaard , Martin , and Jonas Krogh Madsen . 2023 . Anticipated administrative burden: How proximity to upcoming compulsory meetings affect welfare recipients’ experiences of administrative burden . Public Administration 1 – 19 . doi: 10.1111/padm.12928

Baekgaard , Martin , Kim Sass Mikkelsen , Jonas Krogh Madsen , and Julian Christensen . 2021 . Reducing compliance demands in government benefit programs improves the psychological well-being of target group members . Journal of Public Administration Research and Theory 31 ( 4 ): 806 – 21 .

Baekgaard , Martin , Donald P. Moynihan , and Mette Kjaergaard Thomsen . 2021 . Why do policymakers support administrative burdens? The roles of deservingness, political ideology, and personal experience . Journal of Public Administration Research and Theory 31 ( 1 ): 184 – 200 .

Baekgaard , Martin , and Tara Tankink . 2022 . Administrative burden: Untangling a bowl of conceptual spaghetti . Perspectives on Public Management and Governance 5 ( 1 ): 16 – 21 .

Barnes , Carolyn. nd . Decoupling policy and practice: The redemption costs of WIC . Working Paper.

Google Preview

Barnes , Carolyn , and Virginia Riel . 2022 . “I don’t know nothing about that”: How “learning costs” undermine COVID-related efforts to make SNAP and WIC more accessible . Administration & Society 54 ( 10 ): 1902 – 30 .

Barnes , Carolyn Y. 2021 . “It takes a while to get used to”: The costs of redeeming public benefits . Journal of Public Administration Research and Theory 31 ( 2 ): 295 – 310 .

Barnes , Carolyn Y. , and Julia R. Henly . 2018 . ‘They are underpaid and understaffed’: How clients interpret encounters with street-level bureaucrats . Journal of Public Administration Research and Theory 28 ( 2 ): 456 – 456 .

Bashir , Mohsin , and Muhammed Azfar Nisar . 2020 . Expectation versus reality: Political expediency and implementation of information to information laws . Public Administration Quarterly 44 ( 1 ): 3 – 30 .

Bell , Elizabeth , Julian Christensen , Pamela Herd , and Donald P. Moynihan . 2022 . Health in citizen-state interactions: How physical and mental health problems shape experiences of administrative burden and reduce take-up . The American Review of Public Administration 83 ( 2 ): 385 – 400 .

Bell , Elizabeth , and Katharine Meyer . nd . Can reducing workload enhance equity at the front-lines? How street-level bureaucrats’ capacity impacts access to burdensome public programs . Working Paper.

Bell , Elizabeth , and Kylie Smith . 2022 . Working within a system of administrative burden: How street-level bureaucrats’ role perceptions shape access to the promise of higher education . Administration & Society 54 ( 2 ): 167 – 211 .

Bell , Elizabeth , Ani Ter-Mkrtchyan , Wesley Wehde , and Kylie Smith . 2020 . Just or unjust? How ideological beliefs shape street-level bureaucrats’ perceptions of administrative burden . Public Administration Review 81 ( 4 ): 610 – 24 .

Bhanot , Syon P. 2021 . Good for you or good for us? A field experiment on motivating citizen behavior change . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 14 .

Bhargava , Saurabh , and Dayanand Manoli . 2015 . Psychological frictions and the incomplete take-up of social benefits: Evidence from an IRS field experiment . American Economic Review 105 ( 11 ): 3489 – 529 .

Bisgaard , Mette . 2023 . Dealing with bureaucracy: measuring citizens’ bureaucratic self-efficacy . International Review of Public Administration 28 ( 1 ): 45 – 63 .

Bozeman , Barry , and Jan Youtie . 2020 . Robotic bureaucracy: Administrative burden and red tape in university research . Public Administration Review 80 ( 1 ): 157 – 62 .

Brodkin , Evelyn Z. , and Malay Majmundar . 2010 . Administrative exclusion: Organizations and the hidden costs of welfare claiming . Journal of Public Administration Research and Theory 20 ( 4 ): 827 – 48 .

Bruch , Sarah K. , Myra Marx Ferree , and Joe Soss . 2010 . From policy to polity: Democracy, paternalism, and the incorporation of disadvantaged citizens . American Sociological Review 75 ( 2 ): 205 – 26 .

Burden , Barry C. , David T. Canon , Kenneth R. Mayer , and Donald P. Moynihan . 2012 . The effect of administrative burden on bureaucratic perception of policies: Evidence from election administration . Public Administration Review 72 ( 5 ): 741 – 51 .

Campbell , Jesse W. , Sanjay K. Pandey , and Lars Arnesen . 2022 . The ontology, origin, and impact of divisive public sector rules: A meta‐narrative review of the red tape and administrative burden literatures . Public Administration Review 83 ( 2 ): 296 – 315 .

Carey , Gemma , Helen Dickinson , Eleanor Malbon , Megan Weier , and Gordon Duff . 2020 . Burdensome administration and its risks: Competing logics in policy implementation . Administration & Society 52 ( 9 ): 1362 – 81 .

Carey , Gemma , Eleanor Malbon , and James Blackwell . 2021 . Administering inequality? The national disability insurance scheme and administrative burdens on individuals . Australian Journal of Public Administration 80 ( 4 ): 854 – 72 .

Cecchini , Mathilde. nd . Into the unknown—The administrative burden of uncertainty in citizen-state interactions . Working Paper.

Christensen , Julian , Lene Aarøe , Martin Baekgaard , Pamela Herd , and Donald P. Moynihan . 2020 . Human capital and administrative burden: The role of cognitive resources in citizen-state interactions . Public Administration Review 80 ( 1 ): 127 – 36 .

Chudnovsky , Mariana , and Rik Peeters . 2021a . A cascade of exclusion: Administrative burdens and access to citizenship in the case of Argentina’s National Identity Document . International Review of Administrative Sciences 88 ( 4 ): 1068 – 85 .

———. 2021b . The unequal distribution of administrative burden: A framework and an illustrative case study for understanding variation in people’s experience of burdens . Social Policy and Administration 55 ( 4 ): 527 – 42 .

Collie , Alex , Luke Sheehan , Ashley McAllister , and Genevieve Grant . 2021 . The learning, compliance, and psychological costs of applying for the disability support pension . Australian Journal of Public Administration 80 ( 4 ): 873 – 90 .

Compton , Mallory E. , Matthew M. Young , Justin B. Bullock , and Robert Greer . 2022 . Administrative errors and race: Can technology mitigate inequitable administrative outcomes ? Journal of Public Administration Research and Theory 33 : 512 – 28 .

Cook , Kay. 2021 . Gender, malice, obligation and the state: Separated mothers’ experiences of administrative burdens with Australia’s child support program . Australian Journal of Public Administration 80 ( 4 ): 912 – 32 .

Currie , Janet. 2006 . The take-up of social benefits . In Public policy and the distribution of income , ed. A. J. Auerbach , D. Card , and J. M. Quigley , 80 – 148 . New York : Russell Sage Foundation .

Daigneault , Pierre Marc , and Christian Macé . 2020 . Program awareness, administrative burden, and non-take-up of Québec’s supplement to the work premium . International Journal of Public Administration 43 ( 6 ): 527 – 39 .

Döring , Matthias. 2021 . How-to bureaucracy: A concept of citizens’ administrative literacy . Administration & Society 53 ( 8 ): 1155 – 77 .

Döring , Matthias , and Jonas Krogh Madsen . 2022 . Mitigating psychological costs—The role of citizens’ administrative literacy and social capital . Public Administration Review 82 ( 4 ): 671 – 81 .

Doughty , Meghan , and Karen J. Baehler . 2020 . “Hostages to compliance”: Towards a reasonableness test for administrative burdens . Perspectives on Public Management and Governance 3 ( 4 ): 273 – 87 .

George , Bert , Sanjay K. Pandey , Bram Steijn , Adelien Decramer , and Mieke Audenaert . 2020 . Red tape, organizational performance and employee outcomes: Meta‐analysis, meta‐regression and research agenda . Public Administration Review 81 ( 4 ): 638 – 51 .

Gilad , Sharon , and Michaela Assouline . 2022 . Citizens’ choice to voice in response to administrative burdens . International Public Management Journal : 1 – 22 . doi: 10.1080/10967494.2022.2072988

Griffiths , Rita. 2021 . Universal credit and automated decision making: A case of the digital tail wagging the policy dog ? Social Policy and Society : 1 – 18 .

Haeder , Simon F. , Steven M. Sylvester , and Timothy Callaghan . 2021 . Lingering legacies: Public attitudes about Medicaid beneficiaries and work requirements . Journal of Health Politics, Policy and Law 46 ( 2 ): 305 – 55 .

Halling , Aske. nd . Taxing language: Do interpreting fees affect healthcare usage? Working Paper.

Halling , Aske , Pamela Herd , and Donald Moynihan . 2022 . How difficult should it be? Evidence of burden tolerance from a nationally representative sample . Public Management Review : 1 – 20 . doi: 10.1080/14719037.2022.2056910

Halling , Aske , and Niels Bjørn Grund Petersen . nd . Burden feedback: When citizens communicate burdens, frontline employees respond evidence from an experiment . Working Paper.

Hattke , Fabian , Judith Hattke , David Hensel , Pamela Herd , Janne Kaluzca , Donald P. Moynihan , and Rick Vogel . nd . Face-to-face with administrative burdens: Physiological measures and behavioral consequences of psychological costs . Working Paper.

Hattke , Fabian , David Hensel , and Janne Kalucza . 2020 . Emotional responses to bureaucratic red tape . Public Administration Review 80 ( 1 ): 53 – 63 .

Heinrich , Carolyn J. 2016 . The bite of administrative burden: A theoretical and empirical investigation . Journal of Public Administration Research and Theory 26 ( 3 ): 403 – 20 .

———. 2018 . Presidential address: “A thousand petty fortresses”: Administrative burden in US immigration policies and its consequences . Journal of Policy Analysis and Management 37 ( 2 ): 211 – 39 .

Heinrich , Carolyn J. , and Robert Brill . 2015 . Stopped in the name of the law: Administrative burden and its implications for cash transfer program effectiveness . World Development 72 ( August ): 277 – 95 .

Heinrich , Carolyn J. , Sayil Camacho , Kaitlin Binsted , and Shadlan Gale . 2022 . An audit test evaluation of state practices for supporting access to and promoting Covid-19 vaccinations . Social Science and Medicine 301 : 114880 .

Herd , Pamela , Thomas DeLeire , Hope Harvey , and Donald P. Moynihan . 2013 . Shifting administrative burden to the state: The case of Medicaid take-up . Public Administration Review 73 ( s1 ): s69 – 81 .

Herd , Pamela , and Donald P. Moynihan . 2018 . Administrative burden: Policymaking by other means , 1st ed. New York : Russell Sage Foundation .

Hock , Heinrich , John T. Jones , Michael Levere , and David Wittenburg . 2021 . Using behavioral outreach to counteract administrative burden and encourage take-up of simplified disability payment rules . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 15 .

Holler , Roni , and Noam Tarshish . 2022 . Administrative burden in citizen-state encounters: The role of waiting, communication breakdowns and administrative errors . Social Policy and Society : 1 – 18 .

Hoynes , Hilary , Diane W. Schanzenbach , and Douglas Almond . 2016 . Long-run impacts of childhood access to the safety net . American Economic Review 106 ( 4 ): 903 – 34 .

Jakobsen , Morten , Oliver James , Donald P. Moynihan , and Tina Nabatchi . 2016 . JPART virtual issue on citizen-state interactions in public administration research . Journal of Public Administration Research and Theory 29 ( 4 ): e8 – 15 .

Jenkins , Jade Marcus , and Tutrang Nguyen . 2022 . Keeping kids in care: Reducing administrative burden in state child care development fund policy . Journal of Public Administration Research and Theory 32 ( 1 ): 23 – 40 .

Jilke , Sebastian , Wouter Van Dooren , and Sabine Rys . 2018 . Discrimination and administrative burden in public service markets: Does a public-private difference exist ? Journal of Public Administration Research and Theory 28 ( 3 ): 423 – 39 .

Johnson , Donavon , and Alexander Kroll . 2021 . What makes us tolerant of administrative burden? Race, representation, and identity . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 9 .

Kaufmann , Wesley , Alex Ingrams , and Daan Jacobs . 2021 . Being consistent matters: Experimental evidence on the effect of rule consistency on citizen red tape . American Review of Public Administration 51 ( 1 ): 28 – 39 .

Keiser , Lael R. , and Susan M. Miller . 2020 . Does administrative burden influence public support for government programs? Evidence from a survey experiment . Public Administration Review 80 ( 1 ): 137 – 50 .

Kyle , Michael Anne , and Austin B. Frakt . 2021 . Patient administrative burden in the US health care system . Health Services Research 56 ( 5 ): 755 – 65 .

Larsson , Karl Kristian. 2021 . Digitization or equality: When government automation covers some, but not all citizens . Government Information Quarterly 38 ( 1 ): 101547 – 10 .

Lasky-Fink , Jessica , and Linos . Elizabeth . 2023 . Improving delivery of the social safety net: The role of stigma . Journal of Public Administration Research and Theory .

Linos , Elizabeth , Lisa T. Quan , and Elspeth Kirkman . 2020 . Nudging early reduces administrative burden: Three field experiments to improve code enforcement . Journal of Policy Analysis and Management 39 ( 1 ): 243 – 65 .

Linos , Elizabeth , Vikash Reddy , and Jesse Rothstein . 2022 . Demystifying college costs: How nudges can and can’t help . Behavioral Public Policy : 1 – 22 .

Linos , Elizabeth , and Nefara Riesch . 2020 . Thick red tape and the thin blue line: A field study on reducing administrative burden in police recruitment . Public Administration Review 80 ( 1 ): 92 – 103 .

Linos , Katerina , Melissa Carlson , Laura Jakli , Nadia Dalma , Isabelle Cohen , Afroditi Veloudaki , and Stavros Nikiforos Spyrellis . 2022 . How do disadvantaged groups seek information about public services? A randomized controlled trial of communication technologies . Public Administration Review 82 ( 4 ): 708 – 20 .

Lipsky , Michael. 1980 . Street-level bureaucracy: Dilemmas of the individual in public services . New York : Russell Sage Foundation .

Lopoo , Leonard M. , Colleen Heflin , and Joseph Boskovski . 2020 . Testing behavioral interventions designed to improve on-time SNAP recertification . Journal of Behavioral Public Administration 3 ( 2 ): 1 – 8 .

Madsen , Christian Østergaard , Ida Lindgren , and Ulf Melin . 2022 . The accidental caseworker—How digital self-service influences citizens’ administrative burden . Government Information Quarterly 39 ( 1 ): 101653 .

Madsen , Jonas Krogh. nda . Compliant but discouraged? How administrative burden influence unemployment benefit recipients’ job search motivation . Working Paper.

———. ndb . Frictions on both sides of the counter? A study of red tape among street-level bureaucrats and administrative burden among their clients . Working Paper.

Madsen , Jonas Krogh , Martin Baekgaard , and Jon Kvist . 2022 . Scarcity and the mindsets of social welfare recipients: Evidence from a field experiment . Journal of Public Administration Research and Theory 33 : 675 – 87 .

Madsen , Jonas Krogh , and Kim Sass Mikkelsen . 2022 . How salient administrative burden affects job seekers’ locus of control and responsibility attribution: Evidence from a survey experiment . International Public Management Journal 25 ( 2 ): 241 – 60 .

Madsen , Jonas Krogh , Kim Sass Mikkelsen , and Donald P. Moynihan . 2022 . Burdens, sludge, ordeals, red tape, oh my! A user’s guide to the study of frictions . Public Administration 100 ( 2 ): 375 – 93 .

Masood , Ayesha , and Muhammed Azfar Nisar . 2021 . Administrative capital and citizens’ responses to administrative burden . Journal of Public Administration Research and Theory 31 ( 1 ): 56 – 72 .

Mikkelsen , Kim Sass , Jonas Krogh Madsen , and Martin Baekgaard . 2023 . Is stress among street level bureaucrats associated with experiences of administrative burden among clients? A multilevel study of the Danish unemployment sector . Public Administration Review 1 – 13 . doi: 10.1111/puar.13673

Moynihan , Donald , Eric Giannella , Pamela Herd , and Julie Sutherland . 2022 . Matching to categories: Learning and compliance costs in administrative processes . Journal of Public Administration Research and Theory 32 ( 4 ): 750 – 64 .

Moynihan , Donald , and Joe Soss . 2014 . Policy feedback and the politics of administration . Public Administration Review 74 ( 3 ): 320 – 32 .

Moynihan , Donald P. , Pamela Herd , and Hope Harvey . 2015 . Administrative burden: Learning, psychological, and compliance costs in citizen-state interactions . Journal of Public Administration Research and Theory 25 ( 1 ): 43 – 69 .

Moynihan , Donald P. , Pamela Herd , and Elizabeth Ribgy . 2016 . Policymaking by other means: Do states use administrative barriers to limit access to Medicaid . Administration & Society 48 ( 4 ): 497 – 524 .

Newman , Mark E. J. 2003 . Mixing patterns in networks . Physical Review 67 ( 026126 ): 1 – 13 .

Nicholson-Crotty , Jill , Susan M. Miller , and Lael R. Keiser . 2021 . Administrative burden, social construction, and public support for government programs . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 29 .

Nisar , Muhammad A. 2018 . Children of a lesser god: Administrative burden and social equity in citizen-state interactions . Journal of Public Administration Research and Theory 28 ( 1 ): 104 – 19 .

Nisar , Muhammed A. , and Ayesha Masood . nd . Governance by artifacts: Theory and evidence on materiality of administrative burdens . Working Paper.

Olsen , Asmus Leth. 2017 . Human interest or hard numbers? Experiments on citizens’ selection, exposure, and recall of performance information . Public Administration Review 77 ( 3 ): 408 – 20 .

Olsen , Asmus Leth , Jonas Høgh Kyhse-Andersen , and Donald Moynihan . 2020 . The unequal distribution of opportunity: A national audit study of bureaucratic discrimination in primary school access . American Journal of Political Science , Early view version, 66 ( 3 ): 587 – 603 .

Page , Matthew J. , Joanne E. McKenzie , Patrick M. Bossuyt , Isabelle Boutron , Tammy C. Hoffmann , Cynthia D. Mulrow , Larissa Shamseer , et al. 2021 . The PRISMA 2020 statement: An updated guideline for reporting systematic reviews . BMJ 372 : n71 .

Peeters , Rik. 2020 . The political economy of administrative burdens: A theoretical framework for analyzing the organizational origins of administrative burden . Administration & Society 52 ( 4 ): 566 – 92 .

Peeters , Rik , and Sergio A. Campos . 2021 . Taking the bite out of administrative burdens: How beneficiaries of a Mexican social program ease administrative burdens in street-level interactions . Governance 34 ( 4 ): 1001 – 18 .

Peeters , Rik , Humberto Trujillo Jimenez , Elizabeth O’Connor , Pascual Ogarrio Rojas , Michele Gonzalez Galindo , and Daniela Morales Tenorio . 2018 . Low-trust bureaucracy: Understanding the Mexican bureaucratic experience . Public Administration and Development 38 ( 2 ): 65 – 74 .

Peeters , Rik , César Renteria , and Guillermo M. Cejudo . nd . How information capacity shapes administrative burdens: A comparison of the citizen experience of the COVID-19 vaccination programs in the United States, Mexico, and the Netherlands . Working Paper.

Peeters , Rik , and Arjan Widlak . 2018 . The digital cage: Administrative exclusion through information architecture—The case of the Dutch civil registry’s master data management system . Government Information Quarterly 35 : 175 – 83 .

——— . 2023 . Administrative exclusion in the infrastructure-level bureaucracy: The case of the Dutch daycare benefit scandal . Public Administration Review 83 ( 4 ): 863 – 877 . doi: 10.1111/puar.13615

Perianes-Rodriguez , Antonio , Ludo Waltman , and Nees Jan van Eck . 2016 . Constructing bibliometric networks: A comparison between full and fractional counting . Journal of Informetrics 10 ( 4 ): 1178 – 95 .

Petersen , Ole Helby , Jesper Rosenberg Hansen , and Kurt Houlberg . 2022 . The administrative burden of doing business with the government: Learning and compliance costs in Business-Government interactions . Public Administration Review 1 – 19 . doi: 10.1111/padm.12904

Schneider , Anne , and Helen Ingram . 1993 . Social construction of target populations: Implications for politics and policy . American Political Science Review 87 ( 2 ): 334 – 47 .

Selin , Jennifer L. 2019 . The best laid plans: How administrative burden complicates voting rights restoration . Missouri Law Review 84 ( 4 ): 1 – 38 .

Sievert , Martin , and Jonas Bruder . 2023 . Unpacking the effects of burdensome state actions on citizens’ policy perceptions . Public Administration 1 – 21 .

Sievert , Martin , Dominik Vogel , and Mary K. Feeney . 2020 . Formalization and administrative burden as obstacles to employee recruitment: Consequences for the public sector . Review of Public Personnel Administration 42 ( 1 ): 3 – 30 .

Soss , Joe. 1999 . Lessons of welfare: Policy design, political learning, and political action . American Political Science Review 93 ( 2 ): 363 – 80 .

Soss , Joe , Richard Fording , and Sanford F. Schram . 2011 . The organization of discipline: From performance management to perversity and punishment . Journal of Public Administration Research and Theory 21 : i203 – 32 .

Thomsen , Mette Kjaergaard , Martin Baekgaard , and Ulrich Thy Jensen . 2020 . The psychological costs of citizen coproduction . Journal of Public Administration Research and Theory 30 ( 4 ): 656 – 73 .

Widlak , Arjan , and Rik Peeters . 2020 . Administrative errors and the burden of correction and consequence: How information technology exacerbates the consequences of bureaucratic mistakes for citizens . International Journal of Electronic Governance 12 ( 1 ): 1 – 56 .

Yates , Sophie , Gemma Carey , Eleanor Malbon , and Jen Hargrave . 2022 . “Faceless monster, secret society”: Women’s experiences navigating the administrative burden of Australia’s National Disability Insurance Scheme . Health and Social Care in the Community 50 ( 5 ): e2308 – 17 .

As the only exception, we excluded Herd and Moynihan (2018) from the review. The main points in this book have been covered in several journal articles by the authors and including it would therefore introduce the risk of double-counting arguments.

Many of the working papers were later published. The initial number of working papers was 30.

Since this feedback effect is mainly inspired by policy feedback research, for the sake of simplicity we chose not to show this as an independent arrow in the model.

Supplementary data

Email alerts, citing articles via.

Recommend to your Library

Affiliations

Online ISSN 1477-9803
Print ISSN 1053-1858
Copyright © 2024 Public Management Research Association
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
BMC Med Res Methodol

Methods for the synthesis of qualitative research: a critical review

Elaine barnett-page.

1 Evidence for Policy and Practice Information and Co-ordinating (EPPI-) Centre, Social Science Research Unit, 18 Woburn Square, London WC1H 0NS, UK

James Thomas

Associated data.

In recent years, a growing number of methods for synthesising qualitative research have emerged, particularly in relation to health-related research. There is a need for both researchers and commissioners to be able to distinguish between these methods and to select which method is the most appropriate to their situation.

A number of methodological and conceptual links between these methods were identified and explored, while contrasting epistemological positions explained differences in approaches to issues such as quality assessment and extent of iteration. Methods broadly fall into 'realist' or 'idealist' epistemologies, which partly accounts for these differences.

Methods for qualitative synthesis vary across a range of dimensions. Commissioners of qualitative syntheses might wish to consider the kind of product they want and select their method – or type of method – accordingly.

The range of different methods for synthesising qualitative research has been growing over recent years [ 1 , 2 ], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [ 3 ]. While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe how quantitative research is synthesised, far more terms are used to describe the synthesis of qualitative research. This profusion of terms can mask some of the basic similarities in approach that the different methods share, and also lead to some confusion regarding which method is most appropriate in a given situation. This paper does not argue that the various nomenclatures are unnecessary, but rather seeks to draw together and review the full range of methods of synthesis available to assist future reviewers in selecting a method that is fit for their purpose. It also represents an attempt to guide the reader through some of the varied terminology to spring up around qualitative synthesis. Other helpful reviews of synthesis methods have been undertaken in recent years with slightly different foci to this paper. Two recent studies have focused on describing and critiquing methods for the integration of qualitative research with quantitative [ 4 , 5 ] rather than exclusively examining the detail and rationale of methods for the synthesis of qualitative research. Two other significant pieces of work give practical advice for conducting the synthesis of qualitative research, but do not discuss the full range of methods available [ 6 , 7 ]. We begin our Discussion by outlining each method of synthesis in turn, before comparing and contrasting characteristics of these different methods across a range of dimensions. Readers who are more familiar with the synthesis methods described here may prefer to turn straight to the 'dimensions of difference' analysis in the second part of the Discussion.

Overview of synthesis methods

Meta-ethnography.

In their seminal work of 1988, Noblit and Hare proposed meta-ethnography as an alternative to meta-analysis [ 8 ]. They cited Strike and Posner's [ 9 ] definition of synthesis as an activity in which separate parts are brought together to form a 'whole'; this construction of the whole is essentially characterised by some degree of innovation, so that the result is greater than the sum of its parts. They also borrowed from Turner's theory of social explanation [ 10 ], a key tenet of which was building 'comparative understanding' [[ 8 ], p22] rather than aggregating data.

To Noblit and Hare, synthesis provided an answer to the question of 'how to "put together" written interpretive accounts' [[ 8 ], p7], where mere integration would not be appropriate. Noblit and Hare's early work synthesised research from the field of education.

Three different methods of synthesis are used in meta-ethnography. One involves the 'translation' of concepts from individual studies into one another, thereby evolving overarching concepts or metaphors. Noblit and Hare called this process reciprocal translational analysis (RTA). Refutational synthesis involves exploring and explaining contradictions between individual studies. Lines-of-argument (LOA) synthesis involves building up a picture of the whole (i.e. culture, organisation etc) from studies of its parts. The authors conceptualised this latter approach as a type of grounded theorising.

Britten et al [ 11 ] and Campbell et al [ 12 ] have both conducted evaluations of meta-ethnography and claim to have succeeded, by using this method, in producing theories with greater explanatory power than could be achieved in a narrative literature review. While both these evaluations used small numbers of studies, more recently Pound et al [ 13 ] conducted both an RTA and an LOA synthesis using a much larger number of studies (37) on resisting medicines. These studies demonstrate that meta-ethnography has evolved since Noblit and Hare first introduced it. Campbell et al claim to have applied the method successfully to non-ethnographical studies. Based on their reading of Schutz [ 14 ], Britten et al have developed both second and third order constructs in their synthesis (Noblit and Hare briefly allude to the possibility of a 'second level of synthesis' [[ 8 ], p28] but do not demonstrate or further develop the idea).

In a more recent development, Sandelowski & Barroso [ 15 ] write of adapting RTA by using it to ' integrate findings interpretively, as opposed to comparing them interpretively' (p204). The former would involve looking to see whether the same concept, theory etc exists in different studies; the latter would involve the construction of a bigger picture or theory (i.e. LOA synthesis). They also talk about comparing or integrating imported concepts (e.g. from other disciplines) as well as those evolved 'in vivo'.

Grounded theory

Kearney [ 16 ], Eaves [ 17 ] and Finfgeld [ 18 ] have all adapted grounded theory to formulate a method of synthesis. Key methods and assumptions of grounded theory, as originally formulated and subsequently refined by Glaser and Strauss [ 19 ] and Strauss and Corbin [ 20 , 21 ], include: simultaneous phases of data collection and analysis; an inductive approach to analysis, allowing the theory to emerge from the data; the use of the constant comparison method; the use of theoretical sampling to reach theoretical saturation; and the generation of new theory. Eaves cited grounded theorists Charmaz [ 22 ] and Chesler [ 23 ], as well as Strauss and Corbin [ 20 ], as informing her approach to synthesis.

Glaser and Strauss [ 19 ] foresaw a time when a substantive body of grounded research should be pushed towards a higher, more abstract level. As a piece of methodological work, Eaves undertook her own synthesis of the synthesis methods used by these authors to produce her own clear and explicit guide to synthesis in grounded formal theory. Kearney stated that 'grounded formal theory', as she termed this method of synthesis, 'is suited to study of phenomena involving processes of contextualized understanding and action' [[ 24 ], p180] and, as such, is particularly applicable to nurses' research interests.

As Kearney suggested, the examples examined here were largely dominated by research in nursing. Eaves synthesised studies on care-giving in rural African-American families for elderly stroke survivors; Finfgeld on courage among individuals with long-term health problems; Kearney on women's experiences of domestic violence.

Kearney explicitly chose 'grounded formal theory' because it matches 'like' with 'like': that is, it applies the same methods that have been used to generate the original grounded theories included in the synthesis – produced by constant comparison and theoretical sampling – to generate a higher-level grounded theory. The wish to match 'like' with 'like' is also implicit in Eaves' paper. This distinguishes grounded formal theory from more recent applications of meta-ethnography, which have sought to include qualitative research using diverse methodological approaches [ 12 ].

Thematic Synthesis

Thomas and Harden [ 25 ] have developed an approach to synthesis which they term 'thematic synthesis'. This combines and adapts approaches from both meta-ethnography and grounded theory. The method was developed out of a need to conduct reviews that addressed questions relating to intervention need, appropriateness and acceptability – as well as those relating to effectiveness – without compromising on key principles developed in systematic reviews. They applied thematic synthesis in a review of the barriers to, and facilitators of, healthy eating amongst children.

Free codes of findings are organised into 'descriptive' themes, which are then further interpreted to yield 'analytical' themes. This approach shares characteristics with later adaptations of meta-ethnography, in that the analytical themes are comparable to 'third order interpretations' and that the development of descriptive and analytical themes using coding invoke reciprocal 'translation'. It also shares much with grounded theory, in that the approach is inductive and themes are developed using a 'constant comparison' method. A novel aspect of their approach is the use of computer software to code the results of included studies line-by-line, thus borrowing another technique from methods usually used to analyse primary research.

Textual Narrative Synthesis

Textual narrative synthesis is an approach which arranges studies into more homogenous groups. Lucas et al [ 26 ] comment that it has proved useful in synthesising evidence of different types (qualitative, quantitative, economic etc). Typically, study characteristics, context, quality and findings are reported on according to a standard format and similarities and differences are compared across studies. Structured summaries may also be developed, elaborating on and putting into context the extracted data [ 27 ].

Lucas et al [ 26 ] compared thematic synthesis with textual narrative synthesis. They found that 'thematic synthesis holds most potential for hypothesis generation' whereas textual narrative synthesis is more likely to make transparent heterogeneity between studies (as does meta-ethnography, with refutational synthesis) and issues of quality appraisal. This is possibly because textual narrative synthesis makes clearer the context and characteristics of each study, while the thematic approach organises data according to themes. However, Lucas et al found that textual narrative synthesis is 'less good at identifying commonality' (p2); the authors do not make explicit why this should be, although it may be that organising according to themes, as the thematic approach does, is comparatively more successful in revealing commonality.

Paterson et al [ 28 ] have evolved a multi-faceted approach to synthesis, which they call 'meta-study'. The sociologist Zhao [ 29 ], drawing on Ritzer's work [ 30 ], outlined three components of analysis, which they proposed should be undertaken prior to synthesis. These are meta-data-analysis (the analysis of findings), meta-method (the analysis of methods) and meta-theory (the analysis of theory). Collectively, these three elements of analysis, culminating in synthesis, make up the practice of 'meta-study'. Paterson et al pointed out that the different components of analysis may be conducted concurrently.

Paterson et al argued that primary research is a construction; secondary research is therefore a construction of a construction. There is need for an approach that recognises this, and that also recognises research to be a product of its social, historical and ideological context. Such an approach would be useful in accounting for differences in research findings. For Paterson et al, there is no such thing as 'absolute truth'.

Meta-study was developed to study the experiences of adults living with a chronic illness. Meta-data-analysis was conceived of by Paterson et al in similar terms to Noblit and Hare's meta-ethnography (see above), in that it is essentially interpretive and seeks to reveal similarities and discrepancies among accounts of a particular phenomenon. Meta-method involves the examination of the methodologies of the individual studies under review. Part of the process of meta-method is to consider different aspects of methodology such as sampling, data collection, research design etc, similar to procedures others have called 'critical appraisal' (CASP [ 31 ]). However, Paterson et al take their critique to a deeper level by establishing the underlying assumptions of the methodologies used and the relationship between research outcomes and methods used. Meta-theory involves scrutiny of the philosophical and theoretical assumptions of the included research papers; this includes looking at the wider context in which new theory is generated. Paterson et al described meta-synthesis as a process which creates a new interpretation which accounts for the results of all three elements of analysis. The process of synthesis is iterative and reflexive and the authors were unwilling to oversimplify the process by 'codifying' procedures for bringing all three components of analysis together.

Meta-narrative

Greenhalgh et al [ 32 ]'s meta-narrative approach to synthesis arose out of the need to synthesise evidence to inform complex policy-making questions and was assisted by the formation of a multi-disciplinary team. Their approach to review was informed by Thomas Kuhn's The Structure of Scientific Revolutions [ 33 ], in which he proposed that knowledge is produced within particular paradigms which have their own assumptions about theory, about what is a legitimate object of study, about what are legitimate research questions and about what constitutes a finding. Paradigms also tend to develop through time according to a particular set of stages, central to which is the stage of 'normal science', in which the particular standards of the paradigm are largely unchallenged and seen to be self-evident. As Greenhalgh et al pointed out, Kuhn saw paradigms as largely incommensurable: 'that is, an empirical discovery made using one set of concepts, theories, methods and instruments cannot be satisfactorily explained through a different paradigmatic lens' [[ 32 ], p419].

Greenhalgh et al synthesised research from a wide range of disciplines; their research question related to the diffusion of innovations in health service delivery and organisation. They thus identified a need to synthesise findings from research which contains many different theories arising from many different disciplines and study designs.

Based on Kuhn's work, Greenhalgh et al proposed that, across different paradigms, there were multiple – and potentially mutually contradictory – ways of understanding the concept at the heart of their review, namely the diffusion of innovation. Bearing this in mind, the reviewers deliberately chose to select key papers from a number of different research 'paradigms' or 'traditions', both within and beyond healthcare, guided by their multidisciplinary research team. They took as their unit of analysis the 'unfolding "storyline" of a research tradition over time' [[ 32 ], p417) and sought to understand diffusion of innovation as it was conceptualised in each of these traditions. Key features of each tradition were mapped: historical roots, scope, theoretical basis; research questions asked and methods/instruments used; main empirical findings; historical development of the body of knowledge (how have earlier findings led to later findings); and strengths and limitations of the tradition. The results of this exercise led to maps of 13 'meta-narratives' in total, from which seven key dimensions, or themes, were identified and distilled for the synthesis phase of the review.

Critical Interpretive Synthesis

Dixon-Woods et al [ 34 ] developed their own approach to synthesising multi-disciplinary and multi-method evidence, termed 'critical interpretive synthesis', while researching access to healthcare by vulnerable groups. Critical interpretive synthesis is an adaptation of meta-ethnography, as well as borrowing techniques from grounded theory. The authors stated that they needed to adapt traditional meta-ethnographic methods for synthesis, since these had never been applied to quantitative as well as qualitative data, nor had they been applied to a substantial body of data (in this case, 119 papers).

Dixon-Woods et al presented critical interpretive synthesis as an approach to the whole process of review, rather than to just the synthesis component. It involves an iterative approach to refining the research question and searching and selecting from the literature (using theoretical sampling) and defining and applying codes and categories. It also has a particular approach to appraising quality, using relevance – i.e. likely contribution to theory development – rather than methodological characteristics as a means of determining the 'quality' of individual papers [ 35 ]. The authors also stress, as a defining characteristic, critical interpretive synthesis's critical approach to the literature in terms of deconstructing research traditions or theoretical assumptions as a means of contextualising findings.

Dixon-Woods et al rejected reciprocal translational analysis (RTA) as this produced 'only a summary in terms that have already been used in the literature' [[ 34 ], p5], which was seen as less helpful when dealing with a large and diverse body of literature. Instead, Dixon-Woods et al adopted a lines-of-argument (LOA) synthesis, in which – rejecting the difference between first, second and third order constructs – they instead developed 'synthetic constructs' which were then linked with constructs arising directly from the literature.

The influence of grounded theory can be seen in particular in critical interpretive synthesis's inductive approach to formulating the review question and to developing categories and concepts, rejecting a 'stage' approach to systematic reviewing, and in selecting papers using theoretical sampling. Dixon-Woods et al also claim that critical interpretive synthesis is distinct in its 'explicit orientation towards theory generation' [[ 34 ], p9].

Ecological Triangulation

Jim Banning is the author of 'ecological triangulation' or 'ecological sentence synthesis', applying this method to the evidence for what works for youth with disabilities. He borrows from Webb et al [ 36 ] and Denzin [ 37 ] the concept of triangulation, in which phenomena are studied from a variety of vantage points. His rationale is that building an 'evidence base' of effectiveness requires the synthesis of cumulative, multi-faceted evidence in order to find out 'what intervention works for what kind of outcomes for what kind of persons under what kind of conditions' [[ 38 ], p1].

Ecological triangulation unpicks the mutually interdependent relationships between behaviour, persons and environments. The method requires that, for data extraction and synthesis, 'ecological sentences' are formulated following the pattern: 'With this intervention, these outcomes occur with these population foci and within these grades (ages), with these genders ... and these ethnicities in these settings' [[ 39 ], p1].

Framework Synthesis

Brunton et al [ 40 ] and Oliver et al [ 41 ] have applied a 'framework synthesis' approach in their reviews. Framework synthesis is based on framework analysis, which was outlined by Pope, Ziebland and Mays [ 42 ], and draws upon the work of Ritchie and Spencer [ 43 ] and Miles and Huberman [ 44 ]. Its rationale is that qualitative research produces large amounts of textual data in the form of transcripts, observational fieldnotes etc. The sheer wealth of information poses a challenge for rigorous analysis. Framework synthesis offers a highly structured approach to organising and analysing data (e.g. indexing using numerical codes, rearranging data into charts etc).

Brunton et al applied the approach to a review of children's, young people's and parents' views of walking and cycling; Oliver et al to an analysis of public involvement in health services research. Framework synthesis is distinct from the other methods outlined here in that it utilises an a priori 'framework' – informed by background material and team discussions – to extract and synthesise findings. As such, it is largely a deductive approach although, in addition to topics identified by the framework, new topics may be developed and incorporated as they emerge from the data. The synthetic product can be expressed in the form of a chart for each key dimension identified, which may be used to map the nature and range of the concept under study and find associations between themes and exceptions to these [ 40 ].

'Fledgling' approaches

There are three other approaches to synthesis which have not yet been widely used. One is an approach using content analysis [ 45 , 46 ] in which text is condensed into fewer content-related categories. Another is 'meta-interpretation' [ 47 ], featuring the following: an ideographic rather than pre-determined approach to the development of exclusion criteria; a focus on meaning in context; interpretations as raw data for synthesis (although this feature doesn't distinguish it from other synthesis methods); an iterative approach to the theoretical sampling of studies for synthesis; and a transparent audit trail demonstrating the trustworthiness of the synthesis.

In addition to the synthesis methods discussed above, Sandelowski and Barroso propose a method they call 'qualitative metasummary' [ 15 ]. It is mentioned here as a new and original approach to handling a collection of qualitative studies but is qualitatively different to the other methods described here since it is aggregative; that is, findings are accumulated and summarised rather than 'transformed'. Metasummary is a way of producing a 'map' of the contents of qualitative studies and – according to Sandelowski and Barroso – 'reflect [s] a quantitative logic' [[ 15 ], p151]. The frequency of each finding is determined and the higher the frequency of a particular finding, the greater its validity. The authors even discuss the calculation of 'effect sizes' for qualitative findings. Qualitative metasummaries can be undertaken as an end in themselves or may serve as a basis for a further synthesis.

Dimensions of difference

Having outlined the range of methods identified, we now turn to an examination of how they compare with one another. It is clear that they have come from many different contexts and have different approaches to understanding knowledge, but what do these differences mean in practice? Our framework for this analysis is shown in Additional file 1 : dimensions of difference [ 48 ]. We have examined the epistemology of each of the methods and found that, to some extent, this explains the need for different methods and their various approaches to synthesis.

Epistemology

The first dimension that we will consider is that of the researchers' epistemological assumptions. Spencer et al [ 49 ] outline a range of epistemological positions, which might be organised into a spectrum as follows:

Subjective idealism : there is no shared reality independent of multiple alternative human constructions

Objective idealism : there is a world of collectively shared understandings

Critical realism : knowledge of reality is mediated by our perceptions and beliefs

Scientific realism : it is possible for knowledge to approximate closely an external reality

Naïve realism : reality exists independently of human constructions and can be known directly [ 49 , 45 , 46 ].

Thus, at one end of the spectrum we have a highly constructivist view of knowledge and, at the other, an unproblematized 'direct window onto the world' view.

Nearly all of positions along this spectrum are represented in the range of methodological approaches to synthesis covered in this paper. The originators of meta-narrative synthesis, critical interpretive synthesis and meta-study all articulate what might be termed a 'subjective idealist' approach to knowledge. Paterson et al [ 28 ] state that meta-study shies away from creating 'grand theories' within the health or social sciences and assume that no single objective reality will be found. Primary studies, they argue, are themselves constructions; meta-synthesis, then, 'deals with constructions of constructions' (p7). Greenhalgh et al [ 32 ] also view knowledge as a product of its disciplinary paradigm and use this to explain conflicting findings: again, the authors neither seek, nor expect to find, one final, non-contestable answer to their research question. Critical interpretive synthesis is similar in seeking to place literature within its context, to question its assumptions and to produce a theoretical model of a phenomenon which – because highly interpretive – may not be reproducible by different research teams at alternative points in time [[ 34 ], p11].

Methods used to synthesise grounded theory studies in order to produce a higher level of grounded theory [ 24 ] appear to be informed by 'objective idealism', as does meta-ethnography. Kearney argues for the near-universal applicability of a 'ready-to-wear' theory across contexts and populations. This approach is clearly distinct from one which recognises multiple realities. The emphasis is on examining commonalities amongst, rather than discrepancies between, accounts. This emphasis is similarly apparent in most meta-ethnographies, which are conducted either according to Noblit and Hare's 'reciprocal translational analysis' technique or to their 'lines-of-argument' technique and which seek to provide a 'whole' which has a greater explanatory power. Although Noblit and Hare also propose 'refutational synthesis', in which contradictory findings might be explored, there are few examples of this having been undertaken in practice, and the aim of the method appears to be to explain and explore differences due to context, rather than multiple realities.

Despite an assumption of a reality which is perhaps less contestable than those of meta-narrative synthesis, critical interpretive synthesis and meta-study, both grounded formal theory and meta-ethnography place a great deal of emphasis on the interpretive nature of their methods. This still supposes a degree of constructivism. Although less explicit about how their methods are informed, it seems that both thematic synthesis and framework synthesis – while also involving some interpretation of data – share an even less problematized view of reality and a greater assumption that their synthetic products are reproducible and correspond to a shared reality. This is also implicit in the fact that such products are designed directly to inform policy and practice, a characteristic shared by ecological triangulation. Notably, ecological triangulation, according to Banning, can be either realist or idealist. Banning argues that the interpretation of triangulation can either be one in which multiple viewpoints converge on a point to produce confirming evidence (i.e. one definitive answer to the research question) or an idealist one, in which the complexity of multiple viewpoints is represented. Thus, although ecological triangulation views reality as complex, the approach assumes that it can be approximately knowable (at least when the realist view of ecological triangulation is adopted) and that interventions can and should be modelled according to the products of its syntheses.

While pigeonholing different methods into specific epistemological positions is a problematic process, we do suggest that the contrasting epistemologies of different researchers is one way of explaining why we have – and need – different methods for synthesis.

Variation in terms of the extent of iteration during the review process is another key dimension. All synthesis methods include some iteration but the degree varies. Meta-ethnography, grounded theory and thematic synthesis all include iteration at the synthesis stage; both framework synthesis and critical interpretive synthesis involve iterative literature searching – in the case of critical interpretive synthesis, it is not clear whether iteration occurs during the rest of the review process. Meta-narrative also involves iteration at every stage. Banning does not mention iteration in outlining ecological triangulation and neither do Lucas or Thomas and Harden for thematic narrative synthesis.

It seems that the more idealist the approach, the greater the extent of iteration. This might be because a large degree of iteration does not sit well with a more 'positivist' ideal of procedural objectivity; in particular, the notion that the robustness of the synthetic product depends in part on the reviewers stating up front in a protocol their searching strategies, inclusion/exclusion criteria etc, and being seen not to alter these at a later stage.

Quality assessment

Another dimension along which we can look at different synthesis methods is that of quality assessment. When the approaches to the assessment of the quality of studies retrieved for review are examined, there is again a wide methodological variation. It might be expected that the further towards the 'realism' end of the epistemological spectrum a method of synthesis falls, the greater the emphasis on quality assessment. In fact, this is only partially the case.

Framework synthesis, thematic narrative synthesis and thematic synthesis – methods which might be classified as sharing a 'critical realist' approach – all have highly specified approaches to quality assessment. The review in which framework synthesis was developed applied ten quality criteria: two on quality and reporting of sampling methods, four to the quality of the description of the sample in the study, two to the reliability and validity of the tools used to collect data and one on whether studies used appropriate methods for helping people to express their views. Studies which did not meet a certain number of quality criteria were excluded from contributing to findings. Similarly, in the example review for thematic synthesis, 12 criteria were applied: five related to reporting aims, context, rationale, methods and findings; four relating to reliability and validity; and three relating to the appropriateness of methods for ensuring that findings were rooted in participants' own perspectives. Studies which were deemed to have significant flaws were excluded and sensitivity analyses were used to assess the possible impact of study quality on the review's findings. Thomas and Harden's use of thematic narrative synthesis similarly applied quality criteria and developed criteria additional to those they found in the literature on quality assessment, relating to the extent to which people's views and perspectives had been privileged by researchers. It is worth noting not only that these methods apply quality criteria but that they are explicit about what they are: assessing quality is a key component in the review process for both of these methods. Likewise, Banning – the originator of ecological triangulation – sees quality assessment as important and adapts the Design and Implementation Assessment Device (DIAD) Version 0.3 (a quality assessment tool for quantitative research) for use when appraising qualitative studies [ 50 ]. Again, Banning writes of excluding studies deemed to be of poor quality.

Greenhalgh et al's meta-narrative review [ 32 ] modified a range of existing quality assessment tools to evaluate studies according to validity and robustness of methods; sample size and power; and validity of conclusions. The authors imply, but are not explicit, that this process formed the basis for the exclusion of some studies. Although not quite so clear about quality assessment methods as framework and thematic synthesis, it might be argued that meta-narrative synthesis shows a greater commitment to the concept that research can and should be assessed for quality than either meta-ethnography or grounded formal theory. The originators of meta-ethnography, Noblit and Hare [ 8 ], originally discussed quality in terms of quality of metaphor, while more recent use of this method has used amended versions of CASP (the Critical Appraisal Skills Programme tool, [ 31 ]), yet has only referred to studies being excluded on the basis of lack of relevance or because they weren't 'qualitative' studies [ 8 ]. In grounded theory, quality assessment is only discussed in terms of a 'personal note' being made on the context, quality and usefulness of each study. However, contrary to expectation, meta-narrative synthesis lies at the extreme end of the idealism/realism spectrum – as a subjective idealist approach – while meta-ethnography and grounded theory are classified as objective idealist approaches.

Finally, meta-study and critical interpretive synthesis – two more subjective idealist approaches – look to the content and utility of findings rather than methodology in order to establish quality. While earlier forms of meta-study included only studies which demonstrated 'epistemological soundness', in its most recent form [ 51 ] this method has sought to include all relevant studies, excluding only those deemed not to be 'qualitative' research. Critical interpretive synthesis also conforms to what we might expect of its approach to quality assessment: quality of research is judged as the extent to which it informs theory. The threshold of inclusion is informed by expertise and instinct rather than being articulated a priori.

In terms of quality assessment, it might be important to consider the academic context in which these various methods of synthesis developed. The reason why thematic synthesis, framework synthesis and ecological triangulation have such highly specified approaches to quality assessment may be that each of these was developed for a particular task, i.e. to conduct a multi-method review in which randomised controlled trials (RCTs) were included. The concept of quality assessment in relation to RCTs is much less contested and there is general agreement on criteria against which quality should be judged.

Problematizing the literature

Critical interpretive synthesis, the meta-narrative approach and the meta-theory element of meta-study all share some common ground in that their review and synthesis processes include examining all aspects of the context in which knowledge is produced. In conducting a review on access to healthcare by vulnerable groups, critical interpretive synthesis sought to question 'the ways in which the literature had constructed the problematics of access, the nature of the assumptions on which it drew, and what has influenced its choice of proposed solutions' [[ 34 ], p6]. Although not claiming to have been directly influenced by Greenhalgh et al's meta-narrative approach, Dixon-Woods et al do cite it as sharing similar characteristics in the sense that it critiques the literature it reviews.

Meta-study uses meta-theory to describe and deconstruct the theories that shape a body of research and to assess its quality. One aspect of this process is to examine the historical evolution of each theory and to put it in its socio-political context, which invites direct comparison with meta-narrative synthesis. Greenhalgh et al put a similar emphasis on placing research findings within their social and historical context, often as a means of seeking to explain heterogeneity of findings. In addition, meta-narrative shares with critical interpretive synthesis an iterative approach to searching and selecting from the literature.

Framework synthesis, thematic synthesis, textual narrative synthesis, meta-ethnography and grounded theory do not share the same approach to problematizing the literature as critical interpretive synthesis, meta-study and meta-narrative. In part, this may be explained by the extent to which studies included in the synthesis represented a broad range of approaches or methodologies. This, in turn, may reflect the broadness of the review question and the extent to which the concepts contained within the question are pre-defined within the literature. In the case of both the critical interpretive synthesis and meta-narrative reviews, terminology was elastic and/or the question formed iteratively. Similarly, both reviews placed great emphasis on employing multi-disciplinary research teams. Approaches which do not critique the literature in the same way tend to have more narrowly-focused questions. They also tend to include a more limited range of studies: grounded theory synthesis includes grounded theory studies, meta-ethnography (in its original form, as applied by Noblit and Hare) ethnographies. The thematic synthesis incorporated studies based on only a narrow range of qualitative methodologies (interviews and focus groups) which were informed by a similarly narrow range of epistemological assumptions. It may be that the authors of such syntheses saw no need for including such a critique in their review process.

Similarities and differences between primary studies

Most methods of synthesis are applicable to heterogeneous data (i.e. studies which use contrasting methodologies) apart from early meta-ethnography and synthesis informed by grounded theory. All methods of synthesis state that, at some level, studies are compared; many are not so explicit about how this is done, though some are. Meta-ethnography is one of the most explicit: it describes the act of 'translation' where terms and concepts which have resonance with one another are subsumed into 'higher order constructs'. Grounded theory, as represented by Eaves [ 17 ], is undertaken according to a long list of steps and sub-steps, includes the production of generalizations about concepts/categories, which comes from classifying these categories. In meta-narrative synthesis, comparable studies are grouped together at the appraisal phase of review.

Perhaps more interesting are the ways in which differences between studies are explored. Those methods with a greater emphasis on critical appraisal may tend (although this is not always made explicit) to use differences in method to explain differences in finding. Meta-ethnography proposes 'refutational synthesis' to explain differences, although there are few examples of this in the literature. Some synthesis methods – for example, thematic synthesis – look at other characteristics of the studies under review, whether types of participants and their context vary, and whether this can explain differences in perspective.

All of these methods, then, look within the studies to explain differences. Other methods look beyond the study itself to the context in which it was produced. Critical interpretive synthesis and meta-study look at differences in theory or in socio-economic context. Critical interpretive synthesis, like meta-narrative, also explores epistemological orientation. Meta-narrative is unique in concerning itself with disciplinary paradigm (i.e. the story of the discipline as it progresses). It is also distinctive in that it treats conflicting findings as 'higher order data' [[ 32 ], p420], so that the main emphasis of the synthesis appears to be on examining and explaining contradictions in the literature.

Going 'beyond' the primary studies

Synthesis is sometimes defined as a process resulting in a product, a 'whole', which is more than the sum of its parts. However, the methods reviewed here vary in the extent to which they attempt to 'go beyond' the primary studies and transform the data. Some methods – textual narrative synthesis, ecological triangulation and framework synthesis – focus on describing and summarising their primary data (often in a highly structured and detailed way) and translating the studies into one another. Others – meta-ethnography, grounded theory, thematic synthesis, meta-study, meta-narrative and critical interpretive synthesis – seek to push beyond the original data to a fresh interpretation of the phenomena under review. A key feature of thematic synthesis is its clear differentiation between these two stages.

Different methods have different mechanisms for going beyond the primary studies, although some are more explicit than others about what these entail. Meta-ethnography proposes a 'Line of Argument' (LOA) synthesis in which an interpretation is constructed to both link and explain a set of parts. Critical interpretive synthesis based its synthesis methods on those of meta-ethnography, developing an LOA using what the authors term 'synthetic constructs' (akin to 'third order constructs' in meta-ethnography) to create a 'synthesising argument'. Dixon-Woods et al claim that this is an advance on Britten et al's methods, in that they reject the difference between first, second and third order constructs.

Meta-narrative, as outlined above, focuses on conflicting findings and constructs theories to explain these in terms of differing paradigms. Meta study derives questions from each of its three components to which it subjects the dataset and inductively generates a number of theoretical claims in relation to it. According to Eaves' model of grounded theory [ 17 ], mini-theories are integrated to produce an explanatory framework. In ecological triangulation, the 'axial' codes – or second level codes evolved from the initial deductive open codes – are used to produce Banning's 'ecological sentence' [ 39 ].

The synthetic product

In overviewing and comparing different qualitative synthesis methods, the ultimate question relates to the utility of the synthetic product: what is it for? It is clear that some methods of synthesis – namely, thematic synthesis, textual narrative synthesis, framework synthesis and ecological triangulation – view themselves as producing an output that is directly applicable to policy makers and designers of interventions. The example of framework synthesis examined here (on children's, young people's and parents' views of walking and cycling) involved policy makers and practitioners in directing the focus of the synthesis and used the themes derived from the synthesis to infer what kind of interventions might be most effective in encouraging walking and cycling. Likewise, the products of the thematic synthesis took the form of practical recommendations for interventions (e.g. 'do not promote fruit and vegetables in the same way in the same intervention'). The extent to which policy makers and practitioners are involved in informing either synthesis or recommendation is less clear from the documents published on ecological triangulation, but the aim certainly is to directly inform practice.

The outputs of synthesis methods which have a more constructivist orientation – meta-study, meta-narrative, meta-ethnography, grounded theory, critical interpretive synthesis – tend to look rather different. They are generally more complex and conceptual, sometimes operating on the symbolic or metaphorical level, and requiring a further process of interpretation by policy makers and practitioners in order for them to inform practice. This is not to say, however, that they are not useful for practice, more that they are doing different work. However, it may be that, in the absence of further interpretation, they are more useful for informing other researchers and theoreticians.

Looking across dimensions

After examining the dimensions of difference of our included methods, what picture ultimately emerges? It seems clear that, while similar in some respects, there are genuine differences in approach to the synthesis of what is essentially textual data. To some extent, these differences can be explained by the epistemological assumptions that underpin each method. Our methods split into two broad camps: the idealist and the realist (see Table Table1 1 for a summary). Idealist approaches generally tend to have a more iterative approach to searching (and the review process), have less a priori quality assessment procedures and are more inclined to problematize the literature. Realist approaches are characterised by a more linear approach to searching and review, have clearer and more well-developed approaches to quality assessment, and do not problematize the literature.

Summary table

N.B.: In terms of the above dimensions, it is generally a question of degree rather than of absolute distinctions.

Mapping the relationships between methods

What is interesting is the relationship between these methods of synthesis, the conceptual links between them, and the extent to which the originators cite – or, in some cases, don't cite – one another. Some methods directly build on others – framework synthesis builds on framework analysis, for example, while grounded theory and constant comparative analysis build on grounded theory. Others further develop existing methods – meta-study, critical interpretive synthesis and meta-narrative all adapt aspects of meta-ethnography, while also importing concepts from other theorists (critical interpretive synthesis also adapts grounded theory techniques).

Some methods share a clear conceptual link, without directly citing one another: for example, the analytical themes developed during thematic synthesis are comparable to the third order interpretations of meta-ethnography. The meta-theory aspect of meta-study is echoed in both meta-narrative synthesis and critical interpretive synthesis (see 'Problematizing the literature, above); however, the originators of critical interpretive synthesis only refer to the originators of meta-study in relation to their use of sampling techniques.

While methods for qualitative synthesis have many similarities, there are clear differences in approach between them, many of which can be explained by taking account of a given method's epistemology.

However, within the two broad idealist/realist categories, any differences between methods in terms of outputs appear to be small.

Since many systematic reviews are designed to inform policy and practice, it is important to select a method – or type of method – that will produce the kind of conclusions needed. However, it is acknowledged that this is not always simple or even possible to achieve in practice.

The approaches that result in more easily translatable messages for policy-makers and practitioners may appear to be more attractive than the others; but we do need to take account lessons from the more idealist end of the spectrum, that some perspectives are not universal.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

Both authors made substantial contributions, with EBP taking a lead on writing and JT on the analytical framework. Both authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/9/59/prepub

Supplementary Material

Dimensions of difference . Ranging from subjective idealism through objective idealism and critical realism to scientific realism to naïve realism

Acknowledgements

The authors would like to acknowledge the helpful contributions of the following in commenting on earlier drafts of this paper: David Gough, Sandy Oliver, Angela Harden, Mary Dixon-Woods, Trisha Greenhalgh and Barbara L. Paterson. We would also like to thank the peer reviewers: Helen J Smith, Rosaline Barbour and Mark Rodgers for their helpful reviews. The methodological development was supported by the Department of Health (England) and the ESRC through the Methods for Research Synthesis Node of the National Centre for Research Methods (NCRM). An earlier draft of this paper currently appears as a working paper on the National Centre for Research Methods' website http://www.ncrm.ac.uk/ .

Dixon-Woods M, Agarwhal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Pol. 2005; 10 (1):45–53b. doi: 10.1258/1355819052801804. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Barbour RS, Barbour M. Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract. 2003; 9 (2):179–186. doi: 10.1046/j.1365-2753.2003.00371.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Pol. 2005; 10 (Suppl 1):6–20. doi: 10.1258/1355819054308576. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Shaw RL, Smith J, Sutton A, Young B. How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006; 6 :27–44. doi: 10.1177/1468794106058867. [ CrossRef ] [ Google Scholar ]
Pope C, Mays N, Popay J. Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. Maidenhead: Open University Press; 2007. [ Google Scholar ]
Thorne S, Jenson L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004; 14 :1342–1365. doi: 10.1177/1049732304269888. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Centre for Reviews and Dissemination. Systematic Reviews. CRD's Guidance for Undertaking Reviews in Health Care. York: CRD; 2008. [ Google Scholar ]
Noblit GW, Hare RD. Meta-Ethnography: Synthesizing Qualitative Studies. London: Sage; 1988. [ Google Scholar ]
Strike K, Posner G. In: Knowledge Structure and Use. Ward S, Reed L, editor. Philadelphia: Temple University Press; 1983. Types of synthesis and their criteria. [ Google Scholar ]
Turner S. Sociological Explanation as Translation. New York: Cambridge University Press; 1980. [ Google Scholar ]
Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R. Using meta-ethnography to synthesis qualitative research: a worked example. J Health Serv Res. 2002; 7 :209–15. doi: 10.1258/135581902320432732. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003; 65 :671–84. doi: 10.1016/S0277-9536(02)00064-3. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R. Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005; 61 :133–155. doi: 10.1016/j.socscimed.2004.11.063. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Schutz A. Collected Paper. Vol. 1. The Hague: Martinus Nijhoff; 1962. [ Google Scholar ]
Sandelowski M, Barroso J. Handbook for Synthesizing Qualitative Research. New York: Springer Publishing Company; 2007. [ Google Scholar ]
Kearney MH. Enduring love: a grounded formal theory of women's experience of domestic violence. Research Nurs Health. 2001; 24 :270–82. doi: 10.1002/nur.1029. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Eaves YD. A synthesis technique for grounded theory data analysis. J Adv Nurs. 2001; 35 :654–63. doi: 10.1046/j.1365-2648.2001.01897.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Finfgeld D. Courage as a process of pushing beyond the struggle. Qual Health Res. 1999; 9 :803–814. doi: 10.1177/104973299129122298. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine De Gruyter; 1967. [ Google Scholar ]
Strauss AL, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage; 1990. [ Google Scholar ]
Strauss AL, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Thousand Oaks, CA: Sage; 1998. [ Google Scholar ]
Charmaz K. In: Contemporary Field Research: A Collection of Readings. Emerson RM, editor. Waveland Press: Prospect Heights, IL; 1983. The grounded theory method: an explication and interpretation; pp. 109–126. [ Google Scholar ]
Chesler MA. Professionals' Views of the Dangers of Self-Help Groups: Explicating a Grounded Theoretical Approach. [Michigan]: Department of Sociology, University of Michigan, Ann Arbour Centre for Research on Social Organisation, Working Paper Series; 1987. [ Google Scholar ]
Kearney MH. Ready-to-wear: discovering grounded formal theory. Res Nurs Health. 1988; 21 :179–186. doi: 10.1002/(SICI)1098-240X(199804)21:2<179::AID-NUR8>3.0.CO;2-G. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth. 2008; 8 :45. doi: 10.1186/1471-2288-8-45. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lucas PJ, Arai L, Baird, Law C, Roberts HM. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Meth. 2007; 7 (4) [ PMC free article ] [ PubMed ] [ Google Scholar ]
Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A. Applying systematic review methods to studies of people's views: an example from public health research. J Epidemiol Community H. 2004; 58 :794–800. doi: 10.1136/jech.2003.014829. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Paterson BL, Thorne SE, Canam C, Jillings C. Meta-Study of Qualitative Health Research. A Practical Guide to Meta-Analysis and Meta-Synthesis. Thousand Oaks, CA: Sage Publications; 2001. [ Google Scholar ]
Zhao S. Metatheory, metamethod, meta-data-analysis: what, why and how? Sociol Perspect. 1991; 34 :377–390. [ Google Scholar ]
Ritzer G. Metatheorizing in Sociology. Lexington, MA: Lexington Books; 1991. [ Google Scholar ]
CASP (Critical Appraisal Skills Programme) http://www.phru.nhs.uk/Pages/PHD/CASP.htm date unknown.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005; 61 :417–30. doi: 10.1016/j.socscimed.2004.12.001. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Kuhn TS. The Structure of Scientific Revolutions. Chicago: University of Chicago Press; 1962. [ Google Scholar ]
Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Meth. 2006; 6 (35) [ PMC free article ] [ PubMed ] [ Google Scholar ]
Gough D. In: Applied and Practice-based Research. 2. Furlong J, Oancea A, editor. Vol. 22. Special Edition of Research Papers in Education; 2007. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence; pp. 213–228. [ Google Scholar ]
Webb EJ, Campbell DT, Schwartz RD, Sechrest L. Unobtrusive Measures. Chicago: Rand McNally; 1966. [ Google Scholar ]
Denzin NK. The Research Act: a Theoretical Introduction to Sociological Methods. New York: McGraw-Hill; 1978. [ Google Scholar ]
Banning J. Ecological Triangulation. http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Triangualtion.pdf
Banning J. Ecological Sentence Synthesis. http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Sentence%20Synthesis.pdf
Brunton G, Oliver S, Oliver K, Lorenc T. A Synthesis of Research Addressing Children's, Young People's and Parents' Views of Walking and Cycling for Transport. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2006. [ Google Scholar ]
Oliver S, Rees R, Clarke-Jones L, Milne R, Oakley A, Gabbay J, Stein K, Buchanan P, Gyte G. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 2008; 11 :72–84. doi: 10.1111/j.1369-7625.2007.00476.x. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative data. BMJ. 2000; 320 :114–116. doi: 10.1136/bmj.320.7227.114. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Ritchie J, Spencer L. In: Analysing Qualitative Data. Bryman A, Burgess R, editor. London: Routledge; 1993. Qualitative data analysis for applied policy research; pp. 173–194. [ Google Scholar ]
Miles M, Huberman A. Qualitative Data Analysis. London: Sage; 1984. [ Google Scholar ]
Evans D, Fitzgerald M. Reasons for physically restraining patients and residents: a systematic review and content analysis. Int J Nurs Stud. 2002; 39 :739–743. doi: 10.1016/S0020-7489(02)00015-9. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Suikkala A, Leino-Kilpi H. Nursing student-patient relationships: a review of the literature from 1984–1998. J Adv Nurs. 2000; 33 :42–50. doi: 10.1046/j.1365-2648.2001.01636.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Weed M. 'Meta-interpretation': a method for the interpretive synthesis of qualitative research. Forum: Qual Soc Res. 2005; 6 :Art 37. [ Google Scholar ]
Gough D, Thomas J. Dimensions of difference in systematic reviews. http://www.ncrm.ac.uk/RMF2008/festival/programme/sys1
Spencer L, Ritchie J, Lewis J, Dillon L. Quality in Qualitative Evaluation: a Framework for Assessing Research Evidence. London: Government Chief Social Researcher's Office; 2003. [ Google Scholar ]
Banning J. Design and Implementation Assessment Device (DIAD) Version 0.3: A response from a qualitative perspective. http://mycahs.colostate.edu/James.H.Banning/PDFs/Design%20and%20Implementation%20Assessment%20Device.pdf
Paterson BL. In: Reviewing Research Evidence for Nursing Practice. Webb C, Roe B, editor. [Oxford]: Blackwell Publishing Ltd; 2007. Coming out as ill: understanding self-disclosure in chronic illness from a meta-synthesis of qualitative research; pp. 73–83. [ Google Scholar ]

Open access
Published: 03 April 2024

Capturing artificial intelligence applications’ value proposition in healthcare – a qualitative research study

Jasmin Hennrich 1 ,
Eva Ritz 2 ,
Peter Hofmann 1 , 4 &
Nils Urbach 1 , 3

BMC Health Services Research volume 24 , Article number: 420 ( 2024 ) Cite this article

979 Accesses

Metrics details

Artificial intelligence (AI) applications pave the way for innovations in the healthcare (HC) industry. However, their adoption in HC organizations is still nascent as organizations often face a fragmented and incomplete picture of how they can capture the value of AI applications on a managerial level. To overcome adoption hurdles, HC organizations would benefit from understanding how they can capture AI applications’ potential.

We conduct a comprehensive systematic literature review and 11 semi-structured expert interviews to identify, systematize, and describe 15 business objectives that translate into six value propositions of AI applications in HC.

Our results demonstrate that AI applications can have several business objectives converging into risk-reduced patient care, advanced patient care, self-management, process acceleration, resource optimization, and knowledge discovery.

We contribute to the literature by extending research on value creation mechanisms of AI to the HC context and guiding HC organizations in evaluating their AI applications or those of the competition on a managerial level, to assess AI investment decisions, and to align their AI application portfolio towards an overarching strategy.

Peer Review reports

Applications based on artificial intelligence (AI) have the potential to transform the healthcare (HC) industry [ 1 ]. AI applications can be characterized as applications or agents with capabilities that typically demand intelligence [ 2 , 3 ]. In our context, we understand AI as a collection of technological solutions from the field of applied computer science, in which algorithms are trained on medical and HC data to perform tasks that are normally associated with human intelligence (i.e., medical decision-making) [ 4 ]. AI is not a single type of technology, instead, it encompasses a diverse array of technologies spread across various application areas in HC, such as diagnostics (e.g., [ 5 ], biomedical research (e.g., [ 6 ], clinical administration (e.g., [ 7 ], therapy (e.g., [ 8 ], and intelligent robotics (e.g., [ 9 ]. These areas are expected to benefit from AI applications’ capabilities, such as accuracy, objectivity, rapidity, data processing, and automation [ 10 , 11 ]. Accordingly, AI applications are said to have the potential to drive business value and enhance HC [ 12 ], paving the way for transformative innovations in the HC industry [ 13 ]. There are already many promising AI use cases in HC that are expected to improve patient care and create value for HC organizations. For instance, AI applications can advance the quality of patient care by supporting radiologists with more accurate and rapid diagnosis, compensating for humans’ limitations (e.g., data processing speeds) and weaknesses (e.g., inattention, distraction, and fatigue) [ 10 , 14 ]. Klicken oder tippen Sie hier, um Text einzugeben.While the use of AI applications in HC has the overarching goal of creating significant value for patients through improved care, they also come with the potential for business value creation and the opportunity for HC organizations to gain a competitive edge (e.g., [ 15 , 16 ]).

Despite the promised advantages, AI applications’ implementation is slow, and the full realization of their potential within the HC industry is yet to be achieved [ 11 , 17 ]. With just a handful of practical examples of AI applications in the HC industry [ 13 , 18 ], the adoption of AI applications is still in its infancy. The AI in Healthcare Survey Report stated that in 2021, only 9% of respondents worldwide have reached a sophisticated adoption of AI Models, while 32% of respondents are still in the early stages of adopting AI models. According to the survey, the majority of HC organizations (60%) are not actively considering AI as a solution, or they are currently evaluating AI use cases and experimenting with the implementation [ 19 ]. Nevertheless, HC startups are increasingly entering the market [ 20 ], pressuring incumbent HC organizations to evaluate and adopt AI applications. Existing studies already investigate AI technologies in various use cases in HC and provide insights on how to design AI-based services [ 21 ], explain in detail the technical functions and capabilities of AI technologies [ 10 , 11 ], or take on a practical perspective with a focus on concrete examples of AI applications [ 14 ]. However, to foster the adoption of AI applications, HC organizations should understand how they can unfold AI applications’ capabilities into business value to ensure effective investments. Previous studies on the intersection of information systems and value creation have expressed interest into how organizations can actually gain value through the use of technology and thus, enhance their adoption [ 22 , 23 ]. However, to the best of our knowledge, a comprehensive investigation of the value creation of AI applications in the context of HC from a managerial level is currently missing. Thus, our study aims to investigate AI applications’ value creation and capture mechanisms in the specific HC context by answering the following question: How can HC organizations create and capture AI applications’ value?

We conduct a systematic literature analysis and semi structured expert interviews to answer this research question. In the systematic literature analysis, we identify and analyze a heterogeneous set of 21 AI use cases across five different HC application fields and derive 15 business objectives and six value propositions for HC organizations. We then evaluate and refine the categorized business objectives and value propositions with insights from 11 expert interviews. Our study contributes to research on the value creation mechanism of AI applications in the HC context. Moreover, our results have managerial implications for HC organizations since they can draw on our results to evaluate AI applications, assess investment decisions, and align their AI application portfolio toward an overarching strategy.

In what follows, this study first grounds on relevant work to gain a deeper understanding of the underlying constructs of AI in HC. Next, we describe our qualitative research method by describing the process of data collection and analysis, followed by our derived results on capturing AI applications’ value proposition in HC. Afterward, we discuss our results, including this study’s limitations and pathways for further research. Finally, we summarize our findings and their contribution to theory and practice in the conclusion.

Relevant work

In the realm of AI, a thorough exploration of its key subdiscipline, machine learning (ML), is essential [ 24 , 25 ]. ML is a computational model that learns from data without explicitly programming the data [ 24 ] and can be further divided into supervised, unsupervised, and reinforcement learning [ 26 ]. In supervised learning, the machine undergoes training with labeled data, making it well-suited for tasks involving regression and classification problems [ 27 ]. In contrast, unsupervised learning is designed to automatically identify patterns within unlabeled datasets [ 28 ], with its primary utility lying in the extraction of features [ 11 ]. Reinforcement learning, characterized as a method of systematic experimentation or trial and error, involves a situated agent taking specific actions and observing the rewards it gains from those actions, facilitating the learning of behavior in a given environment [ 29 ]. The choice of which type of ML will be used in the different application areas depends on the specific problem, the availability of labeled data, and the nature of the desired outcome.

In recent years, the rapid advances in AI have triggered a revolution in various areas, with numerous impressive advantages. In the financial sector, AI applications can significantly improve security by detecting anomalies and preventing fraud [ 30 ]. Within education, AI has emerged as a powerful tool for tailoring learning experiences, aiming to enhance engagement, understanding, and retention [ 31 ]. In the energy market, the efficacy of AI extends to fault detection and diagnosis in building energy systems, showcasing its robust capabilities in ensuring system integrity [ 32 ]. Moreover, the HC industry is expected to be a promising application area for AI applications. The HC sector is undergoing a significant transformation due to the increasing adoption of digital technologies, with AI technologies at the forefront of this shift. The increasing relevance of AI technologies in HC is underlined by a growing and multidisciplinary stream of AI research, as highlighted by Secinaro et al. [ 33 ]. Taking a closer look at the different application areas in HC, AI applications offer promising potential, as demonstrated by the following exemplary AI use cases. In diagnosis, AI applications can identify complex patterns in medical image data more accurately, resulting in precise and objective disease recognition. This can improve patient safety by reducing the risks of misinterpretation [ 5 ]. Another use case can be found in biomedical research. For example, AI technology is commonly used for de novo drug design. AI can rapidly browse through molecule libraries to detect nearly \({10}^{60}\) drug-like molecules, accelerating the drug development process [ 6 ]. Furthermore, AI applications are used in clinical administration. They enable optimized operation room capacities by automating the process and by including information about absence or waiting times, as well as predicting interruptions [ 34 ]. Furthermore, AI applications are used in therapy by predicting personalized medication dosages. As this helps to reduce the mortality risk, it leads to enhanced patient outcomes and quality of care [ 35 ]. Intelligent prostheses by which patients can improve interactions are another use case. The AI algorithm continuously detects and classifies myoelectric signal patterns to predict movements, leading to reduced training expenditure and more self-management by the patient [ 36 ]. In summary, envisioning that AI applications successfully address persisting challenges, such as lack of transparency (e.g., [ 37 ], bias (e.g., [ 38 ], privacy concerns, and trust issues (e.g., [ 39 ], the potential of AI applications is vast. The conceivable benefits extend to individual practitioners and HC organizations, including hospitals, enabling them to harness AI applications for creating business value and ultimately enhancing competitiveness. Thereby, we follow Schryen’s (p. 141) revisited definition of business value of technologies: “the impact of investments on the multidimensional performance and capabilities of economic entities at various levels, complemented by the ultimate meaning of performance in the economic environment” [ 40 ]. His perspective includes all kinds of tangible value (such as an increase in productivity or reduced costs) to intangible value (such as service innovation or customer satisfaction), as well as internal value for the HC organizations and external value for stakeholders, shareholders, and customers. To create business value, it is essential to have a clear understanding of how the potential of AI applications can be captured. The understanding of how information systems, in general, create value is already covered in the literature. For example, Badakhshan et al. [ 31 ] focus on how process mining can pave the way to create business value. Leidner et al. [ 32 ] examine how enterprise social media adds value for new employees, and Lehrer et al. [ 33 ] answer the question of how big data analytics can enable service. There are also studies focusing on the value creation of information systems in the context of HC. For instance, the study by Haddad and Wickramasinghe [ 41 ] shows that information technology in HC can capture value by improving the quality of HC delivery, increasing safety, or offering additional services. Strong et al. [ 42 ] analyze how electronic health records afford value for HC organizations and determine goal-oriented actions to capture this potential. There is even literature on how machine learning adds value within the discipline of radiology (e.g., [ 43 ].

However, these studies either do not address the context of HC, consider technologies other than AI or information systems in general, or focus only on a small area of HC (e.g., radiology) and a subset of AI technology (e.g., machine learning). Although these studies deliver valuable insights into the value creation of information systems, a comprehensive picture of how HC organizations can capture business value with AI applications is missing.

To answer our research question, we adopted a qualitative inductive research design. This research design is consistent with studies that took a similar perspective on how technologies can create business value [ 44 ]. In conducting our structured literature review, we followed the approach of Webster and Watson [ 45 ] and included recommendations of Wolfswinkel et al. [ 46 ] when considering the inclusion and exclusion criteria. We started by collecting relevant data on different successful AI use cases across five application areas in HC. Siggelkow [ 47 ] argued that use cases are able to provide persuasive arguments for causal relationships. In an initial literature screening, we identified five promising application domains focusing on AI applications for patients and HC providers: disease diagnostics (DD) (e.g., [ 5 ], biomedical research (BR) (e.g., [ 6 ], clinical administration (CA) (e.g., [ 7 ], therapy (T) (e.g., [ 8 ], and intelligent robotics (IR) (e.g., [ 9 ]. Second, to sample AI use cases, we aimed to collect a heterogeneous set of AI use cases within these application domains and consider the heterogeneity in AI applications, underlying data, innovation types, and implementation stages when selecting 21 AI use cases for our in-depth analysis. The AI use case and an exemplary study for each use case are listed in Table 1 .

After sampling the AI use cases, we used PubMed to identify papers for each use case. PubMed is recognized as a common database for biomedical and medical research for HC topics in the information systems domain (e.g., [ 62 , 63 ]. Our search included journal articles, clinical conferences, clinical studies, and comparative studies in English as of 2010. Based on the AI use case sample, we derived a search string based on keywords [ 45 ] considering titles and abstracts by following Shepherd et al. [ 62 ] guidelines. It was aimed to narrow and specific selection to increase data collection replicability for the use cases. Boolean operators (AND, OR) are used to improve results by combining search terms [ 62 ].

((artificial intelligence AND (radiology OR (cancer AND imaging) OR (radiology AND error) OR (cancer AND genomics) OR (speech AND cognitive AND impairment) OR (voice AND parkinson) OR EEG OR (facial AND analysis) OR (drug AND design) OR (Drug AND Biomarker) OR De-identification OR Splicing OR (emergency AND triage) OR (mortality AND prediction) OR (operating AND room) OR text summarization OR (artificial AND pancreas) OR vasopressor OR Chatbot OR (myoelectric prosthesis) OR (automated surgery task) OR (surgery AND workflow)))

The initial search led to 877 results (see Fig. 1 ). After title screening, we eliminated 516 papers that are not relevant (i.e., not covering a specific AI application, only including the description of AI algorithm, or not including a managerial perspective and the value created by AI applications). We further excluded 162 papers because their abstract is not concurrent with any specific use case (e.g., because they were literature reviews on overarching topics and did not include a specific AI application). We screened the remaining 199 papers for eligibility through two content-related criteria. First, papers need to cover an AI use case’s whole value proposition creation path, including information on data, algorithms, functions, competitive advantage, and business value of a certain AI application. The papers often only examine how a certain application works but lack the value proposition perspective, which leads to the exclusion of 63 articles. Second, we removed 89 papers that do not match any of our use cases. This step led to a remaining set of 47 relevant papers. During a backward-forward search according to Webster and Watson [ 45 ] and Levy and Ellis [ 64 ], we additionally included 35 papers. We also incorporated previous and subsequent clinical studies of the same researcher, resulting in an additional six papers. The final set contains 88 relevant papers describing the identified AI use cases, whereby at least three papers describe each AI use case.

Search strategy

In the second step, we engaged in open, axial, and selective coding of the AI use cases following analysis techniques of grounded theory [ 65 ]. We focused on extracting business objectives, detailing how each AI application drives value. We documented these for each AI use case by recording codes of business objectives and value propositions and assigning relationships among the open codes. For example, from the following text passage of Berlyand et al. [ 56 ], who investigate the use case CA1: “Rapidly interpreting clinical data to classify patients and predict outcomes is paramount to emergency department operations, with direct impacts on cost, efficiency, and quality of care”, we derived the code rapid task execution.

After analyzing the AI use cases, we revised the documented tuples to foster consistency and comparability. Then, we iteratively coded the identified tuples by relying on selective coding techniques which is a process to identify and refine categories at a highly generalizable degree [ 65 ]. In all 14 coding iterations, one author continuously compares, relates, and associates categories and properties and discusses the coding results with another author. We modified some tuples during the coding process in two ways. First, we equalized small phrasing disparities for homogenous and refined wording. Second, we carefully adjusted the tuples regarding coherency. Finally, we reviewed the coding schema for internal validity through a final comparison with the data [ 66 ]. Then, we set the core variables “business objectives” and “value propositions”. We refer to business objectives as improvements through implementing the technology that drives a value proposition. We define value proposition as the inherent commitment to deliver reciprocal value to the organization, its customers, and/or partners [ 67 ].

In the third step following Schultze and Avital [ 68 ], we conducted semi structured expert interviews to evaluate and refine the value propositions and business objectives. We developed and refined an interview script following the guidelines of Meyers and Newman [ 69 ] for qualitative interviews. An additional file shows the used interview script (see Additional file 1 ). We conducted expert sampling to select suitable interviewees [ 70 ]. Due to the interdisciplinarity of the research topic, we chose experts in the two knowledge areas, AI and HC. In the process of expert selection, we ensured that interviewees possessed a minimum of two years of experience in their respective fields. We aimed for a well-balanced mix of diverse professions and positions among the interviewees. Additionally, for those with a primary background in HC, we specifically verified their proficiency and understanding of AI, ensuring a comprehensive perspective across the entire expert panel. Table 2 provides an overview of our expert sample. The interviewees were recruited in the authors’ networks and by cold calling. Identified experts were first contacted by email, including some brief information regarding the study. If there was no response within two weeks, they were contacted again by telephone to arrange an interview date. In total, we conducted 11 interviews that took place in a time range between 40 and 75 min. The expert interviews are transcribed verbatim using the software f4. As a coding aid, we use the software MAXQDA—a tool for qualitative data analysis which is frequently used in the analyses of qualitative data in the HC domain (e.g., [ 38 , 71 , 72 ]).

To systematically decompose how HC organizations can realize value propositions from AI applications, we identified 15 business objectives and six value propositions (see Fig. 2 ). These business objectives and value propositions resulted from analyzing the collected data, which we derived from the literature and refined through expert interviews. In the following, we describe the six value propositions and elaborate on how the specific AI business objectives can result in value propositions. This will be followed by a discussion of the results in the discussion of the paper.

Business objectives and value propositions risk-reduced patient care

This value proposition follows business objectives that may identify and reduce threats and adverse factors during medical procedures. HC belongs to a high-risk domain since there are uncertain external factors (E4), including physicians’ fatigue, distractions, or cognitive biases [ 73 , 74 ]. AI applications can reduce certain risks by enabling precise decision support, detecting misconduct, reducing emergent side effects, and reducing invasiveness.

Precise decision support stems from AI applications’ capability to integrate various data types into the decision-making process, gaining a sophisticated overview of a phenomenon. Precise knowledge about all uncertainty factors reduces the ambiguity of decision-making processes [ 49 ]. E5 confirms that AI applications can be seen as a “perceptual enhancement”, enabling more comprehensive and context-based decision support. Humans are naturally prone to innate and socially adapted biases that also affect HC professionals [ 14 ]. Use Case CA1 highlights how rapid decision-making by HC professionals during emergency triage may lead to overlooking subtle yet crucial signs. AI applications can offer decision support based on historical data, enhancing objectivity and accuracy [ 56 ].

Detection of misconduct is possible since AI applications can map and monitor clinical workflows and recognize irregularities early. In this context, E10 highlights that “one of the best examples is the interception of abnormalities.” For instance, AI applications can assist in allocating medications in hospitals (Use case T2). Since HC professionals can be tired or distracted in medication preparation, AI applications may avoid serious consequences for patients by monitoring allocation processes and patients’ reactions. Thus, AI applications can reduce abuse and increase safety.

Reduction of emergent side effects is enabled by AI applications that continuously monitor and process data. If different treatments and medications are combined during a patient’s clinical pathway, it may cause overdosage or evoke co-effects and comorbidities, causing danger for the patient [ 75 ]. AI applications can prevent these by detecting and predicting these effects. For instance, AI applications can calculate the medication dosage for the individual and predict contraindications (Use case T2) [ 76 ]. E3 adds that the reduction of side effects also includes “cross-impacts between medications or possible symptoms that only occur for patients of a certain age or disease.” Avoidable side effects can thus be detected at an early stage, resulting in better outcomes.

Reduction of invasiveness of medical treatments or surgeries is possible by allowing AI applications to compensate for and overcome human weaknesses and limitations. During surgery, AI applications can continuously monitor a robot’s position and accurately predict its trajectories [ 77 ]. Intelligent robots can eliminate human tremors and access hard-to-reach body parts [ 60 ]. E2 validates, “a robot does not tremble; a robot moves in a perfectly straight line.” The precise AI-controlled movement of surgical robots minimizes the risk of injuring nearby vessels and organs [ 61 ]. Use cases DD5 and DD7 elucidate how AI applications enable new methods to perform noninvasive diagnoses. Reducing invasiveness has a major impact on the patient’s recovery, safety, and outcome quality.

Advanced patient care

Advanced patient care follows business objectives that extend patient care to increase the quality of care. One of HC’s primary goals is to provide the most effective treatment outcome. AI applications can advance patient care as they enable personalized care and accurate prognosis.

Personalized care can be enabled by the ability of AI technologies to integrate and process individual structured and unstructured patient data to increase the compatibility of patient and health interventions. For instance, by analyzing genome mutations, AI applications precisely assess cancer, enabling personalized therapy and increasing the likelihood of enhancing outcome quality (Use case DD4). E11 sums up that “we can improve treatment or even make it more specific for the patient. This is, of course, the dream of healthcare”. Use case T1 exemplifies how the integration of AI applications facilitates personalized products, such as an artificial pancreas. The pancreas predicts glucose levels in real time and adapts insulin supplementation. Personalized care allows good care to be made even better by tailoring care to the individual.

Accurate prognosis is achieved by AI applications that track, combine, and analyze HC data and historical data to make accurate predictions. For instance, AI applications can precisely analyze tumor tissue to improve the stratification of cancer patients. Based on this result, the selection of adjuvant therapy can be refined, improving the effectiveness of care [ 48 ]. Use case DD6 shows how AI applications can predict seizure onset zones to enhance the prognosis of epileptic seizures. In this context, E10 adds that an accurate prognosis fosters early and preventive care.

Self-management

Self-management follows the business objectives that increase disease controllability through the support of intelligent medical products. AI applications can foster self-management by self-monitoring and providing a new way of delivering information.

Self-monitoring is enhanced by AI applications, which can automatically process frequently measured data. There are AI-based chatbots, mobile applications, wearables, and other medical products that gather periodic data and are used by people to monitor themselves in the health context (e.g., [ 78 , 79 ]. Frequent data collection of these products (e.g., using sensors) enables AI applications to analyze periodic data and become aware of abnormalities. While the amount of data rises, the applications can improve their performance continuously (E2). Through continuous tracking of heartbeats via wearables, AI applications can precisely detect irregularities, notify their users in the case of irregularities, empower quicker treatment (E2), and may reduce hospital visits (E9). Self-monitoring enhances patient safety and allows the patient to be more physician-independent and involved in their HC.

Information delivery to the patient is enabled by AI applications that give medical advice adjusted to the patient’s needs. Often, patients lack profound knowledge about their anomalies. AI applications can contextualize patients’ symptoms to provide anamnesis support and deliver interactive advice [ 59 ]. While HC professionals must focus on one diagnostic pathway, AI applications can process information to investigate different diagnostic branches simultaneously (E5). Thus, these applications can deliver high-quality information based on the patient’s feedback, for instance, when using an intelligent conversational agent (use case T3). E4 highlights that this can improve doctoral consultations because “the patient is already informed and already has information when he comes to talk to doctors”.

Process acceleration

Process acceleration comprises business objectives that enable speed and low latencies. Speed describes how fast one can perform a task, while latency specifies how much time elapses from an event until a task is executed. AI applications can accelerate processes by rapid task execution and reducing latency.

Rapid task execution can be achieved by the ability of AI applications to process large amounts of data and identify patterns in a short time. In this context, E4 mentions that AI applications can drill diagnosis down to seconds. For instance, whereas doctors need several minutes for profound image-based detection, AI applications have a much faster report turnaround time (use case DD1). Besides, rapid data processing also opens up new opportunities in drug development. AI applications can rapidly browse through molecule libraries to detect nearly 10^60 molecules, which are synthetically available (use case BR1). This immense speed during a discovery process has an essential influence on the business potential and can enormously decrease research costs (E10).

Latency reduction can be enabled by AI technologies monitoring and dynamically processing information and environmental factors. By continuously evaluating vital signs and electrocardiogram records, AI applications can predict the in-house mortality of patients in real time [ 57 ]. The AI application can detect an increased mortality risk faster than HC professionals, enabling a more rapid emergency intervention. In this case, AI applications decrease the time delay between the cause and the reaction, which positively impacts patient care. E7 emphasizes the importance of short latencies: “One of the most important things is that the timeframe between the point when all the data is available, and a decision has been made, […] must be kept short.”

Resource optimization

Resource optimization follows the business objectives that manage limited resources and capacities. The HC industry faces a lack of sufficient resources, especially through a shortage of specialists (E8), which in turn negatively influences waiting times. AI applications can support efficient resource allocation by optimizing device utilization, organizational capacities and unleashing personnel capabilities.

Optimized device utilization can be enhanced by AI applications that track, analyze, and precisely predict load of times of medical equipment in real-time. For instance, AI applications can maximize X-Ray or magnetic resonance tomography device utilization (use case CA3). Besides, AI applications can enable a dynamic replanning of device utilization by including absence or waiting times and predicting interruptions. Intelligent resource optimization may include various key variables (e.g., the maximized lifespan of a radiation scanner) [ 48 ]. Optimized device utilization reduces the time periods when the device is not utilized, and thus, losses are made.

Optimized organizational capacities are possible due to AI applications breaking up static key performance indicators and finding more dynamic measuring approaches for the required workflow changes (E5, E10). The utilization of capacities in hospitals relies on various known and unknown parameters, which are often interdependent [ 80 ]. AI applications can detect and optimize these dependencies to manage capacity. An example is the optimization of clinical occupancy in the hospital (use case CA3), which has a strong impact on cost. E5 adds that the integration of AI applications may increase the reliability of planning HC resources since they can predict capacity trends from historical occupancy rates. Optimized planning of capacities can prevent capacities from remaining unused and fixed costs from being offset by no revenue.

Unleashing personnel capabilities is enabled by AI applications performing analytical and administrative tasks, relieving caregivers’ workload (E8, E10, E11). E7 validates that “our conviction is […] that administrational tasks generate the greatest added value and benefit for doctors and caregivers.” Administrative tasks include the creation of case summaries (use case CA4) or automated de-identification of private health information in electronic health records (use case BR2) [ 54 ]. E8 says that resource optimization enables “more time for direct contact with patients.”

Knowledge discovery

Knowledge discovery follows the business objectives that increase perception and access to novel and previously unrevealed information. AI applications might synthesize and contextualize medical knowledge to create uniform or equalized semantics of information (E5, E11). This semantics enables a translation of knowledge for specific users.

Detection of similarities is enabled by AI applications identifying entities with similar features. AI applications can screen complex and nonlinear databases to identify reoccurring patterns without any a priori understanding of the data (E3). These similarities generate valuable knowledge, which can be applied to enhance scientific research processes such as drug development (use case BR1). In drug development, AI applications can facilitate ligand-based screening to detect new active molecules based on similarities compared with already existing molecular properties. This increases the effectiveness of drug design and reduces risks in clinical trials [ 6 ].

Exploration of new correlations is facilitated by AI applications identifying relationships in data. In diagnostics, AI applications can analyze facial photographs to accurately identify genotype–phenotype correlations and, thus, increase the detection rate of rare diseases (use case DD7). E8 states the potential of AI applications in the field of knowledge discovery: “Well, if you are researching in any medical area, then everybody aims to understand and describe phenomena because science always demands a certain causation.” However, it is crucial to develop transparent and intelligible inferences that are comprehensible for HC professionals and researchers. Exploring new correlations improves diagnoses of rare diseases and ensures earlier treatment.

After describing each business objective and value proposition, we summarize the AI use cases’ contributions to the value propositions in Table 3 .

By revealing 15 business objectives that translate into six value propositions, we contribute to the academic discourse on the value creation of AI (e.g. [ 81 ] and provide prescriptive knowledge on AI applications' value propositions in the HC domain. Our discourse also emphasizes that our findings are not only relevant to the field of value creation research but can also be helpful for adoption research. The value propositions we have identified can be a good starting point to accelerate the adoption of AI in HC, as the understanding of potential value propositions that we foster could mitigate some of the current obstacles to the adoption of AI applications in HC. For example, our findings may help to mitigate the obstacle “added value”, which is presented in the study by Hennrich et al.38 [ 38 ] as users’ concerns that AI might create more burden than benefits.

Further, we deliver valuable implications for practice and provide a comprehensive picture of how organizations in the context of HC can achieve business value with AI applications from a managerial level, which has been missing until now. We guide HC organizations in evaluating their AI applications or those of the competition to assess AI investment decisions and align their AI application portfolio toward an overarching strategy. These results will foster the adoption of AI applications as HC organizations can now understand how they can unfold AI applications’ capabilities into business value. In case a hospital’s major strategy is to reduce patient risks due to limited personal capacities, it might be beneficial for them to invest in AI applications that reduce side effects by calculating medication dosages (use case T2). If an HC organization currently faces issues with overcrowded emergency rooms, the HC organization might acquire AI applications that increase information delivery and help patients decide if and when they should visit the hospital (use case T3) to increase patients’ self-management and, in turn, improve triage. Besides, our findings also offer valuable insights for AI developers. Addressing issues such as transparency and the alignment of AI applications with the needs of HC professionals is crucial. Adapting AI solutions to the specific requirements of the HC sector ensures responsible integration and thus the realization of the expected values.

A closer look at the current challenges in the HC sector reveals that new solutions to mitigate them and improve value creation are needed. Given that a nurse, for example, dedicates a substantial 25% of their working hours to administrative tasks [ 17 ], the rationale behind the respondents’ (E7) recognition of “the greatest added value” in utilizing AI applications for administrative purposes becomes evident. The potential of AI applications in streamlining administrative tasks lies in creating additional time for meaningful patient interactions. Acknowledging the significant impact of the doctor-patient interpersonal relationship on both the patient’s well-being and the processes of diagnosis and healing, as elucidated by Buck et al. [ 82 ] in their interview study, the physicians interviewed emphasized that the mere presence of the doctor in the same room often alleviates the patient’s problems. Consequently, it becomes apparent that the intangible value of AI applications plays a crucial role in the context of HC and is an important factor in the investment decision as to where an AI application should be deployed.

The interviews also indicate that the special context of the HC sector leads to concerns regarding the use of AI applications. For example, one interviewee emphasized a fundamental characteristic of medical staff by pointing out that physicians have a natural desire to understand all phenomena (E8). AI applications, however, are currently struggling with the challenge of transparency. This challenge is described by the so-called black box problem, a phenomenon that makes it impossible to decipher the underlying algorithms that lead to a particular recommendation [ 37 ]. The lack of transparency and the resulting lack of intervention options for medical staff can lead to incorrect decisions by the AI application, which may cause considerable damage. Aware of these risks, physicians are currently struggling with trust issues in AI applications [ 72 ]. The numerous opportunities for value creation through AI applications in HC are offset by the significant risk of causing considerable harm to patients if the technology is not yet fully mature. Ultimately, it remains essential to keep in mind that there are many ethical questions to be answered [ 83 ], and AI applications are still facing many obstacles [ 38 ] that must be overcome in order to realize the expected values and avoid serious harm. One important first step in mitigating the obstacles is disseminating the concerns and risks to relevant stakeholders, emphasizing the urgency for collaborative scientific and public monitoring efforts [ 84 ]. However, keeping these obstacles in mind, by providing prescriptive knowledge, we enhance the understanding of AI’s value creation paths in the HC industry and thus help to drive AI integration forward. For example, looking at the value proposition risk reduced patient care , we demonstrate that this value proposition is determined by four business objectives: precise decision support , detection of misconduct, reduction of side effects, and reduction of invasiveness . Similarly, the AI application’s capability to analyze data more accurately in diagnosis (use case DD1) enables the business objective precise decision support , thereby reducing risks in patient care. Another mechanism can be seen, for example, considering the business objective task execution , which leads to the value proposition process acceleration . The ability of AI applications to rapidly analyze large amounts of data and recognize patterns in biomedical research (use case BR1) allows a faster drug development process.

Further research

By investigating the value creation mechanism of AI applications for HC organizations, we not only make an important contribution to research and practice but also create a valuable foundation for future studies. While we have systematically identified the relations between the business objectives and value propositions, further research is needed to investigate how the business objectives themselves are determined. While the examination of AI capabilities was not the primary research focus, we found first evidence in the use cases that indicates AI technology’s unique capabilities (e.g., to make diagnoses accurate, faster, and more objective) that foster one or several business objectives (e.g., rapid task execution, precise decision support) and unlock one or several value propositions (e.g., Risk-reduced patient care, process acceleration ). In subsequent research, we aim to integrate these into the value creation mechanism by identifying which specific AI capabilities drive business objectives, thereby advancing the understanding of how AI applications in HC create value propositions.

Limitations

This study is subject to certain limitations of methodological and conceptual nature. First, while our methodological approach covers an in-depth analysis of 21 AI use cases, extending the sample of AI use cases would foster the generalizability of the results. This is especially important regarding the latest developments on generative AI and its newcoming use cases. However, our results demonstrate that these AI use cases already provide rich information to derive 15 business objectives, which translate into six value propositions. Second, while many papers assume the potential of AI applications to create value propositions, only a few papers explicitly focus on the value creation and capture mechanisms. To compensate for this paucity of appropriate papers, we used 11 expert interviews to enrich and evaluate the results. Besides, these interviews ensured the practical relevance and reliability of the derived results. Third, we acknowledge limitations of conceptual nature. Our study predominantly takes an optimistic perspective on AI applications in medicine. While we discuss the potential benefits and value propositions in detail, it is important to emphasize that there are still significant barriers and risks currently associated with AI applications that need to be addressed before the identified values can be realized. Furthermore, our investigation is limited because we derive the expected value of AI applications without having extensive real-world use cases to evaluate. It is important to emphasize that our findings are preliminary, and critical reassessment will be essential as the broader implementation of AI applications in medical practice progresses. These limitations emphasize the need for ongoing research and monitoring to understand the true value of AI applications in HC fully.

Conclusions

This study aimed to investigate how AI applications can create value for HC organizations. After elaborating on a diverse and comprehensive set of AI use cases, we are confident that AI applications can create value by making HC, among others, more precise, individualized, self-determined, faster, resource-optimized, and data insight-driven. Especially with regard to the mounting challenges of the industry, such as the aging population and the resulting increase in HC professionals’ workloads, the integration of AI applications and the expected benefits have become more critical than ever. Based on the systematic literature review and expert interviews, we derived 15 business objectives that translate into the following six value propositions that describe how HC organizations can capture the value of AI applications: risk-reduced patient care, advanced patient care, self-management, process acceleration, resource optimization, and knowledge discovery .

By presenting and discussing our results, we enhance the understanding of how HC organizations can unlock AI applications’ value proposition. We provide HC organizations with valuable insights to help them strategically assess their AI applications as well as those deployed by competitors at a management level. Our goal is to facilitate informed decision-making regarding AI investments and enable HC organizations to align their AI application portfolios with a comprehensive and overarching strategy. However, even if various value proposition-creating scenarios exist, AI applications are not yet fully mature in every area or ready for widespread use. Ultimately, it remains essential to take a critical look at which AI applications can be used for which task at which point in time to achieve the promised value. Nonetheless, we are confident that we can shed more light on the value proposition-capturing mechanism and, therefore, support AI application adoption in HC.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Artificial Intelligence

Machine Learning

Fogel AL, Kvedar JC. Artificial intelligence powers digital medicine. NPJ Digit Med. 2018;1:5.

Article PubMed PubMed Central Google Scholar

Rai A, Constantinides P, Sarker S. Next-Generation Digital Platforms: Toward Human–AI Hybrid. Manag Inf Syst Q. 2019;43(1):iii–ix.

Google Scholar

Russell S, Norvig P. Artificial Intelligence: A Modern Approach. Pearson Education Limited; 2016.

He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med. 2019;25(1):30–6.

Article CAS PubMed PubMed Central Google Scholar

Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500–10.

Kadurin A, Aliper A, Kazennov A, Mamoshina P, Vanhaelen Q, Khrabrov K, et al. The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget. 2017;8(7):10883–90.

Article PubMed Google Scholar

Rezazade Mehrizi MH, van Ooijen P, Homan M. Applications of artificial intelligence (AI) in diagnostic radiology: a technography study. Eur Radiol. 2020;31:1805–11.

Dankwa-Mullan I, Rivo M, Sepulveda M, Park Y, Snowdon J, Rhee K. Transforming Diabetes Care Through Artificial Intelligence: The Future Is Here. Popul Health Manag. 2019;22(3):229–42.

Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Artificial Intelligence in Healthcare. Elsevier; 2020. p. 25–60.

Yu K-H, Beam AL, Kohane IS. Artificial Intelligence in Healthcare. Nat Biomed Eng. 2018;2(10):719–31.

Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230–43.

Gilvary C, Madhukar N, Elkhader J, Elemento O. The Missing Pieces of Artificial Intelligence in Medicine. Trends Pharmacol Sci. 2019;40(8):555–64.

Article CAS PubMed Google Scholar

Fernández E. Innovation in Healthcare: Harnessing New Technologies. Journal of the Midwest Association for Information Systems 2017; (2). Available from: http://aisel.aisnet.org/jmwais/vol2017/iss2/8 .

Topol EJ. Deep medicine: How artificial intelligence can make healthcare human again. 1st ed. New York: Basic Books; 2019.

Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 2019;20(5):e262–73.

Plastino E, Purdy M. Game changing value from artificial intelligence: eight strategies. Strategy Leadership. 2018;46(1):16–22.

Article Google Scholar

Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthcare J. 2019;6(2):94–8.

Meskó B, Görög M. A short guide for medical professionals in the era of artificial intelligence. NPJ Digit Med. 2020;3:126.

Statista. What is the stage of AI adoption in your organization?; 2021. Available from: https://www.statista.com/statistics/1225955/stage-of-ai-adoption-in-healthcare-worldwide/ . Cited 2024 Feb. 8

Garbuio M, Lin N. Artificial Intelligence as a Growth Engine for Health Care Startups: Emerging Business Models. Calif Manage Rev. 2019;61(2):59–83.

Väänänen A, Haataja K, Vehviläinen-Julkunen K, Toivanen P. AI in healthcare A narrative review. F1000Res. 2021;10:6.

Kim H-W, Chan HC, Gupta S. Value-based Adoption of Mobile Internet: An empirical investigation. Decis Support Syst. 2007;43(1):111–26.

Lin TC, Wu S, Hsu JSC, Chou YC. The integration of value-based adoption and expectation–confirmation models An example of IPTV continuance intention. Decision Support Systems. 2012;54(1):63–75.

Jordan MI, Mitchell TM. Machine learning: Trends, perspectives, and prospects. Science. 2015;349(6245):255–60.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.

Hamet P, Tremblay J. Artificial intelligence in medicine. Metab Clin Exp. 2017;69S:S36–40.

Lidströmer N, Aresu F, Ashrafian H. Basic Concepts of Artificial Intelligence Primed for Clinicians In Artificial Intelligence in Medicine. Cham: Springer; 2022. p. 3–20.

Choi RY, Coyner AS, Kalpathy-Cramer J, Chiang MF, Campbell JP. Introduction to machine learning, neural networks, and deep learning. Transl Vis Sci Technol. 2020;9(2):1–12.

Lorenz U. Reinforcement Learning: Aktuelle Ansätze verstehen - mit Beispielen in Java und Greenfoot. Berlin, Heidelberg: Springer Berlin Heidelberg; 2020.

Kunduru AR. Artificial intelligence advantages in cloud fintech application security. Central Asian J Mathematical Theory Comp Sci. 2023;4(8):48–53.

Zhai X, Chu X, Chai CS, Jong MSY, Istenic A, Spector M, et al. A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. Complexity. 2021;2021:1–18.

Zhao Y, Li T, Zhang X, Zhang C. Artificial intelligence-based fault detection and diagnosis methods for building energy systems: Advantages, challenges and the future. Renew Sustain Energy Rev. 2019;109:85–101.

Secinaro S, Calandra D, Secinaro A, Muthurangu V, Biancone P. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak. 2021;21(1):125.

Bellini V, Guzzon M, Bigliardi B, Mordonini M, Filippelli S, Bignami E. Artificial intelligence a new tool in operating room management role of machine learning models in operating room optimization. J Med Syst. 2019;44(1):20.

Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24(11):1716–20.

Roland T. Motion Artifact Suppression for Insulated EMG to Control Myoelectric Prostheses. Sensors (Basel). 2020;20(4):1031.

Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20(1):310.

Hennrich J, Fuhrmann H, Eymann T. Accelerating the Adoption of Artificial Intelligence Technologies in Radiology: A Comprehensive Overview on Current Obstacles. Proceedings of the 57th Hawaii International Conference on System Sciences 2024.

Asan O, Bayrak AE, Choudhury A. Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians. J Med Internet Res. 2020;22(6):e15154.

Schryen G. Revisiting IS business value research: what we already know, what we still need to know, and how we can get there. Eur J Inf Syst. 2013;22(2):139–69.

Haddad P, Wickramasinghe N. Conceptualizing Business Value of IT in Healthcare to Design Sustainable e-Health Solutions. Proceedings of Americas Conference on Information Systems 2014.

Strong DM, Volkoff O, Johnson SA, Pelletier LR, Tulu B, Bar-On I, et al. A Theory of Organization-EHR Affordance Actualization. J Assoc Inf Syst. 2014;15(2):53–85.

Hofmann P, Oesterle S, Rust P, Urbach N. Machine Learning Approaches Along the Radiology Value Chain - Rethinking Value Propositions. Proceedings of the European Conference on Information Systems 2019.

Badakhshan P, Wurm B, Grisold T, Geyer-Klingeberg J, Mendling J, Vom Brocke J. Creating business value with process mining. J Strateg Inf Syst. 2022;31(4):101745.

Webster J, Watson RT. Analyzing the Past to Prepare for the Future Writing a Literature Review. MIS Quarterly. 2002;26(2):xiii–xxiii.

Wolfswinkel JF, Furtmueller E, Wilderom CPM. Using Grounded Theory as a Method for Rigorously Reviewing Literature. Eur J Inf Syst. 2013;22(1):45–55.

Siggelkow N. Persuasion With Case Studies. AMJ. 2007;50(1):20–4.

Lakhani P, Prater AB, Hutson RK, Andriole KP, Dreyer KJ, Morey J, et al. Machine Learning in Radiology: Applications Beyond Image Interpretation. J Am Coll Radiol. 2018;15(2):350–9.

Low S-K, Zembutsu H, Nakamura Y. Breast cancer: The translation of big genomic data to cancer precision medicine. Cancer Sci. 2018;109(3):497–506.

Altay EV, Alatas B. Association analysis of Parkinson disease with vocal change characteristics using multi-objective metaheuristic optimization. Med Hypotheses. 2020;141:109722.

Abbasi B, Goldenholz DM. Machine learning applications in epilepsy. Epilepsia. 2019;60(10):2037–47.

Qin B, Quan Q, Wu J, Liang L, Li D. Diagnostic performance of artificial intelligence to detect genetic diseases with facial phenotypes: A protocol for systematic review and meta analysis. Medicine (Baltimore). 2020;99(27):e20989.

Zhavoronkov A, Mamoshina P, Vanhaelen Q, Scheibye-Knudsen M, Moskalev A, Aliper A. Artificial intelligence for aging and longevity research: Recent advances and perspectives. Ageing Res Rev. 2019;49:49–66.

Meystre SM, Friedlin FJ, South BR, Shen S, Samore MH. Automatic de-identification of textual documents in the electronic health record - a review of recent research. BMC Med Res Methodol. 2010;10(70):1–6.

Rhine CL, Neil C, Glidden DT, Cygan KJ, Fredericks AM, Wang J, et al. Future directions for high-throughput splicing assays in precision medicine. Hum Mutat. 2019;40(9):1225–34.

Berlyand Y, Raja AS, Dorner SC, Prabhakar AM, Sonis JD, Gottumukkala RV, et al. How artificial intelligence could transform emergency department operations. Am J Emerg Med. 2018;36(8):1515–7.

Kwon J-M, Kim K-H, Jeon K-H, Park J. Deep learning for predicting in-hospital mortality among heart disease patients based on echocardiography. Echocardiography. 2019;36(2):213–8.

Yang M, Li C, Shen Y, Wu Q, Zhao Z, Chen X. Hierarchical Human-Like Deep Neural Networks for Abstractive Text Summarization. IEEE Trans Neural Netw Learn Syst. 2021;32(6):2744–57.

Hernandez JPT. Network diffusion and technology acceptance of a nurse chatbot for chronic disease self-management support a theoretical perspective. J Med Invest. 2019;66(1.2):24–30.

Bhandari M, Zeffiro T, Reddiboina M. Artificial Intelligence and Robotic Surgery: Current Perspective and Future Directions. Curr Opin Urol. 2020;30(1):48–54.

Padoy N. Machine and deep learning for workflow recognition during surgery. Minim Invasive Ther Allied Technol. 2019;28(2):82–90.

Shepherd M, Abidi SSR, Gao Q, Chen Z, Qi Q, Finley GA. Information Systems and Health Care IX: Accessing Tacit Knowledge and Linking It to the Peer-Reviewed Literature. CAIS 2006; 17.

Wilson EV, Wang W, Sheetz SD. Underpinning a Guiding Theory of Patient-Centered E-Health. CAIS. 2014;34(1):16.

Levy Y, Ellis TJ. A systems approach to conduct an effective literature review in support of information systems research. Informing Sci J. 2006;9:181–211.

Corbin JM, Strauss AL. Basics of qualitative research: Techniques and procedures for developing grounded theory. Fourth edition. Thousand Oaks, Kalifornien: SAGE; 2015.

Glaser BG, Strauss A. Discovery of grounded theory: Strategies for qualitative research. Routledge; 1967.

Feldman S, Horan T. The dynamics of information collaboration: a case study of blended IT value propositions for health information exchange in disability determination. J Assoc Inf Syst. 2011;12(2):189–207.

Schultze U, Avital M. Designing interviews to generate rich data for information systems research. Inf Organ. 2011;21(1):1–16.

Myers MD, Newman M. The qualitative interview in IS research: examining the craft. Inf Organ. 2007;17(1):2–26.

Bhattacherjee A. Social Science Research: Principles, Methods, and Practices. Textbooks Collection 2012; 3.

Moulaei K, Sheikhtaheri A, Fatehi F, Shanbehzadeh M, Bahaadinbeigy K. Patients’ perspectives and preferences toward telemedicine versus in-person visits: a mixed-methods study on 1226 patients. BMC Med Inform Decis Mak. 2023;23(1):261.

Buck C, Hennrich J, Kauffmann A-L. Artificial Intelligence in Radiology – A Qualitative Study on Imaging Specialists’ Perspectives. Proceedings of the 42nd International Conference on Information Systems 2021.

Degnan AJ, Ghobadi EH, Hardy P, Krupinski E, Scali EP, Stratchko L, et al. Perceptual and interpretive error in diagnostic radiology-causes and potential solutions. Acad Radiol. 2019;26(6):833–45.

Pesapane F, Codari M, Sardanelli F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. Eur Radiol Exp. 2018;2(1):35.

Schinkel M, Paranjape K, Nannan Panday RS, Skyttberg N, Nanayakkara PWB. Clinical applications of artificial intelligence in sepsis: A narrative review. Comput Biol Med. 2019;115:103488.

Shamim Nemati, Mohammad Ghassemi, Gari Clifford. Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach. International Conference of the IEEE Engineering in Medicine and Biology Society 2016. Available from: http://ieeexplore.ieee.org/servlet/opac?punumber=7580725 .

Padoy N, Hager GD. Human-Machine Collaborative Surgery Using Learned Methods. Proceedings of IEEE International Conference on Robotics and Automation 2011:5285–92.

Nadarzynski T, Miles O, Cowie A, Ridge D. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: A mixed-methods study. Digit Health. 2019;5:2055207619871808.

PubMed PubMed Central Google Scholar

Trevitt S, Simpson S, Wood A. Artificial Pancreas device systems for the closed-loop control of type 1 diabetes: what systems are in development? J Diabetes Sci Technol. 2016;10(3):714–23.

Luo L, Zhang F, Yao Y, Gong R, Fu M, Xiao J. Machine learning for identification of surgeries with high risks of cancellation. Health Informatics J. 2018;26(1):141–55.

Shollo A, Hopf K, Thiess T, Müller O. Shifting ML value creation mechanisms: a process model of ML value creation. J Strateg Inf Syst. 2022;31(3):101734.

Buck C, Doctor E, Hennrich J, Jöhnk J, Eymann T. General Practitioners’ Attitudes Toward Artificial Intelligence-Enabled Systems: Interview Study. J Med Internet Res. 2022;24(1):e28916.

Bennett SJ. Transmuting values in artificial intelligence: investigating the motivations and contextual constraints shaping the ethics of artificial intelligence practitioners; 2023.

Baumgartner R, Arora P, Bath C, Burljaev D, Ciereszko K, Custers B, et al. Fair and equitable AI in biomedical research and healthcare: Social science perspectives. Artif Intell Med. 2023;144:102658.

Download references

Acknowledgements

The authors thank all physicians who participated in this study.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

FIM Research Institute for Information Management, University of Bayreuth, Branch Business and Information Systems Engineering of the Fraunhofer FIT, Wittelsbacherring 10, 95444, Bayreuth, Germany

Jasmin Hennrich, Peter Hofmann & Nils Urbach

University St. Gallen, Dufourstrasse 50, 9000, St. Gallen, Switzerland

Faculty Business and Law, Frankfurt University of Applied Sciences, Nibelungenplatz 1, 60318, Frankfurt Am Main, Germany

Nils Urbach

appliedAI Initiative GmbH, August-Everding-Straße 25, 81671, Munich, Germany

Peter Hofmann

You can also search for this author in PubMed Google Scholar

Contributions

JH initially composed the introduction, background, and discussion sections; PH contributed to the analysis design, supported data analysis, and critically reviewed the manuscript; ER conducted data collection and analysis and produced the initial draft of the methodology section; NU edited and critically reviewed the manuscript and provided guidance. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jasmin Hennrich .

Ethics declarations

Ethics approval and consent to participate.

The study was based on original data. The authors confirm that all methods were carried out in accordance with relevant guidelines and regulations and confirm that informed consent was obtained from all participants. Ethics approval was granted by the Ethics Committee of the University of Bayreuth (Application-ID 23–032).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hennrich, J., Ritz, E., Hofmann, P. et al. Capturing artificial intelligence applications’ value proposition in healthcare – a qualitative research study. BMC Health Serv Res 24 , 420 (2024). https://doi.org/10.1186/s12913-024-10894-4

Download citation

Received : 26 October 2023

Accepted : 25 March 2024

Published : 03 April 2024

DOI : https://doi.org/10.1186/s12913-024-10894-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Artificial intelligence
Value propositions
Business objectives

BMC Health Services Research

ISSN: 1472-6963

General enquiries: [email protected]

Retirement planning – a systematic review of literature and future research directions

Published: 28 October 2023

Cite this article

Kavita Karan Ingale ORCID: orcid.org/0000-0003-3570-4211 1 &
Ratna Achuta Paluri 2

515 Accesses

Explore all metrics

Rising life expectancy and an aging population across nations are leading to an increased need for long-term financial savings and a focus on the financial well-being of retired individuals amidst changing policy framework. This study is a systematic review based on a scientific way of producing high-quality evidence based on 191 articles from the Scopus and Web of Science databases. It adopts the Theory, Context, Characteristics, and Method (TCCM) framework to analyze literature. This study provides collective insights into financial decision-making for retirement savings and identifies constructs for operationalizing and measuring financial behavior for retirement planning. Further, it indicates the need for an interdisciplinary approach. Though cognitive areas were studied extensively, the non-cognitive areas received little attention. Qualitative research design is gaining prominence in research over other methods, with the sparse application of mixed methods design. The study’s TCCM framework explicates several areas for further research. Furthermore, it guides the practice and policy by integrating empirical evidence and concomitant findings. Coherent synthesis of the extant literature reconciles the highly fragmented field of retirement planning. No research reports prospective areas for further analysis based on the TCCM framework on retirement planning, which highlights the uniqueness of the study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

A Research Proposal to Examine Psychological Factors Influence on Financial Planning for Retirement in China

Domains and determinants of retirement timing: A systematic review of longitudinal studies

Micky Scharn, Ranu Sewdas, … Allard J. van der Beek

Reinventing Retirement

Deanna L. Sharpe

Data Availability

The research data will be made available on request.

Acknowledgment.

Elderly population is defined as a population aged 65 years and over.

Defined benefit plan guarantees benefits to the employee, while defined contribution plan requires employees to decide on their own investment and bear the financial risks identified with it.

“The old-age dependency ratio is defined as the number of individuals aged 65 and over per 100 people of working age defined as those at ages 20 to 64”(OECD 2023 ).

Adams GA, Rau BL (2011) Putting off tomorrow to do what you want today: planning for Retirement. Am Psychol 66(3):180–192. https://doi.org/10.1037/a0022131

Article Google Scholar

Aegon Cfor, Longevity, Retirement ICR (2016) The Aegon Retirement Readiness Survey 2016. In The Aegon Retirement Readiness Survey 2016 . https://www.aegon.com/contentassets/c6a4b1cdded34f1b85a4f21d4c66e5d3/2016-aegon-retirement-readiness-report-india.pdf

Agarwalla SK, Barua SK, Jacob J, Varma JR (2015) Financial Literacy among Working Young in Urban India. World Development , 67 (2013), 101–109. https://doi.org/10.1016/j.worlddev.2014.10.004

Ajzen I (1991) The theory of Planned Behavior. Organ Behav Hum Decis Process 50:179–211. https://doi.org/10.47985/dcidj.475

Anderson A, Baker F, Robinson DT (2017) Precautionary savings, retirement planning, and misperceptions of financial literacy. J Financ Econ 126(2):383–398. https://doi.org/10.1016/j.jfineco.2017.07.008

Atkinson A, Messy FA (2011) Assessing financial literacy in 12 countries: an OECD/INFE international pilot exercise. J Pension Econ Finance 10(4):657–665. https://doi.org/10.1017/S1474747211000539`

Aydin AE, Akben Selcuk E (2019) An investigation of financial literacy, money ethics, and time preferences among college students: a structural equation model. Int J Bank Mark 37(3):880–900. https://doi.org/10.1108/IJBM-05-2018-0120

Bapat D (2020) Antecedents to responsible financial management behavior among young adults: the moderating role of financial risk tolerance. Int J Bank Mark 38(5):1177–1194. https://doi.org/10.1108/IJBM-10-2019-0356

Beckett A, Hewer P, Howcroft B (2000) An exposition of consumer behaviour in the financial services industry. Int J Bank Mark 18(1):15–26. https://doi.org/10.1108/02652320010315325

Białowolski P (2019) Economic sentiment as a driver for household financial behavior. J Behav Experimental Econ 80(August 2017):59–66. https://doi.org/10.1016/j.socec.2019.03.006

Binswanger J, Carman KG (2012) How real people make long-term decisions: the case of retirement preparation. J Economic Behav Organ 81(1):39–60. https://doi.org/10.1016/j.jebo.2011.08.010

Brounen D, Koedijk KG, Pownall RAJ (2016) Household financial planning and savings behavior. J Int Money Finance 69:95–107. https://doi.org/10.1016/j.jimonfin.2016.06.011

Brown R, Jones M (2015) Mapping and exploring the topography of contemporary financial accounting research. Br Acc Rev 47(3):237–261. https://doi.org/10.1016/j.bar.2014.08.006

Brown S, Gray D (2016) Household finances and well-being in Australia: an empirical analysis of comparison effects. J Econ Psychol 53:17–36. https://doi.org/10.1016/j.joep.2015.12.006

Brown S, Taylor K (2014) Household finances and the big five personality traits. J Econ Psychol 45:197–212. https://doi.org/10.1016/j.joep.2014.10.006

Brown S, Taylor K (2016) Early influences on saving behaviour: analysis of British panel data. J Bank Finance 62:1–14. https://doi.org/10.1016/j.jbankfin.2015.09.011

Brüggen EC, Post T, Schmitz K (2019) Interactivity in online pension planners enhances engagement with retirement planning – but not for everyone. J Serv Mark 33(4):488–501. https://doi.org/10.1108/JSM-02-2018-0082

Bruggen E, Post T, Katharina S (2019) Interactivity in online pension planners enhances engagement with retirement planning but not for everyone. J Serv Mark 33(4):488–501

Calcagno R, Monticone C (2015) Financial literacy and the demand for financial advice. J Bank Finance 50:363–380. https://doi.org/10.1016/j.jbankfin.2014.03.013

Campbell JY (2006) Household finance. J Finance 61(4):1553–1604. https://doi.org/10.1111/j.1540-6261.2006.00883.x

Choudhury K (2015) Service quality and customers’ behavioural intentions: class and mass banking and implications for the consumer and society. Asia Pac J Mark Logistics 27(5):735–757

Chowdhry N, Jung J, Dholakia U (2018) Association for consumer research. Adv Consum Res 42:42–46

Google Scholar

Clark GL, Knox-Hayes J, Strauss K (2009) Financial sophistication, salience, and the scale of deliberation in UK retirement planning. Environ Plann A 41(10):2496–2515. https://doi.org/10.1068/a41265

Clark R, Lusardi A, Mitchell OS (2017) Employee Financial Literacy and Retirement Plan Behavior: a case study. Econ Inq 55(1):248–259. https://doi.org/10.1111/ecin.12389

Collins JM, Urban C (2016) The role of information on Retirement Planning: evidence from a field study. Econ Inq 54(4):1860–1872. https://doi.org/10.1111/ecin.12349

Creswell J (2009) Research Design Qualitative Quantitative and Mixed Methods Approaches. In Sage Publishing: Vol. Third edit . https://doi.org/10.1002/tl.20234

Csorba L (2020) The determining factors of financial culture, financial literacy, and financial behavior. Public Finance Q 65:67–83. https://doi.org/10.35551/PFQ_2020_1_6

Davidoff T, Gerhard P, Post T (2017) Reverse mortgages: what homeowners (don’t) know and how it matters. J Economic Behav Organ 133:151–171. https://doi.org/10.1016/j.jebo.2016.11.007

Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly: Management Information Systems 13(3):319–339. https://doi.org/10.2307/249008

Devlin J (2001) Consumer evaluation and competitive advantage in retail financial services - a research agenda. Eur J Mark 35(5/6):639–660

Dholakia U, Tam L, Yoon S, Wong N (2016) The ant and the grasshopper: understanding personal saving orientation of consumers. J Consum Res 43(1):134–155. https://doi.org/10.1093/jcr/ucw004

Dolls M, Doerrenberg P, Peichl A, Stichnoth H (2018) Do retirement savings increase in response to information about retirement and expected pensions? J Public Econ 158(July 2017):168–179. https://doi.org/10.1016/j.jpubeco.2017.12.014

Dragos SL, Dragos CM, Muresan GM (2020) From intention to the decision in purchasing life insurance and private pensions: different effects of knowledge and behavioural factors. J Behav Experimental Econ 87(March):101555. https://doi.org/10.1016/j.socec.2020.101555

Drever AI, Odders-white E, Kalish CW, Hoagland EM, Nelms EN, Drever AI, Odders-white E, Charles W, Else-quest NM, Hoagland EM, Nelms EN (2015) Foundations of Financial Weil-Being: Insights into the Role of Executive Function, Financial Socialization, and Experience-Based Learning in Childhood and Youth Source : The Journal of Consumer Affairs, Vol. 49, No. 1, Special Issue on Starting Ea. The Journal of Consumer Affairs , 49 (1)

Duflo E, Saez E (2002) Participation and investment decisions in a retirement plan: the influence of colleagues’ choices. J Public Econ 85(1):121–148. https://doi.org/10.1016/S0047-2727(01)00098-6

Duxbury D, Summers B, Hudson R, Keasey K (2013) How people evaluate defined contribution, annuity-based pension arrangements: a behavioral exploration. J Econ Psychol 34:256–269. https://doi.org/10.1016/j.joep.2012.10.008

Earl J, Bednall T, Muratore A (2015) A matter of time: why some people plan for retirement and others do not. Work Aging and Retirement 1(2):181–189. https://doi.org/10.1093/workar/wau005

Employees Benefits Research Institute (2020) EBRI Retirement Confidence Survey Report (Issue 202)

Engel JF, Kollat DT, Blackwell RD (1968) A model of consumer motivation and behavior. In: Research in consumer behavior. Holt, Rinehart and Winston, Inc., New York, pp 3–20

Erasmus A, Boshoff E, Rousseau G (2001) Consumer decision-making models within the discipline of consumer science: a critical approach. J Family Ecol Consumer Sci /Tydskrif Vir Gesinsekologie En Verbruikerswetenskappe 29(1):82–90. https://doi.org/10.4314/jfecs.v29i1.52799

Farrell L, Fry TRL, Risse L (2016) The significance of financial self-efficacy in explaining women’s personal finance behaviour. J Econ Psychol 54:85–99

Fernandes D, Lynch JG, Netemeyer RG (2014) Financial literacy, financial education, and downstream financial behaviors. Manage Sci 60(8):1861–1883. https://doi.org/10.1287/mnsc.2013.1849

Filbec G, Ricciardi V, Evensky H, Fan S, Holzhauer H, Spieler A (2017) Behavioral finance: a panel discussion. J Behav Experimental Finance 15:52–58. https://doi.org/10.1016/j.jbef.2015.07.003

Fishbein M (1979) A theory of reasoned action: some applications and implications. Nebraska Symposium on Motivation 27:65–116

Fisher PJ, Montalto CP (2010) Effect of saving motives and horizon on saving behaviors. J Econ Psychol 31(1):92–105. https://doi.org/10.1016/j.joep.2009.11.002

Flores SAM, Vieira KM (2014) Propensity toward indebtedness: an analysis using behavioral factors. J Behav Exp Finance 3:1–10

Foxall GR, Pallister JG (1998) Measuring purchase decision involvement for financial services: comparison of the Zaichkowsky and Mittal scales. Int J Bank Mark 16(5):180–194. https://doi.org/10.1108/02652329810228181

Friedman M (1957) Introduction to “A theory of the consumption function”. In: A theory of the consumption function. Princeton University Press, pp 1–6

Frydman C, Camerer CF (2016) The psychology and neuroscience of financial decision making. Trends Cogn Sci 20(9):661–675. https://doi.org/10.1016/j.tics.2016.07.003

Gardarsdóttir RB, Dittmar H (2012) The relationship of materialism to debt and financial well-being: the case of Iceland’s perceived prosperity. J Econ Psychol 33(3):471–481. https://doi.org/10.1016/j.joep.2011.12.008

Gathergood J (2012) Self-control, financial literacy and consumer over-indebtedness. J Econ Psychol 33(3):590–602

Gerhard P, Gladstone JJ, Hoffmann AOI (2018) Psychological characteristics and household savings behavior: the importance of accounting for latent heterogeneity. J Economic Behav Organ 148:66–82. https://doi.org/10.1016/j.jebo.2018.02.013

Gibbs PT (2009) Time, temporality, and

Goedde-Menke M, Lehmensiek-Starke M, Nolte S (2014) An empirical test of competing hypotheses for the annuity puzzle. J Econ Psychol 43:75–91

Gough O, Nurullah M (2009) Understanding what drives the purchase decision in pension and investment products. J Financial Serv Mark 14(2):152–172. https://doi.org/10.1057/fsm.2009.14

Griffin B, Loe D, Hesketh B (2012) Using Proactivity, Time Discounting, and the theory of Planned Behavior to identify predictors of Retirement Planning. Educ Gerontol 38(12):877–889. https://doi.org/10.1080/03601277.2012.660857

Gritten A (2011) New insights into consumer confidence in financial services. Int J Bank Mark 29(2):90–106. https://doi.org/10.1108/02652321111107602

Grohmann A (2018) Financial literacy and financial behavior: Evidence from the emerging Asian middle class. Pacific Basin Finance Journal , 48 (November 2017), 129–143. https://doi.org/10.1016/j.pacfin.2018.01.007

Grohmann A, Kouwenberg R, Menkhoff L (2015) Childhood roots of financial literacy. J Econ Psychol 51:114–133. https://doi.org/10.1016/j.joep.2015.09.002

Hair JF, Sarstedt M, Ringle CM, Mena JA (2012) An assessment of the use of partial least squares structural equation modeling in marketing research . 414–433. https://doi.org/10.1007/s11747-011-0261-6

Hanna SD, Kim KT, Chen SCC (2016) Retirement savings. In: Handbook of consumer finance research, pp 33–43

Harrison T, Waite K, White P (2006) Analysis by paralysis: the pension purchase decision process. Int J Bank Mark 24(1):5–23. https://doi.org/10.1108/02652320610642317

Hastings J, Mitchell O (2011) How financial literact and impatience shape retirement wealth and investment behaviors. Pengaruh Harga Diskon Dan Persepsi Produk Terhadap Nilai Belanja Serta Perilaku Pembelian Konsumen, NBER Working paper, 1–28

Hauff J, Carlander A, Amelie G, Tommy G, Holmen M (2016) Breaking the ice of low financial involvement: does narrative information format from a trusted sender increase savings in mutual funds? Int J Bank Mark 34(2):151–170

Hentzen JK, Hoffmann A, Dolan R, Pala E (2021) Artificial intelligence in customer-facing financial services: a systematic literature review and agenda for future research. Int J Bank Mark. https://doi.org/10.1108/IJBM-09-2021-0417

Hershey DA, Mowen JC (2000) Psychological determinants of financial preparedness for retirement. Gerontologist 40(6):687–697. https://doi.org/10.1093/geront/40.6.687

Hershey DA, Henkens K, Van Dalen HP (2007) Mapping the minds of retirement planners: a cross-cultural perspective. J Cross-Cult Psychol 38(3):361–382. https://doi.org/10.1177/0022022107300280

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2007b) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2008) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershfield H, Goldstein D, Sharpe W, Fox J, Yeykelis L, Carstensen L, Bailenson J (2011) Increasing saving behavior through age-progressed renderings of the future self. J Mark Res 48:23–37

Hoffmann AOI, Broekhuizen TLJ (2009) Susceptibility to and impact of interpersonal influence in an investment context. J Acad Mark Sci 37:488–503

Hoffmann AOI, Broekhuizen TLJ (2010) Understanding investors’ decisions to purchase innovative products: drivers of adoption timing and range. Int J Res Mark 27(4):342–355. https://doi.org/10.1016/j.ijresmar.2010.08.002

Hoffmann AOI, Plotkina D (2020a) Positive framing when assessing the personal resources to manage one’s finances increases consumers’ retirement self-efficacy and improves retirement goal clarity. Psychol Mark 38(12):2286–2304. https://doi.org/10.1002/mar.21563

Hoffmann AOI, Plotkina D (2020b) Why and when does financial information affect retirement planning intentions and which consumers are more likely to act on them? Journal of Business Research , 117 (September 2019), 411–431. https://doi.org/10.1016/j.jbusres.2020.06.023

Hoffmann AOI, Plotkina D (2021) Let your past define your future. How recalling successful financial experiences can increase beliefs of self-efficacy in financial planning. J Consum Aff 55(3):847–871. https://doi.org/10.1111/joca.12378

Hoffmann AOI, Risse L (2020) Do good things come in pairs? How personality traits help explain individuals’ simultaneous pursuit of a healthy lifestyle and financially responsible behavior. J Consum Aff 54(3):1082–1120. https://doi.org/10.1111/joca.12317

Hsiao YJ, Tsai WC (2018) Financial literacy and participation in the derivatives markets. J Bank Finance 88:15–29

Huhmann BA, McQuitty S (2009) A model of consumer financial numeracy. Int J Bank Mark 27(4):270–293. https://doi.org/10.1108/02652320910968359

Huston SJ (2010) Measuring financial literacy. J Consum Aff 44(2):296–316. https://doi.org/10.1111/j.1745-6606.2010.01170.x

Ijevleva K, Arefjevs I (2014) Analysis of the Aggregate Financial Behaviour of customers using the Transtheoretical Model of Change. Procedia - Social and Behavioral Sciences 156(April):435–438. https://doi.org/10.1016/j.sbspro.2014.11.217

Ingale KK, Paluri RA (2020) Financial literacy and financial behavior: a bibliometric analysis. Rev Behav Finance. https://doi.org/10.1108/RBF-06-2020-0141

Jacobs-Lawson J, Hershey D (2005) Influence of future time perspective, financial knowledge, and financial risk tolerance on retirement savings behavior. Financial Serv Rev 14:331–344. https://doi.org/10.1088/1751-8113/44/8/085201

Jappelli T, Padula M (2013) Investment in financial literacy and saving decisions. J Bank Finance 37(8):2779–2792. https://doi.org/10.1016/j.jbankfin.2013.03.019

Kadoya Y, Rahim Khan MS (2020) Financial literacy in Japan: new evidence using financial knowledge, behavior, and attitude. Sustain (Switzerland) 12(9). https://doi.org/10.3390/su12093683

Kamil NSSN, Musa R, Sahak SZ (2014) Examining the Role of Financial Intelligence Quotient (FiQ) in explaining credit card usage behavior: a conceptual Framework. Procedia - Social and Behavioral Sciences 130:568–576. https://doi.org/10.1016/j.sbspro.2014.04.066

Kerry MJ (2018) Psychological antecedents of retirement planning: a systematic review. Front Psychol 9(OCT). https://doi.org/10.3389/fpsyg.2018.01870

Kerry MJ, Embretson SE (2018) An experimental evaluation of competing age predictions of future time perspective between workplace and retirement domains. Front Psychol 8(JAN):1–9. https://doi.org/10.3389/fpsyg.2017.02316

Kiliyanni AL, Sivaraman S (2016) The perception-reality gap in financial literacy: evidence from the most literate state in India. Int Rev Econ Educ 23:47–64. https://doi.org/10.1016/j.iree.2016.07.001

Kimiyaghalam F, Mansori S, Safari M, Yap S (2017) Parents’ influence on retirement planning in Malaysia. Family Consumer Sci Res J 45(3):315–325

Klapper L, Lusardi A, Panos GA (2013) Financial literacy and its consequences: evidence from Russia during the financial crisis. J Bank Finance 37(10):3904–3923

Koehler DJ, Langstaff J, Liu WQ (2015) A simulated financial savings task for studying consumption and retirement decision-making. J Econ Psychol 46:89–97. https://doi.org/10.1016/j.joep.2014.12.004

Kramer MM (2016) Financial literacy, confidence, and financial advice seeking. Journal of Economic Behavior and Organization , 131 (June 2015), 198–217. https://doi.org/10.1016/j.jebo.2016.08.016

Kumar S, Tomar S, Verma D (2019) Women’s financial planning for retirement: systematic literature review and future research agenda. Int J Bank Mark 37(1):120–141. https://doi.org/10.1108/IJBM-08-2017-0165

Kwon KN, Lee J (2009) The effects of reference point, knowledge, and risk propensity on the evaluation of financial products. J Bus Res 62(7):719–725. https://doi.org/10.1016/j.jbusres.2008.07.002

Landerretche OM, Martínez C (2013) Voluntary savings, financial behavior, and pension finance literacy: evidence from Chile. J Pension Econ Finance 12(3):251–297. https://doi.org/10.1017/S1474747212000340

Lee T (2017) (David). Clear, conspicuous, and improving: US corporate websites for critical financial literacy in retirement. International Journal of Bank Marketing , 35 (5), 761–780. https://doi.org/10.1108/IJBM-01-2016-0010

Liang C-J, Wang Wen‐Hung, Farquhar JD (2009) (2009). The influence of customer perceptions on financial performance in financial services. International Journal of Bank Marketing , 27 (2), 129–149

Liberman N, Trope Y (2003) Construal level theory of intertemporal judgment and decision. In: Loewenstein G, Read D, Baumeister R (eds) Time and decision: economic and psychological perspectives on intertemporal choice, pp 245–276

Lim KL, Soutar GN, Lee JA (2013) Factors affecting investment intentions: a consumer behaviour perspective. J Financ Serv Mark 18:301–315

Lin C, Hsiao YJ, Yeh CY (2017) Financial literacy, financial advisors, and information sources on demand for life insurance. Pac Basin Finance J 43(March):218–237. https://doi.org/10.1016/j.pacfin.2017.04.002

Lown JM (2011) Development and validation of a Financial Self-Efficacy Scale. J Financial Couns Plann 22(2):54–63

Lusardi A, Mitchell OS (2007) Baby Boomer retirement security: the roles of planning, financial literacy, and housing wealth. J Monet Econ 54(1):205–224. https://doi.org/10.1016/j.jmoneco.2006.12.001

Maloney M, McCarthy A (2017) Understanding pension communications at the organizational level: insights from bounded rationality theory & implications for HRM. Hum Resource Manage Rev 27(2):338–352. https://doi.org/10.1016/j.hrmr.2016.08.001

Marjanovic Z, Fiksenbaum L, Greenglass E (2018) Financial threat correlates with acute economic hardship and behavioral intentions that can improve one’s personal finances and health. J Behav Experimental Econ 77(April):151–157. https://doi.org/10.1016/j.socec.2018.09.012

Marques S, Mariano J, Lima ML, Abrams D (2018) Are you talking to the future me? The moderator role of future self-relevance on the effects of aging salience in retirement savings. J Appl Soc Psychol 48(7):360–368. https://doi.org/10.1111/jasp.12516

McKechnie S (1992) Consumer buying behaviour in financial services: an overview. Int J Bank Mark 10(5):5–39. https://doi.org/10.1108/02652329210016803

Milner T, Rosenstreich D (2013a) A review of consumer decision-making models and development of a new model for financial services. J Financial Serv Mark 18(2):106–120. https://doi.org/10.1057/fsm.2013.7

Milner T, Rosenstreich D (2013b) Insights into mature consumers of financial services. J Consumer Mark 30(3):248–257. https://doi.org/10.1108/07363761311328919

Mitchell OS, Mukherjee A (2017) Assessing the demand for micro pensions among India’s poor. J Econ Ageing 9:30–40. https://doi.org/10.1016/j.jeoa.2016.05.004

Mitchell O, Utkus S (2003) Lessons from Behavioral Finance for Retirement Plan Design (PRC WP 2003-6). http://prc.wharton.upenn.edu/prc/prc.html

Modigliani F, Brumberg RH (1954) Utility analysis and the consumption function: an interpretation of cross-section data. In: Kurihara KK (ed) Post-Keynesian economics. Rutgers University Press, New Brunswick, pp 388–436

Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, Atkins D, Barbour V, Barrowman N, Berlin JA, Clark J, Clarke M, Cook D, D’Amico R, Deeks JJ, Devereaux PJ, Dickersin K, Egger M, Ernst E, …, Tugwell P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7). https://doi.org/10.1371/journal.pmed.1000097

Monti M, Pelligra V, Martignon L, Berg N (2014) Retail investors and financial advisors: new evidence on trust and advice taking heuristics. J Bus Res 67(8):1749–1757. https://doi.org/10.1016/j.jbusres.2014.02.022

Mouna A, Anis J (2017) Financial literacy in Tunisia: its determinants and its implications on investment behavior. Res Int Bus Finance 39:568–577

Mullainathan S, Thaler R (2000) Massachusetts Institute of Technology Department of Economics Working Paper Series . September

Nga KH, Yeoh KK (2018) An exploratory model on retirement savings behaviour: a Malaysian study. Int J Bus Soc 19(3):637–659

OECD (2023) Old-age dependency ratio (indicator). https://doi.org/10.1787/e0255c98-en . Accessed 13 Oct 2023

Onwuegbuzie AJ, Collins KM (2007) A typology of mixed methods sampling designs in social science research. Qualitative Rep 12(2):474–498

Pallister JG, Wang HC, Foxall GR (2007) An application of the style/involvement model to financial services. Technovation 27(1–2):78–88. https://doi.org/10.1016/j.technovation.2005.10.001

Pan L, Pezzuti T, Lu W, Pechmann C (2019) Hyperopia and frugality: different motivational drivers and yet similar effects on consumer spending. J Bus Res 95(August 2018):347–356

Parise G, Peijnenburg K (2017) Understanding the Determinants of Financial Outcomes and Choices: The Role of Noncognitive Abilities. BIS Working Papers

Paul J, Rosado-Serrano A (2019) Gradual internationalization vs Born-Global/International new venture models: a review and research agenda. Int Mark Rev 36(6):830–858. https://doi.org/10.1108/IMR-10-2018-0280

Paul J, Criado AR (2020) The art of writing literature review: what do we know and what do we need to know? Int Bus Rev 29(4):101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Paul J, Khatri P, Kaur Duggal H (2023) Frameworks for developing impactful systematic literature reviews and theory building: what, why and how? J Decis Syst 00(00):1–14. https://doi.org/10.1080/12460125.2023.2197700

Petkoska J, Earl JK (2009) Understanding the influence of demographic and psychological variables on Retirement Planning. Psychol Aging 24(1):245–251. https://doi.org/10.1037/a0014096

Piotrowska M (2019) The importance of personality characteristics and behavioral constraints for retirement saving. Econ Anal Policy 64:194–220

Plath DA, Stevenson TH (2005) Financial services consumption behavior across Hispanic American consumers. J Bus Res 58(8):1089–1099. https://doi.org/10.1016/j.jbusres.2004.03.003

Poterba JM (2015) Saver heterogeneity and the challenge of assessing retirement saving adequacy. Natl Tax J 68(2):377–388. https://doi.org/10.17310/ntj.2015.2.06

Potrich ACG, Vieira KM, Kirch G (2018) How well do women do when it comes to financial literacy? Proposition of an indicator and analysis of gender differences. J Behav Experimental Finance 17:28–41. https://doi.org/10.1016/j.jbef.2017.12.005

Rai D, Lin CW (2019) (Wilson). The influence of implicit self-theories on consumer financial decision making. Journal of Business Research , 95 (August 2018), 316–325. https://doi.org/10.1016/j.jbusres.2018.08.016

Ramalho TB, Forte D (2019) Financial literacy in Brazil – do knowledge and self-confidence relate with behavior? RAUSP Manage J 54(1):77–95. https://doi.org/10.1108/RAUSP-04-2018-0008

Rana J, Paul J (2017) Consumer behavior and purchase intention for organic food: a review and research agenda. J Retailing Consumer Serv 38(June):157–165. https://doi.org/10.1016/j.jretconser.2017.06.004

Ranyard R, McNair S, Nicolini G, Duxbury D (2020) An item response theory approach to constructing and evaluating brief and in-depth financial literacy scales. J Consum Aff 54(3):1121–1156. https://doi.org/10.1111/joca.12322

RBI Household Finance Committee (2017) Indian household finance. Reserve Bank of India, Mumbai

Ruefenacht M, Schlager T, Maas P, Puustinen P (2015) Drivers of long-term savings behavior from consumer’s perspective. Electron Libr 34(1):1–5

Scholz JK, Seshadri A, Khitatrakun S (2006) Are Americans saving “optimally” for retirement? J Polit Econ 114(4):607–643

Schuabb T, França LH, Amorim SM (2019) Retirement savings model tested with Brazilian private health care workers. Front Psychol 10(JULY):1–11. https://doi.org/10.3389/fpsyg.2019.01701

Schuhen M, Schurkmann S (2014) International Review of Economics Education. Int Rev Econ Educ 16:1–11

Segel-Karpas D, Werner P (2014) Perceived financial retirement preparedness and its correlates: a national study in Israel. Int J Aging Hum Dev 79(4):279–301. https://doi.org/10.1177/0091415015574177

Seth H, Talwar S, Bhatia A, Saxena A, Dhir A (2020) Consumer resistance and inertia of retail investors: Development of the resistance adoption inertia continuance (RAIC) framework. Journal of Retailing and Consumer Services , 55 (August 2019), 102071. https://doi.org/10.1016/j.jretconser.2020.102071

Sewell M (2008) Behavioural finance. Economist 389(8604):1–13. https://doi.org/10.1057/9780230280786_5

Shefrin HM, Thaler RH (1988) The behavioral life‐cycle hypothesis. Econ Inq 26(4):609–643

Shim S, Serido J, Tang C (2012) The ant and the grasshopper revisited: the present psychological benefits of saving and future oriented financial behavior. J Econ Psychol 33(1):155–165

Simon HA (1978) Information-processing theory of human problem solving. In: Handbook of learning and cognitive processes, vol 5, pp 271–295

Sivaramakrishnan S, Srivastava M, Rastogi A (2017) Attitudinal factors, financial literacy, and stock market participation. Int J Bank Mark 34(1):1–5

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104(August):333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Stawski RS, Hershey DA, Jacobs-Lawson JM (2007) Goal clarity and financial planning activities as determinants of retirement savings contributions. Int J Aging Hum Dev 64(1):13–32. https://doi.org/10.2190/13GK-5H72-H324-16P2

Steinert JI, Zenker J, Filipiak U, Movsisyan A, Cluver LD, Shenderovich Y (2018) Do saving promotion interventions increase household savings, consumption, and investments in Sub-saharan Africa? A systematic review and meta-analysis. World Dev 104:238–256. https://doi.org/10.1016/j.worlddev.2017.11.018

Steinhart Y, Mazursky D (2010) Purchase availability and involvement antecedents among financial products. Int J Bank Mark 28(2):113–135. https://doi.org/10.1108/02652321011018314

Strömbäck C, Lind T, Skagerlund K, Västfjäll D, Tinghög G (2017) Does self-control predict financial behavior and financial well-being? J Behav Experimental Finance 14:30–38. https://doi.org/10.1016/j.jbef.2017.04.002

Strömbäck C, Skagerlund K, Västfjäll D, Tinghög G (2020) Subjective self-control but not objective measures of executive functions predict financial behavior and well-being. Journal of Behavioral and Experimental Finance , 27 . https://doi.org/10.1016/j.jbef.2020.100339

Tam L, Dholakia U (2014) Saving in cycles: how to get people to save more money. Psychol Sci 25(2):531–537. https://doi.org/10.1177/0956797613512129

Tang N, Baker A (2016) Self-esteem, financial knowledge and financial behavior. J Econ Psychol 54:164–176

Tate M, Evermann J, Gable G (2015) An integrated framework for theories of individual attitudes toward technology. Inform Manage 52(6):710–727. https://doi.org/10.1016/j.im.2015.06.005

Taylor MP, Jenkins SP, Sacker A (2011) Financial capability and psychological health. J Econ Psychol 32(5):710–723. https://doi.org/10.1016/j.joep.2011.05.006

Tennyson S, Yang HK (2014) The role of life experience in long-term care insurance decisions. Journal of Economic Psychology , 42 (2014), 175–188. https://doi.org/10.1016/j.joep.2014.04.002

Thaler BRH (1994) Psychology and savings policies. Am Econ Rev 84(2):175–179. http://www.jstor.org/stable/3132220

Thaler R (1980) Toward a positive theory of consumer choice. J Econ Behav Organ 1:39–60

Thaler RH (2005) Advances in behavioral finance. Adv Behav Finance 2:1–694. https://doi.org/10.2307/2329257

Thaler R, Shefrin H (1981) An economic theory of self-control. J Polit Econ 89(2):392–406

Tomar S, Kent Baker H, Kumar S, Hoffmann AOI (2021) Psychological determinants of retirement financial planning behavior. Journal of Business Research , 133 (November 2020), 432–449. https://doi.org/10.1016/j.jbusres.2021.05.007

Topa G, Moriano JA, Depolo M, Alcover CM, Morales JF (2009) Antecedents and consequences of retirement planning and decision-making: a meta-analysis and model. J Vocat Behav 75(1):38–55. https://doi.org/10.1016/j.jvb.2009.03.002

Topa G, Moriano JA, Depolo M, Alcover CM, Moreno A (2011) Retirement and wealth relationships: Meta-analysis and SEM. Res Aging 33(5):501–528. https://doi.org/10.1177/0164027511410549

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222. https://doi.org/10.1111/1467-8551.00375

Ülkümen G, Cheema A (2011) Framing goals to influence personal savings: the role of specificity and construal level. J Mark Res 48(6):958–969. https://doi.org/10.1509/jmr.09.0516

United Nations, Department of Economic and Social, Affairs PD (2020) (2019). World Population Ageing 2019. In United Nations . http://link.springer.com/chapter/10.1007/ 978-94-007-5204-7_6

Utkarsh, Pandey A, Ashta A, Spiegelman E, Sutan A (2020) Catch them young: impact of financial Socialization, financial literacy and attitude towards money on the financial well-being of young adults. Int J Consumer Stud 44(6):531–541. https://doi.org/10.1111/ijcs.12583

Valente TW, Paredes P, Poppe P (1998) Matching the message to the process: the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res 24(3):366–385

Van Rooij M, Teppa F (2014) Personal traits and individual choices: taking action in economic and non-economic decisions. J Econ Behav Organ 100:33–43

van Rooij M, Lusardi A, Alessie R (2011) Financial literacy and stock market participation. J Financ Econ 101(2):449–472. https://doi.org/10.1016/j.jfineco.2011.03.006

Van Rooij MCJ, Lusardi A, Alessie RJM (2011a) Financial literacy and retirement planning in the Netherlands. J Econ Psychol 32(4):593–608. https://doi.org/10.1016/j.joep.2011.02.004

van Schie RJG, Dellaert BGC, Donkers B (2015) Promoting later planned retirement: construal level intervention impact reverses with age. J Econ Psychol 50:124–131. https://doi.org/10.1016/j.joep.2015.06.010

Venkatesh V, Morris M, Davis G, Davis F (2003) Factors influencing the Use of M-Banking by academics: Case Study sms-based M-Banking. MIS Q 27(3):425–478

Vitt LA (2004) Consumers’ financial decisions and the psychology of values. J Financial Service Professionals 58(November):68–77. http://search.ebscohost.com/login.aspx?direct=true &db=bth&AN=14888952&site=ehost-live

Wang L, Lu W, Malhotra NK (2011) Demographics, attitude, personality, and credit card features correlate with credit card debt: a view from China. J Econ Psychol 32(1):179–193. https://doi.org/10.1016/j.joep.2010.11.006

World Economic Forum (2019) Investing in (and for) our future. Issue June. www.weforum.org

Xia T, Wang Z, Li K (2014) Financial literacy overconfidence and stock market participation. Soc Indic Res 119(3):1233–1245. https://doi.org/10.1007/s11205-013-0555-9

Xiao JJ, Chen C, Chen F (2014) Consumer financial capability and financial satisfaction. Soc Indic Res 118(1):415–432. https://doi.org/10.1007/s11205-013-0414-8

Yeung DY, Zhou X (2017) Planning for retirement: longitudinal effect on retirement resources and post-retirement well-being. Front Psychol 8:1300

Zhou R, Pham MT (2004) Promotion and prevention across mental accounts: when financial products dictate consumers’ investment goals. J Consum Res 31(1):125–135. https://doi.org/10.1086/383429

Download references

Acknowledgements

Authors would like to acknowledge the academicians and researchers who guided the search of the article and would like to thank the experts for the valuable inputs to refine the work.

There is no funding received for this research.

Author information

Authors and affiliations.

Symbiosis International (Deemed University), Pune, India

Kavita Karan Ingale

Symbiosis Institute of Operations Management, Symbiosis International (Deemed) University, Pune, India

Ratna Achuta Paluri

You can also search for this author in PubMed Google Scholar

Contributions

Both authors contributed to the conceptualization, research design, methodology, analysis of the data,writing of the manuscript and its revision.

Corresponding author

Correspondence to Kavita Karan Ingale .

Ethics declarations

Conflict of interest.

The authors declare that there is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Ingale, K.K., Paluri, R.A. Retirement planning – a systematic review of literature and future research directions. Manag Rev Q (2023). https://doi.org/10.1007/s11301-023-00377-x

Download citation

Received : 14 December 2022

Accepted : 04 October 2023

Published : 28 October 2023

DOI : https://doi.org/10.1007/s11301-023-00377-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Retirement planning
Systematic literature review
Financial behavior
Household finance
Long-term savings
Pension plan
Financial literacy
TCCM framework
Find a journal
Publish with us
Track your research

Open access
Published: 09 April 2024

Active involvement in scientific research of persons living with dementia and long-term care users: a systematic review of existing methods with a specific focus on good practices, facilitators and barriers of involvement

Janneke M. Groothuijse 1 , 2 ,
Lisa S. van Tol 1 , 2 ,
C. C. M. (Toos) Hoeksel-van Leeuwen 1 , 2 ,
Johannes J. M. van Delden 3 ,
Monique A. A. Caljouw 1 , 2 &
Wilco P. Achterberg 1 , 2

BMC Geriatrics volume 24 , Article number: 324 ( 2024 ) Cite this article

119 Accesses

Metrics details

Active involvement of persons living with dementia (PLWD) and long-term care (LTC) users in research is essential but less developed compared to other patient groups. However, their involvement in research is not only important but also feasible. This study aims to provide an overview of methods, facilitators, and barriers for involving PLWD and LTC users in scientific research.

A systematic literature search across 12 databases in December 2020 identified studies involving PLWD, LTC users, or their carers beyond research subjects and describing methods or models for involvement. Qualitative descriptions of involvement methods underwent a risk of bias assessment using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist 2018. A data collection sheet in Microsoft Excel and thematic analysis were used to synthesize the results.

The eighteen included studies delineated five core involvement methods spanning all research phases: advisory groups, formal and informal research team meetings, action groups, workshops, and co-conducting interviews. Additionally, two co-research models with PLWD and carers were found, while only two studies detailed LTC user involvement methods. Four distinct involvement roles were identified: consulting and advisory roles, co-analysts, co-researchers, and partners. The review also addressed barriers, facilitators, and good practices in the preparation, execution, and translation phases of research, emphasizing the importance of diversity, bias reduction, and resource allocation. Trust-building, clear roles, ongoing training, and inclusive support were highlighted.

Conclusions

Planning enough time for active involvement is important to ensure that researchers have time to build a trusting relationship and meet personal needs and preferences of PLWD, LTC users and carers. Researchers are advised not to presume the meaning of burden and to avoid a deficit perspective. A flexible or emergent design could aid involved persons’ ownership of the research process.

Trial registration

Prospero 2021: CRD42021253736.

Peer Review reports

In research characterized by active involvement, the target group plays a pivotal role in shaping research decisions and outcomes, directly impacting them. Involving patients in health research offers significant benefits [ 1 , 2 ]: it enhances participant recruitment [ 2 ], refines research questions [ 2 ], aligns study results with the target population [ 1 , 2 ], and promotes effective implementation of findings [ 1 ]. Active involvement of patients has also benefits for themselves, namely an enhanced understanding of research, building relationships, personal development, improved health and wellbeing, and enjoyment and satisfaction [ 3 , 4 ]. It gives them a sense of purpose and satisfaction through their tangible impact.

However, for long-term care (LTC) users and persons living with dementia (PLWD) active involvement in research is less developed than for other patient groups [ 5 , 6 ]. PLWD and LTC users share similar care needs, encompassing assistance with activities of daily living (ADLs), medication management, medical condition monitoring, and emotional support. Furthermore, a substantial portion of LTC users comprises individuals living with dementia [ 7 ]. Additionally, statistical data from the United States reveals that one in four older individuals is likely to reside in long-term care (LTC) facilities [ 8 ], and approximately forty to eighty percent of LTC residents in the United States, Japan, Australia, and England experience dementia or severe memory problems [ 7 , 9 ].

Due to these considerations, we have chosen to combine the target audiences of PLWD and LTC users in our systematic review. However, it's important to note that while there are potential advantages to combining these target groups, there may also be challenges. PLWD and LTC users may have varying needs, preferences, and experiences, including differences in care requirements driven by individual factors like the stage of dementia, coexisting conditions, and personal preferences. Therefore, it's imperative to conduct comprehensive research and involve these communities to ensure that involvement approaches are not only inclusive but also tailored to meet their specific requirements.

Given our ageing population and the intricate health challenges faced by PLWD and LTC users, including their vulnerability and shorter life expectancy in old age, it's crucial to establish effective research involvement methods. These individuals have unique needs and preferences that require attention. They possess a voice, and as researchers, it is our responsibility to not only listen to them but also actively involve them in the research process. Consequently, it is essential to identify means through which the voices of PLWD and LTC users can be effectively heard and ensure that their input is incorporated into research.

Fortunately, publication of studies on involvement of PLWD and LTC users in scientific research is slowly increasing [ 5 , 9 , 10 , 11 ]. A few reviews have described how PLWD and LTC users were involved [ 5 , 9 , 10 ]. However, with the increasing attention for involvement, the understanding of when involvement is meaningful grows and stricter requirements can be imposed to increase the quality of active involvement [ 12 , 13 ]. To our knowledge there is no up to date overview of involvement methods used with either or both PLWD and LTC users. Such an overview of involvement methods for PWLD and LTC users would provide a valuable, comprehensive resource encompassing various stages of the research cycle and different aspects of involvement. It would equip researchers with the necessary guidance to navigate the complexities of involving PLWD and LTC users in their research projects.

Recognizing the need to enhance the involvement of PLWD and LTC users in scientific research, this systematic review aims to construct a comprehensive overview of the multiple methodologies employed in previous studies, along with an examination of the facilitators and barriers of involvement. Our overarching goal is to promote inclusive and effective involvement practices within the research community. To achieve this objective, this review will address the following questions: (1) What kind of methods are used and how are these methods implemented to facilitate involvement of PLWD and LTC users in scientific research? (2) What are the facilitators and barriers encountered in previous research projects involving PLWD and LTC users?

Protocol and registration

The search and analysis methods were specified in advance in a protocol. The protocol is registered and published in the PROSPERO database with registration number CRD42021253736. The search and analysis methods are also described below more briefly.

Information sources, search strategy, and eligibility criteria

In preparation of the systematic literature search, key articles and reviews about involvement of PLWD and LTC users in research were screened to identify search terms. In addition, Thesaurus and MeSH terms were used to broaden the search. The search was conducted on December 10, 2020, across multiple databases: PubMed, Medline, Embase, Emcare, Web of Science, Cochrane Library, PsycINFO, Academic Search Premier, JSTOR, Social Services Abstracts, Sociological Abstracts, Psychology and Behavioral Sciences Collection. The search terms were entered in "phrases". The search strategy included synonymous and related terms for dementia, LTC user, involvement, research, method, and long-term care. The full search strategy is provided in supplement 1 .

After conducting the search, records underwent initial screening based on titles and abstracts. Selected reports were retrieved for full-text assessment, and studies were evaluated for eligibility based on several criteria. However, no restriction was made regarding publication date. First, to be included studies had to be written in English, German, French, or Dutch. Second, we only included original research studies. Third, studies were excluded when the target group or their representatives were not involved in research, but only participated as research subjects. Fourth, studies were excluded when not describing involvement in research. Therefore, studies concerning involvement in care, policy, or self-help groups were excluded. Fifth, the focus of this systematic review is on methods. Therefore, studies with a main focus on the results, evaluation, ethical issues, and impact of involvement in research were excluded. Additionally, we have not set specific inclusion or exclusion criteria based on study design since our primary focus is on involvement methodologies, regardless of the chosen research design. Sixth, the included studies had to concern the involvement in research of PLWD or adult LTC users, whether living in the community or in institutional settings, as well as informal caregivers or other representatives of these groups who may represent PLWD and LTC users facing limitations. Studies that involved LTC users that were children or ‘young adults’, or their representatives, were excluded. Studies were also excluded if they involved mental healthcare users if it remained unclear if the care that they received entailed more than only treatment from mental healthcare providers, but for example also assistance with ADL.

Terminology

For readability purposes, we use the abbreviation PLWD to refer to persons diagnosed with dementia, and we use the abbreviation LTC users to refer to persons receiving long-term care, at home or as residents living in nursing homes or other residential facilities. We use the term carers to refer to informal caregivers and other representatives of either PLWD or LTC users. As clear and consistent definitions regarding participatory research remains elusive [ 14 , 15 ], we formulated a broad working definition of involvement in research so as not to exclude any approach to participatory research. We defined involvement in research as “research carried out ‘with’ or ‘by’ the target group” [ 16 ], where the target group or their representatives take part in the governance or conduct of research and have some degree of ownership of the research [ 12 ]. It concerns involvement in research in which lived experienced experts work alongside research teams. We use the terms participation and participants, to refer to people being part of the research as study subjects.

Selection process, data-collection process, and data items

Titles and abstracts were independently screened by the first and second author (JG and LT). Only the studies that both reviewers agreed and met the inclusion criteria were included in the full-text screening process. Any uncertainty about whether the studies truly described a model or approach for involvement, was resolved by a quick screening of the full-text paper. The full-text screening process was then conducted according to the same procedure by JG and LT. Any disagreement was resolved by discussion until consensus was reached. If no agreement could be reached, a third researcher (MC) was consulted. References of the included studies were screened for any missing papers.

The following information was collected on a data collection sheet in Microsoft Excel: year and country of publication, topic, research aim, study design, living situation of involved persons (at home or institutionalized), description of involved persons, study participants (study subjects), theories and methods used, type/role(s) of involvement, research phase(s), recruitment, consent approach, study setting, structure of participatory activities, training, resources, facilitators, barriers, ethics, benefits, impact, and definition of involvement used.

JG independently extracted data from all included studies, the involved co-researcher (THL) independently extracted data from two studies, the second author (LT) from five. Differences in the analysis were discussed with the co-researcher (THL) and second author (LT) until consensus was reached. As only minor differences emerged, limited to the facilitator and barrier categories, data from the remaining studies was extracted by JG.

Risk of bias assessment

Every research article identified through the systematic review exclusively comprised qualitative descriptions of the involvement method(s) employed. Consequently, all articles underwent evaluation using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist 2018 [ 17 ], as opposed to the checklists intended for quantitative or mixed methods research. All included studies were independently assessed on quality by two reviewers (JG,LT) and any disagreement was resolved by discussion until consensus was reached. The CASP Qualitative Checklist consists of ten questions. The checklist does not provide suggestions on scoring, the first author designed a scoring system: zero points if no description was provided (‘no’), one point if a minimal description was provided (‘can’t tell’) and two points when the question was answered sufficiently (‘yes’). The second question of the checklist, “is a qualitative methodology appropriate”, was not applicable to the aims (i.e., to describe involvement) of the included studies and was therefore excluded. The tenth question was translated into a ‘yes’, ‘can’t tell’, or ‘no’ score to fit the scoring system. A maximum of eighteen points could be assigned.

Synthesis methods

Tables were used to summarize the findings and to acquire an overview of (1) the kinds of methods used to enable involvement of PLWD, LTC users, or carers in scientific research, and (2) the facilitators and barriers for involving this target group in scientific research. As to the first research aim, the headings of the first two tables are based on the Guidance for Reporting Involvement of Patients and the Public, long form version 2 (GRIPP2-LF) [ 18 ]. Because our systematic review focusses on methods, only the topics belonging to sections two, three, and four were included. Following Shippee et al., three main research phases were distinguished: preparation, execution, and translation [ 19 ]. Furthermore, the following fields were added to the GRIPP2-LF: First author, year of publication, country of study, setting of involvement, frequency of meetings, and a summary description of activities.

Concerning the second research aim, the extracted facilitators, barriers, and good practices were imported per study in ATLAS.ti for qualitative data analysis. Following the method for thematic synthesis of qualitative studies in systematic reviews [ 20 ], all imported barriers, facilitators and good practices were inductively coded staying 'close' to the results of the original studies, which resulted in 50 initial codes. After multiple rounds of pile sorting [ 21 ], based on similarities and differences and discussions in the research team, this long code list was grouped into a total of 27 categories, which were thereafter subsequently organized into 14 descriptive themes within the three research phases (preparation, execution, translation).

Study selection and characteristics

The Prisma Flow Diagram was used to summarize the study selection process [ 22 ]. In the full text screening, 72 of the 93 remaining studies were excluded because they were not original research articles (n = 5), not about involvement (n = 8), not about involvement in a research project (n = 1), they did not describe a model or method for involvement (n = 34), or they were not about PLWD or LTC users (n = 24). The search resulted in 18 publications eligible for analysis (Fig. 1 ).

Preferred Reporting items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram

Table 1 presents the general study characteristics. Two studies explicitly aimed to develop a model for involvement or good practice, and both focus on co-research either with PLWD [ 23 ] or their carers [ 13 ]. The other sixteen provide a description of the involvement of PLWD [ 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ] or LTC users in their research projects [ 35 , 36 , 37 , 38 , 39 ].

Quality assessment

Table 1 presents the CASP-score per study [ 17 ]. Five scored 16 to 18 points [ 13 , 28 , 29 , 32 , 35 ], indicating high quality with robust methods, clear aims, and strong data analysis. Eleven scored 12 to 15 [ 23 , 24 , 26 , 30 , 32 , 33 , 34 , 36 , 37 , 38 , 39 ], showing generally strong methodologies but with some limitations. Two scored 9 or lower [ 25 , 27 ], signifying significant methodological and analytical shortcomings. Notably, these low-scoring studies were short articles lacking clear recommendations for involvement in research.

Design and implementation of involvement

Phases and methods of involvement.

Table 2 describes the involvement methods used for and the implementation of involvement in research. The included studies jointly presented methods for involvement in the three main research phases [ 19 ]. Regarding the preparation phase, which involves the preparatory work for the study, only three studies provided detailed descriptions of the methods employed [ 26 , 30 , 32 ]. The execution phase, encompassing the actual conduct of the research, was most frequently discussed [ 23 , 24 , 25 , 26 , 27 , 28 , 29 ]. Five studies addressed the translation phase [ 13 , 25 , 31 , 36 , 37 ], where the focus shifts to translating research findings into actionable outcomes.

The eighteen studies introduced a variety of involvement methods, categorizable into five groups: 1) advisory groups, 2) research team meetings (both formal and informal), 3) action groups, 4) workshops, and 5) co-research in interviews. In five studies, individuals including PLWD, LTCF residents, carers, and health professionals participated in advisory/reference groups [ 25 , 26 , 27 , 32 ], working groups [ 27 ], and panels [ 28 ]. These groups offered valuable feedback on research aspects, spanning protocols, design, questionnaires, and implementation of research. Meetings occurred at varying frequencies - monthly, quarterly, or biannually.

Two studies exemplify diverse research collaboration settings. One involving older individuals within an academic research team of five [ 37 ], and another featuring a doctoral student and a co-researcher conducting informal monthly discussions at a local coffee shop [ 31 ]. Brown et al. sought to minimize power differentials and enhance inclusivity [ 37 ], while Mann and Hung focused on benefiting people with dementia and challenging negative discourse on dementia [ 31 ].

An additional five studies employed methods involving frequent meetings, including action [ 35 , 39 ], inquiry [ 23 ], and discussion groups [ 29 , 36 ] In these groups, involved persons with lived experience contributed to developing a shared vision and community improvements, such as enhancing the mealtime experience in care facilities [ 35 ].

Seven studies involved individuals through workshops, often conducted over one or two sessions. These workshops contributed to generating recommendations [ 37 ], informing future e-health designs [ 29 , 30 ], and ensuring diverse perspectives and lived experiences were included in data analysis [ 13 , 24 , 32 , 33 ]. In three studies, representatives worked as co-researchers in interviews, drawing on personal experiences to enhance the interview process, making it more dementia-appropriate and enriching data collection [ 13 , 32 , 34 ]. Finally, one study involved representatives in the recruitment and conduct of interviews [ 38 ].

People involved

The number of persons involved varied from a single co-researcher [ 31 ] to 34 panel individuals providing feedback on their experiences in a clinical trial [ 28 ]. Thirteen studies focussed on PLWD: eleven involved PLWD themselves [ 23 , 24 , 25 , 26 , 27 , 29 , 30 , 31 , 32 , 33 , 34 ], one exclusively focused on caregivers [ 13 ], and another one involved people without or with mild cognitive impairment, who participated in a study examining the risks of developing Alzheimer's disease [ 28 ]. Although not all articles provided descriptions of the dementia stage, available information indicated that individuals involved typically fell within the early to mid-stages of dementia [ 29 , 30 , 32 , 33 , 34 ]. Next to PLWD and carers, two studies additionally involved organizational or advocacy representatives [ 25 , 27 ]. The other five studies concerned older adults living in a LTC facility. Two of them involved older residents themselves [ 35 , 39 ], the other three carers, older community/client representatives or health care practitioners [ 36 , 37 , 38 ].

Roles and level of involvement

Four general roles could be identified. First, consultation and advisory roles were held by PLWD and carers [ 25 , 26 , 27 , 28 , 29 , 30 , 32 ], where involved persons share knowledge and experiences to make suggestions [ 32 ], but the research team retained formal decision-making power [ 25 ]. Second, PLWD were involved as co-analysts in data analysis [ 24 , 32 , 33 ]. Co-analysts influence data analysis, but the decision-making power remained with academic researchers [ 24 ]. Third, in six studies the co-researcher role was part of the research design in which involved persons and researchers steer and conduct research together [ 13 , 23 , 31 , 32 , 34 , 36 ]. Finally, two studies partnered with LTC residents [ 35 , 39 ], with residents at the core of the group, and positioned as experts by experience [ 39 ]. Residents had the decision-making authority regarding how to improve life in LTC facilities [ 35 ].

Models for involvement in research

Only two studies designed a model for co-research with PLWD [ 23 ] or their carers [ 13 ] across all research phases. These models underscored the importance of iterative training for co-researchers [ 13 , 23 ] and academic researchers [ 23 ]. Furthermore, these studies advocate involving co-researchers early on in the research process [ 13 ] and in steering committees [ 23 ]. Co-researchers can be involved in designing research materials [ 23 ], conducting interviews [ 13 , 23 ], analysing data [ 13 ], and co-disseminating findings [ 13 , 23 ]. Additionally, one study stressed involving PLWD in identifying (future) research priorities [ 23 ].

Barriers, facilitators, and good practices in research phases

Preparation phase.

Table 3 describes the barriers, facilitators, and good practices per main research phase. Lack of diversity in ethnicity and stages of dementia in the recruitment of involved persons is mentioned as a recurring barrier [ 26 , 28 , 32 , 33 ]. The exclusion of people with cognitive impairments is partly due to gatekeepers’ and recruiters’ bias towards cognitively healthy people [ 28 , 32 ]. It is stressed that researchers should refrain from making assumptions about the abilities of PLWD and ask the person what he/she is willing to do [ 31 ]. It is considered good practice to involve people regardless of cognitive abilities [ 23 ], based on skills, various personal characteristics [ 13 ] and, if possible, relevant prior experience [ 38 ].

Many studies stress the importance of building a mutual trusting relationship between involved persons and academic researchers [ 13 , 23 , 31 , 33 , 34 , 37 ]. A good relationship is believed to break down social barriers [ 37 ], foster freedom of expression [ 33 ], and thereby avoiding tokenistic involvement [ 13 ]. In addition, spending time with these persons is important to become familiar with an individual’s strengths and limitations [ 31 ].

Opting for naturally evolving involvement roles was mentioned as a barrier, as this may result in conflicting expectations and irrelevant tasks [ 37 ]. A clear role description and clarification of tasks is key to balancing potentially different expectations of the involved persons and researchers [ 26 , 28 , 29 , 32 , 38 ]. When designing a role for involvement in research, good practices dictate taking into account personal skills, preferences, development goals, and motivation for involvement [ 13 , 32 ]. This role should ideally be designed in collaboration with involved persons [ 13 , 32 ].

The perception of providing training to involved persons is ambivalent. Studies cited that training should not aim to transform them into “pseudo-scientist” [ 32 , 37 ] and that it raises the costs for involvement [ 28 ]. However, multiple scholars emphasize the importance of providing iterative training to facilitate meaningful involvement and development opportunities [ 13 , 23 , 28 , 31 , 32 , 33 , 36 , 37 ]. Training can empower involved persons to engage in the research process equally and with confidence, with the skills to fulfil their role [ 13 , 33 , 38 ]. However, the implementation of training may present a potential conflict with the fundamental principle of valuing experiential knowledge [ 37 ] and should avoid the objective of transforming co-researchers into 'expert' researchers [ 32 ]. Academic researchers should also be offered training on how to facilitate meaningful involvement [ 13 , 23 , 28 , 31 ].

Limited time and resources were mentioned as barriers to involvement that can delay the research process [ 13 , 33 , 36 , 39 ], restrict the involvement [ 28 ] and hinder the implementation of developed ideas [ 39 ]. Financial compensation for involvement is encouraged [ 25 , 26 , 27 , 32 ], as it acknowledges the contribution of involved persons [ 13 ]. Thus, meaningful involvement in research requires adequate funding and infrastructure to support the involvement activities [ 13 , 28 , 33 , 37 ].

Execution phase

The use of academic jargon and rapid paced discussions [ 13 , 37 ], power differentials, and the dominant discourse in biomedical research on what is considered “good science” can limit the impact of involvement [ 13 , 24 , 32 , 36 , 37 ]. Facilitating researchers should reflect on power differentials [ 35 ] and how decision-making power is shared [ 31 ]. Other facilitating factors are making a glossary of terms used and planning separate meetings for “technical topics” [ 37 ]. In addition, an emergent research design [ 35 ] or a design with flexible elements [ 28 ] can increase ownership in the research project and provide space for involvement to inform the research agenda [ 28 , 35 ]. This requires academic researchers to value experiential knowledge and to have an open mind towards the evolving research process [ 13 , 23 , 31 ].

Furthermore, managing the involvement process and ensuring equity in the collaboration [ 13 , 32 , 33 ], facilitating researchers must encourage involved persons to voice their perspectives. This means that they sometimes need to be convinced that they are experts of lived experience [ 32 , 33 , 36 , 37 , 39 ]. To enable involvement of PLWD, the use of visual and creative tools to prompt memories can be considered [ 24 , 30 , 33 , 34 ], as well as flexibility in relation to time frames and planning regular breaks to avoid too fast a pace for people who may tire easily [ 24 , 25 , 29 , 30 ].

Involvement can be experienced as stressful [ 13 , 32 , 38 ] and caring responsibilities may interfere [ 26 ]. Tailored [ 29 ] physical and emotional support should therefore be offered [ 13 , 23 , 38 ] without making assumptions about the meaning of burden [ 30 , 31 ]. Moreover, being the only PLWD involved in an advisory group was experienced as intimidating [ 25 ] and, ideally, a larger team of PLWD is involved to mitigate responsibilities [ 37 ]. PLWD having a focal point of contact [ 28 , 37 ] and involving nurses or other staff with experience working with PLWD and their carers [ 29 , 30 ] are mentioned as being beneficial. Some stress the importance of involving carers when engaging with PLWD in research [ 25 , 29 , 30 ].

To avoid an overload of information that is shared with the involved persons, tailoring information-sharing formats to individual preferences and abilities is essential to make communication effective [ 27 , 37 ].

Translation

Two studies indicated a need for more robust evaluation measures to assess the effect of involvement [ 28 , 33 ]. Reflection and evaluation of the involvement serves to improve the collaboration and to foster introspective learning [ 13 , 23 , 26 , 31 ]. The included studies evaluated involvement through the use of reflective diaries [ 13 ] or a template [ 38 ] with open-ended questions [ 33 ].

Two studies postulate that findings should benefit and be accessible to PLWD [ 23 , 31 ]. The use of creative tools not only enables involvement of PLWD, but can also increase accessibility of research findings and expand the present representation of PLWD [ 23 ].

The 18 included studies presented multiple methods for involvement in all three research phases. We found five types of involvement: advisory groups, (formal and informal) research team meetings, action groups, workshops, and co-conducting interviews. Only two studies described methods for involvement of LTC users in research. Involved persons were most often involved in consulting and advisory roles, but also as co-analysts, co-researchers, and partners. Involved persons’ roles can evolve and change over time. Especially as involved persons grow into their role, and gain confidence and knowledge of the specific research project, a more active role with shared responsibilities can become part of the research project. In addition, multiple involvement roles can be used throughout the research depending on the research phase.

Compared to the five types of involvement that we identified, other literature reviews about involvement methods for LTC users and PLWD in research also described advisory groups [ 10 ] and workshops [ 5 , 11 ], and methods that were similar to research team meetings (drop-in sessions and meetings [ 11 ]). Methods for action research (action groups) and co-conducting research (interviews) were not included by these other review studies. In addition to our findings, these other reviews also described as involvement methods interviews and focus groups [ 5 , 10 ] surveys [ 10 ], reader consultation [ 11 ]. Those types of methods were excluded from our study, because our definition of involvement is more strict; collecting opinions is not involvement per se, but sometimes only study participation. Moreover, compared to these previous reviews we set a high standard for transparency about the participation methods and the level of detail at which they are described.

Engaging the target group in research, particularly when collaborating with PLWD, LTC users, and carers, involves navigating unforeseen challenges [ 40 ]. This requires academic researchers to carefully balance academic research goals and expectations, and the expectations, personal circumstances and development goals related to the involved person. The aim is to maximize involvement while being attentive to the individual’s needs and avoiding a deficit perspective. Effective communication should be established, promoting respect, equality, and regular feedback between all stakeholders, including individuals living with dementia and LTCF staff. Building a mutual trusting relationship between involved persons and academic researchers through social interaction and clear communication is key to overcome barriers and ensure meaningful involvement. Inclusivity and empowerment, along with fostering an environment where diverse voices are heard, are crucial for the success of involvement in research. Our results are in line with a recent study concerning the experiences of frail older persons with involvement in research, confirming the importance of avoiding stereotypic views of ageing and frailty, building a trusting relationship, and being sensitive to older persons’ preferences and needs [ 41 ].

Furthermore, our results show that training academic researchers and involved persons is essential to develop the skills to facilitate involvement and to fulfil their role with confidence, respectively. Whilst the need for training is acknowledged by others [ 41 , 42 ], there are legitimate objections to the idea of training involved persons, as the professionalization underpinning the concept of training is at odds with voicing a lay perspective [ 43 , 44 ]. Furthermore, it is argued that experiential knowledge is compromised when training is structured according to the dominant professional epistemology of objectivity [ 45 ]. Therefore, training of involved persons should not focus on what researchers think they ought to know, but on what they want to learn [ 41 ].

Academic culture was frequently mentioned as a barrier to meaningful involvement. This result resonates with the wider debate related to involvement in health research which is concerned about active or “authentic involvement” being replaced with the appropriation of the patient voice as an add-on to conventional research designs [ 12 , 46 ]. It is argued that such tokenistic involvement limits the involved persons’ ability to shape research outcomes [ 46 ]. To reduce tokenism requires a culture shift [ 13 ]. We believe that due to the strict definition of involvement and high transparency standard used in this review, tokenistic approaches were excluded. This may set an example for how to stimulate making this culture shift.

Furthermore, the importance of practical aspects such as funding and, by extension, the availability of time should not be underestimated. Adequate funding is necessary for compensation of involvement, but also to ensure that researchers have ample time to plan involvement activities and provide personalized support for PLWD, LTC residents and their carers. Funding bodies increasingly require involvement of the public to be part of research proposals. Yet, support in terms of financial compensation and time for the implementation of involvement in research is rarely part of funding grants [ 42 ]. In addition, whereas an emergent design could aid the impact of involvement, funders often require a pre-set research proposal in which individual components are already fixed [ 5 , 47 ]. This indicates that not only do academic researchers and culture need to change, academic systems also need to be modified in order to facilitate and nurture meaningful involvement [ 47 ].

Strengths and limitations

A key strength of this review is the inclusion of over ten scientific databases, with a reach beyond the conventional biomedical science databases often consulted in systematic reviews. Besides, we believe that we have overcome the inconsistent use of terminology of involvement in research by including also other terms used, such as participation and engagement, in our search strategy. However, there was also inconsistency in length of publications and precision of the explanation of the process of involvement. E.g., involvement in the execution phase was often elaborated on, contributions to the research proposal and co-authoring research findings were only stated and not described. This presented challenges for data extraction and analysis, as it was not always possible to identify how the target group was involved. Involvement in these research phases is therefore not fully represented in this review.

The included studies in this review, the majority of which are of high quality, provide methods for involvement of PLWD and LTC users in research and they do not explicitly attend to the effectiveness or impact of the method for involvement used. Therefore, a limitation of this review is that it cannot make any statements regarding the effectiveness of the involvement methods included. Moreover, our target population was broad, although PLWD and LTC users are largely overlapping in their care needs and share important features, this may have led to heterogeneous results. In future research, it would be interesting to interpret potential differences between involvement of PLWD, LTC users, and their carers. However, as we expected, the amount of literature included in our analyses was too limited to do so. Furthermore, whereas the broad target group is a limitation it is also a strength of our review. Limiting our search to specifically persons living in LTC facilities would have provided limited methods for involvement of persons living with dementia. Our broad target groups enabled us to learn from research projects in which people living with early staged dementia are directly involved from which we can draw lessons on the involvement of people with more advanced stages of dementia and persons living with cognitive problems who live within LTC facilities.

Since January 2021 quite some research has been published about the importance of involvement in research. Although we had quickly screened for new methods, we realise that we may have missed some involvement methods in the past years. There will be a need for a search update in the future.

Implications for future research

Our review shows that a flexible and emergent design may help to increase involved persons' influence on and ownership in the research process. However, not all research objectives may be suitable for the implementation of an emergent design. Future research should therefore examine how aspects of a flexible emergent design can be integrated in, e.g., clinical research without compromising the validity of research outcomes.

Alzheimer Europe has called for the direct involvement of persons living with dementia in research [ 48 ]. In addition, Swarbrick et al. (this review) advise to involve persons regardless of their cognitive abilities [ 23 ]. These statements question the involvement of proxies, such as carers, professional caregivers and others involved in the care of PLWD. While PLWD and persons with other cognitive problems constitute a significant group within residential and nursing homes [ 7 ], none of the studies included in this review have provided methods to directly involve persons with more advanced stages of dementia. This raises the question if research methods should be adapted to allow those with more advanced stages of dementia to be involved themselves or if, concerning the progressive nature of the disease, it is more appropriate to involve proxies. And secondly who should these proxies be? Those that care for and live with persons with an advanced stage of dementia, or for example a person living with an early stage of dementia to represent the voices of persons with more advanced stages of dementia [ 31 ]?

Future research should adopt our example for stricter requirements for involvement and transparency about the involvement methods used. This will reduce tokenistic involvement and further promote the culture shift towards meaningful involvement. In addition, future research should assess the impact of the involvement methods that are described in this review. One of the first instruments that that may be used to do so in varying healthcare settings is the Public and Patient Engagement Evaluation Tool (PPEET) [ 49 ]. Moreover, scholars in this review stress, and we agree with this, that future research is needed on the involvement of persons with more advanced stages of dementia to ensure their voices are not excluded from research [ 33 , 34 ].

This review provides an overview of the existing methods used to actively involve PLWD, LTC users, and carers in scientific research. Our findings show that their involvement is feasible throughout all research phases. We have identified five different methods for involvement, four different roles, and two models for co-research. Our results suggest that planning enough time for involving PLWD, LTC users, and carers in research, is important to ensure that researchers have time to build a trusting relationship and meet their personal needs and preferences. In addition, researchers are advised not to presume the meaning of burden and to avoid a deficit perspective. A flexible or emergent design could aid involved persons’ ownership in the research process.

Availability of data and materials

The full search strategy is provided in supplement 1 . The data extraction form can be provided by the corresponding author on reasonable request.

Abbreviations

Critical Appraisal Skills Programme

Guidance for Reporting Involvement of Patients and the Public, long form version 2

Long-term care

Persons living with dementia

Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):1–9. https://doi.org/10.1186/1472-6963-14-89 .

Article Google Scholar

Brett J, Staniszewska S, Mockford C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient-Patient-Centered Outcomes Res. 2014;7(4):387–95. https://doi.org/10.1007/s40271-014-0065-0 .

Staley K. Exploring impact: Public involvement in NHS, public health and social care research. 2009. Available from: https://www.invo.org.uk/wp-content/uploads/2011/11/Involve_Exploring_Impactfinal28.10.09.pdf . Accessed 17 Jan 2024

Ashcroft J, Wykes T, Taylor J, et al. Impact on the individual: what do patients and carers gain, lose and expect from being involved in research? J Ment Health. 2016;25(1):28–35. https://doi.org/10.3109/09638237.2015.1101424 .

Article PubMed PubMed Central Google Scholar

Backhouse T, Kenkmann A, Lane K, et al. Older care-home residents as collaborators or advisors in research: a systematic review. Age Ageing. 2016;45(3):337–45. https://doi.org/10.1093/ageing/afv201 .

Bendien E, Groot B, Abma T. Circles of impacts within and beyond participatory action research with older people. Ageing Soc 2020:1–21. https://doi.org/10.1017/S0144686X20001336 .

Lepore MED, Meyer J, Igarashi A. How long-term care quality assurance measures address dementia in Australia, England, Japan, and the United States. Ageing Health Res 2021;1(2). https://doi.org/10.1016/j.ahr.2021.100013 .

Freedman VA, Spillman BC. Disability and care needs among older Americans. Milbank Q. 2014;92(3):509–41. https://doi.org/10.1111/1468-0009.12076 .

Bethell J, Commisso E, Rostad HM, et al. Patient engagement in research related to dementia: a scoping review. Dementia. 2018;17(8):944–75. https://doi.org/10.1177/1471301218789292 .

Article PubMed Google Scholar

Miah J, Dawes P, Edwards S, et al. Patient and public involvement in dementia research in the European Union: a scoping review. BMC Geriatr. 2019;19(1):1–20. https://doi.org/10.1186/s12877-019-1217-9 .

Schilling I, Gerhardus A. Methods for involving older people in health research—a review of the literature. Int J Environ Res Public Health. 2017;14(12):1476. https://doi.org/10.3390/ijerph14121476 .

Andersson N. Participatory research—A modernizing science for primary health care. J Gen Fam Med. 2018;19(5):154–9. https://doi.org/10.1002/jgf2.187 .

Di Lorito C, Godfrey M, Dunlop M, et al. Adding to the knowledge on patient and public involvement: reflections from an experience of co-research with carers of people with dementia. Health Expect. 2020;23(3):691–706. https://doi.org/10.1111/hex.13049 .

Islam S, Small N. An annotated and critical glossary of the terminology of inclusion in healthcare and health research. Res Involve Engage. 2020;6(1):1–9. https://doi.org/10.1186/s40900-020-00186-6 .

Rose D. Patient and public involvement in health research: Ethical imperative and/or radical challenge? J Health Psychol. 2014;19(1):149–58. https://doi.org/10.1177/1359105313500249 .

NIHR. Briefing notes for researchers: public involvement in NHS, health and social care research. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371#Involvement . Accessed 12 Jan 2023.

Critical Appraisal Skills Programme. CASP Qualitative Checklist. 2018. https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf . Accessed 12 Jan 2023.

Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3:13. https://doi.org/10.1186/s40900-017-0062-2 .

Article CAS PubMed PubMed Central Google Scholar

Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66. https://doi.org/10.1111/hex.12090 .

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):1–10. https://doi.org/10.1186/1471-2288-8-45 .

Saldana JM. The coding manual for qualitative researchers. London: SAGE Publications; 2015.

Google Scholar

Page MJ, McKenzie JE, Bossuyt PM, The PRISMA, et al. statement: an updated guideline for reporting systematic reviews. BMJ. 2020;2021:372. https://doi.org/10.1136/bmj.n71 .

Swarbrick C, Doors O, et al. Developing the co-researcher involvement and engagement in dementia model (COINED): A co-operative inquiry. In: Keady J, Hydén L-C, Johnson A, et al., editors. Social research methods in dementia studies: Inclusion and innovation. New York, Routledge: Taylor & Francis Group; 2018. p. 8–19.

Clarke CL, Wilkinson H, Watson J, et al. A seat around the table: participatory data analysis with people living with dementia. Qual Health Res. 2018;28(9):1421–33. https://doi.org/10.1177/1049732318774768 .

Flavin T, Sinclair C. Reflections on involving people living with dementia in research in the Australian context. Australas J Ageing. 2019;38(Suppl 2):6–8. https://doi.org/10.1111/ajag.12596 .

Giebel C, Roe B, Hodgson A, et al. Effective public involvement in the HoST-D programme for dementia home care support: From proposal and design to methods of data collection (innovative practice). Dementia (London). 2019;18(7–8):3173–86. https://doi.org/10.1177/1471301216687698 .

Goeman DP, Corlis M, Swaffer K, et al. Partnering with people with dementia and their care partners, aged care service experts, policymakers and academics: a co-design process. Australas J Ageing. 2019;38(Suppl 2):53–8. https://doi.org/10.1111/ajag.12635 .

Gregory S, Bunnik EM, Callado AB, et al. Involving research participants in a pan-European research initiative: the EPAD participant panel experience. Res Involv Engagem. 2020;6:62. https://doi.org/10.1186/s40900-020-00236-z .

Hanson E, Magnusson L, Arvidsson H, et al. Working together with persons with early stage dementia and their family members to design a user-friendly technology-based support service. Dementia. 2007;6(3):411–34. https://doi.org/10.1177/1471301207081572 .

Hassan L, Swarbrick C, Sanders C, et al. Tea, talk and technology: patient and public involvement to improve connected health “wearables” research in dementia. Res Involv Engagem. 2017;3:12. https://doi.org/10.1186/s40900-017-0063-1 .

Mann J, Hung L. Co-research with people living with dementia for change. Action Research. 2019;17(4):573–90. https://doi.org/10.1177/1476750318787005 .

Poland F, Charlesworth G, Leung P, et al. Embedding patient and public involvement: managing tacit and explicit expectations. Health Expect. 2019;22(6):1231–9. https://doi.org/10.1111/hex.12952 .

Stevenson M, Taylor BJ. Involving individuals with dementia as co-researchers in analysis of findings from a qualitative study. Dementia (London). 2019;18(2):701–12. https://doi.org/10.1177/1471301217690904 .

Tanner D. Co-research with older people with dementia: experience and reflections. J Ment Health. 2012;21(3):296–306. https://doi.org/10.3109/09638237.2011.651658 .

Baur V, Abma T. “The Taste Buddies”: participation and empowerment in a residential home for older people. Ageing Soc. 2012;32(6):1055–78. https://doi.org/10.1017/S0144686X11000766 .

Beukema L, Valkenburg B. Demand-driven elderly care in the Netherlands. Action Research. 2007;5(2):161–80. https://doi.org/10.1177/1476750307077316 .

Brown LJE, Dickinson T, Smith S, et al. Openness, inclusion and transparency in the practice of public involvement in research: a reflective exercise to develop best practice recommendations. Health Expect. 2018;21(2):441–7. https://doi.org/10.1111/hex.12609 .

Froggatt K, Goodman C, Morbey H, et al. Public involvement in research within care homes: benefits and challenges in the APPROACH study. Health Expect. 2016;19(6):1336–45. https://doi.org/10.1111/hex.12431 .

Shura R, Siders RA, Dannefer D. Culture change in long-term care: participatory action research and the role of the resident. Gerontologist. 2011;51(2):212–25. https://doi.org/10.1093/geront/gnq099 .

Cook T. Participatory research: Its meaning and messiness. Beleidsonderzoek Online. 2020;3:1–21. https://doi.org/10.5553/BO/221335502021000003001 .

Haak M, Ivanoff S, Barenfeld E, et al. Research as an essentiality beyond one’s own competence: an interview study on frail older people’s view of research. Res Involv Engage. 2021;7(1):1–8. https://doi.org/10.1186/s40900-021-00333-7 .

Chamberlain SA, Gruneir A, Keefe JM, et al. Evolving partnerships: engagement methods in an established health services research team. Res Involve Engage. 2021;7(1):1–11. https://doi.org/10.1186/s40900-021-00314-w .

Bélisle-Pipon J-C, Rouleau G, Birko S. Early-career researchers’ views on ethical dimensions of patient engagement in research. BMC Med Ethics. 2018;19(1):1–10. https://doi.org/10.1186/s12910-018-0260-y .

Ives J, Damery S, Redwod S. PPI, paradoxes and Plato: who’s sailing the ship? J Med Ethics. 2013;39(3):181–5. https://doi.org/10.1136/medethics-2011-100150 .

De Graaff M, Stoopendaal A, Leistikow I. Transforming clients into experts-by-experience: a pilot in client participation in Dutch long-term elderly care homes inspectorate supervision. Health Policy. 2019;123(3):275–80. https://doi.org/10.1016/j.healthpol.2018.11.006 .

Cook T. Where Participatory Approaches Meet Pragmatism in Funded (Health) Research: The Challenge of Finding Meaningful Spaces. Forum Qualitative Sozialforschung Forum: Qualitative Social Research. 2012;13(1). https://doi.org/10.17169/fqs-13.1.1783 .

Paylor J, McKevitt C. The possibilities and limits of “co-producing” research. Front Sociol. 2019;4:23. https://doi.org/10.3389/fsoc.2019.00023 .

Gove D, Diaz-Ponce A, Georges J, et al. Alzheimer Europe’s position on involving people with dementia in research through PPI (patient and public involvement). Aging Ment Health. 2018;22(6):723–9. https://doi.org/10.1080/13607863.2017.1317334 .

Abelson J, Li K, Wilson G, et al. Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool. Health Expect. 2016;19(4):817–27. https://doi.org/10.1111/hex.12378 .

Download references

Acknowledgements

We thank Jan W. Schoones, information specialist Directorate of Research Policy (formerly: Walaeus Library, Leiden University Medical Centre, Leiden, the Netherlands), for helping with the search.

This systematic review received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Public Health and Primary Care, Leiden University Medical Center, P.O. Box 9600, 2300, RC, Leiden, the Netherlands

Janneke M. Groothuijse, Lisa S. van Tol, C. C. M. (Toos) Hoeksel-van Leeuwen, Monique A. A. Caljouw & Wilco P. Achterberg

University Network for the Care Sector Zuid-Holland, Leiden University Medical Center, Leiden, The Netherlands

Department of Medical Humanities, University Medical Center, Utrecht, The Netherlands

Johannes J. M. van Delden

You can also search for this author in PubMed Google Scholar

Contributions

Concept: HD, LT, MC, WA; design of study protocol: JG, LT, MC; data collection and extraction: JG, LT, MC, THL; data analysis and interpretation: all authors; writing, editing and final approval of the manuscript: all authors.

Corresponding author

Correspondence to Monique A. A. Caljouw .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Competing interest.

There are no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Reprints and permissions

About this article

Cite this article.

Groothuijse, J.M., van Tol, L.S., Leeuwen, C.C.M.(.Hv. et al. Active involvement in scientific research of persons living with dementia and long-term care users: a systematic review of existing methods with a specific focus on good practices, facilitators and barriers of involvement. BMC Geriatr 24 , 324 (2024). https://doi.org/10.1186/s12877-024-04877-7

Download citation

Received : 03 February 2023

Accepted : 05 March 2024

Published : 09 April 2024

DOI : https://doi.org/10.1186/s12877-024-04877-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Involvement in research
Older residents

BMC Geriatrics

ISSN: 1471-2318

Submission enquiries: [email protected]
General enquiries: [email protected]

IMAGES

how to write a qualitative literature review
Sample of Research Literature Review
Sample of Research Literature Review
Role Of Literature Review In Qualitative Research
15 Literature Review Examples (2024)
When to do the literature review in qualitative research? Before, after

VIDEO

Different types of Research Designs|Quantitative|Qualitative|English| part 1|
2023 PhD Research Methods: Qualitative Research and PhD Journey
Academic Writing Workshop
Literature review Qual vs Quan
What is qualitative research?
What is a review of literature in research?

COMMENTS

Qualitative Research: Literature Review
In The Literature Review: A Step-by-Step Guide for Students, Ridley presents that literature reviews serve several purposes (2008, p. 16-17). Included are the following points: Historical background for the research; Overview of current field provided by "contemporary debates, issues, and questions;" Theories and concepts related to your research;
Criteria for Good Qualitative Research: A Comprehensive Review
For this review, a comprehensive literature search was performed from many databases using generic search terms such as Qualitative Research, Criteria, etc. The following databases were chosen for the literature search based on the high number of results: IEEE Explore, ScienceDirect, PubMed, Google Scholar, and Web of Science.
How to Write a Literature Review
Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.
Literature review as a research methodology: An overview and guidelines
As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.
Chapter 9. Reviewing the Literature
A literature review is a comprehensive summary of previous research on a topic. It includes both articles and books—and in some cases reports—relevant to a particular area of research. Ideally, one's research question follows from the reading of what has already been produced. For example, you are interested in studying sports injuries ...
Planning Qualitative Research: Design and Decision Making for New
Qualitative research draws from interpretivist and constructivist paradigms, seeking to deeply understand a research subject rather than predict outcomes, as in the positivist paradigm (Denzin & Lincoln, 2011).Interpretivism seeks to build knowledge from understanding individuals' unique viewpoints and the meaning attached to those viewpoints (Creswell & Poth, 2018).
How to Write a Literature Review
Summarizing themes and patterns. Literature reviews can help develop theory by synthesizing and building on existing knowledge. This is an approach to knowledge called synthesis. A literature review helps draw connections across various studies and brings insights that individual papers may not be able to provide.
What is Qualitative in Qualitative Research
What is qualitative research? If we look for a precise definition of qualitative research, and specifically for one that addresses its distinctive feature of being "qualitative," the literature is meager. In this article we systematically search, identify and analyze a sample of 89 sources using or attempting to define the term ...
How to use and assess qualitative research methods
Qualitative research is defined as "the study of the nature of phenomena ... The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. ... a recent scoping review found that 80% of clinical research did not address the ...
PDF Qualitative Analysis Techniques for the Review of the Literature
literature review is no more complicated than writing a high school term paper" (Boote & Beile, 2005, p. 5). According to Fraenkel and Wallen (2006), "A literature review is ... number of books on research methodology, qualitative research, statistics, measurement, and the like, as noted by Boote and Beile (2005), there has been "a ...
Writing a literature review
Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...
Writing a Literature Review
Qualitative versus quantitative research; Empirical versus theoretical scholarship; Divide the research by sociological, historical, or cultural sources; Theoretical: In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key ...
Qualitative research
Literature review. You will need to write a short, overview literature review to introduce the main theories, concepts and key research areas that you will explore in your dissertation. This set of texts - which may be theoretical, research-based, practice-based or policies - form your theoretical framework.
Qualitative systematic reviews: their importance for our understanding
A qualitative systematic review brings together research on a topic, systematically searching for research evidence from primary qualitative studies and drawing the findings together. There is a debate over whether the search needs to be exhaustive. 1 , 2 Methods for systematic reviews of quantitative research are well established and explicit ...
5. The Literature Review
A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...
What Is Qualitative Research?
Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research. Qualitative research is the opposite of quantitative research, which involves collecting and ...
Methodological Approaches to Literature Review
A literature review is defined as "a critical analysis of a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles." (The Writing Center University of Winconsin-Madison 2022) A literature review is an integrated analysis, not just a summary of scholarly work on a specific topic.
Guidance on Conducting a Systematic Literature Review
Literature review is an essential feature of academic research. Fundamentally, knowledge advancement must be built on prior existing work. To push the knowledge frontier, we must know where the frontier is. By reviewing relevant literature, we understand the breadth and depth of the existing body of work and identify gaps to explore.
How to Operate Literature Review Through Qualitative and ...
3.5 Step 5: Qualitative Analysis. The literature review is an essential part of the research process. There are several types of the literature review [44, 45]. However, in general, the literature review is a process of questioning. It is intended to answer some questions about a particular topic: What are the primary literature sources?
Literature Review
Nonetheless, literature review is a continuous sense-making process -- you need to review the literature continuously in order to organize your thoughts and refine your analysis. A good literature review should be able to: Connect to your research questions; Connect to your choice of methods and research design; Support your data analysis
A Guide to Writing a Qualitative Systematic Review Protocol to Enhance
Methodology: The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction ...
Administrative Burden in Citizen-State Interactions: A Systematic
We adhere to the PRISMA guidelines when conducting our systematic literature review (Page et al. 2021). These guidelines were developed to ensure that literature reviews are comprehensive, transparent, and well documented to minimize reporting biases and ensure reproducibility. ... in addition to more qualitative research, the literature would ...
Why Qualitative Research Needs More and Better Systematic Review
Those doing qualitative research cannot "opt out" of knowing their relevant scholarly conversations. Undertaking a qualitative systematic review provides a vital means to know and tune into the past conversation in your topic area that allows the researcher to position themselves and their work substantively, ontologically, theoretically, and methodologically in this landscape.
Methods for the synthesis of qualitative research: a critical review
Background. The range of different methods for synthesising qualitative research has been growing over recent years [1,2], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [].While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe ...
Capturing artificial intelligence applications' value proposition in
To answer our research question, we adopted a qualitative inductive research design. This research design is consistent with studies that took a similar perspective on how technologies can create business value [].In conducting our structured literature review, we followed the approach of Webster and Watson [] and included recommendations of Wolfswinkel et al. [] when considering the inclusion ...
Book Review: Qualitative Literacy: A Guide to Evaluating Ethnographic
How do we determine whether a book featuring qualitative research is empirically sound and not just a lovely, interesting read? How do you evaluate qualitative research? ... Book Review: Qualitative Literacy: A Guide to Evaluating Ethnographic and Interview Research by Mario Luis Small and Jessica McCrory Calarco Oakland, CA, University of ...
Retirement planning
A literature review reveals a methodological diversity ranging from basic conceptual and exploratory research to descriptive and empirical analysis. Studies in the early days had primarily cross-sectional designs that applied regression analysis. Later, studies intermittently used qualitative research designs to gain deeper insights into the ...
Active involvement in scientific research of persons living with
Every research article identified through the systematic review exclusively comprised qualitative descriptions of the involvement method(s) employed. Consequently, all articles underwent evaluation using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist 2018 [ 17 ], as opposed to the checklists intended for quantitative or ...
Exploring the Human Condition: A Methodological Literature Review of
To investigate current practices and main research themes in fiction-based research, I conducted a critical review to classify and integrate existing studies, closely following best-practice recommendations for (methodological) literature reviews in the process (Aguinis et al., 2023; Celik et al., 2023; Hiebl, 2021; Koseoglu et al., 2022; Kunisch et al., 2023).
Sustainability
The environmental, social and governance (ESG) performance of construction enterprises still needs to be improved. Therefore, in order to better utilize resources effectively to improve enterprise ESG performance, this paper explores the configuration paths for Chinese construction enterprises to improve their ESG performance using the (fuzzy set qualitative comparative analysis) fsQCA method ...

Archer Library

Exploring the literature review

Your Literature Review

1. Select a Topic

Consider Purpose

★ Schedule a research appointment

2. Search the Literature

Books & eBooks: Archer Library & OhioLINK

Databases: Theses & Dissertations

Newspapers: Databases & Internet

Search Strategies & Boolean Operators

Overview of boolean terms

Database Search Limiters

★ Truncating Search Terms

Asterisk (*) Wildcard

★ EBSCO Databases & Google Drive

Researching in an EBSCO database?

EBSCO Databases & Google Drive

Defining Literature Review

Recommended Reading

About this page

Archer Librarians

Archer Library • Ashland University © Copyright 2023. An Equal Opportunity/Equal Access Institution.

Chapter 9. Reviewing the Literature

Examples of Literature Reviews

Reader-Friendly Example: The Power of Peers

Authoritative Academic Journal Example: Working Class Students’ College Expectations

Embracing Theory

Helpful Tips

Where to Start Looking for Literature

Keep a List of Your Keywords

Think Laterally

Read Outside the Canon

Consider Multiple Uses for Literature

Describing Gaps in the Literature

Use Concept Mapping

Ask Yourself, “How Is This Sociology (or Political Science or Public Policy, Etc.)?”

Don’t Treat This as a Chore

Supplement: Two More Literature Review Examples

”Changing Dispositions among the Upwardly Mobile” by Curl, Lareau, and Wu ( 2018 )

Further Readings

The Ultimate Guide to Qualitative Research - Part 1: The Basics

What is a literature review?

Literature reviews

Foundations of scholarly literature

What is the purpose of a literature review?

Identifying a research gap

Summarizing themes and patterns

Writing for your audience

ATLAS.ti helps you make sense of research.

A literature review section

A stand-alone literature review

Organize your data all in one place.

Thematic review

Quantitative analysis

From research question to findings, make it happen with ATLAS.ti.

Writing a Literature Review

Welcome to the Purdue OWL

Where, when, and why would I write a lit review?

What are the parts of a lit review?

How should I organize my lit review?

What are some strategies or tips I can use while writing my lit review?

Dissertations and research projects

Developing a theoretical framework

Why do I need a theoretical framework?

How do I choose the 'right' approach?

Introduction

Literature review

Methodology

A summary of your key findings – what you have concluded from your research, and how far you have been able to successfully answer your research questions.

Alternative structure

Organizing Your Social Sciences Research Paper

Importance of a Good Literature Review

Structure and Writing Style

Writing Tip

Another Writing Tip

Yet Another Writing Tip

Have a language expert improve your writing

What Is Qualitative Research? | Methods & Examples

Table of contents