review of related literature example qualitative research

Review of Related Literature: Format, Example, & How to Make RRL

A review of related literature is a separate paper or a part of an article that collects and synthesizes discussion on a topic. Its purpose is to show the current state of research on the issue and highlight gaps in existing knowledge. A literature review can be included in a research paper or scholarly article, typically following the introduction and before the research methods section.

This article will clarify the definition, significance, and structure of a review of related literature. You’ll also learn how to organize your literature review and discover ideas for an RRL in different subjects.

🔤 What Is RRL?

• ❗ Significance of Literature Review
• 🔎 How to Search for Literature
• 🧩 Literature Review Structure
• 📋 Format of RRL — APA, MLA, & Others
• ✍️ How to Write an RRL
• 📚 Examples of RRL

🔗 References

A review of related literature (RRL) is a part of the research report that examines significant studies, theories, and concepts published in scholarly sources on a particular topic. An RRL includes 3 main components:

• A short overview and critique of the previous research.
• Similarities and differences between past studies and the current one.
• An explanation of the theoretical frameworks underpinning the research.

❗ Significance of Review of Related Literature

Although the goal of a review of related literature differs depending on the discipline and its intended use, its significance cannot be overstated. Here are some examples of how a review might be beneficial:

• It helps determine knowledge gaps .
• It saves from duplicating research that has already been conducted.
• It provides an overview of various research areas within the discipline.
• It demonstrates the researcher’s familiarity with the topic.

🔎 How to Perform a Literature Search

Including a description of your search strategy in the literature review section can significantly increase your grade. You can search sources with the following steps:

🧩 Literature Review Structure Example

The majority of literature reviews follow a standard introduction-body-conclusion structure. Let’s look at the RRL structure in detail.

Introduction of Review of Related Literature: Sample

An introduction should clarify the study topic and the depth of the information to be delivered. It should also explain the types of sources used. If your lit. review is part of a larger research proposal or project, you can combine its introductory paragraph with the introduction of your paper.

Here is a sample introduction to an RRL about cyberbullying:

Bullying has troubled people since the beginning of time. However, with modern technological advancements, especially social media, bullying has evolved into cyberbullying. As a result, nowadays, teenagers and adults cannot flee their bullies, which makes them feel lonely and helpless. This literature review will examine recent studies on cyberbullying.

Sample Review of Related Literature Thesis

A thesis statement should include the central idea of your literature review and the primary supporting elements you discovered in the literature. Thesis statements are typically put at the end of the introductory paragraph.

Look at a sample thesis of a review of related literature:

This literature review shows that scholars have recently covered the issues of bullies’ motivation, the impact of bullying on victims and aggressors, common cyberbullying techniques, and victims’ coping strategies. However, there is still no agreement on the best practices to address cyberbullying.

Literature Review Body Paragraph Example

The main body of a literature review should provide an overview of the existing research on the issue. Body paragraphs should not just summarize each source but analyze them. You can organize your paragraphs with these 3 elements:

• Claim . Start with a topic sentence linked to your literature review purpose.
• Evidence . Cite relevant information from your chosen sources.
• Discussion . Explain how the cited data supports your claim.

Here’s a literature review body paragraph example:

Scholars have examined the link between the aggressor and the victim. Beran et al. (2007) state that students bullied online often become cyberbullies themselves. Faucher et al. (2014) confirm this with their findings: they discovered that male and female students began engaging in cyberbullying after being subject to bullying. Hence, one can conclude that being a victim of bullying increases one’s likelihood of becoming a cyberbully.

Review of Related Literature: Conclusion

A conclusion presents a general consensus on the topic. Depending on your literature review purpose, it might include the following:

• Introduction to further research . If you write a literature review as part of a larger research project, you can present your research question in your conclusion .
• Overview of theories . You can summarize critical theories and concepts to help your reader understand the topic better.
• Discussion of the gap . If you identified a research gap in the reviewed literature, your conclusion could explain why that gap is significant.

Check out a conclusion example that discusses a research gap:

There is extensive research into bullies’ motivation, the consequences of bullying for victims and aggressors, strategies for bullying, and coping with it. Yet, scholars still have not reached a consensus on what to consider the best practices to combat cyberbullying. This question is of great importance because of the significant adverse effects of cyberbullying on victims and bullies.

📋 Format of RRL — APA, MLA, & Others

In this section, we will discuss how to format an RRL according to the most common citation styles: APA, Chicago, MLA, and Harvard.

Writing a literature review using the APA7 style requires the following text formatting:

• When using APA in-text citations , include the author’s last name and the year of publication in parentheses.
• For direct quotations , you must also add the page number. If you use sources without page numbers, such as websites or e-books, include a paragraph number instead.
• When referring to the author’s name in a sentence , you do not need to repeat it at the end of the sentence. Instead, include the year of publication inside the parentheses after their name.
• The reference list should be included at the end of your literature review. It is always alphabetized by the last name of the author (from A to Z), and the lines are indented one-half inch from the left margin of your paper. Do not forget to invert authors’ names (the last name should come first) and include the full titles of journals instead of their abbreviations. If you use an online source, add its URL.

The RRL format in the Chicago style is as follows:

• Author-date . You place your citations in brackets within the text, indicating the name of the author and the year of publication.
• Notes and bibliography . You place your citations in numbered footnotes or endnotes to connect the citation back to the source in the bibliography.
• The reference list, or bibliography , in Chicago style, is at the end of a literature review. The sources are arranged alphabetically and single-spaced. Each bibliography entry begins with the author’s name and the source’s title, followed by publication information, such as the city of publication, the publisher, and the year of publication.

Writing a literature review using the MLA style requires the following text formatting:

• In the MLA format, you can cite a source in the text by indicating the author’s last name and the page number in parentheses at the end of the citation. If the cited information takes several pages, you need to include all the page numbers.
• The reference list in MLA style is titled “ Works Cited .” In this section, all sources used in the paper should be listed in alphabetical order. Each entry should contain the author, title of the source, title of the journal or a larger volume, other contributors, version, number, publisher, and publication date.

The Harvard style requires you to use the following text formatting for your RRL:

• In-text citations in the Harvard style include the author’s last name and the year of publication. If you are using a direct quote in your literature review, you need to add the page number as well.
• Arrange your list of references alphabetically. Each entry should contain the author’s last name, their initials, the year of publication, the title of the source, and other publication information, like the journal title and issue number or the publisher.

✍️ How to Write Review of Related Literature – Sample

Literature reviews can be organized in many ways depending on what you want to achieve with them. In this section, we will look at 3 examples of how you can write your RRL.

Thematic Literature Review

A thematic literature review is arranged around central themes or issues discussed in the sources. If you have identified some recurring themes in the literature, you can divide your RRL into sections that address various aspects of the topic. For example, if you examine studies on e-learning, you can distinguish such themes as the cost-effectiveness of online learning, the technologies used, and its effectiveness compared to traditional education.

Chronological Literature Review

A chronological literature review is a way to track the development of the topic over time. If you use this method, avoid merely listing and summarizing sources in chronological order. Instead, try to analyze the trends, turning moments, and critical debates that have shaped the field’s path. Also, you can give your interpretation of how and why specific advances occurred.

Methodological Literature Review

A methodological literature review differs from the preceding ones in that it usually doesn’t focus on the sources’ content. Instead, it is concerned with the research methods . So, if your references come from several disciplines or fields employing various research techniques, you can compare the findings and conclusions of different methodologies, for instance:

• empirical vs. theoretical studies;
• qualitative vs. quantitative research.

📚 Examples of Review of Related Literature and Studies

We have prepared a short example of RRL on climate change for you to see how everything works in practice!

Climate change is one of the most important issues nowadays. Based on a variety of facts, it is now clearer than ever that humans are altering the Earth's climate. The atmosphere and oceans have warmed, causing sea level rise, a significant loss of Arctic ice, and other climate-related changes. This literature review provides a thorough summary of research on climate change, focusing on climate change fingerprints and evidence of human influence on the Earth's climate system.

Physical Mechanisms and Evidence of Human Influence

Scientists are convinced that climate change is directly influenced by the emission of greenhouse gases. They have carefully analyzed various climate data and evidence, concluding that the majority of the observed global warming over the past 50 years cannot be explained by natural factors alone. Instead, there is compelling evidence pointing to a significant contribution of human activities, primarily the emission of greenhouse gases (Walker, 2014). For example, based on simple physics calculations, doubled carbon dioxide concentration in the atmosphere can lead to a global temperature increase of approximately 1 degree Celsius. (Elderfield, 2022). In order to determine the human influence on climate, scientists still have to analyze a lot of natural changes that affect temperature, precipitation, and other components of climate on timeframes ranging from days to decades and beyond.

Fingerprinting Climate Change

Fingerprinting climate change is a useful tool to identify the causes of global warming because different factors leave unique marks on climate records. This is evident when scientists look beyond overall temperature changes and examine how warming is distributed geographically and over time (Watson, 2022). By investigating these climate patterns, scientists can obtain a more complex understanding of the connections between natural climate variability and climate variability caused by human activity.

Modeling Climate Change and Feedback

To accurately predict the consequences of feedback mechanisms, the rate of warming, and regional climate change, scientists can employ sophisticated mathematical models of the atmosphere, ocean, land, and ice (the cryosphere). These models are grounded in well-established physical laws and incorporate the latest scientific understanding of climate-related processes (Shuckburgh, 2013). Although different climate models produce slightly varying projections for future warming, they all will agree that feedback mechanisms play a significant role in amplifying the initial warming caused by greenhouse gas emissions. (Meehl, 2019).

In conclusion, the literature on global warming indicates that there are well-understood physical processes that link variations in greenhouse gas concentrations to climate change. In addition, it covers the scientific proof that the rates of these gases in the atmosphere have increased and continue to rise fast. According to the sources, the majority of this recent change is almost definitely caused by greenhouse gas emissions produced by human activities. Citizens and governments can alter their energy production methods and consumption patterns to reduce greenhouse gas emissions and, thus, the magnitude of climate change. By acting now, society can prevent the worst consequences of climate change and build a more resilient and sustainable future for generations to come.

Have you ever struggled with finding the topic for an RRL in different subjects? Read the following paragraphs to get some ideas!

Nursing Literature Review Example

Many topics in the nursing field require research. For example, you can write a review of literature related to dengue fever . Give a general overview of dengue virus infections, including its clinical symptoms, diagnosis, prevention, and therapy.

Another good idea is to review related literature and studies about teenage pregnancy . This review can describe the effectiveness of specific programs for adolescent mothers and their children and summarize recommendations for preventing early pregnancy.

📝 Check out some more valuable examples below:

• Hospital Readmissions: Literature Review .
• Literature Review: Lower Sepsis Mortality Rates .
• Breast Cancer: Literature Review .
• Sexually Transmitted Diseases: Literature Review .
• PICO for Pressure Ulcers: Literature Review .
• COVID-19 Spread Prevention: Literature Review .
• Chronic Obstructive Pulmonary Disease: Literature Review .
• Hypertension Treatment Adherence: Literature Review .
• Neonatal Sepsis Prevention: Literature Review .
• Healthcare-Associated Infections: Literature Review .
• Understaffing in Nursing: Literature Review .

Psychology Literature Review Example

If you look for an RRL topic in psychology , you can write a review of related literature about stress . Summarize scientific evidence about stress stages, side effects, types, or reduction strategies. Or you can write a review of related literature about computer game addiction . In this case, you may concentrate on the neural mechanisms underlying the internet gaming disorder, compare it to other addictions, or evaluate treatment strategies.

A review of related literature about cyberbullying is another interesting option. You can highlight the impact of cyberbullying on undergraduate students’ academic, social, and emotional development.

📝 Look at the examples that we have prepared for you to come up with some more ideas:

• Mindfulness in Counseling: A Literature Review .
• Team-Building Across Cultures: Literature Review .
• Anxiety and Decision Making: Literature Review .
• Literature Review on Depression .
• Literature Review on Narcissism .
• Effects of Depression Among Adolescents .
• Causes and Effects of Anxiety in Children .

Literature Review — Sociology Example

Sociological research poses critical questions about social structures and phenomena. For example, you can write a review of related literature about child labor , exploring cultural beliefs and social norms that normalize the exploitation of children. Or you can create a review of related literature about social media . It can investigate the impact of social media on relationships between adolescents or the role of social networks on immigrants’ acculturation .

📝 You can find some more ideas below!

• Single Mothers’ Experiences of Relationships with Their Adolescent Sons .
• Teachers and Students’ Gender-Based Interactions .
• Gender Identity: Biological Perspective and Social Cognitive Theory .
• Gender: Culturally-Prescribed Role or Biological Sex .
• The Influence of Opioid Misuse on Academic Achievement of Veteran Students .
• The Importance of Ethics in Research .
• The Role of Family and Social Network Support in Mental Health .

Education Literature Review Example

For your education studies , you can write a review of related literature about academic performance to determine factors that affect student achievement and highlight research gaps. One more idea is to create a review of related literature on study habits , considering their role in the student’s life and academic outcomes.

You can also evaluate a computerized grading system in a review of related literature to single out its advantages and barriers to implementation. Or you can complete a review of related literature on instructional materials to identify their most common types and effects on student achievement.

📝 Find some inspiration in the examples below:

• Literature Review on Online Learning Challenges From COVID-19 .
• Education, Leadership, and Management: Literature Review .
• Literature Review: Standardized Testing Bias .
• Bullying of Disabled Children in School .
• Interventions and Letter & Sound Recognition: A Literature Review .
• Social-Emotional Skills Program for Preschoolers .
• Effectiveness of Educational Leadership Management Skills .

Business Research Literature Review

If you’re a business student, you can focus on customer satisfaction in your review of related literature. Discuss specific customer satisfaction features and how it is affected by service quality and prices. You can also create a theoretical literature review about consumer buying behavior to evaluate theories that have significantly contributed to understanding how consumers make purchasing decisions.

📝 Look at the examples to get more exciting ideas:

• Leadership and Communication: Literature Review .
• Human Resource Development: Literature Review .
• Project Management. Literature Review .
• Strategic HRM: A Literature Review .
• Customer Relationship Management: Literature Review .
• Literature Review on International Financial Reporting Standards .
• Cultures of Management: Literature Review .

To conclude, a review of related literature is a significant genre of scholarly works that can be applied in various disciplines and for multiple goals. The sources examined in an RRL provide theoretical frameworks for future studies and help create original research questions and hypotheses.

When you finish your outstanding literature review, don’t forget to check whether it sounds logical and coherent. Our text-to-speech tool can help you with that!

• Literature Reviews | University of North Carolina at Chapel Hill
• Writing a Literature Review | Purdue Online Writing Lab
• Learn How to Write a Review of Literature | University of Wisconsin-Madison
• The Literature Review: A Few Tips on Conducting It | University of Toronto
• Writing a Literature Review | UC San Diego
• Conduct a Literature Review | The University of Arizona
• Methods for Literature Reviews | National Library of Medicine
• Literature Reviews: 5. Write the Review | Georgia State University

How to Write an Animal Testing Essay: Tips for Argumentative & Persuasive Papers

Descriptive essay topics: examples, outline, & more.

Archer Library

Qualitative research: literature review .

• Archer Library This link opens in a new window
• Schedule a Reference Appointment This link opens in a new window
• Qualitative Research Handout This link opens in a new window
• Locating Books
• ebook Collections This link opens in a new window
• A to Z Database List This link opens in a new window
• Research & Stats
• Literature Review Resources
• Citation & Reference

Exploring the literature review 

Literature review model: 6 steps.

Adapted from The Literature Review , Machi & McEvoy (2009, p. 13).

Your Literature Review

Step 2: search, boolean search strategies, search limiters, ★ ebsco & google drive.

1. Select a Topic

"All research begins with curiosity" (Machi & McEvoy, 2009, p. 14)

Selection of a topic, and fully defined research interest and question, is supervised (and approved) by your professor. Tips for crafting your topic include:

• Be specific. Take time to define your interest.
• Topic Focus. Fully describe and sufficiently narrow the focus for research.
• Academic Discipline. Learn more about your area of research & refine the scope.
• Avoid Bias. Be aware of bias that you (as a researcher) may have.
• Document your research. Use Google Docs to track your research process.
• Research apps. Consider using Evernote or Zotero to track your research.

Consider Purpose

What will your topic and research address?

In The Literature Review: A Step-by-Step Guide for Students , Ridley presents that literature reviews serve several purposes (2008, p. 16-17).  Included are the following points:

• Historical background for the research;
• Overview of current field provided by "contemporary debates, issues, and questions;"
• Theories and concepts related to your research;
• Introduce "relevant terminology" - or academic language - being used it the field;
• Connect to existing research - does your work "extend or challenge [this] or address a gap;" 
• Provide "supporting evidence for a practical problem or issue" that your research addresses.

★ Schedule a research appointment

At this point in your literature review, take time to meet with a librarian. Why? Understanding the subject terminology used in databases can be challenging. Archer Librarians can help you structure a search, preparing you for step two. How? Contact a librarian directly or use the online form to schedule an appointment. Details are provided in the adjacent Schedule an Appointment box.

2. Search the Literature

Collect & Select Data: Preview, select, and organize

AU Library is your go-to resource for this step in your literature review process. The literature search will include books and ebooks, scholarly and practitioner journals, theses and dissertations, and indexes. You may also choose to include web sites, blogs, open access resources, and newspapers. This library guide provides access to resources needed to complete a literature review.

Books & eBooks: Archer Library & OhioLINK

Databases: scholarly & practitioner journals.

Review the Library Databases tab on this library guide, it provides links to recommended databases for Education & Psychology, Business, and General & Social Sciences.

Expand your journal search; a complete listing of available AU Library and OhioLINK databases is available on the Databases  A to Z list . Search the database by subject, type, name, or do use the search box for a general title search. The A to Z list also includes open access resources and select internet sites.

Databases: Theses & Dissertations

Review the Library Databases tab on this guide, it includes Theses & Dissertation resources. AU library also has AU student authored theses and dissertations available in print, search the library catalog for these titles.

Did you know? If you are looking for particular chapters within a dissertation that is not fully available online, it is possible to submit an ILL article request . Do this instead of requesting the entire dissertation.

Newspapers:  Databases & Internet

Consider current literature in your academic field. AU Library's database collection includes The Chronicle of Higher Education and The Wall Street Journal .  The Internet Resources tab in this guide provides links to newspapers and online journals such as Inside Higher Ed , COABE Journal , and Education Week .

Search Strategies & Boolean Operators

There are three basic boolean operators:  AND, OR, and NOT.

Used with your search terms, boolean operators will either expand or limit results. What purpose do they serve? They help to define the relationship between your search terms. For example, using the operator AND will combine the terms expanding the search. When searching some databases, and Google, the operator AND may be implied.

Overview of boolean terms

About the example: Boolean searches were conducted on November 4, 2019; result numbers may vary at a later date. No additional database limiters were set to further narrow search returns.

Database Search Limiters

Database strategies for targeted search results.

Most databases include limiters, or additional parameters, you may use to strategically focus search results.  EBSCO databases, such as Education Research Complete & Academic Search Complete provide options to:

• Limit results to full text;
• Limit results to scholarly journals, and reference available;
• Select results source type to journals, magazines, conference papers, reviews, and newspapers
• Publication date

Keep in mind that these tools are defined as limiters for a reason; adding them to a search will limit the number of results returned.  This can be a double-edged sword.  How? 

• If limiting results to full-text only, you may miss an important piece of research that could change the direction of your research. Interlibrary loan is available to students, free of charge. Request articles that are not available in full-text; they will be sent to you via email.
• If narrowing publication date, you may eliminate significant historical - or recent - research conducted on your topic.
• Limiting resource type to a specific type of material may cause bias in the research results.

Use limiters with care. When starting a search, consider opting out of limiters until the initial literature screening is complete. The second or third time through your research may be the ideal time to focus on specific time periods or material (scholarly vs newspaper).

★ Truncating Search Terms

Expanding your search term at the root.

Truncating is often referred to as 'wildcard' searching. Databases may have their own specific wildcard elements however, the most commonly used are the asterisk (*) or question mark (?).  When used within your search. they will expand returned results.

Asterisk (*) Wildcard

Using the asterisk wildcard will return varied spellings of the truncated word. In the following example, the search term education was truncated after the letter "t."

Explore these database help pages for additional information on crafting search terms.

• EBSCO Connect: Searching with Wildcards and Truncation Symbols
• EBSCO Connect: Searching with Boolean Operators
• EBSCO Connect: EBSCOhost Search Tips
• EBSCO Connect: Basic Searching with EBSCO
• ProQuest Help: Search Tips
• ERIC: How does ERIC search work?

★ EBSCO Databases & Google Drive

Tips for saving research directly to Google drive.

Researching in an EBSCO database?

It is possible to save articles (PDF and HTML) and abstracts in EBSCOhost databases directly to Google drive. Select the Google Drive icon, authenticate using a Google account, and an EBSCO folder will be created in your account. This is a great option for managing your research. If documenting your research in a Google Doc, consider linking the information to actual articles saved in drive.

EBSCO Databases & Google Drive

EBSCOHost Databases & Google Drive: Managing your Research

This video features an overview of how to use Google Drive with EBSCO databases to help manage your research. It presents information for connecting an active Google account to EBSCO and steps needed to provide permission for EBSCO to manage a folder in Drive.

About the Video:  Closed captioning is available, select CC from the video menu.  If you need to review a specific area on the video, view on YouTube and expand the video description for access to topic time stamps.  A video transcript is provided below.

• EBSCOhost Databases & Google Scholar

Defining Literature Review

What is a literature review.

A definition from the Online Dictionary for Library and Information Sciences .

A literature review is "a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works" (Reitz, 2014). 

A systemic review is "a literature review focused on a specific research question, which uses explicit methods to minimize bias in the identification, appraisal, selection, and synthesis of all the high-quality evidence pertinent to the question" (Reitz, 2014).

Recommended Reading

About this page

EBSCO Connect [Discovery and Search]. (2022). Searching with boolean operators. Retrieved May, 3, 2022 from https://connect.ebsco.com/s/?language=en_US

EBSCO Connect [Discover and Search]. (2022). Searching with wildcards and truncation symbols. Retrieved May 3, 2022; https://connect.ebsco.com/s/?language=en_US

Machi, L.A. & McEvoy, B.T. (2009). The literature review . Thousand Oaks, CA: Corwin Press: 

Reitz, J.M. (2014). Online dictionary for library and information science. ABC-CLIO, Libraries Unlimited . Retrieved from https://www.abc-clio.com/ODLIS/odlis_A.aspx

Ridley, D. (2008). The literature review: A step-by-step guide for students . Thousand Oaks, CA: Sage Publications, Inc.

Archer Librarians

Schedule an appointment.

Contact a librarian directly (email), or submit a request form. If you have worked with someone before, you can request them on the form.

• ★ Archer Library Help • Online Reqest Form
• Carrie Halquist • Reference & Instruction
• Jessica Byers • Reference & Curation
• Don Reams • Corrections Education & Reference
• Diane Schrecker • Education & Head of the IRC
• Tanaya Silcox • Technical Services & Business
• Sarah Thomas • Acquisitions & ATS Librarian
• << Previous: Research & Stats
• Next: Literature Review Resources >>
• Last Updated: Apr 23, 2024 3:47 PM
• URL: https://libguides.ashland.edu/qualitative

Archer Library • Ashland University © Copyright 2023. An Equal Opportunity/Equal Access Institution.

Chapter 9. Reviewing the Literature

What is a “literature review”.

No researcher ever comes up with a research question that is wholly novel. Someone, somewhere, has asked the same thing. Academic research is part of a larger community of researchers, and it is your responsibility, as a member of this community, to acknowledge others who have asked similar questions and to put your particular research into this greater context. It is not simply a convention or custom to begin your study with a review of previous literature (the “ lit review ”) but an important responsibility you owe the scholarly community.

Too often, new researchers pursue a topic to study and then write something like, “No one has ever studied this before” or “This area is underresearched.” It may be that no one has studied this particular group or setting, but it is highly unlikely no one has studied the foundational phenomenon of interest. And that comment about an area being underresearched? Be careful. The statement may simply signal to others that you haven’t done your homework. Rubin ( 2021 ) refers to this as “free soloing,” and it is not appreciated in academic work:

The truth of the matter is, academics don’t really like when people free solo. It’s really bad form to omit talking about the other people who are doing or have done research in your area. Partly, I mean we need to cite their work, but I also mean we need to respond to it—agree or disagree, clarify for extend. It’s also really bad form to talk about your research in a way that does not make it understandable to other academics.…You have to explain to your readers what your story is really about in terms they care about . This means using certain terminology, referencing debates in the literature, and citing relevant works—that is, in connecting your work to something else. ( 51–52 )

A literature review is a comprehensive summary of previous research on a topic. It includes both articles and books—and in some cases reports—relevant to a particular area of research. Ideally, one’s research question follows from the reading of what has already been produced. For example, you are interested in studying sports injuries related to female gymnasts. You read everything you can find on sports injuries related to female gymnasts, and you begin to get a sense of what questions remain open. You find that there is a lot of research on how coaches manage sports injuries and much about cultures of silence around treating injuries, but you don’t know what the gymnasts themselves are thinking about these issues. You look specifically for studies about this and find several, which then pushes you to narrow the question further. Your literature review then provides the road map of how you came to your very specific question, and it puts your study in the context of studies of sports injuries. What you eventually find can “speak to” all the related questions as well as your particular one.

In practice, the process is often a bit messier. Many researchers, and not simply those starting out, begin with a particular question and have a clear idea of who they want to study and where they want to conduct their study but don’t really know much about other studies at all. Although backward, we need to recognize this is pretty common. Telling students to “find literature” after the fact can seem like a purposeless task or just another hurdle for completing a thesis or dissertation. It is not! Even if you were not motivated by the literature in the first place, acknowledging similar studies and connecting your own research to those studies are important parts of building knowledge. Acknowledgment of past research is a responsibility you owe the discipline to which you belong.

Literature reviews can also signal theoretical approaches and particular concepts that you will incorporate into your own study. For example, let us say you are doing a study of how people find their first jobs after college, and you want to use the concept of social capital . There are competing definitions of social capital out there (e.g., Bourdieu vs. Burt vs. Putnam). Bourdieu’s notion is of one form of capital, or durable asset, of a “network of more or less institutionalized relationships of mutual acquaintance or recognition” ( 1984:248 ). Burt emphasizes the “brokerage opportunities” in a social network as social capital ( 1997:355 ). Putnam’s social capital is all about “facilitating coordination and cooperation for mutual benefit” ( 2001:67 ). Your literature review can adjudicate among these three approaches, or it can simply refer to the one that is animating your own research. If you include Bourdieu in your literature review, readers will know “what kind” of social capital you are talking about as well as what kind of social scientist you yourself are. They will likely understand that you are interested more in how some people are advantaged by their social capital relative to others rather than being interested in the mechanics of how social networks operate.

The literature review thus does two important things for you: firstly, it allows you to acknowledge previous research in your area of interest, thereby situating you within a discipline or body of scholars, and, secondly, it demonstrates that you know what you are talking about. If you present the findings of your research study without including a literature review, it can be like singing into the wind. It sounds nice, but no one really hears it, or if they do catch snippets, they don’t know where it is coming from.

Examples of Literature Reviews

To help you get a grasp of what a good literature review looks like and how it can advance your study, let’s take a look at a few examples.

Reader-Friendly Example: The Power of Peers

The first is by Janice McCabe ( 2016 ) and is from an article on peer networks in the journal Contexts . Contexts presents articles in a relatively reader-friendly format, with the goal of reaching a large audience for interesting sociological research. Read this example carefully and note how easily McCabe is able to convey the relevance of her own work by situating it in the context of previous studies:

Scholars who study education have long acknowledged the importance of peers for students’ well-being and academic achievement. For example, in 1961, James Coleman argued that peer culture within high schools shapes students’ social and academic aspirations and successes. More recently, Judith Rich Harris has drawn on research in a range of areas—from sociological studies of preschool children to primatologists’ studies of chimpanzees and criminologists’ studies of neighborhoods—to argue that peers matter much more than parents in how children “turn out.” Researchers have explored students’ social lives in rich detail, as in Murray Milner’s book about high school students, Freaks, Geeks, and Cool Kids , and Elizabeth Armstrong and Laura Hamilton’s look at college students, Paying for the Party . These works consistently show that peers play a very important role in most students’ lives. They tend, however, to prioritize social over academic influence and to use a fuzzy conception of peers rather than focusing directly on friends—the relationships that should matter most for student success. Social scientists have also studied the power of peers through network analysis, which is based on uncovering the web of connections between people. Network analysis involves visually mapping networks and mathematically comparing their structures (such as the density of ties) and the positions of individuals within them (such as how central a given person is within the network). As Nicholas Christakis and James Fowler point out in their book Connected , network structure influences a range of outcomes, including health, happiness, wealth, weight, and emotions. Given that sociologists have long considered network explanations for social phenomena, it’s surprising that we know little about how college students’ friends impact their experiences. In line with this network tradition, I focus on the structure of friendship networks, constructing network maps so that the differences we see across participants are due to the underlying structure, including each participant’s centrality in their friendship group and the density of ties among their friends. ( 23 )

What did you notice? In her very second sentence, McCabe uses “for example” to introduce a study by Coleman, thereby indicating that she is not going to tell you every single study in this area but is going to tell you that (1) there is a lot of research in this area, (2) it has been going on since at least 1961, and (3) it is still relevant (i.e., recent studies are still being done now). She ends her first paragraph by summarizing the body of literature in this area (after giving you a few examples) and then telling you what may have been (so far) left out of this research. In the second paragraph, she shifts to a separate interesting focus that is related to the first but is also quite distinct. Lit reviews very often include two (or three) distinct strands of literature, the combination of which nicely backgrounds this particular study . In the case of our female gymnast study (above), those two strands might be (1) cultures of silence around sports injuries and (2) the importance of coaches. McCabe concludes her short and sweet literature review with one sentence explaining how she is drawing from both strands of the literature she has succinctly presented for her particular study. This example should show you that literature reviews can be readable, helpful, and powerful additions to your final presentation.

Authoritative Academic Journal Example: Working Class Students’ College Expectations

The second example is more typical of academic journal writing. It is an article published in the British Journal of Sociology of Education by Wolfgang Lehmann ( 2009 ):

Although this increase in post-secondary enrolment and the push for university is evident across gender, race, ethnicity, and social class categories, access to university in Canada continues to be significantly constrained for those from lower socio-economic backgrounds (Finnie, Lascelles, and Sweetman 2005). Rising tuition fees coupled with an overestimation of the cost and an underestimation of the benefits of higher education has put university out of reach for many young people from low-income families (Usher 2005). Financial constraints aside, empirical studies in Canada have shown that the most important predictor of university access is parental educational attainment. Having at least one parent with a university degree significantly increases the likelihood of a young person to attend academic-track courses in high school, have high educational and career aspirations, and ultimately attend university (Andres et al. 1999, 2000; Lehmann 2007a). Drawing on Bourdieu’s various writing on habitus and class-based dispositions (see, for example, Bourdieu 1977, 1990), Hodkinson and Sparkes (1997) explain career decisions as neither determined nor completely rational. Instead, they are based on personal experiences (e.g., through employment or other exposure to occupations) and advice from others. Furthermore, they argue that we have to understand these decisions as pragmatic, rather than rational. They are pragmatic in that they are based on incomplete and filtered information, because of the social context in which the information is obtained and processed. New experiences and information can, however, also be allowed into one’s world, where they gradually or radically transform habitus, which in turn creates the possibility for the formation of new and different dispositions. Encountering a supportive teacher in elementary or secondary school, having ambitious friends, or chance encounters can spark such transformations. Transformations can be confirming or contradictory, they can be evolutionary or dislocating. Working-class students who enter university most certainly encounter such potentially transformative situations. Granfield (1991) has shown how initially dislocating feelings of inadequacy and inferiority of working-class students at an elite US law school were eventually replaced by an evolutionary transformation, in which the students came to dress, speak and act more like their middle-class and upper-class peers. In contrast, Lehmann (2007b) showed how persistent habitus dislocation led working-class university students to drop out of university. Foskett and Hemsley-Brown (1999) argue that young people’s perceptions of careers are a complex mix of their own experiences, images conveyed through adults, and derived images conveyed by the media. Media images of careers, perhaps, are even more important for working-class youth with high ambitions as they offer (generally distorted) windows into a world of professional employment to which they have few other sources of access. It has also been argued that working-class youth who do continue to university still face unique, class-specific challenges, evident in higher levels of uncertainty (Baxter and Britton 2001; Lehmann 2004, 2007a; Quinn 2004), their higher education choices (Ball et al. 2002; Brooks 2003; Reay et al. 2001) and fears of inadequacy because of their cultural outsider status (Aries and Seider 2005; Granfield 1991). Although the number of working-class university students in Canada has slowly increased, that of middle-class students at university has risen far more steeply (Knighton and Mizra 2002). These different enrolment trajectories have actually widened the participation gap, which in tum explains our continued concerns with the potential outsider status Indeed, in a study comparing first-generation working-class and traditional students who left university without graduating, Lehmann (2007b) found that first-generation working-class students were more likely to leave university very early in some cases within the first two months of enrollment. They were also more likely to leave university despite solid academic performance. Not “fitting in,” not “feeling university,” and not being able to “relate to these people” were key reasons for eventually withdrawing from university. From the preceding review of the literature, a number of key research questions arise: How do working-class university students frame their decision to attend university? How do they defy the considerable odds documented in the literature to attend university? What are the sources of information and various images that create dispositions to study at university? What role does their social-class background- or habitus play in their transition dispositions and how does this translate into expectations for university? ( 139 )

What did you notice here? How is this different from (and similar to) the first example? Note that rather than provide you with one or two illustrative examples of similar types of research, Lehmann provides abundant source citations throughout. He includes theory and concepts too. Like McCabe, Lehmann is weaving through multiple literature strands: the class gap in higher education participation in Canada, class-based dispositions, and obstacles facing working-class college students. Note how he concludes the literature review by placing his research questions in context.

Find other articles of interest and read their literature reviews carefully. I’ve included two more for you at the end of this chapter . As you learned how to diagram a sentence in elementary school (hopefully!), try diagramming the literature reviews. What are the “different strands” of research being discussed? How does the author connect these strands to their own research questions? Where is theory in the lit review, and how is it incorporated (e.g., Is it a separate strand of its own or is it inextricably linked with previous research in this area)?

One model of how to structure your literature review can be found in table 9.1. More tips, hints, and practices will be discussed later in the chapter.

Table 9.1. Model of Literature Review, Adopted from Calarco (2020:166)

Embracing Theory

A good research study will, in some form or another, use theory. Depending on your particular study (and possibly the preferences of the members of your committee), theory may be built into your literature review. Or it may form its own section in your research proposal/design (e.g., “literature review” followed by “theoretical framework”). In my own experience, I see a lot of graduate students grappling with the requirement to “include theory” in their research proposals. Things get a little squiggly here because there are different ways of incorporating theory into a study (Are you testing a theory? Are you generating a theory?), and based on these differences, your literature review proper may include works that describe, explain, and otherwise set forth theories, concepts, or frameworks you are interested in, or it may not do this at all. Sometimes a literature review sets forth what we know about a particular group or culture totally independent of what kinds of theoretical framework or particular concepts you want to explore. Indeed, the big point of your study might be to bring together a body of work with a theory that has never been applied to it previously. All this is to say that there is no one correct way to approach the use of theory and the writing about theory in your research proposal.

Students are often scared of embracing theory because they do not exactly understand what it is. Sometimes, it seems like an arbitrary requirement. You’re interested in a topic; maybe you’ve even done some research in the area and you have findings you want to report. And then a committee member reads over what you have and asks, “So what?” This question is a good clue that you are missing theory, the part that connects what you have done to what other researchers have done and are doing. You might stumble upon this rather accidentally and not know you are embracing theory, as in a case where you seek to replicate a prior study under new circumstances and end up finding that a particular correlation between behaviors only happens when mediated by something else. There’s theory in there, if you can pull it out and articulate it. Or it might be that you are motivated to do more research on racial microaggressions because you want to document their frequency in a particular setting, taking for granted the kind of critical race theoretical framework that has done the hard work of defining and conceptualizing “microaggressions” in the first place. In that case, your literature review could be a review of Critical Race Theory, specifically related to this one important concept. That’s the way to bring your study into a broader conversation while also acknowledging (and honoring) the hard work that has preceded you.

Rubin ( 2021 ) classifies ways of incorporating theory into case study research into four categories, each of which might be discussed somewhat differently in a literature review or theoretical framework section. The first, the least theoretical, is where you set out to study a “configurative idiographic case” ( 70 ) This is where you set out to describe a particular case, leaving yourself pretty much open to whatever you find. You are not expecting anything based on previous literature. This is actually pretty weak as far as research design goes, but it is probably the default for novice researchers. Your committee members should probably help you situate this in previous literature in some way or another. If they cannot, and it really does appear you are looking at something fairly new that no one else has bothered to research before, and you really are completely open to discovery, you might try using a Grounded Theory approach, which is a methodological approach that foregrounds the generation of theory. In that case, your “theory” section can be a discussion of “Grounded Theory” methodology (confusing, yes, but if you take some time to ponder, you will see how this works). You will still need a literature review, though. Ideally one that describes other studies that have ever looked at anything remotely like what you are looking at—parallel cases that have been researched.

The second approach is the “disciplined configurative case,” in which theory is applied to explain a particular case or topic. You are not trying to test the theory but rather assuming the theory is correct, as in the case of exploring microaggressions in a particular setting. In this case, you really do need to have a separate theory section in addition to the literature review, one in which you clearly define the theoretical framework, including any of its important concepts. You can use this section to discuss how other researchers have used the concepts and note any discrepancies in definitions or operationalization of those concepts. This way you will be sure to design your study so that it speaks to and with other researchers. If everyone who is writing about microaggressions has a different definition of them, it is hard for others to compare findings or make any judgments about their prevalence (or any number of other important characteristics). Your literature review section may then stand alone and describe previous research in the particular area or setting, irrespective of the kinds of theory underlying those studies.

The third approach is “heuristic,” one in which you seek to identify new variables, hypotheses, mechanisms, or paths not yet explained by a theory or theoretical framework. In a way, you are generating new theory, but it is probably more accurate to say that you are extending or deepening preexisting theory. In this case, having a single literature review that is focused on the theory and the ways the theory has been applied and understood (with all its various mechanisms and pathways) is probably your best option. The focus of the literature reviewed is less on the case and more on the theory you are seeking to extend.

The final approach is “theory testing,” which is much rarer in qualitative studies than in quantitative, where this is the default approach. Theory-testing cases are those where a particular case is used to see if an existing theory is accurate or accurate under particular circumstances. As with the heuristic approach, your literature review will probably draw heavily on previous uses of the theory, but you may end up having a special section specifically about cases very close to your own . In other words, the more your study approaches theory testing, the more likely there is to be a set of similar studies to draw on or even one important key study that you are setting your own study up in parallel to in order to find out if the theory generated there operates here.

If we wanted to get very technical, it might be useful to distinguish theoretical frameworks properly from conceptual frameworks. The latter are a bit looser and, given the nature of qualitative research, often fit exploratory studies. Theoretical frameworks rely on specific theories and are essential for theory-testing studies. Conceptual frameworks can pull in specific concepts or ideas that may or may not be linked to particular theories. Think about it this way: A theory is a story of how the world works. Concepts don’t presume to explain the whole world but instead are ways to approach phenomena to help make sense of them. Microaggressions are concepts that are linked to Critical Race Theory. One could contextualize one’s study within Critical Race Theory and then draw various concepts, such as that of microaggressions from the overall theoretical framework. Or one could bracket out the master theory or framework and employ the concept of microaggression more opportunistically as a phenomenon of interest. If you are unsure of what theory you are using, you might want to frame a more practical conceptual framework in your review of the literature.

Helpful Tips

How to maintain good notes for what your read.

Over the years, I have developed various ways of organizing notes on what I read. At first, I used a single sheet of full-size paper with a preprinted list of questions and points clearly addressed on the front side, leaving the second side for more reflective comments and free-form musings about what I read, why it mattered, and how it might be useful for my research. Later, I developed a system in which I use a single 4″ × 6″ note card for each book I read. I try only to use the front side (and write very small), leaving the back for comments that are about not just this reading but things to do or examine or consider based on the reading. These notes often mean nothing to anyone else picking up the card, but they make sense to me. I encourage you to find an organizing system that works for you. Then when you set out to compose a literature review, instead of staring at five to ten books or a dozen articles, you will have ten neatly printed pages or notecards or files that have distilled what is important to know about your reading.

It is also a good idea to store this data digitally, perhaps through a reference manager. I use RefWorks, but I also recommend EndNote or any other system that allows you to search institutional databases. Your campus library will probably provide access to one of these or another system. Most systems will allow you to export references from another manager if and when you decide to move to another system. Reference managers allow you to sort through all your literature by descriptor, author, year, and so on. Even so, I personally like to have the ability to manually sort through my index cards, recategorizing things I have read as I go. I use RefWorks to keep a record of what I have read, with proper citations, so I can create bibliographies more easily, and I do add in a few “notes” there, but the bulk of my notes are kept in longhand.

What kinds of information should you include from your reading? Here are some bulleted suggestions from Calarco ( 2020:113–114 ), with my own emendations:

• Citation . If you are using a reference manager, you can import the citation and then, when you are ready to create a bibliography, you can use a provided menu of citation styles, which saves a lot of time. If you’ve originally formatted in Chicago Style but the journal you are writing for wants APA style, you can change your entire bibliography in less than a minute. When using a notecard for a book, I include author, title, date as well as the library call number (since most of what I read I pull from the library). This is something RefWorks is not able to do, and it helps when I categorize.

I begin each notecard with an “intro” section, where I record the aims, goals, and general point of the book/article as explained in the introductory sections (which might be the preface, the acknowledgments, or the first two chapters). I then draw a bold line underneath this part of the notecard. Everything after that should be chapter specific. Included in this intro section are things such as the following, recommended by Calarco ( 2020 ):

• Key background . “Two to three short bullet points identifying the theory/prior research on which the authors are building and defining key terms.”
• Data/methods . “One or two short bullet points with information about the source of the data and the method of analysis, with a note if this is a novel or particularly effective example of that method.” I use [M] to signal methodology on my notecard, which might read, “[M] Int[erview]s (n-35), B[lack]/W[hite] voters” (I need shorthand to fit on my notecard!).
• Research question . “Stated as briefly as possible.” I always provide page numbers so I can go back and see exactly how this was stated (sometimes, in qualitative research, there are multiple research questions, and they cannot be stated simply).
• Argument/contributions . “Two to three short bullet points briefly describing the authors’ answer to the central research question and its implication for research, theory, and practice.” I use [ARG] for argument to signify the argument, and I make sure this is prominently visible on my notecard. I also provide page numbers here.

For me, all of this fits in the “intro” section, which, if this is a theoretically rich, methodologically sound book, might take up a third or even half of the front page of my notecard. Beneath the bold underline, I report specific findings or particulars of the book as they emerge chapter by chapter. Calarco’s ( 2020 ) next step is the following:

• Key findings . “Three to four short bullet points identifying key patterns in the data that support the authors’ argument.”

All that remains is writing down thoughts that occur upon finishing the article/book. I use the back of the notecard for these kinds of notes. Often, they reach out to other things I have read (e.g., “Robinson reminds me of Crusoe here in that both are looking at the effects of social isolation, but I think Robinson makes a stronger argument”). Calarco ( 2020 ) concludes similarly with the following:

• Unanswered questions . “Two to three short bullet points that identify key limitations of the research and/or questions the research did not answer that could be answered in future research.”

As I mentioned, when I first began taking notes like this, I preprinted pages with prompts for “research question,” “argument,” and so on. This was a great way to remind myself to look for these things in particular. You can do the same, adding whatever preprinted sections make sense to you, given what you are studying and the important aspects of your discipline. The other nice thing about the preprinted forms is that it keeps your writing to a minimum—you cannot write more than the allotted space, even if you might want to, preventing your notes from spiraling out of control. This can be helpful when we are new to a subject and everything seems worth recording!

After years of discipline, I have finally settled on my notecard approach. I have thousands of notecards, organized in several index card filing boxes stacked in my office. On the top right of each card is a note of the month/day I finished reading the item. I can remind myself what I read in the summer of 2010 if the need or desire ever arose to do so…those invaluable notecards are like a memento of what my brain has been up to!

Where to Start Looking for Literature

Your university library should provide access to one of several searchable databases for academic books and articles. My own preference is JSTOR, a service of ITHAKA, a not-for-profit organization that works to advance and preserve knowledge and to improve teaching and learning through the use of digital technologies. JSTOR allows you to search by several keywords and to narrow your search by type of material (articles or books). For many disciplines, the “literature” of the literature review is expected to be peer-reviewed “articles,” but some disciplines will also value books and book chapters. JSTOR is particularly useful for article searching. You can submit several keywords and see what is returned, and you can also narrow your search by a particular journal or discipline. If your discipline has one or two key journals (e.g., the American Journal of Sociology and the American Sociological Review are key for sociology), you might want to go directly to those journals’ websites and search for your topic area. There is an art to when to cast your net widely and when to refine your search, and you may have to tack back and forth to ensure that you are getting all that is relevant but not getting bogged down in all studies that might have some marginal relevance.

Some articles will carry more weight than others, and you can use applications like Google Scholar to see which articles have made and are continuing to make larger impacts on your discipline. Find these articles and read them carefully; use their literature review and the sources cited in those articles to make sure you are capturing what is relevant. This is actually a really good way of finding relevant books—only the most impactful will make it into the citations of journals. Over time, you will notice that a handful of articles (or books) are cited so often that when you see, say, Armstrong and Hamilton ( 2015 ), you know exactly what book this is without looking at the full cite. This is when you know you are in the conversation.

You might also approach a professor whose work is broadly in the area of your interest and ask them to recommend one or two “important” foundational articles or books. You can then use the references cited in those recommendations to build up your literature. Just be careful: some older professors’ knowledge of the literature (and I reluctantly add myself here) may be a bit outdated! It is best that the article or book whose references and sources you use to build your body of literature be relatively current.

Keep a List of Your Keywords

When using searchable databases, it is a good idea to keep a list of all the keywords you use as you go along so that (1) you do not needlessly duplicate your efforts and (2) you can more easily adjust your search as you get a better sense of what you are looking for. I suggest you keep a separate file or even a small notebook for this and you date your search efforts.

Here’s an example:

Table 9.2. Keep a List of Your Keywords

Think Laterally

How to find the various strands of literature to combine? Don’t get stuck on finding the exact same research topic you think you are interested in. In the female gymnast example, I recommended that my student consider looking for studies of ballerinas, who also suffer sports injuries and around whom there is a similar culture of silence. It turned out that there was in fact research about my student’s particular questions, just not about the subjects she was interested in. You might do something similar. Don’t get stuck looking for too direct literature but think about the broader phenomenon of interest or analogous cases.

Read Outside the Canon

Some scholars’ work gets cited by everyone all the time. To some extent, this is a very good thing, as it helps establish the discipline. For example, there are a lot of “Bourdieu scholars” out there (myself included) who draw ideas, concepts, and quoted passages from Bourdieu. This makes us recognizable to one another and is a way of sharing a common language (e.g., where “cultural capital” has a particular meaning to those versed in Bourdieusian theory). There are empirical studies that get cited over and over again because they are excellent studies but also because there is an “echo chamber effect” going on, where knowing to cite this study marks you as part of the club, in the know, and so on. But here’s the problem with this: there are hundreds if not thousands of excellent studies out there that fail to get appreciated because they are crowded out by the canon. Sometimes this happens because they are published in “lower-ranked” journals and are never read by a lot of scholars who don’t have time to read anything other than the “big three” in their field. Other times this happens because the author falls outside of the dominant social networks in the field and thus is unmentored and fails to get noticed by those who publish a lot in those highly ranked and visible spaces. Scholars who fall outside the dominant social networks and who publish outside of the top-ranked journals are in no way less insightful than their peers, and their studies may be just as rigorous and relevant to your work, so it is important for you to take some time to read outside the canon. Due to how a person’s race, gender, and class operate in the academy, there is also a matter of social justice and ethical responsibility involved here: “When you focus on the most-cited research, you’re more likely to miss relevant research by women and especially women of color, whose research tends to be under-cited in most fields. You’re also more likely to miss new research, research by junior scholars, and research in other disciplines that could inform your work. Essentially, it is important to read and cite responsibly, which means checking that you’re not just reading and citing the same white men and the same old studies that everyone has cited before you” ( Calarco 2020:112 ).

Consider Multiple Uses for Literature

Throughout this chapter, I’ve referred to the literature of interest in a rather abstract way, as what is relevant to your study. But there are many different ways previous research can be relevant to your study. The most basic use of the literature is the “findings”—for example, “So-and-so found that Canadian working-class students were concerned about ‘fitting in’ to the culture of college, and I am going to look at a similar question here in the US.” But the literature may be of interest not for its findings but theoretically—for example, employing concepts that you want to employ in your own study. Bourdieu’s definition of social capital may have emerged in a study of French professors, but it can still be relevant in a study of, say, how parents make choices about what preschools to send their kids to (also a good example of lateral thinking!).

If you are engaged in some novel methodological form of data collection or analysis, you might look for previous literature that has attempted that. I would not recommend this for undergraduate research projects, but for graduate students who are considering “breaking the mold,” find out if anyone has been there before you. Even if their study has absolutely nothing else in common with yours, it is important to acknowledge that previous work.

Describing Gaps in the Literature

First, be careful! Although it is common to explain how your research adds to, builds upon, and fills in gaps in the previous research (see all four literature review examples in this chapter for this), there is a fine line between describing the gaps and misrepresenting previous literature by failing to conduct a thorough review of the literature. A little humility can make a big difference in your presentation. Instead of “This is the first study that has looked at how firefighters juggle childcare during forest fire season,” say, “I use the previous literature on how working parents juggling childcare and the previous ethnographic studies of firefighters to explore how firefighters juggle childcare during forest fire season.” You can even add, “To my knowledge, no one has conducted an ethnographic study in this specific area, although what we have learned from X about childcare and from Y about firefighters would lead us to expect Z here.” Read more literature review sections to see how others have described the “gaps” they are filling.

Use Concept Mapping

Concept mapping is a helpful tool for getting your thoughts in order and is particularly helpful when thinking about the “literature” foundational to your particular study. Concept maps are also known as mind maps, which is a delightful way to think about them. Your brain is probably abuzz with competing ideas in the early stages of your research design. Write/draw them on paper, and then try to categorize and move the pieces around into “clusters” that make sense to you. Going back to the gymnasts example, my student might have begun by jotting down random words of interest: gymnasts * sports * coaches * female gymnasts * stress * injury * don’t complain * women in sports * bad coaching * anxiety/stress * careers in sports * pain. She could then have begun clustering these into relational categories (bad coaching, don’t complain culture) and simple “event” categories (injury, stress). This might have led her to think about reviewing literature in these two separate aspects and then literature that put them together. There is no correct way to draw a concept map, as they are wonderfully specific to your mind. There are many examples you can find online.

Ask Yourself, “How Is This Sociology (or Political Science or Public Policy, Etc.)?”

Rubin ( 2021:82 ) offers this suggestion instead of asking yourself the “So what?” question to get you thinking about what bridges there are between your study and the body of research in your particular discipline. This is particularly helpful for thinking about theory. Rubin further suggests that if you are really stumped, ask yourself, “What is the really big question that all [fill in your discipline here] care about?” For sociology, it might be “inequality,” which would then help you think about theories of inequality that might be helpful in framing your study on whatever it is you are studying—OnlyFans? Childcare during COVID? Aging in America? I can think of some interesting ways to frame questions about inequality for any of those topics. You can further narrow it by focusing on particular aspects of inequality (Gender oppression? Racial exclusion? Heteronormativity?). If your discipline is public policy, the big questions there might be, How does policy get enacted, and what makes a policy effective? You can then take whatever your particular policy interest is—tax reform, student debt relief, cap-and-trade regulations—and apply those big questions. Doing so would give you a handle on what is otherwise an intolerably vague subject (e.g., What about student debt relief?).

Sometimes finding you are in new territory means you’ve hit the jackpot, and sometimes it means you’ve traveled out of bounds for your discipline. The jackpot scenario is wonderful. You are doing truly innovative research that is combining multiple literatures or is addressing a new or under-examined phenomenon of interest, and your research has the potential to be groundbreaking. Congrats! But that’s really hard to do, and it might be more likely that you’ve traveled out of bounds, by which I mean, you are no longer in your discipline . It might be that no one has written about this thing—at least within your field— because no one in your field actually cares about this topic . ( Rubin 2021:83 ; emphases added)

Don’t Treat This as a Chore

Don’t treat the literature review as a chore that has to be completed, but see it for what it really is—you are building connections to other researchers out there. You want to represent your discipline or area of study fairly and adequately. Demonstrate humility and your knowledge of previous research. Be part of the conversation.

Supplement: Two More Literature Review Examples

Elites by harvey ( 2011 ).

In the last two decades, there has been a small but growing literature on elites. In part, this has been a result of the resurgence of ethnographic research such as interviews, focus groups, case studies, and participant observation but also because scholars have become increasingly interested in understanding the perspectives and behaviors of leaders in business, politics, and society as a whole. Yet until recently, our understanding of some of the methodological challenges of researching elites has lagged behind our rush to interview them.

There is no clear-cut definition of the term elite, and given its broad understanding across the social sciences, scholars have tended to adopt different approaches. Zuckerman (1972) uses the term ultraelites to describe individuals who hold a significant amount of power within a group that is already considered elite. She argues, for example, that US senators constitute part of the country’s political elite but that among them are the ultraelites: a “subset of particularly powerful or prestigious influentials” (160). She suggests that there is a hierarchy of status within elite groups. McDowell (1998) analyses a broader group of “professional elites” who are employees working at different levels for merchant and investment banks in London. She classifies this group as elite because they are “highly skilled, professionally competent, and class-specific” (2135). Parry (1998:2148) uses the term hybrid elites in the context of the international trade of genetic material because she argues that critical knowledge exists not in traditional institutions “but rather as increasingly informal, hybridised, spatially fragmented, and hence largely ‘invisible,’ networks of elite actors.” Given the undertheorization of the term elite, Smith (2006) recognizes why scholars have shaped their definitions to match their respondents . However, she is rightly critical of the underlying assumption that those who hold professional positions necessarily exert as much influence as initially perceived. Indeed, job titles can entirely misrepresent the role of workers and therefore are by no means an indicator of elite status (Harvey 2010).

Many scholars have used the term elite in a relational sense, defining them either in terms of their social position compared to the researcher or compared to the average person in society (Stephens 2007). The problem with this definition is there is no guarantee that an elite subject will necessarily translate this power and authority in an interview setting. Indeed, Smith (2006) found that on the few occasions she experienced respondents wanting to exert their authority over her, it was not from elites but from relatively less senior workers. Furthermore, although business and political elites often receive extensive media training, they are often scrutinized by television and radio journalists and therefore can also feel threatened in an interview, particularly in contexts that are less straightforward to prepare for such as academic interviews. On several occasions, for instance, I have been asked by elite respondents or their personal assistants what they need to prepare for before the interview, which suggests that they consider the interview as some form of challenge or justification for what they do.

In many cases, it is not necessarily the figureheads or leaders of organizations and institutions who have the greatest claim to elite status but those who hold important social networks, social capital, and strategic positions within social structures because they are better able to exert influence (Burt 1992; Parry 1998; Smith 2005; Woods 1998). An elite status can also change, with people both gaining and losing theirs over time. In addition, it is geographically specific, with people holding elite status in some but not all locations. In short, it is clear that the term elite can mean many things in different contexts, which explains the range of definitions. The purpose here is not to critique these other definitions but rather to highlight the variety of perspectives.

When referring to my research, I define elites as those who occupy senior-management- and board-level positions within organizations. This is a similar scope of definition to Zuckerman’s (1972) but focuses on a level immediately below her ultraelite subjects. My definition is narrower than McDowell’s (1998) because it is clear in the context of my research that these people have significant decision-making influence within and outside of the firm and therefore present a unique challenge to interview. I deliberately use the term elite more broadly when drawing on examples from the theoretical literature in order to compare my experiences with those who have researched similar groups.

”Changing Dispositions among the Upwardly Mobile” by Curl, Lareau, and Wu ( 2018 )

There is growing interest in the role of cultural practices in undergirding the social stratification system. For example, Lamont et al. (2014) critically assess the preoccupation with economic dimensions of social stratification and call for more developed cultural models of the transmission of inequality. The importance of cultural factors in the maintenance of social inequality has also received empirical attention from some younger scholars, including Calarco (2011, 2014) and Streib (2015). Yet questions remain regarding the degree to which economic position is tied to cultural sensibilities and the ways in which these cultural sensibilities are imprinted on the self or are subject to change. Although habitus is a core concept in Bourdieu’s theory of social reproduction, there is limited empirical attention to the precise areas of the habitus that can be subject to change during upward mobility as well as the ramifications of these changes for family life.

In Bourdieu’s (1984) highly influential work on the importance of class-based cultural dispositions, habitus is defined as a “durable system of dispositions” created in childhood. The habitus provides a “matrix of perceptions” that seems natural while also structuring future actions and pathways. In many of his writings, Bourdieu emphasized the durability of cultural tastes and dispositions and did not consider empirically whether these dispositions might be changed or altered throughout one’s life (Swartz 1997). His theoretical work does permit the possibility of upward mobility and transformation, however, through the ability of the habitus to “improvise” or “change” due to “new experiences” (Friedman 2016:131). Researchers have differed in opinion on the durability of the habitus and its ability to change (King 2000). Based on marital conflict in cross-class marriages, for instance, Streib (2015) argues that cultural dispositions of individuals raised in working-class families are deeply embedded and largely unchanging. In a somewhat different vein, Horvat and Davis (2011:152) argue that young adults enrolled in an alternative educational program undergo important shifts in their self-perception, such as “self-esteem” and their “ability to accomplish something of value.” Others argue there is variability in the degree to which habitus changes dependent on life experience and personality (Christodoulou and Spyridakis 2016). Recently, additional studies have investigated the habitus as it intersects with lifestyle through the lens of meaning making (Ambrasat et al. 2016). There is, therefore, ample discussion of class-based cultural practices in self-perception (Horvat and Davis 2011), lifestyle (Ambrasat et al. 2016), and other forms of taste (Andrews 2012; Bourdieu 1984), yet researchers have not sufficiently delineated which aspects of the habitus might change through upward mobility or which specific dimensions of life prompt moments of class-based conflict.

Bourdieu (1999:511; 2004) acknowledged simmering tensions between the durable aspects of habitus and those aspects that have been transformed—that is, a “fractured” or “cleft” habitus. Others have explored these tensions as a “divided” or “fragmented” habitus (Baxter and Britton 2001; Lee and Kramer 2013). Each of these conceptions of the habitus implies that changes in cultural dispositions are possible but come with costs. Exploration of the specific aspects of one’s habitus that can change and generate conflict contributes to this literature.

Scholars have also studied the costs associated with academic success for working-class undergraduates (Hurst 2010; Lee and Kramer 2013; London 1989; Reay 2017; Rondini 2016; Stuber 2011), but we know little about the lasting effects on adults. For instance, Lee and Kramer (2013) point to cross-class tensions as family and friends criticize upwardly mobile individuals for their newly acquired cultural dispositions. Documenting the tension many working-class students experience with their friends and families of origin, they find that the source of their pain or struggle is “shaped not only by their interactions with non-mobile family and friends but also within their own minds, by their own assessments of their social positions, and by how those positions are interpreted by others” (Lee and Kramer 2013:29). Hurst (2010) also explores the experiences of undergraduates who have been academically successful and the costs associated with that success. She finds that decisions about “class allegiance and identity” are required aspects of what it means to “becom[e] educated” (4) and that working-class students deal with these cultural changes differently. Jack (2014, 2016) also argues that there is diversity among lower-income students, which yields varied college experiences. Naming two groups, the “doubly disadvantaged” and the “privileged poor,” he argues that previous experience with “elite environments” (2014:456) prior to college informs students’ ability to take on dominant cultural practices, particularly around engagement, such as help seeking or meeting with professors (2016). These studies shed light on the role college might play as a “lever for mobility” (2016:15) and discuss the pain and difficulty associated with upward mobility among undergraduates, but the studies do not illuminate how these tensions unfold in adulthood. Neither have they sufficiently addressed potential enduring tensions with extended family members as well as the specific nature of the difficulties.

Some scholars point to the positive outcomes upwardly mobile youth (Lehmann 2009) and adults (Stuber 2005) experience when they maintain a different habitus than their newly acquired class position, although, as Jack (2014, 2016) shows, those experiences may vary depending on one’s experience with elite environments in their youth. Researchers have not sufficiently explored the specific aspects of the habitus that upwardly mobile adults change or the conflicts that emerge with family and childhood friends as they reach adulthood and experience colliding social worlds. We contribute to this scholarship with clear examples of self-reported changes to one’s cultural dispositions in three specific areas: “horizons,” food and health, and communication. We link these changes to enduring tension with family members, friends, and colleagues and explore varied responses to this tension based on race.

Further Readings

Bloomberg, Linda Dale, and Marie F. Volpe. 2012. Completing Your Qualitative Dissertation: A Road Map from Beginning to End . 2nd ed. Thousand Oaks, CA: SAGE. In keeping with its general approach to qualitative research, includes a “road map” for conducting a literature review.

Hart, Chris. 1998. Doing a Literature Review: Releasing the Social Science Research Imagination . London: SAGE. A how-to book dedicated entirely to conducting a literature review from a British perspective. Useful for both undergraduate and graduate students.

Machi, Lawrence A., and Brenda T. McEvoy. 2022. The Literature Review: Six Steps to Success . 4th ed. Newbury Park, CA: Corwin. A well-organized guidebook complete with reflection sections to prompt successful thinking about your literature review.

Ridley, Diana. 2008. The Literature Review: A Step-by-Step Guide for Students . London: SAGE. A highly recommended companion to conducting a literature review for doctoral-level students.

The process of systematically searching through pre-existing studies (“literature”) on the subject of research; also, the section of a presentation in which the pre-existing literature is discussed.

Follow-up questions used in a semi-structured interview  to elicit further elaboration.  Suggested prompts can be included in the interview guide  to be used/deployed depending on how the initial question was answered or if the topic of the prompt does not emerge spontaneously.

A tool for identifying relationships among ideas by visually representing them on paper.  Most concept maps depict ideas as boxes or circles (also called nodes), which are structured hierarchically and connected with lines or arrows (also called arcs). These lines are labeled with linking words and phrases to help explain the connections between concepts.  Also known as mind mapping.

The people who are the subjects of an interview-based qualitative study. In general, they are also known as the participants, and for purposes of IRBs they are often referred to as the human subjects of the research.

Introduction to Qualitative Research Methods Copyright © 2023 by Allison Hurst is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 7, 2024 9:40 AM
• URL: https://libguides.usc.edu/writingguide

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care

Affiliations.

• 1 PhD candidate, School of Nursing and Midwifey, Monash University, and Clinical Nurse Specialist, Adult and Pediatric Intensive Care Unit, Monash Health, Melbourne, Victoria, Australia.
• 2 Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
• 3 Senior Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
• PMID: 26790142
• DOI: 10.1111/wvn.12134

Background: The qualitative systematic review is a rapidly developing area of nursing research. In order to present trustworthy, high-quality recommendations, such reviews should be based on a review protocol to minimize bias and enhance transparency and reproducibility. Although there are a number of resources available to guide researchers in developing a quantitative review protocol, very few resources exist for qualitative reviews.

Aims: To guide researchers through the process of developing a qualitative systematic review protocol, using an example review question.

Methodology: The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction techniques; and data synthesis. The paper highlights important considerations during the protocol development process, and uses a previously developed review question as a working example.

Implications for research: This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings.

Linking evidence to action: Qualitative systematic reviews should be based on well planned, peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in clinical practice. Protocols should outline, in detail, the processes which will be used to undertake the review, including key search terms, inclusion and exclusion criteria, and the methods used for critical appraisal, data extraction and data analysis to facilitate transparency of the review process. Additionally, journals should encourage and support the publication of review protocols, and should require reference to a protocol prior to publication of the review results.

Keywords: guidelines; meta synthesis; qualitative; systematic review protocol.

© 2016 Sigma Theta Tau International.

• Evidence-Based Practice / standards*
• Information Seeking Behavior
• Nursing / methods
• Qualitative Research*
• Research Design / standards*
• Systematic Reviews as Topic*
• Writing / standards*

Literature Review

Literature review is important because it:

• Provides ideas about what should be studied;
• Helps us conduct inquires that have not already been done
• Connects our research to existing studies

But…doing a literature review is not simply summarizing (or copying) what you think is related and useful to your work. BEING CRITICAL AND CAREFUL IS A MUST !

In reviewing existing literature, you may try to look for gaps in the field and rework your study in a different setting or with different people. Nonetheless, literature review is a continuous sense-making process -- you need to review the literature continuously in order to organize your thoughts and refine your analysis.

A good literature review should be able to:  

• Connect to your research questions
• Connect to your choice of methods and research design
• Support your data analysis
• Help you draw conclusions and make claims about your research.

Selecting your literature with a purpose

It is impossible to read everything, so when selecting literature  for reviewing, consider these:

• Is it relevant to your topic/field of study?
• Is it a primary source from the researcher(s) or secondary source (e.g. a summary you read in a book about someone’s research)?
• Is it updated?

Nature of literatures:

Your literature review can be of different dimensions. Each has its foci and purposes

Research Process :: Step by Step

• Introduction
• Select Topic
• Identify Keywords
• Background Information
• Develop Research Questions
• Refine Topic
• Search Strategy
• Popular Databases
• Evaluate Sources
• Types of Periodicals
• Reading Scholarly Articles
• Primary & Secondary Sources
• Organize / Take Notes
• Writing & Grammar Resources
• Annotated Bibliography
• Literature Review
• Citation Styles
• Paraphrasing
• Privacy / Confidentiality
• Research Process
• Selecting Your Topic
• Identifying Keywords
• Gathering Background Info
• Evaluating Sources

Organize the literature review into sections that present themes or identify trends, including relevant theory. You are not trying to list all the material published, but to synthesize and evaluate it according to the guiding concept of your thesis or research question.  

What is a literature review?

A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries

A literature review must do these things:

• be organized around and related directly to the thesis or research question you are developing
• synthesize results into a summary of what is and is not known
• identify areas of controversy in the literature
• formulate questions that need further research

Ask yourself questions like these:

• What is the specific thesis, problem, or research question that my literature review helps to define?
• What type of literature review am I conducting? Am I looking at issues of theory? methodology? policy? quantitative research (e.g. on the effectiveness of a new procedure)? qualitative research (e.g., studies of loneliness among migrant workers)?
• What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)? What discipline am I working in (e.g., nursing psychology, sociology, medicine)?
• How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
• Have I critically analyzed the literature I use? Do I follow through a set of concepts and questions, comparing items to each other in the ways they deal with them? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
• Have I cited and discussed studies contrary to my perspective?
• Will the reader find my literature review relevant, appropriate, and useful?

Ask yourself questions like these about each book or article you include:

• Has the author formulated a problem/issue?
• Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
• Could the problem have been approached more effectively from another perspective?
• What is the author's research orientation (e.g., interpretive, critical science, combination)?
• What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
• What is the relationship between the theoretical and research perspectives?
• Has the author evaluated the literature relevant to the problem/issue? Does the author include literature taking positions she or he does not agree with?
• In a research study, how good are the basic components of the study design (e.g., population, intervention, outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
• In material written for a popular readership, does the author use appeals to emotion, one-sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
• How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?
• In what ways does this book or article contribute to our understanding of the problem under study, and in what ways is it useful for practice? What are the strengths and limitations?
• How does this book or article relate to the specific thesis or question I am developing?

Text written by Dena Taylor, Health Sciences Writing Centre, University of Toronto

http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review

• << Previous: Annotated Bibliography
• Next: Step 5: Cite Sources >>
• Last Updated: May 8, 2024 10:57 AM
• URL: https://libguides.uta.edu/researchprocess

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

• Internet Privacy
• Accessibility
• Problems with a guide? Contact Us.

Criteria for Good Qualitative Research: A Comprehensive Review

• Regular Article
• Open access
• Published: 18 September 2021
• Volume 31 , pages 679–689, ( 2022 )

Cite this article

You have full access to this open access article

• Drishti Yadav   ORCID: orcid.org/0000-0002-2974-0323 1  

80k Accesses

28 Citations

71 Altmetric

Explore all metrics

This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then, references of relevant articles were surveyed to find noteworthy, distinct, and well-defined pointers to good qualitative research. This review presents an investigative assessment of the pivotal features in qualitative research that can permit the readers to pass judgment on its quality and to condemn it as good research when objectively and adequately utilized. Overall, this review underlines the crux of qualitative research and accentuates the necessity to evaluate such research by the very tenets of its being. It also offers some prospects and recommendations to improve the quality of qualitative research. Based on the findings of this review, it is concluded that quality criteria are the aftereffect of socio-institutional procedures and existing paradigmatic conducts. Owing to the paradigmatic diversity of qualitative research, a single and specific set of quality criteria is neither feasible nor anticipated. Since qualitative research is not a cohesive discipline, researchers need to educate and familiarize themselves with applicable norms and decisive factors to evaluate qualitative research from within its theoretical and methodological framework of origin.

Similar content being viewed by others

Good Qualitative Research: Opening up the Debate

Beyond qualitative/quantitative structuralism: the positivist qualitative research and the paradigmatic disclaimer.

What is Qualitative in Research

Avoid common mistakes on your manuscript.

Introduction

“… It is important to regularly dialogue about what makes for good qualitative research” (Tracy, 2010 , p. 837)

To decide what represents good qualitative research is highly debatable. There are numerous methods that are contained within qualitative research and that are established on diverse philosophical perspectives. Bryman et al., ( 2008 , p. 262) suggest that “It is widely assumed that whereas quality criteria for quantitative research are well‐known and widely agreed, this is not the case for qualitative research.” Hence, the question “how to evaluate the quality of qualitative research” has been continuously debated. There are many areas of science and technology wherein these debates on the assessment of qualitative research have taken place. Examples include various areas of psychology: general psychology (Madill et al., 2000 ); counseling psychology (Morrow, 2005 ); and clinical psychology (Barker & Pistrang, 2005 ), and other disciplines of social sciences: social policy (Bryman et al., 2008 ); health research (Sparkes, 2001 ); business and management research (Johnson et al., 2006 ); information systems (Klein & Myers, 1999 ); and environmental studies (Reid & Gough, 2000 ). In the literature, these debates are enthused by the impression that the blanket application of criteria for good qualitative research developed around the positivist paradigm is improper. Such debates are based on the wide range of philosophical backgrounds within which qualitative research is conducted (e.g., Sandberg, 2000 ; Schwandt, 1996 ). The existence of methodological diversity led to the formulation of different sets of criteria applicable to qualitative research.

Among qualitative researchers, the dilemma of governing the measures to assess the quality of research is not a new phenomenon, especially when the virtuous triad of objectivity, reliability, and validity (Spencer et al., 2004 ) are not adequate. Occasionally, the criteria of quantitative research are used to evaluate qualitative research (Cohen & Crabtree, 2008 ; Lather, 2004 ). Indeed, Howe ( 2004 ) claims that the prevailing paradigm in educational research is scientifically based experimental research. Hypotheses and conjectures about the preeminence of quantitative research can weaken the worth and usefulness of qualitative research by neglecting the prominence of harmonizing match for purpose on research paradigm, the epistemological stance of the researcher, and the choice of methodology. Researchers have been reprimanded concerning this in “paradigmatic controversies, contradictions, and emerging confluences” (Lincoln & Guba, 2000 ).

In general, qualitative research tends to come from a very different paradigmatic stance and intrinsically demands distinctive and out-of-the-ordinary criteria for evaluating good research and varieties of research contributions that can be made. This review attempts to present a series of evaluative criteria for qualitative researchers, arguing that their choice of criteria needs to be compatible with the unique nature of the research in question (its methodology, aims, and assumptions). This review aims to assist researchers in identifying some of the indispensable features or markers of high-quality qualitative research. In a nutshell, the purpose of this systematic literature review is to analyze the existing knowledge on high-quality qualitative research and to verify the existence of research studies dealing with the critical assessment of qualitative research based on the concept of diverse paradigmatic stances. Contrary to the existing reviews, this review also suggests some critical directions to follow to improve the quality of qualitative research in different epistemological and ontological perspectives. This review is also intended to provide guidelines for the acceleration of future developments and dialogues among qualitative researchers in the context of assessing the qualitative research.

The rest of this review article is structured in the following fashion: Sect.  Methods describes the method followed for performing this review. Section Criteria for Evaluating Qualitative Studies provides a comprehensive description of the criteria for evaluating qualitative studies. This section is followed by a summary of the strategies to improve the quality of qualitative research in Sect.  Improving Quality: Strategies . Section  How to Assess the Quality of the Research Findings? provides details on how to assess the quality of the research findings. After that, some of the quality checklists (as tools to evaluate quality) are discussed in Sect.  Quality Checklists: Tools for Assessing the Quality . At last, the review ends with the concluding remarks presented in Sect.  Conclusions, Future Directions and Outlook . Some prospects in qualitative research for enhancing its quality and usefulness in the social and techno-scientific research community are also presented in Sect.  Conclusions, Future Directions and Outlook .

For this review, a comprehensive literature search was performed from many databases using generic search terms such as Qualitative Research , Criteria , etc . The following databases were chosen for the literature search based on the high number of results: IEEE Explore, ScienceDirect, PubMed, Google Scholar, and Web of Science. The following keywords (and their combinations using Boolean connectives OR/AND) were adopted for the literature search: qualitative research, criteria, quality, assessment, and validity. The synonyms for these keywords were collected and arranged in a logical structure (see Table 1 ). All publications in journals and conference proceedings later than 1950 till 2021 were considered for the search. Other articles extracted from the references of the papers identified in the electronic search were also included. A large number of publications on qualitative research were retrieved during the initial screening. Hence, to include the searches with the main focus on criteria for good qualitative research, an inclusion criterion was utilized in the search string.

From the selected databases, the search retrieved a total of 765 publications. Then, the duplicate records were removed. After that, based on the title and abstract, the remaining 426 publications were screened for their relevance by using the following inclusion and exclusion criteria (see Table 2 ). Publications focusing on evaluation criteria for good qualitative research were included, whereas those works which delivered theoretical concepts on qualitative research were excluded. Based on the screening and eligibility, 45 research articles were identified that offered explicit criteria for evaluating the quality of qualitative research and were found to be relevant to this review.

Figure  1 illustrates the complete review process in the form of PRISMA flow diagram. PRISMA, i.e., “preferred reporting items for systematic reviews and meta-analyses” is employed in systematic reviews to refine the quality of reporting.

PRISMA flow diagram illustrating the search and inclusion process. N represents the number of records

Criteria for Evaluating Qualitative Studies

Fundamental criteria: general research quality.

Various researchers have put forward criteria for evaluating qualitative research, which have been summarized in Table 3 . Also, the criteria outlined in Table 4 effectively deliver the various approaches to evaluate and assess the quality of qualitative work. The entries in Table 4 are based on Tracy’s “Eight big‐tent criteria for excellent qualitative research” (Tracy, 2010 ). Tracy argues that high-quality qualitative work should formulate criteria focusing on the worthiness, relevance, timeliness, significance, morality, and practicality of the research topic, and the ethical stance of the research itself. Researchers have also suggested a series of questions as guiding principles to assess the quality of a qualitative study (Mays & Pope, 2020 ). Nassaji ( 2020 ) argues that good qualitative research should be robust, well informed, and thoroughly documented.

Qualitative Research: Interpretive Paradigms

All qualitative researchers follow highly abstract principles which bring together beliefs about ontology, epistemology, and methodology. These beliefs govern how the researcher perceives and acts. The net, which encompasses the researcher’s epistemological, ontological, and methodological premises, is referred to as a paradigm, or an interpretive structure, a “Basic set of beliefs that guides action” (Guba, 1990 ). Four major interpretive paradigms structure the qualitative research: positivist and postpositivist, constructivist interpretive, critical (Marxist, emancipatory), and feminist poststructural. The complexity of these four abstract paradigms increases at the level of concrete, specific interpretive communities. Table 5 presents these paradigms and their assumptions, including their criteria for evaluating research, and the typical form that an interpretive or theoretical statement assumes in each paradigm. Moreover, for evaluating qualitative research, quantitative conceptualizations of reliability and validity are proven to be incompatible (Horsburgh, 2003 ). In addition, a series of questions have been put forward in the literature to assist a reviewer (who is proficient in qualitative methods) for meticulous assessment and endorsement of qualitative research (Morse, 2003 ). Hammersley ( 2007 ) also suggests that guiding principles for qualitative research are advantageous, but methodological pluralism should not be simply acknowledged for all qualitative approaches. Seale ( 1999 ) also points out the significance of methodological cognizance in research studies.

Table 5 reflects that criteria for assessing the quality of qualitative research are the aftermath of socio-institutional practices and existing paradigmatic standpoints. Owing to the paradigmatic diversity of qualitative research, a single set of quality criteria is neither possible nor desirable. Hence, the researchers must be reflexive about the criteria they use in the various roles they play within their research community.

Improving Quality: Strategies

Another critical question is “How can the qualitative researchers ensure that the abovementioned quality criteria can be met?” Lincoln and Guba ( 1986 ) delineated several strategies to intensify each criteria of trustworthiness. Other researchers (Merriam & Tisdell, 2016 ; Shenton, 2004 ) also presented such strategies. A brief description of these strategies is shown in Table 6 .

It is worth mentioning that generalizability is also an integral part of qualitative research (Hays & McKibben, 2021 ). In general, the guiding principle pertaining to generalizability speaks about inducing and comprehending knowledge to synthesize interpretive components of an underlying context. Table 7 summarizes the main metasynthesis steps required to ascertain generalizability in qualitative research.

Figure  2 reflects the crucial components of a conceptual framework and their contribution to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice (Johnson et al., 2020 ). The synergy and interrelationship of these components signifies their role to different stances of a qualitative research study.

Essential elements of a conceptual framework

In a nutshell, to assess the rationale of a study, its conceptual framework and research question(s), quality criteria must take account of the following: lucid context for the problem statement in the introduction; well-articulated research problems and questions; precise conceptual framework; distinct research purpose; and clear presentation and investigation of the paradigms. These criteria would expedite the quality of qualitative research.

How to Assess the Quality of the Research Findings?

The inclusion of quotes or similar research data enhances the confirmability in the write-up of the findings. The use of expressions (for instance, “80% of all respondents agreed that” or “only one of the interviewees mentioned that”) may also quantify qualitative findings (Stenfors et al., 2020 ). On the other hand, the persuasive reason for “why this may not help in intensifying the research” has also been provided (Monrouxe & Rees, 2020 ). Further, the Discussion and Conclusion sections of an article also prove robust markers of high-quality qualitative research, as elucidated in Table 8 .

Quality Checklists: Tools for Assessing the Quality

Numerous checklists are available to speed up the assessment of the quality of qualitative research. However, if used uncritically and recklessly concerning the research context, these checklists may be counterproductive. I recommend that such lists and guiding principles may assist in pinpointing the markers of high-quality qualitative research. However, considering enormous variations in the authors’ theoretical and philosophical contexts, I would emphasize that high dependability on such checklists may say little about whether the findings can be applied in your setting. A combination of such checklists might be appropriate for novice researchers. Some of these checklists are listed below:

The most commonly used framework is Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ). This framework is recommended by some journals to be followed by the authors during article submission.

Standards for Reporting Qualitative Research (SRQR) is another checklist that has been created particularly for medical education (O’Brien et al., 2014 ).

Also, Tracy ( 2010 ) and Critical Appraisal Skills Programme (CASP, 2021 ) offer criteria for qualitative research relevant across methods and approaches.

Further, researchers have also outlined different criteria as hallmarks of high-quality qualitative research. For instance, the “Road Trip Checklist” (Epp & Otnes, 2021 ) provides a quick reference to specific questions to address different elements of high-quality qualitative research.

Conclusions, Future Directions, and Outlook

This work presents a broad review of the criteria for good qualitative research. In addition, this article presents an exploratory analysis of the essential elements in qualitative research that can enable the readers of qualitative work to judge it as good research when objectively and adequately utilized. In this review, some of the essential markers that indicate high-quality qualitative research have been highlighted. I scope them narrowly to achieve rigor in qualitative research and note that they do not completely cover the broader considerations necessary for high-quality research. This review points out that a universal and versatile one-size-fits-all guideline for evaluating the quality of qualitative research does not exist. In other words, this review also emphasizes the non-existence of a set of common guidelines among qualitative researchers. In unison, this review reinforces that each qualitative approach should be treated uniquely on account of its own distinctive features for different epistemological and disciplinary positions. Owing to the sensitivity of the worth of qualitative research towards the specific context and the type of paradigmatic stance, researchers should themselves analyze what approaches can be and must be tailored to ensemble the distinct characteristics of the phenomenon under investigation. Although this article does not assert to put forward a magic bullet and to provide a one-stop solution for dealing with dilemmas about how, why, or whether to evaluate the “goodness” of qualitative research, it offers a platform to assist the researchers in improving their qualitative studies. This work provides an assembly of concerns to reflect on, a series of questions to ask, and multiple sets of criteria to look at, when attempting to determine the quality of qualitative research. Overall, this review underlines the crux of qualitative research and accentuates the need to evaluate such research by the very tenets of its being. Bringing together the vital arguments and delineating the requirements that good qualitative research should satisfy, this review strives to equip the researchers as well as reviewers to make well-versed judgment about the worth and significance of the qualitative research under scrutiny. In a nutshell, a comprehensive portrayal of the research process (from the context of research to the research objectives, research questions and design, speculative foundations, and from approaches of collecting data to analyzing the results, to deriving inferences) frequently proliferates the quality of a qualitative research.

Prospects : A Road Ahead for Qualitative Research

Irrefutably, qualitative research is a vivacious and evolving discipline wherein different epistemological and disciplinary positions have their own characteristics and importance. In addition, not surprisingly, owing to the sprouting and varied features of qualitative research, no consensus has been pulled off till date. Researchers have reflected various concerns and proposed several recommendations for editors and reviewers on conducting reviews of critical qualitative research (Levitt et al., 2021 ; McGinley et al., 2021 ). Following are some prospects and a few recommendations put forward towards the maturation of qualitative research and its quality evaluation:

In general, most of the manuscript and grant reviewers are not qualitative experts. Hence, it is more likely that they would prefer to adopt a broad set of criteria. However, researchers and reviewers need to keep in mind that it is inappropriate to utilize the same approaches and conducts among all qualitative research. Therefore, future work needs to focus on educating researchers and reviewers about the criteria to evaluate qualitative research from within the suitable theoretical and methodological context.

There is an urgent need to refurbish and augment critical assessment of some well-known and widely accepted tools (including checklists such as COREQ, SRQR) to interrogate their applicability on different aspects (along with their epistemological ramifications).

Efforts should be made towards creating more space for creativity, experimentation, and a dialogue between the diverse traditions of qualitative research. This would potentially help to avoid the enforcement of one's own set of quality criteria on the work carried out by others.

Moreover, journal reviewers need to be aware of various methodological practices and philosophical debates.

It is pivotal to highlight the expressions and considerations of qualitative researchers and bring them into a more open and transparent dialogue about assessing qualitative research in techno-scientific, academic, sociocultural, and political rooms.

Frequent debates on the use of evaluative criteria are required to solve some potentially resolved issues (including the applicability of a single set of criteria in multi-disciplinary aspects). Such debates would not only benefit the group of qualitative researchers themselves, but primarily assist in augmenting the well-being and vivacity of the entire discipline.

To conclude, I speculate that the criteria, and my perspective, may transfer to other methods, approaches, and contexts. I hope that they spark dialog and debate – about criteria for excellent qualitative research and the underpinnings of the discipline more broadly – and, therefore, help improve the quality of a qualitative study. Further, I anticipate that this review will assist the researchers to contemplate on the quality of their own research, to substantiate research design and help the reviewers to review qualitative research for journals. On a final note, I pinpoint the need to formulate a framework (encompassing the prerequisites of a qualitative study) by the cohesive efforts of qualitative researchers of different disciplines with different theoretic-paradigmatic origins. I believe that tailoring such a framework (of guiding principles) paves the way for qualitative researchers to consolidate the status of qualitative research in the wide-ranging open science debate. Dialogue on this issue across different approaches is crucial for the impending prospects of socio-techno-educational research.

Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16 (10), 1472–1482.

Article   Google Scholar  

Barker, C., & Pistrang, N. (2005). Quality criteria under methodological pluralism: Implications for conducting and evaluating research. American Journal of Community Psychology, 35 (3–4), 201–212.

Bryman, A., Becker, S., & Sempik, J. (2008). Quality criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal of Social Research Methodology, 11 (4), 261–276.

Caelli, K., Ray, L., & Mill, J. (2003). ‘Clear as mud’: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2 (2), 1–13.

CASP (2021). CASP checklists. Retrieved May 2021 from https://casp-uk.net/casp-tools-checklists/

Cohen, D. J., & Crabtree, B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. The Annals of Family Medicine, 6 (4), 331–339.

Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (pp. 1–32). Sage Publications Ltd.

Google Scholar  

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38 (3), 215–229.

Epp, A. M., & Otnes, C. C. (2021). High-quality qualitative research: Getting into gear. Journal of Service Research . https://doi.org/10.1177/1094670520961445

Guba, E. G. (1990). The paradigm dialog. In Alternative paradigms conference, mar, 1989, Indiana u, school of education, San Francisco, ca, us . Sage Publications, Inc.

Hammersley, M. (2007). The issue of quality in qualitative research. International Journal of Research and Method in Education, 30 (3), 287–305.

Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., & Mokkink, L. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19 , 1609406920976417.

Hays, D. G., & McKibben, W. B. (2021). Promoting rigorous research: Generalizability and qualitative research. Journal of Counseling and Development, 99 (2), 178–188.

Horsburgh, D. (2003). Evaluation of qualitative research. Journal of Clinical Nursing, 12 (2), 307–312.

Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10 (1), 42–46.

Johnson, J. L., Adkins, D., & Chauvin, S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84 (1), 7120.

Johnson, P., Buehring, A., Cassell, C., & Symon, G. (2006). Evaluating qualitative management research: Towards a contingent criteriology. International Journal of Management Reviews, 8 (3), 131–156.

Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23 (1), 67–93.

Lather, P. (2004). This is your father’s paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10 (1), 15–34.

Levitt, H. M., Morrill, Z., Collins, K. M., & Rizo, J. L. (2021). The methodological integrity of critical qualitative research: Principles to support design and research review. Journal of Counseling Psychology, 68 (3), 357.

Lincoln, Y. S., & Guba, E. G. (1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation, 1986 (30), 73–84.

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163–188). Sage Publications.

Madill, A., Jordan, A., & Shirley, C. (2000). Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies. British Journal of Psychology, 91 (1), 1–20.

Mays, N., & Pope, C. (2020). Quality in qualitative research. Qualitative Research in Health Care . https://doi.org/10.1002/9781119410867.ch15

McGinley, S., Wei, W., Zhang, L., & Zheng, Y. (2021). The state of qualitative research in hospitality: A 5-year review 2014 to 2019. Cornell Hospitality Quarterly, 62 (1), 8–20.

Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and implementation. San Francisco, US.

Meyer, M., & Dykes, J. (2019). Criteria for rigor in visualization design study. IEEE Transactions on Visualization and Computer Graphics, 26 (1), 87–97.

Monrouxe, L. V., & Rees, C. E. (2020). When I say… quantification in qualitative research. Medical Education, 54 (3), 186–187.

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52 (2), 250.

Morse, J. M. (2003). A review committee’s guide for evaluating qualitative proposals. Qualitative Health Research, 13 (6), 833–851.

Nassaji, H. (2020). Good qualitative research. Language Teaching Research, 24 (4), 427–431.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89 (9), 1245–1251.

O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19 , 1609406919899220.

Reid, A., & Gough, S. (2000). Guidelines for reporting and evaluating qualitative research: What are the alternatives? Environmental Education Research, 6 (1), 59–91.

Rocco, T. S. (2010). Criteria for evaluating qualitative studies. Human Resource Development International . https://doi.org/10.1080/13678868.2010.501959

Sandberg, J. (2000). Understanding human competence at work: An interpretative approach. Academy of Management Journal, 43 (1), 9–25.

Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2 (1), 58–72.

Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5 (4), 465–478.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 (2), 63–75.

Sparkes, A. C. (2001). Myth 94: Qualitative health researchers will agree about validity. Qualitative Health Research, 11 (4), 538–552.

Spencer, L., Ritchie, J., Lewis, J., & Dillon, L. (2004). Quality in qualitative evaluation: A framework for assessing research evidence.

Stenfors, T., Kajamaa, A., & Bennett, D. (2020). How to assess the quality of qualitative research. The Clinical Teacher, 17 (6), 596–599.

Taylor, E. W., Beck, J., & Ainsworth, E. (2001). Publishing qualitative adult education research: A peer review perspective. Studies in the Education of Adults, 33 (2), 163–179.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19 (6), 349–357.

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16 (10), 837–851.

Download references

Open access funding provided by TU Wien (TUW).

Author information

Authors and affiliations.

Faculty of Informatics, Technische Universität Wien, 1040, Vienna, Austria

Drishti Yadav

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Drishti Yadav .

Ethics declarations

Conflict of interest.

The author declares no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Yadav, D. Criteria for Good Qualitative Research: A Comprehensive Review. Asia-Pacific Edu Res 31 , 679–689 (2022). https://doi.org/10.1007/s40299-021-00619-0

Download citation

Accepted : 28 August 2021

Published : 18 September 2021

Issue Date : December 2022

DOI : https://doi.org/10.1007/s40299-021-00619-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Evaluative criteria
• Find a journal
• Publish with us
• Track your research

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker
• APA Citation Generator
• MLA Citation Generator
• Chicago Citation Generator
• Vancouver Citation Generator
• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Make a Literature Review in Research (RRL Example)

What is an RRL in a research paper?

A relevant review of the literature (RRL) is an objective, concise, critical summary of published research literature relevant to a topic being researched in an article. In an RRL, you discuss knowledge and findings from existing literature relevant to your study topic. If there are conflicts or gaps in existing literature, you can also discuss these in your review, as well as how you will confront these missing elements or resolve these issues in your study.

To complete an RRL, you first need to collect relevant literature; this can include online and offline sources. Save all of your applicable resources as you will need to include them in your paper. When looking through these sources, take notes and identify concepts of each source to describe in the review of the literature.

A good RRL does NOT:

A literature review does not simply reference and list all of the material you have cited in your paper.

• Presenting material that is not directly relevant to your study will distract and frustrate the reader and make them lose sight of the purpose of your study.
• Starting a literature review with “A number of scholars have studied the relationship between X and Y” and simply listing who has studied the topic and what each scholar concluded is not going to strengthen your paper.

A good RRL DOES:

• Present a brief typology that orders articles and books into groups to help readers focus on unresolved debates, inconsistencies, tensions, and new questions about a research topic.
• Summarize the most relevant and important aspects of the scientific literature related to your area of research
• Synthesize what has been done in this area of research and by whom, highlight what previous research indicates about a topic, and identify potential gaps and areas of disagreement in the field
• Give the reader an understanding of the background of the field and show which studies are important—and highlight errors in previous studies

How long is a review of the literature for a research paper?

The length of a review of the literature depends on its purpose and target readership and can vary significantly in scope and depth. In a dissertation, thesis, or standalone review of literature, it is usually a full chapter of the text (at least 20 pages). Whereas, a standard research article or school assignment literature review section could only be a few paragraphs in the Introduction section .

Building Your Literature Review Bookshelf

One way to conceive of a literature review is to think about writing it as you would build a bookshelf. You don’t need to cut each piece by yourself from scratch. Rather, you can take the pieces that other researchers have cut out and put them together to build a framework on which to hang your own “books”—that is, your own study methods, results, and conclusions.

What Makes a Good Literature Review?

The contents of a literature review (RRL) are determined by many factors, including its precise purpose in the article, the degree of consensus with a given theory or tension between competing theories, the length of the article, the number of previous studies existing in the given field, etc. The following are some of the most important elements that a literature review provides.

Historical background for your research

Analyze what has been written about your field of research to highlight what is new and significant in your study—or how the analysis itself contributes to the understanding of this field, even in a small way. Providing a historical background also demonstrates to other researchers and journal editors your competency in discussing theoretical concepts. You should also make sure to understand how to paraphrase scientific literature to avoid plagiarism in your work.

The current context of your research

Discuss central (or peripheral) questions, issues, and debates in the field. Because a field is constantly being updated by new work, you can show where your research fits into this context and explain developments and trends in research.

A discussion of relevant theories and concepts

Theories and concepts should provide the foundation for your research. For example, if you are researching the relationship between ecological environments and human populations, provide models and theories that focus on specific aspects of this connection to contextualize your study. If your study asks a question concerning sustainability, mention a theory or model that underpins this concept. If it concerns invasive species, choose material that is focused in this direction.

Definitions of relevant terminology

In the natural sciences, the meaning of terms is relatively straightforward and consistent. But if you present a term that is obscure or context-specific, you should define the meaning of the term in the Introduction section (if you are introducing a study) or in the summary of the literature being reviewed.

Description of related relevant research

Include a description of related research that shows how your work expands or challenges earlier studies or fills in gaps in previous work. You can use your literature review as evidence of what works, what doesn’t, and what is missing in the field.

Supporting evidence for a practical problem or issue your research is addressing that demonstrates its importance: Referencing related research establishes your area of research as reputable and shows you are building upon previous work that other researchers have deemed significant.

Types of Literature Reviews

Literature reviews can differ in structure, length, amount, and breadth of content included. They can range from selective (a very narrow area of research or only a single work) to comprehensive (a larger amount or range of works). They can also be part of a larger work or stand on their own.

• A course assignment is an example of a selective, stand-alone work. It focuses on a small segment of the literature on a topic and makes up an entire work on its own.
• The literature review in a dissertation or thesis is both comprehensive and helps make up a larger work.
• A majority of journal articles start with a selective literature review to provide context for the research reported in the study; such a literature review is usually included in the Introduction section (but it can also follow the presentation of the results in the Discussion section ).
• Some literature reviews are both comprehensive and stand as a separate work—in this case, the entire article analyzes the literature on a given topic.

Literature Reviews Found in Academic Journals

The two types of literature reviews commonly found in journals are those introducing research articles (studies and surveys) and stand-alone literature analyses. They can differ in their scope, length, and specific purpose.

Literature reviews introducing research articles

The literature review found at the beginning of a journal article is used to introduce research related to the specific study and is found in the Introduction section, usually near the end. It is shorter than a stand-alone review because it must be limited to very specific studies and theories that are directly relevant to the current study. Its purpose is to set research precedence and provide support for the study’s theory, methods, results, and/or conclusions. Not all research articles contain an explicit review of the literature, but most do, whether it is a discrete section or indistinguishable from the rest of the Introduction.

How to structure a literature review for an article

When writing a literature review as part of an introduction to a study, simply follow the structure of the Introduction and move from the general to the specific—presenting the broadest background information about a topic first and then moving to specific studies that support your rationale , finally leading to your hypothesis statement. Such a literature review is often indistinguishable from the Introduction itself—the literature is INTRODUCING the background and defining the gaps your study aims to fill.

The stand-alone literature review

The literature review published as a stand-alone article presents and analyzes as many of the important publications in an area of study as possible to provide background information and context for a current area of research or a study. Stand-alone reviews are an excellent resource for researchers when they are first searching for the most relevant information on an area of study.

Such literature reviews are generally a bit broader in scope and can extend further back in time. This means that sometimes a scientific literature review can be highly theoretical, in addition to focusing on specific methods and outcomes of previous studies. In addition, all sections of such a “review article” refer to existing literature rather than describing the results of the authors’ own study.

In addition, this type of literature review is usually much longer than the literature review introducing a study. At the end of the review follows a conclusion that once again explicitly ties all of the cited works together to show how this analysis is itself a contribution to the literature. While not absolutely necessary, such articles often include the terms “Literature Review” or “Review of the Literature” in the title. Whether or not that is necessary or appropriate can also depend on the specific author instructions of the target journal. Have a look at this article for more input on how to compile a stand-alone review article that is insightful and helpful for other researchers in your field.

How to Write a Literature Review in 6 Steps

So how do authors turn a network of articles into a coherent review of relevant literature?

Writing a literature review is not usually a linear process—authors often go back and check the literature while reformulating their ideas or making adjustments to their study. Sometimes new findings are published before a study is completed and need to be incorporated into the current work. This also means you will not be writing the literature review at any one time, but constantly working on it before, during, and after your study is complete.

Here are some steps that will help you begin and follow through on your literature review.

Step 1: Choose a topic to write about—focus on and explore this topic.

Choose a topic that you are familiar with and highly interested in analyzing; a topic your intended readers and researchers will find interesting and useful; and a topic that is current, well-established in the field, and about which there has been sufficient research conducted for a review. This will help you find the “sweet spot” for what to focus on.

Step 2: Research and collect all the scholarly information on the topic that might be pertinent to your study.

This includes scholarly articles, books, conventions, conferences, dissertations, and theses—these and any other academic work related to your area of study is called “the literature.”

Step 3: Analyze the network of information that extends or responds to the major works in your area; select the material that is most useful.

Use thought maps and charts to identify intersections in the research and to outline important categories; select the material that will be most useful to your review.

Step 4: Describe and summarize each article—provide the essential information of the article that pertains to your study.

Determine 2-3 important concepts (depending on the length of your article) that are discussed in the literature; take notes about all of the important aspects of this study relevant to the topic being reviewed.

For example, in a given study, perhaps some of the main concepts are X, Y, and Z. Note these concepts and then write a brief summary about how the article incorporates them. In reviews that introduce a study, these can be relatively short. In stand-alone reviews, there may be significantly more texts and more concepts.

Step 5: Demonstrate how these concepts in the literature relate to what you discovered in your study or how the literature connects the concepts or topics being discussed.

In a literature review intro for an article, this information might include a summary of the results or methods of previous studies that correspond to and/or confirm those sections in your own study. For a stand-alone literature review, this may mean highlighting the concepts in each article and showing how they strengthen a hypothesis or show a pattern.

Discuss unaddressed issues in previous studies. These studies that are missing something you address are important to include in your literature review. In addition, those works whose theories and conclusions directly support your findings will be valuable to review here.

Step 6: Identify relationships in the literature and develop and connect your own ideas to them.

This is essentially the same as step 5 but focused on the connections between the literature and the current study or guiding concepts or arguments of the paper, not only on the connections between the works themselves.

Your hypothesis, argument, or guiding concept is the “golden thread” that will ultimately tie the works together and provide readers with specific insights they didn’t have before reading your literature review. Make sure you know where to put the research question , hypothesis, or statement of the problem in your research paper so that you guide your readers logically and naturally from your introduction of earlier work and evidence to the conclusions you want them to draw from the bigger picture.

Your literature review will not only cover publications on your topics but will include your own ideas and contributions. By following these steps you will be telling the specific story that sets the background and shows the significance of your research and you can turn a network of related works into a focused review of the literature.

Literature Review (RRL) Examples

Because creating sample literature reviews would take too long and not properly capture the nuances and detailed information needed for a good review, we have included some links to different types of literature reviews below. You can find links to more literature reviews in these categories by visiting the TUS Library’s website . Sample literature reviews as part of an article, dissertation, or thesis:

• Critical Thinking and Transferability: A Review of the Literature (Gwendolyn Reece)
• Building Customer Loyalty: A Customer Experience Based Approach in a Tourism Context (Martina Donnelly)

Sample stand-alone literature reviews

• Literature Review on Attitudes towards Disability (National Disability Authority)
• The Effects of Communication Styles on Marital Satisfaction (Hannah Yager)

Additional Literature Review Format Guidelines

In addition to the content guidelines above, authors also need to check which style guidelines to use ( APA , Chicago, MLA, etc.) and what specific rules the target journal might have for how to structure such articles or how many studies to include—such information can usually be found on the journals’ “Guide for Authors” pages. Additionally, use one of the four Wordvice citation generators below, choosing the citation style needed for your paper:

Wordvice Writing and Academic Editing Resources

Finally, after you have finished drafting your literature review, be sure to receive professional proofreading services , including paper editing for your academic work. A competent proofreader who understands academic writing conventions and the specific style guides used by academic journals will ensure that your paper is ready for publication in your target journal.

See our academic resources for further advice on references in your paper , how to write an abstract , how to write a research paper title, how to impress the editor of your target journal with a perfect cover letter , and dozens of other research writing and publication topics.

• Open access
• Published: 05 May 2024

A qualitative interview study to determine barriers and facilitators of implementing automated decision support tools for genomic data access

• Vasiliki Rahimzadeh 1 ,
• Jinyoung Baek 2 ,
• Jonathan Lawson 2 &
• Edward S. Dove 3  

BMC Medical Ethics volume  25 , Article number:  51 ( 2024 ) Cite this article

91 Accesses

1 Altmetric

Metrics details

Data access committees (DAC) gatekeep access to secured genomic and related health datasets yet are challenged to keep pace with the rising volume and complexity of data generation. Automated decision support (ADS) systems have been shown to support consistency, compliance, and coordination of data access review decisions. However, we lack understanding of how DAC members perceive the value add of ADS, if any, on the quality and effectiveness of their reviews. In this qualitative study, we report findings from 13 semi-structured interviews with DAC members from around the world to identify relevant barriers and facilitators to implementing ADS for genomic data access management. Participants generally supported pilot studies that test ADS performance, for example in cataloging data types, verifying user credentials and tagging datasets for use terms. Concerns related to over-automation, lack of human oversight, low prioritization, and misalignment with institutional missions tempered enthusiasm for ADS among the DAC members we engaged. Tensions for change in institutional settings within which DACs operated was a powerful motivator for why DAC members considered the implementation of ADS into their access workflows, as well as perceptions of the relative advantage of ADS over the status quo. Future research is needed to build the evidence base around the comparative effectiveness and decisional outcomes of institutions that do/not use ADS into their workflows.

Peer Review reports

Introduction

Genomics is among the most data-prolific scientific fields and is expected to surpass the storage needs and analytic capacities of Twitter, YouTube, and astronomy combined by as soon as 2025 [ 1 ]. To meet rising demands for genomic data and their efficient collection and use, national genomics initiatives [ 2 ] rely on largescale repositories to pool data resources and incentivize data sharing [ 3 , 4 , 5 ]. The “data commons” model has since become the flagship approach for many of these initiatives [ 6 ], and prioritizes research collaboration and data access over proprietary exclusion in the data [ 3 ]. Data access committees (DACs) are principally charged with ensuring only bona fide researchers conducting research permitted by participants’ informed consent are approved to access the data [ 7 ]. DACs are typically staffed by research compliance officers, researchers, and sometimes data security professionals. DAC members can be paid or serve as volunteers and, at a basic level, arbitrate access to data given requests meet minimum requirements for data protection and compliance. Critiques of compliance-only responsibilities and the growing appreciation of data privacy risks among the general public has raised questions about whether DACs ought to weigh in on issues of social and scientific value of the data projects [ 8 ]. Our prior empirical work [ 9 ] suggests there is debate around this scope of DAC oversight, particularly as it relates to considerations of data ethics that are traditionally the domain of institutional ethics committees.

Cheah and Piasecki, for example, propose that DACs have responsibilities to both promote data sharing and protect the interests of individuals and communities about whom the shared data relate: “data access should be granted as long as the data reuse fulfils the criterion of having even a minimal social value, and minimal risk to data subjects and their communities” [ 7 ]. In this way, DACs anchor responsible data sharing ecosystems since they govern access to and compliant use of genomic and, increasingly, other health data [ 10 , 11 , 12 ].

However, DACs may not contribute to efficient data access provisions as effectively as other review models may allow [ 13 ]. In the standard model of data access review, DACs manually review a data requester’s application and assess it against pre-defined criteria. Criteria may include appropriateness of the data requested, data use terms set by data providers, and data privacy and security requirements set by the institution and by law [ 7 ]. As with most, if not all, human-mediated activities, manual review of these criteria can be a laborious and error-prone process. For example, DACs may interpret language describing permitted data uses differently, and the terms themselves can sometimes be ambiguous [ 14 ]. Faced with this ambiguity, DACs are forced to make subjective judgments about whether requests for data access truly align with permitted data uses, if these permissions have been preserved at all. Inconsistencies in how data use terms are articulated in consent forms and subsequently interpreted and executed by DACs across the biomedical ecosystem [ 14 ] can lead to delayed and inconsistent data access decisions, and risk violating the terms by which patients or participants contributed their data in the first place.

Other steps in the data access pipeline can also contribute to research delays. Emerging research suggests there is growing inefficiency, inconsistency, and error in the manual, entirely human-mediated review of data access agreements [ 13 , 15 ] which are executed in finalizing approved data access requests. Many researchers furthermore still rely on the traditional method of copying-and-downloading data once approved. The copy-download approach multiplies security risks [ 11 ], and is quickly becoming unreasonable given the expanding size and complexities of genomic datasets [ 16 , 17 ].

Standards’ developers and software engineers have therefore sought to semi-automate three axes of data access control within cloud environments – user authentication, review of access requests, and concordance of the proposed research with the data use terms of the data requested [ 14 ]. Automated decision support (ADS) systems are a coordinated system of algorithms, software, and ontologies [ 18 ] that aid in categorizing, archiving, and/or acting on decision tasks for data access review. The Data Use Oversight System (DUOS) typifies one such automated decision support [ 19 ]. In recent beta tests, DUOS was successfully shown to concur 100% of the time with human-decided access requests [ 15 ], and also codifies 93% of genomic datasets in NIH’s dbGaP [ 20 ].

While ADS can supplement human DACs with semi-automated technical solutions, no systematic investigation has sought to characterize relevant barriers and facilitators to ADS in practice [ 21 ]. Moreover, we lack understanding of how DAC members perceive the value added by ADS, if any, on the quality and effectiveness of data access review decisions, as well as what challenges they anticipate in adopting ADS considering the myriad organizational structures within which DACs operate.

Now is an opportune time to study the implementation barriers and facilitators to using ADS solutions for data access as their development converges with large-scale data migration to the cloud that can result in near-instant data access decisions. The genomics community can learn important lessons from previous attempts at (premature) ADS implementation without purposeful stakeholder engagement in public health [ 22 ], law enforcement [ 23 ] and in clinical care [ 24 ]. In this article, we report empirical findings on the “constellation of processes” relevant for implementing ADS for genomic data access management and provide practical recommendations for institutional data stewards that are considering or have already implemented ADS in this context.

We conducted a qualitative description study that engaged prospective end users of ADS for genomic data governance to explore: What are the barriers and opportunities of implementing automated workflows to manage access requests to genomic data collections, and what effect do ADS have on DAC review quality and effectiveness? We adopted Damshroder and colleagues’ definition of implementation as the “critical gateway between an organizational decision to adopt an intervention and the routine use of that intervention” [ 25 ] in order to “study the constellation of processes intended to get an intervention into use within an organization” [ 25 ]. We applied the Consolidated Framework for Implementation Research (CFIR) to compare genomic data access processes and procedures to better understand implementation processes for automated workflows to manage genomic data access across international, publicly funded genomic data repositories. The CFIR provides a “menu of constructs” associated with five domains of effective implementation which have been rigorously meta-theorized—that is, synthesized from many implementation theories (Fig.  1 ). In addition, the CFIR provides a practical guide to systematically assess potential barriers and facilitators ahead of an innovation’s implementation (L. Damschroder et al. 2015). The CFIR is also easily customizable to unveiling bioethical issues during implementation in genomics and has been applied in prior work (Burke and Korngiebel, 2015; Smit et al., 2020).

Adapted Consolidated Framework for Implementation Research (CFIR) and associated domains (Intervention Characteristics, Individuals, Process, Inner Setting, Outer Setting) used to structure 13 qualitative interviews on the relevant factors mediating implementation of automated decision support tools for genomic data access management and sharing among publicly funded genomic data repositories worldwide

The interview guide was developed specifically for this study and is available in Supplementary Materials 2 .

Data collection

We conducted a total of 13 semi-structured interviews with 17 DAC members between 27 April and 24 August 2022. Prospective interviewees indicated their interest in being invited to a follow up interview following their participation in a previous survey published elsewhere [ 9 ]. All interviews were conducted virtually and audio/video recorded on Zoom. We used validated interview guides from the official CFIR instrument repository ( https://cfirguide.org/evaluation-design/qualitative-data/ ) to probe the barriers and opportunities of implementing ADS solutions for DAC review of data access requests. Interviews lasted between 45 and 60 min and included 29 questions adapted from the CFIR instrument to fit the ADS context (e.g. Inner Setting, Outer Setting, Intervention Characteristics etc.). The specific interview guide used is available in Supplementary Materials 2 . Interviewees were also recruited from the Data Access Committee Review Standards Working Group (DACReS WG) chaired by authors VR, JL, and ESD, as well as from an internet search of publicly funded genomic data repositories worldwide.

Data analysis

We first applied a deductive coding frame to the interview transcripts based on a framework analysis approach (Pope, Ziebland, and Mays 2000) and the publicly accessible CFIR codebook available in the Supplemental Materials 1 . To ensure the reliability of conclusions drawn, two independent reviewers (VR and JB) tested the coding schema on three transcripts until reaching a recommended interrater reliability score of 0.83 before analyzing the remaining qualitative dataset. All coding discrepancies during the coding pilot were resolved by consensus discussion.

Geographical, Institutional, and Demographic Background of Participants

41% of interviewees worked within U.S.-based DACs, while the remaining 59% of interviewees represented DACs at institutions in Canada, the U.K., Spain, Tunisia, Australia, and Japan (Table  1 ). Nearly 60% of interviewees worked at a non-profit research institute, 24% represented an academic-affiliated research institution, 12% represented a government research agency, and 6% were affiliated with a research consortium. 76% of interview participants identified as female, and 24% as male.

Opportunities for ADS

We categorized the frequency of CFIR implementation factors referenced in our interviews in Table  2 . Our findings suggest that there are three major facilitators to implementing ADS for genomic data governance: (1) external policy and need for efficient workflows, (2) institutional ability to scale the ADS, and (3) interoperability.

External policy and need for efficient workflows

Participants considered adopting ADS to comply with new data sharing mandates from research funders (e.g. National Institutes of Health) and those imposed by peer reviewed journals. The demand for and scope of compliant data access review has had a ripple effect on ethics oversight bodies [ 26 ], including DACs, as a result of these new requirements [ 9 ]. Most DAC members we engaged with currently perform their reviews manually. Members review all data access requests individually or as a committee and make decisions on each request received in the order they were received. Given the anticipated increase in the number of data access requests [ 27 ], our participants noted the reduced workload and costs associated with ADS could contribute to better review efficiencies, without a concomitant loss in review quality and risk of noncompliance with data use conditions.

We found that participants perceived that ADS could reduce DAC member workload by streamlining the intake process for data access requests and verifying that the request matched the terms of use in the original consent obtained at data collection. Indeed, participants noted the initial screening of Data Access Requests (DARs) was a common rate-limiting step in the submission to decision process. DACs often begin the review process by verifying that all necessary information is documented in the request (e.g. study purpose, datasets requested, ethics review). This step can be time-consuming because the requirements can vary depending on the researcher’s institution and the datasets they request. We requested that participants share a copy of their DAR form before, during, or after the interview to compare what information DACs typically required to process a DAR. We found the form fields as well as length of the DAR (from 3 to 18 pages) differed considerably. Our participants believed that this is where ADS could be useful by automatically flagging missing information and documents, verifying the authenticity of a requester’s identity and the submitted documents, and then sending notifications to requesters if more information is needed. As one interviewee put it:

Because one of the biggest concerns in our DAC is that sometimes it takes too much time to be read by all the nine members. … They’re institutional directors or university professors. So I think it will help. Maybe if you have 50% of the work done by an automated system, so you just have to do the 50%. I think … this will be a good motivation for them saying ‘OK’ [to implement ADS].  ‑ Participant M.

Scalability and cost effectiveness

Participants also believed ADS-enabled workflows could be scalable, cost-effective solutions to management of not just newly generated data, but also for legacy data when grant funding ends because ADS can easily store and quickly present data use conditions and audit past DAC reviews. Two interviewees discussed the challenges of finding cost effective solutions to managing legacy datasets:

Actually there are lots of costs related to data sharing, particularly if I’m sharing data from the 1990s, for example. I don’t have any money or budget anymore to prepare the data [for secondary uses]. … And similarly, when it comes to these reports [on data sharing activities], there’s no extra money for doing the work to create those reports. But we’re having to report back over assets from years, decades in fact. And there was always just a little bit of a hint ‘oh well, maybe we’ll find some money’. No, no, you have to find it out on your own.  ‑ Participant F. I mean potentially as we grow over the years, you know what’s going to happen. … we’ve also discussed some scenarios, where, for example, we find ourselves with a larger amount of requests coming in, [and] we only accept applications up to certain days and then, we open this next quarter, close it again. But there potentially could be room for automation depending on the increase in request in the coming years.  ‑ Participant A.

Retention and sustainability of human resources

Participants also discussed retention of repository staff and DAC membership as an evolving human resource factor that would motivate ADS adoption. For example, some participants shared that ADS could be helpful when DAC members or data generators leave the institution, disrupting review continuity and consistency. Unlike for large, well-funded government repositories, many DACs at smaller institutions lack human resources to ensure long-term data preservation and access management for data of increasing complexity and volume:

As the program scales, the participant diversity scales, the data diversity scales. I think it is almost impossible to see a scenario where we do not rely on some level of automation to support human decision making about what is responsible use.  ‑ Participant J.

Interoperability

According to the DAC members we interviewed, ADS tools could provide centralized, interoperable solutions to facilitate inter-organizational and international data sharing. Participants perceived that ADS could motivate use of standardized request forms, access agreements, dataset identifiers, and methods for verifying researcher identities. For example, one participant commented:

But this [ADS] will free up a lot of time in the process is it also potentially means that it will become easier for, if you’re working in a team to hand off tasks as well because you will have a single system. … Also, consistency between organizations. If we have multiple organizations take this up, it’s going to mean less lead time. [Let’s] say people take a new job in a new place. We’ll actually have some software that people will recognize and be able to use and uptake, which we’ve been trying to go towards without ethics approval processes within the hospital and health services… [standardized] systems makes it easier for actual communication between organizations on processes, because everyone kind of begins to know what’s happening.  ‑ Participant E.

(b) Barriers to implementing ADS .

Despite clear advantages of ADS for genomic data access management, our interviewees identified significant barriers to implementation within DAC workflows, including: (1) lower priority compared to more immediate governance challenges, (2) ill equipped personnel and structures within the institution, (3) costs, and (4) degree of human oversight.

Prioritization

Many participants reported that institutional leadership prioritized other competing research data needs over investing in new data governance structures (e.g. generating quality data, increasing diversity in datasets, collaborating with underrepresented groups of researchers and participants, and releasing datasets). Participants believed researchers in general understand why quality and effective review of data access is important for responsible genomic data sharing but are firstly concerned with data quality. Another suspected reason that ADS implementation ranked lower on institutional priorities was that there had not yet been a significant data incident. As one participant put it:

I don’t think that the program thinks it is a very high priority to streamline any of the [data access oversight] process. I think that it will either take something bad happening and then realizing that we need additional capacities on [DAC], or some other hiccup to really promote that need.  ‑ Participant O.

Because budgets for data governance are not always included in grants, researchers may be less motivated to invest in the additional, largely unpaid work related to data governance. Insufficient resourcing for data sharing and governance mechanisms prospectively in research study design inevitably challenge the downstream execution of data governance upon deposit of the research data once generated, according to at least one DAC member we interviewed:

We found that some people don’t prioritize [data governance] because it’s not helpful to them, because it’s not our primary function as a department. You know, we’re producing new data. That’s usually what people, researchers are doing. They’re not thinking about what happens to their old data. So, it’s not much of a priority. Having said that, research funders are getting very keen for us to use their data. So, there is that sort of tug [of war]. … If I go into a senior team meeting, you know, something else will be the priority.  ‑ Participant F.

Structural characteristics of an organization

We also found a close correlation between several structural characteristics of the institution (e.g. years in operation, number of personnel, and database size) and participants’ perceived barriers to ADS implementation. For instance, many participants served on DACs that were established within the last 1–3 years coinciding with the creation of the institution’s database. As the datasets grow, and more researchers are attracted to the resource, there is greater potential to overwhelm existing management processes. It is precisely at this early juncture that DACs would benefit from weighing their ADS options, and proactively address relevant barriers ahead of any plans for implementation. Some DAC members preferred to gain more experience with existing data access management in these early years of data release before integrating ADS “because we’re not sure how [name of participant’s country] citizens feel or consider about the automatic decision on data sharing.” Participant K.

While cost was not a primary concern for ADS implementation at well-funded big data repositories, it was a significant barrier for DAC members working at smaller repositories, individual research departments, or research programs associated with a genomics consortium who were more often supported by research grants or contracts rather than an independent funding source.

“We [data governance office] are supported through project-specific funding. … Governance ends up being a little bit of this indirectly supported component of our work and services. That has limited the ways in which we can innovate around governance. … We don’t have a huge budget.”  ‑ Participant N.

Without dedicated budget for human and material resources, some DAC members were concerned that the initial investment in ADS and significant changes to current workflows would be key issues, to say nothing of new education and training materials and updates to internal policies, among other ancillary revisions to internal workflows.

Lack of human oversight

While some DAC members were enthusiastic about improvements in efficiency and consistency of ADS, participants unanimously rejected the idea of fully automating access management: “no matter what we do with automation that I feel there always needs to be that human element who’s coming in and checking. So, there will always be that barrier to upscaling” Participant E. Other participants emphasized that prior to implementation, they would need to gauge how research participants at their own institution as well as the general public would react to ADS for data access review.

Participants were also skeptical that ADS could adequately assess complex, sensitive data reuse issues which they felt required a deep understanding of ethical, legal, and sociocultural contexts within which data were collected, used, and shared. Some DAC members reported asking data requesters to clarify their study purpose and justify their need for specific datasets in recognition of these sociocultural dimensions.

I’m also someone who thinks that it’s important to be very critical about what’s the nature of the work being done. Maybe it’s solid from a scientific point of view. But are there other concerns from other perspectives that need to be taken into account? That is partly why we have community members on the [committee], and that’s something I’m not sure can be simplified or automated.”

However, when it comes to automating anything that requires reviewing information where there might be a lot of nuances, where there might be a lot of interpretation that’s required, I’m a little bit more hesitant simply because I think to some extent you do need some room for a little bit of mulling over the information, … and I think there are some information that come through with requests, that don’t neatly fit into check boxes.  ‑ Participant B.

Overall, participants perceived that ADS tools could be well positioned to help DACs streamline data access compliance. While believed to beneficial, ADS solutions were unlikely to immediately or directly advance the research organization’s core mission (e.g. collecting quality data and driving scientific discoveries and innovations). One of the most challenging barriers to implementation is the relative low priority of, and lack of institutional investment in, data infrastructures that could adapt as the dynamics of genomic data generation and storage change over time. Participants tended to regard ADS implementation, as well as data governance workflow solutions, as a lower priority compared to regulatory compliance, investigator support, and database curation, among other competing demands on DAC member time.

Most research grants allow investigators to apply for support for data collection and analysis, but rarely establish actual governance structures needed to stand up access management services. We found that executive buy-in was a major driver for ADS support in the cases of some repositories and the lack or administrative or leadership buy in a major detractor for others, namely repositories at smaller research institutions or laboratories. Therefore, part of the challenge of making ADS adoption a higher institutional priority is convincing institutional leadership of their added value and the net benefit of investing in data governance solutions and infrastructures generally.

Delaying infrastructure upgrades has consequences for the future utility of the repository in the longer term. Some of our study participants, for example, believed researchers were drawn to their databases not because of their data access policies and practices, but because of the quality and diversity of their datasets. However, this quality-driven perspective contrasts with findings from a study of genetic researchers suggesting that ease of access is at least marginally important when choosing a database for their research [ 28 ]. We reason that repositories which invest in efficient, scalable, and compliant access decision processes are likely to attract more users to their resources than repositories which do not evolve such processes to meet the pace of data generation and higher data demand. It is also worth noting that funders have a direct role to play in accelerating the pace of data science as researchers are expected to do more with fewer resources and in less time.

Developing more streamlined workflows emerged as a primary benefit that many participants anticipated from adopting ADS. Participants were most enthusiastic about applying ADS for time consuming and tedious tasks, such as preliminary review and quality control checks for data access request forms that are needed to initiate the data access decision process. Applying ADS to facilitate these workflows could free DAC members to dedicate more time to deliberate on more substantive ethics issues raised by data access requests.

While data governance has often been considered auxiliary work, new research findings and new U.S. federal government policies, such as the National Institutes of Health Data Management and Sharing (DMS) Policy, have elevated its importance by placing additional requirements for data sharing [ 29 ]. The new DMS Policy was but one example of distinct legislative reforms that have influenced cultures of data sharing shaping DAC work, as well as the institutional practices and governance tools developed to complement this culture. To be sure, such legislative and institutional context influenced participant responses and particular implementation preparedness factors for ADS such as “structural characteristics of the organization.”

The DMS Policy will accelerate the accumulation of an enormous number of datasets. In the absence of interventions, including but not limited to ADS, the DMS Policy will significantly raise costs associated with data storage and management. We concluded from our participants that databases/repositories are frequently developed specifically to share research data generated from federal funds without attention to existing databases and other resources in mind within which to deposit their data. “Blind” database creation is often done with good intentions; however, it can inadvertently introduce myriad access pathways that make the data effectively “shared” but undiscoverable and is another issue where ADS tools could intervene. One participant’s narrative about their need to transfer legacy data from a repository facing permanent closure puts the problem of unsustainable databases in sharp relief. The participant’s example suggested that there is need for more efficient and sustainable solutions for data access management and sharing that can endure even when repositories themselves do not. Moreover, there is reasonable cause to have a contingency plan for publicly funded data shared via non-publicly supported repositories in the event the repository closes or changes in policy or personnel. Standardized ADS solutions could easily interoperate between the two types of repositories and facilitate legacy data transfer, if and when required.

Limitations

Our results should be considered in light of several methodological limitations. While geographically diverse, many of our interview participants were affiliated with DACs based at large, well-resourced research institutions. It is likely that responses and perceptions of implementation factors related to ADS would differ substantially if more DACs from low- or under-resourced institutions were represented in our sample. Our data collection design relies on self-reports of institutional data access policy and procedures. Many interview participants were aware of the Global Alliance for Genomics and Health, and the data access committee review standards we were principally involved in developing [ 30 ]. Thus, while we endeavored to create a safe, open environment for participants to share their honest views, social desirability bias related to our prior work may have influenced how participants responded. Lastly, CFIR predefines sociological constructs relevant to implementation. Our analysis was therefore limited only to those constructs covered in the framework, whereas others might have emerged inductively if we adopted an alternative analytic frame.

In this article, we reported findings from semi-structured qualitative interviews with DAC members from around the world on the relevant barriers and facilitators of implementing ADS for genomic data access management. Our findings suggest there is general support for pilot studies that test ADS performance for certain tasks in data access management workflows, such as cataloging data types, verifying user credentials, and tagging datasets for use terms. Participants indicated that ADS should supplement, but not replace, DAC member work. This sentiment was especially strong with respect to tasks that were perceived to require sensitivity and human value-judgments such as privacy protections, group harms, and study purpose. Nonetheless, our findings offer cautious optimism regarding the ways in which algorithms, software, and other machine-readable ontologies could streamline aspects of DAC decision-making while also enabling new opportunities for improving consistency and fairness in DAC decisions.

To that end, we conclude with practical recommendations for institutional data stewards that are considering or have already implemented ADS for data access management. First, repositories and institutions that support databases and other resources should prioritize infrastructural upgrades and factor them into associated budgets. Ensuring proper investment in, and human/material resource support for, these upgrades ensures the repository can help ensure its utility even as the complexity and volume of genomic and associated health datasets grow. Second, DACs should prepare to put in place today what data access management and sharing processes they foresee the repository needing tomorrow. For DAC members looking to integrate ADS or other semi-automated tools into their workflows, buy-in from executive leadership should be obtained at the earliest stages of this transition. DAC members should consider substantiating the need for semi/automated solutions with concrete trend data about the frequency of data access requests relative to the time from request to decision and extrapolate these numbers to judge what the anticipated demand for repository will be in 1, 5, and 10 years. Tracking and transparently reporting data access request volume, access decisions, and other committee operations is likewise important not just for internal purposes, but also to demonstrate responsible data stewardship in action to prospective data contributors.

Third, DACs should refrain from implementing ADS wholesale without complementary human oversight of data access request intake and decisions. Pilot testing where ADS tools can be applied to the most time-consuming tasks will require taking inventory of the inputs required for each task along the data access decision workflow. Fourth, DACs should consider what human and material resources will be needed to integrate ADS effectively. These resources include DAC member expertise, computer equipment, and software development, not to mention member education and training resources. Finally, DACs should collaborate on setting standards for how data access requests should be adjudicated and tailor ADS tools in line with these consensus criteria. There is ongoing work to this effect as part of the Ethical Provenance Subgroup of the Global Alliance for Genomics and Health (including the development of an "Ethical Provenance Toolkit"); additional representation from repositories that steward other diverse health datasets would be ideal to coordinate access management strategies across the field.

The explosion in the volume and complexity of genomic and associated health data is converging with the need to manage access more efficiently to these data. Such trends point intuitively to solutions that can help to alleviate, or at least prevent bottlenecks in the access process to preserve the scientific and social value of data generated from public investments in research. To put ADS solutions to the test, future research should compare access decisions and their outcomes between institutions who do/not use such tools for data access management; and examine whether ADS delivers on its efficiency promises and whether it liberates DAC member time previously spent addressing procedural matters – allowing more opportunities for committee deliberation on substantive ethics issues.

Data availability

Materials described in the manuscript and data supporting our findings can be made available upon request. All requests should be directed to Vasiliki Rahimzadeh, PhD at [email protected].

Stephens ZD, Lee SY, Faghri F, Campbell RH, Zhai C, Efron MJ, Iyer R, Schatz MC, Sinha S, Robinson GE. Big Data: Astronomical or Genomical? PLoS Biol. 2015;13:e1002195. https://doi.org/10.1371/journal.pbio.1002195 .

Article   Google Scholar  

IQVIA Institute for Human Data. Science understanding the Global Landscape of genomic initiatives: Progress and Promise.

Cook-Deegan R, McGuire AL. Moving beyond Bermuda: sharing data to build a medical information commons. Genome Res. 2017;27:897–901. https://doi.org/10.1101/gr.216911.116 .

Evans B. (2019). Genomic data commons. Governing Privacy in Knowledge Commons, 75–105.

Contreras JL, Knoppers BM. The genomic commons. Annu Rev Genom Hum Genet. 2018;19:429–53. https://doi.org/10.1146/annurev-genom-083117-021552 .

Grossman RL. Ten lessons for data sharing with a data commons. Sci Data. 2023;10:120. https://doi.org/10.1038/s41597-023-02029-x .

Cheah PY, Piasecki J. Data Access committees. BMC Med Ethics. 2020;21:1–8. https://doi.org/10.1186/s12910-020-0453-z .

Murtagh MJ, Blell MT, Butters OW, Cowley L, Dove ES, Goodman A, Griggs RL, Hall A, Hallowell N, Kumari M, et al. Better governance, better access: practising responsible data sharing in the METADAC governance infrastructure. Hum Genomics. 2018;12. https://doi.org/10.1186/s40246-018-0154-6 .

Lawson J, Rahimzadeh V, Baek J, Dove ES. Achieving Procedural Parity in Managing Access to genomic and related Health data: A Global Survey of Data Access Committee members. Biopreserv Biobank. 2023;bio20220205. https://doi.org/10.1089/bio.2022.0205 .

Shabani M, Thorogood A, Borry P. Who should have access to genomic data and how should they be held accountable? Perspectives of Data Access Committee members and experts. Eur J Hum Genet. 2016;24:1671–5. https://doi.org/10.1038/ejhg.2016.111 .

Ramos EM, Din-Lovinescu C, Bookman EB, McNeil LJ, Baker CC, Godynskiy G, Harris EL, Lehner T, McKeon C, Moss J, A Mechanism for Controlled Access to GWAS Data: Experience of the GAIN Data Access Committee. Am J Hum Genet. 2013;92:479–88. https://doi.org/10.1016/j.ajhg.2012.08.034 .

Shabani M, Knoppers BM, Borry P. From the principles of genomic data sharing to the practices of data access committees. EMBO Mol Med. 2015;7:507–9. https://doi.org/10.15252/emmm.201405002 .

Mello MM, Triantis G, Stanton R, Blumenkranz E, Studdert DM. Waiting for data: barriers to executing data use agreements. Science. 2020;367:150–2. https://doi.org/10.1126/science.aaz7028 .

Lawson J, Cabili MN, Kerry G, Boughtwood T, Thorogood A, Alper P, Bowers SR, Boyles RR, Brookes AJ, Brush M, et al. The Data Use Ontology to streamline responsible access to human biomedical datasets. Cell Genomics. 2021;1:100028. https://doi.org/10.1016/j.xgen.2021.100028 .

Lawson J. Empirical validation of an Automated Approach to Data Use Oversight. Cell Genomics Forthcoming; 2021.

Van der Auwera G, O’Connor B. Genomics in the Cloud: using Docker, GATK, and WDL in Terra. 1st ed. O’Reilly Media; 2020.

Birger C, Hanna M, Salinas E, Neff J, Saksena G, Livitz D, Rosebrock D, Stewart C, Leshchiner I, Baumann A, et al. FireCloud, a scalable cloud-based platform for collaborative genome analysis: strategies for reducing and controlling costs. (Bioinformatics). 2017. https://doi.org/10.1101/209494 .

Rahimzadeh V, Lawson J, Rushton G, Dove ES. Leveraging algorithms to improve decision-making workflows for genomic data Access and Management. Biopreserv Biobank. 2022. https://doi.org/10.1089/bio.2022.0042 .

Data Use Ontology. (2022).

Lawson J, Ghanaim EM, Baek J, Lee H, Rehm HL. Aligning NIH’s existing data use restrictions to the GA4GH DUO standard. Cell Genomics. 2023;3:100381. https://doi.org/10.1016/j.xgen.2023.100381 .

Rahimzadeh V, Lawson J, Baek J, Dove ES. Automating justice: an ethical responsibility of computational Bioethics. Am J Bioeth. 2022;22:30–3. https://doi.org/10.1080/15265161.2022.2075051 .

Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366:447–53. https://doi.org/10.1126/science.aax2342 .

Hartzog W, Conti G, Nelson J, Shay LA. Inefficiently Automated Law Enforcement. Mich State Law Rev. 2016;1763:1763–96.

Google Scholar  

Ledford H. Millions affected by racial bias in healthcare algorithm. Nature. 2019;574:608–10.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4. https://doi.org/10.1186/1748-5908-4-50 .

Rahimzadeh V, Serpico K, Gelinas L. Institutional review boards need new skills to review data sharing and management plans. Nat Med. 2023;1–3. https://doi.org/10.1038/s41591-023-02292-w .

Cabili MN, Lawson J, Saltzman A, Rushton G, O’Rourke P, Wilbanks J, Rodriguez LL, Nyronen T, Courtot M, Donnelly S, et al. Empirical validation of an automated approach to data use oversight. Cell Genomics. 2021;1:100031. https://doi.org/10.1016/j.xgen.2021.100031 .

Trinidad MG, Ryan KA, Krenz CD, Roberts JS, McGuire AL, De Vries R, Zikmund-Fisher BJ, Kardia S, Marsh E, Forman J, et al. Extremely slow and capricious: a qualitative exploration of genetic researcher priorities in selecting shared data resources. Genet Sci. 2023;25:115–24. https://doi.org/10.1016/j.gim.2022.09.003 .

Final NIH. Policy for Data Management and Sharing https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html .

Global Alliance for Genomics and Health. (2021). Data Access Committee Guiding Principles and Procedural Standards Policy.

Download references

Acknowledgements

The authors wish to thank members of the Data Access Committee Review Standards Working Group, and the Regulatory and Ethics Work Stream of the Global Alliance for Genomics and Health for their contributions to the intellectual community that inspired this work.

This study was funded by the National Human Genome Research Institute as an Administrative Supplement grant to the AnVIL program for the Study of Bioethical Issues [U24HGO10262].

Author information

Authors and affiliations.

Center for Medical Ethics and Health Policy, Baylor College of Medicine, 1 Baylor Plaza, Suite 310DF, Houston, TX, 77098, USA

Vasiliki Rahimzadeh

Broad Institute of MIT and Harvard, Cambridge, MA, USA

Jinyoung Baek & Jonathan Lawson

School of Law, University of Edinburgh, Edinburgh, UK

Edward S. Dove

You can also search for this author in PubMed   Google Scholar

Contributions

Authors VR, JL and ED conceptualized, designed, and carried out the study. Author JB led in the data collection and analysis and drafting of early manuscript drafts. All authors, VR, JB, JL and ED took part in writing and editing the manuscript, responding to peer reviewer comments and approved the final version.

Corresponding author

Correspondence to Vasiliki Rahimzadeh .

Ethics declarations

Ethical approval.

This study was reviewed and approved the Stanford University Institutional Review Board. All participants were informed of the purpose of the study, funding, risks and benefits at the time of invitation. Informed consent was obtained from all participants prior to the interview and participants were provided opportunities to ask any questions about the study procedures.

Consent for publication

Not applicable.

Competing interests

All authors are members of the Regulatory and Ethics Work Stream of the Global Alliance for Genomics and Health. JL is co-lead of the Data Use Ontology, leads the Data Use Oversight System and is a member of the Broad Institute Data Access Committee.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rahimzadeh, V., Baek, J., Lawson, J. et al. A qualitative interview study to determine barriers and facilitators of implementing automated decision support tools for genomic data access. BMC Med Ethics 25 , 51 (2024). https://doi.org/10.1186/s12910-024-01050-y

Download citation

Received : 09 January 2024

Accepted : 26 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1186/s12910-024-01050-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Data access committee
• Implementation science
• Decision support
• Genomic data

BMC Medical Ethics

ISSN: 1472-6939

• General enquiries: [email protected]