what should be included in the literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 
• How to write a good literature review 
• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

• Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 
• Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 
• Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 
• Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 
• Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 
• Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

How to write a good literature review

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. 

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• Life Sciences Papers: 9 Tips for Authors Writing in Biological Sciences
• What is an Argumentative Essay? How to Write It (With Examples)

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, what is hedging in academic writing  , how to use ai to enhance your college..., ai + human expertise – a paradigm shift..., how to use paperpal to generate emails &..., ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., do plagiarism checkers detect ai content, word choice problems: how to use the right..., how to avoid plagiarism when using generative ai..., what are journal guidelines on using generative ai....

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

How To Write A Literature Review - A Complete Guide

Table of Contents

A literature review is much more than just another section in your research paper. It forms the very foundation of your research. It is a formal piece of writing where you analyze the existing theoretical framework, principles, and assumptions and use that as a base to shape your approach to the research question.

Curating and drafting a solid literature review section not only lends more credibility to your research paper but also makes your research tighter and better focused. But, writing literature reviews is a difficult task. It requires extensive reading, plus you have to consider market trends and technological and political changes, which tend to change in the blink of an eye.

Now streamline your literature review process with the help of SciSpace Copilot. With this AI research assistant, you can efficiently synthesize and analyze a vast amount of information, identify key themes and trends, and uncover gaps in the existing research. Get real-time explanations, summaries, and answers to your questions for the paper you're reviewing, making navigating and understanding the complex literature landscape easier.

In this comprehensive guide, we will explore everything from the definition of a literature review, its appropriate length, various types of literature reviews, and how to write one.

What is a literature review?

A literature review is a collation of survey, research, critical evaluation, and assessment of the existing literature in a preferred domain.

Eminent researcher and academic Arlene Fink, in her book Conducting Research Literature Reviews , defines it as the following:

“A literature review surveys books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated.

Literature reviews are designed to provide an overview of sources you have explored while researching a particular topic, and to demonstrate to your readers how your research fits within a larger field of study.”

Simply put, a literature review can be defined as a critical discussion of relevant pre-existing research around your research question and carving out a definitive place for your study in the existing body of knowledge. Literature reviews can be presented in multiple ways: a section of an article, the whole research paper itself, or a chapter of your thesis.

A literature review does function as a summary of sources, but it also allows you to analyze further, interpret, and examine the stated theories, methods, viewpoints, and, of course, the gaps in the existing content.

As an author, you can discuss and interpret the research question and its various aspects and debate your adopted methods to support the claim.

What is the purpose of a literature review?

A literature review is meant to help your readers understand the relevance of your research question and where it fits within the existing body of knowledge. As a researcher, you should use it to set the context, build your argument, and establish the need for your study.

What is the importance of a literature review?

The literature review is a critical part of research papers because it helps you:

• Gain an in-depth understanding of your research question and the surrounding area
• Convey that you have a thorough understanding of your research area and are up-to-date with the latest changes and advancements
• Establish how your research is connected or builds on the existing body of knowledge and how it could contribute to further research
• Elaborate on the validity and suitability of your theoretical framework and research methodology
• Identify and highlight gaps and shortcomings in the existing body of knowledge and how things need to change
• Convey to readers how your study is different or how it contributes to the research area

How long should a literature review be?

Ideally, the literature review should take up 15%-40% of the total length of your manuscript. So, if you have a 10,000-word research paper, the minimum word count could be 1500.

Your literature review format depends heavily on the kind of manuscript you are writing — an entire chapter in case of doctoral theses, a part of the introductory section in a research article, to a full-fledged review article that examines the previously published research on a topic.

Another determining factor is the type of research you are doing. The literature review section tends to be longer for secondary research projects than primary research projects.

What are the different types of literature reviews?

All literature reviews are not the same. There are a variety of possible approaches that you can take. It all depends on the type of research you are pursuing.

Here are the different types of literature reviews:

Argumentative review

It is called an argumentative review when you carefully present literature that only supports or counters a specific argument or premise to establish a viewpoint.

Integrative review

It is a type of literature review focused on building a comprehensive understanding of a topic by combining available theoretical frameworks and empirical evidence.

Methodological review

This approach delves into the ''how'' and the ''what" of the research question —  you cannot look at the outcome in isolation; you should also review the methodology used.

Systematic review

This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research and collect, report, and analyze data from the studies included in the review.

Meta-analysis review

Meta-analysis uses statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects than those derived from the individual studies included within a review.

Historical review

Historical literature reviews focus on examining research throughout a period, often starting with the first time an issue, concept, theory, or phenomenon emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and identify future research's likely directions.

Theoretical Review

This form aims to examine the corpus of theory accumulated regarding an issue, concept, theory, and phenomenon. The theoretical literature review helps to establish what theories exist, the relationships between them, the degree the existing approaches have been investigated, and to develop new hypotheses to be tested.

Scoping Review

The Scoping Review is often used at the beginning of an article, dissertation, or research proposal. It is conducted before the research to highlight gaps in the existing body of knowledge and explains why the project should be greenlit.

State-of-the-Art Review

The State-of-the-Art review is conducted periodically, focusing on the most recent research. It describes what is currently known, understood, or agreed upon regarding the research topic and highlights where there are still disagreements.

Can you use the first person in a literature review?

When writing literature reviews, you should avoid the usage of first-person pronouns. It means that instead of "I argue that" or "we argue that," the appropriate expression would be "this research paper argues that."

Do you need an abstract for a literature review?

Ideally, yes. It is always good to have a condensed summary that is self-contained and independent of the rest of your review. As for how to draft one, you can follow the same fundamental idea when preparing an abstract for a literature review. It should also include:

• The research topic and your motivation behind selecting it
• A one-sentence thesis statement
• An explanation of the kinds of literature featured in the review
• Summary of what you've learned
• Conclusions you drew from the literature you reviewed
• Potential implications and future scope for research

Here's an example of the abstract of a literature review

Is a literature review written in the past tense?

Yes, the literature review should ideally be written in the past tense. You should not use the present or future tense when writing one. The exceptions are when you have statements describing events that happened earlier than the literature you are reviewing or events that are currently occurring; then, you can use the past perfect or present perfect tenses.

How many sources for a literature review?

There are multiple approaches to deciding how many sources to include in a literature review section. The first approach would be to look level you are at as a researcher. For instance, a doctoral thesis might need 60+ sources. In contrast, you might only need to refer to 5-15 sources at the undergraduate level.

The second approach is based on the kind of literature review you are doing — whether it is merely a chapter of your paper or if it is a self-contained paper in itself. When it is just a chapter, sources should equal the total number of pages in your article's body. In the second scenario, you need at least three times as many sources as there are pages in your work.

Quick tips on how to write a literature review

To know how to write a literature review, you must clearly understand its impact and role in establishing your work as substantive research material.

You need to follow the below-mentioned steps, to write a literature review:

• Outline the purpose behind the literature review
• Search relevant literature
• Examine and assess the relevant resources
• Discover connections by drawing deep insights from the resources
• Structure planning to write a good literature review

1. Outline and identify the purpose of  a literature review

As a first step on how to write a literature review, you must know what the research question or topic is and what shape you want your literature review to take. Ensure you understand the research topic inside out, or else seek clarifications. You must be able to the answer below questions before you start:

• How many sources do I need to include?
• What kind of sources should I analyze?
• How much should I critically evaluate each source?
• Should I summarize, synthesize or offer a critique of the sources?
• Do I need to include any background information or definitions?

Additionally, you should know that the narrower your research topic is, the swifter it will be for you to restrict the number of sources to be analyzed.

2. Search relevant literature

Dig deeper into search engines to discover what has already been published around your chosen topic. Make sure you thoroughly go through appropriate reference sources like books, reports, journal articles, government docs, and web-based resources.

You must prepare a list of keywords and their different variations. You can start your search from any library’s catalog, provided you are an active member of that institution. The exact keywords can be extended to widen your research over other databases and academic search engines like:

• Google Scholar
• Microsoft Academic
• Science.gov

Besides, it is not advisable to go through every resource word by word. Alternatively, what you can do is you can start by reading the abstract and then decide whether that source is relevant to your research or not.

Additionally, you must spend surplus time assessing the quality and relevance of resources. It would help if you tried preparing a list of citations to ensure that there lies no repetition of authors, publications, or articles in the literature review.

3. Examine and assess the sources

It is nearly impossible for you to go through every detail in the research article. So rather than trying to fetch every detail, you have to analyze and decide which research sources resemble closest and appear relevant to your chosen domain.

While analyzing the sources, you should look to find out answers to questions like:

• What question or problem has the author been describing and debating?
• What is the definition of critical aspects?
• How well the theories, approach, and methodology have been explained?
• Whether the research theory used some conventional or new innovative approach?
• How relevant are the key findings of the work?
• In what ways does it relate to other sources on the same topic?
• What challenges does this research paper pose to the existing theory
• What are the possible contributions or benefits it adds to the subject domain?

Be always mindful that you refer only to credible and authentic resources. It would be best if you always take references from different publications to validate your theory.

Always keep track of important information or data you can present in your literature review right from the beginning. It will help steer your path from any threats of plagiarism and also make it easier to curate an annotated bibliography or reference section.

4. Discover connections

At this stage, you must start deciding on the argument and structure of your literature review. To accomplish this, you must discover and identify the relations and connections between various resources while drafting your abstract.

A few aspects that you should be aware of while writing a literature review include:

• Rise to prominence: Theories and methods that have gained reputation and supporters over time.
• Constant scrutiny: Concepts or theories that repeatedly went under examination.
• Contradictions and conflicts: Theories, both the supporting and the contradictory ones, for the research topic.
• Knowledge gaps: What exactly does it fail to address, and how to bridge them with further research?
• Influential resources: Significant research projects available that have been upheld as milestones or perhaps, something that can modify the current trends

Once you join the dots between various past research works, it will be easier for you to draw a conclusion and identify your contribution to the existing knowledge base.

5. Structure planning to write a good literature review

There exist different ways towards planning and executing the structure of a literature review. The format of a literature review varies and depends upon the length of the research.

Like any other research paper, the literature review format must contain three sections: introduction, body, and conclusion. The goals and objectives of the research question determine what goes inside these three sections.

Nevertheless, a good literature review can be structured according to the chronological, thematic, methodological, or theoretical framework approach.

Literature review samples

1. Standalone

2. As a section of a research paper

How SciSpace Discover makes literature review a breeze?

SciSpace Discover is a one-stop solution to do an effective literature search and get barrier-free access to scientific knowledge. It is an excellent repository where you can find millions of only peer-reviewed articles and full-text PDF files. Here’s more on how you can use it:

Find the right information

Find what you want quickly and easily with comprehensive search filters that let you narrow down papers according to PDF availability, year of publishing, document type, and affiliated institution. Moreover, you can sort the results based on the publishing date, citation count, and relevance.

Assess credibility of papers quickly

When doing the literature review, it is critical to establish the quality of your sources. They form the foundation of your research. SciSpace Discover helps you assess the quality of a source by providing an overview of its references, citations, and performance metrics.

Get the complete picture in no time

SciSpace Discover’s personalized suggestion engine helps you stay on course and get the complete picture of the topic from one place. Every time you visit an article page, it provides you links to related papers. Besides that, it helps you understand what’s trending, who are the top authors, and who are the leading publishers on a topic.

Make referring sources super easy

To ensure you don't lose track of your sources, you must start noting down your references when doing the literature review. SciSpace Discover makes this step effortless. Click the 'cite' button on an article page, and you will receive preloaded citation text in multiple styles — all you've to do is copy-paste it into your manuscript.

Final tips on how to write a literature review

A massive chunk of time and effort is required to write a good literature review. But, if you go about it systematically, you'll be able to save a ton of time and build a solid foundation for your research.

We hope this guide has helped you answer several key questions you have about writing literature reviews.

Would you like to explore SciSpace Discover and kick off your literature search right away? You can get started here .

Frequently Asked Questions (FAQs)

1. how to start a literature review.

• What questions do you want to answer?

• What sources do you need to answer these questions?

• What information do these sources contain?

• How can you use this information to answer your questions?

2. What to include in a literature review?

• A brief background of the problem or issue

• What has previously been done to address the problem or issue

• A description of what you will do in your project

• How this study will contribute to research on the subject

3. Why literature review is important?

The literature review is an important part of any research project because it allows the writer to look at previous studies on a topic and determine existing gaps in the literature, as well as what has already been done. It will also help them to choose the most appropriate method for their own study.

4. How to cite a literature review in APA format?

To cite a literature review in APA style, you need to provide the author's name, the title of the article, and the year of publication. For example: Patel, A. B., & Stokes, G. S. (2012). The relationship between personality and intelligence: A meta-analysis of longitudinal research. Personality and Individual Differences, 53(1), 16-21

5. What are the components of a literature review?

• A brief introduction to the topic, including its background and context. The introduction should also include a rationale for why the study is being conducted and what it will accomplish.

• A description of the methodologies used in the study. This can include information about data collection methods, sample size, and statistical analyses.

• A presentation of the findings in an organized format that helps readers follow along with the author's conclusions.

6. What are common errors in writing literature review?

• Not spending enough time to critically evaluate the relevance of resources, observations and conclusions.

• Totally relying on secondary data while ignoring primary data.

• Letting your personal bias seep into your interpretation of existing literature.

• No detailed explanation of the procedure to discover and identify an appropriate literature review.

7. What are the 5 C's of writing literature review?

• Cite - the sources you utilized and referenced in your research.

• Compare - existing arguments, hypotheses, methodologies, and conclusions found in the knowledge base.

• Contrast - the arguments, topics, methodologies, approaches, and disputes that may be found in the literature.

• Critique - the literature and describe the ideas and opinions you find more convincing and why.

• Connect - the various studies you reviewed in your research.

8. How many sources should a literature review have?

When it is just a chapter, sources should equal the total number of pages in your article's body. if it is a self-contained paper in itself, you need at least three times as many sources as there are pages in your work.

9. Can literature review have diagrams?

• To represent an abstract idea or concept

• To explain the steps of a process or procedure

• To help readers understand the relationships between different concepts

10. How old should sources be in a literature review?

Sources for a literature review should be as current as possible or not older than ten years. The only exception to this rule is if you are reviewing a historical topic and need to use older sources.

11. What are the types of literature review?

• Argumentative review

• Integrative review

• Methodological review

• Systematic review

• Meta-analysis review

• Historical review

• Theoretical review

• Scoping review

• State-of-the-Art review

12. Is a literature review mandatory?

Yes. Literature review is a mandatory part of any research project. It is a critical step in the process that allows you to establish the scope of your research, and provide a background for the rest of your work.

But before you go,

• Six Online Tools for Easy Literature Review
• Evaluating literature review: systematic vs. scoping reviews
• Systematic Approaches to a Successful Literature Review
• Writing Integrative Literature Reviews: Guidelines and Examples

You might also like

Consensus GPT vs. SciSpace GPT: Choose the Best GPT for Research

Literature Review and Theoretical Framework: Understanding the Differences

Types of Essays in Academic Writing - Quick Guide (2024)

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

How To Structure Your Literature Review

3 options to help structure your chapter.

By: Amy Rommelspacher (PhD) | Reviewer: Dr Eunice Rautenbach | November 2020 (Updated May 2023)

Writing the literature review chapter can seem pretty daunting when you’re piecing together your dissertation or thesis. As  we’ve discussed before , a good literature review needs to achieve a few very important objectives – it should:

• Demonstrate your knowledge of the research topic
• Identify the gaps in the literature and show how your research links to these
• Provide the foundation for your conceptual framework (if you have one)
• Inform your own  methodology and research design

To achieve this, your literature review needs a well-thought-out structure . Get the structure of your literature review chapter wrong and you’ll struggle to achieve these objectives. Don’t worry though – in this post, we’ll look at how to structure your literature review for maximum impact (and marks!).

But wait – is this the right time?

Deciding on the structure of your literature review should come towards the end of the literature review process – after you have collected and digested the literature, but before you start writing the chapter. 

In other words, you need to first develop a rich understanding of the literature before you even attempt to map out a structure. There’s no use trying to develop a structure before you’ve fully wrapped your head around the existing research.

Equally importantly, you need to have a structure in place before you start writing , or your literature review will most likely end up a rambling, disjointed mess. 

Importantly, don’t feel that once you’ve defined a structure you can’t iterate on it. It’s perfectly natural to adjust as you engage in the writing process. As we’ve discussed before , writing is a way of developing your thinking, so it’s quite common for your thinking to change – and therefore, for your chapter structure to change – as you write. 

Need a helping hand?

Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components – an  introduction , a  body   and a  conclusion . 

Let’s take a closer look at each of these.

1: The Introduction Section

Just like any good introduction, the introduction section of your literature review should introduce the purpose and layout (organisation) of the chapter. In other words, your introduction needs to give the reader a taste of what’s to come, and how you’re going to lay that out. Essentially, you should provide the reader with a high-level roadmap of your chapter to give them a taste of the journey that lies ahead.

Here’s an example of the layout visualised in a literature review introduction:

Your introduction should also outline your topic (including any tricky terminology or jargon) and provide an explanation of the scope of your literature review – in other words, what you  will   and  won’t   be covering (the delimitations ). This helps ringfence your review and achieve a clear focus . The clearer and narrower your focus, the deeper you can dive into the topic (which is typically where the magic lies). 

Depending on the nature of your project, you could also present your stance or point of view at this stage. In other words, after grappling with the literature you’ll have an opinion about what the trends and concerns are in the field as well as what’s lacking. The introduction section can then present these ideas so that it is clear to examiners that you’re aware of how your research connects with existing knowledge .

2: The Body Section

The body of your literature review is the centre of your work. This is where you’ll present, analyse, evaluate and synthesise the existing research. In other words, this is where you’re going to earn (or lose) the most marks. Therefore, it’s important to carefully think about how you will organise your discussion to present it in a clear way. 

The body of your literature review should do just as the description of this chapter suggests. It should “review” the literature – in other words, identify, analyse, and synthesise it. So, when thinking about structuring your literature review, you need to think about which structural approach will provide the best “review” for your specific type of research and objectives (we’ll get to this shortly).

There are (broadly speaking)  three options  for organising your literature review.

Option 1: Chronological (according to date)

Organising the literature chronologically is one of the simplest ways to structure your literature review. You start with what was published first and work your way through the literature until you reach the work published most recently. Pretty straightforward.

The benefit of this option is that it makes it easy to discuss the developments and debates in the field as they emerged over time. Organising your literature chronologically also allows you to highlight how specific articles or pieces of work might have changed the course of the field – in other words, which research has had the most impact . Therefore, this approach is very useful when your research is aimed at understanding how the topic has unfolded over time and is often used by scholars in the field of history. That said, this approach can be utilised by anyone that wants to explore change over time .

For example , if a student of politics is investigating how the understanding of democracy has evolved over time, they could use the chronological approach to provide a narrative that demonstrates how this understanding has changed through the ages.

Here are some questions you can ask yourself to help you structure your literature review chronologically.

• What is the earliest literature published relating to this topic?
• How has the field changed over time? Why?
• What are the most recent discoveries/theories?

In some ways, chronology plays a part whichever way you decide to structure your literature review, because you will always, to a certain extent, be analysing how the literature has developed. However, with the chronological approach, the emphasis is very firmly on how the discussion has evolved over time , as opposed to how all the literature links together (which we’ll discuss next ).

Option 2: Thematic (grouped by theme)

The thematic approach to structuring a literature review means organising your literature by theme or category – for example, by independent variables (i.e. factors that have an impact on a specific outcome).

As you’ve been collecting and synthesising literature , you’ll likely have started seeing some themes or patterns emerging. You can then use these themes or patterns as a structure for your body discussion. The thematic approach is the most common approach and is useful for structuring literature reviews in most fields.

For example, if you were researching which factors contributed towards people trusting an organisation, you might find themes such as consumers’ perceptions of an organisation’s competence, benevolence and integrity. Structuring your literature review thematically would mean structuring your literature review’s body section to discuss each of these themes, one section at a time.

Here are some questions to ask yourself when structuring your literature review by themes:

• Are there any patterns that have come to light in the literature?
• What are the central themes and categories used by the researchers?
• Do I have enough evidence of these themes?

PS – you can see an example of a thematically structured literature review in our literature review sample walkthrough video here.

Option 3: Methodological

The methodological option is a way of structuring your literature review by the research methodologies used . In other words, organising your discussion based on the angle from which each piece of research was approached – for example, qualitative , quantitative or mixed  methodologies.

Structuring your literature review by methodology can be useful if you are drawing research from a variety of disciplines and are critiquing different methodologies. The point of this approach is to question  how  existing research has been conducted, as opposed to  what  the conclusions and/or findings the research were.

For example, a sociologist might centre their research around critiquing specific fieldwork practices. Their literature review will then be a summary of the fieldwork methodologies used by different studies.

Here are some questions you can ask yourself when structuring your literature review according to methodology:

• Which methodologies have been utilised in this field?
• Which methodology is the most popular (and why)?
• What are the strengths and weaknesses of the various methodologies?
• How can the existing methodologies inform my own methodology?

3: The Conclusion Section

Once you’ve completed the body section of your literature review using one of the structural approaches we discussed above, you’ll need to “wrap up” your literature review and pull all the pieces together to set the direction for the rest of your dissertation or thesis.

The conclusion is where you’ll present the key findings of your literature review. In this section, you should emphasise the research that is especially important to your research questions and highlight the gaps that exist in the literature. Based on this, you need to make it clear what you will add to the literature – in other words, justify your own research by showing how it will help fill one or more of the gaps you just identified.

Last but not least, if it’s your intention to develop a conceptual framework for your dissertation or thesis, the conclusion section is a good place to present this.

Example: Thematically Structured Review

In the video below, we unpack a literature review chapter so that you can see an example of a thematically structure review in practice.

Let’s Recap

In this article, we’ve  discussed how to structure your literature review for maximum impact. Here’s a quick recap of what  you need to keep in mind when deciding on your literature review structure:

• Just like other chapters, your literature review needs a clear introduction , body and conclusion .
• The introduction section should provide an overview of what you will discuss in your literature review.
• The body section of your literature review can be organised by chronology , theme or methodology . The right structural approach depends on what you’re trying to achieve with your research.
• The conclusion section should draw together the key findings of your literature review and link them to your research questions.

If you’re ready to get started, be sure to download our free literature review template to fast-track your chapter outline.

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

27 Comments

Great work. This is exactly what I was looking for and helps a lot together with your previous post on literature review. One last thing is missing: a link to a great literature chapter of an journal article (maybe with comments of the different sections in this review chapter). Do you know any great literature review chapters?

I agree with you Marin… A great piece

I agree with Marin. This would be quite helpful if you annotate a nicely structured literature from previously published research articles.

Awesome article for my research.

I thank you immensely for this wonderful guide

It is indeed thought and supportive work for the futurist researcher and students

Very educative and good time to get guide. Thank you

Great work, very insightful. Thank you.

Thanks for this wonderful presentation. My question is that do I put all the variables into a single conceptual framework or each hypothesis will have it own conceptual framework?

Thank you very much, very helpful

This is very educative and precise . Thank you very much for dropping this kind of write up .

Pheeww, so damn helpful, thank you for this informative piece.

I’m doing a research project topic ; stool analysis for parasitic worm (enteric) worm, how do I structure it, thanks.

comprehensive explanation. Help us by pasting the URL of some good “literature review” for better understanding.

great piece. thanks for the awesome explanation. it is really worth sharing. I have a little question, if anyone can help me out, which of the options in the body of literature can be best fit if you are writing an architectural thesis that deals with design?

I am doing a research on nanofluids how can l structure it?

Beautifully clear.nThank you!

Lucid! Thankyou!

Brilliant work, well understood, many thanks

I like how this was so clear with simple language 😊😊 thank you so much 😊 for these information 😊

Insightful. I was struggling to come up with a sensible literature review but this has been really helpful. Thank you!

You have given thought-provoking information about the review of the literature.

Thank you. It has made my own research better and to impart your work to students I teach

I learnt a lot from this teaching. It’s a great piece.

I am doing research on EFL teacher motivation for his/her job. How Can I structure it? Is there any detailed template, additional to this?

You are so cool! I do not think I’ve read through something like this before. So nice to find somebody with some genuine thoughts on this issue. Seriously.. thank you for starting this up. This site is one thing that is required on the internet, someone with a little originality!

I’m asked to do conceptual, theoretical and empirical literature, and i just don’t know how to structure it

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 9 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

How I harnessed media engagement to supercharge my research career

Career Column 09 APR 24

Ready or not, AI is coming to science education — and students have opinions

Career Feature 08 APR 24

How we landed job interviews for professorships straight out of our PhD programmes

Career Column 08 APR 24

How two PhD students overcame the odds to snag tenure-track jobs

Adopt universal standards for study adaptation to boost health, education and social-science research

Correspondence 02 APR 24

Is ChatGPT corrupting peer review? Telltale words hint at AI use

News 10 APR 24

Rwanda 30 years on: understanding the horror of genocide

Editorial 09 APR 24

Junior Group Leader Position at IMBA - Institute of Molecular Biotechnology

The Institute of Molecular Biotechnology (IMBA) is one of Europe’s leading institutes for basic research in the life sciences. IMBA is located on t...

Austria (AT)

IMBA - Institute of Molecular Biotechnology

Open Rank Faculty, Center for Public Health Genomics

Center for Public Health Genomics & UVA Comprehensive Cancer Center seek 2 tenure-track faculty members in Cancer Precision Medicine/Precision Health.

Charlottesville, Virginia

Center for Public Health Genomics at the University of Virginia

Husbandry Technician I

Memphis, Tennessee

St. Jude Children's Research Hospital (St. Jude)

Lead Researcher – Department of Bone Marrow Transplantation & Cellular Therapy

Researcher in the center for in vivo imaging and therapy.

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Reference management. Clean and simple.

How to write a literature review in 6 steps

What is a literature review?

How to write a literature review, 1. determine the purpose of your literature review, 2. do an extensive search, 3. evaluate and select literature, 4. analyze the literature, 5. plan the structure of your literature review, 6. write your literature review, other resources to help you write a successful literature review, frequently asked questions about writing a literature review, related articles.

A literature review is an assessment of the sources in a chosen topic of research.

A good literature review does not just summarize sources. It analyzes the state of the field on a given topic and creates a scholarly foundation for you to make your own intervention. It demonstrates to your readers how your research fits within a larger field of study.

In a thesis, a literature review is part of the introduction, but it can also be a separate section. In research papers, a literature review may have its own section or it may be integrated into the introduction, depending on the field.

➡️ Our guide on what is a literature review covers additional basics about literature reviews.

• Identify the main purpose of the literature review.
• Do extensive research.
• Evaluate and select relevant sources.
• Analyze the sources.
• Plan a structure.
• Write the review.

In this section, we review each step of the process of creating a literature review.

In the first step, make sure you know specifically what the assignment is and what form your literature review should take. Read your assignment carefully and seek clarification from your professor or instructor if needed. You should be able to answer the following questions:

• How many sources do I need to include?
• What types of sources should I review?
• Should I evaluate the sources?
• Should I summarize, synthesize or critique sources?
• Do I need to provide any definitions or background information?

In addition to that, be aware that the narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good overview of the topic.

Now you need to find out what has been written on the topic and search for literature related to your research topic. Make sure to select appropriate source material, which means using academic or scholarly sources , including books, reports, journal articles , government documents and web resources.

➡️ If you’re unsure about how to tell if a source is scholarly, take a look at our guide on how to identify a scholarly source .

Come up with a list of relevant keywords and then start your search with your institution's library catalog, and extend it to other useful databases and academic search engines like:

• Google Scholar
• Science.gov

➡️ Our guide on how to collect data for your thesis might be helpful at this stage of your research as well as the top list of academic search engines .

Once you find a useful article, check out the reference list. It should provide you with even more relevant sources. Also, keep a note of the:

• authors' names
• page numbers

Keeping track of the bibliographic information for each source will save you time when you’re ready to create citations. You could also use a reference manager like Paperpile to automatically save, manage, and cite your references.

Read the literature. You will most likely not be able to read absolutely everything that is out there on the topic. Therefore, read the abstract first to determine whether the rest of the source is worth your time. If the source is relevant for your topic:

• Read it critically.
• Look for the main arguments.
• Take notes as you read.
• Organize your notes using a table, mind map, or other technique.

Now you are ready to analyze the literature you have gathered. While your are working on your analysis, you should ask the following questions:

• What are the key terms, concepts and problems addressed by the author?
• How is this source relevant for my specific topic?
• How is the article structured? What are the major trends and findings?
• What are the conclusions of the study?
• How are the results presented? Is the source credible?
• When comparing different sources, how do they relate to each other? What are the similarities, what are the differences?
• Does the study help me understand the topic better?
• Are there any gaps in the research that need to be filled? How can I further my research as a result of the review?

Tip: Decide on the structure of your literature review before you start writing.

There are various ways to organize your literature review:

• Chronological method : Writing in the chronological method means you are presenting the materials according to when they were published. Follow this approach only if a clear path of research can be identified.
• Thematic review : A thematic review of literature is organized around a topic or issue, rather than the progression of time.
• Publication-based : You can order your sources by publication, if the way you present the order of your sources demonstrates a more important trend. This is the case when a progression revealed from study to study and the practices of researchers have changed and adapted due to the new revelations.
• Methodological approach : A methodological approach focuses on the methods used by the researcher. If you have used sources from different disciplines that use a variety of research methods, you might want to compare the results in light of the different methods and discuss how the topic has been approached from different sides.

Regardless of the structure you chose, a review should always include the following three sections:

• An introduction, which should give the reader an outline of why you are writing the review and explain the relevance of the topic.
• A body, which divides your literature review into different sections. Write in well-structured paragraphs, use transitions and topic sentences and critically analyze each source for how it contributes to the themes you are researching.
• A conclusion , which summarizes the key findings, the main agreements and disagreements in the literature, your overall perspective, and any gaps or areas for further research.

➡️ If your literature review is part of a longer paper, visit our guide on what is a research paper for additional tips.

➡️ UNC writing center: Literature reviews

➡️ How to write a literature review in 3 steps

➡️ How to write a literature review in 30 minutes or less

The goal of a literature review is to asses the state of the field on a given topic in preparation for making an intervention.

A literature review should have its own independent section. You should indicate clearly in the table of contents where it can be found, and address this section as “Literature Review.”

There is no set amount of words for a literature review; the length depends on the research. If you are working with a large amount of sources, then it will be long. If your paper does not depend entirely on references, then it will be short.

Most research papers include a literature review. By assessing the available sources in your field of research, you will be able to make a more confident argument about the topic.

Literature reviews are most commonly found in theses and dissertations. However, you find them in research papers as well.

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

• Study resources
• Calendar - Graduate
• Calendar - Undergraduate
• Class schedules
• Class cancellations
• Course registration
• Important academic dates
• More academic resources
• Campus services
• IT services
• Job opportunities
• Mental health support
• Student Service Centre (Birks)
• Calendar of events
• Latest news
• Media Relations
• Faculties, Schools & Colleges
• Arts and Science
• Gina Cody School of Engineering and Computer Science
• John Molson School of Business
• School of Graduate Studies
• All Schools, Colleges & Departments
• Directories

• My Library Account (Sofia) View checkouts, fees, place requests and more
• Interlibrary Loans Request books from external libraries
• Zotero Manage your citations and create bibliographies
• E-journals via BrowZine Browse & read journals through a friendly interface
• Article/Chapter Scan & Deliver Request a PDF of an article/chapter we have in our physical collection
• Course Reserves Online course readings
• Spectrum Deposit a thesis or article
• WebPrint Upload documents to print with DPrint
• Sofia Discovery tool
• Databases by subject
• Course Reserves
• E-journals via BrowZine
• E-journals via Sofia
• Article/Chapter Scan & Deliver
• Intercampus Delivery of Bound Periodicals/Microforms
• Interlibrary Loans
• Spectrum Research Repository
• Special Collections
• Additional resources & services
• Loans & Returns (Circulation)
• Subject & course guides
• Open Educational Resources Guide
• General guides for users
• Evaluating...
• Ask a librarian
• Research Skills Tutorial
• Quick Things for Digital Knowledge
• Bibliometrics & research impact guide
• Concordia University Press
• Copyright Guide
• Copyright Guide for Thesis Preparation
• Digital Scholarship
• Digital Preservation
• Open Access
• ORCID at Concordia
• Research data management guide
• Scholarship of Teaching & Learning
• Systematic Reviews
• How to get published speaker series
• Borrow (laptops, tablets, equipment)
• Connect (netname, Wi-Fi, guest accounts)
• Desktop computers, software & availability maps
• Group study, presentation practice & classrooms
• Printers, copiers & scanners
• Technology Sandbox
• Visualization Studio
• Webster Library
• Vanier Library
• Grey Nuns Reading Room
• Book a group study room/scanner
• Study spaces
• Floor plans
• Room booking for academic events
• Exhibitions
• Librarians & staff
• University Librarian
• Memberships & collaborations
• Indigenous Student Librarian program
• Wikipedian in residence
• Researcher-in-Residence
• Feedback & improvement
• Annual reports & fast facts
• Annual Plan
• Library Services Fund
• Giving to the Library
• Webster Transformation blog
• Policies & Code of Conduct

The Campaign for Concordia

Library Research Skills Tutorial

Log into...

• My Library account (Sofia)
• Interlibrary loans
• Article/chapter scan
• Course reserves

Quick links

How to write a literature review

What is a literature review.

The literature review is a written overview of major writings and other sources on a selected topic. Sources covered in the review may include scholarly journal articles, books, government reports, Web sites, etc. The literature review provides a description, summary and evaluation of each source. It is usually presented as a distinct section of a graduate thesis or dissertation.

Purpose of the literature review

The purpose of the literature review is to provide a critical written account of the current state of research on a selected topic:

• Identifies areas of prior scholarship
• Places each source in the context of its contribution to the understanding of the specific issue, area of research, or theory under review.
• Describes the relationship of each source to the others that you have selected
• Identifies new ways to interpret, and shed light on any gaps in, previous research
• Points the way forward for further research.

Components of the literature review

The literature review should include the following:

• Objective of the literature review
• Overview of the subject under consideration.
• particular position, those opposed, and those offering completely different arguments.
• Discussion of both the distinctiveness of each source and its similarities with the others.

Steps in the literature review process

Preparation of a literature review may be divided into four steps:

• Define your subject and the scope of the review.
• Search the library catalogue, subject specific databases and other search tools to find sources that are relevant to your topic.
• Read and evaluate the sources and to determine their suitability to the understanding of topic at hand (see the Evaluating sources section).
• Analyse, interpret and discuss the findings and conclusions of the sources you selected.

Evaluating sources

In assessing each source, consideration should be given to:

• What is the author's expertise in this particular field of study (credentials)?
• Are the author's arguments supported by empirical evidence (e.g. quantitative/qualitative studies)?
• Is the author's perspective too biased in one direction or are opposing studies and viewpoints also considered?
• Does the selected source contribute to a more profound understanding of the subject?

Examples of a published literature review

Literature reviews are often published as scholarly articles, books, and reports. Here is an example of a recent literature review published as a scholarly journal article:

Ledesma, M. C., & Calderón, D. (2015). Critical race theory in education: A review of past literature and a look to the future. Qualitative Inquiry, 21(3), 206-222. Link to the article

Additional sources on writing literature reviews

Further information on the literature review process may be found below:

• Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review
• Fink, A. (2010). Conducting research literature reviews: From the Internet to paper
• Galvin, J. (2006). Writing literature reviews: A guide for students of the social and behavioral sciences
• Machi, L. A., & McEvoy, B. T. (2012). The literature review: Six steps to success

Adapted with permission and thanks from How to Write a Literature Review originally created by Kenneth Lyons, McHenry Library, University of California, Santa Cruz.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 11, 2024 1:27 PM
• URL: https://libguides.usc.edu/writingguide

Academic Writing

• Strategies for Writing
• Punctuation
• Plagiarism & Self-Plagiarism

How to Build a Literature Review

• PRISMA - Systematic reviews & meta-analyses
• Other Resources
• Using Zotero for Bibliographies
• Abstract Writing Tips
• Writing Assistance
• Locating a Journal
• Assessing Potential Journals
• Finding a Publisher
• Types of Peer Review
• Author Rights & Responsibilities
• Copyright Considerations
• What is a Lit Review?
• Why Write a Lit Review?

Structure of a Literature Review

Preliminary steps for literature review.

• Basic Example
• More Examples

What is a Literature Review?

A literature review is a comprehensive summary and analysis of previously published research on a particular topic. Literature reviews should give the reader an overview of the important theories and themes that have previously been discussed on the topic, as well as any important researchers who have contributed to the discourse. This review should connect the established conclusions to the hypothesis being presented in the rest of the paper.

What a Literature Review Is Not:

• Annotated Bibliography: An annotated bibliography summarizes and assesses each resource individually and separately. A literature review explores the connections between different articles to illustrate important themes/theories/research trends within a larger research area. 
• Timeline: While a literature review can be organized chronologically, they are not simple timelines of previous events. They should not be a list of any kind. Individual examples or events should be combined to illustrate larger ideas or concepts.
• Argumentative Paper: Literature reviews are not meant to be making an argument. They are explorations of a concept to give the audience an understanding of what has already been written and researched about an idea. As many perspectives as possible should be included in a literature review in order to give the reader as comprehensive understanding of a topic as possible.

Why Write a Literature Review?

After reading the literature review, the reader should have a basic understanding of the topic. A reader should be able to come into your paper without really knowing anything about an idea, and after reading the literature, feel more confident about the important points.

A literature review should also help the reader understand the focus the rest of the paper will take within the larger topic. If the reader knows what has already been studied, they will be better prepared for the novel argument that is about to be made.

A literature review should help the reader understand the important history, themes, events, and ideas about a particular topic. Connections between ideas/themes should also explored. Part of the importance of a literature review is to prove to experts who do read your paper that you are knowledgeable enough to contribute to the academic discussion. You have to have done your homework.

A literature review should also identify the gaps in research to show the reader what hasn't yet been explored. Your thesis should ideally address one of the gaps identified in the research. Scholarly articles are meant to push academic conversations forward with new ideas and arguments. Before knowing where the gaps are in a topic, you need to have read what others have written.

As mentioned in other tabs, literature reviews should discuss the big ideas that make up a topic. Each literature review should be broken up into different subtopics. Each subtopic should use groups of articles as evidence to support the ideas. There are several different ways of organizing a literature review. It will depend on the patterns one sees in the groups of articles as to which strategy should be used. Here are a few examples of how to organize your review:

Chronological

If there are clear trends that change over time, a chronological approach could be used to organize a literature review. For example, one might argue that in the 1970s, the predominant theories and themes argued something. However, in the 1980s, the theories evolved to something else. Then, in the 1990s, theories evolved further. Each decade is a subtopic, and articles should be used as examples. 

Themes/Theories

There may also be clear distinctions between schools of thought within a topic, a theoretical breakdown may be most appropriate. Each theory could be a subtopic, and articles supporting the theme should be included as evidence for each one. 

If researchers mainly differ in the way they went about conducting research, literature reviews can be organized by methodology. Each type of method could be a subtopic,  and articles using the method should be included as evidence for each one.

• Define your research question
• Compile a list of initial keywords to use for searching based on question
• Search for literature that discusses the topics surrounding your research question
• Assess and organize your literature into logical groups
• Identify gaps in research and conduct secondary searches (if necessary)
• Reassess and reorganize literature again (if necessary)
• Write review

Here is an example of a literature review, taken from the beginning of a research article. You can find other examples within most scholarly research articles. The majority of published scholarship includes a literature review section, and you can use those to become more familiar with these reviews.

Source:  Perceptions of the Police by LGBT Communities

There are many books and internet resources about literature reviews though most are long on how to search and gather the literature. How to literally organize the information is another matter.

Some pro tips:

• Be thoughtful in naming the folders, sub-folders, and sub, sub-folders.  Doing so really helps your thinking and concepts within your research topic.
• Be disciplined to add keywords under the tabs as this will help you search for ALL the items on your concepts/topics.
• Use the notes tab to add reminders, write bibliography/annotated bibliography
• Your literature review easily flows from your statement of purpose (SoP).  Therefore, does your SoP say clearly and exactly the intent of your research?  Your research assumption and argument is obvious?
• Begin with a topic outline that traces your argument. pg99: "First establish the line of argumentation you will follow (the thesis), whether it is an assertion, a contention, or a proposition.
• This means that you should have formed judgments about the topic based on the analysis and synthesis of the literature you are reviewing."
• Keep filling it in; flushing it out more deeply with your references

Other Resources/Examples

• ISU Writing Assistance The Julia N. Visor Academic Center provides one-on-one writing assistance for any course or need. By focusing on the writing process instead of merely on grammar and editing, we are committed to making you a better writer.
• University of Toronto: The Literature Review Written by Dena Taylor, Health Sciences Writing Centre
• Purdue OWL - Writing a Lit Review Goes over the basic steps
• UW Madison Writing Center - Review of Literature A description of what each piece of a literature review should entail.
• USC Libraries - Literature Reviews Offers detailed guidance on how to develop, organize, and write a college-level research paper in the social and behavioral sciences.
• Creating the literature review: integrating research questions and arguments Blog post with very helpful overview for how to organize and build/integrate arguments in a literature review
• Understanding, Selecting, and Integrating a Theoretical Framework in Dissertation Research: Creating the Blueprint for Your “House” Article focusing on constructing a literature review for a dissertation. Still very relevant for literature reviews in other types of content.

A note that many of these examples will be far longer and in-depth than what's required for your assignment. However, they will give you an idea of the general structure and components of a literature review. Additionally, most scholarly articles will include a literature review section. Looking over the articles you have been assigned in classes will also help you.

• Understanding, Selecting, and Integrating a Theoretical Framework in Dissertation Research: Creating the Blueprint for Your “House” Excellent article detailing how to construct your literature review.
• Sample Literature Review (Univ. of Florida) This guide will provide research and writing tips to help students complete a literature review assignment.
• Sociology Literature Review (Univ. of Hawaii) Written in ASA citation style - don't follow this format.
• Sample Lit Review - Univ. of Vermont Includes an example with tips in the footnotes.

Attribution

Content on this page was provided by Grace Allbaugh

• << Previous: Writing a Literature Review
• Next: PRISMA - Systematic reviews & meta-analyses >>
• Last Updated: Dec 6, 2023 3:23 PM
• URL: https://guides.library.illinoisstate.edu/academicwriting

Additional Links

• Directions and Parking
• Accessibility Services
• Library Spaces
• Staff Directory

• Library Guides
• Literature Reviews
• Writing the Review

Literature Reviews: Writing the Review

Outline of review sections.

Your Literature Review should not be a summary and evaluation of each article, one after the other. Your sources should be integrated together to create a narrative on your topic.

Consider the following ways to organize your review:

• By themes, variables, or issues
• By varying perspectives regarding a topic of controversy
• Chronologically, to show how the topic and research have developed over time

Use an outline to organize your sources and ideas in a logical sequence. Identify main points and subpoints, and consider the flow of your review. Outlines can be revised as your ideas develop. They help guide your readers through your ideas and show the hierarchy of your thoughts. What do your readers need to understand first? Where might certain studies fit most naturally? These are the kinds of questions that an outline can clarify.

An example outline for a Literature Review might look like this:

Introduction

• Background information on the topic & definitions
• Purpose of the literature review
• Scope and limitations of the review (what is included /excluded)
• Historical background 
• Overview of the existing research on the topic
• Principle question being asked
• Organization of the literature into categories or themes
• Evaluation of the strengths and weaknesses of each study
• Combining the findings from multiple sources to identify patterns and trends
• Insight into the relationship between your central topic and a larger area of study
• Development of a new research question or hypothesis
• Summary of the key points and findings in the literature
• Discussion of gaps in the existing knowledge
• Implications for future research

Strategies for Writing

Annotated bibliography.

An annotated bibliography collects short descriptions of each source in one place. After you have read each source carefully, set aside some time to write a brief summary. Your summary might be simply informative (e.g. identify the main argument/hypothesis, methods, major findings, and/or conclusions), or it might be evaluative (e.g. state why the source is interesting or useful for your review, or why it is not).

This method is more narrative than the Literature Matrix talked about on the Documenting Your Search page.

Taking the time to write short informative and/or evaluative summaries of your sources while you are researching can help you transition into the drafting stage later on. By making a record of your sources’ contents and your reactions to them, you make it less likely that you will need to go back and re-read many sources while drafting, and you might also start to gain a clearer idea of the overarching shape of your review.

READ EXTANT LIT REVIEWS CLOSELY

As you conduct your research, you will likely read many sources that model the same kind of literature review that you are researching and writing. While your original intent in reading those sources is likely to learn from the studies’ content (e.g. their results and discussion), it will benefit you to re-read these articles rhetorically.

Reading rhetorically means paying attention to how a text is written—how it has been structured, how it presents its claims and analyses, how it employs transitional words and phrases to move from one idea to the next. You might also pay attention to an author’s stylistic choices, like the use of first-person pronouns, active and passive voice, or technical terminology.

See  Finding Example Literature Reviews on the Developing a Research Question page for tips on finding reviews relevant to your topic.

MIND-MAPPING

Creating a mind-map is a form of brainstorming that lets you visualize how your ideas function and relate. Draw the diagram freehand or download software that lets you easily manipulate and group text, images, and shapes ( Coggle ,  FreeMind , MindMaple ).

Write down a central idea, then identify associated concepts, features, or questions around that idea. Make lines attaching various ideas, or arrows to signify directional relationships. Use different shapes, sizes, or colors to indicate commonalities, sequences, or relative importance.

This drafting technique allows you to generate ideas while thinking visually about how they function together. As you follow lines of thought, you can see which ideas can be connected, where certain pathways lead, and what the scope of your project might be. By drawing out a mind-map you may be able to see what elements of your review are underdeveloped and will benefit from more focused attention.

USE VISUALIZATION TOOLS

Attribution.

Thanks to Librarian Jamie Niehof at the University of Michigan for providing permission to reuse and remix this Literature Reviews guide.

Avoiding Bias

Reporting bias.

This occurs when you are summarizing the literature in an unbalanced, inconsistent or distorted way . 

Ways to avoid:

• look for literature that supports multiple perspectives, viewpoints or theories 
• ask multiple people to review your writing for bias
• Last Updated: Apr 9, 2024 3:50 PM
• URL: https://info.library.okstate.edu/literaturereviews

• Undergraduate courses
• Postgraduate courses
• Foundation courses
• Apprenticeships
• Part-time and short courses
• Apply undergraduate
• Apply postgraduate

Search for a course

Search by course name, subject, and more

• Undergraduate
• Postgraduate
• (suspended) - Available in Clearing Not available in Clearing location-sign UCAS

Fees and funding

• Tuition fees
• Scholarships
• Funding your studies
• Student finance
• Cost of living support

Why study at Kent

Student life.

• Careers and employability
• Student support and wellbeing
• Our locations
• Placements and internships
• Year abroad
• Student stories
• Schools and colleges
• International
• International students
• Your country
• Applicant FAQs
• International scholarships
• University of Kent International College
• Campus Tours
• Applicant Events
• Postgraduate events
• Maps and directions
• Research strengths
• Research centres
• Research impact

Research institutes

• Durrell Institute of Conservation and Ecology
• Institute of Cyber Security for Society
• Institute of Cultural and Creative Industries
• Institute of Health, Social Care and Wellbeing

Research students

• Graduate and Researcher College
• Research degrees
• Find a supervisor
• How to apply

Popular searches

• Visits and Open Days
• Jobs and vacancies
• Accommodation
• Student guide
• Library and IT
• Partner with us
• Student Guide
• Student Help
• Health & wellbeing
• Student voice
• Living at Kent
• Careers & volunteering
• Diversity at Kent
• Finance & funding
• Life after graduation

Literature reviews

Writing a literature review.

The following guide has been created for you by the  Student Learning Advisory Service . For more detailed guidance and to speak to one of our advisers, please book an  appointment  or join one of our  workshops . Alternatively, have a look at our  SkillBuilder  skills videos.   

Preparing a literature review involves:

• Searching for reliable, accurate and up-to-date material on a topic or subject
• Reading and summarising the key points from this literature
• Synthesising these key ideas, theories and concepts into a summary of what is known
• Discussing and evaluating these ideas, theories and concepts
• Identifying particular areas of debate or controversy
• Preparing the ground for the application of these ideas to new research

Finding and choosing material

Ensure you are clear on what you are looking for. ask yourself:.

• What is the specific question, topic or focus of my assignment?
• What kind of material do I need (e.g. theory, policy, empirical data)?
• What type of literature is available (e.g. journals, books, government documents)?

What kind of literature is particularly authoritative in this academic discipline (e.g. psychology, sociology, pharmacy)?

How much do you need?

This will depend on the length of the dissertation, the nature of the subject, and the level of study (undergraduate, Masters, PhD). As a very rough rule of thumb – you may choose 8-10 significant pieces (books and/or articles) for an 8,000 word dissertation, up to 20 major pieces of work for 12-15,000 words, and so on. Bear in mind that if your dissertation is based mainly around an interaction with existing scholarship you will need a longer literature review than if it is there as a prelude to new empirical research. Use your judgement or ask your supervisor for guidance.

Where to find suitable material

Your literature review should include a balance between substantial academic books, journal articles and other scholarly publications. All these sources should be as up-to-date as possible, with the exception of ‘classic texts’ such as major works written by leading scholars setting out formative ideas and theories central to your subject. There are several ways to locate suitable material:

Module bibliography: for undergraduate dissertations, look first at the bibliography provided with the module documentation. Choose one or two likely looking books or articles and then scan through the bibliographies provided by these authors. Skim read some of this material looking for clues: can you use these leads to identify key theories and authors or track down other appropriate material?

Library catalogue search engine: enter a few key words to capture a range of items, but avoid over-generalisations; if you type in something as broad as ‘social theory’ you are likely to get several thousand results. Be more specific: for example, ‘Heidegger, existentialism’. Ideally, you should narrow the field to obtain just a few dozen results. Skim through these quickly to identity texts which are most likely to contribute to your study.

Library bookshelves: browse the library shelves in the relevant subject area and examine the books that catch your eye. Check the contents and index pages, or skim through the introductions (or abstracts, in the case of journal articles) to see if they contain relevant material, and replace them if not. Don’t be afraid to ask one of the subject librarians for further help. Your supervisor may also be able to point you in the direction of some of the important literature , but remember this is your literature search, not theirs.

Online: for recent journal articles you will almost certainly need to use one of the online search engines. These can be found on the ‘Indexing Services’ button on the Templeman Library website. Kent students based at Medway still need to use the Templeman pages to access online journals, although you can get to these pages through the Drill Hall Library catalogue. Take a look as well at the Subject Guides on both the Templeman and DHL websites.

Check that you have made the right selection by asking:

• Has my search been wide enough to ensure that I have identified all the relevant material, but narrow enough to exclude irrelevant material?
• Is there a good enough sample of literature for the level (PhD, Masters, undergraduate) of my dissertation or thesis?
• Have I considered as many alternative points of view as possible?
• Will the reader find my literature review relevant and useful?

Assessing the literature

Read the material you have chosen carefully, considering the following:

• The key point discussed by the author: is this clearly defined
• What evidence has the author produced to support this central idea?
• How convincing are the reasons given for the author’s point of view?
• Could the evidence be interpreted in other ways?
• What is the author's research method (e.g. qualitative, quantitative, experimental, etc.)?
• What is the author's theoretical framework (e.g. psychological, developmental, feminist)?
• What is the relationship assumed by the author between theory and practice?
• Has the author critically evaluated the other literature in the field?
• Does the author include literature opposing their point of view?
• Is the research data based on a reliable method and accurate information?
• Can you ‘deconstruct’ the argument – identify the gaps or jumps in the logic?
• What are the strengths and limitations of this study?
• What does this book or article contribute to the field or topic?
• What does this book or article contribute to my own topic or thesis?

As you note down the key content of each book or journal article (together with the reference details of each source) record your responses to these questions. You will then be able to summarise each piece of material from two perspectives:     

Content: a brief description of the content of the book or article. Remember, an author will often make just one key point; so, what is the point they are making, and how does it relate to your own research project or assignment?

Critical analysis: an assessment of the relative strengths and weaknesses of the evidence used, and the arguments presented. Has anything conveniently been left out or skated over? Is there a counter-argument, and has the author dealt with this adequately? Can the evidence presented be interpreted another way? Does the author demonstrate any obvious bias which could affect their reliability? Overall, based on the above analysis of the author’s work, how do you evaluate its contribution to the scholarly understanding and knowledge surrounding the topic?    

Structuring the literature review

In a PhD thesis, the literature review typically comprises one chapter (perhaps 8-10,000 words), for a Masters dissertation it may be around 2-3,000 words, and for an undergraduate dissertation it may be no more than 2,000 words. In each case the word count can vary depending on a range of factors and it is always best, if in doubt, to ask your supervisor.

The overall structure of the section or chapter should be like any other: it should have a beginning, middle and end. You will need to guide the reader through the literature review, outlining the strategy you have adopted for selecting the books or articles, presenting the topic theme for the review, then using most of the word limit to analyse the chosen books or articles thoroughly before pulling everything together briefly in the conclusion.

Some people prefer a less linear approach. Instead of simply working through a list of 8-20 items on your book review list, you might want to try a thematic approach, grouping key ideas, facts, concepts or approaches together and then bouncing the ideas off each other. This is a slightly more creative (and interesting) way of producing the review, but a little more risky as it is harder to establish coherence and logical sequencing.

Whichever approach you adopt, make sure everything flows smoothly – that one idea or book leads neatly to the next. Take your reader effortlessly through a sequence of thought that is clear, accurate, precise and interesting. 

Writing up your literature review

As with essays generally, only attempt to write up the literature review when you have completed all the reading and note-taking, and carefully planned its content and structure. Find an appropriate way of introducing the review, then guide the reader through the material clearly and directly, bearing in mind the following:

• Be selective in the number of points you draw out from each piece of literature; remember that one of your objectives is to demonstrate that you can use your judgement to identify what is central and what is secondary.
• Summarise and synthesise – use your own words to sum up what you think is important or controversial about the book or article.
• Never claim more than the evidence will support. Too many dissertations and theses are let down by sweeping generalisations. Be tentative and careful in the way you interpret the evidence.
• Keep your own voice – you are entitled to your own point of view provided it is based on evidence and clear argument.
• At the same time, aim to project an objective and tentative tone by using the 3rd person, (for example, ‘this tends to suggest’, ‘it could be argued’ and so on).
• Even with a literature review you should avoid using too many, or overlong, quotes. Summarise material in your own words as much as possible. Save the quotes for ‘punch-lines’ to drive a particular point home.
• Revise, revise, revise: refine and edit the draft as much as you can. Check for fluency, structure, evidence, criticality and referencing, and don’t forget the basics of good grammar, punctuation and spelling.

We use cookies on this site to enhance your experience

By clicking any link on this page you are giving your consent for us to set cookies.

A link to reset your password has been sent to your email.

Back to login

We need additional information from you. Please complete your profile first before placing your order.

Thank you. payment completed., you will receive an email from us to confirm your registration, please click the link in the email to activate your account., there was error during payment, orcid profile found in public registry, download history, deciding what to include and exclude as you begin to write your literature review.

• Charlesworth Author Services
• 06 October, 2021

Once you have completed your literature search , you can start thinking about creating a structure to best explain the literature and to link existing studies to your paper. Having a firm structure provides the foundation for laying out your discussions of the literature and/or the development of your research question(s) or hypothesis. Before that, though, you may still need to make some key decisions regarding what literature or texts should be included and excluded in your paper. This article highlights four essential considerations as you start drafting a structure for your literature review .

Comprehensiveness

There are two types of literature review : one is the literature review section within a paper and the other is a literature review synthesis paper that aims to provide a framework for future studies. For the first, a more focused review of only relevant studies would be more appropriate and useful. For the second type, however, you would usually be expected to provide a much more comprehensive review.

For a literature review section within a paper, a focused review that is more tightly related to your study will help you to build arguments more succinctly, and enable you to link existing studies to your own research more easily. If you find that the literature that is most relevant to your study still falls in large, broad categories, then breaking this section down into different, smaller subsections can be helpful for making sure the various ideas and themes are presented clearly and are easy to follow.

Level of detail

In the literature review, you should be aiming to clearly explain prior and current studies so you can better contextualise your own research within the field. However, the level of detail that you include in this section needs to be carefully considered. 

If several studies are key to your paper or sound similar to your study, you may need to compare and contrast them more closely in order to differentiate them from prior studies, create connections between them or to build on existing literature. 

In addition, if you need to draw from specific papers for your methodology or your theoretical framework , it is a good idea to go into slightly more detail and provide as much information as is reasonably possible, rather than assuming that the reader already knows about these studies.

However, too many detailed descriptions can be distracting and it is important to try to strike a balance between providing enough information for your reader to follow your argument without overwhelming them with too much detail. In some instances therefore, offering a summarised key message can work more effectively. 

In order to better gauge the level of detail needed, go through your writing several times to sharpen your focus, ask your colleagues for feedback or engage professional editing services to check that your structure and overall narrative are clear. 

Online sources and extended quotations

Sometimes, you may want to include online sources in your discussion of the literature. For example, government reports or reputable reports released by major organisations can be quite useful for helping you develop your narrative and arguments. These reports may also provide some initial evidence. However, if you do choose to use such studies, they should be engaged alongside other studies from different sources to make them more plausible. 

In addition, unless really necessary, try to avoid very long or extended quotations. A better practice is to paraphrase and/or summarise the key points that you are trying to make. Drawing from your notes can be useful here and will also help to avoid potential concerns about plagiarism . Using your own words to explain complex issues or to summarise long quotations can also make reading easier for the reader.

The literature review is supposed to comprise a summary of thoughts and findings in prior or existing studies related to the topic that you are addressing in your study. Accordingly, the discussion of these studies should be as objective as possible and should not include your personal opinions, comments or even article preferences. This will help you to describe what has already been done in the field more clearly and use this review as a basis for developing your own research.

Use the above four points to help you stay on track as you write. By being very clear about what your literature review will include or exclude, you will be able to provide an effective, focused overview of existing research, upon which you can build and structure your own study .

Read next (fourth) in series: How to refer to other studies or literature in the different sections of a research paper

Read previous (second) in series: How to structure and write your literature review

Maximise your publication success with Charlesworth Author Services .

Charlesworth Author Services, a trusted brand supporting the world’s leading academic publishers, institutions and authors since 1928. 

To know more about our services, visit:  Our Services

Share with your colleagues

Related articles.

Important factors to consider as you Start to Plan your Literature Review

Charlesworth Author Services 06/10/2021 00:00:00

Conducting a Literature Review

Charlesworth Author Services 10/03/2021 00:00:00

How to Structure and Write your Literature Review

Charlesworth Author Services 07/10/2021 00:00:00

Related webinars

Bitesize Webinar: How to write and structure your academic article for publication - Module 1: Know when are you ready to write

Charlesworth Author Services 04/03/2021 00:00:00

Bitesize Webinar: How to write and structure your academic article for publication- Module 3: Understand the structure of an academic paper

Bitesize Webinar: How to write and structure your academic article for publication: Module 5: Conduct a Literature Review

Bitesize Webinar: How to write and structure your academic article for publication: Module 11: Know when your article is ready for submission

Charlesworth Author Services 05/03/2021 00:00:00

Literature search

Why and How to do a literature search

Charlesworth Author Services 17/08/2020 00:00:00

Best tips to do a PubMed search

Charlesworth Author Services 26/08/2021 00:00:00

Best tips to do a Scopus search

Loading metrics

Open Access

Ten Simple Rules for Writing a Literature Review

* E-mail: [email protected]

Affiliations Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France, Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

• Marco Pautasso

Published: July 18, 2013

• https://doi.org/10.1371/journal.pcbi.1003149
• Reader Comments

Citation: Pautasso M (2013) Ten Simple Rules for Writing a Literature Review. PLoS Comput Biol 9(7): e1003149. https://doi.org/10.1371/journal.pcbi.1003149

Editor: Philip E. Bourne, University of California San Diego, United States of America

Copyright: © 2013 Marco Pautasso. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

Competing interests: The author has declared that no competing interests exist.

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

https://doi.org/10.1371/journal.pcbi.1003149.g001

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

• 1. Rapple C (2011) The role of the critical review article in alleviating information overload. Annual Reviews White Paper. Available: http://www.annualreviews.org/userimages/ContentEditor/1300384004941/Annual_Reviews_WhitePaper_Web_2011.pdf . Accessed May 2013.
• View Article
• Google Scholar
• 7. Budgen D, Brereton P (2006) Performing systematic literature reviews in software engineering. Proc 28th Int Conf Software Engineering, ACM New York, NY, USA, pp. 1051–1052. doi: https://doi.org/10.1145/1134285.1134500 .
• 16. Eco U (1977) Come si fa una tesi di laurea. Milan: Bompiani.
• 17. Hart C (1998) Doing a literature review: releasing the social science research imagination. London: SAGE.
• 21. Ridley D (2008) The literature review: a step-by-step guide for students. London: SAGE.

How to write a literature review introduction (+ examples)

The introduction to a literature review serves as your reader’s guide through your academic work and thought process. Explore the significance of literature review introductions in review papers, academic papers, essays, theses, and dissertations. We delve into the purpose and necessity of these introductions, explore the essential components of literature review introductions, and provide step-by-step guidance on how to craft your own, along with examples.

Why you need an introduction for a literature review

When you need an introduction for a literature review, what to include in a literature review introduction, examples of literature review introductions, steps to write your own literature review introduction.

A literature review is a comprehensive examination of the international academic literature concerning a particular topic. It involves summarizing published works, theories, and concepts while also highlighting gaps and offering critical reflections.

In academic writing , the introduction for a literature review is an indispensable component. Effective academic writing requires proper paragraph structuring to guide your reader through your argumentation. This includes providing an introduction to your literature review.

It is imperative to remember that you should never start sharing your findings abruptly. Even if there isn’t a dedicated introduction section .

Instead, you should always offer some form of introduction to orient the reader and clarify what they can expect.

There are three main scenarios in which you need an introduction for a literature review:

• Academic literature review papers: When your literature review constitutes the entirety of an academic review paper, a more substantial introduction is necessary. This introduction should resemble the standard introduction found in regular academic papers.
• Literature review section in an academic paper or essay: While this section tends to be brief, it’s important to precede the detailed literature review with a few introductory sentences. This helps orient the reader before delving into the literature itself.
• Literature review chapter or section in your thesis/dissertation: Every thesis and dissertation includes a literature review component, which also requires a concise introduction to set the stage for the subsequent review.

You may also like: How to write a fantastic thesis introduction (+15 examples)

It is crucial to customize the content and depth of your literature review introduction according to the specific format of your academic work.

In practical terms, this implies, for instance, that the introduction in an academic literature review paper, especially one derived from a systematic literature review , is quite comprehensive. Particularly compared to the rather brief one or two introductory sentences that are often found at the beginning of a literature review section in a standard academic paper. The introduction to the literature review chapter in a thesis or dissertation again adheres to different standards.

Here’s a structured breakdown based on length and the necessary information:

Academic literature review paper

The introduction of an academic literature review paper, which does not rely on empirical data, often necessitates a more extensive introduction than the brief literature review introductions typically found in empirical papers. It should encompass:

• The research problem: Clearly articulate the problem or question that your literature review aims to address.
• The research gap: Highlight the existing gaps, limitations, or unresolved aspects within the current body of literature related to the research problem.
• The research relevance: Explain why the chosen research problem and its subsequent investigation through a literature review are significant and relevant in your academic field.
• The literature review method: If applicable, describe the methodology employed in your literature review, especially if it is a systematic review or follows a specific research framework.
• The main findings or insights of the literature review: Summarize the key discoveries, insights, or trends that have emerged from your comprehensive review of the literature.
• The main argument of the literature review: Conclude the introduction by outlining the primary argument or statement that your literature review will substantiate, linking it to the research problem and relevance you’ve established.
• Preview of the literature review’s structure: Offer a glimpse into the organization of the literature review paper, acting as a guide for the reader. This overview outlines the subsequent sections of the paper and provides an understanding of what to anticipate.

By addressing these elements, your introduction will provide a clear and structured overview of what readers can expect in your literature review paper.

Regular literature review section in an academic article or essay

Most academic articles or essays incorporate regular literature review sections, often placed after the introduction. These sections serve to establish a scholarly basis for the research or discussion within the paper.

In a standard 8000-word journal article, the literature review section typically spans between 750 and 1250 words. The first few sentences or the first paragraph within this section often serve as an introduction. It should encompass:

• An introduction to the topic: When delving into the academic literature on a specific topic, it’s important to provide a smooth transition that aids the reader in comprehending why certain aspects will be discussed within your literature review.
• The core argument: While literature review sections primarily synthesize the work of other scholars, they should consistently connect to your central argument. This central argument serves as the crux of your message or the key takeaway you want your readers to retain. By positioning it at the outset of the literature review section and systematically substantiating it with evidence, you not only enhance reader comprehension but also elevate overall readability. This primary argument can typically be distilled into 1-2 succinct sentences.

In some cases, you might include:

• Methodology: Details about the methodology used, but only if your literature review employed a specialized method. If your approach involved a broader overview without a systematic methodology, you can omit this section, thereby conserving word count.

By addressing these elements, your introduction will effectively integrate your literature review into the broader context of your academic paper or essay. This will, in turn, assist your reader in seamlessly following your overarching line of argumentation.

Introduction to a literature review chapter in thesis or dissertation

The literature review typically constitutes a distinct chapter within a thesis or dissertation. Often, it is Chapter 2 of a thesis or dissertation.

Some students choose to incorporate a brief introductory section at the beginning of each chapter, including the literature review chapter. Alternatively, others opt to seamlessly integrate the introduction into the initial sentences of the literature review itself. Both approaches are acceptable, provided that you incorporate the following elements:

• Purpose of the literature review and its relevance to the thesis/dissertation research: Explain the broader objectives of the literature review within the context of your research and how it contributes to your thesis or dissertation. Essentially, you’re telling the reader why this literature review is important and how it fits into the larger scope of your academic work.
• Primary argument: Succinctly communicate what you aim to prove, explain, or explore through the review of existing literature. This statement helps guide the reader’s understanding of the review’s purpose and what to expect from it.
• Preview of the literature review’s content: Provide a brief overview of the topics or themes that your literature review will cover. It’s like a roadmap for the reader, outlining the main areas of focus within the review. This preview can help the reader anticipate the structure and organization of your literature review.
• Methodology: If your literature review involved a specific research method, such as a systematic review or meta-analysis, you should briefly describe that methodology. However, this is not always necessary, especially if your literature review is more of a narrative synthesis without a distinct research method.

By addressing these elements, your introduction will empower your literature review to play a pivotal role in your thesis or dissertation research. It will accomplish this by integrating your research into the broader academic literature and providing a solid theoretical foundation for your work.

Comprehending the art of crafting your own literature review introduction becomes significantly more accessible when you have concrete examples to examine. Here, you will find several examples that meet, or in most cases, adhere to the criteria described earlier.

Example 1: An effective introduction for an academic literature review paper

To begin, let’s delve into the introduction of an academic literature review paper. We will examine the paper “How does culture influence innovation? A systematic literature review”, which was published in 2018 in the journal Management Decision.

The entire introduction spans 611 words and is divided into five paragraphs. In this introduction, the authors accomplish the following:

• In the first paragraph, the authors introduce the broader topic of the literature review, which focuses on innovation and its significance in the context of economic competition. They underscore the importance of this topic, highlighting its relevance for both researchers and policymakers.
• In the second paragraph, the authors narrow down their focus to emphasize the specific role of culture in relation to innovation.
• In the third paragraph, the authors identify research gaps, noting that existing studies are often fragmented and disconnected. They then emphasize the value of conducting a systematic literature review to enhance our understanding of the topic.
• In the fourth paragraph, the authors introduce their specific objectives and explain how their insights can benefit other researchers and business practitioners.
• In the fifth and final paragraph, the authors provide an overview of the paper’s organization and structure.

In summary, this introduction stands as a solid example. While the authors deviate from previewing their key findings (which is a common practice at least in the social sciences), they do effectively cover all the other previously mentioned points.

Example 2: An effective introduction to a literature review section in an academic paper

The second example represents a typical academic paper, encompassing not only a literature review section but also empirical data, a case study, and other elements. We will closely examine the introduction to the literature review section in the paper “The environmentalism of the subalterns: a case study of environmental activism in Eastern Kurdistan/Rojhelat”, which was published in 2021 in the journal Local Environment.

The paper begins with a general introduction and then proceeds to the literature review, designated by the authors as their conceptual framework. Of particular interest is the first paragraph of this conceptual framework, comprising 142 words across five sentences:

“ A peripheral and marginalised nationality within a multinational though-Persian dominated Iranian society, the Kurdish people of Iranian Kurdistan (a region referred by the Kurds as Rojhelat/Eastern Kurdi-stan) have since the early twentieth century been subject to multifaceted and systematic discriminatory and exclusionary state policy in Iran. This condition has left a population of 12–15 million Kurds in Iran suffering from structural inequalities, disenfranchisement and deprivation. Mismanagement of Kurdistan’s natural resources and the degradation of its natural environmental are among examples of this disenfranchisement. As asserted by Julian Agyeman (2005), structural inequalities that sustain the domination of political and economic elites often simultaneously result in environmental degradation, injustice and discrimination against subaltern communities. This study argues that the environmental struggle in Eastern Kurdistan can be asserted as a (sub)element of the Kurdish liberation movement in Iran. Conceptually this research is inspired by and has been conducted through the lens of ‘subalternity’ ” ( Hassaniyan, 2021, p. 931 ).

In this first paragraph, the author is doing the following:

• The author contextualises the research
• The author links the research focus to the international literature on structural inequalities
• The author clearly presents the argument of the research
• The author clarifies how the research is inspired by and uses the concept of ‘subalternity’.

Thus, the author successfully introduces the literature review, from which point onward it dives into the main concept (‘subalternity’) of the research, and reviews the literature on socio-economic justice and environmental degradation.

While introductions to a literature review section aren’t always required to offer the same level of study context detail as demonstrated here, this introduction serves as a commendable model for orienting the reader within the literature review. It effectively underscores the literature review’s significance within the context of the study being conducted.

Examples 3-5: Effective introductions to literature review chapters

The introduction to a literature review chapter can vary in length, depending largely on the overall length of the literature review chapter itself. For example, a master’s thesis typically features a more concise literature review, thus necessitating a shorter introduction. In contrast, a Ph.D. thesis, with its more extensive literature review, often includes a more detailed introduction.

Numerous universities offer online repositories where you can access theses and dissertations from previous years, serving as valuable sources of reference. Many of these repositories, however, may require you to log in through your university account. Nevertheless, a few open-access repositories are accessible to anyone, such as the one by the University of Manchester . It’s important to note though that copyright restrictions apply to these resources, just as they would with published papers.

Master’s thesis literature review introduction

The first example is “Benchmarking Asymmetrical Heating Models of Spider Pulsar Companions” by P. Sun, a master’s thesis completed at the University of Manchester on January 9, 2024. The author, P. Sun, introduces the literature review chapter very briefly but effectively:

PhD thesis literature review chapter introduction

The second example is Deep Learning on Semi-Structured Data and its Applications to Video-Game AI, Woof, W. (Author). 31 Dec 2020, a PhD thesis completed at the University of Manchester . In Chapter 2, the author offers a comprehensive introduction to the topic in four paragraphs, with the final paragraph serving as an overview of the chapter’s structure:

PhD thesis literature review introduction

The last example is the doctoral thesis Metacognitive strategies and beliefs: Child correlates and early experiences Chan, K. Y. M. (Author). 31 Dec 2020 . The author clearly conducted a systematic literature review, commencing the review section with a discussion of the methodology and approach employed in locating and analyzing the selected records.

Having absorbed all of this information, let’s recap the essential steps and offer a succinct guide on how to proceed with creating your literature review introduction:

• Contextualize your review : Begin by clearly identifying the academic context in which your literature review resides and determining the necessary information to include.
• Outline your structure : Develop a structured outline for your literature review, highlighting the essential information you plan to incorporate in your introduction.
• Literature review process : Conduct a rigorous literature review, reviewing and analyzing relevant sources.
• Summarize and abstract : After completing the review, synthesize the findings and abstract key insights, trends, and knowledge gaps from the literature.
• Craft the introduction : Write your literature review introduction with meticulous attention to the seamless integration of your review into the larger context of your work. Ensure that your introduction effectively elucidates your rationale for the chosen review topics and the underlying reasons guiding your selection.

Get new content delivered directly to your inbox!

Subscribe and receive Master Academia's quarterly newsletter.

The best answers to "What are your plans for the future?"

10 tips for engaging your audience in academic writing, related articles.

How to peer review an academic paper

PhD thesis types: Monograph and collection of articles

How to disagree with reviewers (with examples!)

How to introduce yourself in a conference presentation (in six simple steps)

This paper is in the following e-collection/theme issue:

Published on 10.4.2024 in Vol 26 (2024)

Methodological Frameworks and Dimensions to Be Considered in Digital Health Technology Assessment: Scoping Review and Thematic Analysis

Authors of this article:

• Joan Segur-Ferrer, BSS, PT, MSc   ; 
• Carolina Moltó-Puigmartí, BScPharm, PhD   ; 
• Roland Pastells-Peiró, BA, MA, MsC   ; 
• Rosa Maria Vivanco-Hidalgo, MD, MPH, PhD  

Agency for Health Quality and Assessment of Catalonia, Barcelona, Spain

Corresponding Author:

Joan Segur-Ferrer, BSS, PT, MSc

Agency for Health Quality and Assessment of Catalonia

Roc Boronat Street, 81-95, 2nd Fl

Barcelona, 08005

Phone: 34 935 513 900

Fax:34 935 517 510

Email: [email protected]

Background: Digital health technologies (dHTs) offer a unique opportunity to address some of the major challenges facing health care systems worldwide. However, the implementation of dHTs raises some concerns, such as the limited understanding of their real impact on health systems and people’s well-being or the potential risks derived from their use. In this context, health technology assessment (HTA) is 1 of the main tools that health systems can use to appraise evidence and determine the value of a given dHT. Nevertheless, due to the nature of dHTs, experts highlight the need to reconsider the frameworks used in traditional HTA.

Objective: This scoping review (ScR) aimed to identify the methodological frameworks used worldwide for digital health technology assessment (dHTA); determine what domains are being considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

Methods: The ScR was performed in accordance with the guidelines established in the PRISMA-ScR guidelines. We searched 7 databases for peer reviews and gray literature published between January 2011 and December 2021. The retrieved studies were screened using Rayyan in a single-blind manner by 2 independent authors, and data were extracted using ATLAS.ti software. The same software was used for thematic analysis.

Results: The systematic search retrieved 3061 studies (n=2238, 73.1%, unique), of which 26 (0.8%) studies were included. From these, we identified 102 methodological frameworks designed for dHTA. These frameworks revealed great heterogeneity between them due to their different structures, approaches, and items to be considered in dHTA. In addition, we identified different wording used to refer to similar concepts. Through thematic analysis, we reduced this heterogeneity. In the first phase of the analysis, 176 provisional codes related to different assessment items emerged. In the second phase, these codes were clustered into 86 descriptive themes, which, in turn, were grouped in the third phase into 61 analytical themes and organized through a vertical hierarchy of 3 levels: level 1 formed by 13 domains, level 2 formed by 38 dimensions, and level 3 formed by 11 subdimensions. From these 61 analytical themes, we developed a proposal for a methodological framework for dHTA.

Conclusions: There is a need to adapt the existing frameworks used for dHTA or create new ones to more comprehensively assess different kinds of dHTs. Through this ScR, we identified 26 studies including 102 methodological frameworks and tools for dHTA. The thematic analysis of those 26 studies led to the definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered in dHTA.

Introduction

Digital health technologies (dHTs) are driving the transformation of health care systems. They are changing the way in which health services are delivered, and showing great potential to address some of the major challenges that European health systems, including the Spanish National Health System (SNS), are facing, such as the progressive aging of the population [ 1 , 2 ]; the growing demand for health and long-term care services [ 2 ]; the rise in health care costs, increasing financial pressures on health and welfare systems [ 1 , 3 ]; and the unequal distribution of health services across different geographical regions [ 4 , 5 ]. In addition, dHT can improve the accessibility, sustainability, efficiency, and quality of health care systems [ 6 , 7 ], leading to their becoming a determinant of health on their own [ 6 , 8 ].

However, the digital transformation of health care systems and the implementation of dHT (eg, artificial intelligence [AI]–based solutions, data-driven health care services, or the internet of things) are slow and unequal across different European regions [ 9 , 10 ]. Some of the reasons for this are (1) the immaturity of regulatory frameworks for the use of dHTs [ 9 ], (2) the lack of funding and investment for the implementation of dHTs [ 9 ], (3) the lack of sufficient and appropriate infrastructures and common standards for data management [ 6 , 9 ], (4) the absence of skills and expertise of professionals and users [ 10 ], and (5) the scarcity of strong evidence regarding the real benefits and effects of dHTs on health systems and people’s well-being, as well as the cost-effectiveness of these technologies. This makes decision-making difficult, potentially leading to the development and reproduction of low-value and short-lived dHTs [ 6 , 11 ].

To overcome these challenges, harness the potential of dHTs, and avoid nonintended consequences, the World Health Organization (WHO) [ 4 , 11 ] states that dHTs should be developed under the principles of transparency, accessibility, scalability, privacy, security, and confidentiality. Their implementation should be led by robust strategies that bring together leadership, financial, organizational, human, and technological resources, and decisions should be guided by the best-available evidence [ 4 , 11 ].

Regarding this last aspect, health technology assessment (HTA), defined as a “multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its life cycle,” is a widely accepted tool to inform decision-making and promote equitable, efficient, and high-quality health systems [ 12 , 13 ].

Generally, HTA is conducted according to specific methodological frameworks, such as the HTA Core Model of the European Network for Health Technology Assessment (EUnetHTA) [ 14 ] and the guidelines for the development and adaptation of rapid HTA reports of the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS) [ 15 ]. These frameworks establish the methodologies to follow and the elements to evaluate. Although these frameworks are helpful instruments for evaluating various health technologies, they have certain limitations in comprehensively assessing dHTs. For this reason, in the past few years, different initiatives have emerged to adapt existing methodological frameworks or develop new ones. The objective is to consider additional domains (eg, interoperability, scalability) to cover the intrinsic characteristics of dHTs [ 16 - 18 ]. Examples of these initiatives are the Evidence Standard Framework (ESF) of National Institute for Health and Care Excellence (NICE) [ 19 ] or the Digi-HTA Framework of the Finnish Coordinating Center for Health Technology Assessment (FinCCHTA) [ 16 ]. Nonetheless, the majority of these frameworks have certain constraints, such as being designed for a particular socioeconomic or national setting, which restricts their transferability or suitability for use in other countries; the specificity or exclusion of certain dHTs, resulting in limitations in their application; or the limited evidence regarding their actual usefulness.

In this context, we performed a scoping review (ScR) with the aim of identifying the methodological frameworks that are used worldwide for the evaluation of dHTs; determining what dimensions and aspects are considered for each type of dHT; and generating, through a thematic analysis, a proposal for a methodological framework that is based on the most frequently described dimensions in the literature. This research focused mainly on mobile health (mHealth), non–face-to-face care models and medical devices that integrate AI, as these particular dHTs are the ones most frequently assessed by HTA agencies and units of RedETS.

Identifying Research Questions

This ScR followed by a thematic analysis answered the following research questions:

• What methodological frameworks currently exist for digital health technology assessment (dHTA)?
• What domains and dimensions are considered in dHTA?
• Do the different domains and dimensions considered depend on whether the dHT addressed is a non–face-to-face care model of health care provision, a mobile device (mHealth), or a device that incorporates AI?

Overview of Methods for Conducting the Scoping Review

We conducted an ScR of the literature and a thematic analysis of the studies included according to the published protocol [ 20 ]. The ScR aimed to answer the first research question, while the thematic analysis aimed to answer the second and third research questions. Spanish experts from various domains of HTA and dHT collaborated throughout the study design and development.

The ScR of the available scientific literature was carried out in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews) guidelines ( Multimedia Appendix 1 ) [ 21 ] and following the recommendations of Peters et al [ 22 ] and Pollock et al [ 23 ].

Ethical Considerations

As this work was an ScR, no ethical board approval was required.

Search Strategy

The search strategy ( Multimedia Appendix 2 ) was designed by an experienced information specialist (author RP-P) in accordance with the research questions and using the validated filter of Ayiku et al [ 24 ] for health apps, adding the terms for concepts related to mHealth, remote care models, AI, digital health, methodological frameworks, and HTA. The strategy was peer-reviewed according to the “Peer Review of Electronic Search Strategies Statement” [ 25 ] by authors JS-F and CM-P and was executed in the following 7 databases, considering the characteristics of each in terms of syntax, controlled vocabulary, and proximity operators: Medline (OVID), CINAHL Plus, Embase, Cochrane Library, Scopus, Web of Science, and TripDatabase. Note that no time, language, or other filters were used.

The identification of relevant studies was complemented with a manual search based on the references in the included studies, as well as the websites of the HTA agencies identified through the web pages of EUnetHTA, the International Network for Agencies for Health Technology Assessment (INAHTA), and Health Technology Assessment International (HTAi). Additionally, a search was conducted in Google Scholar, limiting the results to the first 250 items in order to guarantee the inclusion of all pertinent studies [ 26 ].

Inclusion and Exclusion Criteria

The inclusion criteria used in the reference-screening process were based on the previously detailed research questions and are outlined in Textbox 1 using the Population/Problem, Phenomenon of Interest, Context and Design (PICo-D) format [ 27 , 28 ]. The PICo-D format was used instead of the traditional Population/Problem, Intervention, Comparator, Outcomes, Design (PICO-D) format due to the qualitative nature of the research questions and the characteristics of the phenomenon of interest.

Studies were excluded if they were published before 2011, due to the rapid evolution of dHTs in the past few years, did not describe dimensions or evaluation criteria, or were based on methodological frameworks not intended for the assessment of dHTs (eg, EUnetHTA Core Model 3.0). Likewise, we excluded comments, editorials, letters, conference abstracts, frameworks, or tools focusing on the evaluation of dHTs by users (eg, User version of Mobile App Rating Scale [uMARS]) or documents in languages other than English, Spanish. or Catalan.

Population/problem

Digital health technology assessment (dHTA)

Phenomenon of interest

Specific methodological frameworks for the evaluation of digital health (with special focus on mobile health [mHealth]: non–face-to-face care models and medical devices that integrate artificial intelligence [AI] due the type of technologies mostly assessed in the Spanish National Health System [SNS]) that describe the domains to be evaluated in dHTA

Health technology assessment (HTA)

Methodological guidelines and frameworks, scoping reviews (ScRs), systematic reviews (SRs), consensus documents, and qualitative studies

Reference Screening and Data Extraction

The screening of studies was carried out by authors CM-P and JS-F in 2 phases in accordance with the selection criteria detailed earlier ( Textbox 1 ) and in a single-blind peer review manner. The first phase consisted of screening of the titles and abstracts of the studies identified in the bibliographic search. The second phase consisted of full-text screening of the studies included in the previous phase.

Data extraction was performed by 3 authors (CM-P, RP-P, and JS-F) using the web and desktop versions of ATLAS.ti version 22.0 (Scientific Software Development GmbH) [ 29 ] and the data extraction sheets designed ad hoc for this purpose following the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions [ 30 ].

When disagreements emerged in either of the 2 processes, a consensus was reached between the 3 reviewers (CM-P, RP-P, and JS-F). When a consensus was not possible, a fourth reviewer (author RMV-H) was consulted.

Collecting, Summarizing, and Reporting the Results

A descriptive analysis was carried out to evaluate and report the existing methodological frameworks and their characteristics.

Overview of Methods for Thematic Analysis

The thematic analysis was performed following the recommendations and phases described by Thomas and Harden [ 31 ] to determine HTA dimensions for dHTs: (1) line-by-line text coding, (2) development of descriptive topics, and (3) generation of analytical themes. Both analyses were carried out by 3 authors (CM-P, RP-P, and JS-F) using the web and desktop versions of ATLAS.ti version 22.0 [ 29 ].

Dimensions identified from systematic reviews (SRs) that were derived from primary studies also identified in our systematic search were only counted once in order to avoid duplication of data and risk of bias. It is worth mentioning that the primary studies included in the SRs were not directly analyzed but were analyzed through the findings reported in the SRs.

Study Selection and Characteristics

A total of 3042 studies were retrieved throughout the systematic (n=3023, 99.4%) and the manual (n=19, 0.6%) search. Of these, 2238 (73.6%) studies were identified as unique after removing duplicates.

After title and abstract review, 81 (3.6%) studies were selected for full-text review, of which 26 (32.1%) were finally included in the analysis. The excluded studies and reasons for exclusion are detailed in Multimedia Appendix 3 ; in brief, the reasons for exclusion were phenomenon of interest (n=30, 37%), type of publication (n=15, 18.5%), purpose (n=6, 7.4%), language (n=2, 2.5%), and duplicated information (n=2, 2.5%). The study selection process is outlined in Figure 1 [ 32 ].

Of the 26 (32.1%) studies included in this ScR, 19 (73.1%) were designed as specific methodological frameworks for dHTA [ 16 , 17 , 33 - 47 ], 4 (15.4%) were SRs [ 48 - 51 ], 1 (3.9%) was a report from the European mHealth Hub’s working group on mHealth assessment guidelines [ 52 ], 1 (3.9%) was a qualitative study [ 53 ], and 1 (3.9%) was a viewpoint [ 54 ]. In addition, 3 (11.5%) focused on the assessment of non–face-to-face care models [ 33 - 35 ], 8 (30.8%) on mHealth assessment [ 36 - 40 , 52 , 53 , 55 ], 2 (7.7%) on the assessment of AI technology [ 41 , 54 ], 4 (15.4%) on eHealth [ 42 , 43 , 48 , 50 ], and 9 (34.6%) on the overall assessment of digital health [ 16 , 17 , 44 - 47 , 49 , 51 , 56 ].

Research Question 1: Description of Identified Frameworks for dHTA

The 19 methodological frameworks for dHTA [ 16 , 17 , 33 - 47 ] were from various countries: The majority (n=5, 26.3%) originated in Australia [ 17 , 34 , 38 , 41 , 46 ], followed by 3 (15.8%) from the United States [ 43 , 45 , 56 ] and 2 (10.5%) from Switzerland [ 47 , 55 ]; the remaining 9 (47.4%) frameworks were developed in Afghanistan [ 42 ], Denmark [ 33 ], Scotland [ 35 ], Finland [ 16 ], Ireland [ 36 ], Israel [ 40 ], the United Kingdom [ 37 ], Spain [ 39 ], and Sweden [ 44 ].

The 19 methodological frameworks focused on evaluating various types of technologies. Specifically, 3 (15.8%) of them were designed for assessing non–face-to-face care models [ 33 - 35 ], 6 (31.6%) for mHealth [ 36 - 40 ], and 1 (5.3%) for AI solutions [ 41 ]. The other 9 (47.4%) frameworks addressed eHealth [ 42 , 43 , 56 ] or digital health in general [ 16 , 17 , 44 - 47 ], which encompasses non–face-to-face care models, mHealth, and occasionally AI-based solutions [ 18 ] within its scope. It is pertinent to mention that the differentiation between the methodological frameworks designed for the evaluation of eHealth and those designed for dHTA was based on the specific terminology and descriptions used by the authors of those frameworks.

The structures and characteristics of the analyzed methodological frameworks were considered heterogeneous in terms of evaluation specificity (whether they focused on a global evaluation that encompassed more than 1 domain or dimension or on a specific assessment that addressed only 1 domain or dimension), assessment approach (whether they adopted a phased evaluation, a domain evaluation, or a hybrid of both), and number of domains included. Regarding evaluation specificity, 17 (89.5%) methodological frameworks were classified as global as they covered various aspects or domains within their scope [ 16 , 17 , 33 - 36 , 38 - 47 , 55 , 56 ], while 2 (10.5%) were classified as specific as they concentrated exclusively on 1 element or domain of assessment [ 37 , 46 ]. Regarding the assessment approach, 14 (73.7%) methodological frameworks proposed a domain-based evaluation [ 16 , 17 , 33 , 35 , 36 , 38 - 40 , 43 , 44 , 46 , 55 , 56 ], while 4 (21.1%) proposed a hybrid one (phased and domain based) [ 41 , 42 , 45 , 47 ]; the remaining methodological framework did not fit into any of the previous categories, as it was not structured by domains or phases but by types of risk [ 37 ]. Finally, the number of evaluation domains considered ranged from 1 to 14, with an average of 7. Table 1 outlines the primary features of the included methodological frameworks and provides a thorough breakdown of the domains and dimensions they address.

In contrast, from 3 (75%) [ 49 - 51 ] of the 4 SRs [ 48 - 51 ] and the report from the working group on guidelines for the evaluation of mHealth solutions from the European mHealth Hub [ 52 ], we identified other methodological frameworks and tools focusing on the assessment of dHTs. Specifically, we identified 16 methodological frameworks or tools focusing on the evaluation of non–face-to-face care models [ 57 - 72 ], along with 37 for the evaluation of mHealth [ 10 , 52 , 73 - 95 ], 11 for the evaluation of eHealth [ 96 - 107 ], and 17 for the evaluation of dHTs in general [ 108 - 124 ]. Additionally, 5 (26.3%) [ 33 , 34 , 36 , 37 , 42 ] of the 19 methodological frameworks included in this ScR were also identified and analyzed in 1 or more of the 4 literature synthesis documents [ 49 - 52 ]. It is important to note that the difference between the frameworks we retrieved through our systematic search and those identified in the 4 SRs is the result of the narrower perspective we adopted, focusing exclusively on frameworks directly relevant to the HTA field, in line with the aims of our study. In Multimedia Appendix 4 , we provide a more detailed explanation of the methodological frameworks included in the studies mentioned earlier [ 19 , 49 - 52 , 57 - 73 , 75 - 135 ].

a ScR: scoping review.

b mHealth: mobile health.

c N/A: not applicable.

d AI: artificial intelligence.

e dHT: digital health technology.

Research Question 2: Domains and Dimensions Being Considered in dHTA

The 26 (32.1%) studies included encompassed a broad range of items to consider in dHTA and often used diverse expressions for analogous concepts. We reduced this heterogeneity through our thematic analysis according to the recommendations and phases described by Thomas and Harden [ 31 ].

In this sense, in the first phase of thematic analysis, we identified and coded 176 units of meaning (coded as provisional codes) that represented different items (domains or dimensions) of the assessment. These units were then grouped into 86 descriptive themes (second phase), which were further refined into 61 analytical themes that captured the key concepts and relationships between them (third phase). Lastly, the 61 analytical themes were arranged in a 3-level vertical hierarchy based on the evidence: level 1 (12 domains), level 2 (38 dimensions), and level 3 (11 subdimensions). We used the term “domain” to refer to a distinct area or topic of evaluation that is integral to the assessment of the technology in question. A domain may encompass multiple related concepts or dimensions that are relevant to the evaluation. Each dimension, in turn, represents a specific aspect of evaluation that belongs to the domain and contributes to an understanding of its overall significance. Finally, a subdimension refers to a partial element of a dimension that facilitates its analysis. By using these terms, we aimed to provide a clear, rigorous, and comprehensive framework for conducting HTA.

Table 2 displays the 61 analytical themes in descending order of coding frequency, aligned with the hierarchy derived from the data analysis. Additionally, the table specifies the intervention modalities or dHTs that correspond to each code and lists the studies from which each code originated. The network of relationships among the codes can be found in Multimedia Appendix 5 .

a dHT: digital health technology.

c AI: artificial intelligence.

d N/A: not applicable.

Research Question 3: Variability of Domains and Dimensions Among Technologies

Our thematic analysis revealed a significant degree of variability and heterogeneity in the number and type of domains and dimensions considered by the methodological frameworks.

In terms of numbers, the variability was quite pronounced when we compared frameworks addressing different types of dHTs. For instance, the thematic analysis of frameworks for assessing telemedicine only identified 9 (75%) domains and 6 (15.8%) dimensions; instead, in frameworks for assessing mHealth, we identified 10 (83.3%) domains, 20 (52.6%) dimensions, and 6 (54.5%) subdimensions, and in frameworks for assessing AI, we identified 8 (66.7%) different domains, 7 (18.4%) different dimensions, and 6 (54.5%) subdimensions.

In terms of the types of domains considered, certain dimensions and domains were identified as more distinctive for one kind of dHT than for another. For instance, clinical efficacy and effectiveness, technical safety, economic evaluation, and user experience were relevant for the evaluation of models of nonpresential health care and mHealth but not for AI. In contrast, there were specific dimensions and domains of mHealth that were not considered in the evaluation of non–face-to-face health care or AI, such as postmarketing monitoring, scientific basis, technical evaluation and validation, user control and self-determination, accessibility, content and adequacy of information, and data interoperability and integration. Finally, specific methodological frameworks for the evaluation of AI included dimensions such as technical aspects, adoption, use, integration, generalizability, reproducibility, and interpretability, which were not considered in the evaluation of telemedicine or mHealth. In conclusion, greater clarity and structuring in the presentation of these ideas are required to facilitate their understanding and assimilation.

Proposal for Domains, Dimensions, and Subdimensions for dHTA

These findings led to the development of a proposed methodological framework for dHTA, which comprises domains, dimensions, and subdimensions. These evaluation items were established objectively based on thematically analyzed evidence, without incorporating the researcher’s perspective. Consequently, the proposal for domains, dimensions, and subdimensions emerged from the literature and represents the entirety of identified evaluation domains, dimensions, and subdimensions (n=61). Figure 2 presents a visual representation of the proposed framework comprising 12 domains, 38 dimensions, and their corresponding 11 subdimensions. Notably, the figure highlights certain domains, dimensions, and subdimensions that are particularly relevant to the evaluation of non–face-to-face care models, mHealth, and AI according to the evidence.

Principal Findings

In recent years, the interest in digital health has increased significantly, giving rise to a myriad of available technologies. This has brought about a profound transformation in health care systems, fundamentally changing the provision and consumption of health care services [ 9 ]. However, despite these advancements, the shift toward digital health has been accompanied by challenges. One such challenge is the emergence of a plethora of short-lived implementations and an overwhelming diversity of digital tools, which has created a need for careful evaluation and analysis of the benefits and drawbacks of these technologies [ 4 ].

In this context, our ScR aimed to identify the methodological frameworks used worldwide for the assessment of dHTs; determine what domains are considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

Throughout the ScR, we identified a total of 95 methodological frameworks and tools, of which 19 [ 16 , 17 , 33 - 47 ] were directly identified through a systematic search and 75 were indirectly identified through 4 SRs [ 49 - 52 ]. The difference in the number of methodological frameworks identified through the ScR and the 4 evidence synthesis documents [ 49 - 52 ] is attributed to the inclusion of keywords related to the concept of HTA in the search syntax, the exclusion of methodological frameworks published prior to 2011 during the screening process, and the differences in perspectives used for the development of this paper compared to the 4 evidence synthesis documents mentioned earlier. In this sense, these 4 documents [ 49 - 52 ] have analyzed methodological frameworks and tools aimed at evaluating digital health that have not been developed from an HTA perspective despite the authors analyzing them as such. For example, von Huben et al. [ 51 ] included in their analysis the Consolidated Standards of Reporting Trials (CONSORT)-EHEALTH tool [ 97 ], which aims to describe the information that should be reported in papers and reports that focus on evaluating web- and mHealth-based interventions; Koladas et al [ 49 ] included the mobile health evidence reporting and assessment (mERA) checklist [ 73 ], which aims to determine the information that should be reported in trials evaluating mHealth solutions; and the European mHealth Hub document [ 52 ] includes the Isys Score, which is for cataloguing apps for smartphones.

However, as detailed in the Results section, some of the methodological frameworks identified through the ScR were characterized by the authors themselves as being specific for evaluating certain types of dHTs (eg, non–face-to-face care models, mHealth), presenting certain differences according to each typology. It is important to note that the differentiation among various types of dHTs, as described throughout this paper and commonly used in the field of digital health, cannot always be made in a precise and exclusive manner [ 136 ]. This is because a technology often can be classified in more than 1 category. For instance, an mHealth solution may use AI algorithms, while simultaneously being integrated into a non–face-to-face care model [ 137 ]. In this context, future research should consider using alternative taxonomies or classification methods that are based on the intended purpose of the technology, such as those proposed by NICE in the updated version of the Evidence Standards Framework [ 18 ] or the new digital health interventions system classification put forward by WHO [ 138 ].

After conducting a thematic analysis of the 26 included studies, we observed that various methodological frameworks include a set of evaluation items, referred to as domains, dimensions, or criteria. These items primarily focus on the safety; effectiveness; technical aspects; economic impact; and ethical, legal, and social consequences of dHTs. However, there is significant heterogeneity among these frameworks in terms of the way they refer to the evaluation items, the quantity and depth of their description, the degree of granularity, and the proposed evaluation methods, especially when comparing frameworks that focus on different types of dHTs. Despite this heterogeneity, most methodological frameworks consider evaluation items related to the 9 domains described by the HTA Core Model of EUnetHTA, while some frameworks propose additional evaluation elements, such as usability [ 16 , 44 , 45 , 47 , 49 , 56 ], privacy [ 39 - 41 , 44 , 52 , 55 ], and technical stability [ 16 , 38 , 47 , 49 , 52 ] among others. These findings are consistent with earlier research [ 50 , 51 ].

In addition, through the thematic analysis, the heterogeneity identified among the different methodological frameworks included in this ScR was reduced to a total of 61 analytical themes related to various evaluation elements that were arranged in a 3-level vertical hierarchy based on the evidence: level 1 (12 domains), level 2 (38 dimensions), and level 3 (11 subdimensions). At this point, it is pertinent to note that although from the researchers’ perspective, some dimensions could have been classified under different domains (eg, responsibility under ethical aspects) or seen as essential for other kinds of dHTs, an effort was made to maintain the highest degree of objectivity possible. It is for this reason that privacy issues were not described as essential for non–face-to-face care models and why the dimension of accessibility was categorized within the domains of human and sociocultural aspects and technical aspects. This categorization was made because some of the methodological frameworks analyzed associated it with sociocultural elements (eg, evaluating whether users with functional diversity can access the technology and have sufficient ability to use it as expected), while others linked it to technical elements (eg, adequacy of the elements, options, or accessibility functionalities that the system incorporates according to the target audience) [ 16 , 52 ].

The ScR and thematic analysis conducted in this study led to a proposal for a methodological framework for dHTA. This framework was further developed using additional methodologies, such as consensus workshops by the Agency for Health Quality and Assessment of Catalonia (AQuAS), in collaboration with all agencies of RedETS, commissioned by the Ministry of Health of Spain. The final framework is a specific methodological tool for the assessment of dHTs, aimed at describing the domains and dimensions to be considered in dHTA and defining the evidence standards that such technologies must meet based on their associated risk level. The proposed methodological framework enables the assessment of a wide range of dHTs, mainly those classified as medical devices according to the Regulation (EU) 2017/745 for medical devices [ 139 ] and Regulation (EU) 2017/746 for in vitro diagnostic medical devices, although it can be adapted to assess dHTs not classified as medical devices [ 140 ]. Unlike existing frameworks, it establishes a clear link between the identified domains and dimensions and the evidence standards required for dHTs to meet. This approach will enhance the transparency and consistency of dHTAs and support evidence-based decision-making. The final document was published from November 2023 onward and is available on the RedETS website as well as on the main web page of AQuAS in the Spanish language [ 141 ]. From the first week of February, the respective websites have hosted an English version of this document [ 141 ], which also is accessible in the INAHTA database. In addition, the Spanish and English versions of the document will be periodically reviewed and, if necessary, adapted to align with emerging technologies and changes in legislation.

Limitations

Although this ScR was conducted in accordance with the PRISMA-ScR guidelines ( Multimedia Appendix 1 ) and following the recommendations of Peters et al [ 22 ] and Pollock et al [ 23 ], there were some limitations. First, the search incorporated a block of keywords related to the concept of HTA (see Multimedia Appendix 1 ) due to the perspective of our ScR, which may have limited the retrieval of some studies to meet the study objective. However, this limitation was compensated for by the analysis of the 3 SRs and the report of the working group on guidelines for the evaluation of mHealth solutions of the European mHealth Hub. Second, much of the literature related to HTA is gray literature and only published on the websites of the authoring agencies. Despite efforts to address this limitation through expert input and a comprehensive search of the websites of the world’s leading agencies, it is possible that certain studies were not identified. Third, the quality and limitations of the analysis conducted by the authors of methodological frameworks and tools included in SRs may have had an impact on the indirect thematic analysis. Therefore, it is possible that some data could have been omitted or not considered during this process. Fourth, the focus on dHTs encompassed within the 3 previously mentioned categories (mHealth, non–face-to-face care models, and medical devices that integrate AI) may have influenced the outcomes of the thematic analysis conducted. Fifth, only methodological frameworks written in Catalan, Spanish, and English were included.

Comparison With Prior Work

To the best of our knowledge, this is the first ScR to examine the methodological frameworks for dHTA, followed by a thematic analysis with the aim of proposing a new comprehensive framework that incorporates the existing literature in an objective manner and enables the assessment of various technologies included under the concept of digital health. In this sense, existing SRs and other evidence synthesis documents have only analyzed the literature and reported the results in a descriptive manner [ 36 , 48 , 49 , 51 , 56 , 125 , 126 ]. Furthermore, this ScR also considered, in addition to scientific literature, gray literature identified by searching the websites of the agencies, thus covering some limitations of previous reviews [ 50 ]. Moreover, this review was carried out from the perspective of HTA, addressing a clear need expressed by HTA agencies [ 16 ].

Future research should aim to identify what domains and dimensions are relevant at the different stages of the technology life cycle, to establish or develop a standardized set of outcomes for assessing or reporting each domain, and to evaluate the effectiveness and usefulness of the existing methodological frameworks for the different intended users [ 50 , 142 ]. Moreover, future research should aim to determine the specific evaluation criteria that ought to be considered based on the level of risk associated with different types of technologies [ 51 ].

Our ScR revealed a total of 102 methodological frameworks and tools designed for evaluating dHTs, with 19 being directly identified through a systematic search and 83 through 4 evidence synthesis documents. Only 19 of all the identified frameworks were developed from the perspective of HTA. These frameworks vary in assessment items, structure, and specificity, and their proven usefulness in practice is scarce.

The thematic analysis of the 26 studies that met the inclusion criteria led to the identification and definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered when evaluating dHTs. Building on our results, a methodological framework for dHTA was proposed.

Acknowledgments

We acknowledge Benigno Rosón Calvo (Servicio Gallego de Salud [SERGAS]), Carme Carrion (Universitat Oberta de Catalunya [UOC]), Carlos A Molina Carrón (Dirección General de Salud Digital y Sistemas de Información para el SNS. Ministerio de Sanidad, Gobierno de España), Carme Pratdepadua (Fundació Tic Salut i Social [FTSS]), Celia Muñoz (Instituto Aragonés de Ciencias de la Salud [IACS]), David Pijoan (Biocat, BioRegió de Catalunya), Felip Miralles (Eurecat – Centre Tecnològic de Catalunya), Iñaki Guiterrez Ibarluzea (Osasun Teknologien Ebaluazioko Zerbitzua [Osteba]), Janet Puñal Riobóo (Unidad de Asesoramiento Científico-técnico [avalia-t], Agencia Gallega para la Gestión del Conocimiento en Salud [ACIS]), Jordi Piera-Jiménez (Àrea de Sistemes d’Informació del Servei Català de la Salut [CatSalut]), Juan Antonio Blasco (Evaluación de Tecnologías Sanitarias de Andalucía [AETSA]), Liliana Arroyo Moliner (Direcció General de Societat Digital, Departament d’Empresa i Treball de la Generalitat de Catalunya), Lilisbeth Perestelo-Perez (Servicio de Evaluación del Servicio Canario de la Salud [SESCS]), Lucía Prieto Remón (IACS), Marifé Lapeña (Dirección General de Salud Digital y Sistemas de Información para el SNS. Ministerio de Sanidad, Gobierno de España), Mario Cárdaba (Insituto de Salud Carlos III [ISCIII]), Montserrat Daban (Biocat, BioRegió de Catalunya), Montserrat Moharra Frances (Agència de Qualitat i Avaluació Sanitàries de Catalunya), and Oscar Solans (CatSalut) for reviewing the protocol of this scoping review (ScR) and the ScR.

This research was framed within the budget of the work plan of the Spanish Network of Health Technology Assessment Agencies, commissioned by the General Directorate of Common Portfolio of Services of the National Health System and Pharmacy.

Authors' Contributions

JS-F and CM-P were responsible for conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, and visualization. RP-P handled conceptualization, methodology, formal analysis, investigation, resources, and writing—original draft. RMV-H handled conceptualization, writing—review and editing, supervision, and project administration.

Conflicts of Interest

None declared.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR) checklist [ 21 ].

Search strategies for each database.

References excluded at the full-text screening stage.

Methodological frameworks included in systematic reviews.

Network of relationships among the codes.

High-resolution image of Figure 2.

• Avanzas P, Pascual I, Moris C. The great challenge of the public health system in Spain. J Thorac Dis. May 2017;9(Suppl 6):S430-S433. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grubanov BS, Ghio D, Goujon A, Kalantaryan S, Belmonte M, Scipioni M. Health and long-term care workforce: demographic challenges and the potential contribution of migration and digital technology. Luxembourg. European Commission; 2021;2045-2322.
• World Health Organization (WHO). Health 2020: a European policy framework and strategy for the 21st century. Denmark. WHO; 2013.
• World Health Organization (WHO). WHO guideline: recommendations on digital interventions for health system strengthening. Geneva. WHO; Jun 6, 2019.
• Scholz N. Addressing health inequalities in the European Union. Luxembourg. European Parliamentary Research Service; 2020.
• Kickbusch I, Agrawal A, Jack A, Lee N, Horton R. Governing health futures 2030: growing up in a digital world—a joint The Lancet and Financial Times Commission. Lancet. Oct 2019;394(10206):1309. [ CrossRef ]
• Reeves JJ, Ayers JW, Longhurst CA. Telehealth in the COVID-19 era: a balancing act to avoid harm. J Med Internet Res. Feb 01, 2021;23(2):e24785. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Richardson S, Lawrence K, Schoenthaler AM, Mann D. A framework for digital health equity. NPJ Digit Med. Aug 18, 2022;5(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• European Union. State of health in the EU: companion report 2021. Luxembourg. European Union; 2022.
• LATITUD: Anàlisi comparativa de models d'atenció no presencial en l'àmbit de la salut. TIC Salut Social. 2020. URL: https://ticsalutsocial.cat/wp-content/uploads/2021/07/latitud.pdf [accessed 2024-03-28]
• Global strategy on digital health 2020-2025. World Health Organization. Aug 18, 2021. URL: https://www.who.int/publications/i/item/9789240020924 [accessed 2024-03-28]
• Ming J, He Y, Yang Y, Hu M, Zhao X, Liu J, et al. Health technology assessment of medical devices: current landscape, challenges, and a way forward. Cost Eff Resour Alloc. Oct 05, 2022;20(1):54. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Rourke B, Oortwijn W, Schuller T. The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care. May 13, 2020;36(3):187-190. [ CrossRef ]
• EUnetHTA Joint Action 2 Work Package 8. HTA Core Model ® version 3.0 2016. EUnetHTA. 2016. URL: https://www.eunethta.eu/wp-content/uploads/2018/03/HTACoreModel3.0-1.pdf [accessed 2024-03-28]
• Puñal-Riobóo J, Baños, Varela LL, Castillo MM, Atienza MG, Ubago PR. Guía para la elaboración y adaptación de informes rápidos. Santiago de Compostela, Madrid. Agencia Gallega para la Gestión del Conocimientoto en Salud. Unidad de Asesoramiento Científico-técnico, avalia-t. Ministerio de Sanidad, Servicios Sociales e Igualdad; 2016.
• Haverinen J, Keränen N, Falkenbach P, Maijala A, Kolehmainen T, Reponen J. Digi-HTA: health technology assessment framework for digital healthcare services. FinJeHeW. Nov 02, 2019;11(4):326-341. [ CrossRef ]
• Hussain MS, Silvera-Tawil D, Farr-Wharton G. Technology assessment framework for precision health applications. Int J Technol Assess Health Care. May 26, 2021;37(1):e67. [ CrossRef ]
• National Institute for Health and Care Excellence (NICE). Evidence standards framework (ESF) for digital health technologies. Contract no.: 30 de septiembre. London. NICE; 2022.
• Evidence standards framework (ESF) for digital health technologies. National Institute for Health and Care Excellence. URL: https:/​/www.​nice.org.uk/​about/​what-we-do/​our-programmes/​evidence-standards-framework-for-digital-health-technologies [accessed 2024-03-28]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo RM. Methodological frameworks and dimensions to be taken into consideration in digital health technology assessment: protocol for a scoping review. JMIR Res Protoc. Oct 11, 2022;11(10):e39905. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peters M, Marnie C, Tricco A, Pollock D, Munn Z, Alexander L. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119-2126. [ CrossRef ]
• Pollock D, Davies EL, Peters MDJ, Tricco AC, Alexander L, McInerney P, et al. Undertaking a scoping review: a practical guide for nursing and midwifery students, clinicians, researchers, and academics. J Adv Nurs. Apr 04, 2021;77(4):2102-2113. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ayiku L, Hudson T, Glover S, Walsh N, Adams R, Deane J, et al. The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps. Int J Technol Assess Health Care. Oct 27, 2020;37(1):e16. [ CrossRef ]
• McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. Jul 2016;75:40-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. Dec 06, 2017;6(1):245. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc. 2015;13(3):179-187. [ CrossRef ]
• Systematic reviews - research guide. Defining your review question. Murdoch University. 2023. URL: https://libguides.murdoch.edu.au/systematic [accessed 2024-03-28]
• Scientific Software Development GmbH. Atlas.ti qualitative data analysis. 22.0 ed. Berlin. Scientific Software Development GmbH; 2021.
• Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane handbook for systematic reviews of interventions version 6.3. London. Cochrane; 2022.
• Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. Jul 10, 2008;8(1):45. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. Mar 2021;18(3):e1003583. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kidholm K, Ekeland AG, Jensen LK, Rasmussen J, Pedersen CD, Bowes A, et al. A model for assessment of telemedicine applications: MAST. Int J Technol Assess Health Care. Jan 23, 2012;28(1):44-51. [ CrossRef ]
• Nepal S, Li J, Jang-Jaccard J, Alem L. A framework for telehealth program evaluation. Telemed J E Health. Apr 2014;20(4):393-404. [ CrossRef ] [ Medline ]
• Measuring the impact of telehealth and telecare: SCTT toolkit. Glasgow. SCTT; 2013.
• Caulfield B, Reginatto B, Slevin P. Not all sensors are created equal: a framework for evaluating human performance measurement technologies. NPJ Digit Med. Feb 14, 2019;2(1):7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lewis TL, Wyatt JC. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. Sep 15, 2014;16(9):e210. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Moshi MR, Tooher R, Merlin T. Development of a health technology assessment module for evaluating mobile medical applications. Int J Technol Assess Health Care. May 18, 2020;36(3):252-261. [ CrossRef ]
• Puigdomènech E, Poses-Ferrer E, Espallargues CM, Blasco AJ, Varela LL, Paz VL. Evaluación de tecnología basada en mSalud para aplicaciones móviles. Madrid. Ministerio de Sanidad; 2020.
• Henson P, David G, Albright K, Torous J. Deriving a practical framework for the evaluation of health apps. Lancet Digit Health. Jun 2019;1(2):e52-e54. [ CrossRef ]
• Reddy S, Rogers W, Makinen V, Coiera E, Brown P, Wenzel M, et al. Evaluation framework to guide implementation of AI systems into healthcare settings. BMJ Health Care Inform. Oct 12, 2021;28(1):e100444. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Khoja S, Durrani H, Scott RE, Sajwani A, Piryani U. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed J E Health. Jan 2013;19(1):48-53. [ CrossRef ] [ Medline ]
• Sockolow P, Bowles K, Rogers M. Health information technology evaluation framework (HITREF) comprehensiveness as assessed in electronic point-of-care documentation systems evaluations. Amsterdam. IOS Press; 2015.
• Kowatsch T, Otto L, Harperink S, Cotti A, Schlieter H. A design and evaluation framework for digital health interventions. it - Inf Technol. Nov 2019;61(5):253-263. [ CrossRef ]
• Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB. Digital health: a path to validation. NPJ Digit Med. May 13, 2019;2(1):38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Milosevic Z. Ethics in digital health: a deontic accountability framework. 2019. Presented at: IEEE 23rd International Enterprise Distributed Object Computing Conference (EDOC); October 28-31, 2019; Paris, France. [ CrossRef ]
• World Health Organization (WHO). Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Geneva. WHO; 2016.
• Enam A, Torres-Bonilla J, Eriksson H. Evidence-based evaluation of eHealth interventions: systematic literature review. J Med Internet Res. Nov 23, 2018;20(11):e10971. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kolasa K, Kozinski G. How to value digital health interventions? A systematic literature review. Int J Environ Res Public Health. Mar 23, 2020;17(6):2119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vis C, Bührmann L, Riper H, Ossebaard HC. Health technology assessment frameworks for eHealth: a systematic review. Int J Technol Assess Health Care. Apr 16, 2020;36(3):204-216. [ CrossRef ]
• von Huben A, Howell M, Howard K, Carrello J, Norris S. Health technology assessment for digital technologies that manage chronic disease: a systematic review. Int J Technol Assess Health Care. May 26, 2021;37(1):e66. [ CrossRef ]
• Report of the Working Group on mhealth assessment guidelines. European Commission. 2021. URL: https://digital-strategy.ec.europa.eu/en/library/report-working-group-mhealth-assessment-guidelines [accessed 2024-03-28]
• Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, et al. Mobile health technology evaluation: the mhealth evidence workshop. Am J Prev Med. Aug 2013;45(2):228-236. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alami H, Lehoux P, Auclair Y, de Guise M, Gagnon M, Shaw J, et al. Artificial intelligence and health technology assessment: anticipating a new level of complexity. J Med Internet Res. Jul 07, 2020;22(7):e17707. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vokinger KN, Nittas V, Witt CM, Fabrikant SI, von Wyl V. Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Med Wkly. May 04, 2020;150(1920):w20282. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Baumel A, Muench F. Heuristic evaluation of ehealth interventions: establishing standards that relate to the therapeutic process perspective. JMIR Ment Health. Jan 13, 2016;3(1):e5. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alfonzo A, Huerta M, Wong S, Passariello G, Diaz M, La CA, et al. Design of a methodology for assessing an electrocardiographic telemonitoring system. Annu Int Conf IEEE Eng Med Biol Soc. 2007;2007:3729-3732. [ CrossRef ]
• Bashshur R, Shannon G, Sapci H. Telemedicine evaluation. Telemed J E Health. Jun 2005;11(3):296-316. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Beintner I, Vollert B, Zarski A, Bolinski F, Musiat P, Görlich D, et al. Adherence reporting in randomized controlled trials examining manualized multisession online interventions: systematic review of practices and proposal for reporting standards. J Med Internet Res. Aug 15, 2019;21(8):e14181. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brear M. Evaluating telemedicine: lessons and challenges. Health Inf Manag. Jul 21, 2006;35(2):23-31. [ CrossRef ] [ Medline ]
• DeChant HK, Tohme WG, Mun SK, Hayes WS, Schulman KA. Health systems evaluation of telemedicine: a staged approach. Telemed J. Jan 1996;2(4):303-312. [ CrossRef ] [ Medline ]
• Giansanti D, Morelli S, Macellari V. Telemedicine technology assessment part II: tools for a quality control system. Telemed J E Health. Apr 2007;13(2):130-140. [ CrossRef ] [ Medline ]
• Giansanti D, Morelli S, Macellari V. Telemedicine technology assessment part I: setup and validation of a quality control system. Telemed J E Health. Apr 2007;13(2):118-129. [ CrossRef ] [ Medline ]
• Grigsby J, Brega AG, Devore PA. The evaluation of telemedicine and health services research. Telemed J E Health. Jun 2005;11(3):317-328. [ CrossRef ] [ Medline ]
• Hailey D, Jacobs P, Simpson J, Doze S. An assessment framework for telemedicine applications. J Telemed Telecare. Jun 23, 1999;5(3):162-170. [ CrossRef ] [ Medline ]
• Ohinmaa A, Hailey D, Roine R. Elements for assessment of telemedicine applications. Int J Technol Assess Health Care. Jun 30, 2001;17(2):190-202. [ CrossRef ] [ Medline ]
• Rajan B, Tezcan T, Seidmann A. Service systems with heterogeneous customers: investigating the effect of telemedicine on chronic care. Manag Sci. Mar 2019;65(3):1236-1267. [ CrossRef ]
• Sisk JE, Sanders JH. A proposed framework for economic evaluation of telemedicine. Telemed J. Jan 1998;4(1):31-37. [ CrossRef ] [ Medline ]
• Zissman K, Lejbkowicz I, Miller A. Telemedicine for multiple sclerosis patients: assessment using Health Value Compass. Mult Scler. Apr 30, 2012;18(4):472-480. [ CrossRef ] [ Medline ]
• Grustam AS, Vrijhoef HJM, Koymans R, Hukal P, Severens JL. Assessment of a business-to-consumer (B2C) model for telemonitoring patients with chronic heart failure (CHF). BMC Med Inform Decis Mak. Oct 11, 2017;17(1):145. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hebert M. Telehealth success: evaluation framework development. Stud Health Technol Inform. 2001;84(Pt 2):1145-1149. [ Medline ]
• Rojahn K, Laplante S, Sloand J, Main C, Ibrahim A, Wild J, et al. Remote monitoring of chronic diseases: a landscape assessment of policies in four European countries. PLoS One. May 19, 2016;11(5):e0155738. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, et al. WHO mHealth Technical Evidence Review Group. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ. Mar 17, 2016;352:i1174. [ CrossRef ] [ Medline ]
• Safety and quality strategy in mobile health apps. Complete list of recommendations on design, use and assessment of health apps. Agencia de Calidad Sanitaria de Andalucía. 2012. URL: http://www.calidadappsalud.com/en/listado-completo-recomendaciones-app-salud/ [accessed 2023-07-17]
• The app evaluation model. American Psychiatric Association Initiative. 2022. URL: https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/the-app-evaluation-model [accessed 2024-03-28]
• Gdd AppStore. Association of Regional Public Health Services (GGD) and Regional Medical Emergency Preparedness and Planning (GHOR). 2016. URL: https://tinyurl.com/58th4p4w [accessed 2023-07-18]
• AppQ: quality criteria core set for more quality transparency in digital health applications. Bertelsmann Stiftung. Oct 29, 2019. URL: https://www.bertelsmann-stiftung.de/de/publikationen/publikation/did/appq [accessed 2024-03-28]
• Bradway M, Carrion C, Vallespin B, Saadatfard O, Puigdomènech E, Espallargues M, et al. mHealth assessment: conceptualization of a global framework. JMIR Mhealth Uhealth. May 02, 2017;5(5):e60. [ FREE Full text ] [ CrossRef ] [ Medline ]
• PAS 277:2015 - health and wellness apps. Quality criteria across the life cycle. British Standards Institution. URL: https:/​/knowledge.​bsigroup.com/​products/​health-and-wellness-apps-quality-criteria-across-the-life-cycle-code-of-practice/​standard [accessed 2023-07-18]
• MindApps. Centre for Telepsychiatry in the Region of Southern Denmark. 2017. URL: https://mindapps.dk/ [accessed 2023-07-18]
• Dick S, O'Connor Y, Thompson MJ, O'Donoghue J, Hardy V, Wu TJ, et al. Considerations for improved mobile health evaluation: retrospective qualitative investigation. JMIR Mhealth Uhealth. Jan 22, 2020;8(1):e12424. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Federal Institute for Drugs and Medical Devices (BfArM). The fast-track process for digital health applications (diga) according to section 139e sgb v. A guide for manufacturers, service providers and users. Bonn, Germany. Federal Institute for Drugs and Medical Devices; 2020.
• Gorski I, Bram JT, Sutermaster S, Eckman M, Mehta K. Value propositions of mHealth projects. J Med Eng Technol. Aug 12, 2016;40(7-8):400-421. [ CrossRef ] [ Medline ]
• Hogaboam L, Daim T. Technology adoption potential of medical devices: the case of wearable sensor products for pervasive care in neurosurgery and orthopedics. Health Policy Technol. Dec 2018;7(4):409-419. [ CrossRef ]
• Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. Apr 05, 2019;2(4):e192542. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maar MA, Yeates K, Perkins N, Boesch L, Hua-Stewart D, Liu P, et al. A framework for the study of complex mhealth interventions in diverse cultural settings. JMIR Mhealth Uhealth. Apr 20, 2017;5(4):e47. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McMillan B, Hickey E, Patel MG, Mitchell C. Quality assessment of a sample of mobile app-based health behavior change interventions using a tool based on the National Institute of Health and Care Excellence behavior change guidance. Patient Educ Couns. Mar 2016;99(3):429-435. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Health and welness apps: new international guidelines to help to sort the best form the rest. NEN. URL: https://www.nen.nl/en/health-and-welness-apps [accessed 2024-03-28]
• Continua design guidelines. Personal Connected Health Alliance. 2019. URL: https://www.pchalliance.org/continua-design-guidelines [accessed 2023-07-18]
• Philpott D, Guergachi A, Keshavjee K. Design and validation of a platform to evaluate mhealth apps. Stud Health Technol Inform. 2017;235:3-7. [ Medline ]
• Ruck A, Wagner BS, Lowe C. Second draft of guidelines. EU guidelines on assessment of the reliability of mobile health applications. European Commission, Directorate-General of Communications Networks, Content & Technology. Luxembourg; 2016.
• Sax M, Helberger N, Bol N. Health as a means towards profitable ends: mhealth apps, user autonomy, and unfair commercial practices. J Consum Policy. May 22, 2018;41(2):103-134. [ CrossRef ]
• Wyatt JC. How can clinicians, specialty societies and others evaluate and improve the quality of apps for patient use? BMC Med. Dec 03, 2018;16(1):225. [ FREE Full text ] [ CrossRef ] [ Medline ]
• IRBs could address ethical issues related to tracking devices: mobile devices raise new concerns. IRB Advisor. Nov 1, 2017. URL: https:/​/www.​reliasmedia.com/​articles/​141589-irbs-could-address-ethical-issues-related-to-tracking-devices [accessed 2024-03-28]
• App check. Zentrum für Telematik und Telemedizin. URL: https://ztg-nrw.de/ [accessed 2024-03-28]
• Bergmo TS. How to measure costs and benefits of ehealth interventions: an overview of methods and frameworks. J Med Internet Res. Nov 09, 2015;17(11):e254. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. Dec 31, 2011;13(4):e126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shaw NT. 'CHEATS': a generic information communication technology (ICT) evaluation framework. Comput Biol Med. May 2002;32(3):209-220. [ CrossRef ] [ Medline ]
• Brown M, Shaw N. Evaluation practices of a major Canadian telehealth provider: lessons and future directions for the field. Telemed J E Health. Oct 2008;14(8):769-774. [ CrossRef ] [ Medline ]
• Casper GR, Kenron DA. A framework for technology assessment: approaches for the selection of a home technology device. Clin Nurse Spec. 2005;19(4):170-174. [ CrossRef ] [ Medline ]
• Sitting D, Kahol K, Singh H. Sociotechnical evaluation of the safety and effectiveness of point-of-care mobile computing devices: a case study conducted in India. Stud Health Technol Inform. 2013;192:515-519. [ CrossRef ]
• Haute Autorité de Santé. Good practice guidelines on health apps and smart devices (mobile health or mhealth). Paris. Haute Autorité de Santé; Nov 7, 2016.
• Health Information and Quality Authority. International review of consent models for the collection, use and sharing of health information. Dublin. Health Information and Quality Authority; 2020.
• Jurkeviciute M. Planning of a holistic summative ehealth evaluation: the interplay between standards and reality PhD Thesis. Gotherburg. Chalmers University of Technology; 2018.
• Vimarlund V, Davoody N, Koch S. Steps to consider for effective decision making when selecting and prioritizing eHealth services. Stud Health Technol Inform. 2013;192:239-243. [ Medline ]
• Currie WL. TEMPEST: an integrative model for health technology assessment. Health Policy Technol. Mar 2012;1(1):35-49. [ CrossRef ]
• Eivazzadeh S, Anderberg P, Larsson TC, Fricker SA, Berglund J. Evaluating health information systems using ontologies. JMIR Med Inform. Jun 16, 2016;4(2):e20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Our data-driven future in healthcare: people and partnerships at the heart of health related technologies. Academy of Medical Sciences. Nov 2018. URL: https://acmedsci.ac.uk/file-download/74634438 [accessed 2024-03-28]
• Australian Commission on Safety and Quality in Healthcare. National safety and quality digital mental health standards - consultation draft. Australia. Australian Commission on Safety and Quality in Healthcare; 2020.
• A guide to good practice for digital and data-driven health technologies. London. Department of Health & Social Care; 2021.
• Drury P, Roth S, Jones T, Stahl M, Medeiros D. Guidance for investing in digital health. In: Sustainable Development Working Papers. Mandaluyong, the Philippines. Asian Development Bank; May 2018.
• European Commission. Synospis report. Consultation: transformation health and care in the digital single market. Luxembourg. European Commission; 2018.
• Federal Ministry of Health. Regulation on the procedure and requirements for testing the eligibility for reimbursement of digital health applications in the statutory public health insurance (Digital Health Applications Ordinance - DiGAV). Alemania. Federal Ministry of Health; 2020.
• Haute Autorité de Santé. Guide to the specific features of clinical evaluation of a connected medical device (CMD) in view of its application for reimbursement. Paris. Haute Autorité de Santé; 2019.
• How we assess health apps and digital tools. NHS Digital. 2019. URL: https:/​/digital.​nhs.uk/​services/​nhs-apps-library/​guidance-forhealth-app-developers-commissioners-and-assessors/​how-we-assess-healthapps-and-digital-tools [accessed 2023-07-17]
• Lennon MR, Bouamrane M, Devlin AM, O'Connor S, O'Donnell C, Chetty U, et al. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. Feb 16, 2017;19(2):e42. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McNamee P, Murray E, Kelly MP, Bojke L, Chilcott J, Fischer A, et al. Designing and undertaking a health economics study of digital health interventions. Am J Prev Med. Nov 2016;51(5):852-860. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Haute Autorité de Santé. Medical Device and Health Technology Evaluation Committee (CNEDiMTS*). Paris. Haute Autorité de Santé; 2019.
• Draft guidelines for preparing assessment reports for the Medical Services Advisory Committee: draft version 4.0. Australian Government Department of Health and Aged Care. Aug 2020. URL: https:/​/consultations.​health.gov.au/​technology-assessment-access-division/​msac-guidelines-review-consultation/​user_uploads/​draft-msac-guidelines---clean-version---28-august-2020-3.​pdf [accessed 2024-03-28]
• Haute Autorité de Santé. Methodological choices for the clinical development of medical devices. Paris. Haute Autorité de Santé; 2013.
• Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res. Jun 29, 2017;19(6):e232. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. Jul 08, 2015;17(7):e166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. Nov 2016;51(5):843-851. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Steventon A, Grieve R, Bardsley M. An approach to assess generalizability in comparative effectiveness research: a case study of the whole systems demonstrator cluster randomized trial comparing telehealth with usual care for patients with chronic health conditions. Med Decis Making. May 18, 2015;35(8):1023-1036. [ CrossRef ]
• D2.1 Knowledge Tool 1. Health apps assessment frameworks. European mHealth Hub. 2020. URL: https://mhealth-hub.org/download/d2-1-knowledge-tool-1-health-apps-assessment-frameworks [accessed 2024-03-28]
• Moshi MR, Tooher R, Merlin T. Suitability of current evaluation frameworks for use in the health technology assessment of mobile medical applications: a systematic review. Int J Technol Assess Health Care. Sep 11, 2018;34(5):464-475. [ CrossRef ]
• Stensgaard T, Sørensen T. Telemedicine in Greenland — the creation of an evaluation plan. J Telemed Telecare. Jun 22, 2016;7(1_suppl):37-38. [ CrossRef ]
• HL7 Consumer Mobile Health Application Functional Framework (cMHAFF). HL7 International. 2018. URL: http://www.hl7.org/implement/standards/product_brief.cfm?product_id=476 [accessed 2024-03-28]
• mHealth. Kantonen K-uKvBu. 2017. URL: https://www.e-health-suisse.ch/gemeinschaften-umsetzung/ehealth-aktivitaeten/mhealth.html [accessed 2023-07-18]
• mHealthBelgium. mHealthBELGIUM. URL: https://mhealthbelgium.be/ [accessed 2024-03-28]
• Mookherji S, Mehl G, Kaonga N, Mechael P. Unmet need: improving mhealth evaluation rigor to build the evidence base. J Health Commun. Jun 04, 2015;20(10):1224-1229. [ CrossRef ] [ Medline ]
• MySNS Selecção. Serviços Partilhados do Minisério da Saúde. URL: https://mysns.min-saude.pt/mysns-seleccao-processo-de-avaliacao/ [accessed 2023-07-18]
• Nielsen S, Rimpiläinen S. Report on international practice on digital apps. Glasgow. Digital Health & Care Institute; 2018.
• Accreditation of Digital Health solutions is a fundamental foundation for their safe adoption, equipping healthcare providers and practitioners with access to health apps assured to your standards Internet. Organisation for the Review of Care and Health Applications (ORCHA). URL: https://orchahealth.com/services/ [accessed 2023-07-18]
• mHealth. TIC Salut Social. 2022. URL: https://ticsalutsocial.cat/projecte/mhealth/ [accessed 2024-03-28]
• Wienert J, Jahnel T, Maaß L. What are digital public health interventions? First steps toward a definition and an intervention classification framework. J Med Internet Res. Jun 28, 2022;24(6):e31921. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Deniz-Garcia A, Fabelo H, Rodriguez-Almeida AJ, Zamora-Zamorano G, Castro-Fernandez M, Alberiche Ruano MDP, et al. WARIFA Consortium. Quality, usability, and effectiveness of mhealth apps and the role of artificial intelligence: current scenario and challenges. J Med Internet Res. May 04, 2023;25:e44030. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World Health Organization (WHO). Classification of digital interventions, services and applications in health: a shared language to describe the uses of digital technology for health, 2nd ed. Geneva. WHO; Oct 24, 2023.
• European Parliament, Council of the European Union. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. EUR-Lex. 2017. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32017R0745 [accessed 2024-03-28]
• European Parliament, Council of the European Union. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU. EUR-Lex. 2017. URL: https://eur-lex.europa.eu/eli/reg/2017/746/oj [accessed 2024-03-28]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo R. Marco de evaluación de tecnologías sanitarias: adaptación para la evaluación de tecnologías de salud digital. Madrid, Barcelona. Ministerio de Sanidad, Agència de Qualitat i Avaluació Sanitàries de Catalunya; 2023.
• Benedetto V, Filipe L, Harris C, Spencer J, Hickson C, Clegg A. Analytical frameworks and outcome measures in economic evaluations of digital health interventions: a methodological systematic review. Med Decis Making. Oct 19, 2022;43(1):125-138. [ CrossRef ]

Abbreviations

Edited by T Leung; submitted 03.05.23; peer-reviewed by R Gorantla, KL Mauco, M Aymerich, J Haverinen, M Behzadifar; comments to author 10.11.23; revised version received 01.12.23; accepted 20.02.24; published 10.04.24.

©Joan Segur-Ferrer, Carolina Moltó-Puigmartí, Roland Pastells-Peiró, Rosa Maria Vivanco-Hidalgo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Open access
• Published: 09 April 2024

Effects of combination of strength and balance training on postural control and functionality in people with chronic ankle instability: a systematic review and meta analysis

• Yuying Su 1 , 2 ,
• Changbo Pan 1 &

BMC Sports Science, Medicine and Rehabilitation volume  16 , Article number:  79 ( 2024 ) Cite this article

107 Accesses

Metrics details

To identify the effects of strength and balance training on dynamic balance and patient reported outcomes in people with chronic ankle instability(CAI).

Five databases(CNKI, WanFang, Web of Science, EBSCO-SPORTD and PubMed were searched in September 2022. The search was conducted on randomized controlled trials(RCTs) that the effects of strength training, balance training and combination of strength and balance training in people with chronic ankle instability compared to a control group. Using Review Manager 5.3 and Stata-SE 15 to conduct Meta-analysis on the included literature. methodological quality and risk of bias were assessed by using the PEDro scale.

A total of 33 Chinese and English RCTs document were screened and 1154 patients with CAI were included in the study. Compared with control group, strength training, balance training and combination of strength and balance training demonstrated to be more effective in terms of improving patient reported outcomes(strength training: SMD = 0.80, 95%CI = 0.39–1.22; balance training: SMD = 0.79, 95%CI = 0.41–1.17; combination of strength and balance training: SMD = 1.28, 95%CI = 0.57, 1.99). Subgroup analysis: Intervention for 6 weeks, more than 3 times a week and more than 30 min each time were the best rehabilitation programs to improve CAI patientreported outcomes. Compared with control group, balance training demonstrated to be more effective in terms of improving Star Excursion Balance Test (SEBT)((anterior: SMD = 0.71, 95%CI = 0.03–1.40; posterolateral: SMD = 0.84, 95%CI = 0.22–1.46; posteromedial: SMD = 0.88, 95%CI = 0.45–1.32). However, strength training and combination of strength and balance training had no improvement effects on SEBT.

Conclusions

Available evidence showed that, results of the comparison between balance training versus strength training suggest that the combination of strength and balance training achieves greater benefits for patient reported outcomes and balance training could bring greater benefits to dynamic balance. Strength training should be used cautiously in clinic to improve the dynamic balance in individuals with CAI.

Trial registration

( http://www.crd.york.ac.uk/PROSPERO , Registration No. CRD42022371396).

Peer Review reports

Introduction

Ankle sprains are one of the most common types of injuries in daily life, often occurring in acute sports events such as basketball and soccer, as well as in daily rough roads. Among the sports injuries with data statistics, 25% of sports injuries were caused by ankle sprain in different degrees [ 1 ]. Minor ankle sprains can be effectively treated with non-surgical conservative treatment [ 2 ]. However, because many patients do not pay enough attention to ankle sprain in the rehabilitation stage or lack of necessary exercise rehabilitation means, as many as 70% of patients has been habitual sprain and instability in the later stage, and 40%-50% of patients would develop chronic ankle instability (CAI) [ 2 , 3 ]. CAI refers to the structural or functional deficiencies in the ankle joint and surrounding tissues, resulting in ankle joint instability and limited joint movement, with recurrent sprains as the main characteristic [ 4 ]. The main clinical symptoms of CAI include muscle weakness, persistent pain, loss of control, ligament laxity, functional decline, and repeated sprains, accompanied by cartilage damage and synovitis, severely affecting the quality of daily life [ 5 ].

Generally, CAI could be rehabilitated by conservative methods such as ankle fixation, physical therapy and exercise therapy. Exercise therapy is an important treatment modality for the rehabilitation of CAI in the later stage [ 6 ]. Studies had shown that dynamic balance, proprioception, fibular reaction time and lack of valgus strength were the main cause of CAI symptoms [ 7 ]. Strength and balance training were the most commonly used in rehabilitation training. The aim of strength and balance training rehabilitation is to correct modifiable deficits such as reduced muscle strength, decreased neuromuscular control, impaired proprioception, altered gait pattern, and restricted range of motion that are observed in individuals with CAI [ 8 , 9 ].

Previous research has suggested some benefits from strength and balance training for improving dynamic balance and patient reported outcomes in individuals with CAI [ 10 , 11 , 12 ]. But, the comparison of effectiveness between strength training, balance training and combination of strength and balance training on dynamic balance and self-reported function for CAI are still unclear [ 8 ]. We need to evaluate the impact of three types of exercise interventions on dynamic balance and self-reported function. We also need to consider which exercise mode, intervention load, intervention time, and intervention cycle can better treat chronic ankle instability symptoms. These factors are important for clinical development of exercise rehabilitation programs [ 13 ]. In view of the above, the aim of this study was to systematically review the available evidence on the effects of the various therapeutic physical exercise interventions (i.e. strength training, balance training and combination of strength and balance training) in individuals with CAI. Furthermore, meta-analysis was conducted to determine the most effective treatment for improving patient reported outcomes and dynamic balance in this population.

Data sources and search strategies

The literature search for this study was conducted independently and blindly by two researchers, strictly following the PRISMA statement for meta-analyses. The search databases included Wanfang, CNKI, Web of Science, PubMed, and EBSCO-SPORTD. The last search was conducted on September 23, 2022. The literature data is retrieved in both Chinese and English. A secondary search of the reference lists of selected articles was conducted to prevent the omission of important literature. This study was registered in the Research Registry ( http://www.crd.york.ac.uk/PROSPERO , Registration No. CRD42022371396). Derived from the explosion search, the final search strategy was(“chronic ankle instability” OR “ankle instability” OR “function ankle instability”) AND (“exercise intervention” OR “strength training” OR “balance training” OR “neuromuscular control training”).

Selection criteria

The studies were included in this review if they met the following criteria:

(1) Participants: individuals with CAI, According to the standards of the International Ankle Federation, patients who meet the following conditions are included as CAI patients: 1) history of at least 1 ankle sprain(excluding sprains within 3 months), 2) history of habitual sprains or instability, 3) self-reported ankle instability or function confirmed by a validated questionnaire.

(2) Interventions: strength training(elastic bands or resistance exercises); balance training(single leg balance or balance board or proprioception); combination of strength and balance training.

(3) Comparators: control(no exercise) or maintain daily life.

(4) Outcomes: There was no restriction on the measurement of results, but this meta-analysis focused on the star excursion balance text(SEBT) and self-reported function(FAAM, CAIT, FADI, AOFAS).

(5) Study design: randomized controlled trials (RCTs).

Articles were excluded if: (1) Trial conducted in animals and acute ankle sprain; (2) Articles that were a case report or detection or survey or not published as peer-reviewed journal articles, such as book chapters and conference abstracts.

Data extraction

The process of data extraction was conducted independently by two authors (YY.S. and W.L) according to the Cochrane Collaboration Handbook. The data were extracted as follows: basic information (first author, year of publication, characteristics of participants), specific information (measures of intervention, intensity, frequency and duration) and outcome measure appropriate for analysis(SEBT, FAAM, AOFAS, CAIT, FADI and AJFAT scores after intervention). Disagreements were resolved by discussion with a third author (Y.S).

For each included study, the mean and the SD of test and follow-up tests were extracted. If any relevant data was missing, we tried to contact the corresponding author or other authors of that study via email to request it.

Quality assessment

Literature quality evaluation was conducted using the PEDro scale, which is reliable for evaluating the quality of RCTs and and assessing the risk of bias [ 14 ]. The PEDro scale had a total of 10 points (1 point if the corresponding indicators were met, 0 point if the indicators were not met, and the first question were not included in the total score). The results were independently reviewed by two reviewers. Studies with a score above 6 are considered as high quality, and scores of less than 6 for studies would reflect the greater potential for biases to affect results of trial.

Data synthesis and analysis

The included literature was analyzed using Stata-SE 15 and Revman 5.3 software, including effect sizes, publication bias, subgroup analysis and sensitivity analysis. The results of this study were calculated by comparing the average scores of the experimental group and the control group after the intervention. The standardized mean differences (SMD) and SD with 95% CI were calculated for continuous data. Effect sizes were classified as trivial (< 0.2), small (0.2 ~ 0.5), moderate (0.5 ~ 0.8), or large (> 0.8). A randomized effects model was applied, and the inverse variance method was used. The statistical heterogeneity was evaluated using heterogeneity chi-squared (χ 2 ) and I 2 values. The level of heterogeneity was interpreted according to the guidelines from Cochrane,s collaboration: I 2 values of 25, 50, and 75% correspond to low, moderate, and high heterogeneity, respectively [ 15 ]. When I 2  ≥ 50%, sensitivity analysis should be performed to test the stability of the results and subgroup analysis should be conducted based on the characteristics of the literature. In addition, We used Egger's test and funnel plot to quantitatively analyze publication bias of the included studies. If a publication bias is observed, the trim and fill methods are used to adjust the estimation points.

Literature search and screening

According to PRISMA guidelines, the detailed selection process of these trials were showed in Fig.  1 . From the electronic retrieval, a total of 982 relevant studies were obtained from 5 databases, and 597 articles were defined as unqualified after screening by title and abstract. After reading the full text, 33 RCTs were included for Meta analysis, including 11 Chinese literature and 22 English literature [ 10 , 11 , 12 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ] (Table  1 ).

Flowchart of Literature Search and Study Selection

Characteristics of included studies

Participant characteristics.

In the included literature, a total of 1154 subjects were included (Table  1 ). There were 646 people in the experimental group and 508 people in the control group. Including 633 male subjects and 434 female subjects, However, 3 studies did not state gender [ 18 , 24 , 40 ]. The age distribution of the subjects is between 18–50 years old, and they are all healthy adults.

Intervention characteristics

The information of the intervention parameters is included in Table  1 . Six studies in the experimental group intervention program focused on strength training; twenty studies focused on balance training and Twelve studies focused on combination of strength and balance training. Control group interventions consisted of maintenance of daily routine, no intervention, or strength and balance training. Intervention frequency of 4–12 weeks, 2–6 times a week and 20–60 min each time included in the studies. The main outcome measures of dynamic balance is SEBT and self-reported function score includes AOFAS, CAIT, FADI, AJFAT, FAAM-S and FADI-S.

Quality assessment of the included studies

Two reviewers independently evaluated the quality of assessment of the included studies. The scores were evaluated according to 9–10 points as high-quality literature, 6–8 points as higher-quality literature, 4–5 as general quality literature [ 14 ]. In all 33 literature, 27 higher-quality papers with a score of 6 or above, and 6 general quality papers. Most of the literature are concentrated between 6–8 points, with only one high-quality literature. The conditions for including subjects in the literature are relatively clear, and allocation concealment and blinding methods are rarely used in Chinese literature. Overall, the quality assessment of the included studies are higher in Table  2 .

Meta-analysis: comparison of strength, balance and combination training VS. control on self-reported function score

The results the self-reported function score of strength, balance and combination training included 31 studies, including 907 in individuals with CAI. Figure  2 shows significant differences in self-reported function score (SMD = 0.93, 95%CI = 0.62 to 1.24, p  < 0.05) between the strength, balance and combination training group and control group in this study where this outcome was addressed.

Comparison of the effects of strength, balance and combination training versus control training on self-reported function score

Table 3 shows the subgroup analysis of different covariates on self-reported function score. Strength training (SMD = 0.80, 95%CI = 0.39 to 1.22, p  < 0.05), balance training (SMD = 0.79, 95%CI = 0.41 to 1.17, p  < 0.05) and combined training (SMD = 1.28, 95%CI = 0.57 to 1.99, p  < 0.05) can significantly improve the self-reported function score of CAI patients. Combined training improved to a greater extent compared to strength training and balance training. The subgroups of duration and frequency showed that it was best to improve the function score by 6 weeks of intervention, more than 3 times a week and more than 30 min every exercise.

The heterogeneity I 2  = 79% in this study, so through sensitivity analysis discussed whether a certain study has a greater impact on the whole. Figure  3 shows that excluding a certain study has little impact on the overall heterogeneity, and the Meta analysis results are stable.

Sensitivity Analysis of exercise intervention on Functional Score of CAI Patients

Meta-analysis: comparison of strength, balance and combination training VS. control on dynamic balance

Figs  4 , 5 and 6 shows significant differences in A (SMD = 0.55, 95%CI = 0.14 to 0.96, p  < 0.05), PL (SMD = 0.78, 95%CI = 0.23 to 1.33, p  < 0.05), and PM (SMD = 0.63, 95%CI = 0.32 to 0.94, p  < 0.05) between the strength, balance and combination training group and control group in this study where this outcome was addressed.

Comparison of the effects of strength, balance and combination training versus control training on SEBT-A

Comparison of the effects of strength, balance and combination training versus control training on SEBT-PL

Comparison of the effects of strength, balance and combination training versus control training on SEBT-PM

Table 4 shows the subgroup analysis of different Covariates on dynamic balance. Strength training can effectively improve SEBT-A, but has no effect on SEBT-PL and SEBT-PM. Balance training can effectively improve the three directions of SEBT-A, SEBT-PL and SEBT-PM, and can achieve moderate effect size. Combination training has no significant improvement effect in three directions. The intervention duration of 6 weeks, three times a week and less than 20 min each exercise were the best combination to improve the dynamic balance of CAI patients.

Publication bias

The funnel plot and Egger test were used to evaluate the publication bias. No indication of asymmetry or publication bias was found in funnel plot (Figs.  7 , 8 , 9 and 10 ). The Egger test in Table  5 showed that there was no publication bias ( p  > 0.05) in the self-report function ( p  = 0.147), SEBT-A ( p  = 0.204), SEBT-PL ( p  = 0.158), and SEBT-PM ( p  = 0.331).

Funnel plot for analyzing the outcome of the self-reported function score, which is symmetrical and indicates no bias of included studies

Funnel plot for analyzing the outcome of the SEBT-A, which is symmetrical and indicates no bias of included studies

Funnel plot for analyzing the outcome of the SEBT-PL, which is symmetrical and indicates no bias of included studies

Funnel plot for analyzing the outcome of the SEBT-PM, which is symmetrical and indicates no bias of included studies

Patient reported outcomes

Self-reported function can diagnose the rehabilitation effect of CAI patients, generally including pains, swellings, losses of control and other symptoms, as well as the performance of daily life related to running, jumping, landing and squatting. The clinical evaluation of CAI rehabilitation effect by patient reported outcomes has the advantages of saving time and being fast and effective. The scale has been proved to have high test efficiency [ 4 , 46 ]. A total of 31 literatures including 907 CAI patients were included in the patient reported outcomes. The strength training, balance trainging, and combined strength and balance training can significantly improve the chronic ankle instability self-reported function compared to the control group. Combined strength and balance training improved to a greater extent compared to strength training and balance training. The best exercise intervention for improving self-reported function is to exercise for 6 weeks, more than 3 times a week and more than 30 min every exercise. Chronic ankle instability often presents with deficits in neuromuscular control, proprioception, and strength disorders [ 47 ]. These symptoms can affect people's daily quality of life, limit physical exercise, and cause pain [ 11 ]. Because of the incidence rate and commonness of chronic ankle instability, people try to improve it by providing effective technology. Physical exercise is an effective physical method for treating chronic ankle instability, and balance [ 45 , 48 ] and strength [ 36 , 49 ] are common exercise interventions. So this study can provide guidance and recommendations for clinical treatment and design.

Compared to previous literature, previous studies have investigated the effects of balance training and strength training on CAI, but have not compared strength training, balance training, and combined strength and balance training interventions. Luan et al. [ 8 ] concluded that strength training did not improve FAAM compared to the control group and did not include other self-reported function indicators, so only two literature were included. Koshino et al. [ 50 ] compared the effects of several exercise interventions on dynamic balance, but did not include self-reported function. Only one literature compared the effects of strength training and balance training on self-reported function and dynamic balance in their research, and concluded that balance training significantly improved the self-reported function and dynamic balance. Compared to strength training, balance training can better improve self-reported function, but has no advantage in improving dynamic balance [ 51 ].

The main causes of CAI symptoms may include dynamic balance ability, proprioception, fibular reaction time and loss of valgus force [ 7 ]. The loss of dynamic balance and proprioception will cause CAI patients to have symptoms such as blocked nerve information input, weakened muscle control, and muscle lack strength [ 46 ]. Posture control is affected by the input of information such as vision, vestibular sense, position sense and proprioception. Therefore, posture defects are likely to be affected by the impairment of neuromuscular control and proprioception. It is beneficial to improve the obstacle of posture control, neuromuscular control and proprioception through dynamic and static balance training [ 52 , 53 ].

The decline of valgus strength of ankle joint is more likely to cause muscle weakness, ligament relaxation and repeated ankle sprains. The valgus muscle strength of the ankle joint can provide protection for the lateral ankle ligament to resist the varus force. There is a high correlation between valgus muscle loss and chronic ankle instability [ 54 ]. The risk of the ankle sprain can be reduced by strengthening ankle muscle strength after ankle sprain. In clinical practice, it is generally recommended using elastic band and resistance exercise to improve valgus and dorsiflexion strength [ 55 ]. We can think that the combined strength and balance training will be more effective and targeted than the single form of strength and balance training in posture control, dynamic balance improvement and related CAI symptom reduction. This conclusion is also supported by the systematic meta-analysis results of Mollà-Casanova et al. [ 51 ]. Balance training may help to improve ankle function and reduce incidence rate, and the effect is similar to that of strength training. The combination of two training methods yields better results [ 19 ]. Therefore, it is recommended that rehabilitation practitioners and clinicians use the combination of strength and balance training to rehabilitate the ankle joint, which is the best way to improve the patient's symptoms.

• Dynamic balance

SEBT was significantly correlated with postural control and dynamic balance ability of the lower limbs. The ability of dynamic balance was evaluated efficiently by SEBT score [ 56 ]. This study found that strength training had a significant improvement on SEBT-A, but not on SEBT-PL and SEBT-PM. Balance training had a significant improvement in all three directions and achieved moderate effect size. The combination of strength and balance training had no significant improvement on the three directions. Dynamic balance is the ability of the center of gravity of the body to maintain postural stability and orientation in the plane of support while the body is in motion. In the process of human movement, the support plane of the body is constantly changing, and the body posture is constantly adjusted.

Research by Linens [ 39 ] and Anguish [ 10 ] suggested that balance training had the best effect in improving SEBT scores. This study also found that balance training significantly improved dynamic balance ability, while strength training may have a low effect on dynamic balance improvement. Neuromuscular control and proprioceptive reconstruction recovery are prerequisites for improving postural stability [ 5 ]. Balance training requires better control of body stability in both dynamic and static exercise. This is helpful for neural control and proprioception. Dynamic balance is more dependent on the control of body posture, and has great relevance for trunk posture, lower limb muscle strength [ 52 ]. Strength training generally targets the muscles around the ankle joint to improve results. Strength training needs to be carefully chosen to target improvements in neural control and proprioception.

The actions and control in human movement depend on the sensorimotor system, which integrates the nervous system with the sensory system, forming a complex process of integration [ 6 ]. When the ankle sustains a sports injury, damage occurs to the muscles, tendons, and proprioceptors of the ankle joint, impeding the incoming information. The diminished neuro-muscular control, originally responsible for executing the movement, leads to weakened capabilities, making the ankle joint more susceptible to recurrent sprains and secondary injuries during physical activity [ 17 ]. When the ankle is sprained, the recovery of neuro-muscular control and proprioceptive sensation is the physiological foundation for effective rehabilitation [ 57 ]. The balance training program, including dynamic and static exercises, can enhance postural control in unstable conditions, strengthen neuro-muscular control, and reduce the risk of ankle sprains [ 25 , 58 ].

Available evidence showed that, the combination of strength and balance training achieves greater benefits for patient reported outcomes and intervention for 6 weeks, more than 3 times a week and more than 30 min each time were the best rehabilitation programs to improve CAI patientreported outcomes. balance training could bring greater benefits to dynamic balance. Strength training should be used cautiously in clinic to improve the dynamic balance in individuals with CAI.

Limitations of the study

First of all, some studies included in this study did not show the implementation of allocation concealment and blinding, which may reduce the reliability of the study results or cause subjective bias. Second, the balance training intervention program included single-leg balance training, unstable balance training and other balance interventions, etc. This study did not conduct a more detailed subgroup analysis to determine the effect of different forms of balance intervention. The last, the load and patient reported outcomes measurements of the interventions included in the studies were different, which may lead to bias in the results.

Availability of data and materials

The research provides the best intervention plan for the rehabilitation of chronic ankle instability through a combination of strength and balance training; 2 Balance training could bring greater benefits to dynamic balance, and strength training should be used cautiously in clinic to improve the dynamic balance in individuals with CAI.

The data supporting the results of this study can be obtained from the 33 RCTs literature included.

Hupperets MD, Verhagen EA, Van MW. The 2BFit study: is unsupervised balance board training programme given in addition to usual care, effective in preventing ankle sprain recurrences? Design of a randomized controlled trial[J]. BMC Musculoskelet Disord. 2008;20(9):71.

Article   Google Scholar  

Yeung M, Chan KM, So C, et al. An epidemiological survey on ankle sprain[J]. Br J Sports Med. 1994;28(2):112–6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gribble PA, Delahunt E, Bleakley C, et al. Selection criteria for patients with chronic ankle instability in controlled research: a position statement of the International Ankle Consortium[J]. J Orthop Sports Phys Ther. 2013;43(8):585–91.

Article   PubMed   Google Scholar  

Shi XJ, Han J, Liu Y. Research progress on pathological mechanism, evaluation and diagnosis of chronic ankle instability[J]. Chin J Sports Med. 2019;38(9):816–24.

Google Scholar  

Zhang LL, Yang YS, Zheng JJ. Advance in Posture Control forChronic Ankle Instability(review)[J]. Chin J Rehabil Theory Pract. 2019;25(8):908–12.

Schaefer JL. Effects of a 4-week dynamic-balancetraining program supplemented with Graston instrument-assisted soft-tissue mobilization for chronic ankle instability[J]. J Sport Rehabil. 2012;21(4):313–26.

Thompson C, Schabrun S, Romero R, et al. Factors contributing to chronic ankle instability: a systematic review and meta analysis of systematic reviews[J]. Sports Med. 2018;48(1):189–205.

Luan L, Adams R, Witchalls J, et al. Does Strength Training for Chronic Ankle Instability Improve Balance and PatientReported Outcomes and by Clinically Detectable Amounts? A Systematic Review and MetaAnalysis[J]. Phys Ther. 2021;101(7):pzab046.

Tsikopoulos K, Mavridis D, Georgiannos D, et al. Efficacy of non-surgical interventions on dynamic balance in patients with ankle instability: A network meta-analysis[J]. J Sci Med Sport. 2018;21(9):873–9.

Anguish B, Sandrey MA. Two 4-week balance-training programs for chronic ankle instability[J]. J Athl Train. 2018;53(7):662–71.

Article   PubMed   PubMed Central   Google Scholar  

Wright CJ, Linens SW, Cain MS. A randomized controlled trial comparing rehabilitation efficacy in chronic ankle instability[J]. J Sport Rehabil. 2017;26(4):238–49.

Lee E, Cho J, Lee S. Short-foot exercise promotes quantitative somatosensory function in ankle instability: a randomized controlled trial[J]. Med Sci Monit. 2019;7(25):618–26.

Su YY, Yin HM, Shi LC, et al. Meta Analysis of the Effect of Flywheel Eccentric Overload Training on Preventing Disuse Muscular Atrophy in Simulated Weightlessness Population[J]. Chin J Rehabil Med. 2022;37(02):224–30.

Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials[J]. Phy Ther. 2003;83(8):713–21.

Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses[J]. BMJ. 2003;327:557–60.

Jing DJ, Liao ZP. The Effect of Proprioception Training on the Motor Function of Patientswith Chronic Ankle Instability[J]. Medical information. 2020;33(17):98–100.

Liang Y, Gao M, Wang PZ, et al. Effect of dynamic and static balance training on ankle instability[J]. Chin J Geriatr Orthop Rehabil. 2015;1(02):23–8.

Yang Z. Clinical efficacy of functional exercise on chronic ankle instability[D]. Central China Normal University, 2014.  https://www.cnki.net/ .

Liang S S. Effects of different intervention methods on postural stability in patients with functional ankle instability[D]. TianJin University of Sport, 2019.  https://www.cnki.net/ .

Guo Y H. The effect of core training on general people with functional ankle instability[D]. BeiJing Sport University, 2019.  https://www.cnki.net/ .

He WH, Li K, Du JK, et al. Therapeutic effect of muscle function exercise on chronic external ankle instability[J]. The Journal of Practical Medicine. 2014;30(09):1514–5.

Liu X, Zhang YP, Wang L, et al. Effect of Comprehensive Rehabilitation Direction on Functional Ankle Instability[J]. Chin J Rehabil Theory Pract. 2014;20(05):467–9.

Zhang L L. Effect of 3-D and 2-D dynamic balance training on posture control in individuals with functional ankle instability[D]. ShangHai University of Sport, 2019.  https://www.cnki.net/ .

Zhu X T. Effect of Tai Chi combined with rehabilitation training on functional ankle instability[D]. XiAn Physical Education University, 2019.  https://www.cnki.net/ .

Hale SA, Hertel J, Olmsted-Kramer LC. The effect of a 4-week comprehensive rehabilitation program on postural control and lower extremity function in individuals with chronic ankle instability[J]. J Orthop Sports Phys Ther. 2007;37(6):303–11.

Mckeon PO, Ingersoll CD, Kerrigan DC, et al. Balance training improves function and postural control in those with chronic ankle instability[J]. Med Sci Sports Exerc. 2008;40(10):1810–9.

Minoonejad H, Karimizadeh Ardakani M, Rajabi R, et al. Hop stabilization training improves neuromuscular control in college basketball players with chronic ankle instability: a randomized controlled trial[J]. J Sport Rehabil. 2019;28(6):576–83.

Cain MS, Garceau SW, Linens SW. Effects of a four week biomechanical ankle platform system protocol on balance in high school athletes with chronic ankle instability[J]. J Sport Rehabil. 2017;26(1):1–7.

Sierra-Guzman R, Jimenez-Diaz F, Ramirez C, et al. Whole-body-vibration training and balance in recreational athletes with chronic ankle instability[J]. J Athl Train. 2018;53(4):355–63.

Wright CJ, Linens SW, Cain MS. Establishing the minimal clinical important difference and minimal detectable change for the cumberland ankle instability tool[J]. Arch Phys Med Rehabil. 2017;98(9):1806–11.

Cruz-Diaz D, Lomas-Vega R, Osuna-Perez MC, et al. Effects of 6 weeks of balance training on chronic ankle instability in athletes: a randomized controlled trial[J]. Int J Sports Med. 2015;36(9):754–60.

Article   CAS   PubMed   Google Scholar  

Nam SM, Kim K, Lee DY. Effects of visual feedback balance training on the balance and ankle instability in adult men with functional ankle instability[J]. J Phys Ther Sci. 2018;30(1):113–5.

Cloak R, Nevill A, Day S, et al. Six-week combined vibration and wobble board training on balance and stability in footballers with functional ankle instability[J]. Clin J Sport Med. 2013;23(5):384–91.

Kim KJ, Kim YE, Jun HJ, et al. Which treatment is more effective for functional ankle instability: strengthening or combined muscle strengthening and proprioceptive exercises?[J]. J Phys Ther Sci. 2014;26(3):385–8.

Deussen S, Alfuth M. The influence of sensorimotor training modalities on balance, strength, joint function, amd plantar foot sensitivity in recreational athletes with a history of ankle sprain: a randomized controlled pilot study[J]. Int J Sports Phys Ther. 2018;13(6):993–1007.

Hall EA, Chomistek AK, Kingma JJ, et al. Balance-and strength-training protocols to improve chronic ankle instability deficits, part II: assessing patient-reported outcome measures[J]. J Athl Train. 2018;53(6):578–83.

Cain MS, Ban RJ, Chen YP, et al. Four-week ankle-rehabilitation programs in adolescent athletes with chronic ankle instability[J]. J Athl Train. 2020;55(8):801–10.

Melam GR, Alhusaini AA, Perumal V, et al. Effect of weight-bearing overload using elastic tubing on balance and functional performance in athletes with chronic ankle instability[J]. Sci Sports. 2018;33(5):229–36.

Linens SW, Ross SE, Arnold BL. Wobble board rehabilitation for improving balance in ankles with chronic instability[J]. Clin J Sport Med. 2016;26(1):76–82.

Clark VM, Burden AM. A 4-week wobble board exercise programme improved muscle onset latency and perceived stability in individuals with a functionally unstable ankle[J]. Phy Ther Sport. 2005;6(4):181–7.

Collins CK, Masaracchio M, Cleland JA. The effectiveness of strain counterstrain in the treatment of patients with chronic ankle instability: A randomized clinical trial[J]. J Man Manip Ther. 2014;22(3):119–228.

Tang J. Effects of two balance exercises on the stability of patients with functional ankle instability[J]. TianJin University of Sport, 2022.  https://www.cnki.net/ .

Liu Q. Study of balance and proprioception training on ankle stability of patients with chronic ankle instability[D]. WuHan sports University, 2021.  https://www.cnki.net/ .

Kim KM, Estudillo-Martínez MD, Castellote-Caballero Y, et al. Short-term effects of balance training with stroboscopic vision for patients with chronic ankle instability: a single-blinded randomized controlled trial[J]. Int J Environ Res Public Health. 2021;18(10):5364.

Ardakani MK, Wikstrom EA, Minoonejad H, et al. Hop stabilization training and landing biomechanics in athletes with chronic ankle instability: a randomized controlled trial[J]. J Athl Train. 2019;54(12):1296–303.

Yin Y, Luo DM, Liu H, et al. Advance in Mechanism and Self-reported Measures of Functional Ankle Instability (review)[J]. Chin J Rehabil Theory Pract. 2018;24(6):671–7.

Hertel J. Functional Anatomy, Pathomechanics, and Pathophysiology of Lateral Ankle Instability[J]. J Athl Train. 2002;37(4):364–75.

PubMed   PubMed Central   Google Scholar  

Youssef NM, Abdelmohsen AM, Ashour AA, et al. Effect of different balance training programs on postural control in chronic ankle instability: a randomized controlled trial[J]. Acta Bioeng Biomech. 2018;20(2):159–69.

PubMed   Google Scholar  

Hall EA, Docherty CL, Simon J, et al. Strength-training protocols to improve deficits in participants with chronic ankle instability: a randomized controlled trial[J]. J Athl Train. 2015;50(1):36–44.

Koshino Y, Kobayashi T. Effects of Conservative Interventions on Static and Dynamic Balance in Individuals With Chronic Ankle Instability: A Systematic Review and Meta-analysis[J]. Arch Phys Med Rehabil. 2023;104(4):673–85.

Mollà-Casanova S, Inglés M, Serra-Añó P. Effects of balance training on functionality, ankle instability, and dynamic balance outcomes in people with chronic ankle instability: Systematic review and meta-analysis[J]. Clin Rehabil. 2021;35(12):1694–709.

Peng Y. The Effect of Integrative Neuromuscular Training on Sports Performance, Prevention of Lower Limb Injury and Physiological Mechanism Analysis[J]. Journal of Nanjing Sports Institute. 2020;19(8):55–68.

He JJ, Chen DY, Li P. The new thought for adolescent injury prevention training—Integrated Neuromuscular training[J]. ZheJiang Sport Science. 2022;44(33):98–105.

Fox J, Docherty C, Schrader J, et al. Eccentric plantar-flexor torque deficits in participants with functional ankle instability[J]. J Athl Train. 2008;43(1):51–4.

Kim T, Kim E, Choi H. Effects of a 6-week neuromuscular rehabilitation program on ankle-evertor strength and postural stability in elite women field hockey players with chronic ankle instability[J]. J Sport Rehabil. 2017;26(4):269–80.

Shamseddini SF, Hadadi M, Rezaei I, et al. The effect of the combination of whole body vibration and shoe with an unstable surface in chronic ankle instability treatment: a randomized clinical trial[J]. BMC Sports Sci Med Rehabil. 2021;13(1):28.

Su YY, Peng L, Li W, et al. Research progress of integrative neuromuscular training for the rehabilitation and prevention of chronic ankle instability[J]. China Sport Sci Technology. 2023;59(07):47–52.

Tran ST, Thomas S, Dicesare C, et al. A pilot study of biomechanical assessment before and after an integrative training program for adolescents with juvenile fibromyalgia[J]. Pediatr Rheumatol Online J. 2016;14(1):43.

Download references

Acknowledgements

Thank you for the funding support provided by Bohai University. The teachers of the Physical Training College have provided a lot of guidance and suggestions in paper writing and revision. Thank you to all colleagues for their participation and cooperation, which enabled the successful completion of literature writing.

This study was supported by the Basic Scientific Research Project of Liaoning Provincial Department of Education – Youth Project “Study on rehabilitation effect of different exercise interventions on chronic ankle instability” (No. LJKQR20222558).

Author information

Authors and affiliations.

School of Strength and Conditioning Training, Beijing Sport University, Beijing, China

Yuying Su, Wei Li, Changbo Pan & Yu Shi

Physical Education College, Bohai University, Jinzhou, Liaoning, China

You can also search for this author in PubMed   Google Scholar

Contributions

Yuying SU is responsible for selecting topics, collecting literature and writing papers. Wei LI is responsible for reviewing and modifying. Changbo PAN is responsible for searching literature and extracting data. Yu SHI is responsible for organizing the extracted data and collecting information.

Corresponding author

Correspondence to Wei Li .

Ethics declarations

Ethics approval and consent to participate.

(Not applicable).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Su, Y., Li, W., Pan, C. et al. Effects of combination of strength and balance training on postural control and functionality in people with chronic ankle instability: a systematic review and meta analysis. BMC Sports Sci Med Rehabil 16 , 79 (2024). https://doi.org/10.1186/s13102-024-00845-1

Download citation

Received : 03 June 2023

Accepted : 09 February 2024

Published : 09 April 2024

DOI : https://doi.org/10.1186/s13102-024-00845-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Balance training
• Strength training
• Functionality
• Chronic ankle instability

BMC Sports Science, Medicine and Rehabilitation

ISSN: 2052-1847

• General enquiries: [email protected]

• Open access
• Published: 06 April 2024

Delayed discharge in inpatient psychiatric care: a systematic review

• Ashley-Louise Teale   ORCID: orcid.org/0000-0002-1756-7711 1 ,
• Ceri Morgan   ORCID: orcid.org/0000-0002-2417-8677 1 ,
• Tom A. Jenkins   ORCID: orcid.org/0000-0001-7875-4417 1 &
• Pamela Jacobsen   ORCID: orcid.org/0000-0001-8847-7775 1  

International Journal of Mental Health Systems volume  18 , Article number:  14 ( 2024 ) Cite this article

262 Accesses

8 Altmetric

Metrics details

Delayed discharge is problematic. It is financially costly and can create barriers to delivering best patient care, by preventing return to usual functioning and delaying admissions of others in need. This systematic review aimed to collate existing evidence on delayed discharge in psychiatric inpatient settings and to develop understanding of factors and outcomes of delays in these services.

A search of relevant literature published between 2002 and 2022 was conducted on Pubmed, PsycInfo and Embase. Studies of any design, which published data on delayed discharge from psychiatric inpatient care in high income countries were included. Studies examining child and adolescent, general medical or forensic settings were excluded. A narrative synthesis method was utilised. Quality of research was appraised using the Mixed Methods Appraisal Tool (MMAT).

Eighteen studies from England, Canada, Australia, Ireland, and Norway met the inclusion criteria. Six main reasons for delayed discharge were identified: (1) accommodation needs, (2) challenges securing community or rehabilitation support, (3) funding difficulties, (4) family/carer factors, (5) forensic considerations and (6) person being out of area. Some demographic and clinical factors were also found to relate to delays, such as having a diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, and increased service input prior to admission. Being unemployed and socially isolated were also linked to delays. Only one study commented on consequences of delays for patients, finding they experienced feelings of lack of choice and control. Four studies examined consequences on services, identifying high financial costs.

Overall, the findings suggest there are multiple interlinked factors relevant in delayed discharge that should be considered in practice and policy. Suggestions for future research are discussed, including investigating delayed discharge in other high-income countries, examining delayed discharge from child and forensic psychiatric settings, and exploring consequences of delays on patients and staff. We suggest that future research be consistent in terms used to define delayed discharge, to enhance the clarity of the evidence base.

Review registration number on PROSPERO

Date of registration.

9th December 2021.

Delayed discharge, also termed ‘bed blocking’ and ‘delayed transfer of care,’ refers to when patients remain in hospital beyond the time they are determined to be clinically fit to leave [ 1 , 2 ]. It is an international challenge, costly to individuals, health services and governments [ 3 , 4 ], impacting physical health settings, and also psychiatric inpatient services [ 5 ].

Psychiatric inpatient stays are one of the most expensive forms of treatment for mental health conditions, particularly when compared to care delivered in community settings [ 6 ]. Prolonged stays in mental health hospitals likely increase resource use and as such financial expenditure. This is particularly concerning in instances of delayed discharge when stays are determined to not be of clinical benefit. Delayed discharge also could prevent admission of new patients, contributing to bed crises, where there are not enough beds for all who require admission [ 7 ]. This can have consequences on the course of recovery for newly referred patients, either delaying admission, contributing to inappropriate placements, or leading to individuals being placed out of area [ 7 , 8 ]. Extended hospital stay could also detrimentally impact the delayed patient themselves, preventing their return to usual day-to-day functioning and make returning to the community increasingly difficult [ 9 , 10 ].

Existing reviews have examined predictors of longer stays in psychiatric inpatient settings, finding substance use and being employed are associated with shorter length of stay; while being female, having a diagnoses of mood or psychotic disorders and use of Electroconvulsive Therapy are associated with longer stay [ 11 ]. However, there is not to our knowledge a systematic review collating evidence examining delayed discharge in psychiatric settings. As delayed discharge is a unique experience, distinct from long stay driven by clinical need, it requires separate focus to further understand this specific experience.

Furthermore, a large body of evidence has examined delayed discharge in physical health settings with several systematic reviews, examining causes and outcomes. Such reviews have found that delayed discharges were linked to problems in discharge planning, transfer of care difficulties and patient age [ 12 , 13 ]. Outcomes for services included overcrowding and financial costs, whereas outcomes for patients included infections, depression, reduction in activities and mortality. There may be both overlapping and non-overlapping factors associated with delayed discharge between physical and psychiatric inpatient settings. For example, inpatient psychiatric services may differ in organisational structure, daily workings, and treatment focus from general medical services. The clinical population might also differ in psychiatric and physical health settings, for example in age, socio-economic status, and other demographic, plus clinical factors. As such, it is vital that separate attention be given to the area of psychiatric care.

This systematic review aims to fill the current research gap and synthesise existing literature on psychiatric delayed discharges. We aimed to synthesise the available international data from high-income countries, as the prevalence and underlying reasons for delayed discharge are likely to be highly sensitive to context and heterogeneous across countries. This is due to factors such as different models of healthcare funding, and the varying social role of the family in providing care, for example. Developing in-depth understanding of the causes and consequences of delays in a psychiatric inpatient context is important in informing practice and policies at a service, organisational, societal, and government level. This could help develop ways to reduce occurrence of delays and mitigate any negative impacts.

The aim of this review was to increase understanding of what is known about factors influencing delayed discharge in adult psychiatric inpatient settings. Secondary aims were to examine outcomes of delayed discharge for patients and compare findings across different psychiatric settings and age groups.

The systematic review protocol was pre-registered on PROSPERO before the review was started and the searches were run (PROSPERO: 292515). The review is reported in line with PRISMA guidelines [ 14 , 15 ]. The primary research question of this review is: What is known about factors associated with delayed discharge from inpatient psychiatric care settings?

Secondary research questions were:

What are the outcomes for those who have experienced delayed discharge from inpatient psychiatric settings, for example, in mental health outcomes, health outcomes, readmissions and quality of life?

What are the outcomes on services in terms of resources and costs from delayed psychiatric inpatient discharge?

What are the experiences of staff and patients of delayed discharge from inpatient mental health wards?

Are there differences between types of inpatient services, including acute, rehabilitation or specialist inpatient wards, in factors and costs, are there differences between working age adults and older adults, in experiences of delayed discharge, search strategy.

Initial searches were conducted on the 15th of January 2022, and updated on the 5th of August 2022. Pubmed, PsycInfo and Embase were searched.

Search terms (Appendix B in supplementary materials) were developed through examining key words of published studies on the topic, reviewing the terms used in comparative reviews based in physical health settings and thesaurus mapping. Terms included: “delayed discharge,” “bed blocking” and “long stays.” Search terms were piloted on each database prior to running the final search.

The search included studies published from 2002. A 20-year search timeframe was selected, as psychiatric inpatient care has adapted in response to changing need and updated knowledge over time. As such, studies published before 2002 are likely to be less relevant to current practice.

Following database searches, reference lists of included papers were examined, to identify any relevant studies missed in the search. A forward citation search was also conducted, to identify any relevant studies that were cited in the included papers.

Inclusion and exclusion criteria

Studies were included if they reported data related to delayed discharge or associated outcomes, in adult psychiatric inpatient wards. Specialist and rehabilitation psychiatric inpatient settings were included. Studies of any design were included, providing they were published in a peer-reviewed journal. Both quantitative and qualitative studies were included.

Studies exploring delayed discharge in child or adolescent units and/or forensic units were excluded. This was because the causes and outcomes of delays in such settings are likely unique, given the specialist context. For example, there is likely different systemic involvement from families and different governing legislation in these contexts. As such, it was determined that such settings were too disparate, and synthesising studies from these settings together with adult psychiatric settings could lead to inaccurate conclusions. Physical health settings were also excluded, given the different processes, procedures and treatment focus involved in such settings. In addition, reviews have already been conducted examining delayed discharge from such settings. Studies not conducted in high-income countries were also excluded. In this review, we included high-income countries as defined by World Bank criteria, accessed in January 2022 [ 16 ] (see Appendix C in supplementary material for the list of included countries). Globally, countries differ in the conceptualisation of mental health and provisions offered, therefore, limiting this review to only high-income countries would enable comparisons to be made.

Study selection and data extraction

Screening was conducted using Covidence Systematic Review Software [ 17 ]. All records were independently double-screened by two reviewers at both title/abstract and full-text stage. Conflicts were resolved by discussion to reach consensus, with referral to the senior author (PJ) when needed.

A standardised template was used for data extraction, with all included studies being independently double extracted by two reviewers, with consensus achieved by discussion where needed.

A narrative synthesis method was used. For data examining reasons for delayed discharge, a deductive approach was taken initially. Authors identified possible reasons for delays based on existing literature and organised data under these categories/themes. Any data that did not fit into the pre-defined categories was pooled as ‘other’. All categories were then reviewed, with particular attention placed on the ‘other’ categories, to determine if additional categories need to be added or existing categories adapted. Sub-categories were identified when appropriate through coding. Once categories were established, the number of papers which reported each reason/factor were tabulated and data was reviewed to examine relationships, exploring both links and disparities within and between studies. The final synthesis was checked by three authors (AT, TJ, and CM), to achieve final agreement.

Data relating to outcomes/consequences of delayed discharge was synthesised in a similar way, with data initially organised into three categories: (1) consequences for patients, (2) consequences for service, (3) consequences for staff. Categories were reviewed by the authors following synthesis. Financial costs were converted to US dollars by the authors to support comparison.

Quality assessment of the included studies formed part of the synthesis with the appraisal of quality considered in the interpretation of results.

Quality Assessment

Quality assessment of studies was completed during the synthesis stage. In the protocol, we initially outlined that the Quality Assessment Tool for Studies of Diverse Designs (QATSDD) would be utilised [ 18 ]. However, following a trial of this tool with the included papers, we noted disparities in interpretations between authors. Therefore, the Mixed Methods Appraisal Tool (MMAT) was established to be a more suitable appraisal of quality for the included studies. The MMAT was developed for assessing and comparing the quality of studies using quantitative, qualitative and mixed-methods design, in one tool [ 19 ]. This tool was selected as studies of different designs were included in the review and this tool allows for quality appraisal across five different study types, distinguishing between methodology.

Two initial screening questions were answered to determine appropriateness of using the MMAT to assess quality of the study (are there clear research questions and do the collected data address the research questions). If screening questions are not passed, this tool is deemed inappropriate. Providing the screening questions were passed, quality was assessed on five questions within one of five categories. The category in which questions were answered was determined by study design. The MMAT discourages from scoring and assigning qualitative labels to describe quality, instead advises a more detailed evaluation of quality [ 19 ]. This approach has therefore been taken in this paper.

To achieve reliable and accurate quality ratings, every study was quality rated by two members of the research team and conflicts were discussed to reach consensus.

Identification of studies

Figure  1 (PRISMA flowchart) shows the study selection process. After removing duplicates, a total of 4891 papers were identified for screening. 4397 papers were excluded at title and abstract stage. Full texts were then obtained for 492 papers. Two full texts could not be obtained via the library service and the authors did not respond to a request for the paper. There were four papers obtained that were erratum’s, all of which related to excluded studies that were not examining delayed discharge and as such, were not linked to the included studies. Following full text screening 18 papers were eligible for inclusion. Each paper represented a different study.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flowchart

Study characteristics

Table  1 shows the characteristics of the 18 included studies. Twelve of these studies examined delayed discharge as a primary outcome, with three of these studies specifically examining Housing Related Delayed Discharge (HRDD). HRDD is defined as instances where delayed discharge is attributed to housing issues. The remaining studies ( n  = 6) reported delayed discharge as secondary outcomes. Fifteen studies were of quantitative observational design, two studies used mixed methodologies and one was qualitative.

In the included studies, there was a range of psychiatric inpatient settings: psychiatric/general mental health units ( n  = 11), Psychiatric Intensive Care Units (PICUs) ( n  = 2), older adult psychiatric units ( n  = 3) and Mental Health Trusts ( n  = 1). One study looked across three inpatient settings: acute psychiatric, PICU and older adult. Studies were conducted in five high income countries (England = 10, Ireland = 1, Australia = 3, Canada = 3, and Norway = 1). There were no studies from any other high-income countries, identified in the search.

The MMAT quality scores are shown (Table  2 ). One included study [ 20 ] did not meet initial criteria to be assessed using this tool, as the research questions were unclear.

All studies were of fairly good quality, with all studies meeting at least three out of five of the quality assessment criteria. Quality was highest in Australian and Canadian studies, with included papers in these countries meeting all five quality assessment criteria [ 21 , 22 , 23 , 24 , 25 , 26 ]. Quality assessment ratings indicate that three quantitative descriptive studies included, did not clearly report use of a representative sample or appropriate measures. Ratings per question are shown in Table two.

Research Q1

What is known about factors associated with delayed discharge.

Thirteen studies identified reasons for delayed discharge (Table  1 ). The results showed that there are many complex reasons for delays with often overlapping contributing factors. We categorised reasons for delay into six categories: (1) accommodation needs, (2) difficulty securing rehabilitation or community support, (3) finance/funding challenges, (4) family/carer factors, (5) forensic factors, (6) patient being out of area.

The most common reason for delays was due to accommodation and placement factors. This was identified as a contributing reason for delay in twelve studies and a further two studies assessed Housing-Related Delayed Discharge (HRDD), suggesting accommodation factors contributing to delay in these cases. Accommodation/placement factors included limited availability of placements ( n  = 7), difficulty finding appropriate placements ( n  = 5), awaiting or undergoing placement assessment ( n  = 3), challenges in person returning to accommodation ( n  = 3), e.g., awaiting repairs or adaptations to their home, individuals being rejected from placement ( n  = 2), patients/family rejecting placement ( n  = 2) and awaiting transfer ( n  = 1). It should be noted that one of the studies which examined specific accommodation factors was unable to be quality assessed due to not having clear research questions and therefore did not meet the screening criteria for assessment with the MMAT [ 20 ], and two studies only met three of the five quality assessment criteria, with queries regarding the quality of measures used and analysis technique for one study [ 27 ], and some difficulties integrating and meeting the full quality criteria for the mixed methods approaches used in the second [ 28 ]. The second reason identified for delays was difficulty sourcing support for the person to enable discharge, such as community, rehabilitation, and homecare support. This contributed to delays in twelve studies. Eight of these studies met four to five of the quality assessment criteria, one was not able to be assessed [ 20 ], and three only met three of the five quality assessment criteria [ 27 , 28 , 29 ]. A third reason for delay was finance/funding challenges identified in nine studies. These included challenges obtaining funding, patients/families’ refusal to pay for placements and funding applications being rejected. Six studies identified family/carers factors in creating delays, such as family conflict, family not wanting the person to live with them and ongoing family discussion. The quality of two of the studies identifying family and finance factors should be considered, as one of these studies was unable to be quality assessed due to a lack of clear research questions [ 20 ] and a second met only three of the five quality assessment criteria [ 28 ]. The fifth reason identified in this review as contributing to delay was forensic factors, which accounted for delays in three studies, all of good methodological quality. Forensic delays incorporated delay by Ministry of Justice and awaiting forensic assessment. Person being out of area was highlighted as a reason for delay in only one study and it was not possible to quality assess this study due to no specific research questions identified [ 20 ], suggesting limited exploration or evidence for out of areas contributing to delays.

Fourteen studies included in this review examined the demographic and clinical factors relevant in delays, with eight conducting significance testing to establish associations. Significant associations with delay were having a diagnosis of schizophrenia or other psychotic disorder ( n  = 4), cognitive impairment ( n  = 3) and type/amount of service input prior to admission ( n  = 3). All studies reporting these significant results were of a good methodological quality, achieving at least four of the five MMAT quality criteria. Results were mainly consistent across those studies which examined significance, however, there was one study of good quality that did not find significant association with schizophrenia diagnosis [ 22 ]. The impact of physical health differed between Australia and England, where in one English study having fair-excellent health was more associated with delays [ 30 ], though two Australian studies found poorer physical health linked to delays [ 24 , 25 ]. Findings related to demographic characteristics, including gender, age, ethnicity, socio-economic status, were inconsistent across studies. The only consistent finding was that a smaller proportion of the delayed group were employed ( n  = 3). One of these studies found significant association between being unemployed and delayed discharge. The two other studies found only one member of the delayed group was employed, less than non-delayed groups, though this was not significance tested. There was some indication that being not being married and lacking a support network, was higher in delayed groups. One study found significant relationships to being unmarried and another finding that the delayed group were visited significantly less often by relatives. The other studies did not conduct significance testing. However, there was no significant relationship related to marriage between delayed and non-delayed groups in two studies [ 22 , 31 ]. One of these studies only clearly met three of the quality assessment criteria [ 31 ], though the other met all five quality assessment criteria. Being male was significantly associated with delays in two Canadian studies [ 21 , 22 ]. No significant association with gender was found in other studies.

The supplementary materials provide additional analysis of results for research question one, further describing each study’s findings. Additional materials also include tables showing tabulation of which study examined each variable.

Research Q2

What are the outcomes for those who have experienced delayed discharge from inpatient psychiatric settings for example, in mental health outcomes, health outcomes, readmissions and quality of life.

Only one study examined individual outcomes of delayed discharge for patients [ 26 ]. As such, there is limited data to draw conclusions to answer this research question. The study that evaluated patient outcomes was of qualitative design and good quality. The study explored Housing-Related Delayed Discharge (HRDD) in Australia for 10 patients using semi-structured interviews. They found consequences of lack of choice and control for patients, which impacted mental wellbeing, physical health and created a sense of anticipation for transition to community. Some participants highlighted a positive outcome of delayed discharge in preventing homelessness.

Research Q3

What is the outcome on services in terms of resources and costs from delayed psychiatric inpatient discharge.

Four studies assessed financial costs of delayed discharge for services, providing limited evidence in terms of financial outcomes. Each study focused on a different country. At an old age psychiatry unit in England, delayed discharges were estimated to cost over $855,820 for the year [ 20 ]. Notably, this study was not quality assessed due to the omission of research questions. In a high-quality paper from Australia, HRDD cost the health district $2,828,174 over one year [ 25 ]. While both papers present yearly costs, there is disparity in area covered, contributing to difficulty making comparisons regarding financial expenditure. Two studies calculated financial expenditure and did not present the cost per year. In a Canadian study, using the median number of delayed days ( M  = 17), it was calculated that the average cost incurred by one episode of delayed days was approximately $5,746 [ 21 ]. Furthermore, in Norway, $491,406 was allocated to delays on the acute ward included in the study, though methodological quality might be queried, due to lack of clarity on whether the sample was representative and the appropriateness of measures utilised [ 29 ]. The information necessary to calculate costs per year or costs per delayed day, to enable comparisons to be made across studies, has not included in the studies.

Aside from financial costs, no other type of outcome for services were assessed.

Research Q4

None of the included studies explored specific experiences of delayed discharge for staff. Some information on experiences for patients is detailed in question two.

Research Q5

This systematic review identified studies in acute psychiatric, older adult and Psychiatric Intensive Care Unit (PICU) settings. Only one study included Learning Disability inpatient care settings [ 28 ]. This study was of mixed-method design and met three quality assessment criteria. No studies reported data from rehabilitation units. There were few differences identified between types of setting. Prevalence of delayed discharge was highest in older adult settings (56.9%) [ 30 ] and PICU settings (51.1%) [ 32 ], compared to working age adult settings (18–32%) [ 31 , 33 ]. However, the highest proportion of delayed days was found in acute psychiatric settings in Norway acute psychiatric units (54.8%) [ 29 ]. More information on prevalence is provided in supplementary materials.

Reasons for delay did not vary much across type of setting. There is a potential service difference in the impact of physical health in delays, as having fair-excellent health was more associated with delays in an English older adult study [ 30 ], while in a working age adult sample in Australian studies [ 24 , 25 ], having poor health was more associated with delays. However, this could represent a disparity in country. There were some other differences across countries found. Forensic reasons for delay were only found in the UK ( n  = 2), as was due to patient being out of area ( n  = 1). In UK settings, there was no significant difference found in gender between those delayed and those not [ 30 , 34 ], though there was in Canada [ 21 ]. England and Australia were the only countries identifying funding issues as contributing to delay. Each country will have its own respective funding system, which could impact delays. For example, two Australian studies identified difficulties with their own National Disability Insurance Scheme [ 24 , 25 ].

Research Q6

Only five of the included studies looked specifically at older adult settings, all of which were in the UK. A further five studies, from the UK and Canada, included older adults within their sample, despite not examining a specific older adult setting.

The highest proportion of inpatients experiencing delayed discharge were from older adult settings, with one study identifying 56.9% [ 30 ] of inpatients experiencing delays. There were lower rates of delayed patients in working age adult psychiatric inpatient settings in comparison, with 3.5% [ 21 , 25 ] to 39.1% [ 29 ] of patients experiencing delay. Similarly, two studies in Canada identified that a higher proportion of older adults made up the delayed group compared to the non-delayed group, suggesting that older adult inpatients are more likely to experience delay [ 21 , 22 ]. However, two English studies found delayed discharge was not associated with age [ 31 , 35 ]. One of these studies met only three quality assessment criteria, with lack of clarity regarding the quality of sampling and representativeness of the sample [ 31 ].

In terms of reasons for delay, no clear differences were found across age groups. Although when limiting comparisons to studies conducted in the UK, family/carer factors was identified as a reason for delay more frequently in older adult samples ( n  = 3) compared to studies looking at working age adults ( n  = 1). To support this finding, one study in England found that eight older adult trusts identified patient/carer exercising choice as a reason for delay, whereas the same was true for only four working age adult trusts [ 28 ]. However, this finding cannot be generalised across all countries. There is also some indication that cognitive impairment/dementia might increase likelihood of delay in older adult samples, as two studies identified the role of dementia and greater cognitive impairment in the delayed older adult groups [ 20 , 30 ]. A further two studies examined the impact of cognitive impairment, finding association with delay [ 21 , 22 ]. However, these studies included working age samples, so it is unclear who in the sample this impacted. In addition, physical health status could cause delays differently in older adult populations. In an older adult UK sample having fair-excellent health was more associated with delays [ 30 ], whereas two Australian studies in working age adult inpatient settings found poorer physical health increased delays [ 24 , 25 ]. This difference could however be attributed to country or setting. Funding was identified as a reason for delay in all studies in older adult settings ( n  = 5), but the same was not true for the other setting types. Forensic factors were not found to be a reason for delay in any of the studies with older adult inpatients, conversely patient being out of area was only identified as a reason for delay in an older adult sample [ 20 ].

This systematic review aimed to fill a research gap and examine factors contributing to delayed discharge in adult psychiatric inpatient settings and explore associated consequences. This adds a unique contribution to the evidence base, which predominantly has focused on delayed discharge from physical health settings. Eighteen studies were included for synthesis.

The findings suggest that there are varying inter-related reasons for delay, including accommodation or placement needs, difficulties securing the required support services, funding and finance challenges, family/carer factors, forensic factors and the person being out of area. There were mixed findings regarding demographic and clinical characteristics associated with delays. However, this review showed that delays could be associated with the person having diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, being unemployed and receiving increased service input prior to admission.

There were only a few studies that commented on outcomes of delays. Only one study examined outcomes for patients, identifying feelings of lack of choice and control, while four studies looked at financial outcomes for services, finding large costs associated with delays. This points to a lack of evidence examining the outcomes and experiences of psychiatric delayed discharge, and therefore requires further attention in research.

This review adds to and expands on existing findings, identifying similarities and differences between longer stay generally. For example, one review [ 11 ] found that long stay was associated with mood and psychotic disorders, use of Electroconvulsive Therapy, and being female. Being married, employed, and using substances were associated with a shorter stay [ 11 ]. Our review found that psychiatric delayed discharge was also associated with diagnosis of schizophrenia or other psychotic disorder and being unemployed. However, we found delayed discharge to be associated with cognitive impairment and increased service input prior to admission, but not gender or treatment. This could suggest some important differences in those at risk of delays or those requiring longer inpatient treatment. It is important to note however, the review by Gopalakrishina and colleagues did not distinguish between those patients with long stay clinically warranted and delayed discharge patients [ 11 ]. It would be of benefit for future research on long stay patients to better define their sample based on those who clinically needed treatment or longer stay patients in the context of delayed discharge, allowing similarities and differences to be better explored. This will support policy makers and service managers to better identify those at risk of delays that are not clinically necessary, and those who might need additional clinical input. The findings in this review provide some suggestion that there could be benefit in considering a person’s social context when they are admitted to psychiatric inpatient care, including their living situation at admission, employment status and cognitive functioning. Identifying patients at higher risk of delays earlier in admission might be useful, to ensure more time be given to organise and find appropriate accommodations, placements and service support and facilitate discharge. Wider policy and structural changes are needed, such as improving the availability of appropriate accommodation placements.

It is important to highlight that there were discrepancies across studies in language used to term delayed discharge, e.g., ‘alternate level of care,’ ‘waiting days’ and ‘prolonged stay.’ Due to such discrepancies in definitions and terminology, during the screening process it was at times difficult to determine if studies were focused on delayed discharge or longer lengths of stay clinically required. In this review, studies were excluded if the focus was unclear to prevent incorrect conclusions being drawn related to the unique experience of delayed discharge. However, this means other relevant findings might have been missed. It would therefore be useful for future research on psychiatric inpatient care to ensure clarity in the terminology and definitions used in reports. There were also discrepancies in the way financial costs related to delays were reported, i.e., whether reported as cost per day, cost per year. This made comparing the costs across countries challenging and prevented clear conclusions being drawn. Future research should therefore aim to ensure clarity when reporting financial expenditure, for example, by calculating the daily cost of delays. It is important to highlight that only eighteen studies were identified over the 20-year search period, suggesting this area has not yet been subject to much research focus. All high-income countries met inclusion, but the final sample included studies from only five countries. It might have been expected that studies in other high-income countries be identified, particularly given the expensive nature of inpatient stays and as such delayed discharge. It might be beneficial for future research to further examine delayed discharge in psychiatric settings across other countries, particularly in the USA and EU. For the purposes of this review, studies not conducted in high-income countries were excluded. This was because lower-income countries might experience different factors contributing to delays due to differences in healthcare funding and social factors. As such, separate attention should be given to these settings, to understand similarities or differences in reasons for delays across low- and mid- income countries. Studies on forensic psychiatric settings and child and adolescent settings were also excluded in this instance, so again, there might be benefit in future research examining these areas.

Furthermore, future research could look not only at factors creating delays, but those causing longer delays. Some of the studies in this review began examining this, but more research in this area could be of interest. Finally, while the quality of included studies was relatively high, the studies were primarily of quantitative audit design and infrequently conducted significance testing. As such, further exploration of associations using significance testing would strengthen the evidence base.

In conclusion, 18 studies identified reasons for delayed discharge, including accommodation and placement related factors, challenges securing appropriate support, funding difficulties, family/carer factors, forensic factors and person being out of area. Delay was associated with having a diagnosis of schizophrenia or other psychotic disorder, cognitive impairment, increased service involvement prior to admission, and being unemployed. Service, societal and policy changes might be indicated, to improve accommodation and care provisions following discharge. Future research should continue to examine prolonged inpatient psychiatric stays, ensuring to distinguish between long stays and delayed discharge and improve clarity in terminology used.

Data availability

The data on which this review is based will be made publicly available on publication. A link to data for anonymous peer-review is here: https://osf.io/j4kng/?view_only=1fbf2558d9d044bbb1778fccd5fd6f51 .

Bryan K. Policies for reducing delayed discharge from hospital. Br Med Bull. 2010;95(1):33–46.

Article   PubMed   Google Scholar  

NHS England. Monthly delayed transfers of care situation reports: definitions and guidance. London: NHS England; 2015.

Google Scholar  

House of Commons Health Committee. Delayed Discharges (third report). In: Committee H, editor. 2002.

Rojas-García A, Turner S, Pizzo E, Hudson E, Thomas J, Raine R. Impact and experiences of delayed discharge: a mixed‐studies systematic review. Health Expect. 2018;21(1):41–56.

Christ W. Factors delaying discharge of psychiatric patients. Health Soc Work. 1984;9(3):178–87.

Article   CAS   Google Scholar  

Department of Health. NHS reference costs 2013 to 2014. Department of Health London; 2013.

Glasby J, Lester H. Delayed hospital discharge and mental health: the policy implications of recent research. Social Policy Adm. 2004;38(7):744–57.

Article   Google Scholar  

National Institute of Health and Care Excellence [NICE]. Transition between inpatient mental health settings and community or care home settings. National Institue for Health and Care Excellence; 2016.

Katsakou C, Rose D, Amos T, Bowers L, McCabe R, Oliver D, et al. Psychiatric patients’ views on why their involuntary hospitalisation was right or wrong: a qualitative study. Soc Psychiatry Psychiatr Epidemiol. 2012;47:1169–79.

Csipke E, Williams P, Rose D, Koeser L, McCrone P, Wykes T, et al. Following the Francis report: investigating patient experience of mental health in-patient care. Br J Psychiatry. 2016;209(1):35–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gopalakrishna G, Ithman M, Malwitz K. Predictors of length of stay in a psychiatric hospital. Int J Psychiatry Clin Pract. 2015;19(4):238–44.

Micallef A, Buttigieg SC, Tomaselli G, Garg L. Defining delayed discharges of inpatients and their impact in acute hospital care: a scoping review. Int J Health Policy Manage. 2022;11(2):103.

Cadel L, Guilcher SJ, Kokorelias KM, Sutherland J, Glasby J, Kiran T, et al. Initiatives for improving delayed discharge from a hospital setting: a scoping review. BMJ open. 2021;11(2):e044291.

Article   PubMed   PubMed Central   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group* t. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol. 2021;134:103–12.

World Bank. World Bank Country and Lending Groups 2022 [ https://datahelpdesk.worldbank.org/knowledgebase/articles/906519.]

Covidence systematic review software. Veritas health innovation Melbourne, Australia. 2020. Available at: www.covidence.org.

Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. 2012;18(4):746–52.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The mixed methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inform. 2018;34(4):285–91.

Hanif I, Rathod B. Delays in discharging elderly psychiatric in-patients. Psychiatr Bull. 2008;32(6):211–3.

Little J, Hirdes JP, Daniel I, editors. ALC status in in-patient mental health settings: evidence based on the Ontario Mental Health Reporting System. Healthcare Management Forum. Los Angeles, CA: SAGE Publications Sage CA; 2015.

Little J, Hirdes JP, Perlman CM, Meyer SB. Clinical predictors of delayed discharges in inpatient mental health settings across Ontario. Adm Policy Mental Health Mental Health Serv Res. 2019;46:105–14.

Aflalo M, Soucy N, Xue X, Colacone A, Jourdenais E, Boivin J-F. Characteristics and needs of psychiatric patients with prolonged hospital stay. Can J Psychiatry. 2015;60(4):181–8.

Honey A, Arblaster K, Nguyen J, Heard R. Predicting Housing Related delayed discharge from Mental Health Inpatient units: a Case Control Study. Adm Policy Mental Health Mental Health Serv Res. 2022;49(6):962–72.

Nguyen J, Honey A, Arblaster K, Heard R. Housing-related delayed discharge from inpatient mental health units: Magnitude and contributors in a metropolitan mental health service. Australian J Social Issues. 2022;57(1):144–63.

Chuah CPT, Honey A, Arblaster K. I’m institutionalised… there’s not much I can do’: lived experience of housing related delayed discharge. Aust Occup Ther J. 2022;69(5):574–84.

Cowman J, Whitty P. Prevalence of housing needs among inpatients: a 1 year audit of housing needs in the acute mental health unit in Tallaght Hospital. Ir J Psychol Med. 2016;33(3):159–64.

Article   CAS   PubMed   Google Scholar  

Lewis R, Glasby J. Delayed discharge from mental health hospitals: results of an English postal survey. Health Soc Care Commun. 2006;14(3):225–30.

Berg JE, Restan A. Duration of bed occupancy as calculated at a random chosen day in an acute care ward. Implications for the use of scarce resources in psychiatric care. Ann Gen Psychiatry. 2005;4(1):1–6.

Tucker S, Hargreaves C, Wilberforce M, Brand C, Challis D. What becomes of people admitted to acute old age psychiatry wards? An exploration of factors affecting length of stay, delayed discharge and discharge destination. Int J Geriatr Psychiatry. 2017;32(9):1027–36.

Tyrer P, Suryanarayan G, Rao B, Cicchetti D, Fulop N, Roberts F, et al. The bed requirement inventory: a simple measure to estimate the need for a psychiatric bed. Int J Soc Psychiatry. 2006;52(3):267–77.

Onyon R, Khan S, George M. Delayed discharges from a psychiatric intensive care unit–are we detaining patients unlawfully? J Psychiatric Intensive Care. 2006;2(2):59–64.

Impey M, Milner E. Delayed discharge from mental health inpatient care in the UK. Mental Health Pract. 2013;16(9).

Haw C, Otuwehinmi O, Kotterbova E. Out of area admissions to two independent sector PICUs: patient characteristics, length of stay and delayed discharges. J Psychiatric Intensive Care. 2017;13(1):27–36.

Poole R, Pearsall A, Ryan T. Delayed discharges in an urban in-patient mental health service in England. Psychiatric Bull. 2014;38(2):66–70.

Commander M, Rooprai D. Survey of long-stay patients on acute psychiatric wards. Psychiatr Bull. 2008;32(10):380–3.

Paton JM, Fahy MA, Livingston GA. Delayed discharge—a solvable problem? The place of intermediate care in mental health care of older people. Aging Ment Health. 2004;8(1):34–9.

Download references

No funding to declare.

Author information

Authors and affiliations.

Department of Psychology, University of Bath, Bath, BA2 7AY, UK

Ashley-Louise Teale, Ceri Morgan, Tom A. Jenkins & Pamela Jacobsen

You can also search for this author in PubMed   Google Scholar

Contributions

AT and PJ formulated the initial research questions and developed the systematic review protocol. AT ran the searches on databases. AT, CM and TJ conducted the screening, data extraction and quality assessment. PJ acted as senior reviewer to resolve any conflicts. AT synthesised the results. All authors contributed to data synthesis and interpretation. AT wrote the paper. All authors read and approved the final version of manuscript for submission.

Corresponding author

Correspondence to Ashley-Louise Teale .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Teale, AL., Morgan, C., Jenkins, T.A. et al. Delayed discharge in inpatient psychiatric care: a systematic review. Int J Ment Health Syst 18 , 14 (2024). https://doi.org/10.1186/s13033-024-00635-9

Download citation

Received : 28 July 2023

Accepted : 25 March 2024

Published : 06 April 2024

DOI : https://doi.org/10.1186/s13033-024-00635-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Delayed discharge
• Bed blocking
• Delayed transfer
• Psychiatric inpatient
• Inpatient treatment
• Prolonged stays
• Length of stay

International Journal of Mental Health Systems

ISSN: 1752-4458

• Submission enquiries: [email protected]