limitations of narrative literature review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Grad Med Educ
• v.14(4); 2022 Aug

Narrative Reviews: Flexible, Rigorous, and Practical

Javeed sukhera.

Javeed Sukhera, MD, PhD, FRCPC , is Chair/Chief, Department of Psychiatry, Institute of Living and Hartford Hospital

Introduction

Narrative reviews are a type of knowledge synthesis grounded in a distinct research tradition. They are often framed as non-systematic, which implies that there is a hierarchy of evidence placing narrative reviews below other review forms. 1 However, narrative reviews are highly useful to medical educators and researchers. While a systematic review often focuses on a narrow question in a specific context, with a prespecified method to synthesize findings from similar studies, a narrative review can include a wide variety of studies and provide an overall summary, with interpretation and critique. 1 Examples of narrative review types include state-of-the-art, critical, and integrative reviews, among many others.

Foundations

Narrative reviews are situated within diverse disciplines in the social sciences and humanities. Most forms of narrative reviews align with subjectivist and interpretivist paradigms. These worldviews emphasize that reality is subjective and dynamic. They contrast with the positivist and post-positivist worldviews that are the foundations of systematic reviews: a single reality can be known through experimental research. Unlike systematic reviews, narrative reviews offer researchers the ability to synthesize multiple points of view and harness unique review team perspectives, which will shape the analysis. Therefore, insights gained from a narrative review will vary depending on the individual, organizational, or historical contexts in which the review was conducted. 1 - 5

Why Choose a Narrative Review?

Narrative reviews allow researchers to describe what is known on a topic while conducting a subjective examination and critique of an entire body of literature. Authors can describe the topic's current status while providing insights on advancing the field, new theories, or current evidence viewed from different or unusual perspectives. 3 Therefore, such reviews can be useful by exploring topics that are under-researched as well as for new insights or ways of thinking regarding well-developed, robustly researched fields.

Narrative reviews are often useful for topics that require a meaningful synthesis of research evidence that may be complex or broad and that require detailed, nuanced description and interpretation. 1 See Boxes 1 and 2 for resources on writing a narrative review as well as a case example of a program director's use of a narrative review for an interprofessional education experience. This Journal of Graduate Medical Education (JGME) special review series will continue to use the Case of Dr. Smith to consider the same question using different review methodologies.

Box 1 The Case of Dr. Smith

Dr. Smith, a program director, has been tasked to develop an interprofessional education (IPE) experience for the residency program. Dr. Smith decides that conducting a literature review would be a savvy way to examine the existing evidence and generate a publication useful to others. Using PubMed and a general subject search with “interprofessional education,” Dr. Smith identifies 24 000 matches. Dr. Smith begins to randomly sample the papers and notes the huge diversity of types and approaches: randomized trials, qualitative investigations, critical perspectives, and more.

Dr. Smith decides to do a meta-narrative review, because she notes that there are tensions and contradictions in the ways in which IPE is discussed by different health professions education communities, such as in nursing literature vs in medical journals.

Box 2 Resources

Ferrari R. Writing narrative style literature reviews. Med Writing . 2015;24(4):230-235. doi: 10.1179/2047480615Z.000000000329

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med . 2006;5(3):101-117. doi: 10.1016/S0899-3467(07)60142-6

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews—tasks, tips and traps for aspiring authors. Heart Lung Circ . 2018;27(7):893-898. doi: 10.1016/j.hlc.2018.03.027

Murphy CM. Writing an effective review article. J Med Toxicol . 2012;8(2):89-90. doi: 10.1007/s13181-012-0234-2

Process and Rigor

While each type of narrative review has its own associated markers of rigor, the following guidelines are broadly applicable to narrative reviews and can help readers critically appraise their quality. These principles may also guide researchers who wish to conduct narrative reviews. When engaging with a narrative review as a reader or a researcher, scholars are advised to be conversant with the following 5 foundational elements of narrative reviews.

Rationale for a Narrative Review

First, scholars should consider the framing of the research question. Does the topic being studied align with the type of knowledge synthesis performed through a narrative review? Authors should have a clear research question and a specific audience target. Authors should also provide a rationale for why a narrative review method was chosen. 6 The manuscript should include the initial research question as well as details about any iterative refinements to the question.

Clarity of Boundaries, Scope, and Definitions

Second, although narrative reviews do not typically involve strict predetermined inclusion or exclusion criteria, scholars should explicitly demarcate the boundaries and scope of their topic. They should also clearly define key terms related to the topic and research question and any definitions used. Authors should elaborate why they chose a particular definition if others were available. As narrative reviews are flexible, the initial scope may change through the review process. In such circumstances, authors should provide reasonable justification for the evolution of inclusion and exclusion criteria and a description of how this affected the literature search.

Justification for Inclusion and Exclusion Criteria

Third, authors of narrative reviews should explain which search terms and databases were included in the synthesis and why. For example, did authors include research studies from a particular database, time frame, or study design? Did they include commentaries or empirical articles? Did they include grey literature such as trade publications, reports, or digital media? Each of the authors' choices should be outlined with appropriate reasoning. 7 Narrative reviews tend to be iterative and involve multiple cycles of searching, analysis, and interpretation. High-quality narrative reviews usually include pivotal or seminal papers that address the phenomenon of interest and other manuscripts that are relevant to the research question.

Reflexivity and a Saturation/Sufficiency Statement

Fourth, narrative reviews should clearly specify any factors that may have shaped the authors' interpretations and analysis. One fundamental distinction between narrative and non-narrative reviews is that narrative reviews explicitly recognize that they may not include all relevant literature on a topic. Since narrative reviews do not aim to be inclusive of all literature addressing the phenomenon of interest, a justification for the selection of manuscripts must be included. Authors should carefully outline how researchers conducted analyses and how they determined that sufficient analysis and interpretation was achieved. This latter concept is similar to considerations of saturation or thematic sufficiency in primary qualitative research. 8

Details on Analysis and Interpretation

Lastly, since several different categories of reviews fall under the narrative review umbrella, the analysis conducted in a narrative review varies by type. Regardless of the type of narrative review carried out, authors should clearly describe how analyses were conducted and provide justification for their approach. Narrative reviews are enhanced when researchers are explicit about how their perspectives and experiences informed problem identification, interpretation, and analysis. Given that authors' unique perspectives shape the selection of literature and its interpretation, narrative reviews may be reproduced, but different authors will likely yield different insights and interpretations.

Distinctive Methods and Subtypes

The narrative review has been commonly framed as an umbrella term that includes several different subtypes of reviews. These narrative medicine subtypes share the goals of deepening an understanding of a topic, while describing why researchers chose to explore and analyze the topic in a specific way.

There are several subtypes of narrative reviews with distinctive methodologies; each offers a unique way of approaching the research question and analyzing and interpreting the literature. This article will describe some common narrative review types that will also be discussed in upcoming JGME special articles on reviews: state-of-the-art , meta-ethnographic , critical , and theory integration reviews.

A state-of-the-art review attempts to summarize the research concerning a specific topic along a timeline of significant changes in understanding or research orientations. By focusing on such turning points in the history of evolving understandings of a phenomenon, state-of-the-art reviews offer a summary of the current state of understanding, how such an understanding was developed, and an idea of future directions. A state-of-the art review seeks to offer a 3-part description: where are we now in our understanding, how did we get here, and where should we go next?

A meta-ethnographic review involves choosing and interpreting qualitative research evidence about a specific topic. Working exclusively with qualitative data, this type of knowledge synthesis aims to generate new insights or new conclusions about a topic. It draws together insights and analyses from existing publications of qualitative research to construct new knowledge that spans across these individual, and often small scale, studies.

A meta-narrative review seeks to explore and make sense of contradictions and tensions within the literature. A meta-narrative review maps how a certain topic is understood in distinct ways, conducts a focused search to describe and compare narratives, and then seeks to make sense of how such narratives are interpreted across different disciplines or historical contexts, as part of the analysis. 9

A critical review is a narrative synthesis of literature that brings an interpretative lens: the review is shaped by a theory, a critical point of view, or perspectives from other domains to inform the literature analysis. Critical reviews involve an interpretative process that combines the reviewer's theoretical premise with existing theories and models to allow for synthesis and interpretation of diverse studies. First, reviewers develop and outline their interpretive theoretical position, which is informed by individual knowledge and experience. Next, a noncomprehensive search is completed to capture and identify dominant themes focused on a research question. 8 , 10

An integrative review typically has 1 of 2 different orientations. Empirical integrative reviews analyze and synthesize publications of evidence-based studies with diverse methodologies. In contrast, theoretical integrative reviews conduct an analysis of the available theories addressing a phenomenon, critically appraise those theories, and propose an advancement in the development of those theories. Both types of integrative reviews follow a multistage approach including problem identification, searching, evaluation, analysis, and presentation. 11

Strengths and Weaknesses

Narrative reviews have many strengths. They are flexible and practical, and ideally provide a readable, relevant synthesis of a diverse literature. Narrative reviews are often helpful for teaching or learning about a topic because they deliver a general overview. They are also useful for setting the stage for future research, as they offer an interpretation of the literature, note gaps, and critique research to date.

Such reviews may be useful for providing general background; however, a more comprehensive form of review may be necessary. Narrative reviews do not offer an evidence-based synthesis for focused questions, nor do they offer definitive guideline statements. All types of narrative reviews offer interpretations that are open to critique and will vary depending on the author team or context of the review.

Conclusions

Well-done narrative reviews provide a readable, thoughtful, and practical synthesis on a topic. They allow review authors to advance new ideas while describing and interpreting literature in the field. Narrative reviews do not aim to be systematic syntheses that answer a specific, highly focused question; instead, they offer carefully thought out and rigorous interpretations of a body of knowledge. Such reviews will not provide an exhaustive, comprehensive review of the literature; however, they are useful for a rich and meaningful summary of a topic.

• About the LSE Impact Blog
• Comments Policy
• Popular Posts
• Recent Posts
• Subscribe to the Impact Blog
• Write for us
• LSE comment

Neal Haddaway

October 19th, 2020, 8 common problems with literature reviews and how to fix them.

3 comments | 315 shares

Estimated reading time: 5 minutes

Literature reviews are an integral part of the process and communication of scientific research. Whilst systematic reviews have become regarded as the highest standard of evidence synthesis, many literature reviews fall short of these standards and may end up presenting biased or incorrect conclusions. In this post, Neal Haddaway highlights 8 common problems with literature review methods, provides examples for each and provides practical solutions for ways to mitigate them.

Enjoying this blogpost? 📨 Sign up to our  mailing list  and receive all the latest LSE Impact Blog news direct to your inbox.

Researchers regularly review the literature – it’s an integral part of day-to-day research: finding relevant research, reading and digesting the main findings, summarising across papers, and making conclusions about the evidence base as a whole. However, there is a fundamental difference between brief, narrative approaches to summarising a selection of studies and attempting to reliably and comprehensively summarise an evidence base to support decision-making in policy and practice.

So-called ‘evidence-informed decision-making’ (EIDM) relies on rigorous systematic approaches to synthesising the evidence. Systematic review has become the highest standard of evidence synthesis and is well established in the pipeline from research to practice in the field of health . Systematic reviews must include a suite of specifically designed methods for the conduct and reporting of all synthesis activities (planning, searching, screening, appraising, extracting data, qualitative/quantitative/mixed methods synthesis, writing; e.g. see the Cochrane Handbook ). The method has been widely adapted into other fields, including environment (the Collaboration for Environmental Evidence ) and social policy (the Campbell Collaboration ).

Despite the growing interest in systematic reviews, traditional approaches to reviewing the literature continue to persist in contemporary publications across disciplines. These reviews, some of which are incorrectly referred to as ‘systematic’ reviews, may be susceptible to bias and as a result, may end up providing incorrect conclusions. This is of particular concern when reviews address key policy- and practice- relevant questions, such as the ongoing COVID-19 pandemic or climate change.

These limitations with traditional literature review approaches could be improved relatively easily with a few key procedures; some of them not prohibitively costly in terms of skill, time or resources.

In our recent paper in Nature Ecology and Evolution , we highlight 8 common problems with traditional literature review methods, provide examples for each from the field of environmental management and ecology, and provide practical solutions for ways to mitigate them.

There is a lack of awareness and appreciation of the methods needed to ensure systematic reviews are as free from bias and as reliable as possible: demonstrated by recent, flawed, high-profile reviews. We call on review authors to conduct more rigorous reviews, on editors and peer-reviewers to gate-keep more strictly, and the community of methodologists to better support the broader research community. Only by working together can we build and maintain a strong system of rigorous, evidence-informed decision-making in conservation and environmental management.

Note: This article gives the views of the authors, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our  comments policy  if you have any concerns on posting a comment below

Image credit:  Jaeyoung Geoffrey Kang  via unsplash

About the author

Neal Haddaway is a Senior Research Fellow at the Stockholm Environment Institute, a Humboldt Research Fellow at the Mercator Research Institute on Global Commons and Climate Change, and a Research Associate at the Africa Centre for Evidence. He researches evidence synthesis methodology and conducts systematic reviews and maps in the field of sustainability and environmental science. His main research interests focus on improving the transparency, efficiency and reliability of evidence synthesis as a methodology and supporting evidence synthesis in resource constrained contexts. He co-founded and coordinates the Evidence Synthesis Hackathon (www.eshackathon.org) and is the leader of the Collaboration for Environmental Evidence centre at SEI. @nealhaddaway

Why is mission creep a problem and not a legitimate response to an unexpected finding in the literature? Surely the crucial points are that the review’s scope is stated clearly and implemented rigorously, not when the scope was finalised.

• Pingback: Quick, but not dirty – Can rapid evidence reviews reliably inform policy? | Impact of Social Sciences

#9. Most of them are terribly boring. Which is why I teach students how to make them engaging…and useful.

Leave a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Notify me of follow-up comments by email.

Related Posts

“But I’m not ready!” Common barriers to writing and how to overcome them

November 16th, 2020.

“Remember a condition of academic writing is that we expose ourselves to critique” – 15 steps to revising journal articles

January 18th, 2017.

A simple guide to ethical co-authorship

March 29th, 2021.

How common is academic plagiarism?

February 8th, 2024.

Visit our sister blog LSE Review of Books

• Open access
• Published: 04 September 2016

Improving the peer review of narrative literature reviews

• Jennifer A. Byrne 1 , 2  

Research Integrity and Peer Review volume  1 , Article number:  12 ( 2016 ) Cite this article

24k Accesses

73 Citations

29 Altmetric

Metrics details

As the size of the published scientific literature has increased exponentially over the past 30 years, review articles play an increasingly important role in helping researchers to make sense of original research results. Literature reviews can be broadly classified as either “systematic” or “narrative”. Narrative reviews may be broader in scope than systematic reviews, but have been criticised for lacking synthesis and rigour. The submission of more scientific manuscripts requires more researchers acting as peer reviewers, which requires adding greater numbers of new reviewers to the reviewing population over time. However, whereas there are many easily accessible guides for reviewers of primary research manuscripts, there are few similar resources to assist reviewers of narrative reviews. Here, I summarise why literature reviews are valued by their diverse readership and how peer reviewers with different levels of content expertise can improve the reliability and accessibility of narrative review articles. I then provide a number of recommendations for peer reviewers of narrative literature reviews, to improve the integrity of the scientific literature, while also ensuring that narrative review articles meet the needs of both expert and non-expert readers.

Peer Review reports

Over the past 30 years, the size of the published scientific literature has expanded exponentially [ 1 ]. While it has been argued that this rate of expansion is unsustainable [ 2 ], underlying factors such as greater numbers of scientists and scientific journals [ 3 ] are unlikely to change in the short term. The submission of more manuscripts for publication requires more peer reviewers, yet the current demand for capable, available manuscript reviewers is not being met [ 3 ]. This has serious adverse consequences for the validity of published research and overall trust in science [ 3 ].

Review articles help both experts and non-experts to make sense of the increasing volume of original publications [ 4 , 5 ]. Busy clinicians have a particular reliance upon review articles, because of their constant need for reliable, up-to-date information, yet limited available time [ 6 ]. Literature reviews can also help other content experts such as researchers and policymakers to identify gaps in their own reading and knowledge. However, literature reviews are also sought by readers with little or no prior understanding of the reviewed topic, such as researchers seeking to rapidly triage results from high-throughput analyses and students for whom literature reviews can represent entry points into a new field. For the benefit of both expert and non-expert readers, it is essential that review articles accurately synthesise the relevant literature in a comprehensive, transparent and objective manner [ 7 , 8 ].

Numbers of review articles are increasing in fields where this has been measured [ 4 ], as is the diversity of review types published [ 9 , 10 ]. Although there are now many review sub-types that can be distinguished based upon the literature search, appraisal, synthesis and analysis methods used [ 9 , 10 ], review articles can be broadly classified as either “systematic” or “narrative” [ 5 , 11 ]. Systematic reviews take defined approaches to the identification and synthesis of study findings and include other review sub-types such as evidence maps [ 12 ]. The systematic review is considered to be the gold standard of evidence synthesis, but also carries the potential disadvantages of narrow scope [ 11 ], and requiring more time and resources to prepare and update [ 7 ]. Narrative reviews, also referred to as “traditional reviews” [ 5 ] and “literature reviews” [ 9 ], constitute the majority of review articles published in some fields [ 7 ]. Other review sub-types, such as rapid and scoping reviews also present information in a narrative format [ 9 ]. Narrative reviews have been criticised for rarely employing peer-reviewed methodologies, or duplicate curation of evidence [ 5 ], and for often failing to disclose study inclusion criteria [ 11 ]. Despite these limitations, narrative reviews remain frequent within the literature, as they offer breadth of literature coverage and flexibility to deal with evolving knowledge and concepts [ 11 ]. In this article, I will provide advice regarding the peer review of narrative reviews, and the advice presented aims to be broadly applicable. I will not attempt to provide advice regarding the peer review of systematic reviews [ 13 , 14 ].

Given the broad readership of literature reviews, content and methodology experts as well as reviewers with less directly relevant expertise can play important roles in the peer-review process [ 15 ]. Peer reviewers with related content expertise are best placed to assess the reliability of the information presented, while other reviewers can ensure that this information remains accessible to readers with different levels of prior knowledge. However, whereas there are easily accessible guides for reviewers of primary research manuscripts [ 16 , 17 ], there are few similar resources available for reviewers of literature reviews [ 15 , 18 ]. This article therefore proposes a number of recommendations for peer reviewers (Table  1 ) to ensure that narrative literature review articles make the best possible contributions to their fields, while also meeting their readers’ often diverse needs.

Ask whether the literature review justifies its place in the literature

Lower than expected ratios between numbers of original publications and review articles suggest excessive numbers of reviews in some fields, which may contribute to the very problem that review articles aim to solve [ 4 ]. With rapidly rising publication rates in many fields [ 2 ], even content-expert peer reviewers should check publication databases for similar and/or overlapping review articles as part of the peer-review process. Pre-empting such scrutiny, authors should clearly define the review’s scope and what it intends to achieve [ 8 ]. If there have been other recent reviews of the same or similar topics, the authors should explain how their manuscript is unique. This could be through combining literature from related fields, by updating existing reviews in light of new research evidence [ 8 ], or because published reviews may have been subject to bias. A clear definition of a review’s scope is a recognised tool to reduce evidence selection bias [ 19 ]. Review authors can also define their subject by referring to literature reviews of related topics that will not be explored in depth. These definitions and statements should form part of an overall narrative structure that helps readers to anticipate and understand the information presented [ 20 ].

Ask whether the literature searches conducted were clearly defined

A criticism frequently levelled at traditional or narrative reviews is that they do not always state or follow rules regarding literature searches [ 5 , 7 , 11 ]. Providing evidence that comprehensive literature searches have been conducted, preferably according to pre-defined eligibility criteria [ 19 ], increases confidence that the review’s findings and conclusions are reliable, and have not been subject to selection bias. Ideally, any literature search choices made by the authors should be clearly stated, transparent and reproducible [ 11 ].

Check for citation breadth and balance

Consider whether the authors have cited a comprehensive range of literature or whether they have tended to cite papers that support their own point of view. If there are important papers that have not been cited, suggest to the authors that these be added, and explain why. If only a limited number of articles can be cited due to the journal’s requirements, check that these studies are representative of those available.

Where possible, verify that information has been summarised correctly

Many different types of citation errors can be identified in the research literature [ 21 , 22 ], and these may occur regardless of the journal impact factor [ 22 ]. The increasing size and complexity of primary reports [ 3 ] also render data extraction and summary more challenging. Realistically, it is unlikely that individual peer reviewers will have detailed knowledge of any full review topic [ 19 ]. Nonetheless, if you are a content expert, take time to cross-reference at least some individual statements to citations, for the particular benefit of non-expert readers. If your level of expertise means that you are unable to verify the accuracy of particular sections of the review, you should indicate this to your editor. Peer reviewers can also ask about data extraction methods, if these were not described in the manuscript. Adopting systematic review practices, such as duplicate independent data extraction, or independent data extraction and validation, can reduce content errors and increase reliability [ 19 ].

Check that original references have been cited

Authors sometimes incorrectly cite original studies, both in original manuscripts and reviews [ 23 , 24 ]. While checking the content, ask whether descriptions of original findings were referenced accordingly, as opposed to being incorrectly attributed to reviews [ 23 ].

Consider how studies were critically evaluated

Beyond correct data summary, narrative literature reviews should include critical data appraisal and some level of data synthesis. How this should be done varies according to the review scope and methodology [ 9 , 10 , 19 ]. While some narrative reviews reasonably focus on breadth as opposed to depth of literature coverage [ 10 ], limited or poor data appraisal risks placing undue emphasis on poor quality research [ 9 ]. Evaluating at least some aspects of the methods used by individual studies can improve reliability [ 7 ]. Similarly, ask how the authors have interpreted conflicting findings or studies with apparently outlying results [ 9 , 11 ].

Evaluate whether tables/figures/diagrams support the text

While not all literature reviews need to include figures or tables, these can help to summarise findings and make key messages clearer. Some detailed information may be best presented in tables, with a shorter summary within the text. Tables can improve the availability of quantitative data for cross-checking, better demonstrate the results of qualitative or quantitative data synthesis, and reassure both peer reviewers and readers that comprehensive, objective analyses have been performed. If figures or tables are included, these need to be original; otherwise, the authors need to have obtained permission to reproduce these from an original source.

Consider whether the review will help someone entering the field

Literature reviews are not always read by subject experts, and it is important that the peer-review process considers this. Reviewers who are not direct content experts may valuably request clarification of nomenclature and/or historical issues that may have seemed too obvious for the authors to have explained. Summary diagrams suggested by peer reviewers may help make a literature review more accessible to a broader audience.

Ask whether the review expands the body of knowledge

Ultimately, the goal of a literature review should be to further the body of knowledge [ 18 ]. Extending or developing ideas is clearly a difficult task, and is often the weakest section of a review [ 25 ]. Consider therefore whether the authors have derived and clearly presented new ideas and/or new research directions from any identified knowledge gaps. Having read the manuscript with fresh eyes, peer reviewers may have valuable ideas to contribute.

Do not forget the rules for reviewing manuscripts in general

The review of literature reviews has some particular considerations, but all the usual manuscript review rules also apply, such as managing conflicts of interest and allocating appropriate time [ 16 , 17 ]. Try to separate the assessment of language and grammar from the more important assessment of scientific quality and remain aware that expert reviewers risk bringing their own biases to the peer-review process [ 15 ].

Conclusions

More quality peer reviewers are needed within the scientific community [ 3 ], including those with the capacity and confidence to review narrative literature reviews. Although it has been difficult to identify predictors of peer-reviewer performance and effective training methods, younger reviewer age has been reproducibly associated with better quality manuscript reviews [ 26 , 27 ]. This association suggests that peer reviewers should be recruited relatively early in their careers, and encouraged to participate widely in manuscript review. Associations between younger peer-reviewer age and better manuscript reviews may also highlight the need for regular training, to ensure that the peer-review community remains up-to-date regarding new approaches to editing or reviewing manuscripts. Indeed, a recent industry survey reported that over three quarters of researchers were interested in further reviewer training [ 28 ]. I therefore hope that this article will add to existing resources [ 29 ] to encourage less experienced peer reviewers to extend their efforts towards narrative literature reviews.

Bornmann L, Mutz R. Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J Assoc Inform Sci Tech. 2015;66(11):2215–22.

Google Scholar  

Pautasso M. Publication growth in biological sub-fields: patterns, predictability and sustainability. Sustainability. 2012;4(12):3234–47.

Article   Google Scholar  

Siebert S, Machesky LM, Insall RH. Overflow in science and its implications for trust. Elife. 2015;4: doi: 10.7554/eLife.10825 .

Ketcham CM, Crawford JM. The impact of review articles. Lab Invest. 2007;87(12):1174–85.

Dijkers MP. Task Force on Systematic Reviews and Guidelines. The value of traditional reviews in the era of systematic reviewing. Am J Phys Med Rehabil. 2009;88(5):423–30.

McAlister FA, Clark HD, van Walraven C, Straus SE, Lawson FM, Moher D, et al. The medical review article revisited: has the science improved? Ann Intern Med. 1999;131(12):947–51.

Haddaway NR, Woodcock P, Macura B, Collins A. Making literature reviews more reliable through application of lessons from systematic reviews. Conserv Biol. 2015;29(6):1596–605.

Pautasso M. Ten simple rules for writing a literature review. PLoS Comput Biol. 2013;9(7):e1003149.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inform Lib J. 2009;26(2):91–108.

Paré G, Trudel M-C, Jaana M, Kitsiou S. Synthesizing information systems knowledge: a typology of literature reviews. Inform Management. 2015;52(2):183–99.

Collins JA, Fauser BCJM. Balancing the strengths of systematic and narrative reviews. Hum Reprod Update. 2005;11(2):103–4.

Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016;5:28.

Higgins JPT, Green S. Handbook for systematic reviews of interventions. The Cochrane Collaboration, John Wiley & Sons Ltd; 2011.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700.

Oxman AJ. Checklists for review articles. BMJ. 1994;309(6955):648–51.

Bourne PE, Korngreen A. Ten simple rules for reviewers. PLoS Comput Biol. 2006;2(9):e110.

Nicholas KA, Gordon W. A quick guide to writing a solid peer review. Eos. 2011;92(28):233–4.

Jennex ME. Literature reviews and the review process: an editor-in-chief’s perspective. CAIS. 2015;36:8.

O’Connor A, Sargeant J. Research synthesis in veterinary science: narrative reviews, systematic reviews and meta-analysis. Vet J. 2015;206(3):261–7.

Docherty M, Smith R. The case for structuring the discussion of scientific papers. BMJ. 1999;318(7193):1224–5.

Davids JR, Weigl DM, Edmonds JP, Blackhurst DW. Reference accuracy in peer-reviewed pediatric orthopaedic literature. J Bone Joint Surg Am. 2010;92(5):1155–61.

Awrey J, Inaba K, Barmparas G, Recinos G, Teixeira PG, Chan LS, et al. Reference accuracy in the general surgery literature. World J Surg. 2011;35(3):475–9.

Gavras H. Inappropriate attribution: the “lazy author syndrome”. Am J Hypertens. 2002;15(9):831.

Katz TJ. Propagation of errors in review articles. Science. 2006;313(5791):1236.

Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. MIS Q. 2002;26:2.

Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA. 1998;280(3):231–3.

Callaham ML, Tercier J. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS Med. 2007;4(1):e40.

Warne V. Rewarding reviewers- sense or sensibility? A Wiley study explained. Learned Pub. 2016;29(1):41–50.

COPE Ethical Guidelines for Peer Reviewers. Available: http://publicationethics.org/resources/guidelines-new/cope-ethical-guidelines-peer-reviewers . Accessed 10 Aug, 2016.

Download references

Acknowledgements

I thank Dr Mona Shehata (Princess Margaret Cancer Centre, Toronto, Canada) for discussions, Ms Sarah Frost for critical reading, reviewers of this manuscript for many constructive comments, and reviewers of past publications for feedback which also contributed towards the development of this manuscript.

Not applicable.

Availability of data and materials

Authors’ contributions.

JAB drafted, wrote and approved the final manuscript.

Competing interests

The author declares that she has no competing interests.

Consent for publication

Ethics approval and consent to participate, author information, authors and affiliations.

Molecular Oncology Laboratory, Children’s Cancer Research Unit, Kids Research Institute, The Children’s Hospital at Westmead, Locked Bag 4001, Westmead, 2145, NSW, Australia

Jennifer A. Byrne

The University of Sydney Discipline of Child and Adolescent Health, The Children’s Hospital at Westmead, Locked Bag 4001, Westmead, 2145, NSW, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jennifer A. Byrne .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Byrne, J.A. Improving the peer review of narrative literature reviews. Res Integr Peer Rev 1 , 12 (2016). https://doi.org/10.1186/s41073-016-0019-2

Download citation

Received : 17 June 2016

Accepted : 05 August 2016

Published : 04 September 2016

DOI : https://doi.org/10.1186/s41073-016-0019-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Peer review
• Narrative literature review

Research Integrity and Peer Review

ISSN: 2058-8615

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

• Search Menu
• Advance articles
• Grand Theme Reviews
• Author Guidelines
• Submission Site
• Reasons to Publish
• Open Access
• About Human Reproduction Update
• About the European Society of Human Reproduction and Embryology
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Terms and Conditions
• Contact ESHRE
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Balancing the strengths of systematic and narrative reviews.

• Article contents
• Figures & tables
• Supplementary Data

John A. Collins, Bart C.J.M. Fauser, Balancing the strengths of systematic and narrative reviews, Human Reproduction Update , Volume 11, Issue 2, March/April 2005, Pages 103–104, https://doi.org/10.1093/humupd/dmh058

• Permissions Icon Permissions

The mandate of Human Reproduction Update involves several roles: (i) to provide a synthesis of evidence that can aid scientists and clinicians in their daily work; (ii) to help reproductive specialists understand concepts from related disciplines; and (iii) to summarize current knowledge generated by basic science as the foundation of future scientific and clinical advancement. Given that review and synthesis are central to good scientific and clinical practice, and that a grasp of the current state of knowledge is a prerequisite to designing new studies, it is pertinent to ask which reviews are most likely to fulfil the needs of readers. A related question concerns whether systematic reviews meet the needs of all review topics and all readers.

Summarizing evidence or knowledge is a difficult problem in reproductive medicine, as in other branches of science and medical care ( Eddy et al. , 1992 ). For each question there may be multiple studies that use different designs and inclusion criteria. For clinical questions, the interventions, outcomes and measures of effect may vary: the effect measures in treatment studies include odds ratios, relative risks and absolute differences. For scientific questions, the experimental species, models and designs may differ. Moreover, it is always uncertain whether all of the relevant evidence has been evaluated. Even when the search has been exhaustive, there are no simple guides on how to interpret conflicting results and whether to accept apparently outlying studies. The choices that the reviewer makes to address the variable conditions and uncertainties may be conservative, strict and exclusive, or liberal, open and inclusive. The decisions made by the reviewer may not be consistent throughout and these choices may or may not satisfy the reader who seeks out the review to address a clinical or research question. Faced with uncertainty and doubt, readers nonetheless must form an impression of the evidence and synthesize the state of knowledge in order to address the clinical or research question that stimulated their interest in the review. We argue that the reader is better served when the choices made in the review, regardless of whether they are strict or open, should be explicit, transparent, clearly stated and reproducible by interested readers.

This list of objectives for reviews is more easily satisfied by systematic reviews, which use explicit methods to methodically search, critically appraise and synthesize the available literature on a specific issue. The question or issue need not be clinical: indeed, the concept evolved primarily in psychology studies ( Light and Pillemer, 1984 ). The systematic review attempts to reduce reviewer bias through the use of objective, reproducible criteria to select relevant individual publications and assess their validity. A systematic review may include a meta-analysis or statistical summary of the individual study results: the aggregate of effects from several studies yields an average treatment effect that is more precise than the individual study results ( Schlesselman and Collins, 2003 ). Thus, the systematic review involves explicit, transparent methods which are clearly stated, and reproducible by others. Whether a systematic review of randomized controlled trials adheres to the guidelines can easily be evaluated by means of a widely used checklist (the QUORUM statement) ( Moher et al. , 1999 ). The strengths of the systematic review include the narrow focus of the question, the comprehensive search for evidence, the criterion-based selection of relevant evidence, the rigorous appraisal of validity, the objective or quantitative summary, and the evidence-based inferences ( Cook et al. , 1997 ).

For some review topics, however, the strengths of the systematic review may turn into weaknesses. The primary problem is that the narrow focus and prescribed methods of the systematic review do not allow for comprehensive coverage. For example, the historical review is an irreplaceable means of tracing the development of a scientific principle or clinical concept, but the narrative thread could be lost in the strict rules of systematic review. As other examples, it would be burdensome to apply systematic methods to a survey on aneuploidy and fertility in the aging female or to an assessment of mouse knockout models and polycystic ovarian phenotype. Such topics would require the wider scope of a traditional narrative review, in which less explicit methods are the trade-off for broader coverage.

The majority of review articles are narrative rather than systematic. Narrative reviews generally are comprehensive and cover a wide range of issues within a given topic, but they do not necessarily state or follow rules about the search for evidence. Also, typical narrative reviews do not reveal how the decisions were made about relevance of studies and the validity of the included studies. Of course, the results of the search, selection and assessment procedures must meet the referees’ and editors’ sense of propriety, but readers may not be privy to the methods and thus could not make judgments about the authors choices.

Neither the systematic reviews with their narrow scope nor the narrative reviews with their individuality can satisfy the range of topics for review. Currently, progress in reproductive medicine depends primarily on knowledge of developments in molecular biology, genetics and pharmacology. Background knowledge, evolving concepts and controversy require the flexibility of a narrative review with broad coverage and situational choices about the inclusion of evidence. In contrast, the rigour of a systematic review is needed for effectiveness of diagnostic and treatment interventions and for the outcomes of natural and therapeutic exposures, including adverse events and costs. The choice is more open for many other scientific and clinical topics.

Recognizing that there is a need for both systematic and narrative reviews, could one review type learn from the other? Because readers value transparency and reproducibility, some narrative reviews could gain by drawing from the rigour of systematic reviews. Authors could arrange the subject matter in a series of objective questions, each section based on specified procedures for search, relevance and validity and tied to other sections by appropriate descriptive links. One of the many types of statistical summarization would be helpful to readers. Inferences would adhere to the cited evidence and abstain from opinion. Systematic reviews, on the other hand, could adopt some of the strengths of the narrative review without compromising validity. Their formulaic nature can be boring to read, but this could be countered by non-technical idiomatic language, novel approaches to graphics, and new ways to deal with the baggage of massive tables. Also the excessive concentration in systematic reviews on odds ratios and relative risks is anachronistic, now that absolute differences and numbers needed to treat are the preferred measures of treatment effects ( Sackett and Cook, 1994 ). The procedures for calculating summary absolute effects and their heterogeneity are similar to those for relative effects ( Greenland, 1987 ; Deeks et al. , 2001 ).

Review journals such as Human Reproduction Update have high impact factors because readers need and appreciate comprehensive, relevant, valid summaries that clearly synthesize scientific and clinical evidence. While systematic reviews are more appropriate for focused topics and traditional narrative reviews are better suited to comprehensive topics, either approach can be adapted to clinical or scientific subjects. An infusion of systematic review methods would strengthen narrative reviews and in turn systematic reviews could benefit from the presentation strengths of narrative reviews. The goal is to ensure that the methods of all reviews should be explicit, transparent, clearly stated and reproducible by interested readers.

Cook DJ, Mulrow CD and Haynes RB ( 1997 ) Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 126 , 376 –380.

Google Scholar

Deeks JJ, Altman DG and Bradburn MJ ( 2001 ) Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In Egger M, Davey Smith G, and Altman DG (eds) Systematic Reviews in Health Care: Meta-analysis in Context. BMJ Publishing Group, London, pp 285 –312.

Eddy DM, Hasselblad V and Shachter R ( 1992 ) Meta-analysis by the Confidence Profile Method. Academic Press, Boston.

Greenland S ( 1987 ) Quantitative methods in the review of epidemiologic literature. Epidemiol Rev 9 , 1 –30.

Light RJ and Pillemer DB ( 1984 ) Summing Up: The Science of Reviewing Research. Harvard University Press, Boston.

Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D and Stroup DF ( 1999 ) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet 354 , 1896 –1900.

Sackett DL and Cook RJ ( 1994 ) Understanding clinical trials: what measures of efficacy should journal articles provide busy clinicians? Br Med J 309 , 755 –756.

Schlesselman JJ and Collins JA ( 2003 ) Evaluating systematic reviews and meta-analyses. Semin Reprod Med 21 , 95 –105.

Author notes

1Department of Obstetrics and Gynecology, McMaster University, Hamilton, Ontario, 2Department of Obstetrics and Gynaecology, Dalhousie University, Halifax, Nova Scotia, Canada and 3Department of Reproductive Medicine, University Medical Center, Utrecht, The Netherlands

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1460-2369
• Copyright © 2024 Human Reproduction Update
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Systematic Reviews: Narrative Reviews

• Systematic Review Defined
• Guides, Standards and Resources
• Data Extraction
• Risk of Bias
• WCM's Data Retention Policy & Systematic Reviews
• Scoping Reviews
• Narrative Reviews
• Covidence This link opens in a new window
• Collaborations 2022-present
• Collaborations 2012-2021

What is a narrative review?

Narrative reviews (sometimes called a topic or literature review) are general discussions of a topic, often with no stated hypothesis. They are often invited papers written by experts in the field. 

They can provide rationales for future research and can speculate on new types of interventions available. (Ferrari, 2015)

A narrative review is not a "less good" systematic/scoping review. The aims and methodology are different; they are different study types. 

Rossella Ferrari (2015) Writing narrative style literature reviews, Medical Writing, 24:4, 230-235, DOI: 10.1179/2047480615Z.000000000329

What are some limitations of narrative reviews?

Some limitations of narrative reviews are that they often do not include methodology (i.e., how did they get these results?) and therefore can be subjective in study selection, which can potentially lead to bias. This also makes narrative reviews not reproducible. Because it is not necessary for a narrative review to comprehensive, it is often difficult to discern how much weight to apply to their claims. 

However, narrative reviews can be important and necessary contributions to medical literature.

When should I perform a narrative review versus a systematic/scoping review?

It's very important to remember that not all research questions require or are appropriate for a systematic/scoping review (SR).

• When the question is too broad for a comprehensive, methodologically rigorous SR to take place.
• When there is not enough literature yet available to perform a SR. 
• When there is not enough time to perform a SR.
• When there is not a large enough team  available to perform a SR.

How should I write a narrative review?

Because there are no formal guidelines that narrative reviews must adhere to (such as PRISMA for systematic reviews) their quality can vary greatly. However, there are some resources to help you build your review.

• Writing narrative style literature reviews
• SANRA—a scale for the quality assessment of narrative review articles
• << Previous: Scoping Reviews
• Next: Covidence >>
• Last Updated: Apr 23, 2024 9:13 AM
• URL: https://med.cornell.libguides.com/systematicreviews

LitR-Ex.com

LitR-Ex.com is generously sustained by 

[email protected]

NARRATIVE REVIEWS

Click or scroll down below to view content

Narrative reviews are an umbrella term for a collection of reviews that offer the opportunity to undertake an extensive description and interpretation of previously published literature. Narrative reviews are flexible yet rigorous and have five elements that are consistent across most sub-types. Reflexivity is an important component of narrative reviews, which provides readers with clear insight into how the researcher's perspectives and experiences informed interpretation.

Not sure this is the right review type to answer your research question(s)?

Click Types of Reviews to help you decide.

STRENGTHS + WEAKNESSES

Narrative Reviews

Flexible yet rigorous approach for knowledge synthesis, which is useful to many educators and researchers

Helpful for teaching or learning about a topic because they deliver a general overview

Useful for setting the stage for future research, as they offer an interpretation of the literature, note gaps, and critique research to date

Are not often reproducible related to the influence of the authors and setting on screening, sampling, and analysis

Do not include an exhaustive search of all possible evidence on a given topic

They are selective, which may make them harder to critically appraise against strict criteria. Researchers can address this potential shortcoming by being thoughtful, purposive, and transparent about the choices they make throughout the review process, as well as being explicit in their justifications for these choices

NARRATIVE REVIEWS:

Supporting Documents to Download

NARRATIVE REVIEWS: 5-Steps Downloadable How-To Guide

Narrative reviews: theoretical foundations (journal of graduate medical education publication), narrative reviews: how-to guide (journal of graduate medical education publication), author spotlight.

JAVEED SUKHERA, MD PhD

Javeed Sukhera, MD PhD, is the Chair of Psychiatry at the Institute of Living and Chief of Psychiatry at Hartford Hospital in Hartford Connecticut where he has (pending) academic affiliations with Yale University and the University of Connecticut. He is an MD/PhD Scientist in Health Professions Education and researches novel approaches to addressing stigma and bias in healthcare.