types of review of literature in research methodology

Types of Literature Review

There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:

Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review

Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.

When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.

Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.

Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.

Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.

Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.

Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.

At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods.  Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.

[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology

  John Dudovskiy

Literature Reviews

• Types of reviews
• Getting started

Types of reviews and examples

Choosing a review type.

• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

• Meta-analysis
• Systematized

Definition:

"A term used to describe a conventional overview of the literature, particularly when contrasted with a systematic review (Booth et al., 2012, p. 265).

Characteristics:

• Provides examination of recent or current literature on a wide range of subjects
• Varying levels of completeness / comprehensiveness, non-standardized methodology
• May or may not include comprehensive searching, quality assessment or critical appraisal

Mitchell, L. E., & Zajchowski, C. A. (2022). The history of air quality in Utah: A narrative review.  Sustainability ,  14 (15), 9653.  doi.org/10.3390/su14159653

Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review. London: SAGE Publications Ltd.

"An assessment of what is already known about a policy or practice issue...using systematic review methods to search and critically appraise existing research" (Grant & Booth, 2009, p. 100).

• Assessment of what is already known about an issue
• Similar to a systematic review but within a time-constrained setting
• Typically employs methodological shortcuts, increasing risk of introducing bias, includes basic level of quality assessment
• Best suited for issues needing quick decisions and solutions (i.e., policy recommendations)

Learn more about the method:

Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach.  Systematic reviews, 1 (1), 1-9.  https://doi.org/10.1186/2046-4053-1-10

Virginia Commonwealth University Libraries. (2021). Rapid Review Protocol .

Quarmby, S., Santos, G., & Mathias, M. (2019). Air quality strategies and technologies: A rapid review of the international evidence.  Sustainability, 11 (10), 2757.  https://doi.org/10.3390/su11102757

Grant, M.J. & Booth, A. (2009). A typology of reviews: an analysis of the 14 review types and associated methodologies.  Health Information & Libraries Journal , 26(2), 91-108. https://www.doi.org/10.1111/j.1471-1842.2009.00848.x

Developed and refined by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), this review "map[s] out and categorize[s] existing literature on a particular topic, identifying gaps in research literature from which to commission further reviews and/or primary research" (Grant & Booth, 2009, p. 97).

Although mapping reviews are sometimes called scoping reviews, the key difference is that mapping reviews focus on a review question, rather than a topic

Mapping reviews are "best used where a clear target for a more focused evidence product has not yet been identified" (Booth, 2016, p. 14)

Mapping review searches are often quick and are intended to provide a broad overview

Mapping reviews can take different approaches in what types of literature is focused on in the search

Cooper I. D. (2016). What is a "mapping study?".  Journal of the Medical Library Association: JMLA ,  104 (1), 76–78. https://doi.org/10.3163/1536-5050.104.1.013

Miake-Lye, I. M., Hempel, S., Shanman, R., & Shekelle, P. G. (2016). What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products.  Systematic reviews, 5 (1), 1-21.  https://doi.org/10.1186/s13643-016-0204-x

Tainio, M., Andersen, Z. J., Nieuwenhuijsen, M. J., Hu, L., De Nazelle, A., An, R., ... & de Sá, T. H. (2021). Air pollution, physical activity and health: A mapping review of the evidence.  Environment international ,  147 , 105954.  https://doi.org/10.1016/j.envint.2020.105954

Booth, A. (2016). EVIDENT Guidance for Reviewing the Evidence: a compendium of methodological literature and websites . ResearchGate. https://doi.org/10.13140/RG.2.1.1562.9842 . 

Grant, M.J. & Booth, A. (2009). A typology of reviews: an analysis of the 14 review types and associated methodologies.  Health Information & Libraries Journal , 26(2), 91-108.  https://www.doi.org/10.1111/j.1471-1842.2009.00848.x

"A type of review that has as its primary objective the identification of the size and quality of research in a topic area in order to inform subsequent review" (Booth et al., 2012, p. 269).

• Main purpose is to map out and categorize existing literature, identify gaps in literature—great for informing policy-making
• Search comprehensiveness determined by time/scope constraints, could take longer than a systematic review
• No formal quality assessment or critical appraisal

Learn more about the methods :

Arksey, H., & O'Malley, L. (2005) Scoping studies: towards a methodological framework.  International Journal of Social Research Methodology ,  8 (1), 19-32.  https://doi.org/10.1080/1364557032000119616

Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science: IS, 5, 69. https://doi.org/10.1186/1748-5908-5-69

Example : 

Rahman, A., Sarkar, A., Yadav, O. P., Achari, G., & Slobodnik, J. (2021). Potential human health risks due to environmental exposure to nano-and microplastics and knowledge gaps: A scoping review.  Science of the Total Environment, 757 , 143872.  https://doi.org/10.1016/j.scitotenv.2020.143872

A review that "[compiles] evidence from multiple...reviews into one accessible and usable document" (Grant & Booth, 2009, p. 103). While originally intended to be a compilation of Cochrane reviews, it now generally refers to any kind of evidence synthesis.

• Compiles evidence from multiple reviews into one document
• Often defines a broader question than is typical of a traditional systematic review

Choi, G. J., & Kang, H. (2022). The umbrella review: a useful strategy in the rain of evidence.  The Korean Journal of Pain ,  35 (2), 127–128.  https://doi.org/10.3344/kjp.2022.35.2.127

Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13(3), 132–140. https://doi.org/10.1097/XEB.0000000000000055

Rojas-Rueda, D., Morales-Zamora, E., Alsufyani, W. A., Herbst, C. H., Al Balawi, S. M., Alsukait, R., & Alomran, M. (2021). Environmental risk factors and health: An umbrella review of meta-analyses.  International Journal of Environmental Research and Public Dealth ,  18 (2), 704.  https://doi.org/10.3390/ijerph18020704

A meta-analysis is a "technique that statistically combines the results of quantitative studies to provide a more precise effect of the result" (Grant & Booth, 2009, p. 98).

• Statistical technique for combining results of quantitative studies to provide more precise effect of results
• Aims for exhaustive, comprehensive searching
• Quality assessment may determine inclusion/exclusion criteria
• May be conducted independently or as part of a systematic review

Berman, N. G., & Parker, R. A. (2002). Meta-analysis: Neither quick nor easy. BMC Medical Research Methodology , 2(1), 10. https://doi.org/10.1186/1471-2288-2-10

Hites R. A. (2004). Polybrominated diphenyl ethers in the environment and in people: a meta-analysis of concentrations.  Environmental Science & Technology ,  38 (4), 945–956.  https://doi.org/10.1021/es035082g

A systematic review "seeks to systematically search for, appraise, and [synthesize] research evidence, often adhering to the guidelines on the conduct of a review" provided by discipline-specific organizations, such as the Cochrane Collaboration (Grant & Booth, 2009, p. 102).

• Aims to compile and synthesize all known knowledge on a given topic
• Adheres to strict guidelines, protocols, and frameworks
• Time-intensive and often takes months to a year or more to complete
• The most commonly referred to type of evidence synthesis. Sometimes confused as a blanket term for other types of reviews

Gascon, M., Triguero-Mas, M., Martínez, D., Dadvand, P., Forns, J., Plasència, A., & Nieuwenhuijsen, M. J. (2015). Mental health benefits of long-term exposure to residential green and blue spaces: a systematic review.  International Journal of Environmental Research and Public Health ,  12 (4), 4354–4379.  https://doi.org/10.3390/ijerph120404354

"Systematized reviews attempt to include one or more elements of the systematic review process while stopping short of claiming that the resultant output is a systematic review" (Grant & Booth, 2009, p. 102). When a systematic review approach is adapted to produce a more manageable scope, while still retaining the rigor of a systematic review such as risk of bias assessment and the use of a protocol, this is often referred to as a  structured review  (Huelin et al., 2015).

• Typically conducted by postgraduate or graduate students
• Often assigned by instructors to students who don't have the resources to conduct a full systematic review

Salvo, G., Lashewicz, B. M., Doyle-Baker, P. K., & McCormack, G. R. (2018). Neighbourhood built environment influences on physical activity among adults: A systematized review of qualitative evidence.  International Journal of Environmental Research and Public Health ,  15 (5), 897.  https://doi.org/10.3390/ijerph15050897

Huelin, R., Iheanacho, I., Payne, K., & Sandman, K. (2015). What’s in a name? Systematic and non-systematic literature reviews, and why the distinction matters. https://www.evidera.com/resource/whats-in-a-name-systematic-and-non-systematic-literature-reviews-and-why-the-distinction-matters/

• Review Decision Tree - Cornell University For more information, check out Cornell's review methodology decision tree.
• LitR-Ex.com - Eight literature review methodologies Learn more about 8 different review types (incl. Systematic Reviews and Scoping Reviews) with practical tips about strengths and weaknesses of different methods.
• << Previous: Getting started
• Next: 1. Define your research question >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

• University of Wisconsin–Madison
• University of Wisconsin-Madison
• Research Guides
• Evidence Synthesis, Systematic Review Services
• Literature Review Types, Taxonomies

Evidence Synthesis, Systematic Review Services : Literature Review Types, Taxonomies

• Develop a Protocol
• Develop Your Research Question
• Select Databases
• Select Gray Literature Sources
• Write a Search Strategy
• Manage Your Search Process
• Register Your Protocol
• Citation Management
• Article Screening
• Risk of Bias Assessment
• Synthesize, Map, or Describe the Results
• Find Guidance by Discipline
• Manage Your Research Data
• Browse Evidence Portals by Discipline
• Automate the Process, Tools & Technologies
• Additional Resources

Choosing a Literature Review Methodology

Growing interest in evidence-based practice has driven an increase in review methodologies. Your choice of review methodology (or literature review type) will be informed by the intent (purpose, function) of your research project and the time and resources of your team. 

• Decision Tree (What Type of Review is Right for You?) Developed by Cornell University Library staff, this "decision-tree" guides the user to a handful of review guides given time and intent.

Types of Evidence Synthesis*

Critical Review - Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or model.

Mapping Review (Systematic Map) - Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature.

Meta-Analysis - Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.

Mixed Studies Review (Mixed Methods Review) - Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies.

Narrative (Literature) Review - Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness.

Overview - Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics.

Qualitative Systematic Review or Qualitative Evidence Synthesis - Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies.

Rapid Review - Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research.

Scoping Review or Evidence Map - Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research.

State-of-the-art Review - Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research.

Systematic Review - Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review. (An emerging subset includes Living Reviews or Living Systematic Reviews - A [review or] systematic review which is continually updated, incorporating relevant new evidence as it becomes available.)

Systematic Search and Review - Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis.’

Umbrella Review - Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results.

*These definitions are in Grant & Booth's "A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies."

Literature Review Types/Typologies, Taxonomies

Grant, M. J., and A. Booth. "A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies."  Health Information and Libraries Journal  26.2 (2009): 91-108.  DOI: 10.1111/j.1471-1842.2009.00848.x  Link

Munn, Zachary, et al. “Systematic Review or Scoping Review? Guidance for Authors When Choosing between a Systematic or Scoping Review Approach.” BMC Medical Research Methodology , vol. 18, no. 1, Nov. 2018, p. 143. DOI: 10.1186/s12874-018-0611-x. Link

Sutton, A., et al. "Meeting the Review Family: Exploring Review Types and Associated Information Retrieval Requirements."  Health Information and Libraries Journal  36.3 (2019): 202-22.  DOI: 10.1111/hir.12276  Link

• << Previous: Home
• Next: The Systematic Review Process >>
• Last Updated: May 8, 2024 5:33 PM
• URL: https://researchguides.library.wisc.edu/literature_review

Literature Review: Types of literature reviews

• Traditional or narrative literature reviews
• Scoping Reviews
• Systematic literature reviews
• Annotated bibliography
• Keeping up to date with literature
• Finding a thesis
• Evaluating sources and critical appraisal of literature
• Managing and analysing your literature
• Further reading and resources

Types of literature reviews

The type of literature review you write will depend on your discipline and whether you are a researcher writing your PhD, publishing a study in a journal or completing an assessment task in your undergraduate study.

A literature review for a subject in an undergraduate degree will not be as comprehensive as the literature review required for a PhD thesis.

An undergraduate literature review may be in the form of an annotated bibliography or a narrative review of a small selection of literature, for example ten relevant articles. If you are asked to write a literature review, and you are an undergraduate student, be guided by your subject coordinator or lecturer.

The common types of literature reviews will be explained in the pages of this section.

• Narrative or traditional literature reviews
• Critically Appraised Topic (CAT)
• Scoping reviews
• Annotated bibliographies

These are not the only types of reviews of literature that can be conducted. Often the term "review" and "literature" can be confusing and used in the wrong context. Grant and Booth (2009) attempt to clear up this confusion by discussing 14 review types and the associated methodology, and advantages and disadvantages associated with each review.

Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26 , 91–108. doi:10.1111/j.1471-1842.2009.00848.x

What's the difference between reviews?

Researchers, academics, and librarians all use various terms to describe different types of literature reviews, and there is often inconsistency in the ways the types are discussed. Here are a couple of simple explanations.

• The image below describes common review types in terms of speed, detail, risk of bias, and comprehensiveness:

"Schematic of the main differences between the types of literature review" by Brennan, M. L., Arlt, S. P., Belshaw, Z., Buckley, L., Corah, L., Doit, H., Fajt, V. R., Grindlay, D., Moberly, H. K., Morrow, L. D., Stavisky, J., & White, C. (2020). Critically Appraised Topics (CATs) in veterinary medicine: Applying evidence in clinical practice. Frontiers in Veterinary Science, 7 , 314. https://doi.org/10.3389/fvets.2020.00314 is licensed under CC BY 3.0

• The table below lists four of the most common types of review , as adapted from a widely used typology of fourteen types of reviews (Grant & Booth, 2009).  

Grant, M.J. & Booth, A. (2009).  A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26 (2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

See also the Library's  Literature Review guide.

Critical Appraised Topic (CAT)

For information on conducting a Critically Appraised Topic or CAT

Callander, J., Anstey, A. V., Ingram, J. R., Limpens, J., Flohr, C., & Spuls, P. I. (2017).  How to write a Critically Appraised Topic: evidence to underpin routine clinical practice.  British Journal of Dermatology (1951), 177(4), 1007-1013. https://doi.org/10.1111/bjd.15873 

Books on Literature Reviews

• << Previous: Home
• Next: Traditional or narrative literature reviews >>
• Last Updated: May 12, 2024 12:18 PM
• URL: https://libguides.csu.edu.au/review

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

State-of-the-art literature review methodology: A six-step approach for knowledge synthesis

• Original Article
• Open access
• Published: 05 September 2022
• Volume 11 , pages 281–288, ( 2022 )

Cite this article

You have full access to this open access article

• Erin S. Barry   ORCID: orcid.org/0000-0003-0788-7153 1 , 2 ,
• Jerusalem Merkebu   ORCID: orcid.org/0000-0003-3707-8920 3 &
• Lara Varpio   ORCID: orcid.org/0000-0002-1412-4341 3  

28k Accesses

8 Citations

18 Altmetric

Explore all metrics

Introduction

Researchers and practitioners rely on literature reviews to synthesize large bodies of knowledge. Many types of literature reviews have been developed, each targeting a specific purpose. However, these syntheses are hampered if the review type’s paradigmatic roots, methods, and markers of rigor are only vaguely understood. One literature review type whose methodology has yet to be elucidated is the state-of-the-art (SotA) review. If medical educators are to harness SotA reviews to generate knowledge syntheses, we must understand and articulate the paradigmatic roots of, and methods for, conducting SotA reviews.

We reviewed 940 articles published between 2014–2021 labeled as SotA reviews. We (a) identified all SotA methods-related resources, (b) examined the foundational principles and techniques underpinning the reviews, and (c) combined our findings to inductively analyze and articulate the philosophical foundations, process steps, and markers of rigor.

In the 940 articles reviewed, nearly all manuscripts (98%) lacked citations for how to conduct a SotA review. The term “state of the art” was used in 4 different ways. Analysis revealed that SotA articles are grounded in relativism and subjectivism.

This article provides a 6-step approach for conducting SotA reviews. SotA reviews offer an interpretive synthesis that describes: This is where we are now. This is how we got here. This is where we could be going. This chronologically rooted narrative synthesis provides a methodology for reviewing large bodies of literature to explore why and how our current knowledge has developed and to offer new research directions.

Similar content being viewed by others

An analysis of current practices in undertaking literature reviews in nursing: findings from a focused mapping review and synthesis

Reading and interpreting reviews for health professionals: a practical review

Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review

Avoid common mistakes on your manuscript.

Literature reviews play a foundational role in scientific research; they support knowledge advancement by collecting, describing, analyzing, and integrating large bodies of information and data [ 1 , 2 ]. Indeed, as Snyder [ 3 ] argues, all scientific disciplines require literature reviews grounded in a methodology that is accurate and clearly reported. Many types of literature reviews have been developed, each with a unique purpose, distinct methods, and distinguishing characteristics of quality and rigor [ 4 , 5 ].

Each review type offers valuable insights if rigorously conducted [ 3 , 6 ]. Problematically, this is not consistently the case, and the consequences can be dire. Medical education’s policy makers and institutional leaders rely on knowledge syntheses to inform decision making [ 7 ]. Medical education curricula are shaped by these syntheses. Our accreditation standards are informed by these integrations. Our patient care is guided by these knowledge consolidations [ 8 ]. Clearly, it is important for knowledge syntheses to be held to the highest standards of rigor. And yet, that standard is not always maintained. Sometimes scholars fail to meet the review’s specified standards of rigor; other times the markers of rigor have never been explicitly articulated. While we can do little about the former, we can address the latter. One popular literature review type whose methodology has yet to be fully described, vetted, and justified is the state-of-the-art (SotA) review.

While many types of literature reviews amalgamate bodies of literature, SotA reviews offer something unique. By looking across the historical development of a body of knowledge, SotA reviews delves into questions like: Why did our knowledge evolve in this way? What other directions might our investigations have taken? What turning points in our thinking should we revisit to gain new insights? A SotA review—a form of narrative knowledge synthesis [ 5 , 9 ]—acknowledges that history reflects a series of decisions and then asks what different decisions might have been made.

SotA reviews are frequently used in many fields including the biomedical sciences [ 10 , 11 ], medicine [ 12 , 13 , 14 ], and engineering [ 15 , 16 ]. However, SotA reviews are rarely seen in medical education; indeed, a bibliometrics analysis of literature reviews published in 14 core medical education journals between 1999 and 2019 reported only 5 SotA reviews out of the 963 knowledge syntheses identified [ 17 ]. This is not to say that SotA reviews are absent; we suggest that they are often unlabeled. For instance, Schuwirth and van der Vleuten’s article “A history of assessment in medical education” [ 14 ] offers a temporally organized overview of the field’s evolving thinking about assessment. Similarly, McGaghie et al. published a chronologically structured review of simulation-based medical education research that “reviews and critically evaluates historical and contemporary research on simulation-based medical education” [ 18 , p. 50]. SotA reviews certainly have a place in medical education, even if that place is not explicitly signaled.

This lack of labeling is problematic since it conceals the purpose of, and work involved in, the SotA review synthesis. In a SotA review, the author(s) collects and analyzes the historical development of a field’s knowledge about a phenomenon, deconstructs how that understanding evolved, questions why it unfolded in specific ways, and posits new directions for research. Senior medical education scholars use SotA reviews to share their insights based on decades of work on a topic [ 14 , 18 ]; their junior counterparts use them to critique that history and propose new directions [ 19 ]. And yet, SotA reviews are generally not explicitly signaled in medical education. We suggest that at least two factors contribute to this problem. First, it may be that medical education scholars have yet to fully grasp the unique contributions SotA reviews provide. Second, the methodology and methods of SotA reviews are poorly reported making this form of knowledge synthesis appear to lack rigor. Both factors are rooted in the same foundational problem: insufficient clarity about SotA reviews. In this study, we describe SotA review methodology so that medical educators can explicitly use this form of knowledge synthesis to further advance the field.

We developed a four-step research design to meet this goal, illustrated in Fig.  1 .

Four-step research design process used for developing a State-of-the-Art literature review methodology

Step 1: Collect SotA articles

To build our initial corpus of articles reporting SotA reviews, we searched PubMed using the strategy (″state of the art review″[ti] OR ″state of the art review*″) and limiting our search to English articles published between 2014 and 2021. We strategically focused on PubMed, which includes MEDLINE, and is considered the National Library of Medicine’s premier database of biomedical literature and indexes health professions education and practice literature [ 20 ]. We limited our search to 2014–2021 to capture modern use of SotA reviews. Of the 960 articles identified, nine were excluded because they were duplicates, erratum, or corrigendum records; full text copies were unavailable for 11 records. All articles identified ( n  = 940) constituted the corpus for analysis.

Step 2: Compile all methods-related resources

EB, JM, or LV independently reviewed the 940 full-text articles to identify all references to resources that explained, informed, described, or otherwise supported the methods used for conducting the SotA review. Articles that met our criteria were obtained for analysis.

To ensure comprehensive retrieval, we also searched Scopus and Web of Science. Additionally, to find resources not indexed by these academic databases, we searched Google (see Electronic Supplementary Material [ESM] for the search strategies used for each database). EB also reviewed the first 50 items retrieved from each search looking for additional relevant resources. None were identified. Via these strategies, nine articles were identified and added to the collection of methods-related resources for analysis.

Step 3: Extract data for analysis

In Step 3, we extracted three kinds of information from the 940 articles papers identified in Step 1. First, descriptive data on each article were compiled (i.e., year of publication and the academic domain targeted by the journal). Second, each article was examined and excerpts collected about how the term state-of-the-art review was used (i.e., as a label for a methodology in-and-of itself; as an adjective qualifying another type of literature review; as a term included in the paper’s title only; or in some other way). Finally, we extracted excerpts describing: the purposes and/or aims of the SotA review; the methodology informing and methods processes used to carry out the SotA review; outcomes of analyses; and markers of rigor for the SotA review.

Two researchers (EB and JM) coded 69 articles and an interrater reliability of 94.2% was achieved. Any discrepancies were discussed. Given the high interrater reliability, the two authors split the remaining articles and coded independently.

Step 4: Construct the SotA review methodology

The methods-related resources identified in Step 2 and the data extractions from Step 3 were inductively analyzed by LV and EB to identify statements and research processes that revealed the ontology (i.e., the nature of reality that was reflected) and the epistemology (i.e., the nature of knowledge) underpinning the descriptions of the reviews. These authors studied these data to determine if the synthesis adhered to an objectivist or a subjectivist orientation, and to synthesize the purposes realized in these papers.

To confirm these interpretations, LV and EB compared their ontology, epistemology, and purpose determinations against two expectations commonly required of objectivist synthesis methods (e.g., systematic reviews): an exhaustive search strategy and an appraisal of the quality of the research data. These expectations were considered indicators of a realist ontology and objectivist epistemology [ 21 ] (i.e., that a single correct understanding of the topic can be sought through objective data collection {e.g., systematic reviews [ 22 ]}). Conversely, the inverse of these expectations were considered indicators of a relativist ontology and subjectivist epistemology [ 21 ] (i.e., that no single correct understanding of the topic is available; there are multiple valid understandings that can be generated and so a subjective interpretation of the literature is sought {e.g., narrative reviews [ 9 ]}).

Once these interpretations were confirmed, LV and EB reviewed and consolidated the methods steps described in these data. Markers of rigor were then developed that aligned with the ontology, epistemology, and methods of SotA reviews.

Of the 940 articles identified in Step 1, 98% ( n  = 923) lacked citations or other references to resources that explained, informed, or otherwise supported the SotA review process. Of the 17 articles that included supporting information, 16 cited Grant and Booth’s description [ 4 ] consisting of five sentences describing the overall purpose of SotA reviews, three sentences noting perceived strengths, and four sentences articulating perceived weaknesses. This resource provides no guidance on how to conduct a SotA review methodology nor markers of rigor. The one article not referencing Grant and Booth used “an adapted comparative effectiveness research search strategy that was adapted by a health sciences librarian” [ 23 , p. 381]. One website citation was listed in support of this strategy; however, the page was no longer available in summer 2021. We determined that the corpus was uninformed by a cardinal resource or a publicly available methodology description.

In Step 2 we identified nine resources [ 4 , 5 , 24 , 25 , 26 , 27 , 28 ]; none described the methodology and/or processes of carrying out SotA reviews. Nor did they offer explicit descriptions of the ontology or epistemology underpinning SotA reviews. Instead, these resources provided short overview statements (none longer than one paragraph) about the review type [ 4 , 5 , 24 , 25 , 26 , 27 , 28 ]. Thus, we determined that, to date, there are no available methodology papers describing how to conduct a SotA review.

Step 3 revealed that “state of the art” was used in 4 different ways across the 940 articles (see Fig.  2 for the frequency with which each was used). In 71% ( n  = 665 articles), the phrase was used only in the title, abstract, and/or purpose statement of the article; the phrase did not appear elsewhere in the paper and no SotA methodology was discussed. Nine percent ( n  = 84) used the phrase as an adjective to qualify another literature review type and so relied entirely on the methodology of a different knowledge synthesis approach (e.g., “a state of the art systematic review [ 29 ]”). In 5% ( n  = 52) of the articles, the phrase was not used anywhere within the article; instead, “state of the art” was the type of article within a journal. In the remaining 15% ( n  = 139), the phrase denoted a specific methodology (see ESM for all methodology articles). Via Step 4’s inductive analysis, the following foundational principles of SotA reviews were developed: (1) the ontology, (2) epistemology, and (3) purpose of SotA reviews.

Four ways the term “state of the art” is used in the corpus and how frequently each is used

Ontology of SotA reviews: Relativism

SotA reviews rest on four propositions:

The literature addressing a phenomenon offers multiple perspectives on that topic (i.e., different groups of researchers may hold differing opinions and/or interpretations of data about a phenomenon).

The reality of the phenomenon itself cannot be completely perceived or understood (i.e., due to limitations [e.g., the capabilities of current technologies, a research team’s disciplinary orientation] we can only perceive a limited part of the phenomenon).

The reality of the phenomenon is a subjective and inter-subjective construction (i.e., what we understand about a phenomenon is built by individuals and so their individual subjectivities shape that understanding).

The context in which the review was conducted informs the review (e.g., a SotA review of literature about gender identity and sexual function will be synthesized differently by researchers in the domain of gender studies than by scholars working in sex reassignment surgery).

As these propositions suggest, SotA scholars bring their experiences, expectations, research purposes, and social (including academic) orientations to bear on the synthesis work. In other words, a SotA review synthesizes the literature based on a specific orientation to the topic being addressed. For instance, a SotA review written by senior scholars who are experts in the field of medical education may reflect on the turning points that have shaped the way our field has evolved the modern practices of learner assessment, noting how the nature of the problem of assessment has moved: it was first a measurement problem, then a problem that embraced human judgment but needed assessment expertise, and now a whole system problem that is to be addressed from an integrated—not a reductionist—perspective [ 12 ]. However, if other scholars were to examine this same history from a technological orientation, learner assessment could be framed as historically constricted by the media available through which to conduct assessment, pointing to how artificial intelligence is laying the foundation for the next wave of assessment in medical education [ 30 ].

Given these foundational propositions, SotA reviews are steeped in a relativist ontology—i.e., reality is socially and experientially informed and constructed, and so no single objective truth exists. Researchers’ interpretations reflect their conceptualization of the literature—a conceptualization that could change over time and that could conflict with the understandings of others.

Epistemology of SotA reviews: Subjectivism

SotA reviews embrace subjectivism. The knowledge generated through the review is value-dependent, growing out of the subjective interpretations of the researcher(s) who conducted the synthesis. The SotA review generates an interpretation of the data that is informed by the expertise, experiences, and social contexts of the researcher(s). Furthermore, the knowledge developed through SotA reviews is shaped by the historical point in time when the review was conducted. SotA reviews are thus steeped in the perspective that knowledge is shaped by individuals and their community, and is a synthesis that will change over time.

Purpose of SotA reviews

SotA reviews create a subjectively informed summary of modern thinking about a topic. As a chronologically ordered synthesis, SotA reviews describe the history of turning points in researchers’ understanding of a phenomenon to contextualize a description of modern scientific thinking on the topic. The review presents an argument about how the literature could be interpreted; it is not a definitive statement about how the literature should or must be interpreted. A SotA review explores: the pivotal points shaping the historical development of a topic, the factors that informed those changes in understanding, and the ways of thinking about and studying the topic that could inform the generation of further insights. In other words, the purpose of SotA reviews is to create a three-part argument: This is where we are now in our understanding of this topic. This is how we got here. This is where we could go next.

The SotA methodology

Based on study findings and analyses, we constructed a six-stage SotA review methodology. This six-stage approach is summarized and guiding questions are offered in Tab.  1 .

Stage 1: Determine initial research question and field of inquiry

In Stage 1, the researcher(s) creates an initial description of the topic to be summarized and so must determine what field of knowledge (and/or practice) the search will address. Knowledge developed through the SotA review process is shaped by the context informing it; thus, knowing the domain in which the review will be conducted is part of the review’s foundational work.

Stage 2: Determine timeframe

This stage involves determining the period of time that will be defined as SotA for the topic being summarized. The researcher(s) should engage in a broad-scope overview of the literature, reading across the range of literature available to develop insights into the historical development of knowledge on the topic, including the turning points that shape the current ways of thinking about a topic. Understanding the full body of literature is required to decide the dates or events that demarcate the timeframe of now in the first of the SotA’s three-part argument: where we are now . Stage 2 is complete when the researcher(s) can explicitly justify why a specific year or event is the right moment to mark the beginning of state-of-the-art thinking about the topic being summarized.

Stage 3: Finalize research question(s) to reflect timeframe

Based on the insights developed in Stage 2, the researcher(s) will likely need to revise their initial description of the topic to be summarized. The formal research question(s) framing the SotA review are finalized in Stage 3. The revised description of the topic, the research question(s), and the justification for the timeline start year must be reported in the review article. These are markers of rigor and prerequisites for moving to Stage 4.

Stage 4: Develop search strategy to find relevant articles

In Stage 4, the researcher(s) develops a search strategy to identify the literature that will be included in the SotA review. The researcher(s) needs to determine which literature databases contain articles from the domain of interest. Because the review describes how we got here , the review must include literature that predates the state-of-the-art timeframe, determined in Stage 2, to offer this historical perspective.

Developing the search strategy will be an iterative process of testing and revising the search strategy to enable the researcher(s) to capture the breadth of literature required to meet the SotA review purposes. A librarian should be consulted since their expertise can expedite the search processes and ensure that relevant resources are identified. The search strategy must be reported (e.g., in the manuscript itself or in a supplemental file) so that others may replicate the process if they so choose (e.g., to construct a different SotA review [and possible different interpretations] of the same literature). This too is a marker of rigor for SotA reviews: the search strategies informing the identification of literature must be reported.

Stage 5: Analyses

The literature analysis undertaken will reflect the subjective insights of the researcher(s); however, the foundational premises of inductive research should inform the analysis process. Therefore, the researcher(s) should begin by reading the articles in the corpus to become familiar with the literature. This familiarization work includes: noting similarities across articles, observing ways-of-thinking that have shaped current understandings of the topic, remarking on assumptions underpinning changes in understandings, identifying important decision points in the evolution of understanding, and taking notice of gaps and assumptions in current knowledge.

The researcher(s) can then generate premises for the state-of-the-art understanding of the history that gave rise to modern thinking, of the current body of knowledge, and of potential future directions for research. In this stage of the analysis, the researcher(s) should document the articles that support or contradict their premises, noting any collections of authors or schools of thinking that have dominated the literature, searching for marginalized points of view, and studying the factors that contributed to the dominance of particular ways of thinking. The researcher(s) should also observe historical decision points that could be revisited. Theory can be incorporated at this stage to help shape insights and understandings. It should be highlighted that not all corpus articles will be used in the SotA review; instead, the researcher(s) will sample across the corpus to construct a timeline that represents the seminal moments of the historical development of knowledge.

Next, the researcher(s) should verify the thoroughness and strength of their interpretations. To do this, the researcher(s) can select different articles included in the corpus and examine if those articles reflect the premises the researcher(s) set out. The researcher(s) may also seek out contradictory interpretations in the literature to be sure their summary refutes these positions. The goal of this verification work is not to engage in a triangulation process to ensure objectivity; instead, this process helps the researcher(s) ensure the interpretations made in the SotA review represent the articles being synthesized and respond to the interpretations offered by others. This is another marker of rigor for SotA reviews: the authors should engage in and report how they considered and accounted for differing interpretations of the literature, and how they verified the thoroughness of their interpretations.

Stage 6: Reflexivity

Given the relativist subjectivism of a SotA review, it is important that the manuscript offer insights into the subjectivity of the researcher(s). This reflexivity description should articulate how the subjectivity of the researcher(s) informed interpretations of the data. These reflections will also influence the suggested directions offered in the last part of the SotA three-part argument: where we could go next. This is the last marker of rigor for SotA reviews: researcher reflexivity must be considered and reported.

SotA reviews have much to offer our field since they provide information on the historical progression of medical education’s understanding of a topic, the turning points that guided that understanding, and the potential next directions for future research. Those future directions may question the soundness of turning points and prior decisions, and thereby offer new paths of investigation. Since we were unable to find a description of the SotA review methodology, we inductively developed a description of the methodology—including its paradigmatic roots, the processes to be followed, and the markers of rigor—so that scholars can harness the unique affordances of this type of knowledge synthesis.

Given their chronology- and turning point-based orientation, SotA reviews are inherently different from other types of knowledge synthesis. For example, systematic reviews focus on specific research questions that are narrow in scope [ 32 , 33 ]; in contrast, SotA reviews present a broader historical overview of knowledge development and the decisions that gave rise to our modern understandings. Scoping reviews focus on mapping the present state of knowledge about a phenomenon including, for example, the data that are currently available, the nature of that data, and the gaps in knowledge [ 34 , 35 ]; conversely, SotA reviews offer interpretations of the historical progression of knowledge relating to a phenomenon centered on significant shifts that occurred during that history. SotA reviews focus on the turning points in the history of knowledge development to suggest how different decisions could give rise to new insights. Critical reviews draw on literature outside of the domain of focus to see if external literature can offer new ways of thinking about the phenomenon of interest (e.g., drawing on insights from insects’ swarm intelligence to better understand healthcare team adaptation [ 36 ]). SotA reviews focus on one domain’s body of literature to construct a timeline of knowledge development, demarcating where we are now, demonstrating how this understanding came to be via different turning points, and offering new research directions. Certainly, SotA reviews offer a unique kind of knowledge synthesis.

Our six-stage process for conducting these reviews reflects the subjectivist relativism that underpins the methodology. It aligns with the requirements proposed by others [ 24 , 25 , 26 , 27 ], what has been written about SotA reviews [ 4 , 5 ], and the current body of published SotA reviews. In contrast to existing guidance [ 4 , 5 , 20 , 21 , 22 , 23 ], our description offers a detailed reporting of the ontology, epistemology, and methodology processes for conducting the SotA review.

This explicit methodology description is essential since many academic journals list SotA reviews as an accepted type of literature review. For instance, Educational Research Review [ 24 ], the American Academy of Pediatrics [ 25 ], and Thorax all lists SotA reviews as one of the types of knowledge syntheses they accept [ 27 ]. However, while SotA reviews are valued by academia, guidelines or specific methodology descriptions for researchers to follow when conducting this type of knowledge synthesis are conspicuously absent. If academics in general, and medical education more specifically, are to take advantage of the insights that SotA reviews can offer, we need to rigorously engage in this synthesis work; to do that, we need clear descriptions of the methodology underpinning this review. This article offers such a description. We hope that more medical educators will conduct SotA reviews to generate insights that will contribute to further advancing our field’s research and scholarship.

Cooper HM. Organizing knowledge syntheses: a taxonomy of literature reviews. Knowl Soc. 1988;1:104.

Google Scholar  

Badger D, Nursten J, Williams P, Woodward M. Should all literature reviews be systematic? Eval Res Educ. 2000;14:220–30.

Article   Google Scholar  

Snyder H. Literature review as a research methodology: an overview and guidelines. J Bus Res. 2019;104:333–9.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91–108.

Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36:202–22.

Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

Tricco AC, Langlois E, Straus SE, World Health Organization, Alliance for Health Policy and Systems Research. Rapid reviews to strengthen health policy and systems: a practical guide. Geneva: World Health Organization; 2017.

Jackson R, Feder G. Guidelines for clinical guidelines: a simple, pragmatic strategy for guideline development. Br Med J. 1998;317:427–8.

Greenhalgh T, Thorne S, Malterud K. Time to challenge the spurious hierarchy of systematic over narrative reviews? Eur J Clin Invest. 2018;48:e12931.

Bach QV, Chen WH. Pyrolysis characteristics and kinetics of microalgae via thermogravimetric analysis (TGA): a state-of-the-art review. Bioresour Technol. 2017;246:88–100.

Garofalo C, Milanović V, Cardinali F, Aquilanti L, Clementi F, Osimani A. Current knowledge on the microbiota of edible insects intended for human consumption: a state-of-the-art review. Food Res Int. 2019;125:108527.

Carbone S, Dixon DL, Buckley LF, Abbate A. Glucose-lowering therapies for cardiovascular risk reduction in type 2 diabetes mellitus: state-of-the-art review. Mayo Clin Proc. 2018;93:1629–47.

Hofkens PJ, Verrijcken A, Merveille K, et al. Common pitfalls and tips and tricks to get the most out of your transpulmonary thermodilution device: results of a survey and state-of-the-art review. Anaesthesiol Intensive Ther. 2015;47:89–116.

Schuwirth LW, van der Vleuten CP. A history of assessment in medical education. Adv Health Sci Educ Theory Pract. 2020;25:1045–56.

Arena A, Prete F, Rambaldi E, et al. Nanostructured zirconia-based ceramics and composites in dentistry: a state-of-the-art review. Nanomaterials. 2019;9:1393.

Bahraminasab M, Farahmand F. State of the art review on design and manufacture of hybrid biomedical materials: hip and knee prostheses. Proc Inst Mech Eng H. 2017;231:785–813.

Maggio LA, Costello JA, Norton C, Driessen EW, Artino AR Jr. Knowledge syntheses in medical education: a bibliometric analysis. Perspect Med Educ. 2021;10:79–87.

McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44:50–63.

Krishnan DG, Keloth AV, Ubedulla S. Pros and cons of simulation in medical education: a review. Education. 2017;3:84–7.

National Library of Medicine. MEDLINE: overview. 2021. https://www.nlm.nih.gov/medline/medline_overview.html . Accessed 17 Dec 2021.

Bergman E, de Feijter J, Frambach J, et al. AM last page: a guide to research paradigms relevant to medical education. Acad Med. 2012;87:545.

Maggio LA, Samuel A, Stellrecht E. Systematic reviews in medical education. J Grad Med Educ. 2022;14:171–5.

Bandari J, Wessel CB, Jacobs BL. Comparative effectiveness in urology: a state of the art review utilizing a systematic approach. Curr Opin Urol. 2017;27:380–94.

Elsevier. A guide for writing scholarly articles or reviews for the educational research review. 2010. https://www.elsevier.com/__data/promis_misc/edurevReviewPaperWriting.pdf . Accessed 3 Mar 2020.

American Academy of Pediatrics. Pediatrics author guidelines. 2020. https://pediatrics.aappublications.org/page/author-guidelines . Accessed 3 Mar 2020.

Journal of the American College of Cardiology. JACC instructions for authors. 2020. https://www.jacc.org/pb-assets/documents/author-instructions-jacc-1598995793940.pdf . Accessed 3 Mar 2020.

Thorax. Authors. 2020. https://thorax.bmj.com/pages/authors/ . Accessed 3 Mar 2020.

Berven S, Carl A. State of the art review. Spine Deform. 2019;7:381.

Ilardi CR, Chieffi S, Iachini T, Iavarone A. Neuropsychology of posteromedial parietal cortex and conversion factors from mild cognitive impairment to Alzheimer’s disease: systematic search and state-of-the-art review. Aging Clin Exp Res. 2022;34:289–307.

Chan KS, Zary N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ. 2019;5:e13930.

World Health Organization. Framework for action on interprofessional education and collaborative practice. 2010. https://www.who.int/publications/i/item/framework-for-action-on-interprofessional-education-collaborative-practice . Accessed July 1 2021.

Hammersley M. On ‘systematic’ reviews of research literatures: a ‘narrative’ response to Evans & Benefield. Br Educ Res J. 2001;27:543–54.

Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ. 2017;51:585–97.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.

Matsas B, Goralnick E, Bass M, Barnett E, Nagle B, Sullivan E. Leadership development in US undergraduate medical education: a scoping review of curricular content and competency frameworks. Acad Med. 2022;97:899–908.

Cristancho SM. On collective self-healing and traces: How can swarm intelligence help us think differently about team adaptation? Med Educ. 2021;55:441–7.

Download references

Acknowledgements

We thank Rhonda Allard for her help with the literature review and compiling all available articles. We also want to thank the PME editors who offered excellent development and refinement suggestions that greatly improved this manuscript.

Author information

Authors and affiliations.

Department of Anesthesiology, F. Edward Hébert School of Medicine, Uniformed Services University, Bethesda, MD, USA

Erin S. Barry

School of Health Professions Education (SHE), Maastricht University, Maastricht, The Netherlands

Department of Medicine, F. Edward Hébert School of Medicine, Uniformed Services University, Bethesda, MD, USA

Jerusalem Merkebu & Lara Varpio

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Erin S. Barry .

Ethics declarations

Conflict of interest.

E.S. Barry, J. Merkebu and L. Varpio declare that they have no competing interests.

Additional information

The opinions and assertions contained in this article are solely those of the authors and are not to be construed as reflecting the views of the Uniformed Services University of the Health Sciences, the Department of Defense, or the Henry M. Jackson Foundation for the Advancement of Military Medicine.

Supplementary Information

40037_2022_725_moesm1_esm.docx.

For information regarding the search strategy to develop the corpus and search strategy for confirming capture of any available State of the Art review methodology descriptions. Additionally, a list of the methodology articles found through the search strategy/corpus is included

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Barry, E.S., Merkebu, J. & Varpio, L. State-of-the-art literature review methodology: A six-step approach for knowledge synthesis. Perspect Med Educ 11 , 281–288 (2022). https://doi.org/10.1007/s40037-022-00725-9

Download citation

Received : 03 December 2021

Revised : 25 July 2022

Accepted : 27 July 2022

Published : 05 September 2022

Issue Date : October 2022

DOI : https://doi.org/10.1007/s40037-022-00725-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• State-of-the-art literature review
• Literature review
• Literature review methodology
• Find a journal
• Publish with us
• Track your research

• Duke NetID Login
• 919.660.1100
• Duke Health Badge: 24-hour access
• Accounts & Access
• Databases, Journals & Books
• Request & Reserve
• Training & Consulting
• Request Articles & Books
• Renew Online
• Reserve Spaces
• Reserve a Locker
• Study & Meeting Rooms
• Course Reserves
• Digital Health Device Collection
• Pay Fines/Fees
• Recommend a Purchase
• Access From Off Campus
• Building Access
• Computers & Equipment
• Wifi Access
• My Accounts
• Mobile Apps
• Known Access Issues
• Report an Access Issue
• All Databases
• Article Databases
• Basic Sciences
• Clinical Sciences
• Dissertations & Theses
• Drugs, Chemicals & Toxicology
• Grants & Funding
• Interprofessional Education
• Non-Medical Databases
• Search for E-Journals
• Search for Print & E-Journals
• Search for E-Books
• Search for Print & E-Books
• E-Book Collections
• Biostatistics
• Global Health
• MBS Program
• Medical Students
• MMCi Program
• Occupational Therapy
• Path Asst Program
• Physical Therapy
• Researchers
• Community Partners

Conducting Research

• Archival & Historical Research
• Black History at Duke Health
• Data Analytics & Viz Software
• Data: Find and Share
• Evidence-Based Practice
• NIH Public Access Policy Compliance
• Publication Metrics
• Qualitative Research
• Searching Animal Alternatives

Systematic Reviews

• Test Instruments

Using Databases

• JCR Impact Factors
• Web of Science

Finding & Accessing

• COVID-19: Core Clinical Resources
• Health Literacy
• Health Statistics & Data
• Library Orientation

Writing & Citing

• Creating Links
• Getting Published
• Reference Mgmt
• Scientific Writing

Meet a Librarian

• Request a Consultation
• Find Your Liaisons
• Register for a Class
• Request a Class
• Self-Paced Learning

Search Services

• Literature Search
• Systematic Review
• Animal Alternatives (IACUC)
• Research Impact

Citation Mgmt

• Other Software

Scholarly Communications

• About Scholarly Communications
• Publish Your Work
• Measure Your Research Impact
• Engage in Open Science
• Libraries and Publishers
• Directions & Maps
• Floor Plans

Library Updates

• Annual Snapshot
• Conference Presentations
• Contact Information
• Gifts & Donations
• What is a Systematic Review?

Types of Reviews

• Manuals and Reporting Guidelines
• Our Service
• 1. Assemble Your Team
• 2. Develop a Research Question
• 3. Write and Register a Protocol
• 4. Search the Evidence
• 5. Screen Results
• 6. Assess for Quality and Bias
• 7. Extract the Data
• 8. Write the Review
• Additional Resources
• Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

• Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project. 

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108.  doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: What is a Systematic Review?
• Next: Manuals and Reporting Guidelines >>
• Last Updated: May 17, 2024 12:11 PM
• URL: https://guides.mclibrary.duke.edu/sysreview
• Duke Health
• Duke University
• Duke Libraries
• Medical Center Archives
• Duke Directory
• Seeley G. Mudd Building
• 10 Searle Drive
• [email protected]

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Systematic Reviews: Types of literature review, methods, & resources

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

Analytical reviews

GUIDELINES FOR HOW TO CARRY OUT AN ANALYTICAL REVIEW OF QUANTITATIVE RESEARCH

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

When comparing therapies :

PRISMA (Guideline on how to perform and write-up a systematic review and/or meta-analysis of the outcomes reported in multiple clinical trials of therapeutic interventions. PRISMA  replaces the previous QUORUM statement guidelines ):  Liberati, A,, Altman, D,, Moher, D, et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  Plos Medicine, 6 (7):e1000100. doi:10.1371/journal.pmed.1000100 

When comparing diagnostic methods :

Checklist for Artificial Intelligence in Medical Imaging (CLAIM). CLAIM is modeled after the STARD guideline and has been extended to address applications of AI in medical imaging that include classification, image reconstruction, text analysis, and workflow optimization. The elements described here should be viewed as a “best practice” to guide authors in presenting their research. Reported in Mongan, J., Moy, L., & Kahn, C. E., Jr (2020). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.  Radiology. Artificial intelligence ,  2 (2), e200029. https://doi.org/10.1148/ryai.2020200029

STAndards for the Reporting of Diagnostic accuracy studies (STARD) Statement. (Reporting guidelines for writing up a study comparing the accuracy of competing diagnostic methods)  http://www.stard-statement.org/

When evaluating clinical practice guidelines :

AGREE Research Trust (ART) (2013).  Appraisal of Guidelines for Research & Evaluation (AGREE-II) . (A 23-item instrument for as sessing th e quality of Clinical Practice Guidelines. Used internationally for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.)

National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument (2019). (A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.)

When reviewing genetics studies

Human genetics review reporting guidelines.  Little J, Higgins JPT (eds.). The HuGENet™ HuGE Review Handbook, version 1.0 . 

When you need to re-analyze individual participant data

If you wish to collect, check, and re-analyze individual participant data (IPD) from clinical trials addressing a particular research question, you should follow the  PRISMA-IPD  guidelines as reported in  Stewart, L.A., Clarke, M., Rovers, M., et al. (2015). Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement. JAMA, 313(16):1657-1665. doi:10.1001/jama.2015.3656 .

When comparing Randomized studies involving animals, livestock, or food:

O’Connor AM, et al. (2010).  The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement.  Zoonoses Public Health. 57(2):95-104. Epub 2010/01/15. doi: 10.1111/j.1863-2378.2009.01311.x. PubMed PMID: 20070653.

Sargeant JM, et al. (2010).  The REFLECT Statement: Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety: Explanation and Elaboration.  Zoonoses Public Health. 57(2):105-36. Epub 2010/01/15. doi: JVB1312 [pii] 10.1111/j.1863-2378.2009.01312.x. PubMed PMID: 20070652.

GUIDELINES FOR HOW TO WRITE UP FOR PUBLICATION THE RESULTS OF ONE QUANTITATIVE CLINICAL TRIAL

When reporting the results of a Randomized Controlled Trial :

Consolidated Standards of Reporting Trials (CONSORT) Statement. (2010 reporting guideline for writing up a Randomized Controlled Clinical Trial).  http://www.consort-statement.org . Since updated in 2022, see Butcher, M. A., et al. (2022). Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension . JAMA : the Journal of the American Medical Association, 328(22), 2252–2264. https://doi.org/10.1001/jama.2022.21022

Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biology, 8(6), e1000412–e1000412. https://doi.org/10.1371/journal.pbio.1000412 (A 20-item checklist, following the CONSORT approach, listing the information that published articles reporting research using animals should include, such as the number and specific characteristics of animals used; details of housing and husbandry; and the experimental, statistical, and analytical methods used to reduce bias.)

Narrative reviews

GUIDELINES  FOR HOW TO CARRY OUT  A  NARRATIVE REVIEW / QUALITATIVE RESEARCH /  OBSERVATIONAL STUDIES

Campbell, M. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368. doi: https://doi.org/10.1136/bmj.l6890  (guideline on how to analyse evidence for a narrative review, to provide a recommendation based on heterogenous study types).

Community Preventive Services Task Force (2021).  The Methods Manual for Community Guide Systematic Reviews . (Public Health Prevention systematic review guidelines)

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Observational studies, Qualitative research, Quality improvement studies, and Economic evaluations). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from  https://methods.cochrane.org/qi/training-resources . (Training materials for how to do a meta-synthesis, or qualitative evidence synthesis). 

Cornell University Library (2019). Planning worksheet for structured literature reviews. Retrieved 4/8/22 from  https://osf.io/tnfm7/  (offers a framework for a narrative literature review).

Green, B. N., Johnson, C. D., & Adams, A. (2006).  Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine, 5(3): 101-117. DOI: 10.1016/ S0899-3467 (07)60142-6.  This is a very good article about what to take into consideration when writing any type of narrative review.

When reviewing observational studies/qualitative research :

STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement. (Reporting guidelines for various types of health sciences observational studies).  http://www.strobe-statement.org 

Meta-analysis of Observational Studies in Epidemiology (MOOSE)  http://jama.jamanetwork.com/article.aspx?articleid=192614

RATS Qualitative research systematic review guidelines.  https://www.equator-network.org/reporting-guidelines/qualitative-research-review-guidelines-rats/

Methods/Guidance

Right Review , this decision support website provides an algorithm to help reviewers choose a review methodology from among 41 knowledge synthesis methods.

The Systematic Review Toolbox , an online catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Maintained by the UK University of York Health Economics Consortium, Newcastle University NIHR Innovation Observatory, and University of Sheffield School of Health and Related Research.

Institute of Medicine. (2011).  Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: National Academies  (Systematic review guidelines from the Health and Medicine Division (HMD) of the U.S. National Academies of Sciences, Engineering, and Medicine (formerly called the Institute of Medicine)).

International Committee of Medical Journal Editors (2022).  Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals . Guidance on how to prepare a manuscript for submission to a Medical journal.

Cochrane Handbook of Systematic Reviews of Interventions (International Cochrane Collaboration systematic review guidelines). The various Cochrane review groups comporise around 30,000 physicians around the world working in the disciplines on reviews of interventions with very detailed methods for verifying the validity of the research methods and analysis performed in screened-in Randmized Controlled Clinical Trials. Typically published Cochrane Reviews are the most exhaustive review of the evidence of effectiveness of a particular drug or intervention, and include a statistical meta-analysis. Similar to practice guidelines, Cochrane reviews are periodically revised and updated.

Joanna Briggs Institute (JBI) Manual of Evidence Synthesis . (International systematic review guidelines). Based at the University of Adelaide, South Australia, and collaborating with around 80 academic and medical entities around the world. Unlike Cochrane Reviews that strictly focus on efficacy of interventions, JBI offers a broader, inclusive approach to evidence, to accommodate a range of diverse questions and study designs. The JBI manual provides guidance on how to analyse and include both quantitative and qualitative research.

Cochrane Methods Support Unit, webinar recordings on methodological support questions 

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from https://methods.cochrane.org/qi/training-resources . (How to do a meta-synthesis, or qualitative evidence synthesis). 

Center for Reviews and Dissemination (University of York, England) (2009).  Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care . (British systematic review guidelines). 

Agency for Health Research & Quality (AHRQ) (2013). Methods guide for effectiveness and comparative effectiveness reviews . (U.S. comparative effectiveness review guidelines)

Hunter, K. E., et al. (2022). Searching clinical trials registers: guide for systematic reviewers.  BMJ (Clinical research ed.) ,  377 , e068791. https://doi.org/10.1136/bmj-2021-068791

Patient-Centered Outcomes Research Institute (PCORI).  The PCORI Methodology Report . (A 47-item methodology checklist for U.S. patient-centered outcomes research. Established under the Patient Protection and Affordable Care Act, PCORI funds the development of guidance on the comparative effectivess of clinical healthcare, similar to the UK National Institute for Clinical Evidence but without reporting cost-effectiveness QALY metrics). 

Canadian Agency for Drugs and Technologies in Health (CADTH) (2019). Grey Matters: a practical tool for searching health-related grey literature. Retrieved from https://www.cadth.ca/resources/finding-evidence/grey-matters . A checklist of N American & international online databases and websites you can use to search for unpublished reports, posters, and policy briefs, on topics including general medicine and nursing, public and mental health, health technology assessment, drug and device regulatory, approvals, warnings, and advisories.

Hempel, S., Xenakis, L., & Danz, M. (2016). Systematic Reviews for Occupational Safety and Health Questions: Resources for Evidence Synthesis. Retrieved 8/15/16 from http://www.rand.org/pubs/research_reports/RR1463.html . NIOSH guidelines for how to carry out a systematic review in the occupational safety and health domain.

A good source for reporting guidelines is the  NLM's  Research Reporting Guidelines and Initiatives .

Grading of Recommendations Assessment, Development and Evaluation (GRADE). (An international group of academics/clinicians working to promote a common approach to grading the quality of evidence and strength of recommendations.) 

Phillips, B., Ball, C., Sackett, D., et al. (2009). Oxford Centre for Evidence Based Medicine: Levels of Evidence. Retrieved 3/20/17 from https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf . (Another commonly used criteria for grading the quality of evidence and strength of recommendations, developed in part by EBM guru David Sackett.) 

Systematic Reviews for Animals & Food  (guidelines including the REFLECT statement for carrying out a systematic review on animal health, animal welfare, food safety, livestock, and agriculture)

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x. (Describes 14 different types of literature and systematic review, useful for thinking at the outset about what sort of literature review you want to do.)

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements . Health information and libraries journal, 36(3), 202–222. doi:10.1111/hir.12276  (An updated look at different types of literature review, expands on the Grant & Booth 2009 article listed above).

Garrard, J. (2007).  Health Sciences Literature Review Made Easy: The Matrix Method  (2nd Ed.).   Sudbury, MA:  Jones & Bartlett Publishers. (Textbook of health sciences literature search methods).

Zilberberg, M. (2012).  Between the lines: Finding the truth in medical literature . Goshen, MA: Evimed Research Press. (Concise book on foundational concepts of evidence-based medicine).

Lang, T. (2009). The Value of Systematic Reviews as Research Activities in Medical Education . In: Lang, T. How to write, publish, & present in the health sciences : a guide for clinicians & laboratory researchers. Philadelphia : American College of Physicians.  (This book chapter has a helpful bibliography on systematic review and meta-analysis methods)

Brown, S., Martin, E., Garcia, T., Winter, M., García, A., Brown, A., Cuevas H.,  & Sumlin, L. (2013). Managing complex research datasets using electronic tools: a meta-analysis exemplar . Computers, Informatics, Nursing: CIN, 31(6), 257-265. doi:10.1097/NXN.0b013e318295e69c. (This article advocates for the programming of electronic fillable forms in Adobe Acrobat Pro to feed data into Excel or SPSS for analysis, and to use cloud based file sharing systems such as Blackboard, RefWorks, or EverNote to facilitate sharing knowledge about the decision-making process and keep data secure. Of particular note are the flowchart describing this process, and their example screening form used for the initial screening of abstracts).

Brown, S., Upchurch, S., & Acton, G. (2003). A framework for developing a coding scheme for meta-analysis . Western Journal Of Nursing Research, 25(2), 205-222. (This article describes the process of how to design a coded data extraction form and codebook, Table 1 is an example of a coded data extraction form that can then be used to program a fillable form in Adobe Acrobat or Microsoft Access).

Elamin, M. B., Flynn, D. N., Bassler, D., Briel, M., Alonso-Coello, P., Karanicolas, P., & ... Montori, V. M. (2009). Choice of data extraction tools for systematic reviews depends on resources and review complexity .  Journal Of Clinical Epidemiology ,  62 (5), 506-510. doi:10.1016/j.jclinepi.2008.10.016  (This article offers advice on how to decide what tools to use to extract data for analytical systematic reviews).

Riegelman R.   Studying a Study and Testing a Test: Reading Evidence-based Health Research , 6th Edition.  Lippincott Williams & Wilkins, 2012. (Textbook of quantitative statistical methods used in health sciences research).

Rathbone, J., Hoffmann, T., & Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 480. doi:10.1186/s13643-015-0067-6

Guyatt, G., Rennie, D., Meade, M., & Cook, D. (2015). Users' guides to the medical literature (3rd ed.). New York: McGraw-Hill Education Medical.  (This is a foundational textbook on evidence-based medicine and of particular use to the reviewer who wants to learn about the different types of published research article e.g. "what is a case report?" and to understand what types of study design best answer what types of clinical question).

Glanville, J., Duffy, S., Mccool, R., & Varley, D. (2014). Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? Journal of the Medical Library Association : JMLA, 102(3), 177–183. https://doi.org/10.3163/1536-5050.102.3.007

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan a web and mobile app for systematic reviews.  Systematic Reviews, 5 : 210, DOI: 10.1186/s13643-016-0384-4. http://rdcu.be/nzDM

Kwon Y, Lemieux M, McTavish J, Wathen N. (2015). Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 103 (4): 184-8. doi: 10.3163/1536-5050.103.4.004. https://www.ncbi.nlm.nih.gov/pubmed/26512216

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 104 (3):240-3. doi: 10.3163/1536-5050.104.3.014. Erratum in: J Med Libr Assoc. 2017 Jan;105(1):111. https://www.ncbi.nlm.nih.gov/pubmed/27366130

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–46. doi: 10.1016/j.jclinepi.2016.01.021 . PRESS is a guideline with a checklist for librarians to critically appraise the search strategy for a systematic review literature search.

Clark, JM, Sanders, S, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Bateup, S, Linthwaite, B, May, N, Munn, J, Ramsay, L, Rickett, K, Rutter, C, Smith, A, Sondergeld, P, Wallin, M, Jones, M & Beller, E 2020, 'Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial',  Journal of the Medical Library Association , vol. 108, no. 2, pp. 195-207.

Journal articles describing systematic review methods can be searched for in PubMed using this search string in the PubMed search box: sysrev_methods [sb] . 

Software tools for systematic reviews

• Covidence GW in 2019 has bought a subscription to this Cloud based tool for facilitating screening decisions, used by the Cochrane Collaboration. Register for an account.
• NVIVO for analysis of qualitative research NVIVO is used for coding interview data to identify common themes emerging from interviews with several participants. GW faculty, staff, and students may download NVIVO software.
• RedCAP RedCAP is software that can be used to create survey forms for research or data collection or data extraction. It has very detailed functionality to enable data exchange with Electronic Health Record Systems, and to integrate with study workflow such as scheduling follow up reminders for study participants.
• SRDR tool from AHRQ Free, web-based and has a training environment, tutorials, and example templates of systematic review data extraction forms
• RevMan 5 RevMan 5 is the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews
• Rayyan Free, web-based tool for collecting and screening citations. It has options to screen with multiple people, masking each other.
• GradePro Free, web application to create, manage and share summaries of research evidence (called Evidence Profiles and Summary of Findings Tables) for reviews or guidelines, uses the GRADE criteria to evaluate each paper under review.
• DistillerSR Needs subscription. Create coded data extraction forms from templates.
• EPPI Reviewer Needs subscription. Like DistillerSR, tool for text mining, data clustering, classification and term extraction
• SUMARI Needs subscription. Qualitative data analysis.
• Dedoose Needs subscription. Qualitative data analysis, similar to NVIVO in that it can be used to code interview transcripts, identify word co-occurence, cloud based.
• Meta-analysis software for statistical analysis of data for quantitative reviews SPSS, SAS, and STATA are popular analytical statistical software that include macros for carrying out meta-analysis. Himmelfarb has SPSS on some 3rd floor computers, and GW affiliates may download SAS to your own laptop from the Division of IT website. To perform mathematical analysis of big data sets there are statistical analysis software libraries in the R programming language available through GitHub and RStudio, but this requires advanced knowledge of the R and Python computer languages and data wrangling/cleaning.
• PRISMA 2020 flow diagram generator The PRISMA Statement website has a page listing example flow diagram templates and a link to software for creating PRISMA 2020 flow diagrams using R software.

GW researchers may want to consider using Refworks to manage citations, and GW Box to store the full text PDF's of review articles. You can also use online survey forms such as Qualtrics, RedCAP, or Survey Monkey, to design and create your own coded fillable forms, and export the data to Excel or one of the qualitative analytical software tools listed above.

Forest Plot Generators

• RevMan 5 the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews.
• Meta-Essentials a free set of workbooks designed for Microsoft Excel that, based on your input, automatically produce meta-analyses including Forest Plots. Produced for Erasmus University Rotterdam joint research institute.
• Neyeloff, Fuchs & Moreira Another set of Excel worksheets and instructions to generate a Forest Plot. Published as Neyeloff, J.L., Fuchs, S.C. & Moreira, L.B. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes 5, 52 (2012). https://doi-org.proxygw.wrlc.org/10.1186/1756-0500-5-52
• For R programmers instructions are at https://cran.r-project.org/web/packages/forestplot/vignettes/forestplot.html and you can download the R code package from https://github.com/gforge/forestplot
• << Previous: Home
• Next: Protocol and registration >>

• Last Updated: May 8, 2024 11:07 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

• Library Hours
• Strategic Plan
• Giving to the Libraries
• Jobs at the Libraries
• Find Your Librarian
• View All →
• Google Scholar
• Research Guides
• Textbook/Reserves
• Government Documents
• Get It For Me
• Print/Copy/Scan
• Renew Materials
• Study Rooms
• Use a Computer
• Borrow Tech Gear
• Student Services
• Faculty Services
• Users with Disabilities
• Visitors & Alumni
• Special Collections
• Find Information

Basics of Systematic Reviews

• About Systematic Review

Types of Reviews

Literature review.

Collects key sources on a topic and discusses those sources in conversation with each other

• Standard for research articles in most disciplines
• Tells the reader what is known, or not known, about a particular issue, topic, or subject
• Demonstrates knowledge and understanding of a topic
• Establishes context or background for a case or argument
• Helps develop the author’s ideas and perspective

Rapid Review

Thorough methodology but with process limitations in place to expeditethe completion of a review.

• For questions that require timely answers
• 3-4 months vs. 12-24 months
• Limitations - scope, comprehensiveness bias, and quality of appraisal
• Discusses potential effects that the limited methods may have had on results

Scoping Review

Determine the scope or coverage of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview of its focus.

• Identify types of available evidence in a given field
• Clarify key concepts/definitions in the literature
• Examine how research is conducted on a certain topic or field
• Identify key factors related to a concept
• Key difference is focus
• Identify and analyze knowledge gaps

Systematic Review

Attempts to identify, appraise, and summarize all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question.

• clearly defined question with inclusion/exclusion criteria
• rigorous and systematic search of the literature
• thorough screening of results
• data extraction and management
• analysis and interpretation of results
• risk of bias assessment of included studies

Meta-Analysis

Used to systematically synthesize or merge the findings of single, independent studies, using statistical methods to calculate an overall or ‘absolute’ effect.

• Combines results from multiple empirical studies
• Requires systematic review first
• Use well recognized, systematic methods to account for differences in sample size, variability (heterogeneity) in study approach and findings (treatment effects)
• Test how sensitive their results are to their own systematic review protocol

For additional types of reviews please see these articles:

• Sutton, A., Clowes, M., Preston, L. and Booth, A. (2019), Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J, 36: 202-222. https://doi.org/10.1111/hir.12276
• Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
• << Previous: About Systematic Review
• Next: Sources >>
• Last Updated: May 17, 2024 10:04 AM
• URL: https://libguides.utsa.edu/systematicreview
• Library Locations
• Staff Directory
• 508 Compliance
• Site Search
• © The University of Texas at San Antonio
• Information: 210-458-4011
• Campus Alerts
• Required Links
• UTSA Policies
• Report Fraud

• Search Menu
• Advance articles
• Editor's Choice
• Author Guidelines
• Submission Site
• Open Access
• About The British Journal of Social Work
• About the British Association of Social Workers
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

• Introduction
• Limitations
• Supplementary material

Social Workers’ Perceived Barriers and Facilitators to Social Work Practice in Schools: A Scoping Review

• Article contents
• Figures & tables
• Supplementary Data

Sarah Binks, Lyndal Hickey, Airin Heath, Anna Bornemisza, Lauren Goulding, Arno Parolini, Social Workers’ Perceived Barriers and Facilitators to Social Work Practice in Schools: A Scoping Review, The British Journal of Social Work , 2024;, bcae046, https://doi.org/10.1093/bjsw/bcae046

• Permissions Icon Permissions

The aim of this scoping review was to establish the breadth of the academic literature regarding the barriers and facilitators to social work practice in schools as perceived by School Social Workers (SSWs). Following the PRISMA-ScR Scoping Review Framework, 42 articles were identified as meeting the inclusion criteria. Five interrelated themes related to the barriers and facilitators to SSW practice were identified: (1) Inadequacy of service delivery infrastructure; (2) SSWs’ role ambiguities and expectations; (3) SSWs’ competency, knowledge and support; (4) School climate and context; and (5) Cultivating relationships and engagement. This scoping review found that social workers perceive far greater barriers than facilitators when delivering services in school settings, with limited evidence related to the facilitators that enhance School Social Work (SSW) practice. Further research regarding the facilitators of SSW practice is needed, specifically in countries where research on this topic is emergent.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1468-263X
• Print ISSN 0045-3102
• Copyright © 2024 British Association of Social Workers
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Systematic Review
• Open access
• Published: 20 May 2024

Prevalence of hearing impairment in neonatal encephalopathy due to hypoxia-ischemia: a systematic review and meta-analysis

• Dinesh Pawale 1 ,
• Anurag Fursule 1 ,
• Jason Tan 1 , 2 ,
• Deepika Wagh 1 , 2 ,
• Sanjay Patole 2 , 3 &
• Shripada Rao 1 , 2  

Pediatric Research ( 2024 ) Cite this article

Metrics details

This systematic review was undertaken to estimate the overall prevalence of hearing impairment in survivors of neonatal HIE.

PubMed, EMBASE, CINAHL, EMCARE and Cochrane databases, mednar (gray literature) were searched till January 2023. Randomized controlled trials and observational studies were included. The main outcome was estimation of overall prevalence of hearing impairment in survivors of HIE.

A total of 71studies (5821 infants assessed for hearing impairment) were included of which 56 were from high income countries (HIC) and 15 from low- or middle-income countries (LMIC). Overall prevalence rate of hearing impairment in cooled infants was 5% (95% CI: 3–6%, n  = 4868) and 3% (95% CI: 1–6%, n  = 953) in non-cooled HIE infants. The prevalence rate in cooled HIE infants in LMICs was 7% (95% CI: 2–15%) and in HICs was 4% (95% CI: 3–5%). The prevalence rate in non-cooled HIE infants in LMICs was 8% (95% CI: 2–17%) and HICs was 2% (95% CI: 0–4%).

Conclusions

These results would be useful for counseling parents, and in acting as benchmark when comparing institutional data, and while monitoring future RCTs testing new interventions in HIE. There is a need for more data from LMICs and standardization of reporting hearing impairment.

The overall prevalence rate of hearing impairment in cooled infants with HIE was 5% (95% CI: 3–6%) and 3% (95% CI: 1–6%) in the non-cooled infants.

The prevalence rate in cooled HIE infants in LMICs was 7% (95% CI: 2–15%) and in HICs was 4% (95% CI: 3–5%).

The prevalence rate in non-cooled HIE infants in LMICs was 8% (95% CI: 2–17%) and HICs was 2% (95% CI: 0–4%).

These results would be useful for counseling parents, and in acting as benchmark when comparing institutional data, and while monitoring future RCTs testing new interventions in HIE.

Similar content being viewed by others

Growth and developmental outcomes of infants with hypoxic ischemic encephalopathy

Three-year outcome following neonatal encephalopathy in a high-survival cohort

Prevention of severe brain injury in very preterm neonates: A quality improvement initiative

Introduction.

Hypoxic ischemic encephalopathy (HIE) is a leading cause of neonatal brain injury, with an incidence of 1.5 per 1000 live births in developed countries and 2.3–26.5 per 1000 live births in lower and middle-income countries. 1 , 2 The sequelae of HIE encompass motor deficits, intellectual disability, and hearing and vision impairments. 3 , 4 , 5 , 6 Hearing loss has the potential to hinder a child’s linguistic progress, communication abilities, social wellbeing and overall quality of life, especially in socio-demographically disadvantaged children 7 , 8 , 9 Despite its significance, the overall prevalence of hearing impairment in survivors of HIE remains unclear. The 2013 Cochrane meta-analysis of seven Randomized controlled trials (RCTs) found that the incidence of hearing impairment was 3.8% (15/396) among survivors who received hypothermia and 5.8% (19/324) in those who received normothermia. 4 The results of the Cochrane review suggested that there was no significant impact of cooling on hearing impairment (RR 0.66, 95% CI: 0.35, 1.26). 4 The recent network meta-analysis included only seven RCTs evaluating hearing impairment, but the incidence of hearing impairment in neonatal HIE was not reported. 10 The other recent systematic reviews did not report on hearing outcomes. 11 , 12 , 13 While managing an infant with HIE, an important question that clinicians and parents face is “what are the chances of developing hearing impairment if the infant were to survive?”. To answer that question, it is essential to know the prevalence rates of hearing impairment in infants with HIE based on the current literature. Such information would be useful for counselling parents, and allocating resources for early intervention and in acting as benchmark when comparing instituitional data. Hence, this systematic review was undertaken to estimate the overall prevalence of hearing impairment in survivors of neonatal HIE.

Guidelines from the Joanna Biggs Institute were followed for conducting and reporting this systematic review. 14 Ethics approval was not required. The protocol was registered in PROSPERO (CRD42022335943).

Literature search

MEDLINE through PubMed, Embase, CINAHL, Emcare and Cochrane databases were searched in January 2023. Gray literature was searched through Mednar ( https://mednar.com ). Two reviewers conducted the literature search independently. The reference lists of included studies and other relevant articles were searched to identify additional studies. No language restrictions were applied.

MEDLINE was searched through PubMed using the following search terms:

((((((HIE) OR (hypoxic ischemic encephalopathy)) OR (birth asphyxia)) OR (perinatal asphyxia)) OR (neonatal encephalopathy)) AND ((((((((((deafness) OR (hearing loss)) OR (hearing impairment)) OR (sensorineural deafness)) OR (sensorineural hearing loss)) OR (sensorineural hearing impairment)) OR (auditory dysfunction)) OR (auditory impairment)) OR (cochlear implant)) OR (outcome))) OR ((hypoxic ischemic encephalopathy) AND (disability)).

The automatic mapping system of PubMed expanded it to the following terms:

((“HIE” OR (“hypoxic ischemic encephalopathy” OR “hypoxia ischemia, brain” OR (“hypoxia ischemia” AND “brain”) OR “brain hypoxia-ischemia” OR (“hypoxic” AND “ischemic” AND “encephalopathy”) OR “hypoxic ischemic encephalopathy”) OR (“asphyxia neonatorum” OR (“asphyxia” AND “neonatorum”) OR “asphyxia neonatorum” OR (“birth” AND “asphyxia”) OR “birth asphyxia”) OR ((“perinatal” OR “perinatally” OR “perinatals”) AND (“asphyxia” OR “asphyxia” OR “asphyxias”)) OR ((“infant, newborn” OR (“infant” AND “newborn”) OR “newborn infant” OR “neonatal” OR “neonate” OR “neonates” OR “neonatality” OR “neonatals” OR “neonate s”) AND (“brain diseases” OR (“brain” AND “diseases”) OR “brain diseases” OR “encephalopathies” OR “encephalopathy”))) AND (“deafness” OR “deafness” OR “deafnesses” OR (“hearing loss” OR (“hearing” AND “loss”) OR “hearing loss”) OR (“hearing loss” OR (“hearing” AND “loss”) OR “hearing loss” OR (“hearing” AND “impairment”) OR “hearing impairment”) OR (“hearing loss, sensorineural” OR (“hearing” AND “loss” AND “sensorineural”) OR “sensorineural hearing loss” OR (“sensorineural” AND “deafness”) OR “sensorineural deafness”) OR (“hearing loss, sensorineural” OR (“hearing” AND “loss” AND “sensorineural”) OR “sensorineural hearing loss” OR (“sensorineural” AND “hearing” AND “loss”)) OR (“hearing loss, sensorineural” OR (“hearing” AND “loss” AND “sensorineural”) OR “sensorineural hearing loss” OR (“sensorineural” AND “hearing” AND “impairment”) OR “sensorineural hearing impairment”) OR (“hearing disorders” OR (“hearing” AND” disorders”) OR “hearing disorders” OR (“auditory” AND “dysfunction”) OR “auditory dysfunction”) OR ((“auditorially” OR “auditory”) AND (“impair” OR “impaired” OR “impairement” OR “impairements” OR “impairing” OR “impairment” OR “impairments” OR “impairs”)))) OR ((“hypoxic ischemic encephalopathy” OR “hypoxia ischemia, brain” OR (“hypoxia ischemia” AND “brain”) OR “brain hypoxia-ischemia” OR (“hypoxic” AND “ischemic” AND “encephalopathy”) OR “hypoxic ischemic encephalopathy”) AND (“disabilities” OR “disability” OR “disabled persons” OR (“disabled” AND “persons”) OR “disabled persons” OR “disabled” OR “disablement” OR “disablements” OR “disabling” OR “disability”)). Similar terms were used for other databases.

Inclusion criteria

Eligible studies from year 2000 onwards which reported the incidence of hearing impairment in survivors of HIE were included. Studies in which neonates received hypothermia as standard of care or as part of clinical trial were included. Similarly, all studies in which neonates with HIE received normothermia as standard of care or as part of clinical trial were included. Studies published from year 2000 onwards were included to ensure appropriate comparison and contemporaneousness of the data, considering the publication of the first pilot study on therapeutic hypothermia in HIE at the time. 15 Review articles, editorials, case reports, letters and commentaries were excluded.

The diagnosis of hearing impairment could have been based on the auditory brainstem response (ABR) or otoacoustic emissions (OAE) prior to discharge or subsequent audiology assessments or could have been reported as part of formal developmental assessments.

Quality assessment

We used the quality assessment tool from the Joanna Briggs Institute. 16 which has the following criteria: (1) Was the sample frame appropriate to address the target population? (2) Was the study population sampled in an appropriate way? (3) Was the sample size adequate? (4) Were the study subjects and settings described in detail? (5) Was the data analysis conducted with sufficient coverage of the identified sample? (6) Were valid methods used for identification of the condition? (7) Was the condition measured in a standard, reliable way for all the participants? (8) Was there appropriate statistical analysis? (9) Was the response rate adequate, and if not, was the low response rate managed appropriately? The criteria were rated as either yes, no, not clear, or not applicable.

Data extraction

Two reviewers independently extracted the data using a prespecified data collection form. Information about the study design and outcomes was verified by three reviewers independently. Disagreements were resolved through discussions. Where necessary, authors of the included studies were contacted, requesting additional information from their studies.

Data synthesis

Meta-analysis was conducted using Stata 18 (StataCorp, 4905 Lakeway Drive, College Station, Texas 77845 USA). 17 Using the metaprop command to derive the pooled estimation of prevalence. 18 The Freeman–Tukey Double Arcsine Transformation was used to stabilize the variances. The prevalence rates of hearing impairment in individual studies were computed using the formula: (number of infants with hearing impairment/number of infants assessed for hearing impairment). We used the absolute number of observed events and calculated the proportions and 95% confidence intervals (CIs), assuming a binomial distribution. A logistic normal random effects model was fitted. Heterogeneity was assessed by using the χ2 test and the I 2 statistic. In addition, the proportions with their 95% CI values from individual studies were also presented in a forest plot. Funnel plots were used for assessing publication bias. Eggers’s test was used as formal test of funnel plot asymmetry and publication bias. Summary estimates of prevalence rates of hearing impairment were calculated separately for cooled and non-cooled neonates.

Figure  1 provides details of the study selection process. Included studies were segregated into two groups based on whether infants received cooling or not. Infants in the non-cooled arms of RCTs were grouped together with observational studies in which infants were not cooled. Similarly, infants in the cooled arms of RCTs were grouped together with observational studies wherein infants were cooled. A total of 71 studies were included in the systematic review, of which 52 provided information on cooled infants, seven provided information on non-cooled infants and the remaining 11 provided information on both cooled and non-cooled infants in the published manuscripts. Upon request, one author provided information from their published RCT. 19

Preferred reporting items for systematic reviews and meta-analyses flow diagram.

Studies in which infants with HIE were cooled

A total of 64 studies reported the outcomes on hearing assessment in cooled HIE infants (Table  1 ). 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 Amongst included studies, 15 were RCTs 19 , 22 , 31 , 32 , 35 , 43 , 47 , 60 , 61 , 64 , 70 , 72 , 73 , 74 , 75 and 49 were observational studies. 20 , 21 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 33 , 34 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 44 , 45 , 46 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 62 , 63 , 65 , 66 , 67 , 68 , 69 , 71 , 72 , 76 , 77 , 78 , 79 , 80 , 81 , 82 The total sample size was 7990 cooled infants, out of which 4868 were assessed for hearing impairment. The median sample size in the included studies was 61 (IQR 27–104; range: 5–1484).

Full information on the total number of survivors from all included studies was not available. However, based on the available data from 38 studies where such information was available, the median follow up rates among survivors in individual studies were 96.3% (IQR: 86.3–100%, range: 13.5–100%).The median age at reported follow up was 22 months (IQR: 18–24 months). In the majority of studies, hearing impairment was reported as a part of developmental assessment using validated tools such as Bayley or Griffiths Scales. Fifteen studies reported about the diagnostic tools used for hearing impairment assessment, with ABR or OAE being the commonly used tools. 19 , 20 , 26 , 33 , 36 , 44 , 48 , 50 , 51 , 58 , 66 , 72 , 78 , 79 , 82 Five studies reported about diagnosis of hearing impairment before initial discharge from hospital. 19 , 26 , 48 , 66 , 79

Studies in which infants with HIE were not cooled

A total of 19 studies reported the outcome of hearing impairment in non-cooled HIE infants (Table  2 ). 19 , 22 , 31 , 35 , 43 , 47 , 60 , 64 , 72 , 73 , 74 , 75 , 83 , 84 , 85 , 86 , 87 , 88 , 89 Amongst included studies, 12 were non-cooled arms of RCTs. 19 , 22 , 31 , 35 , 43 , 47 , 60 , 64 , 72 , 73 , 74 , 75 and seven were observational studies. 83 , 84 , 85 , 86 , 87 , 88 , 89 The total sample size was 1402 infants, out of which 953 were assessed for hearing impairment. The median sample size in the included studies was 61 (IQR 22–110; range: 3–206). Full information on the total number of survivors from all included studies was not available. Based on 11 studies on non-cooled infants in which such information was available, median follow-up rates amongst survivors in individual studies was 100% (IQR: 83.3–100%, Range: 29.3–100%). In majority of studies, hearing impairment was reported as a part of developmental assessment. The median age at reported follow up was 20 months (IQR: 13–22 months). Three studies reported information about the diagnostic tools used for hearing impairment assessment, all of them using the ABR. 19 , 72 , 84 .

Meta-analysis

The overall prevalence rate of hearing impairment among survivors of cooled HIE infants was 5% (95% CI: 3–6%; I 2  = 59.88%, Fig.  2 ). The prevalence rate of hearing impairment in survivors of non-cooled HIE infants was 3% (95% CI: 1–6%; I 2  = 51.42%, Fig.  3 ).

Forrest plot of studies with cooled HIE infants.

Forrest plot of studies with non-cooled HIE infants.

Subgroup analysis

To explore heterogeneity, a subgroup analysis was conducted based on income level of countries. 90 Out of 71 included studies, 15 (21.1%) were conducted in low or middle-income countries (LMICs), 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 84 , 88 whereas 56 (78.8%) were from high-income countries (HICs). 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 83 , 85 , 86 , 87 , 89 The prevalence rate of hearing impairment in cooled infants in LMICs was 7% (95% CI: 2–15%) and in HICs was 4% (95% CI: 3–5%) (Fig.  2 ). The prevalence rate in non-cooled infants in LMICs was 8% (95% CI: 2–17%) and in HICs was 2% (95% CI: 0–4%) (Fig.  3 ).

Publication bias

The funnel plots and the p values on the Egger’s test for the studies with cooled infants ( P  = 0.023) suggested publication bias while funnel plot for the studies with non-cooled infants ( P  = 0.103) did not suggest statistically significant publication bias (Figs.  4 and 5 ).

Funnel plot of studies with cooled HIE infants.

Funnel plot of studies with non-cooled HIE infants.

Sensitivity analysis

In further explore heterogeneity, the following analyses were conducted.

Prevalence of hearing impairment was estimated separately for selective head cooling, whole body cooling and mixed cooling. The results were similar with a prevalence between 4 and 6%. (Table  3 ).

In many studies, cooling was initiated even in infants with Stage I HIE. 21 , 31 , 37 , 38 , 40 , 43 , 46 , 48 , 54 , 55 , 65 , 68 , 72 , 76 , 83 , 84 , 87 The inclusion of such infants in the meta-analysis could have resulted in an underestimation of the overall prevalence rates in moderate to severe HIE. Hence, we conducted a sensitivity analysis by analyzing them separately. The overall prevalence was 5% in moderate to severe HIE whereas it was 3% in studies that had included infants even with mild HIE (Table  3 ).

The prevalence of hearing impairment in cooled HIE surviving infants among studies that had ≥80% follow-up was 6% and in <80% follow-up was 4%. The prevalence of hearing impairment in non-cooled HIE infants with studies that had more than ≥80% follow-up was 1% and that in <80% follow-up was 7% (Table  3 ).

The prevalence of hearing impairment in cooled HIE surviving infants among studies that had mentioned the use of BERA/OAE was 9%. The prevalence of hearing impairment was 4% where studies reported it as part of formal developmental assessment. The prevalence in non-cooled HIE surviving infants among studies that had mentioned the use of BERA/OAE was 3%. The prevalence of hearing impairment was 4% where studies reported it as part of formal developmental assessment (Table  3 ).

Risk of bias

The risk of bias was low in the majority of the domains in the included studies (Tables  4 and 5 ).

In this systematic review that included 5821 infants from 71 studies, the overall prevalence rate of hearing impairment in surviving cooled infants with HIE was 5% [(95% CI 3–6%), n  = 4868 from 64 studies)] and 3% [(95% CI 1–6%), n  = 953 infants from 19 studies] in non-cooled infants during the same time period. These rates are significantly higher than the global prevalence of 0.1–0.4% (1–4 per 1000 live births) in general neonatal population 91 and 1.57% (15.7 per 1000) in NICU population. 92 These results would be useful as a benchmark for comparing institutional data, counseling parents and guiding policy makers. 93 The results will also be useful for data and safety monitoring committees (DSMCs) of randomized controlled trials evaluating  potentially ototoxic drugs in neonates with HIE. 94 The dose finding study of bumetanide for neonatal seizures (NEMO trial) was stopped prematurely by the DSMC due insufficient efficacy and a potential increased prevalence of hearing loss (3/11 or 27%) in infants with HIE who had received bumetanide. 58 In that context, the results of our meta-analysis have the potential to guide the DSMCs while monitoring new interventions in neonates with HIE.

Our overall results are similar to the Cochrane meta-analysis of seven RCTs that found the incidence to be 3.8% (15/396) among survivors who had received hypothermia and 5.8% (19/324) in those who received normothermia. 4 Even though our study was not about comparing cooling versus normothermia, the prevalence rates of 5% [(95% CI 3–6%), and 3% [(95% CI 1–6%) respectively with overlapping confidence intervals suggests that cooling may not have an impact on hearing impairment, similar to the findings of the Cochrane review.

Our meta-analysis found a higher prevalence of hearing impairment in LMICs compared to HICs. The probable reasons for this finding include the use of ototoxic antibiotics without optimal monitoring of drug levels, and higher incidence of sepsis, low birthweight and growth restricted infants in LMICs. Even though the burden of HIE is more in LMICs, there are very limited number of studies from such countries reporting on hearing impairment. Institutions managing neonates with HIE should be provided adequate resources to enable audiology and long-term developmental assessments.

This review highlights the inconsistency among studies in the reporting various aspects of hearing impairment. In the majority of studies, screening or diagnostic tools used for hearing impairment and timing of assessment were not mentioned. In many studies, the severity of hearing impairment was categorized as not requiring amplification, corrected by amplification, or not corrected by amplification. 21 , 22 , 24 , 32 , 34 , 35 , 37 , 42 , 43 , 45 , 46 , 47 , 52 , 56 , 60 , 61 , 64 , 72 , 73 , 75 , 80 , 83 However, in some studies, the severity was more objectively classified based on the level of hearing threshold in decibels. 26 , 32 , 35 , 48 , 50 , 51 , 58 , 84 Details regarding the laterality of hearing impairment were found in only 10 studies, 26 , 33 , 35 , 50 , 51 , 58 , 65 , 76 , 79 , 84 while the severity was addressed in only 15 studies. 20 , 26 , 28 , 37 , 39 , 40 , 42 , 43 , 51 , 56 , 58 , 60 , 64 , 67 , 72 , 89 Even among the subset of fifteen studies, only three provided a grading system for hearing impairment severity. 26 , 51 , 60 Given the importance of hearing in acquiring language and communication skills, future studies should incorporate crucial information such as the age at screening, age at definitive diagnosis, tools used for screening and definitive diagnosis, severity, laterality, age at intervention, and type of intervention to enable comprehensive understanding. A standardized approach would facilitate more accurate comparisons across studies and enable healthcare providers to develop evidence-based strategies for the prevention and management of hearing impairment in this population.

The limitations of the systematic review include the presence of heterogeneity, insufficient information on the severity and laterality of hearing impairment and on the methods used for assessing hearing. The strengths of our review include the robust and comprehensive literature search, large sample size, inclusion of RCTs as well as real life data, formal assessment of publication bias and exploration of heterogeneity through sensitivity and subgroup analyses.

The overall prevalence rate of hearing impairment in cooled surviving infants was 5% (95% CI 3–6%), and 3% (95% CI 1–6%) in the non-cooled surviving infants with HIE. These results would be useful for counseling parents, and in acting as benchmark when comparing institutional data, and while monitoring future RCTs testing new interventions in HIE.

Kurinczuk, J. J., White-Koning, M. & Badawi, N. Epidemiology of neonatal encephalopathy and hypoxic-ischaemic encephalopathy. Early Hum. Dev. 86 , 329–338 (2010).

Article   PubMed   Google Scholar  

Lawn, J. E. et al. Two million intrapartum-related stillbirths and neonatal deaths: where, why, and what can be done? Int. J. Gynaecol. Obstet. 107 , S5–S18 (2009).

Eunson, P. The long-term health, social, and financial burden of hypoxic-ischaemic encephalopathy. Dev. Med. Child Neurol. 57 , 48–50 (2015).

Jacobs, S. E. et al. Cooling for newborns with hypoxic ischaemic encephalopathy. Cochrane Database Syst. Rev. 2013 , Cd003311 (2013).

PubMed   PubMed Central   Google Scholar  

Zhang, S. et al. Birth Asphyxia Is Associated With Increased Risk of Cerebral Palsy: A Meta-Analysis. Front. Neurol. 11 , 704 (2020).

Article   PubMed   PubMed Central   Google Scholar  

Lee, B. L. & Glass, H. C. Cognitive outcomes in late childhood and adolescence of neonatal hypoxic-ischemic encephalopathy. Clin. Exp. Pediatr. 64 , 608–618 (2021).

GBD 2019 Hearing Loss Collaborators. Hearing loss prevalence and years lived with disability, 1990-2019: findings from the Global Burden of Disease Study 2019. Lancet . 397 , 996–1009 (2021).

Lieu, J. E. C., Kenna, M., Anne, S. & Davidson, L. Hearing Loss in Children: A Review. JAMA 324 , 2195–2205 (2020).

Mietzsch, U. et al. Effects of Hypoxic-Ischemic Encephalopathy and Whole-Body Hypothermia on Neonatal Auditory Function: A Pilot Study. Am. J. Perinatol. 25 , 435–441 (2008).

Lee, C. Y. Z., Chakranon, P. & Lee, S. W. H. Comparative Efficacy and Safety of Neuroprotective Therapies for Neonates With Hypoxic Ischemic Encephalopathy: A Network Meta-Analysis. Front. Pharm. 10 , 1221 (2019).

Article   CAS   Google Scholar  

Bellos, I., Devi, U. & Pandita, A. Therapeutic Hypothermia for Neonatal Encephalopathy in Low- and Middle-Income Countries: A Meta-Analysis. Neonatology 119 , 300–310 (2022).

Diggikar, S. & Krishnegowda, R. Therapeutic Hypothermia for Neonatal Encephalopathy in Low- and Middle-Income Countries: A Literature Review. J. Trop. Pediatr. 68 , fmac016 (2022).

Abate, B. B. et al. Effects of therapeutic hypothermia on death among asphyxiated neonates with hypoxic-ischemic encephalopathy: A systematic review and meta-analysis of randomized control trials. PLoS One 16 , e0247229 (2021).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Peters, M. D. J., Godfrey, C., McInerney, P., Munn, Z., Tricco, A. C.& Khalil, H. Chapter 11: Scoping Reviews. In JBI Manual for Evidence Synthesis (ed Aromataris, E. M. Z.) https://synthesismanual.jbi.global (2020).

Gunn, A. J., Gluckman, P. D. & Gunn, T. R. Selective head cooling in newborn infants after perinatal asphyxia: a safety study. Pediatrics 102 , 885–892 (1998).

Article   CAS   PubMed   Google Scholar  

Aromataris, E. et al. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int. J. Evid. Based Health. 13 , 132–140 (2015).

Article   Google Scholar  

StataCorp L. Stata Statistical Software . Release 18 (StataCorp, 2023).

Nyaga, V. N., Arbyn, M. & Aerts, M. Metaprop: a Stata command to perform meta-analysis of binomial data. Arch. Public Health 72 , 39 (2014).

Soul, J. S. et al. A Pilot Randomized, Controlled, Double-Blind Trial of Bumetanide to Treat Neonatal Seizures. Ann. Neurol. 89 , 327–340 (2021).

Ancora, G. et al. Prognostic value of brain proton MR spectroscopy and diffusion tensor imaging in newborns with hypoxic-ischemic encephalopathy treated by brain cooling. Neuroradiology 55 , 1017–1025 (2013).

Arriaga-Redondo, M. et al. Prognostic value of somatosensory-evoked potentials in the newborn with hypoxic-ischemic encephalopathy after the introduction of therapeutic hypothermia. Eur. J. Pediatr. 181 , 1609–1618 (2022).

Azzopardi, D. V. et al. Moderate hypothermia to treat perinatal asphyxial encephalopathy. N. Engl. J. Med. 361 , 1349–1358 (2009).

Barta, H. et al. Prognostic value of early, conventional proton magnetic resonance spectroscopy in cooled asphyxiated infants. BMC Pediatr. 18 , 302 (2018).

Buchiboyina, A. et al. Servo controlled versus manual cooling methods in neonates with hypoxic ischemic encephalopathy. Early Hum. Dev. 112 , 35–41 (2017).

Cainelli, E. et al. Evoked potentials predict psychomotor development in neonates with normal MRI after hypothermia for hypoxic-ischemic encephalopathy. Clin. Neurophysiol. 129 , 1300–1306 (2018).

Chen, D. Y., Lee, I. C., Wang, X. A. & Wong, S. H. Early Biomarkers and Hearing Impairments in Patients with Neonatal Hypoxic-Ischemic Encephalopathy. Diagnostics 11 , 2056 (2021).

Dereymaeker, A. et al. Automated EEG background analysis to identify neonates with hypoxic-ischemic encephalopathy treated with hypothermia at risk for adverse outcome: A pilot study. Pediatr. Neonatol. 60 , 50–58 (2019).

Dingley, J. T. J. et al. Xenon ventilation during therapeutic hypothermia in neonatal encephalopathy: a feasibility study. Pediatrics 133 , 809–818 (2014).

Edmonds, C. J., Cianfaglione, R., Cornforth, C. & Vollmer, B. Children with neonatal Hypoxic Ischaemic Encephalopathy (HIE) treated with therapeutic hypothermia are not as school ready as their peers. Acta Paediatrica Int. J. Paediatrics 110 , 2756–2765 (2021).

Erdi-Krausz, G. et al. Neonatal hypoxic-ischaemic encephalopathy: Motor impairment beyond cerebral palsy. Eur. J. Paediatr. Neurol. 35 , 74–81 (2021).

Eicher, D. J. et al. Moderate hypothermia in neonatal encephalopathy: efficacy outcomes. Pediatr. Neurol. 32 , 11–17 (2005).

Filippi, L. et al. Safety and efficacy of topiramate in neonates with hypoxic ischemic encephalopathy treated with hypothermia (NeoNATI): a feasibility study. J. Matern. Fetal Neonatal Med. 31 , 973–980 (2018).

Fitzgerald, M. P. et al. Hearing impairment and hypoxia ischaemic encephalopathy: Incidence and associated factors. Eur. J. Paediatr. Neurol. 23 , 81–86 (2019).

Giesinger, R. E. et al. Neurodevelopmental outcome following hypoxic ischaemic encephalopathy and therapeutic hypothermia is related to right ventricular performance at 24-hour postnatal age. Arch. Dis. Child Fetal Neonatal Ed. 107 , 70–75 (2022).

Gluckman, P. D. et al. Selective head cooling with mild systemic hypothermia after neonatal encephalopathy: multicentre randomised trial. Lancet 365 , 663–670 (2005).

Gouveia, R. et al. Low incidence of neurosensorial deafness in infants treated with hypothermia for neonatal hypoxic-ischaemic encephalopathy can be related to avoidance of ototoxic antibiotics. Arch. Dis. Child. 2 , A397 (2014).

Google Scholar  

Grass, B. et al. Short-term neurological improvement in neonates with hypoxic-ischemic encephalopathy predicts neurodevelopmental outcome at 18–24 months. J. Perinat. Med. 48 , 296–303 (2020).

Grass, B. et al. Therapeutic hypothermia for hypoxic-ischemic encephalopathy after perinatal sentinel events: less brain injury on MRI and improved neurodevelopmental outcome at 18-36 months. J. Perinatol. 40 , 633–639 (2020).

Groenendaal, F. et al. Introduction of hypothermia for neonates with perinatal asphyxia in the Netherlands and Flanders. Neonatology 104 , 15–21 (2013).

Robertsson Grossmann, K., Eriksson Westblad, M., Blennow, M.& Lindström, K. Outcome at early school age and adolescence after hypothermia-treated hypoxic-ischaemic encephalopathy: an observational, population-based study. Arch. Dis. Child Fetal Neonatal Ed. 108 , 295–301 (2022).

Guillot, M. et al. Influence of timing of initiation of therapeutic hypothermia on brain MRI and neurodevelopment at 18 months in infants with HIE: a retrospective cohort study. BMJ Paediatr. Open 3 , e000442 (2019).

Herrera, T. I. et al. Outcomes of preterm infants treated with hypothermia for hypoxic-ischemic encephalopathy. Early Hum. Dev. 125 , 1–7 (2018).

Jacobs, S. E. et al. Whole-body hypothermia for term and near-term newborns with hypoxic-ischemic encephalopathy: a randomized controlled trial. Arch. Pediatr. Adolesc. Med . 165 , 692–700 (2011).

Kumar, K. et al. Hearing loss in infants with hypoxic-ischemic encephalopathy. Pediatrics 141 , 525 (2016).

Labat, J. et al. Hypothermia for neonatal hypoxic-ischemic encephalopathy: Retrospective descriptive study of features associated with poor outcome. Arch Pediatr. 30 , 93–99 (2022).

Lally, P. J. et al. Magnetic resonance spectroscopy assessment of brain injury after moderate hypothermia in neonatal encephalopathy: a prospective multicentre cohort study. Lancet Neurol. 18 , 35–45 (2019).

Laptook, A. R. et al. Effect of Therapeutic Hypothermia Initiated After 6 h of Age on Death or Disability Among Newborns With Hypoxic-Ischemic Encephalopathy: A Randomized Clinical Trial. JAMA J. Am. Med. Assoc. 318 , 1550–1560 (2017).

Lee, I. C., Yang, J. J. & Liou, Y. M. Early Blood Glucose Level Post-Admission Correlates with the Outcomes and Oxidative Stress in Neonatal Hypoxic-Ischemic Encephalopathy. Antioxidants 11 , 39 (2021).

Lee-Kelland, R. et al. School-age outcomes of children without cerebral palsy cooled for neonatal hypoxic-ischaemic encephalopathy in 2008-2010. Arch. Dis. Child Fetal Neonatal Ed. 105 , 8–13 (2020).

Massaro, A. N. et al. Quantitative analysis of magnetic resonance images and neurological outcome in encephalopathic neonates treated with whole-body hypothermia. J. Perinatol. 30 , 596–603 (2010).

Michniewicz, B. et al. Hearing Impairment in Infants with Hypoxic Ischemic Encephalopathy Treated with Hypothermia. Therapeutic Hypothermia Temp. Manag. 12 , 8–15 (2022).

Montaldo, P. et al. Electrocardiographic and echocardiographic changes during therapeutic hypothermia in encephalopathic infants with long-term adverse outcome. Resuscitation 130 , 99–104 (2018).

Monzani, A. et al. Early instrumental predictors of long term neurodevelopmental impairment in newborns with perinatal asphyxia treated with therapeutic hypothermia. Signa Vitae 14 , 81–85 (2018).

Natarajan, G. et al. Opioid exposure during therapeutic hypothermia and short-term outcomes in neonatal encephalopathy. J. Perinatol. 42 , 1017–1025 (2022).

Peeples, E. S. et al. Predictive Models of Neurodevelopmental Outcomes After Neonatal Hypoxic-Ischemic Encephalopathy. Pediatrics 147 , e2020022962 (2021).

Pereira, C. O. et al. Prognostic value of near-infrared spectroscopy in hypoxic-ischaemic encephalopathy. An Pediatr 94 , 136–143 (2021).

Pokorna, P. et al. Whole body hypothermia for hypoxic-ischemic encephalopathy: A 6 - Years evaluation of neonatal therapy and follow-up in the Czech Republic. Pediatr. Crit. Care Med. 1 , 81 (2014).

Pressler, R. M. et al. Bumetanide for the treatment of seizures in newborn babies with hypoxic ischaemic encephalopathy (NEMO): an open-label, dose finding, and feasibility phase 1/2 trial. Lancet Neurol. 14 , 469–477 (2015).

Salam, M., Hashim, A. & Singh, J. Role of VEEG monitoring in neonates with HIE undergoing therapeutic hypothermia with correlation of seizures and neurodevelopmental outcome. Neurology 82 , P1.274 (2014).

Shankaran, S. et al. Whole-body hypothermia for neonates with hypoxic-ischemic encephalopathy. N. Engl. J. Med. 353 , 1574–1584 (2005).

Shankaran, S. et al. Effect of Depth and Duration of Cooling on Death or Disability at Age 18 Months Among Neonates With Hypoxic-Ischemic Encephalopathy: A Randomized Clinical Trial. JAMA 318 , 57–67 (2017).

Shellhaas, R. A. et al. An Evaluation of Cerebral and Systemic Predictors of 18-Month Outcomes for Neonates With Hypoxic Ischemic Encephalopathy. J. Child Neurol. 30 , 1526–1531 (2015).

Shibasaki, J. et al. Outcomes related to 10-min Apgar scores of zero in Japan. Arch. Dis. Child Fetal Neonatal Ed. 105 , 64–68 (2020).

Simbruner, G., Mittal, R. A., Rohlmann, F. & Muche, R. Systemic hypothermia after neonatal encephalopathy: outcomes of neo.nEURO.network RCT. Pediatrics 126 , e771–e778 (2010).

Skranes, J. H. et al. Amplitude-Integrated Electroencephalography Improves the Identification of Infants with Encephalopathy for Therapeutic Hypothermia and Predicts Neurodevelopmental Outcomes at 2 Years of Age. J. Pediatr. 187 , 34–42 (2017).

Smit, E. et al. Factors associated with permanent hearing impairment in infants treated with therapeutic hypothermia. J. Pediatr. 163 , 995–1000 (2013).

Thoresen, M. et al. MRI combined with early clinical variables are excellent outcome predictors for newborn infants undergoing therapeutic hypothermia after perinatal asphyxia. EClinicalMedicine 36 , 100885 (2021).

Tsuda, K. et al. Three-year outcome following neonatal encephalopathy in a high-survival cohort. Sci. Rep. 12 , 7945 (2022).

Valera, I. T. et al. Erythropoietin with hypothermia improves outcomes in neonatal hypoxic ischemic encephalopathy. J. Clin. Neonatol. 4 , 244–249 (2015).

Wu, Y. W. et al. Trial of Erythropoietin for Hypoxic-Ischemic Encephalopathy in Newborns. N. Engl. J. Med. 387 , 148–159 (2022).

Xu, Q. et al. Pattern of Brain Injury Predicts Long-Term Epilepsy Following Neonatal Encephalopathy. J. Child Neurol. 34 , 199–209 (2019).

Battin, M. R. et al. Neurodevelopmental outcome of infants treated with head cooling and mild hypothermia after perinatal asphyxia. Pediatrics 107 , 480–484 (2001).

Das, S. et al. Neurological Outcome at 30 Months of Age after Mild Hypothermia via Selective Head Cooling in Term Neonates with Perinatal Asphyxia Using Low-Cost CoolCap: a Single-Center Randomized Control Pilot Trial in India. J. Pediatr. Neurol. 15 , 157–165 (2017).

Gane, B. D. et al. Effect of therapeutic hypothermia on DNA damage and neurodevelopmental outcome among term neonates with perinatal asphyxia: a randomized controlled trial. J. Trop. Pediatr. 60 , 134–140 (2014).

Thayyil, S. et al. Hypothermia for moderate or severe neonatal encephalopathy in low-income and middle-income countries (HELIX): a randomised controlled trial in India, Sri Lanka, and Bangladesh. Lancet Glob. Health 9 , e1273–e1285 (2021).

Celik, K. et al. Neurodevelopmental outcomes in therapeutic hypothermia: A single center experience. Turkish J. Pediatr. Dis. 15 , 99–103 (2021).

Koshy, B., Padankatti, C. S., George, K. C. & Thomas, N. Neurodevelopmental outcome following whole body cooling for perinatal asphyxia. Indian Pediatr. 48 , 982–983 (2011).

PubMed   Google Scholar  

Khuwuthyakorn, V. et al. Two-year outcomes of therapeutic hypothermia in perinatal hypoxic-ischaemic encephalopathy cases at Chiang Mai University Hospital. Arch. Dis. Child. 106 , 516–517 (2021).

Martinez-Hernandez, A. et al. Therapeutic hypothermia in newborns with ischemic hypoxic encephalopathy. Rev. Mexicana de. Pediatr. 87 , 176–182 (2020).

Perez, J. M., Golombek, S. G., Alpan, G. & Sola, A. Using a novel laminar flow unit provided effective total body hypothermia for neonatal hypoxic encephalopathy. Acta Paediatr. 104 , e483–e488 (2015).

Procianoy, R. S. et al. Outcome and Feasibility after 7 Years of Therapeutic Hypothermia in Southern Brazil. Am. J. Perinatol. 37 , 955–961 (2020).

Kadioglu Simsek, G. et al. Hearing screening failure rate in newborn infants with hypoxic ischemic encephalopathy. Int. J. Pediatr. Otorhinolaryngol. 128 , 109691 (2020).

Chalak, L. F. et al. Prospective research in infants with mild encephalopathy identified in the first six hours of life: neurodevelopmental outcomes at 18-22 months. Pediatr. Res. 84 , 861–868 (2018).

Jiang, Z. D., Zhang, Z. & Wilkinson, A. R. Distortion product otoacoustic emissions in term infants after hypoxia-ischaemia. Eur. J. Pediatr. 164 , 84–87 (2005).

Jyoti, R., O’Neil, R. & Hurrion, E. Predicting outcome in term neonates with hypoxic-ischaemic encephalopathy using simplified MR criteria. Pediatr. Radio. 36 , 38–42 (2006).

Kodama, Y., Sameshima, H., Ikeda, T. & Ikenoue, T. Intrapartum fetal heart rate patterns in infants (> or =34 weeks) with poor neurological outcome. Early Hum. Dev. 85 , 235–238 (2009).

Martinez-Biarge, M. et al. White matter and cortical injury in hypoxic-ischemic encephalopathy: Antecedent factors and 2-year outcome. J. Pediatrics 161 , 799–807 (2012).

Mwakyusa, S. D., Manji, K. P. & Massawe, A. W. The hypoxic ischaemic encephalopathy score in predicting neurodevelopmental outcomes among infants with birth asphyxia at the Muhimbili National Hospital, Dar-es-Salaam, Tanzania. J. Trop. Pediatr. 55 , 8–14 (2009).

Parmentier, C. E. J. et al. Outcome of non-cooled asphyxiated infants with under-recognised or delayed-onset encephalopathy. Arch. Dis. Child Fetal Neonatal Ed. 107 , 364–370 (2022).

Bank, T. W. World Bank Country and Lending Groups Internet. Available from: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups#:~:text=World%20Bank%20Country%20and%20Lending%20Groups%20%20,%20Taiwan%2C%20China%20%208%20more%20rows%20 (2023).

Edmond, K. et al. Effectiveness of universal newborn hearing screening: A systematic review and meta-analysis. J. Glob. Health 12 , 12006 (2022).

Bussé, A. M. L. et al. Prevalence of permanent neonatal hearing impairment: systematic review and Bayesian meta-analysis. Int. J. Audio. 59 , 475–485 (2020).

Chadha, S., Kamenov, K. & Cieza, A. The world report on hearing, 2021. Bull. World Health Organ 99 , 242 (2021).

DAMOCLES Study Group. A proposed charter for clinical trial data monitoring committees: helping them to do their job well. Lancet . 365 , 711–722. (2005).

Download references

Open Access funding enabled and organized by CAUL and its Member Institutions.

Author information

Authors and affiliations.

Department of Neonatology, Perth Children’s Hospital, Perth, WA, Australia

Dinesh Pawale, Anurag Fursule, Jason Tan, Deepika Wagh & Shripada Rao

School of Medicine, University of Western Australia, Crawley, WA, Australia

Jason Tan, Deepika Wagh, Sanjay Patole & Shripada Rao

Department of Neonatology, King Edwards Memorial Hospital, Perth, WA, Australia

Sanjay Patole

You can also search for this author in PubMed   Google Scholar

Contributions

D.P., A.F., J.T., and D.W. did the review of literature. S.R., D.P., and A.F. did the meta-analyses. D.P., and A.F. coordinated the investigator group and wrote the first draft of the paper. S.R. and S.P. advised on data analysis and edited the paper. S.R., J.T., and S.P. contributed to successive versions of the paper. S. Rao was responsible for overall supervision of the project from inception till publication.

Corresponding author

Correspondence to Shripada Rao .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Prisma_2020_checklist, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Pawale, D., Fursule, A., Tan, J. et al. Prevalence of hearing impairment in neonatal encephalopathy due to hypoxia-ischemia: a systematic review and meta-analysis. Pediatr Res (2024). https://doi.org/10.1038/s41390-024-03261-w

Download citation

Received : 01 March 2024

Revised : 17 April 2024

Accepted : 29 April 2024

Published : 20 May 2024

DOI : https://doi.org/10.1038/s41390-024-03261-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Open access
• Published: 10 May 2024

Challenges and opportunities of English as the medium of instruction in diploma midwifery programs in Bangladesh: a mixed-methods study

• Anna Williams 1 ,
• Jennifer R. Stevens 2 ,
• Rondi Anderson 3 &
• Malin Bogren 4  

BMC Medical Education volume  24 , Article number:  523 ( 2024 ) Cite this article

246 Accesses

Metrics details

English is generally recognized as the international language of science and most research on evidence-based medicine is produced in English. While Bangla is the dominant language in Bangladesh, public midwifery degree programs use English as the medium of instruction (EMI). This enables faculty and student access to the latest evidence-based midwifery content, which is essential for provision of quality care later. Yet, it also poses a barrier, as limited English mastery among students and faculty limits both teaching and learning.

This mixed-methods study investigates the challenges and opportunities associated with the implementation of EMI in the context of diploma midwifery education in Bangladesh. Surveys were sent to principals at 38 public midwifery education institutions, and 14 English instructors at those schools. Additionally, ten key informant interviews were held with select knowledgeable stakeholders with key themes identified.

Surveys found that English instructors are primarily guest lecturers, trained in general or business English, without a standardized curriculum or functional English language laboratories. Three themes were identified in the key informant interviews. First, in addition to students’ challenges with English, faculty mastery of English presented challenges as well. Second, language labs were poorly maintained, often non-functional, and lacked faculty. Third, an alternative education model, such as the English for Specific Purposes (ESP) curriculum,  has potential to strengthen English competencies within midwifery schools.

Conclusions

ESP, which teaches English for application in a specific discipline, is one option available in Bangladesh for midwifery education. Native language instruction and the middle ground of multilingualism are also useful options. Although a major undertaking, investing in an ESP model and translation of technical midwifery content into relevant mother tongues may provide faster and more complete learning. In addition, a tiered system of requirements for English competencies tied to higher levels of midwifery education could build bridges to students to help them access global evidence-based care resources. Higher levels might emphasize English more heavily, while the diploma level would follow a multilingualism approach, teach using an ESP curriculum, and have complementary emphasis on the mother tongue.

Peer Review reports

Introduction

As the international language of science, English holds an important position in the education of healthcare professionals. Globally, most scientific papers are published in English. In many non-native English-speaking countries, English is used as the language of instruction in higher education [ 1 ]. The dominant status held by the English language in the sciences is largely considered to increase global access to scientific information by unifying the scientific community under a single lingua franca [ 2 ].

In Bangladesh, where the mother tongue is Bangla and midwifery diploma programs are taught in English, knowledge of English facilitates student and instructor access to global, continuously updated evidence-based practice guidance. This includes basic and scientific texts, media-based instructional materials (including on life-saving skills), professional journals, and proceedings of medical conferences. Many of these resources are available for free online, which can be particularly useful in healthcare settings that have not integrated evidence-based practice.

In addition to opportunity though, English instruction also creates several challenges. Weak student and faculty English competency may impede midwifery education quality in Bangladesh. Globally, literature has linked limited instructor competency in the language of instruction with reduced depth, nuance, and accuracy in conveying subject matter content [ 3 ]. This can lead to the perpetuation of patterns of care in misalignment with global evidence. In addition, students’ native language proficiency in their topic of study can decline when instruction is in English, limiting native language communication between colleagues on the job later on [ 4 , 5 ].

In this paper, we examine the current status of English language instruction within public diploma midwifery programs in Bangladesh. Midwifery students are not required to demonstrate a certain skill level in English to enter the program. However, they are provided with English classes in the program. Midwifery course materials are in English, while—for ease and practicality—teaching aids and verbal classroom instruction are provided in Bangla. Following graduation, midwifery students must pass a national licensing exam given in English to practice. Upon passing, some new midwives are deployed as public employees and are posted to sub-district health facilities where English is not used by either providers or clients. Others will seek employment as part of non-governmental organization (NGO) projects where English competency can be of value for interacting with global communities, and for participating in NGO-specific on-the-job learning opportunities. The mix of both challenge and opportunity in this context is complex.

Our analysis examines the reasons for the identified English competency gaps within midwifery programs, and potential solutions. We synthesize the findings and discuss solutions in the context of the global literature. Finally, we present a set of viable options for strengthening English competencies among midwifery faculty and students to enable better quality teaching and greater learning comprehension among students.

Study design

We employed a mixed-methods study design [ 6 ] in order to assess the quality of English instruction within education programs, and options for its improvement. Data collection consisted of two surveys of education institutes, a web-search of available English programs in Bangladesh, and key informant interviews. Both surveys followed a structured questionnaire with a combination of open- and closed-ended questions and were designed by the authors. One survey targeted the 38 institute principals and the other targeted 14 of the institutes’ 38 English instructors (those for whom contact information was shared). The web-search focused on generating a list of available English programs in Bangladesh that had viable models that could be tapped into to strengthen English competencies among midwifery faculty and students. Key informant interviews were unstructured and intended to substantiate and deepen understanding of the survey and web-search findings.

No minimum requirements exist for students’ English competencies upon entry into midwifery diploma programs. Students enter directly from higher secondary school (12th standard) and complete the midwifery program over a period of three years. Most students come from modest economic backgrounds having completed their primary and secondary education in Bangla. While English instruction is part of students’ secondary education, skill attainment is low, and assessment standards are not in place to ensure student mastery. To join the program, midwifery students are required to pass a multi-subject entrance exam that includes a component on English competency. However, as no minimum English standard must be met, the exam does not screen out potential midwifery students. Scoring, for instance, is not broken down by subject. This makes it possible to answer zero questions correctly in up to three of the subjects, including English, and pass the exam.

Processes/data collection

Prior to the first survey, principals were contacted by UNFPA with information about the survey and all provided verbal consent to participate. The survey of principals collected general information about the resources available for English instruction at the institutes. It was a nine-item questionnaire with a mix of Yes/No, multiple choice and write-in questions. Specific measures of interest were whether and how many English instructors the institutes had, instructors’ hiring criteria, whether institutes had language labs and if they were in use, and principals’ views on the need for English courses and their ideal mode of delivery (e.g., in-person, online, or a combination). This survey also gathered contact information of institute English instructors. These measures were chosen as they were intended to provide a high-level picture of institutes’ English resources such as faculty availability and qualifications, and use of language labs. To ensure questions were appropriately framed, a pilot test was conducted with two institute principals and small adjustments were subsequently made. Responses were shared via an electronic form sent by email and were used to inform the second survey as well as the key informant interviews. Of the 38 principals, 36 completed the survey.

The second survey, targeting English instructors, gathered information on instructors’ type of employment (e.g., institute faculty or adjunct lecturers); length of employment; student academic focus (e.g., midwifery or nursing); hours of English instruction provided as part of the midwifery diploma program; whether a standard English curriculum was used and if it was tailored toward the healthcare profession; use of digital content in teaching; education and experience in English teaching; and their views on student barriers to learning English. These measures were chosen to provide a basic criterion for assessing quality of English instruction, materials and resources available to students. For instance, instructors’ status as faculty would indicate a stronger degree of integration and belonging to the institute midwifery program than a guest lecturer status which allows for part time instruction with little job security. In addition, use of a standard, professionally developed English curriculum and integration of digital content into classroom learning would be indicative of higher quality than learning materials developed informally by instructors themselves without use of listening content by native speakers in classrooms. The survey was piloted with two English instructors. Based on their feedback, minor adjustments were made to one question, and it was determined that responses were best gathered by phone due to instructors’ limited internet access. Of the 14 instructors contacted, 11 were reached and provided survey responses by phone.

The web-search gathered information on available English language instruction programs for adults in Bangladesh, and the viability of tapping into any of them to improve English competency among midwifery students and faculty. Keywords Bangladesh  +  English courses , English training , English classes , study English and learn English were typed into Google’s search platform. Eleven English language instruction programs were identified. Following this, each program was contacted either by phone or email and further detail about the program’s offerings was collected.

Unstructured key informant interviews were carried out with select knowledgeable individuals to substantiate and enhance the credibility of the survey and web-search findings. Three in-country expert English language instructors and four managers of English language teaching programs were interviewed. In addition, interviews were held with three national-level stakeholders knowledgeable about work to make functional technologically advanced English language laboratories that had been installed at many of the training institutes. Question prompts included queries such as, ‘In your experience, what are the major barriers to Bangla-medium educated students studying in English at the university level?’, ‘What effective methods or curricula are you aware of for improving student English to an appropriate competency level for successful learning in English?’, and, ‘What options do you see for the language lab/s being used, either in their originally intended capacity or otherwise?’

Data analysis

All data were analyzed by the lead researcher. Survey data were entered into a master Excel file and grouped descriptively to highlight trends and outliers, and ultimately enable a clear description of the structure and basic quality attributes (e.g., instructors’ education, hours of English instruction, and curriculum development resources used). Web-search findings were compiled in a second Excel file with columns distinguishing whether they taught general English (often aimed at preparing students for international standard exams), Business English, or English for Specific Purposes (ESP). This enabled separation of standalone English courses taught by individual instructors as part of vocational or academic programs of study in other fields, and programs with an exclusive focus on English language acquisition. Key informant interviews were summarized in a standard notes format using Word. An inductive process of content analysis was carried out, in which content categories were identified and structured to create coherent meaning [ 7 ]. From this, the key overall findings and larger themes that grew from the initial survey and web-search results were drawn out.

The surveys (Tables  1 and 2 ) found that English instructors are primarily long-term male guest lecturers employed at each institute for more than two years. All principal respondents indicated that there is a need for English instruction—18 of the 19 reported that this is best done through a combination of in-person and computer-based instruction. Ten institutes reported that they have an English language lab, but none were used as such. The other institutes did not have language labs. The reported reasons for the labs not being in use were a lack of trained staff to operate them and some components of the technology not being installed or working properly. The findings from the instructors’ survey indicated that English instructors typically develop their own learning materials and teach general English without tailoring content to healthcare contexts. Only two mentioned using a standard textbook to guide their instruction and one described consulting a range of English textbooks to develop learning content. None reported using online or other digital tools for language instruction in their classrooms. Most instructors had an advanced degree (i.e., master’s degree) in English, and seven had received training in teaching English. Interviews with instructors also revealed that they themselves did not have mastery of English, as communication barriers in speaking over the phone appeared consistently across 10 of the 11 instructor respondents.

The web-search and related follow up interviews found that most English instruction programs (10 out of the 11) were designed for teaching general English and/or business English. The majority were offered through private entities aiming to reach individuals intending to study abroad, access employment that required English, or improve their ability to navigate business endeavors in English. One program, developed by the British Council, had flexibility to tailor its structure and some of its content to the needs of midwifery students. However, this was limited in that a significant portion of the content that would be used was developed for global audiences and thus not tailored to a Bangladeshi audience or to any specific discipline. One of the university English programs offered a promising ESP model tailored to midwifery students. It was designed by BRAC University’s Institute of Language for the university’s private midwifery training program.

Three themes emerged from the other key informant interviews (Table  3 ). The first was that, in addition to students’ challenges with English, faculty mastery of English presented challenges as well. Of the 34 faculty members intending to participate in the 2019–2020 cohort for the Dalarna master’s degree, half did not pass the prerequisite English exam. Ultimately, simultaneous English-Bangla translation was necessary for close to half of the faculty to enable their participation in the master’s program. English language limitations also precluded one faculty member from participating in an international PhD program in midwifery.

The second theme highlighted the language labs’ lack of usability. The language labs consisted of computers, an interactive whiteboard, audio-visual equipment, and associated software to allow for individualized direct interactions between teacher and student. However, due to the lack of appropriately trained staff to manage, care for and use the language lab equipment, the investment required to make the labs functional appeared to outweigh the learning advantages doing so would provide. Interviews revealed that work was being done, supported by a donor agency, on just one language lab, to explore whether it could be made functional. The work was described as costly and challenging, and required purchasing a software license from abroad, thus likely being impractical to apply to the other labs and sustain over multiple years.

The third theme was around the ESP curriculum model. The program developers had employed evidence-informed thinking to develop the ESP learning content and consulted student midwives on their learning preferences. Due to the student input, at least 80% of the content was designed to directly relate to the practice of midwifery in Bangladesh, while the remaining 10–20% references globally relevant content. This balance was struck based on students’ expressed interest in having some exposure to English usage outside of Bangladesh for their personal interest. For conversation practice, the modules integrated realistic scenarios of midwives interacting with doctors, nurses and patients. Also built into written activities were exercises where students were prompted to describe relevant health topics they are concurrently studying in their health, science or clinical classes. Given the midwifery students’ educational backgrounds and intended placements in rural parts of Bangladesh, an ESP curriculum model appeared to be the most beneficial existing program to pursue tapping into to strengthen English competencies within midwifery programs. This was because the content would likely be more accessible to students than a general English course by having vocabulary, activities and examples directly relevant to the midwifery profession.

The study findings demonstrate key weaknesses in the current model of English instruction taught in public midwifery programs. Notably, the quantitative findings revealed that some English instructors do not have training in teaching English, and none used standard curricula or online resources to structure and enhance their classroom content. In addition, weak mastery of English among midwifery faculty was identified in the qualitative data, which calls into question faculty’s ability to fully understand and accurately convey content from English learning materials. Global literature indicates that this is not a unique situation. Many healthcare faculty and students in low-resource settings, in fact, are faced with delivering and acquiring knowledge in a language they have not sufficiently mastered [ 8 ]. As a significant barrier to knowledge and skill acquisition for evidence-based care, this requires more attention from global midwifery educators [ 9 ].

Also holding back students’ English development is the finding from both the quantitative and qualitative data that none of the high-tech language labs were being used as intended. This indicates a misalignment with the investment against the reality of the resources at the institutes to use them. While setting up the costly language labs appears to have been a large investment with little to no return, it does demonstrate that strengthening English language instruction in post-secondary public education settings is a priority that the Bangladesh government is willing to invest in. However, scaling up access to an ESP curriculum model tailored to future midwifery practitioners in Bangladesh may be a more worthwhile investment than language labs [ 10 ]. 

The ESP approach teaches English for application in a specific discipline. It does this by using vocabulary, examples, demonstrations, scenarios and practice activities that are directly related to the context and professions those studying English live and work (or are preparing to work) in. One way ESP has been described, attributed to Hutchinson and Waters (1987), is, “ESP should properly be seen not as any particular language product but as an approach to language teaching in which all decisions as to content and method are based on the learner’s reason for learning” [ 11 ]. It is proposed by linguistic education researchers as a viable model for strengthening language mastery and subject matter comprehension in EMI university contexts [ 12 ].

Though it did not arise as a finding, reviewing the literature highlighted that Bangla language instruction may be an additional, potentially viable option. Linguistic research has long shown that students learn more thoroughly and efficiently in their mother tongue [ 12 ]. Another perhaps more desirable option may be multilingualism, which entails recognizing native languages as complementary in EMI classrooms, and using them through verbal instruction and supplemental course materials. Kirkpatrick, a leading scholar of EMI in Asia, suggests that multilingualism be formally integrated into EMI university settings [ 13 ]. This approach is supported by evidence showing that the amount of native language support students need for optimal learning is inversely proportional to their degree of English proficiency [ 14 ].

Ultimately, despite the language related learning limitations identified in this study, and the opportunities presented by native language and multilingualism approaches, there remains a fundamental need for members of the midwifery profession in Bangladesh to use up-to-date guidance on evidence-based midwifery care [ 11 ]. Doing that currently requires English language competence. Perhaps a tiered system of requirements for English competencies that are tied to diploma, Bachelor’s, Master’s and PhD midwifery programs could build bridges for more advanced students to access global resources. Higher academic levels might emphasize English more heavily, while the diploma level could follow a multilingualism approach—teaching using an ESP curriculum and integrating Bangla strategically to support optimal knowledge acquisition for future practice in rural facilities. Ideally, scores on a standard English competency exam would be used to assess students’ language competencies prior to entrance in English-based programs and that this would require more stringent English skill development prior to entering a midwifery program.

Methodological considerations

One of the limitations of this study is that it relied on self-reports and observation, rather than tested language and subject matter competencies. Its strengths though are in the relatively large number of education institutes that participated in the study, and the breadth of knowledge about faculty and student subject matter expertise among study co-authors. It was recognized that the lead researcher might be biased toward pre-determined perceptions of English competencies being a barrier to teaching and learning held by the lead institution (UNFPA). It was also recognized that due to the inherent power imbalance between researcher and participants, the manner of gathering data and engaging with stakeholders may contribute to confirmation bias, with respondents primarily sharing what they anticipated the researcher wished to hear (e.g., that English needed strengthening and the lead agency should take action to support the strengthening). The researcher thus engaged with participants independently of UNFPA and employed reflexivity by designing and carrying out the surveys to remotely collect standard data from institutes, as well as casting a wide net across institutes to increase broad representation. In addition, while institutes were informed that the surveys were gathering information about the English instruction within the institutes, no information was shared about potential new support to institutes. Finally, the researcher validated and gathered further details on the relevant information identified in the surveys through key informant interviews, which were held with stakeholders independent of UNFPA.

Adapting and scaling up the existing ESP modules found in this study, and integrating Bangla where it can enhance subject-matter learning, may be a useful way to help midwifery students and faculty improve their knowledge, skills, and critical thinking related to the field of midwifery. Given the educational backgrounds and likely work locations of most midwives in Bangladesh and many other LMICs, practitioners may want to consider investing in more opportunities for local midwives to teach and learn in their mother tongue. This type of investment would ideally be paired with a tiered system in which more advanced English competencies are required at higher-levels of education to ensure integration of global, evidence-based approaches into local standards of care.

Declarations.

Data availability

The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

Bangladesh Rehabilitation Assistance Committee

English medium instruction

English for Specific Purposes

Low- and Middle-Income Countries

Ministry of Health and Family Welfare

United Nations Population Fund

Macaro E. English medium instruction: global views and countries in focus. Lang Teach. 2019;52(2):231–48.

Article   Google Scholar  

Montgomery S. Does science need a global language? English and the future of research. University of Chicago Press; 2013.

Doiz A, Lasagabaster D, Pavón V. The integration of language and content in English-medium instruction courses: lecturers’ beliefs and practices. Ibérica. 2019;38:151–76.

Google Scholar  

Gallo F, Bermudez-Margareto B, et al. First language attrition: what it is, what it isn’t, and what it can be. National Research University Higher School of Economics; 2019.

Yilmaz G, Schmidt M. First language attrition and bilingualism, adult speakers. Bilingual cognition and language, the state of the science across its sub-fields (Ch. 11). John Benjamin’s Publishing Company.

Polit DF, Beck CT. (2021). Nursing research: generating and assessing evidence for nursing practice. Eleventh edition. Philadelphia, Wolters Kluwer.

Scheufele, B. (2008). Content Analysis, Qualitative. The international encyclopedia of communication John Wiley & Sons.

Pelicioni PHS, Michell A, Rocha dos Santos PC, Schulz JS. Facilitating Access to Current, evidence-based Health Information for Non-english speakers. Healthcare. 2023;11(13):1932.

Pakenham-Walsh N. Improving the availability of health research in languages other than English. Lancet. 2018;8. http://dx.doi.org/10.1016/ S2214-109X(18)30384-X.

Islam M. The differences and similarities between English for Specific purposes(ESP) and English for General purposes(EGP) teachers. Journal of Research in Humanities; 2015.

Lamri C, Dr et al. (2016-2017). English for Specific Purposes (1st Semester) Third Year ‘License’ Level. Department of English Language, Faculty of Arts and Language, University of Tlemcen

Jiang L, Zhang LJ, May S. (2016). Implementing English-medium instruction (EMI) in China: teachers’ practices and perceptions, and students’ learning motivation and needs. Int J Bilingual Educ Bilinguaism 22(2).

Kirkpatrick A. The rise of EMI: challenges for Asia. In, English medium instruction: global views and countries in focus. Lang Teach. 2015;52(2):231–48.

Kavaliauskiene G. Role of the mother tongue in learning English for specific purposes. ESP World. 2009;1(22):8.

Download references

Acknowledgements

The authors acknowledge Farida Begum, Rabeya Basri, and Pronita Raha for their contributions to data collection for this assessment.

This project under which this study was carried out was funded by funded by the Foreign Commonwealth and Development Office.

Open access funding provided by University of Gothenburg.

Author information

Authors and affiliations.

Data, Design + Writing, Portland, OR, USA

Anna Williams

Goodbirth Network, North Adams, USA, MA

Jennifer R. Stevens

Project HOPE, Washington DC, USA

Rondi Anderson

University of Gothenburg, Gothenburg, Sweden

Malin Bogren

You can also search for this author in PubMed   Google Scholar

Contributions

Authors contributions in the development of this paper were as follows: AW- Concept, acquisition, drafting, revision, analysis, interpretation. JRS- Concept, revision. RA- Concept, analysis MB- Revision, analysis, interpretationAll authors read and approved the final manuscript.

Ethics declarations

Ethics approval.

This study was part of a larger project in Bangladesh approved by the Ministry of Health and Family Welfare (MOHFW) with project ID UZJ31. The MOHFW project approval allows data collection of this type, that is carried out as part of routine program monitoring and improvement, including informed verbal consent for surveys and key informant interviews.

Consent for publication

Not applicable.

Competing interests

The authors of this study have no competing interests and no conflicts of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Williams, A., Stevens, J., Anderson, R. et al. Challenges and opportunities of English as the medium of instruction in diploma midwifery programs in Bangladesh: a mixed-methods study. BMC Med Educ 24 , 523 (2024). https://doi.org/10.1186/s12909-024-05499-8

Download citation

Received : 31 July 2023

Accepted : 02 May 2024

Published : 10 May 2024

DOI : https://doi.org/10.1186/s12909-024-05499-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• “English for special purposes”
• “English medium instruction”

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley Open Access Collection

An overview of methodological approaches in systematic reviews

Prabhakar veginadu.

1 Department of Rural Clinical Sciences, La Trobe Rural Health School, La Trobe University, Bendigo Victoria, Australia

Hanny Calache

2 Lincoln International Institute for Rural Health, University of Lincoln, Brayford Pool, Lincoln UK

Akshaya Pandian

3 Department of Orthodontics, Saveetha Dental College, Chennai Tamil Nadu, India

Mohd Masood

Associated data.

APPENDIX B: List of excluded studies with detailed reasons for exclusion

APPENDIX C: Quality assessment of included reviews using AMSTAR 2

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full‐text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non‐English, gray, and unpublished literature, and (c) use of text‐mining approaches during title and abstract screening.

The overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

1. INTRODUCTION

Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the “gold standard” of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search, appraise, and synthesize the available evidence. 3 Several guidelines, developed by various organizations, are available for the conduct of an SR; 4 , 5 , 6 , 7 among these, Cochrane is considered a pioneer in developing rigorous and highly structured methodology for the conduct of SRs. 8 The guidelines developed by these organizations outline seven fundamental steps required in SR process: defining the scope of the review and eligibility criteria, literature searching and retrieval, selecting eligible studies, extracting relevant data, assessing risk of bias (RoB) in included studies, synthesizing results, and assessing certainty of evidence (CoE) and presenting findings. 4 , 5 , 6 , 7

The methodological rigor involved in an SR can require a significant amount of time and resource, which may not always be available. 9 As a result, there has been a proliferation of modifications made to the traditional SR process, such as refining, shortening, bypassing, or omitting one or more steps, 10 , 11 for example, limits on the number and type of databases searched, limits on publication date, language, and types of studies included, and limiting to one reviewer for screening and selection of studies, as opposed to two or more reviewers. 10 , 11 These methodological modifications are made to accommodate the needs of and resource constraints of the reviewers and stakeholders (e.g., organizations, policymakers, health care professionals, and other knowledge users). While such modifications are considered time and resource efficient, they may introduce bias in the review process reducing their usefulness. 5

Substantial research has been conducted examining various approaches used in the standardized SR methodology and their impact on the validity of SR results. There are a number of published reviews examining the approaches or modifications corresponding to single 12 , 13 or multiple steps 14 involved in an SR. However, there is yet to be a comprehensive summary of the SR‐level evidence for all the seven fundamental steps in an SR. Such a holistic evidence synthesis will provide an empirical basis to confirm the validity of current accepted practices in the conduct of SRs. Furthermore, sometimes there is a balance that needs to be achieved between the resource availability and the need to synthesize the evidence in the best way possible, given the constraints. This evidence base will also inform the choice of modifications to be made to the SR methods, as well as the potential impact of these modifications on the SR results. An overview is considered the choice of approach for summarizing existing evidence on a broad topic, directing the reader to evidence, or highlighting the gaps in evidence, where the evidence is derived exclusively from SRs. 15 Therefore, for this review, an overview approach was used to (a) identify and collate evidence from existing published SR articles evaluating various methodological approaches employed in each of the seven fundamental steps of an SR and (b) highlight both the gaps in the current research and the potential areas for future research on the methods employed in SRs.

An a priori protocol was developed for this overview but was not registered with the International Prospective Register of Systematic Reviews (PROSPERO), as the review was primarily methodological in nature and did not meet PROSPERO eligibility criteria for registration. The protocol is available from the corresponding author upon reasonable request. This overview was conducted based on the guidelines for the conduct of overviews as outlined in The Cochrane Handbook. 15 Reporting followed the Preferred Reporting Items for Systematic reviews and Meta‐analyses (PRISMA) statement. 3

2.1. Eligibility criteria

Only published SRs, with or without associated MA, were included in this overview. We adopted the defining characteristics of SRs from The Cochrane Handbook. 5 According to The Cochrane Handbook, a review was considered systematic if it satisfied the following criteria: (a) clearly states the objectives and eligibility criteria for study inclusion; (b) provides reproducible methodology; (c) includes a systematic search to identify all eligible studies; (d) reports assessment of validity of findings of included studies (e.g., RoB assessment of the included studies); (e) systematically presents all the characteristics or findings of the included studies. 5 Reviews that did not meet all of the above criteria were not considered a SR for this study and were excluded. MA‐only articles were included if it was mentioned that the MA was based on an SR.

SRs and/or MA of primary studies evaluating methodological approaches used in defining review scope and study eligibility, literature search, study selection, data extraction, RoB assessment, data synthesis, and CoE assessment and reporting were included. The methodological approaches examined in these SRs and/or MA can also be related to the substeps or elements of these steps; for example, applying limits on date or type of publication are the elements of literature search. Included SRs examined or compared various aspects of a method or methods, and the associated factors, including but not limited to: precision or effectiveness; accuracy or reliability; impact on the SR and/or MA results; reproducibility of an SR steps or bias occurred; time and/or resource efficiency. SRs assessing the methodological quality of SRs (e.g., adherence to reporting guidelines), evaluating techniques for building search strategies or the use of specific database filters (e.g., use of Boolean operators or search filters for randomized controlled trials), examining various tools used for RoB or CoE assessment (e.g., ROBINS vs. Cochrane RoB tool), or evaluating statistical techniques used in meta‐analyses were excluded. 14

2.2. Search

The search for published SRs was performed on the following scientific databases initially from inception to third week of November 2020 and updated in the last week of February 2022: MEDLINE (via Ovid), Embase (via Ovid), Web of Science Core Collection, Cochrane Database of Systematic Reviews, and American Psychological Association (APA) PsycINFO. Search was restricted to English language publications. Following the objectives of this study, study design filters within databases were used to restrict the search to SRs and MA, where available. The reference lists of included SRs were also searched for potentially relevant publications.

The search terms included keywords, truncations, and subject headings for the key concepts in the review question: SRs and/or MA, methods, and evaluation. Some of the terms were adopted from the search strategy used in a previous review by Robson et al., which reviewed primary studies on methodological approaches used in study selection, data extraction, and quality appraisal steps of SR process. 14 Individual search strategies were developed for respective databases by combining the search terms using appropriate proximity and Boolean operators, along with the related subject headings in order to identify SRs and/or MA. 16 , 17 A senior librarian was consulted in the design of the search terms and strategy. Appendix A presents the detailed search strategies for all five databases.

2.3. Study selection and data extraction

Title and abstract screening of references were performed in three steps. First, one reviewer (PV) screened all the titles and excluded obviously irrelevant citations, for example, articles on topics not related to SRs, non‐SR publications (such as randomized controlled trials, observational studies, scoping reviews, etc.). Next, from the remaining citations, a random sample of 200 titles and abstracts were screened against the predefined eligibility criteria by two reviewers (PV and MM), independently, in duplicate. Discrepancies were discussed and resolved by consensus. This step ensured that the responses of the two reviewers were calibrated for consistency in the application of the eligibility criteria in the screening process. Finally, all the remaining titles and abstracts were reviewed by a single “calibrated” reviewer (PV) to identify potential full‐text records. Full‐text screening was performed by at least two authors independently (PV screened all the records, and duplicate assessment was conducted by MM, HC, or MG), with discrepancies resolved via discussions or by consulting a third reviewer.

Data related to review characteristics, results, key findings, and conclusions were extracted by at least two reviewers independently (PV performed data extraction for all the reviews and duplicate extraction was performed by AP, HC, or MG).

2.4. Quality assessment of included reviews

The quality assessment of the included SRs was performed using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). The tool consists of a 16‐item checklist addressing critical and noncritical domains. 18 For the purpose of this study, the domain related to MA was reclassified from critical to noncritical, as SRs with and without MA were included. The other six critical domains were used according to the tool guidelines. 18 Two reviewers (PV and AP) independently responded to each of the 16 items in the checklist with either “yes,” “partial yes,” or “no.” Based on the interpretations of the critical and noncritical domains, the overall quality of the review was rated as high, moderate, low, or critically low. 18 Disagreements were resolved through discussion or by consulting a third reviewer.

2.5. Data synthesis

To provide an understandable summary of existing evidence syntheses, characteristics of the methods evaluated in the included SRs were examined and key findings were categorized and presented based on the corresponding step in the SR process. The categories of key elements within each step were discussed and agreed by the authors. Results of the included reviews were tabulated and summarized descriptively, along with a discussion on any overlap in the primary studies. 15 No quantitative analyses of the data were performed.

From 41,556 unique citations identified through literature search, 50 full‐text records were reviewed, and nine systematic reviews 14 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 were deemed eligible for inclusion. The flow of studies through the screening process is presented in Figure  1 . A list of excluded studies with reasons can be found in Appendix B .

Study selection flowchart

3.1. Characteristics of included reviews

Table  1 summarizes the characteristics of included SRs. The majority of the included reviews (six of nine) were published after 2010. 14 , 22 , 23 , 24 , 25 , 26 Four of the nine included SRs were Cochrane reviews. 20 , 21 , 22 , 23 The number of databases searched in the reviews ranged from 2 to 14, 2 reviews searched gray literature sources, 24 , 25 and 7 reviews included a supplementary search strategy to identify relevant literature. 14 , 19 , 20 , 21 , 22 , 23 , 26 Three of the included SRs (all Cochrane reviews) included an integrated MA. 20 , 21 , 23

Characteristics of included studies

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.

The included SRs evaluated 24 unique methodological approaches (26 in total) used across five steps in the SR process; 8 SRs evaluated 6 approaches, 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 while 1 review evaluated 18 approaches. 14 Exclusion of gray or unpublished literature 21 , 26 and blinding of reviewers for RoB assessment 14 , 23 were evaluated in two reviews each. Included SRs evaluated methods used in five different steps in the SR process, including methods used in defining the scope of review ( n  = 3), literature search ( n  = 3), study selection ( n  = 2), data extraction ( n  = 1), and RoB assessment ( n  = 2) (Table  2 ).

Summary of findings from review evaluating systematic review methods

There was some overlap in the primary studies evaluated in the included SRs on the same topics: Schmucker et al. 26 and Hopewell et al. 21 ( n  = 4), Hopewell et al. 20 and Crumley et al. 19 ( n  = 30), and Robson et al. 14 and Morissette et al. 23 ( n  = 4). There were no conflicting results between any of the identified SRs on the same topic.

3.2. Methodological quality of included reviews

Overall, the quality of the included reviews was assessed as moderate at best (Table  2 ). The most common critical weakness in the reviews was failure to provide justification for excluding individual studies (four reviews). Detailed quality assessment is provided in Appendix C .

3.3. Evidence on systematic review methods

3.3.1. methods for defining review scope and eligibility.

Two SRs investigated the effect of excluding data obtained from gray or unpublished sources on the pooled effect estimates of MA. 21 , 26 Hopewell et al. 21 reviewed five studies that compared the impact of gray literature on the results of a cohort of MA of RCTs in health care interventions. Gray literature was defined as information published in “print or electronic sources not controlled by commercial or academic publishers.” Findings showed an overall greater treatment effect for published trials than trials reported in gray literature. In a more recent review, Schmucker et al. 26 addressed similar objectives, by investigating gray and unpublished data in medicine. In addition to gray literature, defined similar to the previous review by Hopewell et al., the authors also evaluated unpublished data—defined as “supplemental unpublished data related to published trials, data obtained from the Food and Drug Administration  or other regulatory websites or postmarketing analyses hidden from the public.” The review found that in majority of the MA, excluding gray literature had little or no effect on the pooled effect estimates. The evidence was limited to conclude if the data from gray and unpublished literature had an impact on the conclusions of MA. 26

Morrison et al. 24 examined five studies measuring the effect of excluding non‐English language RCTs on the summary treatment effects of SR‐based MA in various fields of conventional medicine. Although none of the included studies reported major difference in the treatment effect estimates between English only and non‐English inclusive MA, the review found inconsistent evidence regarding the methodological and reporting quality of English and non‐English trials. 24 As such, there might be a risk of introducing “language bias” when excluding non‐English language RCTs. The authors also noted that the numbers of non‐English trials vary across medical specialties, as does the impact of these trials on MA results. Based on these findings, Morrison et al. 24 conclude that literature searches must include non‐English studies when resources and time are available to minimize the risk of introducing “language bias.”

3.3.2. Methods for searching studies

Crumley et al. 19 analyzed recall (also referred to as “sensitivity” by some researchers; defined as “percentage of relevant studies identified by the search”) and precision (defined as “percentage of studies identified by the search that were relevant”) when searching a single resource to identify randomized controlled trials and controlled clinical trials, as opposed to searching multiple resources. The studies included in their review frequently compared a MEDLINE only search with the search involving a combination of other resources. The review found low median recall estimates (median values between 24% and 92%) and very low median precisions (median values between 0% and 49%) for most of the electronic databases when searched singularly. 19 A between‐database comparison, based on the type of search strategy used, showed better recall and precision for complex and Cochrane Highly Sensitive search strategies (CHSSS). In conclusion, the authors emphasize that literature searches for trials in SRs must include multiple sources. 19

In an SR comparing handsearching and electronic database searching, Hopewell et al. 20 found that handsearching retrieved more relevant RCTs (retrieval rate of 92%−100%) than searching in a single electronic database (retrieval rates of 67% for PsycINFO/PsycLIT, 55% for MEDLINE, and 49% for Embase). The retrieval rates varied depending on the quality of handsearching, type of electronic search strategy used (e.g., simple, complex or CHSSS), and type of trial reports searched (e.g., full reports, conference abstracts, etc.). The authors concluded that handsearching was particularly important in identifying full trials published in nonindexed journals and in languages other than English, as well as those published as abstracts and letters. 20

The effectiveness of checking reference lists to retrieve additional relevant studies for an SR was investigated by Horsley et al. 22 The review reported that checking reference lists yielded 2.5%–40% more studies depending on the quality and comprehensiveness of the electronic search used. The authors conclude that there is some evidence, although from poor quality studies, to support use of checking reference lists to supplement database searching. 22

3.3.3. Methods for selecting studies

Three approaches relevant to reviewer characteristics, including number, experience, and blinding of reviewers involved in the screening process were highlighted in an SR by Robson et al. 14 Based on the retrieved evidence, the authors recommended that two independent, experienced, and unblinded reviewers be involved in study selection. 14 A modified approach has also been suggested by the review authors, where one reviewer screens and the other reviewer verifies the list of excluded studies, when the resources are limited. It should be noted however this suggestion is likely based on the authors’ opinion, as there was no evidence related to this from the studies included in the review.

Robson et al. 14 also reported two methods describing the use of technology for screening studies: use of Google Translate for translating languages (for example, German language articles to English) to facilitate screening was considered a viable method, while using two computer monitors for screening did not increase the screening efficiency in SR. Title‐first screening was found to be more efficient than simultaneous screening of titles and abstracts, although the gain in time with the former method was lesser than the latter. Therefore, considering that the search results are routinely exported as titles and abstracts, Robson et al. 14 recommend screening titles and abstracts simultaneously. However, the authors note that these conclusions were based on very limited number (in most instances one study per method) of low‐quality studies. 14

3.3.4. Methods for data extraction

Robson et al. 14 examined three approaches for data extraction relevant to reviewer characteristics, including number, experience, and blinding of reviewers (similar to the study selection step). Although based on limited evidence from a small number of studies, the authors recommended use of two experienced and unblinded reviewers for data extraction. The experience of the reviewers was suggested to be especially important when extracting continuous outcomes (or quantitative) data. However, when the resources are limited, data extraction by one reviewer and a verification of the outcomes data by a second reviewer was recommended.

As for the methods involving use of technology, Robson et al. 14 identified limited evidence on the use of two monitors to improve the data extraction efficiency and computer‐assisted programs for graphical data extraction. However, use of Google Translate for data extraction in non‐English articles was not considered to be viable. 14 In the same review, Robson et al. 14 identified evidence supporting contacting authors for obtaining additional relevant data.

3.3.5. Methods for RoB assessment

Two SRs examined the impact of blinding of reviewers for RoB assessments. 14 , 23 Morissette et al. 23 investigated the mean differences between the blinded and unblinded RoB assessment scores and found inconsistent differences among the included studies providing no definitive conclusions. Similar conclusions were drawn in a more recent review by Robson et al., 14 which included four studies on reviewer blinding for RoB assessment that completely overlapped with Morissette et al. 23

Use of experienced reviewers and provision of additional guidance for RoB assessment were examined by Robson et al. 14 The review concluded that providing intensive training and guidance on assessing studies reporting insufficient data to the reviewers improves RoB assessments. 14 Obtaining additional data related to quality assessment by contacting study authors was also found to help the RoB assessments, although based on limited evidence. When assessing the qualitative or mixed method reviews, Robson et al. 14 recommends the use of a structured RoB tool as opposed to an unstructured tool. No SRs were identified on data synthesis and CoE assessment and reporting steps.

4. DISCUSSION

4.1. summary of findings.

Nine SRs examining 24 unique methods used across five steps in the SR process were identified in this overview. The collective evidence supports some current traditional and modified SR practices, while challenging other approaches. However, the quality of the included reviews was assessed to be moderate at best and in the majority of the included SRs, evidence related to the evaluated methods was obtained from very limited numbers of primary studies. As such, the interpretations from these SRs should be made cautiously.

The evidence gathered from the included SRs corroborate a few current SR approaches. 5 For example, it is important to search multiple resources for identifying relevant trials (RCTs and/or CCTs). The resources must include a combination of electronic database searching, handsearching, and reference lists of retrieved articles. 5 However, no SRs have been identified that evaluated the impact of the number of electronic databases searched. A recent study by Halladay et al. 27 found that articles on therapeutic intervention, retrieved by searching databases other than PubMed (including Embase), contributed only a small amount of information to the MA and also had a minimal impact on the MA results. The authors concluded that when the resources are limited and when large number of studies are expected to be retrieved for the SR or MA, PubMed‐only search can yield reliable results. 27

Findings from the included SRs also reiterate some methodological modifications currently employed to “expedite” the SR process. 10 , 11 For example, excluding non‐English language trials and gray/unpublished trials from MA have been shown to have minimal or no impact on the results of MA. 24 , 26 However, the efficiency of these SR methods, in terms of time and the resources used, have not been evaluated in the included SRs. 24 , 26 Of the SRs included, only two have focused on the aspect of efficiency 14 , 25 ; O'Mara‐Eves et al. 25 report some evidence to support the use of text‐mining approaches for title and abstract screening in order to increase the rate of screening. Moreover, only one included SR 14 considered primary studies that evaluated reliability (inter‐ or intra‐reviewer consistency) and accuracy (validity when compared against a “gold standard” method) of the SR methods. This can be attributed to the limited number of primary studies that evaluated these outcomes when evaluating the SR methods. 14 Lack of outcome measures related to reliability, accuracy, and efficiency precludes making definitive recommendations on the use of these methods/modifications. Future research studies must focus on these outcomes.

Some evaluated methods may be relevant to multiple steps; for example, exclusions based on publication status (gray/unpublished literature) and language of publication (non‐English language studies) can be outlined in the a priori eligibility criteria or can be incorporated as search limits in the search strategy. SRs included in this overview focused on the effect of study exclusions on pooled treatment effect estimates or MA conclusions. Excluding studies from the search results, after conducting a comprehensive search, based on different eligibility criteria may yield different results when compared to the results obtained when limiting the search itself. 28 Further studies are required to examine this aspect.

Although we acknowledge the lack of standardized quality assessment tools for methodological study designs, we adhered to the Cochrane criteria for identifying SRs in this overview. This was done to ensure consistency in the quality of the included evidence. As a result, we excluded three reviews that did not provide any form of discussion on the quality of the included studies. The methods investigated in these reviews concern supplementary search, 29 data extraction, 12 and screening. 13 However, methods reported in two of these three reviews, by Mathes et al. 12 and Waffenschmidt et al., 13 have also been examined in the SR by Robson et al., 14 which was included in this overview; in most instances (with the exception of one study included in Mathes et al. 12 and Waffenschmidt et al. 13 each), the studies examined in these excluded reviews overlapped with those in the SR by Robson et al. 14

One of the key gaps in the knowledge observed in this overview was the dearth of SRs on the methods used in the data synthesis component of SR. Narrative and quantitative syntheses are the two most commonly used approaches for synthesizing data in evidence synthesis. 5 There are some published studies on the proposed indications and implications of these two approaches. 30 , 31 These studies found that both data synthesis methods produced comparable results and have their own advantages, suggesting that the choice of the method must be based on the purpose of the review. 31 With increasing number of “expedited” SR approaches (so called “rapid reviews”) avoiding MA, 10 , 11 further research studies are warranted in this area to determine the impact of the type of data synthesis on the results of the SR.

4.2. Implications for future research

The findings of this overview highlight several areas of paucity in primary research and evidence synthesis on SR methods. First, no SRs were identified on methods used in two important components of the SR process, including data synthesis and CoE and reporting. As for the included SRs, a limited number of evaluation studies have been identified for several methods. This indicates that further research is required to corroborate many of the methods recommended in current SR guidelines. 4 , 5 , 6 , 7 Second, some SRs evaluated the impact of methods on the results of quantitative synthesis and MA conclusions. Future research studies must also focus on the interpretations of SR results. 28 , 32 Finally, most of the included SRs were conducted on specific topics related to the field of health care, limiting the generalizability of the findings to other areas. It is important that future research studies evaluating evidence syntheses broaden the objectives and include studies on different topics within the field of health care.

4.3. Strengths and limitations

To our knowledge, this is the first overview summarizing current evidence from SRs and MA on different methodological approaches used in several fundamental steps in SR conduct. The overview methodology followed well established guidelines and strict criteria defined for the inclusion of SRs.

There are several limitations related to the nature of the included reviews. Evidence for most of the methods investigated in the included reviews was derived from a limited number of primary studies. Also, the majority of the included SRs may be considered outdated as they were published (or last updated) more than 5 years ago 33 ; only three of the nine SRs have been published in the last 5 years. 14 , 25 , 26 Therefore, important and recent evidence related to these topics may not have been included. Substantial numbers of included SRs were conducted in the field of health, which may limit the generalizability of the findings. Some method evaluations in the included SRs focused on quantitative analyses components and MA conclusions only. As such, the applicability of these findings to SR more broadly is still unclear. 28 Considering the methodological nature of our overview, limiting the inclusion of SRs according to the Cochrane criteria might have resulted in missing some relevant evidence from those reviews without a quality assessment component. 12 , 13 , 29 Although the included SRs performed some form of quality appraisal of the included studies, most of them did not use a standardized RoB tool, which may impact the confidence in their conclusions. Due to the type of outcome measures used for the method evaluations in the primary studies and the included SRs, some of the identified methods have not been validated against a reference standard.

Some limitations in the overview process must be noted. While our literature search was exhaustive covering five bibliographic databases and supplementary search of reference lists, no gray sources or other evidence resources were searched. Also, the search was primarily conducted in health databases, which might have resulted in missing SRs published in other fields. Moreover, only English language SRs were included for feasibility. As the literature search retrieved large number of citations (i.e., 41,556), the title and abstract screening was performed by a single reviewer, calibrated for consistency in the screening process by another reviewer, owing to time and resource limitations. These might have potentially resulted in some errors when retrieving and selecting relevant SRs. The SR methods were grouped based on key elements of each recommended SR step, as agreed by the authors. This categorization pertains to the identified set of methods and should be considered subjective.

5. CONCLUSIONS

This overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process. Limited evidence was also identified on some methodological modifications currently used to expedite the SR process. Overall, findings highlight the dearth of SRs on SR methodologies, warranting further work to confirm several current recommendations on conventional and expedited SR processes.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

APPENDIX A: Detailed search strategies

ACKNOWLEDGMENTS

The first author is supported by a La Trobe University Full Fee Research Scholarship and a Graduate Research Scholarship.

Open Access Funding provided by La Trobe University.

Veginadu P, Calache H, Gussy M, Pandian A, Masood M. An overview of methodological approaches in systematic reviews . J Evid Based Med . 2022; 15 :39–54. 10.1111/jebm.12468 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]