literature review techniques

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

How To Write A Literature Review - A Complete Guide

Table of Contents

A literature review is much more than just another section in your research paper. It forms the very foundation of your research. It is a formal piece of writing where you analyze the existing theoretical framework, principles, and assumptions and use that as a base to shape your approach to the research question.

Curating and drafting a solid literature review section not only lends more credibility to your research paper but also makes your research tighter and better focused. But, writing literature reviews is a difficult task. It requires extensive reading, plus you have to consider market trends and technological and political changes, which tend to change in the blink of an eye.

Now streamline your literature review process with the help of SciSpace Copilot. With this AI research assistant, you can efficiently synthesize and analyze a vast amount of information, identify key themes and trends, and uncover gaps in the existing research. Get real-time explanations, summaries, and answers to your questions for the paper you're reviewing, making navigating and understanding the complex literature landscape easier.

In this comprehensive guide, we will explore everything from the definition of a literature review, its appropriate length, various types of literature reviews, and how to write one.

What is a literature review?

A literature review is a collation of survey, research, critical evaluation, and assessment of the existing literature in a preferred domain.

Eminent researcher and academic Arlene Fink, in her book Conducting Research Literature Reviews , defines it as the following:

“A literature review surveys books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated.

Literature reviews are designed to provide an overview of sources you have explored while researching a particular topic, and to demonstrate to your readers how your research fits within a larger field of study.”

Simply put, a literature review can be defined as a critical discussion of relevant pre-existing research around your research question and carving out a definitive place for your study in the existing body of knowledge. Literature reviews can be presented in multiple ways: a section of an article, the whole research paper itself, or a chapter of your thesis.

A literature review does function as a summary of sources, but it also allows you to analyze further, interpret, and examine the stated theories, methods, viewpoints, and, of course, the gaps in the existing content.

As an author, you can discuss and interpret the research question and its various aspects and debate your adopted methods to support the claim.

What is the purpose of a literature review?

A literature review is meant to help your readers understand the relevance of your research question and where it fits within the existing body of knowledge. As a researcher, you should use it to set the context, build your argument, and establish the need for your study.

What is the importance of a literature review?

The literature review is a critical part of research papers because it helps you:

• Gain an in-depth understanding of your research question and the surrounding area
• Convey that you have a thorough understanding of your research area and are up-to-date with the latest changes and advancements
• Establish how your research is connected or builds on the existing body of knowledge and how it could contribute to further research
• Elaborate on the validity and suitability of your theoretical framework and research methodology
• Identify and highlight gaps and shortcomings in the existing body of knowledge and how things need to change
• Convey to readers how your study is different or how it contributes to the research area

How long should a literature review be?

Ideally, the literature review should take up 15%-40% of the total length of your manuscript. So, if you have a 10,000-word research paper, the minimum word count could be 1500.

Your literature review format depends heavily on the kind of manuscript you are writing — an entire chapter in case of doctoral theses, a part of the introductory section in a research article, to a full-fledged review article that examines the previously published research on a topic.

Another determining factor is the type of research you are doing. The literature review section tends to be longer for secondary research projects than primary research projects.

What are the different types of literature reviews?

All literature reviews are not the same. There are a variety of possible approaches that you can take. It all depends on the type of research you are pursuing.

Here are the different types of literature reviews:

Argumentative review

It is called an argumentative review when you carefully present literature that only supports or counters a specific argument or premise to establish a viewpoint.

Integrative review

It is a type of literature review focused on building a comprehensive understanding of a topic by combining available theoretical frameworks and empirical evidence.

Methodological review

This approach delves into the ''how'' and the ''what" of the research question —  you cannot look at the outcome in isolation; you should also review the methodology used.

Systematic review

This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research and collect, report, and analyze data from the studies included in the review.

Meta-analysis review

Meta-analysis uses statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects than those derived from the individual studies included within a review.

Historical review

Historical literature reviews focus on examining research throughout a period, often starting with the first time an issue, concept, theory, or phenomenon emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and identify future research's likely directions.

Theoretical Review

This form aims to examine the corpus of theory accumulated regarding an issue, concept, theory, and phenomenon. The theoretical literature review helps to establish what theories exist, the relationships between them, the degree the existing approaches have been investigated, and to develop new hypotheses to be tested.

Scoping Review

The Scoping Review is often used at the beginning of an article, dissertation, or research proposal. It is conducted before the research to highlight gaps in the existing body of knowledge and explains why the project should be greenlit.

State-of-the-Art Review

The State-of-the-Art review is conducted periodically, focusing on the most recent research. It describes what is currently known, understood, or agreed upon regarding the research topic and highlights where there are still disagreements.

Can you use the first person in a literature review?

When writing literature reviews, you should avoid the usage of first-person pronouns. It means that instead of "I argue that" or "we argue that," the appropriate expression would be "this research paper argues that."

Do you need an abstract for a literature review?

Ideally, yes. It is always good to have a condensed summary that is self-contained and independent of the rest of your review. As for how to draft one, you can follow the same fundamental idea when preparing an abstract for a literature review. It should also include:

• The research topic and your motivation behind selecting it
• A one-sentence thesis statement
• An explanation of the kinds of literature featured in the review
• Summary of what you've learned
• Conclusions you drew from the literature you reviewed
• Potential implications and future scope for research

Here's an example of the abstract of a literature review

Is a literature review written in the past tense?

Yes, the literature review should ideally be written in the past tense. You should not use the present or future tense when writing one. The exceptions are when you have statements describing events that happened earlier than the literature you are reviewing or events that are currently occurring; then, you can use the past perfect or present perfect tenses.

How many sources for a literature review?

There are multiple approaches to deciding how many sources to include in a literature review section. The first approach would be to look level you are at as a researcher. For instance, a doctoral thesis might need 60+ sources. In contrast, you might only need to refer to 5-15 sources at the undergraduate level.

The second approach is based on the kind of literature review you are doing — whether it is merely a chapter of your paper or if it is a self-contained paper in itself. When it is just a chapter, sources should equal the total number of pages in your article's body. In the second scenario, you need at least three times as many sources as there are pages in your work.

Quick tips on how to write a literature review

To know how to write a literature review, you must clearly understand its impact and role in establishing your work as substantive research material.

You need to follow the below-mentioned steps, to write a literature review:

• Outline the purpose behind the literature review
• Search relevant literature
• Examine and assess the relevant resources
• Discover connections by drawing deep insights from the resources
• Structure planning to write a good literature review

1. Outline and identify the purpose of  a literature review

As a first step on how to write a literature review, you must know what the research question or topic is and what shape you want your literature review to take. Ensure you understand the research topic inside out, or else seek clarifications. You must be able to the answer below questions before you start:

• How many sources do I need to include?
• What kind of sources should I analyze?
• How much should I critically evaluate each source?
• Should I summarize, synthesize or offer a critique of the sources?
• Do I need to include any background information or definitions?

Additionally, you should know that the narrower your research topic is, the swifter it will be for you to restrict the number of sources to be analyzed.

2. Search relevant literature

Dig deeper into search engines to discover what has already been published around your chosen topic. Make sure you thoroughly go through appropriate reference sources like books, reports, journal articles, government docs, and web-based resources.

You must prepare a list of keywords and their different variations. You can start your search from any library’s catalog, provided you are an active member of that institution. The exact keywords can be extended to widen your research over other databases and academic search engines like:

• Google Scholar
• Microsoft Academic
• Science.gov

Besides, it is not advisable to go through every resource word by word. Alternatively, what you can do is you can start by reading the abstract and then decide whether that source is relevant to your research or not.

Additionally, you must spend surplus time assessing the quality and relevance of resources. It would help if you tried preparing a list of citations to ensure that there lies no repetition of authors, publications, or articles in the literature review.

3. Examine and assess the sources

It is nearly impossible for you to go through every detail in the research article. So rather than trying to fetch every detail, you have to analyze and decide which research sources resemble closest and appear relevant to your chosen domain.

While analyzing the sources, you should look to find out answers to questions like:

• What question or problem has the author been describing and debating?
• What is the definition of critical aspects?
• How well the theories, approach, and methodology have been explained?
• Whether the research theory used some conventional or new innovative approach?
• How relevant are the key findings of the work?
• In what ways does it relate to other sources on the same topic?
• What challenges does this research paper pose to the existing theory
• What are the possible contributions or benefits it adds to the subject domain?

Be always mindful that you refer only to credible and authentic resources. It would be best if you always take references from different publications to validate your theory.

Always keep track of important information or data you can present in your literature review right from the beginning. It will help steer your path from any threats of plagiarism and also make it easier to curate an annotated bibliography or reference section.

4. Discover connections

At this stage, you must start deciding on the argument and structure of your literature review. To accomplish this, you must discover and identify the relations and connections between various resources while drafting your abstract.

A few aspects that you should be aware of while writing a literature review include:

• Rise to prominence: Theories and methods that have gained reputation and supporters over time.
• Constant scrutiny: Concepts or theories that repeatedly went under examination.
• Contradictions and conflicts: Theories, both the supporting and the contradictory ones, for the research topic.
• Knowledge gaps: What exactly does it fail to address, and how to bridge them with further research?
• Influential resources: Significant research projects available that have been upheld as milestones or perhaps, something that can modify the current trends

Once you join the dots between various past research works, it will be easier for you to draw a conclusion and identify your contribution to the existing knowledge base.

5. Structure planning to write a good literature review

There exist different ways towards planning and executing the structure of a literature review. The format of a literature review varies and depends upon the length of the research.

Like any other research paper, the literature review format must contain three sections: introduction, body, and conclusion. The goals and objectives of the research question determine what goes inside these three sections.

Nevertheless, a good literature review can be structured according to the chronological, thematic, methodological, or theoretical framework approach.

Literature review samples

1. Standalone

2. As a section of a research paper

How SciSpace Discover makes literature review a breeze?

SciSpace Discover is a one-stop solution to do an effective literature search and get barrier-free access to scientific knowledge. It is an excellent repository where you can find millions of only peer-reviewed articles and full-text PDF files. Here’s more on how you can use it:

Find the right information

Find what you want quickly and easily with comprehensive search filters that let you narrow down papers according to PDF availability, year of publishing, document type, and affiliated institution. Moreover, you can sort the results based on the publishing date, citation count, and relevance.

Assess credibility of papers quickly

When doing the literature review, it is critical to establish the quality of your sources. They form the foundation of your research. SciSpace Discover helps you assess the quality of a source by providing an overview of its references, citations, and performance metrics.

Get the complete picture in no time

SciSpace Discover’s personalized suggestion engine helps you stay on course and get the complete picture of the topic from one place. Every time you visit an article page, it provides you links to related papers. Besides that, it helps you understand what’s trending, who are the top authors, and who are the leading publishers on a topic.

Make referring sources super easy

To ensure you don't lose track of your sources, you must start noting down your references when doing the literature review. SciSpace Discover makes this step effortless. Click the 'cite' button on an article page, and you will receive preloaded citation text in multiple styles — all you've to do is copy-paste it into your manuscript.

Final tips on how to write a literature review

A massive chunk of time and effort is required to write a good literature review. But, if you go about it systematically, you'll be able to save a ton of time and build a solid foundation for your research.

We hope this guide has helped you answer several key questions you have about writing literature reviews.

Would you like to explore SciSpace Discover and kick off your literature search right away? You can get started here .

Frequently Asked Questions (FAQs)

1. how to start a literature review.

• What questions do you want to answer?

• What sources do you need to answer these questions?

• What information do these sources contain?

• How can you use this information to answer your questions?

2. What to include in a literature review?

• A brief background of the problem or issue

• What has previously been done to address the problem or issue

• A description of what you will do in your project

• How this study will contribute to research on the subject

3. Why literature review is important?

The literature review is an important part of any research project because it allows the writer to look at previous studies on a topic and determine existing gaps in the literature, as well as what has already been done. It will also help them to choose the most appropriate method for their own study.

4. How to cite a literature review in APA format?

To cite a literature review in APA style, you need to provide the author's name, the title of the article, and the year of publication. For example: Patel, A. B., & Stokes, G. S. (2012). The relationship between personality and intelligence: A meta-analysis of longitudinal research. Personality and Individual Differences, 53(1), 16-21

5. What are the components of a literature review?

• A brief introduction to the topic, including its background and context. The introduction should also include a rationale for why the study is being conducted and what it will accomplish.

• A description of the methodologies used in the study. This can include information about data collection methods, sample size, and statistical analyses.

• A presentation of the findings in an organized format that helps readers follow along with the author's conclusions.

6. What are common errors in writing literature review?

• Not spending enough time to critically evaluate the relevance of resources, observations and conclusions.

• Totally relying on secondary data while ignoring primary data.

• Letting your personal bias seep into your interpretation of existing literature.

• No detailed explanation of the procedure to discover and identify an appropriate literature review.

7. What are the 5 C's of writing literature review?

• Cite - the sources you utilized and referenced in your research.

• Compare - existing arguments, hypotheses, methodologies, and conclusions found in the knowledge base.

• Contrast - the arguments, topics, methodologies, approaches, and disputes that may be found in the literature.

• Critique - the literature and describe the ideas and opinions you find more convincing and why.

• Connect - the various studies you reviewed in your research.

8. How many sources should a literature review have?

When it is just a chapter, sources should equal the total number of pages in your article's body. if it is a self-contained paper in itself, you need at least three times as many sources as there are pages in your work.

9. Can literature review have diagrams?

• To represent an abstract idea or concept

• To explain the steps of a process or procedure

• To help readers understand the relationships between different concepts

10. How old should sources be in a literature review?

Sources for a literature review should be as current as possible or not older than ten years. The only exception to this rule is if you are reviewing a historical topic and need to use older sources.

11. What are the types of literature review?

• Argumentative review

• Integrative review

• Methodological review

• Systematic review

• Meta-analysis review

• Historical review

• Theoretical review

• Scoping review

• State-of-the-Art review

12. Is a literature review mandatory?

Yes. Literature review is a mandatory part of any research project. It is a critical step in the process that allows you to establish the scope of your research, and provide a background for the rest of your work.

But before you go,

• Six Online Tools for Easy Literature Review
• Evaluating literature review: systematic vs. scoping reviews
• Systematic Approaches to a Successful Literature Review
• Writing Integrative Literature Reviews: Guidelines and Examples

You might also like

Consensus GPT vs. SciSpace GPT: Choose the Best GPT for Research

Literature Review and Theoretical Framework: Understanding the Differences

Types of Essays in Academic Writing - Quick Guide (2024)

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 29, 2024 1:49 PM
• URL: https://libguides.usc.edu/writingguide

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

Want to make a difference? Try working at an environmental non-profit organization

Career Feature 26 APR 24

Scientists urged to collect royalties from the ‘magic money tree’

Career Feature 25 APR 24

NIH pay rise for postdocs and PhD students could have US ripple effect

News 25 APR 24

Algorithm ranks peer reviewers by reputation — but critics warn of bias

Nature Index 25 APR 24

Researchers want a ‘nutrition label’ for academic-paper facts

Nature Index 17 APR 24

How young people benefit from Swiss apprenticeships

Spotlight 17 APR 24

How reliable is this research? Tool flags papers discussed on PubPeer

News 29 APR 24

W2 Professorship with tenure track to W3 in Animal Husbandry (f/m/d)

The Faculty of Agricultural Sciences at the University of Göttingen invites applications for a temporary professorship with civil servant status (g...

Göttingen (Stadt), Niedersachsen (DE)

Georg-August-Universität Göttingen

W1 professorship for „Tissue Aspects of Immunity and Inflammation“

Kiel University (CAU) and the University of Lübeck (UzL) are striving to increase the proportion of qualified female scientists in research and tea...

University of Luebeck

W1 professorship for "Bioinformatics and artificial intelligence that preserve privacy"

Kiel, Schleswig-Holstein (DE)

Universität Kiel - Medizinische Fakultät

W1 professorship for "Central Metabolic Inflammation“

W1 professorship for "Congenital and adaptive lymphocyte regulation"

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• University of Texas Libraries

Literature Reviews

Steps in the literature review process.

• What is a literature review?
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• You may need to some exploratory searching of the literature to get a sense of scope, to determine whether you need to narrow or broaden your focus
• Identify databases that provide the most relevant sources, and identify relevant terms (controlled vocabularies) to add to your search strategy
• Finalize your research question
• Think about relevant dates, geographies (and languages), methods, and conflicting points of view
• Conduct searches in the published literature via the identified databases
• Check to see if this topic has been covered in other discipline's databases
• Examine the citations of on-point articles for keywords, authors, and previous research (via references) and cited reference searching.
• Save your search results in a citation management tool (such as Zotero, Mendeley or EndNote)
• De-duplicate your search results
• Make sure that you've found the seminal pieces -- they have been cited many times, and their work is considered foundational 
• Check with your professor or a librarian to make sure your search has been comprehensive
• Evaluate the strengths and weaknesses of individual sources and evaluate for bias, methodologies, and thoroughness
• Group your results in to an organizational structure that will support why your research needs to be done, or that provides the answer to your research question  
• Develop your conclusions
• Are there gaps in the literature?
• Where has significant research taken place, and who has done it?
• Is there consensus or debate on this topic?
• Which methodological approaches work best?
• For example: Background, Current Practices, Critics and Proponents, Where/How this study will fit in 
• Organize your citations and focus on your research question and pertinent studies
• Compile your bibliography

Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.

Videos Tutorials about Literature Reviews

This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.

Recommended Reading

• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• Harvard Library
• Research Guides
• Harvard Graduate School of Design - Frances Loeb Library

Write and Cite

• Literature Review
• Academic Integrity
• Citing Sources
• Fair Use, Permissions, and Copyright
• Writing Resources
• Grants and Fellowships
• Last Updated: Apr 26, 2024 10:28 AM
• URL: https://guides.library.harvard.edu/gsd/write

Harvard University Digital Accessibility Policy

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

Organizing and Creating Information

• Citation and Attribution

What Is a Literature Review?

Review the literature, write the literature review, further reading, learning objectives, attribution.

This guide is designed to:

• Identify the sections and purpose of a literature review in academic writing
• Review practical strategies and organizational methods for preparing a literature review

A literature review is a summary and synthesis of scholarly research on a specific topic. It should answer questions such as:

• What research has been done on the topic?
• Who are the key researchers and experts in the field?
• What are the common theories and methodologies?
• Are there challenges, controversies, and contradictions?
• Are there gaps in the research that your approach addresses?

The process of reviewing existing research allows you to fine-tune your research question and contextualize your own work. Preparing a literature review is a cyclical process. You may find that the research question you begin with evolves as you learn more about the topic.

Once you have defined your research question , focus on learning what other scholars have written on the topic.

In order to  do a thorough search of the literature  on the topic, define the basic criteria:

• Databases and journals: Look at the  subject guide  related to your topic for recommended databases. Review the  tutorial on finding articles  for tips. 
• Books: Search BruKnow, the Library's catalog. Steps to searching ebooks are covered in the  Finding Ebooks tutorial .
• What time period should it cover? Is currency important?
• Do I know of primary and secondary sources that I can use as a way to find other information?
• What should I be aware of when looking at popular, trade, and scholarly resources ? 

One strategy is to review bibliographies for sources that relate to your interest. For more on this technique, look at the tutorial on finding articles when you have a citation .

Tip: Use a Synthesis Matrix

As you read sources, themes will emerge that will help you to organize the review. You can use a simple Synthesis Matrix to track your notes as you read. From this work, a concept map emerges that provides an overview of the literature and ways in which it connects. Working with Zotero to capture the citations, you build the structure for writing your literature review.

How do I know when I am done?

A key indicator for knowing when you are done is running into the same articles and materials. With no new information being uncovered, you are likely exhausting your current search and should modify search terms or search different catalogs or databases. It is also possible that you have reached a point when you can start writing the literature review.

Tip: Manage Your Citations

These citation management tools also create citations, footnotes, and bibliographies with just a few clicks:

Zotero Tutorial

Endnote Tutorial

Your literature review should be focused on the topic defined in your research question. It should be written in a logical, structured way and maintain an objective perspective and use a formal voice.

Review the Summary Table you created for themes and connecting ideas. Use the following guidelines to prepare an outline of the main points you want to make. 

• Synthesize previous research on the topic.
• Aim to include both summary and synthesis.
• Include literature that supports your research question as well as that which offers a different perspective.
• Avoid relying on one author or publication too heavily.
• Select an organizational structure, such as chronological, methodological, and thematic.

The three elements of a literature review are introduction, body, and conclusion.

Introduction

• Define the topic of the literature review, including any terminology.
• Introduce the central theme and organization of the literature review.
• Summarize the state of research on the topic.
• Frame the literature review with your research question.
• Focus on ways to have the body of literature tell its own story. Do not add your own interpretations at this point.
• Look for patterns and find ways to tie the pieces together.
• Summarize instead of quote.
• Weave the points together rather than list summaries of each source.
• Include the most important sources, not everything you have read.
• Summarize the review of the literature.
• Identify areas of further research on the topic.
• Connect the review with your research.
• DeCarlo, M. (2018). 4.1 What is a literature review? In Scientific Inquiry in Social Work. Open Social Work Education. https://scientificinquiryinsocialwork.pressbooks.com/chapter/4-1-what-is-a-literature-review/
• Literature Reviews (n.d.) https://writingcenter.unc.edu/tips-and-tools/literature-reviews/ Accessed Nov. 10, 2021

This guide was designed to: 

• Identify the sections and purpose of a literature review in academic writing 
• Review practical strategies and organizational methods for preparing a literature review​

Content on this page adapted from: 

Frederiksen, L. and Phelps, S. (2017).   Literature Reviews for Education and Nursing Graduate Students.  Licensed CC BY 4.0

• << Previous: EndNote
• Last Updated: Jan 9, 2024 3:05 PM
• URL: https://libguides.brown.edu/organize

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: Jan 17, 2024 10:05 AM
• URL: https://libguides.northwestern.edu/literaturereviews

How To Write An A-Grade Literature Review

3 straightforward steps (with examples) + free template.

By: Derek Jansen (MBA) | Expert Reviewed By: Dr. Eunice Rautenbach | October 2019

Quality research is about building onto the existing work of others , “standing on the shoulders of giants”, as Newton put it. The literature review chapter of your dissertation, thesis or research project is where you synthesise this prior work and lay the theoretical foundation for your own research.

Long story short, this chapter is a pretty big deal, which is why you want to make sure you get it right . In this post, I’ll show you exactly how to write a literature review in three straightforward steps, so you can conquer this vital chapter (the smart way).

Overview: The Literature Review Process

• Understanding the “ why “
• Finding the relevant literature
• Cataloguing and synthesising the information
• Outlining & writing up your literature review
• Example of a literature review

But first, the “why”…

Before we unpack how to write the literature review chapter, we’ve got to look at the why . To put it bluntly, if you don’t understand the function and purpose of the literature review process, there’s no way you can pull it off well. So, what exactly is the purpose of the literature review?

Well, there are (at least) four core functions:

• For you to gain an understanding (and demonstrate this understanding) of where the research is at currently, what the key arguments and disagreements are.
• For you to identify the gap(s) in the literature and then use this as justification for your own research topic.
• To help you build a conceptual framework for empirical testing (if applicable to your research topic).
• To inform your methodological choices and help you source tried and tested questionnaires (for interviews ) and measurement instruments (for surveys ).

Most students understand the first point but don’t give any thought to the rest. To get the most from the literature review process, you must keep all four points front of mind as you review the literature (more on this shortly), or you’ll land up with a wonky foundation.

Okay – with the why out the way, let’s move on to the how . As mentioned above, writing your literature review is a process, which I’ll break down into three steps:

• Finding the most suitable literature
• Understanding , distilling and organising the literature
• Planning and writing up your literature review chapter

Importantly, you must complete steps one and two before you start writing up your chapter. I know it’s very tempting, but don’t try to kill two birds with one stone and write as you read. You’ll invariably end up wasting huge amounts of time re-writing and re-shaping, or you’ll just land up with a disjointed, hard-to-digest mess . Instead, you need to read first and distil the information, then plan and execute the writing.

Step 1: Find the relevant literature

Naturally, the first step in the literature review journey is to hunt down the existing research that’s relevant to your topic. While you probably already have a decent base of this from your research proposal , you need to expand on this substantially in the dissertation or thesis itself.

Essentially, you need to be looking for any existing literature that potentially helps you answer your research question (or develop it, if that’s not yet pinned down). There are numerous ways to find relevant literature, but I’ll cover my top four tactics here. I’d suggest combining all four methods to ensure that nothing slips past you:

Method 1 – Google Scholar Scrubbing

Google’s academic search engine, Google Scholar , is a great starting point as it provides a good high-level view of the relevant journal articles for whatever keyword you throw at it. Most valuably, it tells you how many times each article has been cited, which gives you an idea of how credible (or at least, popular) it is. Some articles will be free to access, while others will require an account, which brings us to the next method.

Method 2 – University Database Scrounging

Generally, universities provide students with access to an online library, which provides access to many (but not all) of the major journals.

So, if you find an article using Google Scholar that requires paid access (which is quite likely), search for that article in your university’s database – if it’s listed there, you’ll have access. Note that, generally, the search engine capabilities of these databases are poor, so make sure you search for the exact article name, or you might not find it.

Method 3 – Journal Article Snowballing

At the end of every academic journal article, you’ll find a list of references. As with any academic writing, these references are the building blocks of the article, so if the article is relevant to your topic, there’s a good chance a portion of the referenced works will be too. Do a quick scan of the titles and see what seems relevant, then search for the relevant ones in your university’s database.

Method 4 – Dissertation Scavenging

Similar to Method 3 above, you can leverage other students’ dissertations. All you have to do is skim through literature review chapters of existing dissertations related to your topic and you’ll find a gold mine of potential literature. Usually, your university will provide you with access to previous students’ dissertations, but you can also find a much larger selection in the following databases:

• Open Access Theses & Dissertations
• Stanford SearchWorks

Keep in mind that dissertations and theses are not as academically sound as published, peer-reviewed journal articles (because they’re written by students, not professionals), so be sure to check the credibility of any sources you find using this method. You can do this by assessing the citation count of any given article in Google Scholar. If you need help with assessing the credibility of any article, or with finding relevant research in general, you can chat with one of our Research Specialists .

Alright – with a good base of literature firmly under your belt, it’s time to move onto the next step.

Need a helping hand?

Step 2: Log, catalogue and synthesise

Once you’ve built a little treasure trove of articles, it’s time to get reading and start digesting the information – what does it all mean?

While I present steps one and two (hunting and digesting) as sequential, in reality, it’s more of a back-and-forth tango – you’ll read a little , then have an idea, spot a new citation, or a new potential variable, and then go back to searching for articles. This is perfectly natural – through the reading process, your thoughts will develop , new avenues might crop up, and directional adjustments might arise. This is, after all, one of the main purposes of the literature review process (i.e. to familiarise yourself with the current state of research in your field).

As you’re working through your treasure chest, it’s essential that you simultaneously start organising the information. There are three aspects to this:

• Logging reference information
• Building an organised catalogue
• Distilling and synthesising the information

I’ll discuss each of these below:

2.1 – Log the reference information

As you read each article, you should add it to your reference management software. I usually recommend Mendeley for this purpose (see the Mendeley 101 video below), but you can use whichever software you’re comfortable with. Most importantly, make sure you load EVERY article you read into your reference manager, even if it doesn’t seem very relevant at the time.

2.2 – Build an organised catalogue

In the beginning, you might feel confident that you can remember who said what, where, and what their main arguments were. Trust me, you won’t. If you do a thorough review of the relevant literature (as you must!), you’re going to read many, many articles, and it’s simply impossible to remember who said what, when, and in what context . Also, without the bird’s eye view that a catalogue provides, you’ll miss connections between various articles, and have no view of how the research developed over time. Simply put, it’s essential to build your own catalogue of the literature.

I would suggest using Excel to build your catalogue, as it allows you to run filters, colour code and sort – all very useful when your list grows large (which it will). How you lay your spreadsheet out is up to you, but I’d suggest you have the following columns (at minimum):

• Author, date, title – Start with three columns containing this core information. This will make it easy for you to search for titles with certain words, order research by date, or group by author.
• Categories or keywords – You can either create multiple columns, one for each category/theme and then tick the relevant categories, or you can have one column with keywords.
• Key arguments/points – Use this column to succinctly convey the essence of the article, the key arguments and implications thereof for your research.
• Context – Note the socioeconomic context in which the research was undertaken. For example, US-based, respondents aged 25-35, lower- income, etc. This will be useful for making an argument about gaps in the research.
• Methodology – Note which methodology was used and why. Also, note any issues you feel arise due to the methodology. Again, you can use this to make an argument about gaps in the research.
• Quotations – Note down any quoteworthy lines you feel might be useful later.
• Notes – Make notes about anything not already covered. For example, linkages to or disagreements with other theories, questions raised but unanswered, shortcomings or limitations, and so forth.

If you’d like, you can try out our free catalog template here (see screenshot below).

2.3 – Digest and synthesise

Most importantly, as you work through the literature and build your catalogue, you need to synthesise all the information in your own mind – how does it all fit together? Look for links between the various articles and try to develop a bigger picture view of the state of the research. Some important questions to ask yourself are:

• What answers does the existing research provide to my own research questions ?
• Which points do the researchers agree (and disagree) on?
• How has the research developed over time?
• Where do the gaps in the current research lie?

To help you develop a big-picture view and synthesise all the information, you might find mind mapping software such as Freemind useful. Alternatively, if you’re a fan of physical note-taking, investing in a large whiteboard might work for you.

Step 3: Outline and write it up!

Once you’re satisfied that you have digested and distilled all the relevant literature in your mind, it’s time to put pen to paper (or rather, fingers to keyboard). There are two steps here – outlining and writing:

3.1 – Draw up your outline

Having spent so much time reading, it might be tempting to just start writing up without a clear structure in mind. However, it’s critically important to decide on your structure and develop a detailed outline before you write anything. Your literature review chapter needs to present a clear, logical and an easy to follow narrative – and that requires some planning. Don’t try to wing it!

Naturally, you won’t always follow the plan to the letter, but without a detailed outline, you’re more than likely going to end up with a disjointed pile of waffle , and then you’re going to spend a far greater amount of time re-writing, hacking and patching. The adage, “measure twice, cut once” is very suitable here.

In terms of structure, the first decision you’ll have to make is whether you’ll lay out your review thematically (into themes) or chronologically (by date/period). The right choice depends on your topic, research objectives and research questions, which we discuss in this article .

Once that’s decided, you need to draw up an outline of your entire chapter in bullet point format. Try to get as detailed as possible, so that you know exactly what you’ll cover where, how each section will connect to the next, and how your entire argument will develop throughout the chapter. Also, at this stage, it’s a good idea to allocate rough word count limits for each section, so that you can identify word count problems before you’ve spent weeks or months writing!

PS – check out our free literature review chapter template…

3.2 – Get writing

With a detailed outline at your side, it’s time to start writing up (finally!). At this stage, it’s common to feel a bit of writer’s block and find yourself procrastinating under the pressure of finally having to put something on paper. To help with this, remember that the objective of the first draft is not perfection – it’s simply to get your thoughts out of your head and onto paper, after which you can refine them. The structure might change a little, the word count allocations might shift and shuffle, and you might add or remove a section – that’s all okay. Don’t worry about all this on your first draft – just get your thoughts down on paper.

Once you’ve got a full first draft (however rough it may be), step away from it for a day or two (longer if you can) and then come back at it with fresh eyes. Pay particular attention to the flow and narrative – does it fall fit together and flow from one section to another smoothly? Now’s the time to try to improve the linkage from each section to the next, tighten up the writing to be more concise, trim down word count and sand it down into a more digestible read.

Once you’ve done that, give your writing to a friend or colleague who is not a subject matter expert and ask them if they understand the overall discussion. The best way to assess this is to ask them to explain the chapter back to you. This technique will give you a strong indication of which points were clearly communicated and which weren’t. If you’re working with Grad Coach, this is a good time to have your Research Specialist review your chapter.

Finally, tighten it up and send it off to your supervisor for comment. Some might argue that you should be sending your work to your supervisor sooner than this (indeed your university might formally require this), but in my experience, supervisors are extremely short on time (and often patience), so, the more refined your chapter is, the less time they’ll waste on addressing basic issues (which you know about already) and the more time they’ll spend on valuable feedback that will increase your mark-earning potential.

Literature Review Example

In the video below, we unpack an actual literature review so that you can see how all the core components come together in reality.

Let’s Recap

In this post, we’ve covered how to research and write up a high-quality literature review chapter. Let’s do a quick recap of the key takeaways:

• It is essential to understand the WHY of the literature review before you read or write anything. Make sure you understand the 4 core functions of the process.
• The first step is to hunt down the relevant literature . You can do this using Google Scholar, your university database, the snowballing technique and by reviewing other dissertations and theses.
• Next, you need to log all the articles in your reference manager , build your own catalogue of literature and synthesise all the research.
• Following that, you need to develop a detailed outline of your entire chapter – the more detail the better. Don’t start writing without a clear outline (on paper, not in your head!)
• Write up your first draft in rough form – don’t aim for perfection. Remember, done beats perfect.
• Refine your second draft and get a layman’s perspective on it . Then tighten it up and submit it to your supervisor.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

38 Comments

Thank you very much. This page is an eye opener and easy to comprehend.

This is awesome!

I wish I come across GradCoach earlier enough.

But all the same I’ll make use of this opportunity to the fullest.

Thank you for this good job.

Keep it up!

You’re welcome, Yinka. Thank you for the kind words. All the best writing your literature review.

Thank you for a very useful literature review session. Although I am doing most of the steps…it being my first masters an Mphil is a self study and one not sure you are on the right track. I have an amazing supervisor but one also knows they are super busy. So not wanting to bother on the minutae. Thank you.

You’re most welcome, Renee. Good luck with your literature review 🙂

This has been really helpful. Will make full use of it. 🙂

Thank you Gradcoach.

Really agreed. Admirable effort

thank you for this beautiful well explained recap.

Thank you so much for your guide of video and other instructions for the dissertation writing.

It is instrumental. It encouraged me to write a dissertation now.

Thank you the video was great – from someone that knows nothing thankyou

an amazing and very constructive way of presetting a topic, very useful, thanks for the effort,

It is timely

It is very good video of guidance for writing a research proposal and a dissertation. Since I have been watching and reading instructions, I have started my research proposal to write. I appreciate to Mr Jansen hugely.

I learn a lot from your videos. Very comprehensive and detailed.

Thank you for sharing your knowledge. As a research student, you learn better with your learning tips in research

I was really stuck in reading and gathering information but after watching these things are cleared thanks, it is so helpful.

Really helpful, Thank you for the effort in showing such information

This is super helpful thank you very much.

Thank you for this whole literature writing review.You have simplified the process.

I’m so glad I found GradCoach. Excellent information, Clear explanation, and Easy to follow, Many thanks Derek!

You’re welcome, Maithe. Good luck writing your literature review 🙂

Thank you Coach, you have greatly enriched and improved my knowledge

Great piece, so enriching and it is going to help me a great lot in my project and thesis, thanks so much

This is THE BEST site for ANYONE doing a masters or doctorate! Thank you for the sound advice and templates. You rock!

Thanks, Stephanie 🙂

This is mind blowing, the detailed explanation and simplicity is perfect.

I am doing two papers on my final year thesis, and I must stay I feel very confident to face both headlong after reading this article.

thank you so much.

if anyone is to get a paper done on time and in the best way possible, GRADCOACH is certainly the go to area!

This is very good video which is well explained with detailed explanation

Thank you excellent piece of work and great mentoring

Thanks, it was useful

Thank you very much. the video and the information were very helpful.

Good morning scholar. I’m delighted coming to know you even before the commencement of my dissertation which hopefully is expected in not more than six months from now. I would love to engage my study under your guidance from the beginning to the end. I love to know how to do good job

Thank you so much Derek for such useful information on writing up a good literature review. I am at a stage where I need to start writing my one. My proposal was accepted late last year but I honestly did not know where to start

Like the name of your YouTube implies you are GRAD (great,resource person, about dissertation). In short you are smart enough in coaching research work.

This is a very well thought out webpage. Very informative and a great read.

Very timely.

I appreciate.

Very comprehensive and eye opener for me as beginner in postgraduate study. Well explained and easy to understand. Appreciate and good reference in guiding me in my research journey. Thank you

Thank you. I requested to download the free literature review template, however, your website wouldn’t allow me to complete the request or complete a download. May I request that you email me the free template? Thank you.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• Strategy: Literature Reviews

What is a literature review?

Typically, a literature review is a written discussion that examines publications about  a particular subject area or topic. 

A Literature Review provides an overview of selected sources on a topic.

Depending on disciplines, publications, or authors a literature review may be:  

a summary of sources an organized presentation of sources a synthesis or interpretation of sources an evaluative analysis of sources

A Literature Review may be part of a process or a product . 

It may be: 

a part of your research process a part of your final research publication an independent publication

Why do a literature review?

The Literature Review will place your research in context . 

It will help you and your readers: 

Locate   patterns, relationships, connections, agreements, disagreements, & gaps in understanding Identify methodological and theoretical foundations Identify landmark and exemplary works Situate your voice in a broader conversation with other writers, thinkers, and scholars

The Literature Review will aid your research process. 

It will help you to: 

establish your knowledge understand what has been said define your questions establish a relevant methodology refine your voice situate your voice in the conversation

What does a literature review look like?

The Literature Review structure and organization . 

an introduction or overview a body or organizational sub-divisions a conclusion or an explanation of significance

The body of a literature review may be organized: 

chronologically: organized by date of publication  methodologically: organized by type of research method used  thematically: organized by concept, trend, or theme  ideologically: organized by belief, ideology, or school of thought

• Find a focus
• Find models
• Review your target publication
• Track citations 
• Read critically
• Manage your citations
• Ask friends, faculty, and librarians

• Next: Managing Sources >>
• Literature Review
• Managing Sources

More sources

• Byrne, D. (2017). Reviewing the literature. Project Planner. 10.4135/9781526408518.
• Literature Review: By UNC Writing Center
• PhD on Track
• CU Graduate Students Thesis & Dissertation
• CU Honors Thesis
• University of Colorado Boulder Libraries
• Research Guides
• Site: Research Strategies
• Last Updated: Aug 10, 2020 11:33 AM
• URL: https://libguides.colorado.edu/strategies/litreview

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

Literature reviews

What this guide covers, what is a literature review, literature review resources, types of literature reviews, what is the difference between a literature review and a systematic review, related information and guides, further help.

• Conducting your search
• Store and organise the literature
• Evaluate and critique the literature
• Different subject areas
• Find literature reviews

Reusing content from this guide

Attribute our work under a Creative Commons Attribution-NonCommercial 4.0 International License.

The literature review process involves a number of steps.

This guide focuses on:

• evaluating.

A literature review is a survey and critical analysis of what has been written on a particular topic, theory, question or method.

"In writing the literature review, the purpose is to explore what knowledge and ideas have been established on a topic, what approaches and viewpoints have been adopted, and what are their strengths and weaknesses."

Source: "Focus and frame". (2008). In Eriksson, P. & Kovalainen, A. Introducing Qualitative Methods: Qualitative methods in business research (pp. 44) . London: SAGE Publications Ltd. doi: 10.4135/9780857028044.

Get an overview on doing a literature review:

• Sage research methods online - Literature review methods map Information on the literature review methodology with links to further resources - the Project Planner, books, articles, videos and more.
• Ten simple rules for writing a literature review Gives 10 tips on how to approach and carry out a literature review. By Pautasso M (2013) Ten Simple Rules for Writing a Literature Review. PLoS Comput Biol9(7): e1003149.
• The literature review. In: Doing your undergraduate program This chapter looks at the purpose of literature reviews, how it is done, setting the boundaries of your search and more.

• More books on literature reviews A selection of literature review books available via UQ Library Search.

The type of literature review you do will depend on a variety of factors:

• Your discipline
• The purpose - undergraduate assessment, PHD thesis, journal article?
• Your lecturer or supervisor's requirements

Always follow the guidelines outlined by your lecturer or supervisor or consult the instructions for authors (for journal articles), when conducting your literature review.

• is an overview of the significant literature on a topic
• typically includes a critical analysis of each work included
• demonstrates the reviewers knowledge of the topic
• is a list of citations of research sources (books, journal articles, websites etc) on a topic
• includes a brief summary and analysis or evaluation of each citation = the annotation
• a critical assessment of all research studies on a particular research question
• has specific criteria for collecting and evaluating the literature
• includes a synthesis of the findings of the included studies
• This method developed by Griffith University's School of Environment bridges the gap between traditional narrative review methods and meta-analyses to enable students to produce results that are reliable, quantifiable and reproducible.

The requirements of narrative literature reviews are usually quite different than systematic reviews . However, you may be required to adopt some of the characteristics of a systematic approach when doing your literature review. Check the guidelines or criteria that have been set by your supervisor so you know what is expected of you.

Characteristics of reviews

• Meeting the review family: Exploring review types and associated information retrieval requirements This article defines different review types and discusses appropriate search methods for each type.
• Writing literature reviews - Student Support Student Support provides information on how to write effective literature reviews.
• Writing skills Learn strategies for good writing from the Graduate School.
• Systematic reviews An overview of systematic reviews and resources to support producing one.
• Subject guides See recommended resources in different subject areas.
• Grey literature Find literature that is not available in traditional channels of publishing and distribution.
• How to find guides Techniques and resources to find specific information formats.

Contact the Librarian team .

Phone: + 617 334 64312 during opening hours

Email: [email protected]

• Next: Conducting your search >>
• Last Updated: Dec 15, 2023 12:09 PM
• URL: https://guides.library.uq.edu.au/research-techniques/literature-reviews

Methods for Indirect Treatment Comparison: Results from a Systematic Literature Review

Affiliations.

• 1 Department of Public Health, Aix-Marseille University, 13005 Marseille, France.
• 2 Pierre Fabre Laboratories, 92100 Paris, France.
• 3 Amaris, Montréal, QC H2Y 2N1, Canada.
• 4 InovIntell, 3023GJ Rotterdam, The Netherlands.
• PMID: 38660413
• PMCID: PMC11036291
• DOI: 10.3390/jmahp12020006

Introduction: Health technology assessment (HTA) agencies express a clear preference for randomized controlled trials when assessing the comparative efficacy of two or more treatments. However, an indirect treatment comparison (ITC) is often necessary where a direct comparison is unavailable or, in some cases, not possible. Numerous ITC techniques are described in the literature. A systematic literature review (SLR) was conducted to identify all the relevant literature on existing ITC techniques, provide a comprehensive description of each technique and evaluate their strengths and limitations from an HTA perspective in order to develop guidance on the most appropriate method to use in different scenarios.

Methods: Electronic database searches of Embase and PubMed, as well as grey literature searches, were conducted on 15 November 2021. Eligible articles were peer-reviewed papers that specifically described the methods used for different ITC techniques and were written in English. The review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Results: A total of 73 articles were included in the SLR, reporting on seven different ITC techniques. All reported techniques were forms of adjusted ITC. Network meta-analysis (NMA) was the most frequently described technique (in 79.5% of the included articles), followed by matching-adjusted indirect comparison (MAIC) (30.1%), network meta-regression (24.7%), the Bucher method (23.3%), simulated treatment comparison (STC) (21.9%), propensity score matching (4.1%) and inverse probability of treatment weighting (4.1%). The appropriate choice of ITC technique is critical and should be based on the feasibility of a connected network, the evidence of heterogeneity between and within studies, the overall number of relevant studies and the availability of individual patient-level data (IPD). MAIC and STC were found to be common techniques in the case of single-arm studies, which are increasingly being conducted in oncology and rare diseases, whilst the Bucher method and NMA provide suitable options where no IPD is available.

Conclusion: ITCs can provide alternative evidence where direct comparative evidence may be missing. ITCs are currently considered by HTA agencies on a case-by-case basis; however, their acceptability remains low. Clearer international consensus and guidance on the methods to use for different ITC techniques is needed to improve the quality of ITCs submitted to HTA agencies. ITC techniques continue to evolve quickly, and more efficient techniques may become available in the future.

Keywords: Bucher; indirect treatment comparison (ITC); matching-adjusted indirect comparison (MAIC); methodology; methods; network meta-analysis (NMA); oncology; systematic literature review (SLR).

© 2024 by the authors.

Publication types

Grants and funding.

Exploring current research trends in sound event detection: a systematic literature review

• Published: 29 April 2024

Cite this article

• Sallauddin Mohmmad   ORCID: orcid.org/0000-0002-4525-4894 1 , 2 &
• Suresh Kumar Sanampudi 3  

Sound Event Detection (SED) plays a significant role in the present research, implemented in several areas such as Computer Science, Healthcare, Environmental Science, Security and Surveillance, etc. With the advancement of technology, SED can be deployed to mimic the human auditory system. In this paper, we have undertaken a Systematic Literature Review focused on sound event detection, presenting a comprehensive and well-structured analysis and in-depth discussions. This review is based on the authors' extensive knowledge and expertise in the field, and it compares various algorithms employed for sound event detection. The primary objective of this study is to offer valuable insights into datasets, feature extraction techniques, and execution models commonly used in SED, along with an examination of their corresponding accuracy, challenges, and limitations. Furthermore, the paper delves into identifying potential trends within the field, offering forward-looking information that can be invaluable for future research and development efforts in sound event detection. This systematic review aims to contribute to the continued advancement of SED technologies and applications by synthesizing existing knowledge and identifying emerging directions. It provides a foundation for researchers, practitioners, and stakeholders to make informed decisions and explore new possibilities within this evolving domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study .

Kitchenham B, Brereton OP, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering–a systematic literature review. Inf Softw Technol 51(1): 7–15

Mesaros A, Heittola T, Virtanen T (2016) TUT database for acoustic scene classification and sound event detection. 24th European signal processing conference (EUSIPCO), pp 1128–1132. https://doi.org/10.1109/EUSIPCO.2016.7760424

Lim H, Park J, Han Y (2017) Rare sound event detection using 1D convolutional recurrent neural networks. In: Proceedings of the detection and classification of acoustic scenes and events 2017 workshop (DCASE2017), pp 80–84

Kawaguchi Y, Tanabe R, Endo T, Ichige K, Hamada K (2019) Anomaly detection based on an ensemble of dereverberation and anomalous sound extraction. In: ICASSP 2019–2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 865–869

Adavanne S, Virtanen T (2017) Sound event detection using weakly labeled dataset with stacked convolutional and recurrent neural network. arXiv preprint arXiv:1710.02998

Archontis P, Mesaros A, Adavanne S, Heittola T, Virtanen T (2020) Overview and evaluation of sound event localization and detection in DCASE2019. IEEE/ACM transactions on audio, speech, and language processing, 29 pp 684–698

Kawaguchi Y, Endo T, Ichige K, Hamada K (2018) Non-negative novelty extraction: A new non-negativity constraint for NMF. 16th international workshop on acoustic signal enhancement (IWAENC), pp 256–260

Küçükbay SE, Sert M (2015) Audio-based event detection in office live environments using optimized MFCC-SVM approach. In: Proceedings of the 2015 IEEE 9th international conference on semantic computing (IEEE ICSC 2015), pp 475–480

Mak M-W, Kung S-Y (2012) Low-power SVM classifiers for sound event classification on mobile devices. In: 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1985–1988

Parathai P, Tengtrairat N, Woo WL, Abdullah MAM, Rafiee G, Alshabrawy O (2020) Efficient noisy sound-event mixture classification using adaptive-sparse complex-valued matrix factorization and OvsO SVM. Sensors 20(16):4368

Tran HD, Li H (2010) Sound event recognition with probabilistic distance SVMs. IEEE Trans Audio Speech Lang Process 19(6):1556–1568

Article   Google Scholar  

Yu C-Y, Liu H, Qi Z-M (2017) Sound event detection using deep random forest. Detection and Classification of Acoustic Scenes and Events

Phan H, Maaß M, Mazur R, Mertins A (2014) Random regression forests for acoustic event detection and classification. IEEE/ACM Trans Audio Speech Lang Process 23(1):20–31

Xia X, Togneri R, Sohel F, Huang D (2017) Random forest classification based acoustic event detection. IEEE International Conference on Multimedia and Expo (ICME), pp 163–168

Xia X, Togneri R, Sohel F, Huang D (2018) Random forest classification based acoustic event detection utilizing contextual-information and bottleneck features. Pattern Recogn 81(2018):1–13

Stoller D, Ewert S, Dixon S (2018) Wave-u-net: A multi-scale neural network for end-to-end audio source separation. arXiv preprint arXiv:1806.03185

Park J, Shin J, Lee K (2018) Separation of instrument sounds using non-negative matrix factorization with spectral envelope constraints. arXiv preprint arXiv:1801.04081

Chan TK, Chin CS, Li Y (2020) Non-negative matrix factorization-convolutional neural network (NMF-CNN) for sound event detection. arXiv preprint arXiv:2001.07874

Bisot V, Essid S, Richard G (2017) Overlapping sound event detection with supervised nonnegative matrix factorization. IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 31–35

Imoto K, Tonami N, Koizumi Y, Yasuda M, Yamanishi R, Yamashita Y (2020) Sound event detection by multitask learning of sound events and scenes with soft scene labels. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 621–625

Wei W, Zhu H, Benetos E, Wang Y (2020) A-crnn: A domain adaptation model for sound event detection. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 276–280

Innami S, Kasai H (2012) NMF-based environmental sound source separation using time-variant gain features. Comput Math Appl 64(5):1333–1342

Komatsu T, Senda Y, Kondo R (2016) Acoustic event detection based on non-negative matrix factorization with mixtures of local dictionaries and activation aggregation. IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2259–2263

Noh K, Chang J-H (2020) Joint optimization of deep neural network-based dereverberation and beam forming for sound event detection in multi-channel environments. Sensors 20(7):1883

Turpault N, Serizel R, Wisdom S, Erdogan H, Hershey JR, Fonseca E, Seetharaman P, Salamon J (2021) Sound event detection and separation: a benchmark on desed synthetic soundscapes. In: ICASSP 2021–2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 840–844

Komatsu T, Toizumi T, Kondo R, Senda Y (2016) Acoustic event detection method using semi-supervised non-negative matrix factorization with a mixture of local dictionaries. In: Proceedings of the detection and classification of acoustic scenes and events 2016 workshop (DCASE2016), pp 45–49

Kong Q, Cao Y, Iqbal T, Xu Y, Wang W, Plumbley MD (2019) Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems. arXiv preprint arXiv:1904.03476

Grondin F, Glass J, Sobieraj I, Plumbley MD (2019) Sound event localization and detection using CRNN on pairs of microphones. arXiv preprint arXiv:1910.10049

Adavanne S, Politis A, Virtanen T (2019) A multi-room reverberant dataset for sound event localization and detection. arXiv preprint arXiv:1905.08546

Zhang J, Ding W, He L (2019) Data augmentation and prior knowledge-based regularization for sound event localization and detection. DCASE 2019 detection and classification of acoustic scenes and events 2019 Challenge

Cao Y, Iqbal T, Kong Q, Galindo M, Wang W, Plumbley M (2019) Two-stage sound event localization and detection using intensity vector and generalized cross-correlation. DCASE2019 Challenge, Tech. Rep

Adavanne S, Politis A, Nikunen J, Virtanen T (2018) Sound event localization and detection of overlapping sources using convolutional recurrent neural networks. IEEE J Sel Top Signal Process 13(1):34–48

Xue W, Tong Y, Zhang C, Ding G, He X, Zhou B (2020) Sound event localization and detection based on multiple DOA beam forming and multi-task learning. Proc. Interspeech 2020 : 5091-5095

Nguyen TNT, Jones DL, Gan W (2020) Ensemble of sequence matching networks for dynamic sound event localization detection and tracking. In: Detection and classification of acoustic scenes and events 2020 workshop (DCASE2020)

Trowitzsch I, Schymura C, Kolossa D, Obermayer K (2019) Joining sound event detection and localization through spatial segregation. IEEE/ACM Trans Audio Speech Lang Process 28:487–502

Kim B, Pardo B (2019) Sound event detection using point-labeled data. IEEE workshop on applications of signal processing to audio and acoustics (WASPAA), pp 1–5

Xia X, Togneri R, Sohel F, Huang D (2018) Auxiliary classifier generative adversarial network with soft labels in imbalanced acoustic event detection. IEEE Trans Multimedia 21(6):1359–1371

Basaran D, Essid S, Peeters G (2018) Main melody extraction with source-filter NMF and CRNN. In: 19th International Society for Music Information Retreival. 2018

Boulanger-Lewandowski N, Mysore GJ, Hoffman M (2014) Exploiting long-term temporal dependencies in NMF using recurrent neural networks with application to source separation. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 6969–6973. IEEE

Liu S, Guo L, Wiggins GA (2018) A parallel fusion approach to piano music transcription based on convolutional neural network. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 391–395. IEEE

Hsieh T-H, Su L, Yang Y-H (2019) A streamlined encoder/decoder architecture for melody extraction. In: ICASSP 2019–2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 156–160. IEEE

Machado RB, Aguiar L, Jones G (2017) Do acoustic indices reflect the characteristics of bird communities in the savannas of Central Brazil? Landsc Urban Plan 162:36–43

Ross S-J, Friedman NR, Dudley KL, Yoshimura M, Yoshida T, Economo EP (2018) Listening to ecosystems: data-rich acoustic monitoring through landscape-scale sensor networks. Ecol Res 33(1):135–147

Gómez WE, Isaza CV, Daza JM (2018) Identifying disturbed habitats: a new method from acoustic indices. Eco Inform 45:16–25

Khanaposhtani MG, Gasc A, Francomano D, Villanueva-Rivera LJ, Jung J, Mossman MJ, Pijanowski BC (2019) Effects of highways on bird distribution and soundscape diversity around Aldo Leopold’s shack in Baraboo, Wisconsin, USA. Landsc Urban Plan 192:103666

Siddagangaiah S, Chen C-F, Wei-Chun Hu, Pieretti N (2019) A complexity-entropy based approach for the detection of fish choruses. Entropy 21(10):977

Roma G, Nogueira W, Herrera P (2013) Recurrence quantification analysis features for environmental sound recognition. In: 2013 IEEE workshop on applications of signal processing to audio and acoustics, pp 1–4. IEEE

Sobieraj I, Kong Q, Plumbley MD (2017) Masked non-negative matrix factorization for bird detection using weakly labeled data. In: 2017 25th European signal processing conference (EUSIPCO), pp 1769–1773. IEEE

Yu S, Yi Yu, Chen Xi, Li W (2021) HANME: hierarchical attention network for singing melody extraction. IEEE Signal Process Lett 28:1006–1010

Surampudi N, Srirangan M, Christopher J (2019) Enhanced feature extraction approaches for detection of sound events. In: 2019 IEEE 9th international conference on advanced computing (IACC), pp 223–229. IEEE

Gumelar AB, Kurniawan A, Sooai AG, Purnomo MH, Yuniarno ME, Sugiarto I, Widodo A, Kristanto AA, Fahrudin TM (2019) Human voice emotion identification using prosodic and spectral feature extraction based on deep neural networks. In: 2019 IEEE 7th international conference on serious games and applications for health (SeGAH), pp 1–8. IEEE

Jain U, Nathani K, Ruban N, Raj ANJ, Zhuang Z, Mahesh VGV (2018) Cubic SVM classifier based feature extraction and emotion detection from speech signals. In: 2018 international conference on sensor networks and signal processing (SNSP), pp 386–391. IEEE

Lee S, Pang H-S (2020) Feature extraction based on the non-negative matrix factorization of convolutional neural networks for monitoring domestic activity with acoustic signals. IEEE Access 8:122384–122395

Piczak KJ (2015) ESC: Dataset for environmental sound classification. In: Proceedings of the 23rd ACM international conference on multimedia, pp 1015–1018

Zinemanas P, Cancela P, Rocamora M (2019) MAVD: a dataset for sound event detection in urban environments. Detection and classification of acoustic scenes and events, DCASE 2019, New York, NY, USA, 25–26 Oct, page 263–267

Mesaros A, Heittola T, Virtanen T (2016) August. TUT database for acoustic scene classification and sound event detection. In: 2016 24th European Signal Processing Conference (EUSIPCO), pp 1128–1132). IEEE

Salamon J, Jacoby C, Bello JP (2014) A dataset and taxonomy for urban sound research. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 1041–1044

Stowell D, Plumbley MD (2013) An open dataset for research on audio field recording archives: freefield1010. arXiv preprint arXiv:1309.5275

Vozáriková E, Juhár J, Čižmár A (2011) Acoustic events detection using MFCC and MPEG-7 descriptors. In: International conference on multimedia communications, services and security, pp 191–197. Springer, Berlin, Heidelberg

Johnson DS, Lorenz W, Taenzer M, Mimilakis S, Grollmisch S, Abeßer J, Lukashevich H (2021) Desed-fl and urban-fl: Federated learning datasets for sound event detection. In: 2021 29th European signal processing conference (EUSIPCO), pp 556–560. IEEE

Purohit H, Tanabe R, Ichige K, Endo T, Nikaido Y, Suefusa K, Kawaguchi Y (2019) MIMII dataset: sound dataset for malfunctioning industrial machine investigation and inspection. arXiv preprint arXiv:1909.09347

Hertel L, Phan H, Mertins A (2016) Comparing time and frequency domain for audio event recognition using deep learning. In: 2016 International Joint Conference on Neural Networks (Ijcnn), pp 3407–3411. IEEE

Gemmeke JF, Ellis DP, Freedman D, Jansen A, Lawrence W, Moore RC, Plakal M, Ritter M (2017) Audio set: An ontology and human-labeled dataset for audio events. In: 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 776–780. IEEE

Ooi K, Watcharasupat KN, Peksi S, Karnapi FA, Ong ZT, Chua D, Leow HW, Kwok LL, Ng XL, Loh ZA, Gan WS (2021) A strongly-labelled polyphonic dataset of urban sounds with spatiotemporal context. arXiv preprint arXiv:2111.02006

Cartwright M, Cramer J, Mendez AEM, Wang Y, Wu HH, Lostanlen V, Fuentes M, Dove G, Mydlarz C, Salamon J, Nov O (2020) SONYC-UST-V2: An urban sound tagging dataset with spatiotemporal context. arXiv preprint arXiv:2009.05188

Fonseca E, Favory X, Pons J, Font F, Serra X (2020) FSD50k: an open dataset of human-labeled sound events. arXiv preprint arXiv:2010.00475

Abeßer J (2021) USM-SED-A dataset for polyphonic sound event detection in urban sound monitoring scenarios. arXiv preprint arXiv:2105.02592

McFee B, Bertin-Mahieux T, Ellis DP, Lanckriet GR (2012) The million song dataset challenge. In: Proceedings of the 21st International Conference on World Wide Web, pp 909–916

Gao S, Zheng Y, Guo X (2020) Gated recurrent unit-based heart sound analysis for heart failure screening. Biomed Eng Online 19(1):1–17

Fonseca E, Pons Puig J, Favory X, Font Corbera F, Bogdanov D, Ferraro A, Oramas S, Porter A, Serra X (2017) Freesound datasets: a platform for the creation of open audio datasets. In: Hu X, Cunningham SJ, Turnbull D, Duan Z (eds) Proceedings of the 18th ISMIR Conference; 2017 oct 23–27; Suzhou, China.[Canada]: International Society for Music Information Retrieval, pp 486–93. International Society for Music Information Retrieval (ISMIR)

Koizumi Y, Saito S, Uematsu H, Harada N, Imoto K (2019) ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection. In: 2019 IEEE workshop on applications of signal processing to audio and acoustics (WASPAA), pp 313–317. IEEE

Cartwright M, Mendez AEM, Cramer J, Lostanlen V, Dove G, Wu HH, Salamon J, Nov O, Bello J (2019) SONYC Urban Sound Tagging (SONYC-UST): A multilabel dataset from an urban acoustic sensor network

Li Y, Liu M, Drossos K, Virtanen T (2020) Sound event detection via dilated convolutional recurrent neural networks. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 286–290. IEEE

Mesaros A, Heittola T, Virtanen T (2018) A multi-device dataset for urban acoustic scene classification. arXiv preprint arXiv:1807.09840

Wan M, Wang R, Wang B, Bai J, Chen C, Fu Z, Chen J, Zhang X, Rahardja S (2019) Ciaic-ASC system for DCASE 2019 challenge task1. Tech. Rep., DCASE2019 Challenge

Heittola T, Mesaros A, Virtanen T (2020) Acoustic scene classification in dcase 2020 challenge: generalization across devices and low complexity solutions. arXiv preprint arXiv:2005.14623

Rakotomamonjy A, Gasso G (2014) Histogram of gradients of time–frequency representations for audio scene classification. IEEE/ACM Trans Audio Speech Lang Process 23(1):142–153

Google Scholar  

Mesaros A, Heittola T, Diment A, Elizalde B, Shah A, Vincent E, Raj B, Virtanen T (2017) DCASE 2017 challenge setup: Tasks, datasets and baseline system. In: DCASE 2017-workshop on detection and classification of acoustic scenes and events

Koduru A, Valiveti HB, Budati AK (2020) Feature extraction algorithms to improve the speech emotion recognition rate. Int J Speech Technol 23(1):45–55

Zhang Keming, Cai Yuanwen, Ren Yuan, Ye Ruida, He Liang (2020) MTF-CRNN: multiscale time-frequency convolutional recurrent neural network for sound event detection. IEEE Access 8:147337–147348

Özseven T, Düğenci M (2018) SPeech ACoustic (SPAC): A novel tool for speech feature extraction and classification. Appl Acoust 136:1–8

Dang A, Vu TH, Wang JC (2018) Acoustic scene classification using convolutional neural networks and multi-scale multi-feature extraction. In: 2018 IEEE international conference on consumer electronics (ICCE), pp. 1–4. IEEE

Glowacz Adam (2018) Acoustic-based fault diagnosis of commutator motor. Electronics 7(11):299

Deng M, Meng T, Cao J, Wang S, Zhang J, Fan H (2020) Heart sound classification based on improved MFCC features and convolutional recurrent neural networks. Neural Netw 130:22–32

Heittola T, Mesaros A, Eronen A, Virtanen T (2013) Context-dependent sound event detection. EURASIP J Audio Speech Music Process 2013(1):1–13

Mesaros A, Heittola T, Dikmen O, Virtanen T (2015) Sound event detection in real life recordings using coupled matrix factorization of spectral representations and class activity annotations. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 151–155. IEEE

Ohishi Y, Mochihashi D, Matsui T, Nakano M, Kameoka H, Izumitani T, Kashino K (2013) Bayesian semi-supervised audio event transcription based on Markov Indian buffet process. In: 2013 IEEE international conference on acoustics, speech and signal processing, pp 3163–3167. IEEE

Cakir E, Heittola T, Huttunen H, Virtanen T (2015) Multi-label vs. combined single-label sound event detection with deep neural networks. In: 2015 23rd European signal processing conference (EUSIPCO), pp. 2551–2555. IEEE

Parascandolo G, Huttunen H, Virtanen T (2016) Recurrent neural networks for polyphonic sound event detection in real life recordings. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 6440–6444. IEEE

Adavanne S, Parascandolo G, Pertilä P, Heittola T, Virtanen T (2017) Sound event detection in multichannel audio using spatial and harmonic features. arXiv preprint arXiv:1706.02293

Cakır E, Parascandolo G, Heittola T, Huttunen H, Virtanen T (2017) Convolutional recurrent neural networks for polyphonic sound event detection. IEEE/ACM Trans Audio Speech Lang Process 25(6):1291–1303

Jung S, Park J, Lee S (2019) Polyphonic sound event detection using convolutional bidirectional lstm and synthetic data-based transfer learning. In: ICASSP 2019–2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 885–889. IEEE

Adavanne S, Pertilä P, Virtanen T (2017) Sound event detection using spatial features and convolutional recurrent neural network. In: 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 771–775. IEEE

Mondal Ashok, Banerjee Poulami, Tang Hong (2018) A novel feature extraction technique for pulmonary sound analysis based on EMD. Comput Methods Programs Biomed 159:199–209

Mushtaq Zohaib, Shun-Feng Su (2020) Environmental sound classification using a regularized deep convolutional neural network with data augmentation. Appl Acoust 167:107389

Lin L, Wang X, Liu H, Qian Y (2019) Guided learning convolution system for dcase 2019 task 4. arXiv preprint arXiv:1909.06178

Altinors Ayhan, Yol Ferhat, Yaman Orhan (2021) A sound based method for fault detection with statistical feature extraction in UAV motors. Appl Acoust 183:108325

Adavanne S, Politis A, Virtanen T (2018) Multichannel sound event detection using 3D convolutional neural networks for learning inter-channel features. In: 2018 international joint conference on neural networks (IJCNN), pp 1–7. IEEE

Kong Q, Xu Y, Wang W, Plumbley MD (2020) Sound event detection of weakly labelled data with CNN-transformer and automatic threshold optimization. IEEE/ACM Trans Audio Speech Lang Process 28:2450–2460

Lin L, Wang X, Liu H, Qian Y (2020) Guided learning for weakly-labeled semi-supervised sound event detection. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP, pp 626–630. IEEE

Alías F, Socoró JC, Sevillano X (2016) A review of physical and perceptual feature extraction techniques for speech, music and environmental sounds. Appl Sci 6(5):143

Piczak KJ (2015) Environmental sound classification with convolutional neural networks. In: 2015 IEEE 25th international workshop on machine learning for signal processing (MLSP), pp 1–6. IEEE

Cakir E, Heittola T, Huttunen H, Virtanen T (2015) Polyphonic sound event detection using multi label deep neural networks. In: 2015 international joint conference on neural networks (IJCNN), pp 1–7. IEEE

Madhu A, Kumaraswamy S (2019) Data augmentation using generative adversarial network for environmental sound classification. In: 2019 27th European signal processing conference (EUSIPCO), pp 1–5. IEEE

Kao CC, Wang W, Sun M, Wang C (2018) R-CRNN: Region-based convolutional recurrent neural network for audio event detection. arXiv preprint arXiv:1808.06627

Cakir E, Adavanne S, Parascandolo G, Drossos K, Virtanen T (2017) Convolutional recurrent neural networks for bird audio detection. In: 2017 25th European signal processing conference (EUSIPCO), pp 1744–1748. IEEE

Sharma G (2018) Acoustic signal classification for deforestation monitoring: tree cutting problem. J Comput Sci Syst Biol 11:178–184

Incze A, Jancsó H-B, Szilágyi Z, Farkas A, Sulyok C (2018) Bird sound recognition using a convolutional neural network. In: 2018 IEEE 16th international symposium on intelligent systems and informatics (SISY), pp 000295–000300. IEEE

Chatterjee CC, Mulimani M, Koolagudi SG (2020) Polyphonic sound event detection using transposed convolutional recurrent neural network. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 661–665. IEEE

Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics. 2009 3rd international symposium on empirical software engineering and measurement, pp 367–377. https://doi.org/10.1109/ESEM.2009.5314233

Bansal A, Garg NK (2022) Environmental sound classification: a descriptive review of the literature. Intell Syst Appl 200115

Chan TK, Chin CS (2020) A comprehensive review of polyphonic sound event detection. IEEE Access 8:103339–103373

Mesaros Annamaria, Heittola Toni, Virtanen Tuomas, Plumbley Mark D (2021) Sound event detection: a tutorial. IEEE Signal Process Mag 38(5):67–83

Nogueira AFR, Oliveira HS, Machado JJM, Tavares JMRS (2022) Sound classification and processing of urban environments: a systematic literature review. Sensors 22(22):8608

Shreyas N, Venkatraman M, Malini S, Chandrakala S (2020) Trends of sound event recognition in audio surveillance: a recent review and study. The Cognitive Approach in Cloud Computing and Internet of Things Technologies for Surveillance Tracking Systems 95–106

Abayomi-Alli Olusola O, Damaševičius Robertas, Qazi Atika, Adedoyin-Olowe Mariam, Misra Sanjay (2022) Data augmentation and deep learning methods in sound classification: a systematic review. Electronics 11(22):3795

Mesaros Annamaria, Heittola Toni, Virtanen Tuomas (2016) Metrics for polyphonic sound event detection. Appl Sci 6(6):162

Xiao Y, Khandelwal T, Das RK (2023) FMSG submission for DCASE 2023 challenge task 4 on sound event detection with weak labels and synthetic soundscapes. Proc. DCASE Challenge

Martín-Morató I, Harju M, Ahokas P, Mesaros A (2023) Training sound event detection with soft labels from crowdsourced annotations. In: ICASSP 2023–2023 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1–5. IEEE

Cai X, Gan Y, Wu M, Wu J (2023) Weak supervised sound event detection based on Puzzle CAM. IEEE Access

Xu L, Wang L, Bi S, Liu H, Wang J (2023) Semi-Supervised sound event detection with pre-trained model. In: ICASSP 2023–2023 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1–5. IEEE

Wang Qing, Jun Du, Hua-Xin Wu, Pan Jia, Ma Feng, Lee Chin-Hui (2023) A four-stage data augmentation approach to ResNet-Conformer based acoustic modeling for sound event localization and detection. IEEE/ACM Trans Audio Speech Lang Process 31:1251–1264

Download references

Not applicable.

Author information

Authors and affiliations.

School of Computer Science and Artificial Intelligence, SR University, Warangal, Telangana, India

Sallauddin Mohmmad

Research Scholar, JNTU, Hyderabad, India

Department of Information Technology, JNTUH College of Engineering, Nachupally, Kondagattu, Jagtial, Telangana, India

Suresh Kumar Sanampudi

You can also search for this author in PubMed   Google Scholar

Contributions

All authors equally contributed and approved the final manuscript.

Corresponding author

Correspondence to Sallauddin Mohmmad .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflicts.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Mohmmad, S., Sanampudi, S.K. Exploring current research trends in sound event detection: a systematic literature review. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18740-9

Download citation

Received : 29 November 2022

Revised : 16 January 2024

Accepted : 24 February 2024

Published : 29 April 2024

DOI : https://doi.org/10.1007/s11042-024-18740-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sound event detection
• Deep learning
• Machine learning
• Systematic literature review
• Feature extraction
• Find a journal
• Publish with us
• Track your research

REVIEW article

The application of multi-criteria decision analysis in evaluating the value of drug-oriented intervention: a literature review.

• 1 Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
• 2 China Academy of Chinese Medical Sciences, Beijing, China
• 3 School of Public Health, Nantong University, Nantong, Jiangsu, China

Objectives: Multi-Criteria Decision Analysis (MCDA) has gained increasing attention in supporting drug risk-benefit assessment, pricing and reimbursement, as well as optimization of clinical interventions. The objective of this study was to systematically collect and categorize evaluation criteria and techniques of weighting and scoring of MCDA for drug value assessment.

Methods: A systematic review of the literature was conducted across seven databases to identify articles utilizing the MCDA frameworks for the evaluation of drug value. Evaluation criteria mentioned in the included studies were extracted and assigned to 5 dimensions including clinical, economic, innovative, societal and humanistic value. A descriptive statistical analysis was performed on the identified drug value evaluation criteria, as well as the weighting and scoring techniques employed. The more a criterion or technique were mentioned in articles, the more important we consider it.

Results: Out of the 82 articles included, 111 unique criteria were identified to evaluate the value of drug. Among the 56 unique criteria (448 times) used to measure clinical value, the most frequently mentioned were “comparative safety/tolerability” (58 times), “comparative effectiveness/efficacy” (56 times), “comparative patient-perceived health/patient reported outcomes” (37 times), “disease severity” (34 times), and “unmet needs” (25 times). Regarding economic value measurement, out of the 20 unique criteria (124 times), the most frequently utilized criteria were “cost of intervention” (17 times), “comparative other medical costs” (16 times), and “comparative non-medical costs” (18 times). Out of the 10 criteria (18 times) for assessing innovative value, “a novel pharmacological mechanism” was the most frequently mentioned criterion (5 times). Among the 22 criteria (73 times) used to measure societal value, “system capacity and appropriate use of intervention” was the most frequently cited criterion (14 times). Out of the 3 criteria (15 times) utilized to measure humanistic value, “political/historical/cultural context” was the most frequently mentioned criterion (9 times). Furthermore, 11 scoring and 11 weighting techniques were found from various MCDA frameworks. “Swing weighting” and “a direct rating scale” were the most frequently used techniques in included articles.

Conclusion: This study comprehensively presented the current evaluation dimensions, criteria, and techniques for scoring and weighting in drug-oriented MCDA articles. By highlighting the frequently cited evaluation criteria and techniques for scoring and weighting, this analysis will provide a foundation to reasonably select appropriate evaluation criteria and technique in constructing the MCDA framework that aligns with research objectives.

1 Background

In recent decades, several “value frameworks” have been applied to evaluate the drug value in the health technology assessment (HTA) in the world ( de Andrés-Nogales et al., 2020 ; de Andrés-Nogales et al., 2021 ; Hummel et al., 2012 ). These frameworks are employed to facilitate dynamic drug supervision, promote clinical rational drug use, adjust coverage and reimbursement decisions, and support drug marketing ( Yu et al., 2023 ). In China, HTA assessment evidence based on comprehensive drug evaluations has also been used to update the national reimbursement drug list since 2017 ( Chen et al., 2023 ). Despite differing evaluation goals, the introduction of these frameworks has expanded a broader concept of drug value. In the field of drug value evaluation, there is a growing trend towards incorporating a wide range of dimensions, including clinical, economic, societal and humanistic values. As proposed by Garrison LP Jr, the value of drugs cover life years gained, improvement in quality of life, cost savings within health system, productivity, cost savings outside health system, scientific spillovers, insurance value, real option value, value of hope and reduction in uncertainty ( Garrison et al., 2017 ). Moreover, the criteria utilized to assess each dimension of value are continuously evolving. For instance, the criteria for evaluating drug efficacy have advanced from merely focusing on primary or endpoint outcomes to a comprehensive assessment of efficacy including therapeutic benefits type, multiple outcomes (primary outcome, secondary outcome, endpoint outcome, patients report outcome) and types of clinical research ( Migliore et al., 2015 ; Vermersch et al., 2019 ; Institute for Quality and Efficiency in Healthcare, 2022). In situations requiring intricate decision-making, researchers tend to prioritize a holistic evaluation of drug value ( Goetghebeur et al., 2012 ). The substantial and scattered evaluation criteria of drugs value posed inevitable challenges for the decision-making process. Collecting and categorizing these value criteria are increasingly essential to aid value-based decision-making.

Multi-Criteria Decision Analysis (MCDA), as a valuable approach for integrating diverse and complex trade-offs among different stakeholders, can provide comprehensive, transparent, and structured evaluations of drug value ( Mt-Isa et al., 2014 ). Moreover, this approach has gained widespread use in the healthcare fields, particularly in the assessment of interventions for authorization, prioritization for coverage or reimbursement, selection for clinicians and patients, and the allocation of research funds ( Marsh et al., 2014 ; European Medicines Agency, 2012 ). It is noteworthy that some prominent organizations, such as the European Medicines Agency, the Food and Drug Administration, and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), have acknowledged and endorsed MCDA as a valuable tool for assessing the benefits and safety of medicinal products ( European Medicines Agency, 2012 ; Marsh et al., 2016 ; Angelis and Kanavos, 2021 ). Due to different evaluation purposes, various MCDA frameworks have been developed, including EVDEM (Evidence and Value: Impact on Decision Making) framework, Benefit-Risk framework, AVF (the Advance Value Framework), MAUT (Multi-attribute Utility Analysis), and others ( Marsh et al., 2016 ). Each MCDA framework exhibits unique characteristic in selecting criteria, scoring and weighting techniques. For example, EVIDEM framework, introduced by Goetghebeur MM in 2012, employs a scoring scale ranging from 0–3 for 15 criteria (e.g., disease severity, cost-effectiveness, etc.) ( Goetghebeur et al., 2012 ). Whereas, the benefit-risk MCDA framework utilizes a technique of “0–100 value scales” to determine the criteria score ( Mendoza-Sanchez et al., 2018 ).

The objective of this study was to systematically collect and analyze drug value evaluation dimensions, criteria and techniques of weighting and scoring used in various MCDA frameworks based on the existing literature. This analysis aims to provide a foundation for the construction of a drug evaluation framework that could support regulatory approval, drug pricing, prescription decisions making, medical insurance reimbursement, and other related purposes.

2.1 Search strategies and articles selection

The related Chinese and English articles, published before 30 June 2022, had been searched at seven databases including China National Knowledge Infrastructure (CNKI), WanFang Database, the Chinese Scientific Journals Full-text Database (VIP), the Chinese Biomedical Literature Database (CBM), PubMed, Cochrane Library and Embase. We used a search strategy combining MeSH terms with free words. The search terms were composed of “MCDA” (or “multi-criteria decision analysis” or “multicriteria decision analysis” or “multicriteria decision aiding” “multi-attribute utility” or “MAU” or “MAUT” or “MACBETH” or “Evidence and Value: Impact on Decision Making” or “EVIDEM” or “Advance Value Framework”)and “intervention” (or “treatment” or “drug” or “drug assessment” or “medicine” or “medication” or “pharmacy” or “prescription”). Additional literature through other sources were also identified, such as references of relevant reviews. The detailed search strategies were described in Supplementary File S1 .

Endnote X9.1 software was utilized for managing all retrieved articles. Following the removal of duplicates, two researchers (PS and KZ) independently screened articles based on predetermined inclusion and exclusion criteria. The research was limited to the application of MCDA to assess value of drug-oriented intervention and excluded the following types of studies: not in the field of medicine and health; no specific evaluation criteria; methodological or theoretical studies of MCDA; editorials; conference article and bibliographies of relevant articles.

2.2 Data extraction

The essential data from eligible studies were extracted using Microsoft Excel 2019 software. Two researchers (PS and KZ) independently extracted basic information using a specially designed form including publication data (publication date, title, and authors’ names, country, journal), details of MCDA (MCDA method subtypes, indication, drugs, evaluation dimensions, criteria, stakeholders, the techniques of scoring and weighting, the method of aggregation and uncertainty analysis). Any disagreement during literature screening and data extraction was resolved by consensus or consulting a third researcher (HD).

2.3 Descriptive statistics

The number of unique criteria and their frequencies of citation (times) in the included studies were calculated. Due to the diverse terminology used to describe similar evaluation criteria, internal deliberations were held to identify unique criteria. For example, terms referring to the same concept “comparative effectiveness/efficacy” (e.g., “overall survival period” and “progression-free survival period”) were consolidated into one criterion. Classification standard of qualitative criteria or quantitative criteria was mainly guided by the structure of the EVIDEM framework, which includes an adaptable set of qualitative criteria or quantitative criteria. Then, these unique evaluation criteria were assigned to 5 dimensions including clinical, economic, innovative, societal and humanistic value. The techniques for weighting and scoring derived from 82 articles were also counted. The greater the quantity of unique criteria, the more comprehensive the assessment perspective of these value dimensions. The more a criterion or technique were mentioned in articles, the more important we consider it.

3.1 Literature screening

Among the 4659 articles retrieved, 82 (69 in English and 13 in Chinese) articles were included in this analysis, as shown in Figure 1 . There were 24 articles on EVDEM framework, 25 on Benefit-Risk Framework, 3 on AVF, 5 on MAUT, and 20 designed for specific research purposes. The first study on MCDA was published in 1991 ( Schumacher, 1991 ), after which the number of articles showed an overall increasing trend, reaching the highest in 2016 and 2018, with 11 articles each. Spain had the highest number of articles on this subject, with a total of 17 articles. The number of criteria ranged from 4 to 38 ( Sidi and Harel, 2020 ; Byun et al., 2016 ; Al-Badriyeh et al., 2016 ; Tervonen et al., 2015 ; Hsu et al., 2015 ; Erjaee et al., 2012 ; Bettinger et al., 2007 ; Pérez Encinas et al., 1998 ). Additional information regarding the literature included can be accessed in Supplementary File S2 .

Figure 1 . Prisma flow diagram.

3.2 Evaluation dimensions and criteria

Drug value evaluation covered 5 dimensions (clinical, economic, societal, humanistic, and innovative value) and 111 unique criteria (678 times) in the 82 included articles. These criteria could be divided into quantitative criteria and qualitative criteria, as shown in Figure 2 .

Figure 2 . Research diagram of each dimension criteria.

3.2.1 Clinical value

This study identified 56 unique clinical value criteria (448 times), with the criterion of “comfort in drug consumption” being the sole qualitative criterion, as shown in Figure 3 . These criteria were further categorized into five groups: disease-related criteria, evidence-related criteria, effectiveness/efficacy-related criteria, safety-related criteria and patient preferences-related criteria.

Figure 3 . The frequency plot of clinical value criteria occurrence. Note: Clinical value criteria were further categorized into five groups: disease-related criteria (A) , evidence-related criteria (B) , effectiveness/efficacy-related criteria (C) , safety-related criteria (D) and patient preferences-related criteria (E) . The blue bars represent quantitative criteria, while orange bars represent qualitative criteria. QTI stands for “quantitative” and QLI stands for “qualitative.”

3.2.1.1 Disease-related criteria

In the analysis of 14 unique disease-related criteria (111 times) ( Figure 3A ), the most frequently mentioned were “disease severity” (34 times), “unmet needs” (25 times), and “size of affected population” (23 times). It is worth noting that the criterion of “the size of the population being affected by the disease” was not recommended by AVF due to ethical concerns related to evaluating the clinical value of drug based on high or low disease prevalence ( Angelis and Kanavos, 2017 ). “Disease rarity” could be used as a supplementary criterion to assess the degree of rarity of the disease.

3.2.1.2 Evidence-related criteria

Within the 18 unique evidence-related criteria (63 times) ( Figure 3B ), the two criteria mentioned over 10 times were “quality of evidence” (20 times) and “expert consensus/clinical practice guidelines” (18 times).

3.2.1.3 Effectiveness/efficacy-related criteria

Among the 12 unique effectiveness/efficacy-related criteria (161 times) ( Figure 3C ), the five criteria that were cited more than 10 times included “comparative effectiveness/efficacy” (56 times), “comparative patient-perceived health/patient reported outcomes” (37 times), “benefits” (24 times), “type of therapeutic benefit” (20 times), and “type of preventive benefit” (15 times).

3.2.1.4 Safety-related criteria

Within the 5 unique safety-related criteria (87 times) ( Figure 3D ), the two criteria mentioned over 10 times were “comparative safety/tolerability” (58 times) and “risks” (22 times).

3.2.1.5 Patient preference-related criteria

Among the 7 unique criteria related to patient preferences (26 times) ( Figure 3E ), the top three most frequently mentioned were “applicability” (9 times), “patient convenience” (6 times), and “patient adherence” (4 times).

3.2.2 Economic value

Among the 16 quantitative criteria (111 times) and 4 qualitative criteria (13 times) which were employed to assess the economic value, the most frequently utilized top three criteria were “cost of intervention” (17 times), “comparative other medical costs” (16 times), and “comparative non-medical costs” (18 times), as shown in Figure 4 . In certain instances, comprehensive economic indicators, such as “budget impact on health plan” (12 times) and “cost-effectiveness analysis” (12 times) were also employed for economic value assessment. Furthermore, the AVF frameworks employed medical costs impact, rather than intervention cost, to assess the socioeconomic impact. Additionally, the qualitative criteria of “opportunity costs and affordability” (8 times), “opportunity cost-efficiency” (3 times), “healthcare system capacity to assume the technology cost” (1 time), “cost-opportunity associated to healthcare system intervention” (1 time) were also utilized to assess the economic value.

Figure 4 . The frequency plot of economic value criteria occurrence. Note: The blue bars represent quantitative criteria, while orange bars represent qualitative criteria. QTI stands for “quantitative” and QLI stands for “qualitative.”

3.2.3 Innovative value

There were 10 quantitative criteria (18 times) used to assess the innovation of drugs, as shown in Figure 5 . Among these criteria, the most frequent occurrence was “a novel pharmacological mechanism” (5 times). The criteria of “spill-over effect” and “innovation of patient convenience” were mentioned 3 times each.

Figure 5 . The frequency plot of innovative value criteria occurrence. Note: The blue bars represent quantitative criteria QTI stands for “quantitative.”

3.2.4 Societal value

10 quantitative (52 times) and 12 qualitative criteria (21 times) were used to measure societal value, as shown in Figure 6 . Among these criteria, there were four criteria mentioned over 10 times, namely, “system capacity and appropriate use of intervention” (14 times), “common goal and specific interests” (10 times), “mandate and scope of the healthcare system” (10 times), and “population priorities and access and fairness” (10 times). It is also worth noting that all of these four criteria were qualitative.

Figure 6 . The frequency plot of societal value criteria occurrence. Note: The blue bars represent quantitative criteria, while orange bars represent qualitative criteria. QTI stands for “quantitative” and QLI stands for “qualitative.”

3.2.5 Humanistic value

Out of 3 qualitative criteria (15 times) used to measure the humanistic value of drugs, “political/historical/cultural context” was the most frequently mentioned criterion (9 times), followed by “environmental impact” (5 times) and “environmental sustainability” (1 time), as shown in Figure 7 .

Figure 7 . The frequency plot of humanistic value criteria occurrence. Note: The orange bars represent qualitative criteria. QLI stands for “qualitative.”

3.3 The techniques for weighting

11 unique techniques for weighting were identified, including “5-point weighting scale,” “10-point scale,” “Hierarchical Point Allocation (HPA),” “Visual Analogue Scale (VAS),” “Swing Weighting,” “best-worst scale,” “Discrete Choice Experiment (DCE),” “Analytic Hierarchy Process (AHP),” “determined subjectively,” “a novel rank-based weighting methodology” and “Adaptive Conjoint Analysis (ACA).” “Swing Weighting” was the most commonly utilized technique among articles, as illustrated in Figure 8 .

Figure 8 . The techniques for weighting.

Moreover, the application of weighting techniques exhibited variability among various MCDA frameworks. Within benefit-risk MCDA studies, 81.82% (18/22) adopted “Swing Weighting.” All the AVF Framework studies (3/3) utilized “Swing Weighting” to determine the criteria weights. In MUAT studies, researchers predominantly determined criteria weights through a subjective process, combining the level of criteria and evidence from the literature (3/5). Other MCDA studies with specific research objectives utilized AHP (10/22) to weight criteria.

3.4 The technique for scoring

11 techniques were utilized for estimating scoring, including “0–1 preference value scales,” “3-point scale,” “4-point scale,” “5-point scale,” “a direct rating scale,” “7-point scale,” “11-point cardinal scoring scale,” “0–100 value scales,” “best-worst scale,” “lower-higher reference levels,” and “grade scoring.” “A direct rating scale” was the most commonly employed, being utilized in 18 of the articles analyzed, as depicted in Figure 9 . This technique primarily measured the criteria scoring based on the “performance matrix” of each intervention through expert meetings. Alternatively, researchers scored directly based on the literature.

Figure 9 . The technique for scoring.

In the EVIDEM framework studies, 81.82% (18/22) employed “a direct rating scale,” while the others (4/22) used a “4-point scale.” Among the benefit-risk MCDA framework studies, 81.25% (13/16) utilized “0–100 value scales” to measure scores of criteria. The AVF framework adopted “lower-higher reference levels” to establish the range of criteria scores (3/3). In MAUT studies, criteria scoring was measured through grade scoring (3/4) and a 5-point scale (1/4). The grade levels for each criterion were predetermined and subsequently utilized by researchers to assign corresponding scores after conducting a thorough review of the literature. Some MCDA studies with specific research purposes adopted more flexible methods of scoring, such as 0–100 value scales (5/13), grade scoring (2/13), 5-point scales (2/13), or other techniques (4/13).

4 Discussion

Our analysis of 82 literature revealed that the evaluation of drug values covered 5 dimensions and 111 criteria (678 times). 94 (94/111, 84.68%) criteria were quantitative. Of these, 56 of the 94 criteria (59.57%) measured clinical value. Furthermore, we identified 11 scoring and 11 weighting techniques used in various MCDA frameworks. “Swing weighting” and “a direct rating scale” were the most frequently techniques used in MCDA literature. To the best of our knowledge, our study was the first attempt to categorize drug-oriented MCDA criteria based on their clinical, economic, societal, innovative and humanistic values.

The evaluation of drug value is a multifaceted process that requires thorough consideration of various dimensions, such as medical, ethical, economic, and social equity criteria ( Garrison et al., 2017 ; Lakdawalla et al., 2018 ). Therefore, it is imperative to categorize the criteria for drug evaluation based on different value dimensions. However, previous literature reviews on MCDAs in healthcare have exhibited certain limitations, including oversimplified categorization of criteria. Lalla Aida Guindo classified the criteria based on the evaluation dimensions of EVIDEM framework ( Guindo et al., 2012 ), while Tamas Zelei sorted out the decision criteria based on cost and outcome dimensions ( Zelei et al., 2021 ). Additionally, a wider range of interventions was encompassed, such as pharmaceuticals, public health interventions (e.g., smoking cessation, obesity), screening, surgical strategies, devices, ( Marsh et al., 2014 ). Further detailed research is required to explore the unique characteristics of different interventions in value evaluation. The National Institute for Health and Care Excellence (NICE) utilizes topic classification to develop health technology assessment, encompassing devices, diagnostics, interventional procedures, medicines, combination or integrated technologies, digital technologies and other technologies ( National Institute for Health and Care Excellence, 2022 ). Our research focused on pharmaceuticals and categorized for drug-oriented MCDA criteria according to 5 dimensions (clinical, economic, societal, innovative and humanistic value dimensions). We found that clinical value criteria were commonly utilized, followed by economic value criteria, which aligns with previous studies ( Park et al., 2012 ; Kwon et al., 2017 ; Tanios et al., 2013 ; Golan et al., 2011 ; Baji et al., 2016 ). Clinical value is the core factor in drug development and use, which can reflect the extent of a drug’s ability to meet clinical needs and provide clinical benefits for its target population ( Liu et al., 2021 ).

In relation to clinical value, the regulatory and reimbursement processes for drugs prioritize the assessment of the effectiveness-safety balance, also referred to as efficacy-tolerability or benefit-risk balance ( Dang et al., 2020 ). For example, the Food and Drug Administration (FDA) conducted a thorough benefit-risk evaluation of new drugs, taking into account the substantial evidence of safety and effectiveness provided by the sponsor ( Food and Drug Administration, 2021 ). Similarly, the Institute for Quality and Efficiency in Healthcare (IQWiG) in Germany made a trade-off between benefit and harm aspect in the early benefit assessment of new drugs, and determined the extent of added benefit (minor, considerable, and major treatment effects) ( Institute for Quality and Efficiency in Health Care, 2022 ). Our study also confirmed that “comparative safety/tolerability” was the most frequently used clinical value criterion, followed by “comparative effectiveness/efficacy.” Although there are subtle differences between efficacy and effectiveness, as well as safety and tolerability ( Berger et al., 2012 ), we aggregated “effectiveness” and “efficacy” criteria into a single “comparative effectiveness/efficacy” criterion considering greater operability in practice, as well as “comparative safety/tolerability.”

With regard to the economic value, we found the most frequently utilized criteria were “cost of intervention,” “comparative other medical costs” and “comparative non-medical costs.” However, it is constantly discussing whether to incorporate cost (and related criteria such as cost-effectiveness, budget impact) as a criterion in MCDA ( Golan et al., 2011 ; Marsh et al., 2018a ; Hansen and Ombler, 2008 ). Hansen argued “cost is not a value criterion, but a measure of what has to be given up to achieve the value criteria” ( Hansen and Devlin, 2019 ). Including cost related criteria in the calculation of MCDA values may violate the principle of “structurally independent” of the multidimensionality of value and damage opportunity cost in the allocation of limited resources ( Hansen and Devlin, 2019 ; Wilson et al., 2022 ). Wilson compared the extent of consistency in ranking importance of the four criteria of “treatment effectiveness,” “cost of the intervention” “risk of serious harms,” and “risk of mild-to-moderate harms” based on MCDA and cost-effectiveness analysis models and found “cost of the intervention” should be excluded in MCDA for prioritizing intervention setting ( Wilson et al., 2022 ). Golan and Angelis also excluded the cost of the treatments in the MCDA value metric. Golan used MCDA to aggregate health-related benefits of multiple dimensions and calculated the final value score by formula “cost/health-related benefits value” ( Golan and Hansen, 2012 ). Angelis used costs per unit of MCDA value to calculate overall weighted preference value scores (i.e., the final value scores) ( Angelis et al., 2020 ). However, cost related criteria were incorporated in EVIDEM frameworks ( Goetghebeur et al., 2012 ). A set of criteria including “direct medical costs,” “direct non-medical costs” and “indirect costs” were used as the “modulators” of core model to measure economic consequences of intervention ( Casellas Caro et al., 2022 ). With the soaring drug costs and limited financial resources, we think cost of the interventions should be considered in decision making, especially in prioritizing drugs for reimbursement.

Medical innovation is a complex concept lacking a definitive consensus, which mainly related to the realm of “therapeutic innovation” ( Tanios et al., 2013 ). Therapeutic innovation is commonly associated with generating improved health outcomes that were previously unattainable and addressing the unmet medical need ( Morgan et al., 2008 ). Nevertheless, we found the AVF frameworks offer a distinct viewpoint for quantitatively assessing the value of innovation, utilizing criteria such as the mechanism of action of the medicine, its spill-over effects, and its utility for patients (e.g., convenience) ( Angelis and Phillips, 2021 ). The criterion of “mechanism of action” can be evaluated by Anatomical Therapeutic Chemical (ATC) Classification System of World Health Organization (WHO). The degree of innovation “spill-over effects,” as argued by Angelis, can be gauged by the number of new indications being explored for the medicine at each stage of clinical development (e.g., Phase I, Phase II, Phase III, Marketing Authorization phase). “Patient usefulness” could be perceived as ease and convenience, which were related to mode of administration, dosing schedule, medication restrictions, and product-specific designs.

Once the criteria are agreed upon in MCDA frameworks, the selection of the appropriate techniques of scoring and weighting are subsequently considered by the researchers. A published study in healthcare found AHP is the most frequently applied technique for weighting, followed by “Swing Weighting” ( Camps et al., 2019 ). However, our research showed that “Swing Weighting” and “a direct rating scale” were the most frequently techniques used in the field of pharmaceuticals assessment. “Swing Weighting” could obtain more discriminative weights by obtaining the expected performance floating range of each criterion in advance. It should be noted that the weight of each criterion is typically determined by a panel of experts, independent of the specific drug ( Hansen and Devlin, 2019 ; Camps et al., 2020 ; Guarga et al., 2019 ; Jiménez et al., 2018 ). Due to the weights and scores were closely related to the panel of experts participating in MCDA, the process of weight and score determination were subjective to some extent. So, sensitivity analysis was often used to assess the robustness of MCDA results by re-testing using alternative weighting and scoring techniques. Furthermore, as Marsh argued, it is important to incorporate well-trained experts and working groups to perform the MCDA ( Marsh et al., 2018b ).

Some limitations were worth mentioning. Firstly, the identified literature was only searched in seven publicly available databases, and supplemented with relevant review references and expert consultations. Grey articles were not acquired in this study, which may lead to a publication bias to a certain extent. Secondly, due to language limitations, only Chinese and English literature were identified. Excluding articles published in other languages may have had an impact on our results. Thirdly, this study did not recognize the critical dimensions and criteria targeted for different stages of life cycles in drug value evaluation. In future, we will continue to carry out the research on drug value assessment in different stages of life cycles.

5 Conclusion

This study comprehensively presented the current evaluation dimensions, criteria, and techniques for scoring and weighting in drug-oriented MCDA articles. By highlighting the frequently cited evaluation criteria and techniques for scoring and weighting, this analysis can serve as a resource to reasonably select these evaluation details in constructing the appropriate MCDA framework. The ultimate objective is to provide a solid foundation for the construction of a drug evaluation framework to advancing the structured decision in drug management. In future, as research on MCDA drug value evaluation deepens, more attention should be paid to the assessment criteria and techniques of weighting and scoring for different types of drugs. For instance, specific criteria should be considered when assessing traditional Chinese medicine in China.

Author contributions

PS, KZ, JX, HX, JL, ZW, QL, YY, and HD did the conception and design of this study. YY and HD provided the guide of the methodology. PS, KZ, HX, JL, and QL collected related data. PS and KZ wrote original draft and JX, HX, JL, ZW, QL, YY, and HD revised the paper. All authors contributed to the article and approved the submitted version.

This work was funded by the Innovation Fund of China Academy of Chinese Medical Sciences (CI 2021A04707), the National Nature Science Foundation of China (Grant No. 82105052), and China National Institute of Healthcare Security, Capital Medical University (No. YB 2022B10).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2024.1245825/full#supplementary-material

Abbreviations

ACA, adaptive conjoint analysis; AHP, analytic hierarchy process; ATC, anatomical therapeutic chemical; AVF, advance value framework; DCE, discrete choice experiment; EVDEM, evidence and value: impact on decision making; FDA, food and drug administration; HPA, hierarchical point allocation; HTA, health technology assessment; IQWiG, institute for quality and efficiency in healthcare; ISPOR, international society for pharmacoeconomics and outcomes research; MAUT, multi-attribute utility analysis; MCDA, multi-criteria decision analysis; NICE, national institute for health and care excellence; VAS, visual analogue scale; WHO, world health organization.

Al-Badriyeh, D., Alabbadi, I., Fahey, M., Al-Khal, A., and Zaidan, M. (2016). Multi-indication pharmacotherapeutic multicriteria decision analytic model for the comparative formulary inclusion of proton pump inhibitors in Qatar. Clin. Ther. 38 (5), 1158–1173. doi:10.1016/j.clinthera.2016.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Angelis, A., and Kanavos, P. (2017). Multiple criteria decision analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: the advance value framework. Soc. Sci. Med. 188, 137–156. doi:10.1016/j.socscimed.2017.06.024

Angelis, A., Linch, M., Montibeller, G., Molina-Lopez, T., Zawada, A., Orzel, K., et al. (2020). Multiple criteria decision analysis for HTA across four EU member states: piloting the advance value framework. Soc. Sci. Med. 246, 112595. doi:10.1016/j.socscimed.2019.112595

Angelis, A., and Phillips, L. D. (2021). Advancing structured decision-making in drug regulation at the FDA and EMA. Br. J. Clin. Pharmacol. 87 (2), 395–405. doi:10.1111/bcp.14425

Baji, P., García-Goñi, M., Gulácsi, L., Mentzakis, E., and Paolucci, F. (2016). Comparative analysis of decision maker preferences for equity/efficiency attributes in reimbursement decisions in three European countries. Eur. J. Health Econ. 17 (7), 791–799. doi:10.1007/s10198-015-0721-x

Berger, M. L., Dreyer, N., Anderson, F., Towse, A., Sedrakyan, A., and Normand, S. L. (2012). Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health 15 (2), 217–230. doi:10.1016/j.jval.2011.12.010

Bettinger, T. L., Shuler, G., Jones, D. R., and Wilson, J. P. (2007). Schizophrenia: multi-attribute utility theory approach to selection of atypical antipsychotics. Ann. Pharmacother. 41 (2), 201–207. doi:10.1345/aph.1G607

Byun, J. H., Kwon, S. H., Ha, J. H., and Lee, E. K. (2016). A benefit-risk assessment model for statins using multicriteria decision analysis based on a discrete choice experiment in Korean patients. Ther. Clin. Risk Manag. 12, 965–974. doi:10.2147/TCRM.S100438

Camps, C., Badia, X., García-Campelo, R., García-Foncillas, J., López, R., Massuti, B., et al. (2020). Development of a multicriteria decision analysis framework for evaluating and positioning oncologic treatments in clinical practice. JCO. Oncol. Pract. 16 (3), e298–e305. doi:10.1200/JOP.19.00487

Casellas Caro, M., Hidalgo, M. J. C., García-Erce, J. A., Baquero Úbeda, J. L., Torras Boatella, M. G., Gredilla Díaz, E., et al. (2022). Applying reflective multicriteria decision analysis to understand the value of therapeutic alternatives in the management of gestational and peripartum anaemia in Spain. BMC Pregnancy Childbirth 22 (1), 157. doi:10.1186/s12884-022-04481-w

Chen, Y., Zhao, K., Liu, G., and Chen, W. (2023). Health technology assessment to inform decision making in China: progress, challenges, and sustainability. BMJ 381, e068910. doi:10.1136/bmj-2021-068910

Dang, H. X., Zhang, L., Liu, J., Wang, Z., and Shen, C. T. (2020). Introduction and analysis of FDA's benefit-risk drug value assessment framework. Drug Eval. Res. 43 (04), 665–669. doi:10.7501/j.issn.1674-6376.2020.04.014

CrossRef Full Text | Google Scholar

de Andrés-Nogales, F., Casado, M. Á., Trillo, J. L., Ruiz-Moreno, J. M., Martínez-Sesmero, J. M., Peralta, G., et al. (2020). A multiple stakeholder multicriteria decision analysis in diabetic macular edema management: the MULTIDEX-EMD study. Open 4 (4), 615–624. doi:10.1007/s41669-020-00201-2

de Andrés-Nogales, F., Cruz, E., Calleja, M. Á., Delgado, O., Gorgas, M. Q., Espín, J., et al. (2021). A multi-stakeholder multicriteria decision analysis for the reimbursement of orphan drugs (FinMHU-MCDA study). Orphanet. J. Rare Dis. 16 (1), 186. doi:10.1186/s13023-021-01809-1

Erjaee, A., Bagherpour, M., Razeghi, S., Dehghani, S. M., Imanieh, M. H., and Haghighat, M. (2012). A multi-criteria decision making model for treatment of Helicobacter pylori infection in children. Hong kong J. Paediatrics 17, 237–242. doi:10.1089/bfm.2012.9981

European Medicines Agency (2012). Benefit-risk methodology project; work package 4 report: benefit-risk tools and processes. Available at: https://www.ema.europa.eu/documents/report/benefit-risk-methodology-project-workpackage-4-report-benefit-risk-toolsprocesses_en.pdf .

Google Scholar

Food and Drug Administration (2021). Benefit-risk assessment for new drug and biological products. Available at: https://www.fda.gov/media/152544/download .

Garrison, L. P., Kamal-Bahl, S., and Towse, A. (2017). Toward a broader concept of value: identifying and defining elements for an expanded cost-effectiveness analysis. Value Health 20 (2), 213–216. doi:10.1016/j.jval.2016.12.005

Goetghebeur, M. M., Wagner, M., Khoury, H., Levitt, R. J., Erickson, L. J., and Rindress, D. (2012). Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA): applying the EVIDEM framework to medicines appraisal. Med. Decis. Mak. 32 (2), 376–388. doi:10.1177/0272989X11416870

Golan, O., and Hansen, P. (2012). Which health technologies should be funded? A prioritization framework based explicitly on value for money. Isr. J. Health Policy Res. 1 (1), 44. doi:10.1186/2045-4015-1-44

Golan, O., Hansen, P., Kaplan, G., and Tal, O. (2011). Health technology prioritization: which criteria for prioritizing new technologies and what are their relative weights? Health Policy 102 (2-3), 126–135. doi:10.1016/j.healthpol.2010.10.012

Guarga, L., Badia, X., Obach, M., Fontanet, M., Prat, A., Vallano, A., et al. (2019). Implementing reflective multicriteria decision analysis (MCDA) to assess orphan drugs value in the Catalan Health Service (CatSalut). Orphanet. J. Rare Dis. 14 (1), 157. doi:10.1186/s13023-019-1121-6

Guindo, L. A., Wagner, M., Baltussen, R., Rindress, D., van Til, J., Kind, P., et al. (2012). From efficacy to equity: literature review of decision criteria for resource allocation and healthcare decisionmaking. Resour. Alloc. 10 (1), 9. doi:10.1186/1478-7547-10-9

Hansen, P., and Devlin, N. (2019). Multi-criteria decision analysis (MCDA) in healthcare decision-making . Oxford, United Kingdom: Oxford University Press .

Hansen, P., and Ombler, F. (2008). A new method for scoring additive multi-attribute value models using pairwise rankings of alternatives. J. Multi. Criteria Decis. Anal. 15 (3-4), 87–107. doi:10.1002/mcda.428

Hsu, J. C., Hsieh, C. Y., Yang, Y. H., and Lu, C. Y. (2015). Net clinical benefit of oral anticoagulants: a multiple criteria decision analysis. PLoS. One. 10 (4), e0124806. doi:10.1371/journal.pone.0124806

Hummel, M. J., Volz, F., van Manen, J. G., Danner, M., Dintsios, C. M., Ijzerman, M. J., et al. (2012). Using the analytic hierarchy process to elicit patient preferences: prioritizing multiple outcome measures of antidepressant drug treatment. Patient 5 (4), 225–237. doi:10.1007/BF03262495

Institute for Quality and Efficiency in Health Care (IQWiG) (2022). General methods version 6.1. Available at: https://www.iqwig.de/methoden/general-methods_version-6-1.pdf .

Jiménez, A., Ais, A., Beaudet, A., and Gil, A. (2018). Determining the value contribution of selexipag for the treatment of pulmonary arterial hypertension (PAH) in Spain using reflective multi-criteria decision analysis (MCDA). Orphanet. J. Rare Dis. 13 (1), 220. doi:10.1186/s13023-018-0966-4

Kwon, S. H., Park, S. K., Byun, J. H., and Lee, E. K. (2017). Eliciting societal preferences of reimbursement decision criteria for anticancer drugs in South Korea. Expert Rev. Pharmacoecon Outcomes Res. 17 (4), 411–419. doi:10.1080/14737167.2017.1277144

Lakdawalla, D. N., Doshi, J. A., Garrison, L. P., Phelps, C. E., Basu, A., and Danzon, P. M. (2018). Defining elements of value in health care-A health economics approach: an ISPOR special task force report [3]. Value Health 21 (2), 131–139. doi:10.1016/j.jval.2017.12.007

Liu, B. L., Yang, N., Wu, Y., and Hou, C. C. (2021). Consideration of clinical value of drugs in clinical trials of new drugs. China Food & Drug Adm. Mag. (04), 24–31. doi:10.3969/j.issn.1673-5390.2021.04.003

Marsh, K., Ijzerman, M., Thokala, P., Baltussen, R., Boysen, M., Kaló, Z., et al. (2016). Multiple criteria decision analysis for health care decision making--emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health 19 (2), 125–137. doi:10.1016/j.jval.2015.12.016

Marsh, K., Lanitis, T., Neasham, D., Orfanos, P., and Caro, J. (2014). Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics 32 (4), 345–365. doi:10.1007/s40273-014-0135-0

Marsh, K., Thokala, P., Youngkong, S., and Chalkidou, K. (2018a). Incorporating MCDA into HTA: challenges and potential solutions, with a focus on lower income settings. Resour. Alloc. 16 (Suppl. 1), 43. doi:10.1186/s12962-018-0125-8

Marsh, K. D., Sculpher, M., Caro, J. J., and Tervonen, T. (2018b). The use of MCDA in HTA: great potential, but more effort needed. Value Health 21 (4), 394–397. doi:10.1016/j.jval.2017.10.001

Mendoza-Sanchez, J., Silva, F., Rangel, L., Jaramillo, L., Mendoza, L., Garzon, J., et al. (2018). Benefit, risk and cost of new oral anticoagulants and warfarin in atrial fibrillation; A multi-criteria decision analysis. PLoS. One. 13 (5), e0196361. doi:10.1371/journal.pone.0196361

Migliore, A., Integlia, D., Bizzi, E., and Piaggio, T. (2015). Is it the time to rethink clinical decision-making strategies? From a single clinical outcome evaluation to a Clinical Multi-criteria Decision Assessment (CMDA). Med. Hypotheses. 85 (4), 433–440. doi:10.1016/j.mehy.2015.06.024

Morgan, S., Lopert, R., and Greyson, D. (2008). Toward a definition of pharmaceutical innovation. Open Med. 2 (1), e4–e7. PMID: 21602949, PMCID: PMC3091590.

PubMed Abstract | Google Scholar

Mt-Isa, S., Hallgreen, C. E., Wang, N., Callreus, T., Genov, G., Hirsch, I., et al. (2014). Balancing benefit and risk of medicines: a systematic review and classification of available methodologies. Pharmacoepidemiol Drug Saf. 23 (7), 667–678. doi:10.1002/pds.3636

National Institute for Health and Care Excellence (NICE) (2022). NICE health technology evaluation topic selection: the manual. Available at: https://www.nice.org.uk/process/pmg37/resources/nice-health-technology-evaluation-topic-selection-the-manual-pdf-72286780924357 .

Oliveira, M. D., Mataloto, I., and Kanavos, P. (2019). Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art. Eur. J. Health Econ. 20 (6), 891–918. doi:10.1007/s10198-019-01052-3

Park, S. E., Lim, S. H., Choi, H. W., Lee, S. M., Kim, D. W., Yim, E. Y., et al. (2012). Evaluation on the first 2 years of the positive list system in South Korea. Health Policy 104 (1), 32–39. doi:10.1016/j.healthpol.2011.09.006

Pérez Encinas, M., Fernández, M. A., Martín, M. L., Calvo, M. V., Gómez-Alonso, A., Dominguez-Gil, A., et al. (1998). Multicriteria decision analysis for determining drug therapy for intermittent claudication. Methods Find. Exp. Clin. Pharmacol. 20 (5), 425–431. doi:10.1358/mf.1998.20.5.485704

Schumacher, G. E. (1991). Multiattribute evaluation in formulary decision making as applied to calcium-channel blockers. Am. J. Hosp. Pharm. 48 (2), 301–308. doi:10.1093/ajhp/48.2.301

Sidi, Y., and Harel, O. (2020). Comprehensive benefit-risk assessment of noninferior treatments using multicriteria decision analysis. Value Health 23 (12), 1622–1629. doi:10.1016/j.jval.2020.09.002

Tanios, N., Wagner, M., Tony, M., Baltussen, R., van Til, J., Rindress, D., et al. (2013). Which criteria are considered in healthcare decisions? Insights from an international survey of policy and clinical decision makers. Int. J. Technol. Assess. Health Care. 29 (4), 456–465. doi:10.1017/S0266462313000573

Tervonen, T., Naci, H., van Valkenhoef, G., Ades, A. E., Angelis, A., Hillege, H. L., et al. (2015). Applying multiple criteria decision analysis to comparative benefit-risk assessment: choosing among statins in primary prevention. Med. Decis. Mak. 35 (7), 859–871. doi:10.1177/0272989X15587005

Vermersch, P., Martinelli, V., Pfleger, C., Rieckmann, P., Alonso-Magdalena, L., Galazka, A., et al. (2019). Benefit-risk assessment of cladribine using multi-criteria decision analysis (MCDA) for patients with relapsing-remitting multiple sclerosis. Clin. Ther. 41 (2), 249–260. doi:10.1016/j.clinthera.2018.12.015

Wilson, R., Chua, J., Pryymachenko, Y., Pathak, A., Sharma, S., and Abbott, J. H. (2022). Prioritizing healthcare interventions: a comparison of multicriteria decision analysis and cost-effectiveness analysis. Value Health 25 (2), 268–275. doi:10.1016/j.jval.2021.08.008

Yu, Y. N., Liu, J., Li, B., and Dang, H. X. (2023). Expert consensus on core indicators for lifecycle value assessment of Chinese patent medicine. China J. Chin. Mater. Med. 48 (19), 5389–5396. Specialist Group of Specialty Committee of Data Monitoring and Decision Making of the World Federation of Chinese Medicine Societies. doi:10.19540/j.cnki.cjcmm.20230516.501

Zelei, T., Mendola, N. D., Elezbawy, B., Németh, B., and Campbell, J. D. (2021). Criteria and scoring functions used in multi-criteria decision analysis and value frameworks for the assessment of rare disease therapies: a systematic literature review. Pharmacoecon. Open 5 (4), 605–612. doi:10.1007/s41669-021-00271-w

Keywords: multi-criteria decision analysis, drug value, criteria, weighting techniques, scoring techniques

Citation: Su P, Zhi K, Xu H, Xiao J, Liu J, Wang Z, Liu Q, Yu Y and Dang H (2024) The application of multi-criteria decision analysis in evaluating the value of drug-oriented intervention: a literature review. Front. Pharmacol. 15:1245825. doi: 10.3389/fphar.2024.1245825

Received: 28 June 2023; Accepted: 10 April 2024; Published: 24 April 2024.

Reviewed by:

Copyright © 2024 Su, Zhi, Xu, Xiao, Liu, Wang, Liu, Yu and Dang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Yanan Yu, [email protected] ; Haixia Dang, [email protected]

† These authors have contributed equally to this work

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.