qualitative systematic review dissertation

• UNC Libraries
• HSL Subject Research
• Qualitative Research Resources
• Integrating Qualitative Research into Systematic Reviews

Qualitative Research Resources: Integrating Qualitative Research into Systematic Reviews

Created by health science librarians.

• What is Qualitative Research?
• Qualitative Research Basics
• Special Topics
• Training Opportunities: UNC & Beyond
• Help at UNC
• Qualitative Software for Coding/Analysis
• Software for Audio, Video, Online Surveys
• Finding Qualitative Studies
• Assessing Qualitative Research
• Writing Up Your Research

About this Page

Articles, manuals, handbooks, meta-ethnography & interpretive (vs. aggregative) approaches, websites/tutorials, quality assessment tools.

• Publishing Qualitative Research
• Presenting Qualitative Research
• Qualitative & Libraries: a few gems
• Data Repositories

Why is this information important?

• Researchers in health science fields are increasingly recognizing the value of including qualitative studies in systematic reviews.
• Because qualitative and quantitative studies can be so different, however, it can be hard to know how to integrate them productively.

On this page you will find the following helpful resources:

• Articles and chapters that discuss methods for integrating qualitative studies into systematic reviews
• Examples of systematic reviews that include qualitative studies
• Selected books on qualitative studies and systematic reviews that are owned by UNC Libraries
• A short list of helpful websites and tutorials on the topic

See also:  Assessing Qualitative Research

The following articles and chapters offer some advice on how to include qualitative research in systematic reviews, as well as some examples of reviews that have done so.

Methods for Including Qualitative Research in Systematic Reviews

Butler, Ashleigh, Helen Hall, Beverley Copnell. (2016). A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care. Worldviews on Evidence-Based Nursing 13(3): 241-249.

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3  (updated February 2022) . Cochrane, 2022. Available from www.training.cochrane.org/handbook .. 

• Chapter 21: Qualitative Evidence ( Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, Garside R, Hannes K, Pantoja T, Thomas J. Chapter 21: Qualitative evidence. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook . )

See Journal of Clinical Epidemiology, Volume 97, May 2018, Cochrane Qualitative and Implementation Methods Group Guidance Series. UNC Chapel Hill users direct link .

• Current version can be accessed here: Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022) . .
•   See Chapter 21: Qualitative Evidence ( Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, Garside R, Hannes K, Pantoja T, Thomas J. Chapter 21: Qualitative evidence. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook . )

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B. How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research 2006; 6: 27-44.

Dixon-Woods, Mary, Shona Argawal, David Jones, Bridget Young, and Alex Sutton. Synthesizing qualitative and quantitative evidence: a review of possible methods.   J Health Serv Res Policy January 1, 2005 vol. 10 no. 1 45-53B

Harden, Angela. Mixed-Methods Systematic Reviews: Integrating Quantitative and Qualitative Findings. FOCUS 25, 2010.

JBI/Joanna Briggs Institute : Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. Aromataris E, Munn Z, editors . JBI Manual for Evidence Synthesis.  JBI; 2020. Available from:  https://synthesismanual.jbi.global .   https://doi.org/10.46658/JBIMES-20-03 . In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from  JBI Manual for Evidence Synthesis  

Ludvigsen, Mette S., Elizabeth O.C. Hall, Gabriele Meyer, Liv Fegran, Hanne Aagaard, Lisbeth Uhrenfeldt. (2015) Using Sandelowski and Barroso’s Meta-Synthesis Method in Advancing Qualitative Evidence . Qualitative Health Research 26(3). PMID:25794523 DOI: 10.1177/1049732315576493

• UNC Chapel Hill users:  link to article in SAGE Journals Qualitative Health .  

Glenton C, Bohren MA, Downe S, Paulsen EJ, Lewin S, on behalf of the Cochrane People, Health Systems and Public Health Thematic Group and Cochrane Norway. (2023 ). Cochrane Qualitative Evidence Syntheses: Protocol and review template v1.4. doi.org/10.5281/zenodo.10050961  (downloadable template available at the link) 

2021 Campbell Webinar Series, webinar 9, presented by Ruth Garside, Co-chair and Editor of the Campbell Methods Coordinating Group, on 10 August 2021.  Beyond barriers and facilitators: what questions can qualitative evidence synthesis address? www.youtube.com/watch?v=CyHbLa69Tmg

Examples of Systematic Reviews Incorporating Qualitative Research

Finfgeld-Connett, Deborah. Intimate partner abuse among older women: qualitative systematic review . Clinical Nursing Research 2014, 23:6 664-683.

Lucas, Patricia, Janis Baird, Lisa Arai, Catherine Law, and Helen M. Roberts. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews . BMC Medical Research Methodology 2007, 7 :4  doi:10.1186/1471-2288-7-4

Added Value of Qualitative Research with Randomized Clinical Trials

O'Cathain, Alicia, Jackie Goode, Sarah J. Drabble, Kate J. Thomas, Anne Rudolph, and Jenny Hewison.  Getting added value from using qualitative research wtih randomized controlled trials:  a qualitative interview study .  Trials 2014, 15 (June): 215.  doi:10.1186/1745-6215-15-215 

Snowden, Claire  (Trials)  and David Gough  (Systematic Reviews)  (eds)  Qualitative Methods, Trials, and Systematic Reviews.  Joint Publication,  Trials  and  Systematic Reviews.  

• Snowden, Claire (2015).  Trials  editorial about the special joint publication:  Qualitative and mixed methods research in trials .  Trials  16: 558. 
• Gough, David (2015).   Systematic Reviews  editorial about the special joint publication:  Qualitative and mixed methods in systematic reviews .  Systemaatic Reviews  4:81.
• The two sister-journals,  Trials  and  Systematic Reviews , have, on the face of it, different readerships and deal with different issues. In both journals there is, however, a common and growing interest in the contribution of qualitative methods. We are seeing an expansion of the use and application of a range of techniques with entry into novel research areas and pursuit of new lines of inquiry. Our contributors are working within specific methods, with mixed methods, and across paradigms.  This special issue covers these innovative and challenging areas, with the aim of sharing methodological practice, findings and reflections to drive forward and further the respective fields.

Systematic Review of Qualitative Research (Meta-Synthesis)

Korhonen, Anne, Tuovi Hakulinen-Viitanen, Virpi Jylha, Arja Holopainen.  Meta-synthesis and evidence-based heath care:  a method for systemic review.   Scandinavian Journal of Caring Sciences 2013, 27:4, 1027-1034.  doi: 10.1111/scs.12003

GRADE-CERQUAL Approach

PLOS Medicine Staff. (2016). Correction: Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual). PLoS Medicine , 13 (6), e1002065. https://doi.org/10.1371/journal.pmed.1002065  PMID:27284910 PMCID: PMC4902189

• Both the corrected and uncorrected versions are available at the link above. 

See additional information in Website/Tutorial box, below. 

Cochrane Methods:  Qualitative & Implementation - Core Library of Qualitative Synthesis Methodology

• Core Library .  NB: items represent key methodology resources.  No endorsement of individual methods is implied by inclusion in this list.  See  Supplemental Handbook Guidance .   

Meta-ethnography is generally considered an interpretative (vs. aggregative) qualitative synthesis approach.  

eMERGe Project

Funded by the National Institute for Health Research of the National Health Service (NHS) in the UK,  The aim of the eMERGe project, which ran from June 2015 to May 2017, was to develop a guideline to improve the way researchers report meta-ethnographies. The website includes many resources and publications.  From the website: 

• Meta-ethnography is an interpretive qualitative synthesis approach developed by George W. Noblit and R. Dwight Hare, in the field of education, in the 1980s. They designed the approach to address the inability of an aggregative synthesis of five ethnographic studies to explain the failure of racial desegregation in schools. In a meta-ethnography, the reviewers conducting the meta-ethnography aim to produce new interpretations that transcend the findings of individual studies, rather than simply to aggregate findings. Noblit and Hare described it as ‘making a whole into something more than the parts alone imply’ (Noblit & Hare, 1988, p. 28), i.e. going beyond the findings of any individual study.

Meta-ethnography differs from other qualitative evidence synthesis approaches in its underpinning theory, use of the authors’ interpretations (e.g. concepts, themes) from primary qualitative studies as data, and creation of new interpretations through its unique analytic synthesis process. Researchers select, analyse and interpret qualitative studies to answer focused questions on a specific topic (e.g. people’s experiences of having and being treated for arthritis) to come up with new insights and conclusions. The aim of the eMERGe project was to develop a guideline to improve the way researchers report meta-ethnographies.

• EQUATOR NETWORK: guidelines for reporting meta-ethnography EQUATOR Network (Enhancing the QUAlity and Transparency Of health Research)
• The JBI (Joanna Briggs Institute) Approach to Qualitative Synthesis "The JBI uses a meta-aggregative approach to the synthesis of qualitative evidence. Meta aggregation is sensitive to the nature and traditions of qualitative research while being predicated on the process of systematic review (Pearson 2004). The meta-aggregative approach is sensitive to the practicality and usability of the primary author’s findings and does not seek to re-interpret those findings as some other methods of qualitative synthesis do. A strong feature of the meta-aggregative approach is that it seeks to enable generalizable statements in the form of recommendations to guide practitioners and policy makers (Hannes and Lockwood 2011). In this regard, meta aggregation contrasts with meta-ethnography or the critical interpretive approach to qualitative evidence synthesis, which have a focus on re-interpretation and theory generation rather than aggregation." more... less... Lockwood C, Porrit K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-20-03
• Cochrane Methods: Qualitative & Implementation--Core Library of Qualitative Synthesis Methodology Includes the following 4 references.

Note from the Core Library: The following items represent key methodology resources.  No endorsement of individual methods is implied by inclusion in this list.  See  Supplemental Handbook Guidance .   

• Campbell R, Pound P, Morgan M, Daker-White G, Britten N, Pill R, Yardley L, Pope C, Donovan J.  Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research .  Health Technol Assess . 2011 Dec;15(43):1-164.
• France, E. F., Ring, N., Thomas, R., Noyes, J., Maxwell, M., & Jepson, R. (2014). A methodological systematic review of what's wrong with meta-ethnography reporting. BMC medical research methodology, 14(1), 119.
• France, E. F., Wells, M., Lang, H., & Williams, B. (2016).  Why, when and how to update a meta-ethnography qualitative synthesis . Systematic Reviews, 5(1). doi:10.1186/s13643-016-0218-4
• Toye, F., Seers, K., Allcock, N., Briggs, M., Carr, E., Andrews, J., & Barker, K. (2013). 'Trying to pin down jelly'-exploring intuitive processes in quality assessment for meta-ethnography. BMC Medical Research Methodology, 13(1), 46.

Some additional meta-ethnography resources and examples:

• SAGE Research Methods (Qualitative) contains many resources for meta-ethnography.  "Meta-ethnography" can be searched from the general Sage Methods page. 
• To see some examples of published meta-ethnographies in Scopus (which indexes both social science and health-related literature), you can run a quick and dirty search such as "public health AND meta-ethnography" or "public health AND (policy OR management) and meta-ethnography)" .  

Historically speaking:

• This is the seminal work on meta-ethnography
• UNC Chapel Hill users e-book edition
• " This chapter explains meta-ethnography as created by Noblit and Hare and how the method has been used since."
• UNC Chapel Hill users e-book edition .
• Booth, A. (2001), 'Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research?' In  Qualitative Evidence-based conference: Taking a critical stance,  Coventry University.  Download the article from ResearchGate  

Temple University Libraries: What is a Meta-Synthesis

Cochrane Qualitative and Implementation Methods Group

JBI/Joanna Briggs Institute Manual for Evidence Synthesis  Chapter 2 (Systematic Reviews of Qualitative Evidence):  Lockwood C, Porrit K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z (Editors) . JBI Manual for Evidence Synthesis.  JBI, 2020. Available from  https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03

Canadian Cochrane Center YouTube Tutorial on Qualitative Evidence Synthesis, published Dec 2, 2013

This recording is of the first webinar of the 2013-2014 Different Evidence, Different Syntheses Series. Jane Noyes of the Cochrane Qualitative and Implementation Methods Group for the discussion was the presenter for the webinar held on November 28, 2013.This webinar explored: a) when to consider undertaking a synthesis of qualitative evidence; b) some frequently used methods and examples of developing methods for synthesising qualitative evidence; and c) approaches for integrating qualitative and quantitative findings.

Seminar on CerQual: a new approach to qualitative evidence synthesis analysis Oct 13, 2014

• Appendix A: Tools To Assess Risk of Bias of Individual Outcomes In: Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters LM, Santaguida PL, Shamliyan T, Singh K, Tsertsvadze A, Treadwell JR. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Agency for Healthcare Research and Quality Methods Guide for Comparative Effectiveness Reviews. March 2012. AHRQ Publication No. 12-EHC047-EF. Available at: www.effectivehealthcare.ahrq.gov/
• Observational Epidemiology Quality Rating Tool Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol 2007;36:666-76.
• Diagnostic Accuracy Tool Whiting P, Rutjes AWS, Dinnes J, et al. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technol Assess 2004;8(25):iii, 1-234.
• CASP Checklists A set of critical appraisal checklists. Checklists are available for systematic reviews, qualitative studies, RCTs, case-control studies, diagnostic studies, cohort studies, and economic evaluations.
• LEGEND Evidence Evaluation Tools A series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
• The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses Validated tool for assessing case-control and cohort studies.

Search the UNC Library Catalog

The Books section of this page contains a small selection of the books available on including qualitative research in systematic reviews. To find more, click the links below to search the UNC library catalog.

• Catalog Search -- Review Literature as Topic
• Catalog Search -- Systematic Reviews (Medical Research)
• Catalog Search -- Meta-Analysis as Topic

Contact HSL About Systematic Reviews

Ready to start a systematic review? HSL Librarians can help!

Fill out the Systematic Review Request Form and the best-suited librarian will get back to you promptly.  Our systematic review service is only available to faculty, staff, students, and others who are affiliated with UNC Chapel Hill.

• << Previous: Writing Up Your Research
• Next: Publishing Qualitative Research >>
• Last Updated: May 14, 2024 12:50 PM
• URL: https://guides.lib.unc.edu/qual

A qualitative systematic review of experiences and perceptions of youth suicide

Affiliations.

• 1 School of Nursing, College of Health and Medicine, University of Tasmania, Sydney, NSW, Australia.
• 2 School of Health Sciences, College of Health and Medicine, University of Tasmania, Sydney, NSW, Australia.
• PMID: 31188855
• PMCID: PMC6561633
• DOI: 10.1371/journal.pone.0217568

Background: Suicide remains a global issue with over 800,000 people dying from suicide every year. Youth suicide is especially serious due to the years of life lost when a young person takes their own life. Social interactions, perceived support, genetic predisposition and mental illnesses are factors associated with suicide ideation.

Objectives: To review and synthesize qualitative studies that explored the experiences and perceptions of suicide in people 25 years old and younger.

Design: Qualitative systematic review.

Data sources: PubMed, PsycINFO, Scopus and CINAHL were searched alongside hand-searching reference lists up to October 2018.

Methods: Methodological quality was assessed using the qualitative Critical Appraisal Skills Programme checklist. The 27 studies included in the review centered around youth suicide and included interviews with young people and members of the wider community. Thematic synthesis focused on factors leading to suicide attempts, elements important to recovery, beliefs within the community, and treatment/prevention strategies.

Results: Thematic analysis of the articles revealed four categories: i) triggers and risks leading to suicidality; ii) factors involved in recovery; iii) need for institutional treatment/prevention strategies; and iv) beliefs about suicide at a community level. The first category was further subdivided into: i) behaviours; ii) feelings/emotions; iii) family influences; iv) peer influences; and v) other. The second category was split into: i) interpersonal; ii) cultural; and iii) individual influences, while the third category was divided into i) education; and ii) treatment.

Conclusion: Youth suicide is a complex issue with many causes and risks factors which interact with one another. For successful treatment and prevention, procedural reform is needed, along with a shift in societal attitudes toward emotional expression and suicide.

Publication types

• Research Support, Non-U.S. Gov't
• Systematic Review
• Educational Status
• Emotional Regulation / physiology
• Genetic Predisposition to Disease
• Mental Disorders / physiopathology
• Mental Disorders / psychology*
• Psychological Distress
• Qualitative Research
• Risk Factors
• Social Networking*
• Suicidal Ideation*
• Suicide / psychology*
• Suicide Prevention
• Suicide, Attempted / prevention & control
• Suicide, Attempted / psychology*

Grants and funding

Dissertations and research projects

• Book a session
• Planning your research

Developing a theoretical framework

Reflecting on your position, extended literature reviews, presenting qualitative data.

• Quantitative research
• Writing up your research project
• e-learning and books
• SkillsCheck This link opens in a new window
• ⬅ Back to Skills Centre This link opens in a new window
• Review this resource

What is a theoretical framework?

Developing a theoretical framework for your dissertation is one of the key elements of a qualitative research project. Through writing your literature review, you are likely to have identified either a problem that need ‘fixing’ or a gap that your research may begin to fill.

The theoretical framework is your toolbox . In the toolbox are your handy tools: a set of theories, concepts, ideas and hypotheses that you will use to build a solution to the research problem or gap you have identified.

The methodology is the instruction manual: the procedure and steps you have taken, using your chosen tools, to tackle the research problem.

Why do I need a theoretical framework?

Developing a theoretical framework shows that you have thought critically about the different ways to approach your topic, and that you have made a well-reasoned and evidenced decision about which approach will work best. theoretical frameworks are also necessary for solving complex problems or issues from the literature, showing that you have the skills to think creatively and improvise to answer your research questions. they also allow researchers to establish new theories and approaches, that future research may go on to develop., how do i create a theoretical framework for my dissertation.

First, select your tools. You are likely to need a variety of tools in qualitative research – different theories, models or concepts – to help you tackle different parts of your research question.  

When deciding what tools would be best for the job of answering your research questions or problem, explore what existing research in your area has used. You may find that there is a ‘standard toolbox’ for qualitative research in your field that you can borrow from or apply to your own research.

You will need to justify why your chosen tools are best for the job of answering your research questions, at what stage they are most relevant, and how they relate to each other. Some theories or models will neatly fit together and appear in the toolboxes of other researchers. However, you may wish to incorporate a model or idea that is not typical for your research area – the ‘odd one out’ in your toolbox. If this is the case, make sure you justify and account for why it is useful to you, and look for ways that it can be used in partnership with the other tools you are using.

You should also be honest about limitations, or where you need to improvise (for example, if the ‘right’ tool or approach doesn’t exist in your area).

This video from the Skills Centre includes an overview and example of how you might create a theoretical framework for your dissertation:

How do I choose the 'right' approach?

When designing your framework and choosing what to include, it can often be difficult to know if you’ve chosen the ‘right’ approach for your research questions. One way to check this is to look for consistency between your objectives, the literature in your framework, and your overall ethos for the research. This means ensuring that the literature you have used not only contributes to answering your research objectives, but that you also use theories and models that are true to your beliefs as a researcher.

Reflecting on your values and your overall ambition for the project can be a helpful step in making these decisions, as it can help you to fully connect your methodology and methods to your research aims.

Should I reflect on my position as a researcher?

If you feel your position as a researcher has influenced your choice of methods or procedure in any way, the methodology is a good place to reflect on this.  Positionality  acknowledges that no researcher is entirely objective: we are all, to some extent, influenced by prior learning, experiences, knowledge, and personal biases. This is particularly true in qualitative research or practice-based research, where the student is acting as a researcher in their own workplace, where they are otherwise considered a practitioner/professional. It's also important to reflect on your positionality if you belong to the same community as your participants where this is the grounds for their involvement in the research (ie. you are a mature student interviewing other mature learners about their experences in higher education). 

The following questions can help you to reflect on your positionality and gauge whether this is an important section to include in your dissertation (for some people, this section isn’t necessary or relevant):

• How might my personal history influence how I approach the topic?
• How am I positioned in relation to this knowledge? Am I being influenced by prior learning or knowledge from outside of this course?
• How does my gender/social class/ ethnicity/ culture influence my positioning in relation to this topic?
• Do I share any attributes with my participants? Are we part of a s hared community? How might this have influenced our relationship and my role in interviews/observations?
• Am I invested in the outcomes on a personal level? Who is this research for and who will feel the benefits?

One option for qualitative projects is to write an extended literature review. This type of project does not require you to collect any new data. Instead, you should focus on synthesising a broad range of literature to offer a new perspective on a research problem or question.  

The main difference between an extended literature review and a dissertation where primary data is collected, is in the presentation of the methodology, results and discussion sections. This is because extended literature reviews do not actively involve participants or primary data collection, so there is no need to outline a procedure for data collection (the methodology) or to present and interpret ‘data’ (in the form of interview transcripts, numerical data, observations etc.) You will have much more freedom to decide which sections of the dissertation should be combined, and whether new chapters or sections should be added.

Here is an overview of a common structure for an extended literature review:

Introduction

• Provide background information and context to set the ‘backdrop’ for your project.
• Explain the value and relevance of your research in this context. Outline what do you hope to contribute with your dissertation.
• Clarify a specific area of focus.
• Introduce your research aims (or problem) and objectives.

Literature review

You will need to write a short, overview literature review to introduce the main theories, concepts and key research areas that you will explore in your dissertation. This set of texts – which may be theoretical, research-based, practice-based or policies – form your theoretical framework. In other words, by bringing these texts together in the literature review, you are creating a lens that you can then apply to more focused examples or scenarios in your discussion chapters.

Methodology

As you will not be collecting primary data, your methodology will be quite different from a typical dissertation. You will need to set out the process and procedure you used to find and narrow down your literature. This is also known as a search strategy.

Including your search strategy

A search strategy explains how you have narrowed down your literature to identify key studies and areas of focus. This often takes the form of a search strategy table, included as an appendix at the end of the dissertation. If included, this section takes the place of the traditional 'methodology' section.

If you choose to include a search strategy table, you should also give an overview of your reading process in the main body of the dissertation.  Think of this as a chronology of the practical steps you took and your justification for doing so at each stage, such as:

• Your key terms, alternatives and synonyms, and any terms that you chose to exclude.
• Your choice and combination of databases;
• Your inclusion/exclusion criteria, when they were applied and why. This includes filters such as language of publication, date, and country of origin;
• You should also explain which terms you combined to form search phrases and your use of Boolean searching (AND, OR, NOT);
• Your use of citation searching (selecting articles from the bibliography of a chosen journal article to further your search).
• Your use of any search models, such as PICO and SPIDER to help shape your approach.
• Search strategy template A simple template for recording your literature searching. This can be included as an appendix to show your search strategy.

The discussion section of an extended literature review is the most flexible in terms of structure. Think of this section as a series of short case studies or ‘windows’ on your research. In this section you will apply the theoretical framework you formed in the literature review – a combination of theories, models and ideas that explain your approach to the topic – to a series of different examples and scenarios. These are usually presented as separate discussion ‘chapters’ in the dissertation, in an order that you feel best fits your argument.

Think about an order for these discussion sections or chapters that helps to tell the story of your research. One common approach is to structure these sections by common themes or concepts that help to draw your sources together. You might also opt for a chronological structure if your dissertation aims to show change or development over time. Another option is to deliberately show where there is a lack of chronology or narrative across your case studies, by ordering them in a fragmentary order! You will be able to reflect upon the structure of these chapters elsewhere in the dissertation, explaining and defending your decision in the methodology and conclusion.

A summary of your key findings – what you have concluded from your research, and how far you have been able to successfully answer your research questions.

• Recommendations – for improvements to your own study, for future research in the area, and for your field more widely.
• Emphasise your contributions to knowledge and what you have achieved.

Alternative structure

Depending on your research aims, and whether you are working with a case-study type approach (where each section of the dissertation considers a different example or concept through the lens established in your literature review), you might opt for one of the following structures:

Splitting the literature review across different chapters:

This structure allows you to pull apart the traditional literature review, introducing it little by little with each of your themed chapters. This approach works well for dissertations that attempt to show change or difference over time, as the relevant literature for that section or period can be introduced gradually to the reader.

Whichever structure you opt for, remember to explain and justify your approach. A marker will be interested in why you decided on your chosen structure, what it allows you to achieve/brings to the project and what alternatives you considered and rejected in the planning process. Here are some example sentence starters:

In qualitative studies, your results are often presented alongside the discussion, as it is difficult to include this data in a meaningful way without explanation and interpretation. In the dsicussion section, aim to structure your work thematically, moving through the key concepts or ideas that have emerged from your qualitative data. Use extracts from your data collection - interviews, focus groups, observations - to illustrate where these themes are most prominent, and refer back to the sources from your literature review to help draw conclusions. 

Here's an example of how your data could be presented in paragraph format in this section:

Example from  'Reporting and discussing your findings ', Monash University .

• << Previous: Planning your research
• Next: Quantitative research >>
• Last Updated: Apr 17, 2024 1:52 PM
• URL: https://libguides.shu.ac.uk/researchprojects

How to Write a Systematic Review Dissertation: With Examples

Writing a systematic review dissertation isn’t easy because you must follow a thorough and accurate scientific process. You must be an expert in research methodology to synthesise studies. In this article, I will provide a step-by-step approach to writing a top-notch systematic review dissertation.

Table of Contents

However, for students who may find this process challenging and seek professional assistance, I recommend exploring SystematicReviewPro —a reliable systematic review writing service. By signing up and placing a free inquiry and engaging with the admin team at any time, students can avail themselves of an exclusive offer of up to 50% off on their systematic review order. Additionally, there is already a 30% discount running on the website, making it an excellent opportunity to ease your dissertation journey.

As an Undergraduate or Master’s student, you’re are allowed to pick a systematic review for your dissertation. As a PhD student, you can use a systematic review methodology in the second chapter (literature review) of your dissertation. A systematic review is considered the highest level of empirical evidence, especially in clinical sciences like nursing and medicine. When developing new practice guidelines, new services, or new products, systematic reviews are searched and synthesised first on that topic or idea.

Factors to Consider When Writing a Systematic Review Dissertation

The nature of your research topic or research question.

Some research topics or questions strictly conform to qualitative or quantitative methods. For example, if you’re exploring the lived experiences, attitudes, perceptions, and meaning-making in a given population, you’ll need qualitative methods. However, you will require quantitative methods if looking into quantifiable variables like happiness, depression, academic performance, sleep, etc. That said, the nature of your research question should guide you. If your topic is qualitative, you’ll need qualitative studies only. If your topic is quantitative, you’ll need quantitative studies only. Systematic reviews of qualitative studies are less intricate than of quantitative studies. Still, they require a thoughtful approach in synthesizing findings from various qualitative studies.

If you choose to review quantitative studies, you might need to conduct a meta-analysis in your systematic review. A meta-analysis refers to statistical techniques used in pooling findings from various independent studies to compute a summary statistic. For example, in your dissertation, you may aim to investigate the effect of a student well-being programme embedded in university classes on the happiness of university students. Various studies that have investigated the same or a related intervention and quantitively measured happiness among university students must be synthesised together using a statistical technique. The ultimate outcome of that meta-analysis is to provide an overview of the overall trend of the effect of the intervention on university student’s happiness. For more information about how to formulate a research question for a systematic review with a meta-analysis, visit this link.

An example meta-analysis showing the statistical combination of findings from various studies to indicate the overall effect of a psychological intervention on the psychological well-being of university students.

Availability of primary studies

Finding primary studies for your systematic review is the hardest thing you can encounter with this approach. You can choose your topic and plan your journey so well. Upon reaching the point you need primary studies to answer your research question, you get stuck. Retrieving primary studies is challenging because it requires advanced search strategies on various online databases. Doing an advanced search strategy can be an uphill task for someone who has never done a systematic review. This is because, more often than not, depending on the topic, primary studies are not readily available on the Internet. Remember, secondary studies, like systematic reviews and literature reviews, are not eligible for systematic reviews.

Supervisor’s recommendation

Always confirm with your supervisor if you can do a systematic review dissertation. Some supervisors may feel it better for you to do a primary study. So, always confirm with your supervisor before doing much.

Your confidence

Always ensure you’re confident that you can do a systematic review on your own. Writing a systematic review isn’t easy. You need to be aware that doing a systematic review may even be harder than doing interviews or surveys in primary research. Why? A systematic review involves combining many primary studies together in a scientific manner. That means you must have expertise in various research methodologies to know the best way to integrate or synthesise the various studies.

Availability of time and resources

The main advantage of doing a systematic review dissertation is that it saves a lot of time. Conducting interviews or surveys can be time- and resource-consuming. However, with a systematic review, you do everything from your desk. It will save you a lot of time and resources. If you find that you meet many of the requirements of successfully conducting a systematic review, the next step is to engage in the actual process. The step-by-step approach used in writing systematic reviews is outlined below.

Step-by-Step Process in Writing a Systematic Review Dissertation

The following steps are iterative, meaning you can start over again and again until you meet your research objectives. The step-by-step guide on how to write a systematic review dissertation is summarized in the infographic shown below.

Step-by-step guide on how to write a systematic review dissertation

Step 1: Formulate the systematic review research question

The starting point of a systematic review is to formulate a research question. As stated above, the nature of your research question will help you make key decisions. For example, you will be able to know which design (quantitative versus qualitative) to consider in your inclusion and exclusion criteria.

Step 2: Do a preliminary search

The next step is to perform a preliminary search on the Internet to determine if another systematic review has been published. It is not acceptable to repeat what has already been done. Your research should be novel and contribute to a knowledge gap. However, if you find that another systematic review has already been published on your topic. You should consider the publication date.

In most cases, systematic reviews on given topics are outdated. They have not used recent studies published on that topic, thus missing important updates. That can be a good reason you’re conducting your study. Suppose there’s an updated systematic review on your topic. In that case, you should consider reformulating your research question to address a specific knowledge gap.

Step 3: Develop your systematic review inclusion and exclusion criteria

One unique thing about systematic reviews is that they must be based on a very specific population, intervention/exposure, and assess a specific outcome. Let’s say, for example, you write on Intervention A’s effectiveness in reducing depression symptoms in older frail people. In that case, you must retrieve studies that strictly assess the effectiveness of Intervention A, the outcome being depression symptoms and the population being older frail people.

Therefore, it will be against the principles of a systematic review to focus on Intervention B (different intervention/exposure) on anxiety (different outcomes) in younger people (different populations). Also, depending on your research question, you will need to determine the research design (qualitative versus quantitative) of the studies you will review. Other criteria to consider are the country of publication, the publication date, language, etc.

Step 4: Develop your systematic review search strategy

As said, the main challenge in writing a systematic review is to identify papers. Your literature search should be thorough so that you don’t leave out some relevant studies. Developing a literature search strategy isn’t easy because you must start identifying relevant keywords and search terms for your topic. You must start by knowing common terminologies used in your subject of interest.

Afterward, combine the keywords using Boolean connectors like “AND” & “OR.” For example, suppose my topic is the effectiveness of cognitive behavioural therapy in treating anxiety in adolescents. In that regard, I can combine my keywords as follows: (Cognitive behavioural therapy OR CBT) AND (anxiety) AND (adolescents OR youth). If you use terminologies unknown in your discipline, you will likely not find relevant studies for review.

Step 5: Plan and perform systematic review database selection

At this stage, you identify the databases you’ll use to execute your search strategy. When writing a systematic review dissertation, you also need to report the databases that you searched. Commonly searched ones in the field of social and health sciences include PubMed, Google Scholar, Cochrane, PsycInfo, and many others. You need to know how each database works. Also, apart from Google Scholar and PubMed, most of these databases require paid or institutional access. Liaise with your supervisor or librarian to help in identifying good databases for subject and discipline.

Step 6: Perform systematic review screening using titles and abstracts

When you execute your search strategy on each database, results or search hits will be displayed. This is also another difficult step because of tedious work involved. You start by screening the titles. Then, eliminate results that contain irrelevant titles. You need to be careful at this point because sometimes people eliminate even relevant studies. The title doesn’t need to contain exactly your keywords. Some titles appear totally irrelevant but they actually contain useful data inside.

After screening titles, the next step is to screen abstracts. You may be surprised at this point that the titles you thought were irrelevant actually contain relevant information. For instance, some studies may indicate in the title that their study focused on depression as an outcome when you’re interested in anxiety. However, reading the abstract may surprise you that depression was only a primary outcome. The authors also measured secondary outcomes, among them anxiety. In such an article, you can decide to focus on anxiety results only because they are relevant to your study.

Step 7: Do a manual search to supplement database search

After screening articles identified using various databases, the next step is to augment the search strategy with a manual search. This will ensure you don’t miss relevant studies in your systematic review dissertation. The manual search involves identifying more studies in the bibliographies of the identified articles using a database search. It is also about contacting the authors and experts sourced from the found articles to give access to more articles that may not be found online. Finally, you can also identify key journals from the articles and perform a hand search. For example, suppose I identify the Journal of Cognitive Psychology. In that case, I will visit that journal’s website and perform a manual search there. A properly done manual search can help you identify more articles that you couldn’t have identified using databases only.

Step 8: Perform systematic review screening using the full-body texts

After having all your articles intact, the next step is to screen for full-text bodies. In most cases, the titles and abstracts may not contain enough information for screening purposes. You must read the full texts of the articles to determine their full eligibility. At this point, you screen articles identified through database search and manual search altogether. For example, sometimes you may be interested in healthy adolescents. In the abstract, the author of the articles may only report adolescents without providing any specifics about them. Upon reading the full text, you may discover that the authors included adolescents with mental issues that are not within your study’s scope. Therefore, always do a full-text screening before you move to the next step.

Step 9: Perform systematic review quality assessment using PRISMA, etc

Systematic review dissertations can be used to inform the formulation of practice guidelines and even inform policies. You must strive to review only studies with rigorous methodological quality. The quality assessment tool will depend on your study’s design. The commonly used ones for student dissertations include CASP Checklists and Joanna Briggs Institute (JBI) Checklists. You can consult with your supervisor before arriving at the final decision. Transparently report your quality assessment findings. For example, indicate the score of each study under each item of each tool and calculate the overall score in the form of a percentage. Also, always have a cut-off of 65%, and studies whose methodological rigour is below the cut-off are excluded.

Step 10: Perform systematic review data extraction

The next step is to extract relevant data from your studies. Your data extraction approach depends on the research design of the studies you used. If you use qualitative studies, your data extraction can focus on individual studies’ findings, particularly themes. You can also extract data that can aid in-depth analysis, such as country of study, population characteristics, etc. Using quantitative studies, you can collect quantitative data that will aid your analysis, such as means and standard deviations and other crucial information relevant to your analysis technique. Always chart your data in a tabular format to facilitate easy management and handling.

Step 11: Carry on with systematic review data analysis

The data analysis approach used in your systematic review dissertation will depend on the research design. Using qualitative studies, you will rely on qualitative approaches to analyse your data. For example, you can do a thematic analysis or a narrative synthesis. If you used quantitative studies, you might need to perform a meta-analysis or narrative synthesis. A meta-analysis is done when you have homogenous studies (such as population, outcome variables, measurement tools, etc.) that are experimental in nature. Particularly, meta-analysis is performed when reviewing controlled randomized trials or other interventional studies. In other words, meta-analysis is appropriately used when reviewing the effectiveness of interventions. However, if your quantitative studies are heterogenous, such as using different research designs, you must perform a narrative synthesis.

Step 12: Prepare the written report

The final step is to produce a written report of your systematic review dissertation. One of the ethical concerns in systematic reviews is transparency. You can improve the transparency of your reporting by using an established protocol like PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses).

Approximate price: $ 22

Calculate the price of your order

• Free title page and bibliography
• Unlimited revisions
• Plagiarism-free guarantee

Money-back guarantee

• 24/7 support
• Systematic Review Service
• Meta Analysis Services
• Literature Search Service
• Literature Review Assistance
• Scientific Article Writing Service
• Manuscript Publication Assistance
• 275 words per page
• 12 pt Arial/Times New Roman
• Double line spacing
• Any citation style (APA, MLA, Chicago/Turabian, Harvard, etc)

Our guarantees

Delivering a high-quality product at a reasonable price is not enough anymore. That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.

You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.

Zero-plagiarism guarantee

Each paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.

Free-revision policy

Thanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.

Privacy policy

Your email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.

Fair-cooperation guarantee

By sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PMC10248995

Guidance to best tools and practices for systematic reviews

Kat kolaski.

1 Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC USA

Lynne Romeiser Logan

2 Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY USA

John P. A. Ioannidis

3 Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA USA

Associated Data

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.

A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.

Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13643-023-02255-9.

Part 1. The state of evidence synthesis

Evidence syntheses are commonly regarded as the foundation of evidence-based medicine (EBM). They are widely accredited for providing reliable evidence and, as such, they have significantly influenced medical research and clinical practice. Despite their uptake throughout health care and ubiquity in contemporary medical literature, some important aspects of evidence syntheses are generally overlooked or not well recognized. Evidence syntheses are mostly retrospective exercises, they often depend on weak or irreparably flawed data, and they may use tools that have acknowledged or yet unrecognized limitations. They are complicated and time-consuming undertakings prone to bias and errors. Production of a good evidence synthesis requires careful preparation and high levels of organization in order to limit potential pitfalls [ 1 ]. Many authors do not recognize the complexity of such an endeavor and the many methodological challenges they may encounter. Failure to do so is likely to result in research and resource waste.

Given their potential impact on people’s lives, it is crucial for evidence syntheses to correctly report on the current knowledge base. In order to be perceived as trustworthy, reliable demonstration of the accuracy of evidence syntheses is equally imperative [ 2 ]. Concerns about the trustworthiness of evidence syntheses are not recent developments. From the early years when EBM first began to gain traction until recent times when thousands of systematic reviews are published monthly [ 3 ] the rigor of evidence syntheses has always varied. Many systematic reviews and meta-analyses had obvious deficiencies because original methods and processes had gaps, lacked precision, and/or were not widely known. The situation has improved with empirical research concerning which methods to use and standardization of appraisal tools. However, given the geometrical increase in the number of evidence syntheses being published, a relatively larger pool of unreliable evidence syntheses is being published today.

Publication of methodological studies that critically appraise the methods used in evidence syntheses is increasing at a fast pace. This reflects the availability of tools specifically developed for this purpose [ 4 – 6 ]. Yet many clinical specialties report that alarming numbers of evidence syntheses fail on these assessments. The syntheses identified report on a broad range of common conditions including, but not limited to, cancer, [ 7 ] chronic obstructive pulmonary disease, [ 8 ] osteoporosis, [ 9 ] stroke, [ 10 ] cerebral palsy, [ 11 ] chronic low back pain, [ 12 ] refractive error, [ 13 ] major depression, [ 14 ] pain, [ 15 ] and obesity [ 16 , 17 ]. The situation is even more concerning with regard to evidence syntheses included in clinical practice guidelines (CPGs) [ 18 – 20 ]. Astonishingly, in a sample of CPGs published in 2017–18, more than half did not apply even basic systematic methods in the evidence syntheses used to inform their recommendations [ 21 ].

These reports, while not widely acknowledged, suggest there are pervasive problems not limited to evidence syntheses that evaluate specific kinds of interventions or include primary research of a particular study design (eg, randomized versus non-randomized) [ 22 ]. Similar concerns about the reliability of evidence syntheses have been expressed by proponents of EBM in highly circulated medical journals [ 23 – 26 ]. These publications have also raised awareness about redundancy, inadequate input of statistical expertise, and deficient reporting. These issues plague primary research as well; however, there is heightened concern for the impact of these deficiencies given the critical role of evidence syntheses in policy and clinical decision-making.

Methods and guidance to produce a reliable evidence synthesis

Several international consortiums of EBM experts and national health care organizations currently provide detailed guidance (Table ​ (Table1). 1 ). They draw criteria from the reporting and methodological standards of currently recommended appraisal tools, and regularly review and update their methods to reflect new information and changing needs. In addition, they endorse the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system for rating the overall quality of a body of evidence [ 27 ]. These groups typically certify or commission systematic reviews that are published in exclusive databases (eg, Cochrane, JBI) or are used to develop government or agency sponsored guidelines or health technology assessments (eg, National Institute for Health and Care Excellence [NICE], Scottish Intercollegiate Guidelines Network [SIGN], Agency for Healthcare Research and Quality [AHRQ]). They offer developers of evidence syntheses various levels of methodological advice, technical and administrative support, and editorial assistance. Use of specific protocols and checklists are required for development teams within these groups, but their online methodological resources are accessible to any potential author.

Guidance for development of evidence syntheses

Notably, Cochrane is the largest single producer of evidence syntheses in biomedical research; however, these only account for 15% of the total [ 28 ]. The World Health Organization requires Cochrane standards be used to develop evidence syntheses that inform their CPGs [ 29 ]. Authors investigating questions of intervention effectiveness in syntheses developed for Cochrane follow the Methodological Expectations of Cochrane Intervention Reviews [ 30 ] and undergo multi-tiered peer review [ 31 , 32 ]. Several empirical evaluations have shown that Cochrane systematic reviews are of higher methodological quality compared with non-Cochrane reviews [ 4 , 7 , 9 , 11 , 14 , 32 – 35 ]. However, some of these assessments have biases: they may be conducted by Cochrane-affiliated authors, and they sometimes use scales and tools developed and used in the Cochrane environment and by its partners. In addition, evidence syntheses published in the Cochrane database are not subject to space or word restrictions, while non-Cochrane syntheses are often limited. As a result, information that may be relevant to the critical appraisal of non-Cochrane reviews is often removed or is relegated to online-only supplements that may not be readily or fully accessible [ 28 ].

Influences on the state of evidence synthesis

Many authors are familiar with the evidence syntheses produced by the leading EBM organizations but can be intimidated by the time and effort necessary to apply their standards. Instead of following their guidance, authors may employ methods that are discouraged or outdated 28]. Suboptimal methods described in in the literature may then be taken up by others. For example, the Newcastle–Ottawa Scale (NOS) is a commonly used tool for appraising non-randomized studies [ 36 ]. Many authors justify their selection of this tool with reference to a publication that describes the unreliability of the NOS and recommends against its use [ 37 ]. Obviously, the authors who cite this report for that purpose have not read it. Authors and peer reviewers have a responsibility to use reliable and accurate methods and not copycat previous citations or substandard work [ 38 , 39 ]. Similar cautions may potentially extend to automation tools. These have concentrated on evidence searching [ 40 ] and selection given how demanding it is for humans to maintain truly up-to-date evidence [ 2 , 41 ]. Cochrane has deployed machine learning to identify randomized controlled trials (RCTs) and studies related to COVID-19, [ 2 , 42 ] but such tools are not yet commonly used [ 43 ]. The routine integration of automation tools in the development of future evidence syntheses should not displace the interpretive part of the process.

Editorials about unreliable or misleading systematic reviews highlight several of the intertwining factors that may contribute to continued publication of unreliable evidence syntheses: shortcomings and inconsistencies of the peer review process, lack of endorsement of current standards on the part of journal editors, the incentive structure of academia, industry influences, publication bias, and the lure of “predatory” journals [ 44 – 48 ]. At this juncture, clarification of the extent to which each of these factors contribute remains speculative, but their impact is likely to be synergistic.

Over time, the generalized acceptance of the conclusions of systematic reviews as incontrovertible has affected trends in the dissemination and uptake of evidence. Reporting of the results of evidence syntheses and recommendations of CPGs has shifted beyond medical journals to press releases and news headlines and, more recently, to the realm of social media and influencers. The lay public and policy makers may depend on these outlets for interpreting evidence syntheses and CPGs. Unfortunately, communication to the general public often reflects intentional or non-intentional misrepresentation or “spin” of the research findings [ 49 – 52 ] News and social media outlets also tend to reduce conclusions on a body of evidence and recommendations for treatment to binary choices (eg, “do it” versus “don’t do it”) that may be assigned an actionable symbol (eg, red/green traffic lights, smiley/frowning face emoji).

Strategies for improvement

Many authors and peer reviewers are volunteer health care professionals or trainees who lack formal training in evidence synthesis [ 46 , 53 ]. Informing them about research methodology could increase the likelihood they will apply rigorous methods [ 25 , 33 , 45 ]. We tackle this challenge, from both a theoretical and a practical perspective, by offering guidance applicable to any specialty. It is based on recent methodological research that is extensively referenced to promote self-study. However, the information presented is not intended to be substitute for committed training in evidence synthesis methodology; instead, we hope to inspire our target audience to seek such training. We also hope to inform a broader audience of clinicians and guideline developers influenced by evidence syntheses. Notably, these communities often include the same members who serve in different capacities.

In the following sections, we highlight methodological concepts and practices that may be unfamiliar, problematic, confusing, or controversial. In Part 2, we consider various types of evidence syntheses and the types of research evidence summarized by them. In Part 3, we examine some widely used (and misused) tools for the critical appraisal of systematic reviews and reporting guidelines for evidence syntheses. In Part 4, we discuss how to meet methodological conduct standards applicable to key components of systematic reviews. In Part 5, we describe the merits and caveats of rating the overall certainty of a body of evidence. Finally, in Part 6, we summarize suggested terminology, methods, and tools for development and evaluation of evidence syntheses that reflect current best practices.

Part 2. Types of syntheses and research evidence

A good foundation for the development of evidence syntheses requires an appreciation of their various methodologies and the ability to correctly identify the types of research potentially available for inclusion in the synthesis.

Types of evidence syntheses

Systematic reviews have historically focused on the benefits and harms of interventions; over time, various types of systematic reviews have emerged to address the diverse information needs of clinicians, patients, and policy makers [ 54 ] Systematic reviews with traditional components have become defined by the different topics they assess (Table 2.1 ). In addition, other distinctive types of evidence syntheses have evolved, including overviews or umbrella reviews, scoping reviews, rapid reviews, and living reviews. The popularity of these has been increasing in recent years [ 55 – 58 ]. A summary of the development, methods, available guidance, and indications for these unique types of evidence syntheses is available in Additional File 2 A.

Types of traditional systematic reviews

Both Cochrane [ 30 , 59 ] and JBI [ 60 ] provide methodologies for many types of evidence syntheses; they describe these with different terminology, but there is obvious overlap (Table 2.2 ). The majority of evidence syntheses published by Cochrane (96%) and JBI (62%) are categorized as intervention reviews. This reflects the earlier development and dissemination of their intervention review methodologies; these remain well-established [ 30 , 59 , 61 ] as both organizations continue to focus on topics related to treatment efficacy and harms. In contrast, intervention reviews represent only about half of the total published in the general medical literature, and several non-intervention review types contribute to a significant proportion of the other half.

Evidence syntheses published by Cochrane and JBI

a Data from https://www.cochranelibrary.com/cdsr/reviews . Accessed 17 Sep 2022

b Data obtained via personal email communication on 18 Sep 2022 with Emilie Francis, editorial assistant, JBI Evidence Synthesis

c Includes the following categories: prevalence, scoping, mixed methods, and realist reviews

d This methodology is not supported in the current version of the JBI Manual for Evidence Synthesis

Types of research evidence

There is consensus on the importance of using multiple study designs in evidence syntheses; at the same time, there is a lack of agreement on methods to identify included study designs. Authors of evidence syntheses may use various taxonomies and associated algorithms to guide selection and/or classification of study designs. These tools differentiate categories of research and apply labels to individual study designs (eg, RCT, cross-sectional). A familiar example is the Design Tree endorsed by the Centre for Evidence-Based Medicine [ 70 ]. Such tools may not be helpful to authors of evidence syntheses for multiple reasons.

Suboptimal levels of agreement and accuracy even among trained methodologists reflect challenges with the application of such tools [ 71 , 72 ]. Problematic distinctions or decision points (eg, experimental or observational, controlled or uncontrolled, prospective or retrospective) and design labels (eg, cohort, case control, uncontrolled trial) have been reported [ 71 ]. The variable application of ambiguous study design labels to non-randomized studies is common, making them especially prone to misclassification [ 73 ]. In addition, study labels do not denote the unique design features that make different types of non-randomized studies susceptible to different biases, including those related to how the data are obtained (eg, clinical trials, disease registries, wearable devices). Given this limitation, it is important to be aware that design labels preclude the accurate assignment of non-randomized studies to a “level of evidence” in traditional hierarchies [ 74 ].

These concerns suggest that available tools and nomenclature used to distinguish types of research evidence may not uniformly apply to biomedical research and non-health fields that utilize evidence syntheses (eg, education, economics) [ 75 , 76 ]. Moreover, primary research reports often do not describe study design or do so incompletely or inaccurately; thus, indexing in PubMed and other databases does not address the potential for misclassification [ 77 ]. Yet proper identification of research evidence has implications for several key components of evidence syntheses. For example, search strategies limited by index terms using design labels or study selection based on labels applied by the authors of primary studies may cause inconsistent or unjustified study inclusions and/or exclusions [ 77 ]. In addition, because risk of bias (RoB) tools consider attributes specific to certain types of studies and study design features, results of these assessments may be invalidated if an inappropriate tool is used. Appropriate classification of studies is also relevant for the selection of a suitable method of synthesis and interpretation of those results.

An alternative to these tools and nomenclature involves application of a few fundamental distinctions that encompass a wide range of research designs and contexts. While these distinctions are not novel, we integrate them into a practical scheme (see Fig. ​ Fig.1) 1 ) designed to guide authors of evidence syntheses in the basic identification of research evidence. The initial distinction is between primary and secondary studies. Primary studies are then further distinguished by: 1) the type of data reported (qualitative or quantitative); and 2) two defining design features (group or single-case and randomized or non-randomized). The different types of studies and study designs represented in the scheme are described in detail in Additional File 2 B. It is important to conceptualize their methods as complementary as opposed to contrasting or hierarchical [ 78 ]; each offers advantages and disadvantages that determine their appropriateness for answering different kinds of research questions in an evidence synthesis.

Distinguishing types of research evidence

Application of these basic distinctions may avoid some of the potential difficulties associated with study design labels and taxonomies. Nevertheless, debatable methodological issues are raised when certain types of research identified in this scheme are included in an evidence synthesis. We briefly highlight those associated with inclusion of non-randomized studies, case reports and series, and a combination of primary and secondary studies.

Non-randomized studies

When investigating an intervention’s effectiveness, it is important for authors to recognize the uncertainty of observed effects reported by studies with high RoB. Results of statistical analyses that include such studies need to be interpreted with caution in order to avoid misleading conclusions [ 74 ]. Review authors may consider excluding randomized studies with high RoB from meta-analyses. Non-randomized studies of intervention (NRSI) are affected by a greater potential range of biases and thus vary more than RCTs in their ability to estimate a causal effect [ 79 ]. If data from NRSI are synthesized in meta-analyses, it is helpful to separately report their summary estimates [ 6 , 74 ].

Nonetheless, certain design features of NRSI (eg, which parts of the study were prospectively designed) may help to distinguish stronger from weaker ones. Cochrane recommends that authors of a review including NRSI focus on relevant study design features when determining eligibility criteria instead of relying on non-informative study design labels [ 79 , 80 ] This process is facilitated by a study design feature checklist; guidance on using the checklist is included with developers’ description of the tool [ 73 , 74 ]. Authors collect information about these design features during data extraction and then consider it when making final study selection decisions and when performing RoB assessments of the included NRSI.

Case reports and case series

Correctly identified case reports and case series can contribute evidence not well captured by other designs [ 81 ]; in addition, some topics may be limited to a body of evidence that consists primarily of uncontrolled clinical observations. Murad and colleagues offer a framework for how to include case reports and series in an evidence synthesis [ 82 ]. Distinguishing between cohort studies and case series in these syntheses is important, especially for those that rely on evidence from NRSI. Additional data obtained from studies misclassified as case series can potentially increase the confidence in effect estimates. Mathes and Pieper provide authors of evidence syntheses with specific guidance on distinguishing between cohort studies and case series, but emphasize the increased workload involved [ 77 ].

Primary and secondary studies

Synthesis of combined evidence from primary and secondary studies may provide a broad perspective on the entirety of available literature on a topic. This is, in fact, the recommended strategy for scoping reviews that may include a variety of sources of evidence (eg, CPGs, popular media). However, except for scoping reviews, the synthesis of data from primary and secondary studies is discouraged unless there are strong reasons to justify doing so.

Combining primary and secondary sources of evidence is challenging for authors of other types of evidence syntheses for several reasons [ 83 ]. Assessments of RoB for primary and secondary studies are derived from conceptually different tools, thus obfuscating the ability to make an overall RoB assessment of a combination of these study types. In addition, authors who include primary and secondary studies must devise non-standardized methods for synthesis. Note this contrasts with well-established methods available for updating existing evidence syntheses with additional data from new primary studies [ 84 – 86 ]. However, a new review that synthesizes data from primary and secondary studies raises questions of validity and may unintentionally support a biased conclusion because no existing methodological guidance is currently available [ 87 ].

Recommendations

We suggest that journal editors require authors to identify which type of evidence synthesis they are submitting and reference the specific methodology used for its development. This will clarify the research question and methods for peer reviewers and potentially simplify the editorial process. Editors should announce this practice and include it in the instructions to authors. To decrease bias and apply correct methods, authors must also accurately identify the types of research evidence included in their syntheses.

Part 3. Conduct and reporting

The need to develop criteria to assess the rigor of systematic reviews was recognized soon after the EBM movement began to gain international traction [ 88 , 89 ]. Systematic reviews rapidly became popular, but many were very poorly conceived, conducted, and reported. These problems remain highly prevalent [ 23 ] despite development of guidelines and tools to standardize and improve the performance and reporting of evidence syntheses [ 22 , 28 ]. Table 3.1  provides some historical perspective on the evolution of tools developed specifically for the evaluation of systematic reviews, with or without meta-analysis.

Tools specifying standards for systematic reviews with and without meta-analysis

a Currently recommended

b Validated tool for systematic reviews of interventions developed for use by authors of overviews or umbrella reviews

These tools are often interchangeably invoked when referring to the “quality” of an evidence synthesis. However, quality is a vague term that is frequently misused and misunderstood; more precisely, these tools specify different standards for evidence syntheses. Methodological standards address how well a systematic review was designed and performed [ 5 ]. RoB assessments refer to systematic flaws or limitations in the design, conduct, or analysis of research that distort the findings of the review [ 4 ]. Reporting standards help systematic review authors describe the methodology they used and the results of their synthesis in sufficient detail [ 92 ]. It is essential to distinguish between these evaluations: a systematic review may be biased, it may fail to report sufficient information on essential features, or it may exhibit both problems; a thoroughly reported systematic evidence synthesis review may still be biased and flawed while an otherwise unbiased one may suffer from deficient documentation.

We direct attention to the currently recommended tools listed in Table 3.1  but concentrate on AMSTAR-2 (update of AMSTAR [A Measurement Tool to Assess Systematic Reviews]) and ROBIS (Risk of Bias in Systematic Reviews), which evaluate methodological quality and RoB, respectively. For comparison and completeness, we include PRISMA 2020 (update of the 2009 Preferred Reporting Items for Systematic Reviews of Meta-Analyses statement), which offers guidance on reporting standards. The exclusive focus on these three tools is by design; it addresses concerns related to the considerable variability in tools used for the evaluation of systematic reviews [ 28 , 88 , 96 , 97 ]. We highlight the underlying constructs these tools were designed to assess, then describe their components and applications. Their known (or potential) uptake and impact and limitations are also discussed.

Evaluation of conduct

Development.

AMSTAR [ 5 ] was in use for a decade prior to the 2017 publication of AMSTAR-2; both provide a broad evaluation of methodological quality of intervention systematic reviews, including flaws arising through poor conduct of the review [ 6 ]. ROBIS, published in 2016, was developed to specifically assess RoB introduced by the conduct of the review; it is applicable to systematic reviews of interventions and several other types of reviews [ 4 ]. Both tools reflect a shift to a domain-based approach as opposed to generic quality checklists. There are a few items unique to each tool; however, similarities between items have been demonstrated [ 98 , 99 ]. AMSTAR-2 and ROBIS are recommended for use by: 1) authors of overviews or umbrella reviews and CPGs to evaluate systematic reviews considered as evidence; 2) authors of methodological research studies to appraise included systematic reviews; and 3) peer reviewers for appraisal of submitted systematic review manuscripts. For authors, these tools may function as teaching aids and inform conduct of their review during its development.

Description

Systematic reviews that include randomized and/or non-randomized studies as evidence can be appraised with AMSTAR-2 and ROBIS. Other characteristics of AMSTAR-2 and ROBIS are summarized in Table 3.2 . Both tools define categories for an overall rating; however, neither tool is intended to generate a total score by simply calculating the number of responses satisfying criteria for individual items [ 4 , 6 ]. AMSTAR-2 focuses on the rigor of a review’s methods irrespective of the specific subject matter. ROBIS places emphasis on a review’s results section— this suggests it may be optimally applied by appraisers with some knowledge of the review’s topic as they may be better equipped to determine if certain procedures (or lack thereof) would impact the validity of a review’s findings [ 98 , 100 ]. Reliability studies show AMSTAR-2 overall confidence ratings strongly correlate with the overall RoB ratings in ROBIS [ 100 , 101 ].

Comparison of AMSTAR-2 and ROBIS

a ROBIS includes an optional first phase to assess the applicability of the review to the research question of interest. The tool may be applicable to other review types in addition to the four specified, although modification of this initial phase will be needed (Personal Communication via email, Penny Whiting, 28 Jan 2022)

b AMSTAR-2 item #9 and #11 require separate responses for RCTs and NRSI

Interrater reliability has been shown to be acceptable for AMSTAR-2 [ 6 , 11 , 102 ] and ROBIS [ 4 , 98 , 103 ] but neither tool has been shown to be superior in this regard [ 100 , 101 , 104 , 105 ]. Overall, variability in reliability for both tools has been reported across items, between pairs of raters, and between centers [ 6 , 100 , 101 , 104 ]. The effects of appraiser experience on the results of AMSTAR-2 and ROBIS require further evaluation [ 101 , 105 ]. Updates to both tools should address items shown to be prone to individual appraisers’ subjective biases and opinions [ 11 , 100 ]; this may involve modifications of the current domains and signaling questions as well as incorporation of methods to make an appraiser’s judgments more explicit. Future revisions of these tools may also consider the addition of standards for aspects of systematic review development currently lacking (eg, rating overall certainty of evidence, [ 99 ] methods for synthesis without meta-analysis [ 105 ]) and removal of items that assess aspects of reporting that are thoroughly evaluated by PRISMA 2020.

Application

A good understanding of what is required to satisfy the standards of AMSTAR-2 and ROBIS involves study of the accompanying guidance documents written by the tools’ developers; these contain detailed descriptions of each item’s standards. In addition, accurate appraisal of a systematic review with either tool requires training. Most experts recommend independent assessment by at least two appraisers with a process for resolving discrepancies as well as procedures to establish interrater reliability, such as pilot testing, a calibration phase or exercise, and development of predefined decision rules [ 35 , 99 – 101 , 103 , 104 , 106 ]. These methods may, to some extent, address the challenges associated with the diversity in methodological training, subject matter expertise, and experience using the tools that are likely to exist among appraisers.

The standards of AMSTAR, AMSTAR-2, and ROBIS have been used in many methodological studies and epidemiological investigations. However, the increased publication of overviews or umbrella reviews and CPGs has likely been a greater influence on the widening acceptance of these tools. Critical appraisal of the secondary studies considered evidence is essential to the trustworthiness of both the recommendations of CPGs and the conclusions of overviews. Currently both Cochrane [ 55 ] and JBI [ 107 ] recommend AMSTAR-2 and ROBIS in their guidance for authors of overviews or umbrella reviews. However, ROBIS and AMSTAR-2 were released in 2016 and 2017, respectively; thus, to date, limited data have been reported about the uptake of these tools or which of the two may be preferred [ 21 , 106 ]. Currently, in relation to CPGs, AMSTAR-2 appears to be overwhelmingly popular compared to ROBIS. A Google Scholar search of this topic (search terms “AMSTAR 2 AND clinical practice guidelines,” “ROBIS AND clinical practice guidelines” 13 May 2022) found 12,700 hits for AMSTAR-2 and 1,280 for ROBIS. The apparent greater appeal of AMSTAR-2 may relate to its longer track record given the original version of the tool was in use for 10 years prior to its update in 2017.

Barriers to the uptake of AMSTAR-2 and ROBIS include the real or perceived time and resources necessary to complete the items they include and appraisers’ confidence in their own ratings [ 104 ]. Reports from comparative studies available to date indicate that appraisers find AMSTAR-2 questions, responses, and guidance to be clearer and simpler compared with ROBIS [ 11 , 101 , 104 , 105 ]. This suggests that for appraisal of intervention systematic reviews, AMSTAR-2 may be a more practical tool than ROBIS, especially for novice appraisers [ 101 , 103 – 105 ]. The unique characteristics of each tool, as well as their potential advantages and disadvantages, should be taken into consideration when deciding which tool should be used for an appraisal of a systematic review. In addition, the choice of one or the other may depend on how the results of an appraisal will be used; for example, a peer reviewer’s appraisal of a single manuscript versus an appraisal of multiple systematic reviews in an overview or umbrella review, CPG, or systematic methodological study.

Authors of overviews and CPGs report results of AMSTAR-2 and ROBIS appraisals for each of the systematic reviews they include as evidence. Ideally, an independent judgment of their appraisals can be made by the end users of overviews and CPGs; however, most stakeholders, including clinicians, are unlikely to have a sophisticated understanding of these tools. Nevertheless, they should at least be aware that AMSTAR-2 and ROBIS ratings reported in overviews and CPGs may be inaccurate because the tools are not applied as intended by their developers. This can result from inadequate training of the overview or CPG authors who perform the appraisals, or to modifications of the appraisal tools imposed by them. The potential variability in overall confidence and RoB ratings highlights why appraisers applying these tools need to support their judgments with explicit documentation; this allows readers to judge for themselves whether they agree with the criteria used by appraisers [ 4 , 108 ]. When these judgments are explicit, the underlying rationale used when applying these tools can be assessed [ 109 ].

Theoretically, we would expect an association of AMSTAR-2 with improved methodological rigor and an association of ROBIS with lower RoB in recent systematic reviews compared to those published before 2017. To our knowledge, this has not yet been demonstrated; however, like reports about the actual uptake of these tools, time will tell. Additional data on user experience is also needed to further elucidate the practical challenges and methodological nuances encountered with the application of these tools. This information could potentially inform the creation of unifying criteria to guide and standardize the appraisal of evidence syntheses [ 109 ].

Evaluation of reporting

Complete reporting is essential for users to establish the trustworthiness and applicability of a systematic review’s findings. Efforts to standardize and improve the reporting of systematic reviews resulted in the 2009 publication of the PRISMA statement [ 92 ] with its accompanying explanation and elaboration document [ 110 ]. This guideline was designed to help authors prepare a complete and transparent report of their systematic review. In addition, adherence to PRISMA is often used to evaluate the thoroughness of reporting of published systematic reviews [ 111 ]. The updated version, PRISMA 2020 [ 93 ], and its guidance document [ 112 ] were published in 2021. Items on the original and updated versions of PRISMA are organized by the six basic review components they address (title, abstract, introduction, methods, results, discussion). The PRISMA 2020 update is a considerably expanded version of the original; it includes standards and examples for the 27 original and 13 additional reporting items that capture methodological advances and may enhance the replicability of reviews [ 113 ].

The original PRISMA statement fostered the development of various PRISMA extensions (Table 3.3 ). These include reporting guidance for scoping reviews and reviews of diagnostic test accuracy and for intervention reviews that report on the following: harms outcomes, equity issues, the effects of acupuncture, the results of network meta-analyses and analyses of individual participant data. Detailed reporting guidance for specific systematic review components (abstracts, protocols, literature searches) is also available.

PRISMA extensions

PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses

a Note the abstract reporting checklist is now incorporated into PRISMA 2020 [ 93 ]

Uptake and impact

The 2009 PRISMA standards [ 92 ] for reporting have been widely endorsed by authors, journals, and EBM-related organizations. We anticipate the same for PRISMA 2020 [ 93 ] given its co-publication in multiple high-impact journals. However, to date, there is a lack of strong evidence for an association between improved systematic review reporting and endorsement of PRISMA 2009 standards [ 43 , 111 ]. Most journals require a PRISMA checklist accompany submissions of systematic review manuscripts. However, the accuracy of information presented on these self-reported checklists is not necessarily verified. It remains unclear which strategies (eg, authors’ self-report of checklists, peer reviewer checks) might improve adherence to the PRISMA reporting standards; in addition, the feasibility of any potentially effective strategies must be taken into consideration given the structure and limitations of current research and publication practices [ 124 ].

Pitfalls and limitations of PRISMA, AMSTAR-2, and ROBIS

Misunderstanding of the roles of these tools and their misapplication may be widespread problems. PRISMA 2020 is a reporting guideline that is most beneficial if consulted when developing a review as opposed to merely completing a checklist when submitting to a journal; at that point, the review is finished, with good or bad methodological choices. However, PRISMA checklists evaluate how completely an element of review conduct was reported, but do not evaluate the caliber of conduct or performance of a review. Thus, review authors and readers should not think that a rigorous systematic review can be produced by simply following the PRISMA 2020 guidelines. Similarly, it is important to recognize that AMSTAR-2 and ROBIS are tools to evaluate the conduct of a review but do not substitute for conceptual methodological guidance. In addition, they are not intended to be simple checklists. In fact, they have the potential for misuse or abuse if applied as such; for example, by calculating a total score to make a judgment about a review’s overall confidence or RoB. Proper selection of a response for the individual items on AMSTAR-2 and ROBIS requires training or at least reference to their accompanying guidance documents.

Not surprisingly, it has been shown that compliance with the PRISMA checklist is not necessarily associated with satisfying the standards of ROBIS [ 125 ]. AMSTAR-2 and ROBIS were not available when PRISMA 2009 was developed; however, they were considered in the development of PRISMA 2020 [ 113 ]. Therefore, future studies may show a positive relationship between fulfillment of PRISMA 2020 standards for reporting and meeting the standards of tools evaluating methodological quality and RoB.

Choice of an appropriate tool for the evaluation of a systematic review first involves identification of the underlying construct to be assessed. For systematic reviews of interventions, recommended tools include AMSTAR-2 and ROBIS for appraisal of conduct and PRISMA 2020 for completeness of reporting. All three tools were developed rigorously and provide easily accessible and detailed user guidance, which is necessary for their proper application and interpretation. When considering a manuscript for publication, training in these tools can sensitize peer reviewers and editors to major issues that may affect the review’s trustworthiness and completeness of reporting. Judgment of the overall certainty of a body of evidence and formulation of recommendations rely, in part, on AMSTAR-2 or ROBIS appraisals of systematic reviews. Therefore, training on the application of these tools is essential for authors of overviews and developers of CPGs. Peer reviewers and editors considering an overview or CPG for publication must hold their authors to a high standard of transparency regarding both the conduct and reporting of these appraisals.

Part 4. Meeting conduct standards

Many authors, peer reviewers, and editors erroneously equate fulfillment of the items on the PRISMA checklist with superior methodological rigor. For direction on methodology, we refer them to available resources that provide comprehensive conceptual guidance [ 59 , 60 ] as well as primers with basic step-by-step instructions [ 1 , 126 , 127 ]. This section is intended to complement study of such resources by facilitating use of AMSTAR-2 and ROBIS, tools specifically developed to evaluate methodological rigor of systematic reviews. These tools are widely accepted by methodologists; however, in the general medical literature, they are not uniformly selected for the critical appraisal of systematic reviews [ 88 , 96 ].

To enable their uptake, Table 4.1  links review components to the corresponding appraisal tool items. Expectations of AMSTAR-2 and ROBIS are concisely stated, and reasoning provided.

Systematic review components linked to appraisal with AMSTAR-2 and ROBIS a

CoI conflict of interest, MA meta-analysis, NA not addressed, PICO participant, intervention, comparison, outcome, PRISMA-P Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols, RoB risk of bias

a Components shown in bold are chosen for elaboration in Part 4 for one (or both) of two reasons: 1) the component has been identified as potentially problematic for systematic review authors; and/or 2) the component is evaluated by standards of an AMSTAR-2 “critical” domain

b Critical domains of AMSTAR-2 are indicated by *

Issues involved in meeting the standards for seven review components (identified in bold in Table 4.1 ) are addressed in detail. These were chosen for elaboration for one (or both) of two reasons: 1) the component has been identified as potentially problematic for systematic review authors based on consistent reports of their frequent AMSTAR-2 or ROBIS deficiencies [ 9 , 11 , 15 , 88 , 128 , 129 ]; and/or 2) the review component is judged by standards of an AMSTAR-2 “critical” domain. These have the greatest implications for how a systematic review will be appraised: if standards for any one of these critical domains are not met, the review is rated as having “critically low confidence.”

Research question

Specific and unambiguous research questions may have more value for reviews that deal with hypothesis testing. Mnemonics for the various elements of research questions are suggested by JBI and Cochrane (Table 2.1 ). These prompt authors to consider the specialized methods involved for developing different types of systematic reviews; however, while inclusion of the suggested elements makes a review compliant with a particular review’s methods, it does not necessarily make a research question appropriate. Table 4.2  lists acronyms that may aid in developing the research question. They include overlapping concepts of importance in this time of proliferating reviews of uncertain value [ 130 ]. If these issues are not prospectively contemplated, systematic review authors may establish an overly broad scope, or develop runaway scope allowing them to stray from predefined choices relating to key comparisons and outcomes.

Research question development

a Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Hulley SB, Cummings SR, Browner WS, editors. Designing clinical research: an epidemiological approach; 4th edn. Lippincott Williams & Wilkins; 2007. p. 14–22

b Doran, GT. There’s a S.M.A.R.T. way to write management’s goals and objectives. Manage Rev. 1981;70:35-6.

c Johnson BT, Hennessy EA. Systematic reviews and meta-analyses in the health sciences: best practice methods for research syntheses. Soc Sci Med. 2019;233:237–51

Once a research question is established, searching on registry sites and databases for existing systematic reviews addressing the same or a similar topic is necessary in order to avoid contributing to research waste [ 131 ]. Repeating an existing systematic review must be justified, for example, if previous reviews are out of date or methodologically flawed. A full discussion on replication of intervention systematic reviews, including a consensus checklist, can be found in the work of Tugwell and colleagues [ 84 ].

Protocol development is considered a core component of systematic reviews [ 125 , 126 , 132 ]. Review protocols may allow researchers to plan and anticipate potential issues, assess validity of methods, prevent arbitrary decision-making, and minimize bias that can be introduced by the conduct of the review. Registration of a protocol that allows public access promotes transparency of the systematic review’s methods and processes and reduces the potential for duplication [ 132 ]. Thinking early and carefully about all the steps of a systematic review is pragmatic and logical and may mitigate the influence of the authors’ prior knowledge of the evidence [ 133 ]. In addition, the protocol stage is when the scope of the review can be carefully considered by authors, reviewers, and editors; this may help to avoid production of overly ambitious reviews that include excessive numbers of comparisons and outcomes or are undisciplined in their study selection.

An association with attainment of AMSTAR standards in systematic reviews with published prospective protocols has been reported [ 134 ]. However, completeness of reporting does not seem to be different in reviews with a protocol compared to those without one [ 135 ]. PRISMA-P [ 116 ] and its accompanying elaboration and explanation document [ 136 ] can be used to guide and assess the reporting of protocols. A final version of the review should fully describe any protocol deviations. Peer reviewers may compare the submitted manuscript with any available pre-registered protocol; this is required if AMSTAR-2 or ROBIS are used for critical appraisal.

There are multiple options for the recording of protocols (Table 4.3 ). Some journals will peer review and publish protocols. In addition, many online sites offer date-stamped and publicly accessible protocol registration. Some of these are exclusively for protocols of evidence syntheses; others are less restrictive and offer researchers the capacity for data storage, sharing, and other workflow features. These sites document protocol details to varying extents and have different requirements [ 137 ]. The most popular site for systematic reviews, the International Prospective Register of Systematic Reviews (PROSPERO), for example, only registers reviews that report on an outcome with direct relevance to human health. The PROSPERO record documents protocols for all types of reviews except literature and scoping reviews. Of note, PROSPERO requires authors register their review protocols prior to any data extraction [ 133 , 138 ]. The electronic records of most of these registry sites allow authors to update their protocols and facilitate transparent tracking of protocol changes, which are not unexpected during the progress of the review [ 139 ].

Options for protocol registration of evidence syntheses

a Authors are advised to contact their target journal regarding submission of systematic review protocols

b Registration is restricted to approved review projects

c The JBI registry lists review projects currently underway by JBI-affiliated entities. These records include a review’s title, primary author, research question, and PICO elements. JBI recommends that authors register eligible protocols with PROSPERO

d See Pieper and Rombey [ 137 ] for detailed characteristics of these five registries

e See Pieper and Rombey [ 137 ] for other systematic review data repository options

Study design inclusion

For most systematic reviews, broad inclusion of study designs is recommended [ 126 ]. This may allow comparison of results between contrasting study design types [ 126 ]. Certain study designs may be considered preferable depending on the type of review and nature of the research question. However, prevailing stereotypes about what each study design does best may not be accurate. For example, in systematic reviews of interventions, randomized designs are typically thought to answer highly specific questions while non-randomized designs often are expected to reveal greater information about harms or real-word evidence [ 126 , 140 , 141 ]. This may be a false distinction; randomized trials may be pragmatic [ 142 ], they may offer important (and more unbiased) information on harms [ 143 ], and data from non-randomized trials may not necessarily be more real-world-oriented [ 144 ].

Moreover, there may not be any available evidence reported by RCTs for certain research questions; in some cases, there may not be any RCTs or NRSI. When the available evidence is limited to case reports and case series, it is not possible to test hypotheses nor provide descriptive estimates or associations; however, a systematic review of these studies can still offer important insights [ 81 , 145 ]. When authors anticipate that limited evidence of any kind may be available to inform their research questions, a scoping review can be considered. Alternatively, decisions regarding inclusion of indirect as opposed to direct evidence can be addressed during protocol development [ 146 ]. Including indirect evidence at an early stage of intervention systematic review development allows authors to decide if such studies offer any additional and/or different understanding of treatment effects for their population or comparison of interest. Issues of indirectness of included studies are accounted for later in the process, during determination of the overall certainty of evidence (see Part 5 for details).

Evidence search

Both AMSTAR-2 and ROBIS require systematic and comprehensive searches for evidence. This is essential for any systematic review. Both tools discourage search restrictions based on language and publication source. Given increasing globalism in health care, the practice of including English-only literature should be avoided [ 126 ]. There are many examples in which language bias (different results in studies published in different languages) has been documented [ 147 , 148 ]. This does not mean that all literature, in all languages, is equally trustworthy [ 148 ]; however, the only way to formally probe for the potential of such biases is to consider all languages in the initial search. The gray literature and a search of trials may also reveal important details about topics that would otherwise be missed [ 149 – 151 ]. Again, inclusiveness will allow review authors to investigate whether results differ in gray literature and trials [ 41 , 151 – 153 ].

Authors should make every attempt to complete their review within one year as that is the likely viable life of a search. (1) If that is not possible, the search should be updated close to the time of completion [ 154 ]. Different research topics may warrant less of a delay, for example, in rapidly changing fields (as in the case of the COVID-19 pandemic), even one month may radically change the available evidence.

Excluded studies

AMSTAR-2 requires authors to provide references for any studies excluded at the full text phase of study selection along with reasons for exclusion; this allows readers to feel confident that all relevant literature has been considered for inclusion and that exclusions are defensible.

Risk of bias assessment of included studies

The design of the studies included in a systematic review (eg, RCT, cohort, case series) should not be equated with appraisal of its RoB. To meet AMSTAR-2 and ROBIS standards, systematic review authors must examine RoB issues specific to the design of each primary study they include as evidence. It is unlikely that a single RoB appraisal tool will be suitable for all research designs. In addition to tools for randomized and non-randomized studies, specific tools are available for evaluation of RoB in case reports and case series [ 82 ] and single-case experimental designs [ 155 , 156 ]. Note the RoB tools selected must meet the standards of the appraisal tool used to judge the conduct of the review. For example, AMSTAR-2 identifies four sources of bias specific to RCTs and NRSI that must be addressed by the RoB tool(s) chosen by the review authors. The Cochrane RoB-2 [ 157 ] tool for RCTs and ROBINS-I [ 158 ] for NRSI for RoB assessment meet the AMSTAR-2 standards. Appraisers on the review team should not modify any RoB tool without complete transparency and acknowledgment that they have invalidated the interpretation of the tool as intended by its developers [ 159 ]. Conduct of RoB assessments is not addressed AMSTAR-2; to meet ROBIS standards, two independent reviewers should complete RoB assessments of included primary studies.

Implications of the RoB assessments must be explicitly discussed and considered in the conclusions of the review. Discussion of the overall RoB of included studies may consider the weight of the studies at high RoB, the importance of the sources of bias in the studies being summarized, and if their importance differs in relationship to the outcomes reported. If a meta-analysis is performed, serious concerns for RoB of individual studies should be accounted for in these results as well. If the results of the meta-analysis for a specific outcome change when studies at high RoB are excluded, readers will have a more accurate understanding of this body of evidence. However, while investigating the potential impact of specific biases is a useful exercise, it is important to avoid over-interpretation, especially when there are sparse data.

Synthesis methods for quantitative data

Syntheses of quantitative data reported by primary studies are broadly categorized as one of two types: meta-analysis, and synthesis without meta-analysis (Table 4.4 ). Before deciding on one of these methods, authors should seek methodological advice about whether reported data can be transformed or used in other ways to provide a consistent effect measure across studies [ 160 , 161 ].

Common methods for quantitative synthesis

CI confidence interval (or credible interval, if analysis is done in Bayesian framework)

a See text for descriptions of the types of data combined in each of these approaches

b See Additional File 4  for guidance on the structure and presentation of forest plots

c General approach is similar to aggregate data meta-analysis but there are substantial differences relating to data collection and checking and analysis [ 162 ]. This approach to syntheses is applicable to intervention, diagnostic, and prognostic systematic reviews [ 163 ]

d Examples include meta-regression, hierarchical and multivariate approaches [ 164 ]

e In-depth guidance and illustrations of these methods are provided in Chapter 12 of the Cochrane Handbook [ 160 ]

Meta-analysis

Systematic reviews that employ meta-analysis should not be referred to simply as “meta-analyses.” The term meta-analysis strictly refers to a specific statistical technique used when study effect estimates and their variances are available, yielding a quantitative summary of results. In general, methods for meta-analysis involve use of a weighted average of effect estimates from two or more studies. If considered carefully, meta-analysis increases the precision of the estimated magnitude of effect and can offer useful insights about heterogeneity and estimates of effects. We refer to standard references for a thorough introduction and formal training [ 165 – 167 ].

There are three common approaches to meta-analysis in current health care–related systematic reviews (Table 4.4 ). Aggregate meta-analyses is the most familiar to authors of evidence syntheses and their end users. This standard meta-analysis combines data on effect estimates reported by studies that investigate similar research questions involving direct comparisons of an intervention and comparator. Results of these analyses provide a single summary intervention effect estimate. If the included studies in a systematic review measure an outcome differently, their reported results may be transformed to make them comparable [ 161 ]. Forest plots visually present essential information about the individual studies and the overall pooled analysis (see Additional File 4  for details).

Less familiar and more challenging meta-analytical approaches used in secondary research include individual participant data (IPD) and network meta-analyses (NMA); PRISMA extensions provide reporting guidelines for both [ 117 , 118 ]. In IPD, the raw data on each participant from each eligible study are re-analyzed as opposed to the study-level data analyzed in aggregate data meta-analyses [ 168 ]. This may offer advantages, including the potential for limiting concerns about bias and allowing more robust analyses [ 163 ]. As suggested by the description in Table 4.4 , NMA is a complex statistical approach. It combines aggregate data [ 169 ] or IPD [ 170 ] for effect estimates from direct and indirect comparisons reported in two or more studies of three or more interventions. This makes it a potentially powerful statistical tool; while multiple interventions are typically available to treat a condition, few have been evaluated in head-to-head trials [ 171 ]. Both IPD and NMA facilitate a broader scope, and potentially provide more reliable and/or detailed results; however, compared with standard aggregate data meta-analyses, their methods are more complicated, time-consuming, and resource-intensive, and they have their own biases, so one needs sufficient funding, technical expertise, and preparation to employ them successfully [ 41 , 172 , 173 ].

Several items in AMSTAR-2 and ROBIS address meta-analysis; thus, understanding the strengths, weaknesses, assumptions, and limitations of methods for meta-analyses is important. According to the standards of both tools, plans for a meta-analysis must be addressed in the review protocol, including reasoning, description of the type of quantitative data to be synthesized, and the methods planned for combining the data. This should not consist of stock statements describing conventional meta-analysis techniques; rather, authors are expected to anticipate issues specific to their research questions. Concern for the lack of training in meta-analysis methods among systematic review authors cannot be overstated. For those with training, the use of popular software (eg, RevMan [ 174 ], MetaXL [ 175 ], JBI SUMARI [ 176 ]) may facilitate exploration of these methods; however, such programs cannot substitute for the accurate interpretation of the results of meta-analyses, especially for more complex meta-analytical approaches.

Synthesis without meta-analysis

There are varied reasons a meta-analysis may not be appropriate or desirable [ 160 , 161 ]. Syntheses that informally use statistical methods other than meta-analysis are variably referred to as descriptive, narrative, or qualitative syntheses or summaries; these terms are also applied to syntheses that make no attempt to statistically combine data from individual studies. However, use of such imprecise terminology is discouraged; in order to fully explore the results of any type of synthesis, some narration or description is needed to supplement the data visually presented in tabular or graphic forms [ 63 , 177 ]. In addition, the term “qualitative synthesis” is easily confused with a synthesis of qualitative data in a qualitative or mixed methods review. “Synthesis without meta-analysis” is currently the preferred description of other ways to combine quantitative data from two or more studies. Use of this specific terminology when referring to these types of syntheses also implies the application of formal methods (Table 4.4 ).

Methods for syntheses without meta-analysis involve structured presentations of the data in any tables and plots. In comparison to narrative descriptions of each study, these are designed to more effectively and transparently show patterns and convey detailed information about the data; they also allow informal exploration of heterogeneity [ 178 ]. In addition, acceptable quantitative statistical methods (Table 4.4 ) are formally applied; however, it is important to recognize these methods have significant limitations for the interpretation of the effectiveness of an intervention [ 160 ]. Nevertheless, when meta-analysis is not possible, the application of these methods is less prone to bias compared with an unstructured narrative description of included studies [ 178 , 179 ].

Vote counting is commonly used in systematic reviews and involves a tally of studies reporting results that meet some threshold of importance applied by review authors. Until recently, it has not typically been identified as a method for synthesis without meta-analysis. Guidance on an acceptable vote counting method based on direction of effect is currently available [ 160 ] and should be used instead of narrative descriptions of such results (eg, “more than half the studies showed improvement”; “only a few studies reported adverse effects”; “7 out of 10 studies favored the intervention”). Unacceptable methods include vote counting by statistical significance or magnitude of effect or some subjective rule applied by the authors.

AMSTAR-2 and ROBIS standards do not explicitly address conduct of syntheses without meta-analysis, although AMSTAR-2 items 13 and 14 might be considered relevant. Guidance for the complete reporting of syntheses without meta-analysis for systematic reviews of interventions is available in the Synthesis without Meta-analysis (SWiM) guideline [ 180 ] and methodological guidance is available in the Cochrane Handbook [ 160 , 181 ].

Familiarity with AMSTAR-2 and ROBIS makes sense for authors of systematic reviews as these appraisal tools will be used to judge their work; however, training is necessary for authors to truly appreciate and apply methodological rigor. Moreover, judgment of the potential contribution of a systematic review to the current knowledge base goes beyond meeting the standards of AMSTAR-2 and ROBIS. These tools do not explicitly address some crucial concepts involved in the development of a systematic review; this further emphasizes the need for author training.

We recommend that systematic review authors incorporate specific practices or exercises when formulating a research question at the protocol stage, These should be designed to raise the review team’s awareness of how to prevent research and resource waste [ 84 , 130 ] and to stimulate careful contemplation of the scope of the review [ 30 ]. Authors’ training should also focus on justifiably choosing a formal method for the synthesis of quantitative and/or qualitative data from primary research; both types of data require specific expertise. For typical reviews that involve syntheses of quantitative data, statistical expertise is necessary, initially for decisions about appropriate methods, [ 160 , 161 ] and then to inform any meta-analyses [ 167 ] or other statistical methods applied [ 160 ].

Part 5. Rating overall certainty of evidence

Report of an overall certainty of evidence assessment in a systematic review is an important new reporting standard of the updated PRISMA 2020 guidelines [ 93 ]. Systematic review authors are well acquainted with assessing RoB in individual primary studies, but much less familiar with assessment of overall certainty across an entire body of evidence. Yet a reliable way to evaluate this broader concept is now recognized as a vital part of interpreting the evidence.

Historical systems for rating evidence are based on study design and usually involve hierarchical levels or classes of evidence that use numbers and/or letters to designate the level/class. These systems were endorsed by various EBM-related organizations. Professional societies and regulatory groups then widely adopted them, often with modifications for application to the available primary research base in specific clinical areas. In 2002, a report issued by the AHRQ identified 40 systems to rate quality of a body of evidence [ 182 ]. A critical appraisal of systems used by prominent health care organizations published in 2004 revealed limitations in sensibility, reproducibility, applicability to different questions, and usability to different end users [ 183 ]. Persistent use of hierarchical rating schemes to describe overall quality continues to complicate the interpretation of evidence. This is indicated by recent reports of poor interpretability of systematic review results by readers [ 184 – 186 ] and misleading interpretations of the evidence related to the “spin” systematic review authors may put on their conclusions [ 50 , 187 ].

Recognition of the shortcomings of hierarchical rating systems raised concerns that misleading clinical recommendations could result even if based on a rigorous systematic review. In addition, the number and variability of these systems were considered obstacles to quick and accurate interpretations of the evidence by clinicians, patients, and policymakers [ 183 ]. These issues contributed to the development of the GRADE approach. An international working group, that continues to actively evaluate and refine it, first introduced GRADE in 2004 [ 188 ]. Currently more than 110 organizations from 19 countries around the world have endorsed or are using GRADE [ 189 ].

GRADE approach to rating overall certainty

GRADE offers a consistent and sensible approach for two separate processes: rating the overall certainty of a body of evidence and the strength of recommendations. The former is the expected conclusion of a systematic review, while the latter is pertinent to the development of CPGs. As such, GRADE provides a mechanism to bridge the gap from evidence synthesis to application of the evidence for informed clinical decision-making [ 27 , 190 ]. We briefly examine the GRADE approach but only as it applies to rating overall certainty of evidence in systematic reviews.

In GRADE, use of “certainty” of a body of evidence is preferred over the term “quality.” [ 191 ] Certainty refers to the level of confidence systematic review authors have that, for each outcome, an effect estimate represents the true effect. The GRADE approach to rating confidence in estimates begins with identifying the study type (RCT or NRSI) and then systematically considers criteria to rate the certainty of evidence up or down (Table 5.1 ).

GRADE criteria for rating certainty of evidence

a Applies to randomized studies

b Applies to non-randomized studies

This process results in assignment of one of the four GRADE certainty ratings to each outcome; these are clearly conveyed with the use of basic interpretation symbols (Table 5.2 ) [ 192 ]. Notably, when multiple outcomes are reported in a systematic review, each outcome is assigned a unique certainty rating; thus different levels of certainty may exist in the body of evidence being examined.

GRADE certainty ratings and their interpretation symbols a

a From the GRADE Handbook [ 192 ]

GRADE’s developers acknowledge some subjectivity is involved in this process [ 193 ]. In addition, they emphasize that both the criteria for rating evidence up and down (Table 5.1 ) as well as the four overall certainty ratings (Table 5.2 ) reflect a continuum as opposed to discrete categories [ 194 ]. Consequently, deciding whether a study falls above or below the threshold for rating up or down may not be straightforward, and preliminary overall certainty ratings may be intermediate (eg, between low and moderate). Thus, the proper application of GRADE requires systematic review authors to take an overall view of the body of evidence and explicitly describe the rationale for their final ratings.

Advantages of GRADE

Outcomes important to the individuals who experience the problem of interest maintain a prominent role throughout the GRADE process [ 191 ]. These outcomes must inform the research questions (eg, PICO [population, intervention, comparator, outcome]) that are specified a priori in a systematic review protocol. Evidence for these outcomes is then investigated and each critical or important outcome is ultimately assigned a certainty of evidence as the end point of the review. Notably, limitations of the included studies have an impact at the outcome level. Ultimately, the certainty ratings for each outcome reported in a systematic review are considered by guideline panels. They use a different process to formulate recommendations that involves assessment of the evidence across outcomes [ 201 ]. It is beyond our scope to describe the GRADE process for formulating recommendations; however, it is critical to understand how these two outcome-centric concepts of certainty of evidence in the GRADE framework are related and distinguished. An in-depth illustration using examples from recently published evidence syntheses and CPGs is provided in Additional File 5 A (Table AF5A-1).

The GRADE approach is applicable irrespective of whether the certainty of the primary research evidence is high or very low; in some circumstances, indirect evidence of higher certainty may be considered if direct evidence is unavailable or of low certainty [ 27 ]. In fact, most interventions and outcomes in medicine have low or very low certainty of evidence based on GRADE and there seems to be no major improvement over time [ 202 , 203 ]. This is still a very important (even if sobering) realization for calibrating our understanding of medical evidence. A major appeal of the GRADE approach is that it offers a common framework that enables authors of evidence syntheses to make complex judgments about evidence certainty and to convey these with unambiguous terminology. This prevents some common mistakes made by review authors, including overstating results (or under-reporting harms) [ 187 ] and making recommendations for treatment. This is illustrated in Table AF5A-2 (Additional File 5 A), which compares the concluding statements made about overall certainty in a systematic review with and without application of the GRADE approach.

Theoretically, application of GRADE should improve consistency of judgments about certainty of evidence, both between authors and across systematic reviews. In one empirical evaluation conducted by the GRADE Working Group, interrater reliability of two individual raters assessing certainty of the evidence for a specific outcome increased from ~ 0.3 without using GRADE to ~ 0.7 by using GRADE [ 204 ]. However, others report variable agreement among those experienced in GRADE assessments of evidence certainty [ 190 ]. Like any other tool, GRADE requires training in order to be properly applied. The intricacies of the GRADE approach and the necessary subjectivity involved suggest that improving agreement may require strict rules for its application; alternatively, use of general guidance and consensus among review authors may result in less consistency but provide important information for the end user [ 190 ].

GRADE caveats

Simply invoking “the GRADE approach” does not automatically ensure GRADE methods were employed by authors of a systematic review (or developers of a CPG). Table 5.3 lists the criteria the GRADE working group has established for this purpose. These criteria highlight the specific terminology and methods that apply to rating the certainty of evidence for outcomes reported in a systematic review [ 191 ], which is different from rating overall certainty across outcomes considered in the formulation of recommendations [ 205 ]. Modifications of standard GRADE methods and terminology are discouraged as these may detract from GRADE’s objectives to minimize conceptual confusion and maximize clear communication [ 206 ].

Criteria for using GRADE in a systematic review a

a Adapted from the GRADE working group [ 206 ]; this list does not contain the additional criteria that apply to the development of a clinical practice guideline

Nevertheless, GRADE is prone to misapplications [ 207 , 208 ], which can distort a systematic review’s conclusions about the certainty of evidence. Systematic review authors without proper GRADE training are likely to misinterpret the terms “quality” and “grade” and to misunderstand the constructs assessed by GRADE versus other appraisal tools. For example, review authors may reference the standard GRADE certainty ratings (Table 5.2 ) to describe evidence for their outcome(s) of interest. However, these ratings are invalidated if authors omit or inadequately perform RoB evaluations of each included primary study. Such deficiencies in RoB assessments are unacceptable but not uncommon, as reported in methodological studies of systematic reviews and overviews [ 104 , 186 , 209 , 210 ]. GRADE ratings are also invalidated if review authors do not formally address and report on the other criteria (Table 5.1 ) necessary for a GRADE certainty rating.

Other caveats pertain to application of a GRADE certainty of evidence rating in various types of evidence syntheses. Current adaptations of GRADE are described in Additional File 5 B and included on Table 6.3 , which is introduced in the next section.

Concise Guide to best practices for evidence syntheses, version 1.0 a

AMSTAR A MeaSurement Tool to Assess Systematic Reviews, CASP Critical Appraisal Skills Programme, CERQual Confidence in the Evidence from Reviews of Qualitative research, ConQual Establishing Confidence in the output of Qualitative research synthesis, COSMIN COnsensus-based Standards for the selection of health Measurement Instruments, DTA diagnostic test accuracy, eMERGe meta-ethnography reporting guidance, ENTREQ enhancing transparency in reporting the synthesis of qualitative research, GRADE Grading of Recommendations Assessment, Development and Evaluation, MA meta-analysis, NRSI non-randomized studies of interventions, P protocol, PRIOR Preferred Reporting Items for Overviews of Reviews, PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses, PROBAST Prediction model Risk Of Bias ASsessment Tool, QUADAS quality assessment of studies of diagnostic accuracy included in systematic reviews, QUIPS Quality In Prognosis Studies, RCT randomized controlled trial, RoB risk of bias, ROBINS-I Risk Of Bias In Non-randomised Studies of Interventions, ROBIS Risk of Bias in Systematic Reviews, ScR scoping review, SWiM systematic review without meta-analysis

a Superscript numbers represent citations provided in the main reference list. Additional File 6 lists links to available online resources for the methods and tools included in the Concise Guide

b The MECIR manual [ 30 ] provides Cochrane’s specific standards for both reporting and conduct of intervention systematic reviews and protocols

c Editorial and peer reviewers can evaluate completeness of reporting in submitted manuscripts using these tools. Authors may be required to submit a self-reported checklist for the applicable tools

d The decision flowchart described by Flemming and colleagues [ 223 ] is recommended for guidance on how to choose the best approach to reporting for qualitative reviews

e SWiM was developed for intervention studies reporting quantitative data. However, if there is not a more directly relevant reporting guideline, SWiM may prompt reviewers to consider the important details to report. (Personal Communication via email, Mhairi Campbell, 14 Dec 2022)

f JBI recommends their own tools for the critical appraisal of various quantitative primary study designs included in systematic reviews of intervention effectiveness, prevalence and incidence, and etiology and risk as well as for the critical appraisal of systematic reviews included in umbrella reviews. However, except for the JBI Checklists for studies reporting prevalence data and qualitative research, the development, validity, and reliability of these tools are not well documented

g Studies that are not RCTs or NRSI require tools developed specifically to evaluate their design features. Examples include single case experimental design [ 155 , 156 ] and case reports and series [ 82 ]

h The evaluation of methodological quality of studies included in a synthesis of qualitative research is debatable [ 224 ]. Authors may select a tool appropriate for the type of qualitative synthesis methodology employed. The CASP Qualitative Checklist [ 218 ] is an example of a published, commonly used tool that focuses on assessment of the methodological strengths and limitations of qualitative studies. The JBI Critical Appraisal Checklist for Qualitative Research [ 219 ] is recommended for reviews using a meta-aggregative approach

i Consider including risk of bias assessment of included studies if this information is relevant to the research question; however, scoping reviews do not include an assessment of the overall certainty of a body of evidence

j Guidance available from the GRADE working group [ 225 , 226 ]; also recommend consultation with the Cochrane diagnostic methods group

k Guidance available from the GRADE working group [ 227 ]; also recommend consultation with Cochrane prognostic methods group

l Used for syntheses in reviews with a meta-aggregative approach [ 224 ]

m Chapter 5 in the JBI Manual offers guidance on how to adapt GRADE to prevalence and incidence reviews [ 69 ]

n Janiaud and colleagues suggest criteria for evaluating evidence certainty for meta-analyses of non-randomized studies evaluating risk factors [ 228 ]

o The COSMIN user manual provides details on how to apply GRADE in systematic reviews of measurement properties [ 229 ]

The expected culmination of a systematic review should be a rating of overall certainty of a body of evidence for each outcome reported. The GRADE approach is recommended for making these judgments for outcomes reported in systematic reviews of interventions and can be adapted for other types of reviews. This represents the initial step in the process of making recommendations based on evidence syntheses. Peer reviewers should ensure authors meet the minimal criteria for supporting the GRADE approach when reviewing any evidence synthesis that reports certainty ratings derived using GRADE. Authors and peer reviewers of evidence syntheses unfamiliar with GRADE are encouraged to seek formal training and take advantage of the resources available on the GRADE website [ 211 , 212 ].

Part 6. Concise Guide to best practices

Accumulating data in recent years suggest that many evidence syntheses (with or without meta-analysis) are not reliable. This relates in part to the fact that their authors, who are often clinicians, can be overwhelmed by the plethora of ways to evaluate evidence. They tend to resort to familiar but often inadequate, inappropriate, or obsolete methods and tools and, as a result, produce unreliable reviews. These manuscripts may not be recognized as such by peer reviewers and journal editors who may disregard current standards. When such a systematic review is published or included in a CPG, clinicians and stakeholders tend to believe that it is trustworthy. A vicious cycle in which inadequate methodology is rewarded and potentially misleading conclusions are accepted is thus supported. There is no quick or easy way to break this cycle; however, increasing awareness of best practices among all these stakeholder groups, who often have minimal (if any) training in methodology, may begin to mitigate it. This is the rationale for inclusion of Parts 2 through 5 in this guidance document. These sections present core concepts and important methodological developments that inform current standards and recommendations. We conclude by taking a direct and practical approach.

Inconsistent and imprecise terminology used in the context of development and evaluation of evidence syntheses is problematic for authors, peer reviewers and editors, and may lead to the application of inappropriate methods and tools. In response, we endorse use of the basic terms (Table 6.1 ) defined in the PRISMA 2020 statement [ 93 ]. In addition, we have identified several problematic expressions and nomenclature. In Table 6.2 , we compile suggestions for preferred terms less likely to be misinterpreted.

Terms relevant to the reporting of health care–related evidence syntheses a

a Reproduced from Page and colleagues [ 93 ]

Terminology suggestions for health care–related evidence syntheses

a For example, meta-aggregation, meta-ethnography, critical interpretative synthesis, realist synthesis

b This term may best apply to the synthesis in a mixed methods systematic review in which data from different types of evidence (eg, qualitative, quantitative, economic) are summarized [ 64 ]

We also propose a Concise Guide (Table 6.3 ) that summarizes the methods and tools recommended for the development and evaluation of nine types of evidence syntheses. Suggestions for specific tools are based on the rigor of their development as well as the availability of detailed guidance from their developers to ensure their proper application. The formatting of the Concise Guide addresses a well-known source of confusion by clearly distinguishing the underlying methodological constructs that these tools were designed to assess. Important clarifications and explanations follow in the guide’s footnotes; associated websites, if available, are listed in Additional File 6 .

To encourage uptake of best practices, journal editors may consider adopting or adapting the Concise Guide in their instructions to authors and peer reviewers of evidence syntheses. Given the evolving nature of evidence synthesis methodology, the suggested methods and tools are likely to require regular updates. Authors of evidence syntheses should monitor the literature to ensure they are employing current methods and tools. Some types of evidence syntheses (eg, rapid, economic, methodological) are not included in the Concise Guide; for these, authors are advised to obtain recommendations for acceptable methods by consulting with their target journal.

We encourage the appropriate and informed use of the methods and tools discussed throughout this commentary and summarized in the Concise Guide (Table 6.3 ). However, we caution against their application in a perfunctory or superficial fashion. This is a common pitfall among authors of evidence syntheses, especially as the standards of such tools become associated with acceptance of a manuscript by a journal. Consequently, published evidence syntheses may show improved adherence to the requirements of these tools without necessarily making genuine improvements in their performance.

In line with our main objective, the suggested tools in the Concise Guide address the reliability of evidence syntheses; however, we recognize that the utility of systematic reviews is an equally important concern. An unbiased and thoroughly reported evidence synthesis may still not be highly informative if the evidence itself that is summarized is sparse, weak and/or biased [ 24 ]. Many intervention systematic reviews, including those developed by Cochrane [ 203 ] and those applying GRADE [ 202 ], ultimately find no evidence, or find the evidence to be inconclusive (eg, “weak,” “mixed,” or of “low certainty”). This often reflects the primary research base; however, it is important to know what is known (or not known) about a topic when considering an intervention for patients and discussing treatment options with them.

Alternatively, the frequency of “empty” and inconclusive reviews published in the medical literature may relate to limitations of conventional methods that focus on hypothesis testing; these have emphasized the importance of statistical significance in primary research and effect sizes from aggregate meta-analyses [ 183 ]. It is becoming increasingly apparent that this approach may not be appropriate for all topics [ 130 ]. Development of the GRADE approach has facilitated a better understanding of significant factors (beyond effect size) that contribute to the overall certainty of evidence. Other notable responses include the development of integrative synthesis methods for the evaluation of complex interventions [ 230 , 231 ], the incorporation of crowdsourcing and machine learning into systematic review workflows (eg the Cochrane Evidence Pipeline) [ 2 ], the shift in paradigm to living systemic review and NMA platforms [ 232 , 233 ] and the proposal of a new evidence ecosystem that fosters bidirectional collaborations and interactions among a global network of evidence synthesis stakeholders [ 234 ]. These evolutions in data sources and methods may ultimately make evidence syntheses more streamlined, less duplicative, and more importantly, they may be more useful for timely policy and clinical decision-making; however, that will only be the case if they are rigorously reported and conducted.

We look forward to others’ ideas and proposals for the advancement of methods for evidence syntheses. For now, we encourage dissemination and uptake of the currently accepted best tools and practices for their development and evaluation; at the same time, we stress that uptake of appraisal tools, checklists, and software programs cannot substitute for proper education in the methodology of evidence syntheses and meta-analysis. Authors, peer reviewers, and editors must strive to make accurate and reliable contributions to the present evidence knowledge base; online alerts, upcoming technology, and accessible education may make this more feasible than ever before. Our intention is to improve the trustworthiness of evidence syntheses across disciplines, topics, and types of evidence syntheses. All of us must continue to study, teach, and act cooperatively for that to happen.

Acknowledgements

Michelle Oakman Hayes for her assistance with the graphics, Mike Clarke for his willingness to answer our seemingly arbitrary questions, and Bernard Dan for his encouragement of this project.

Authors’ contributions

All authors participated in the development of the ideas, writing, and review of this manuscript. The author(s) read and approved the final manuscript.

The work of John Ioannidis has been supported by an unrestricted gift from Sue and Bob O’Donnell to Stanford University.

Declarations

The authors declare no competing interests.

This article has been published simultaneously in BMC Systematic Reviews, Acta Anaesthesiologica Scandinavica, BMC Infectious Diseases, British Journal of Pharmacology, JBI Evidence Synthesis, the Journal of Bone and Joint Surgery Reviews , and the Journal of Pediatric Rehabilitation Medicine .

Publisher’ s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Library Services

UCL LIBRARY SERVICES

• Guides and databases
• Library skills
• Systematic reviews

Formulating a research question

• What are systematic reviews?
• Types of systematic reviews
• Identifying studies
• Searching databases
• Describing and appraising studies
• Synthesis and systematic maps
• Software for systematic reviews
• Online training and support
• Live and face to face training
• Individual support
• Further help

Clarifying the review question leads to specifying what type of studies can best address that question and setting out criteria for including such studies in the review. This is often called inclusion criteria or eligibility criteria. The criteria could relate to the review topic, the research methods of the studies, specific populations, settings, date limits, geographical areas, types of interventions, or something else.

Systematic reviews address clear and answerable research questions, rather than a general topic or problem of interest. They also have clear criteria about the studies that are being used to address the research questions. This is often called inclusion criteria or eligibility criteria.

Six examples of types of question are listed below, and the examples show different questions that a review might address based on the topic of influenza vaccination. Structuring questions in this way aids thinking about the different types of research that could address each type of question. Mneumonics can help in thinking about criteria that research must fulfil to address the question. The criteria could relate to the context, research methods of the studies, specific populations, settings, date limits, geographical areas, types of interventions, or something else.

Examples of review questions

• Needs - What do people want? Example: What are the information needs of healthcare workers regarding vaccination for seasonal influenza?
• Impact or effectiveness - What is the balance of benefit and harm of a given intervention? Example: What is the effectiveness of strategies to increase vaccination coverage among healthcare workers. What is the cost effectiveness of interventions that increase immunisation coverage?
• Process or explanation - Why does it work (or not work)? How does it work (or not work)?  Example: What factors are associated with uptake of vaccinations by healthcare workers?  What factors are associated with inequities in vaccination among healthcare workers?
• Correlation - What relationships are seen between phenomena? Example: How does influenza vaccination of healthcare workers vary with morbidity and mortality among patients? (Note: correlation does not in itself indicate causation).
• Views / perspectives - What are people's experiences? Example: What are the views and experiences of healthcare workers regarding vaccination for seasonal influenza?
• Service implementation - What is happening? Example: What is known about the implementation and context of interventions to promote vaccination for seasonal influenza among healthcare workers?

Examples in practice :  Seasonal influenza vaccination of health care workers: evidence synthesis / Loreno et al. 2017

Example of eligibility criteria

Research question: What are the views and experiences of UK healthcare workers regarding vaccination for seasonal influenza?

• Population: healthcare workers, any type, including those without direct contact with patients.
• Context: seasonal influenza vaccination for healthcare workers.
• Study design: qualitative data including interviews, focus groups, ethnographic data.
• Date of publication: all.
• Country: all UK regions.
• Studies focused on influenza vaccination for general population and pandemic influenza vaccination.
• Studies using survey data with only closed questions, studies that only report quantitative data.

Consider the research boundaries

It is important to consider the reasons that the research question is being asked. Any research question has ideological and theoretical assumptions around the meanings and processes it is focused on. A systematic review should either specify definitions and boundaries around these elements at the outset, or be clear about which elements are undefined. 

For example if we are interested in the topic of homework, there are likely to be pre-conceived ideas about what is meant by 'homework'. If we want to know the impact of homework on educational attainment, we need to set boundaries on the age range of children, or how educational attainment is measured. There may also be a particular setting or contexts: type of school, country, gender, the timeframe of the literature, or the study designs of the research.

Research question: What is the impact of homework on children's educational attainment?

• Scope : Homework - Tasks set by school teachers for students to complete out of school time, in any format or setting.
• Population: children aged 5-11 years.
• Outcomes: measures of literacy or numeracy from tests administered by researchers, school or other authorities.
• Study design: Studies with a comparison control group.
• Context: OECD countries, all settings within mainstream education.
• Date Limit: 2007 onwards.
• Any context not in mainstream primary schools.
• Non-English language studies.

Mnemonics for structuring questions

Some mnemonics that sometimes help to formulate research questions, set the boundaries of question and inform a search strategy.

Intervention effects

PICO  Population – Intervention– Outcome– Comparison

Variations: add T on for time, or ‘C’ for context, or S’ for study type,

Policy and management issues

ECLIPSE : Expectation – Client group – Location – Impact ‐ Professionals involved – Service

Expectation encourages  reflection on what the information is needed for i.e. improvement, innovation or information.  Impact looks at what  you would like to achieve e.g. improve team communication .

• How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information / Wildridge & Bell, 2002

Analysis tool for management and organisational strategy

PESTLE:  Political – Economic – Social – Technological – Environmental ‐ Legal

An analysis tool that can be used by organizations for identifying external factors which may influence their strategic development, marketing strategies, new technologies or organisational change.

• PESTLE analysis / CIPD, 2010

Service evaluations with qualitative study designs

SPICE:  Setting (context) – Perspective– Intervention – Comparison – Evaluation

Perspective relates to users or potential users. Evaluation is how you plan to measure the success of the intervention.

• Clear and present questions: formulating questions for evidence based practice / Booth, 2006

Read more about some of the frameworks for constructing review questions:

• Formulating the Evidence Based Practice Question: A Review of the Frameworks / Davis, 2011
• << Previous: Stages in a systematic review
• Next: Identifying studies >>
• Last Updated: Apr 4, 2024 10:09 AM
• URL: https://library-guides.ucl.ac.uk/systematic-reviews

• Open access
• Published: 16 May 2024

Use of the supportive and palliative care indicators tool (SPICT™) for end-of-life discussions: a scoping review

• Melanie Mahura 1 ,
• Brigitte Karle 2 ,
• Louise Sayers 3 ,
• Felicity Dick-Smith 3 &
• Rosalind Elliott 3 , 4  

BMC Palliative Care volume  23 , Article number:  119 ( 2024 ) Cite this article

82 Accesses

5 Altmetric

Metrics details

In order to mitigate the distress associated with life limiting conditions it is essential for all health professionals not just palliative care specialists to identify people with deteriorating health and unmet palliative care needs and to plan care. The SPICT™ tool was designed to assist with this.

The aim was to examine the impact of the SPICT™ on advance care planning conversations and the extent of its use in advance care planning for adults with chronic life-limiting illness.

In this scoping review records published between 2010 and 2024 reporting the use of the SPICT™, were included unless the study aim was to evaluate the tool for prognostication purposes. Databases searched were EBSCO Medline, PubMed, EBSCO CINAHL, APA Psych Info, ProQuest One Theses and Dissertations Global.

From the search results 26 records were reviewed, including two systematic review, two theses and 22 primary research studies. Much of the research was derived from primary care settings. There was evidence that the SPICT™ assists conversations about advance care planning specifically discussion and documentation of advance care directives, resuscitation plans and preferred place of death. The SPICT™ is available in at least eight languages (many versions have been validated) and used in many countries.

Conclusions

Use of the SPICT™ appears to assist advance care planning. It has yet to be widely used in acute care settings and has had limited use in countries beyond Europe. There is a need for further research to validate the tool in different languages.

Key message

What is already known on this topic?

• The SPICT ™ was developed to assist clinicians to screen patients for palliative care needs.

What this review found.

• The SPICT™ assists conversations about advance care planning and facilitates changes in documented goals of care.

• The SPICT™ is available in at least eight languages (and used in many countries.

How the findings of this review may affect practice and research.

• Evidence suggests that formalising screening for palliative care needs using the SPICT™ is advantageous for advance care planning; clinicians should consider using the SPICT™ to initiate discussions with people with life limiting conditions.

• Further research is required to validate the tool in different languages and extend its use in acute care settings and with other patient cohorts.

Peer Review reports

Introduction

The demand for palliative care services globally has outpaced service availability, particularly in low and middle-income countries [ 1 ]. This is expected to continue as the population ages and the burden of noncommunicable disease increases. Thus, non-specialist palliative care health professionals may be required to manage care. The Supportive And Palliative Care Indicators Tool (SPICT™) [ 2 ] is one instrument available for non-specialist palliative care clinicians which may assist them in assessing unmet palliative needs and care planning.

Evidence suggests that clinicians feel inadequately prepared to conduct end-of-life discussions with patients who are terminally ill [ 3 , 4 , 5 ] and are also unsure of the appropriate time to start these discussions or whether to involve a specialist palliative care team [ 5 , 6 , 7 ]. Clinicians have reported their discomfort when addressing the topic of death with seriously ill patients [ 5 ].

From the perspective of patients with advanced illness, honest information from a trusted health care professional is the preference of most [ 7 ]. A survey study conducted in Canada involving 434 patients with advanced illness found over half of patients felt it was ‘very important’ to have a sense of control over decision-making regarding their care and 56% felt it was ‘extremely important’ not to be kept alive on life support if there was little hope of recovery [ 7 ]. The default medical decision to do everything to save life may be contributing to delays in referral to a specialist palliative care team, burdensome medical treatment and poorer quality of life for many patients [ 8 ]. Thus, a standardised, reliable and validated method of assessing and planning care in collaboration with the patient is required.

The terms ‘end-of-life’ and ‘terminally ill’ have been conceptualised as synonymous and ‘apply to patients with progressive disease with months or less of expected survival.’ [ 9 ]. In the United States there is consensus that referral to specialist palliative care services is required at the time of diagnosis for patients with neurologic disease, frailty, multimorbidity, advanced cancer, organ or cognitive impairment, patients with a high symptom burden and patients with onerous family or caregiver needs [ 10 ]. However with an ageing population and increased levels of dementia and frailty non-palliative care clinicians need a tool with a common language to identify those who are nearing the end of life and to promote a palliative approach to care. According to the High Authority for Health, an independent organisation that promotes quality outcomes in the fields of health, sociology and medicine a palliative approach is, “a way of addressing end-of-life issues early on: make time to talk about ethical questions, psychological support, comfort care, the right care, and give a timely thought to the likely palliative care needs of people approaching the end-of-life.” [ 11 ], p1.

Advance care planning, “a process that supports adults at any age or stage of health in understanding and sharing their personal values, life goals, and preferences regarding future medical care” [ 12 ] is one aspect of palliative care often provided by medical professionals which may assist in ensuring people’s needs are met, and care and communication are enhanced. Early advance care planning is vital, particularly for patients with neurodegenerative conditions before they lose capacity to express their wishes [ 8 ] “to help ensure that people receive medical care that is consistent with their values, goals and preferences during serious and chronic illness.” [ 12 ] Research has revealed that patients who have had the opportunity to discuss their preferences at the end-of-life are more likely to receive care that is consistent with those preferences. Findings also include greater patient and carer satisfaction and less conflict regarding decision making when end-of-life preferences have been examined [ 13 ].

People who have life limiting conditions may benefit from the delivery of advance care planning using a systematic approach. The SPICT™, although not designed for this purpose may enhance the approach particularly when health professionals who have limited palliative care experience are required to facilitate advance care planning.

The SPICT™ [ 2 ] was designed to identify patients at risk of deteriorating or dying and to screen for unmet palliative care needs. The tool includes general indicators of deterioration and clinical indicators of life-limiting conditions. The accompanying SPICT™ guide provides prompts and tips and a suggested framework (REMAP Ready, Expect, Diagnosis, Matters, Actions and Plan) [ 14 ] for conducting future care planning conversations. The tool is reported to be simple to use and designed for use by all multidisciplinary team members in any care setting [ 13 ].

The SPICT™ was evaluated using a mixed methods participatory approach [ 2 ]. Peer review and consensus was gathered for the 15 revisions of the SPICT™ over an 18-month period. Each iteration of the tool was distributed to clinicians and policy makers internationally until consensus was reached [ 2 ]. The research team worked concurrently with clinicians in four participating units at an acute tertiary hospital in Scotland to screen all patients with advanced organ disease whose admission to hospital was unplanned ( n  = 130) using a checklist that included the SPICT™ general indicators, disease specific indicators and the surprise question (SQ), “Would you be surprised if this patient were to die in the next 6 to 12 months?”. Data were gathered over an 8-week period and patients were followed up for 12 months [ 2 ]. A significantly greater number of patients who died at 12-months had two or more admissions in the previous 6 months before being screened. These patients also had increased care needs and persistent symptoms despite optimal treatment. The researchers proposed that better identification, assessment and pre-emptive care planning could reduce the risk of unplanned hospital admission and prolonged inpatient stays [ 2 ]. Of note the patients’ diagnoses were limited to advanced illness which was non-malignant and ethnicity was homogenous [ 2 ]. The SQ was removed from subsequent versions of the SPICT™ and the rationale for removing it remains unclear. The SPICT™ continues to be revised and versions are available in different languages [ 2 ].

The intention of this review was to examine the impact of the SPICT™ on advance care planning and the extent of its use. The patient cohorts, languages, and contexts in which the SPICT™ is available and used were examined.

Review questions

The following primary question was addressed:

How does use of the SPICT™ assist with conversations about advance care planning?

Secondary review questions were:

What is the extent of the use of the SPICT™ (which patient cohorts, contexts, and countries is it used)?

In which languages has the spict™ been validated.

Does use of the SPICT™ facilitate changes in documented goals of care?

Design and methods

This scoping review was performed in accordance with the Joanna Briggs Institute Manual for Evidence Synthesis Scoping Review Framework [ 15 ] and the Meta-Analyses Scoping Review extension for scoping reviews (PRISMA-ScR) checklist [ 16 ] was used to guide the reporting.

Preliminary literature search

An initial search focussed on inpatients with a chronic illness nearing the end of life however the search was expanded to include all care settings where the SPICT™ was being used for adults with a life-limiting chronic illness to evaluate its efficacy in advance care planning. Thus the search reflected the International Association for Hospice and Palliative Care definition of palliative care “the active holistic care of individuals across all ages with serious health-related suffering due to severe illness, and especially of those near the end of life.” [ 17 ]. A life-limiting illness or condition encompasses both malignant and non-malignant diseases as well as the effects of ageing.

A preliminary search of EBSCO Medline, the Cochrane database of systematic reviews, Prospero and JBI Evidence Synthesis was conducted in June 2022. No current or planned systematic or scoping reviews specifically on this topic were identified. A systematic review by Teike Luthi, et al. [ 18 ], examining instruments for the identification of patients in need of palliative care in the hospital setting was identified. The current scoping review differs from the systematic review by Teike Luthi, et al. [ 18 ] as the aim was to identify and describe all research related to how the SPICT™ is used in end-of-life discussions and what influence this has on advance care planning and goals of care.

Inclusion criteria

Participants.

The population of interest was adult patients (> 18 years) with a life-limiting chronic illness.

The concept of interest was the SPICT™. Any studies incorporating the SPICT™ were included in this review since its development in 2010. Studies evaluating the SPICT™ for prognostication purposes were excluded as this was not the intent of this review.

Published and unpublished studies in any language for which a translation could be obtained were included. Published and unpublished studies in any setting that met the eligibility criteria were included.

Evidence sources

This scoping review included both experimental and quasi-experimental study designs. In addition, analytical observational studies including prospective and retrospective cohort studies, case-control studies and analytical cross-sectional studies were considered for inclusion. Systematic reviews that met the inclusion criteria were included. Qualitative studies, theses and dissertations were also considered if they met the inclusion criteria.

Search strategy

An initial search on this topic in the EBSCO Medline and PubMed databases was reviewed for relevant abstracts and titles to determine keywords and index terms. MESH terms used in the final search strategy included: Communication; Documentation; Palliative Care; Patient Care Planning; Advance Care Plan; Decision Making and Chronic Disease. The research abstract for this scoping review was registered on the Center for Open Science website ( https://doi.org/10.17605/OSF.IO/DN27C ) in August 2022 prior to performing the definitive search in September. The search was conducted on 28th September 2022 and date limited i.e., 2010-September 2022. The database and grey literature searches were updated on 27th January 2024 to identify further studies published beyond this date.

Electronic databases searched included EBSCO Medline, PubMed, EBSCO CINAHL, APA Psych Info, ProQuest One Theses and Dissertations Global. Publications listed on the SPICT website ( www.spict.org.uk ) were cross checked with the records included from the electronic databases, duplicates were removed and further records were added to the Endnote library for screening. Reference lists of included studies were reviewed for additional studies.

All websites searched for additional records (grey literature sources) are included in supplementary file 1 . The expanded search strategy for the EBSCO Medline database is also provided in supplementary file 1 .

Study selection

All records were collated in an EndNote library. Duplicate records were removed manually by RE. The screening process involved two independent reviewers (MM and RE) reading titles and abstracts. Full text screening was conducted independently by the same two reviewers. Any discrepancies between the two reviewers at each stage of the process was resolved following review and consultation of a third reviewer (BK). Studies that did not meet the inclusion criteria were excluded with a reason recorded. Data extracted from included studies has been recorded in the standardised data extraction form (supplementary file 2 ). Critical appraisal of included studies was not performed and thus studies were not excluded based on methodological quality.

Data synthesis

Key aspects of the included studies were summarised in tables. Also consistent with the approach for a scoping review a textual narrative synthesis [ 19 ] was performed with the primary aim of addressing the review questions.

Over 2,000 records were retrieved. Five guidelines and six conference abstracts were found but these either did not relate to the review questions or did not contain sufficient information to be included. After applying the exclusion criteria 26 reports were included in this scoping review. The flow diagram (Fig.  1 ) presents the number of records retrieved, screened, excluded and included.

Flow diagram of number of records retrieved, screened, excluded and included **Abstract and title screening involved assessing each record for relevance to the review questions i.e., if no mention of the SPICT™ or/and advance planning conversation the record was excluded from further consideration

There were multiple study designs including validation and translation ( n  = 8) studies [ 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 ] and clinical improvement projects ( n  = 3) [ 28 , 29 , 30 ]. The focus of the clinical improvement projects was to increase the identification of palliative care needs and care planning through the use of the SPICT™. Two reviews (one of these included a survey study) 18 31 and two theses were included 28 29 (Table  1 ).

Research reveals that the SPICT™ appears to assist clinicians with conversations about advance care planning by providing a proforma for essential aspects of end-of-life care, a framework for end-of-life conversations and a common language to collaborate within the multidisciplinary team.

For example, in a prospective exploratory feasibility study to explore the practical use of the SPICT™ resulted in increased palliative care planning [ 32 ]. In this study general practitioners (GPs) [ n  = 10] were trained in the use of the German version of the SPICT™ (SPICT-DE™) and during a two-month intervention period were asked to use the tool with any adult patients diagnosed with a life-limiting disease ( n  = 79) and these patients were followed up at 6 months. The GPs’ actions as recommended by the SPICT-DE™ were considered appropriate with the most frequent actions being “Agree a current and future care plan with the person and their family; support family carers” ( n  = 59 [75%)),“Review current treatment and medication to ensure the person receives optimal care; minimise polypharmacy”( n  = 53 [67%]), and “Plan ahead early if loss of decision-making capacity is likely”(n = 49 [62%]). Of note “Consider referral for specialist palliative care consultation to manage complex symptoms” was considered appropriate for 25 (32%) patients. The effect of the SPICT™ was evident at the 6-month follow-up; the most frequently initiated actions were “Review current treatment and medication to ensure the person receives optimal care; minimise polypharmacy” (n = 36 [46%]) and “Plan ahead early if loss of decision-making capacity is likely” (n = 29 [37%]).

Further implementation research by Afshar et al. [ 33 ] with GPs in Germany revealed that GPs considered that the tool supported the communication and coordination of care and considered it broadened their perspectives of the meeting the needs of people especially those with non-cancer diagnoses. Of note over 50% of patients in this study had their agreed care plan initiated at the 6-month follow-up. Some GPs who had extensive experience and training claimed that the tool had no effect on their practice. However overall more than two thirds of the sample reported that they could envisage using the SPICT-DE™ in everyday practice.

In addition, three studies found that nurses who were trained to use the SPICT™ increased their self-efficacy in identifying patients who may be nearing the end of life and promoted an advance care plan discussion with these patients 28 29 34 . 21−23 In the study set in a renal ward, patients were screened on admission to identify those nearing the end of life by nurses using the SPICT™ [ 34 ]. An alert was added to the ward patient name list when a patient was identified as nearing the end of life (‘SPICT™ positive’) which prompted a review by the physician and multidisciplinary team. In this study 16% (25/152) of newly admitted patients were screened as ‘SPICT™ positive’; all of these patients received a palliative care consult and were discharged with an advance care directive including a resuscitation plan [ 34 ]. Incidentally nurses reported a significant increase in their ability to identify patients approaching end of life.

Similarly high SPICT™ screening rates and end of life conversations and referrals were revealed in a clinical improvement project designed to improve palliative care screening and consultation on admission to the cardiopulmonary unit of a long-term acute care facility using the SPICT™ [ 29 ]. In this project involving patients requiring mechanical ventilation and cardiac monitoring, 83% (59/71) of nurses working in the unit were trained in the use of the tool and screened all 50 newly admitted patients in the study period, 48 of whom were ‘SPICT™ positive’. Only 7 received a palliative care consultation within a week of admission however all 7 of these patients received a resuscitation plan and an advanced directive. Of note the use of the SPICT™ for screening resulted in a doubling of the facility’s monthly average number of palliative care referrals (from 32 to 84). In another clinical improvement project designed to increase screening and referral for palliative care among ambulatory care patients, nurse practitioners found the SPICT™ ‘. opens the door to a discussion of palliation .’ and was ‘. helpful in determining eligibility for palliative care. .’ p 22 28 . This project using both quantitative and qualitative approaches revealed an increase in palliative care referrals from 16% ( n  = 8/50) to 50% ( n  = 25/50) after the SPICT™ was introduced.

Two studies designed to translate and validate the SPICT-DK™ (Danish) [ 21 ] and SPICT-SE™ (Swedish) [ 24 ] involving focus groups with health care professionals revealed positive responses from doctors and nurses. The tool was described as a linguistic framework among these professionals and that use of the SPICT™ gave them a common language in which to collaborate and focus when treating and caring for patients [ 21 ]. The specificity of the tool was highlighted by nurses and medical doctors [ 24 ].

Conversely the expert committee comprising family physicians and palliative and home care specialists who provided input to the translation and cross-cultural adaptation of the SPICT™ into Japanese were more circumspect [ 27 ]. These experts were concerned that the tool might not be appropriate for framing advance planning conversations as a ‘not-telling the truth’ culture was prevalent and health care was heavily siloed into specialities so that care planning was fragmented.

The SPICT™ has been used to screen for palliative care needs in many patient cohorts, settings and countries. The cohorts in which the SPICT™ has been used include people over 65 years [ 35 ], those with advanced cancer 32 36 and with chronic diseases including cardiovascular disease [ 28 ], renal disease [ 34 ] and pulmonary disease [ 29 ].

Ten of the included studies were conducted in primary care and general practice settings 20 , 21 , 22 24 25 30 , 31 , 32 37 38 . The SPICT™ was also used in outpatient clinic settings 23 28 39 and residential aged care 29 35 . One cross sectional survey of community households in India used the SPICT™ to identify patients with palliative care needs in two rural communities [ 40 ]. The SPICT™ was originally developed for use in a hospital setting but not formally validated during its development [ 2 ]. All of the contexts in which the SPICT™ has been used are listed in Table  1 .

Of the included records ten were studies conducted in European countries 20 , 21 , 22 24 30 , 31 , 32 , 33 37 38 ; seven in Asia 23 25 27 39 , 40 , 41 , 42 ; three in the USA 28 29 36 ; two in Australia 34 35 ; one in South Africa [ 43 ] one in Chile [ 26 ] and one in Peru [ 44 ], and one paper was a review performed by authors based in Switzerland [ 18 ]. Of note the systematic review and survey of European primary care GP practice to identify patients for palliative care revealed that the United Kingdom was the only European country at the time that incorporated the SPICT™ to identify palliative care needs in primary and secondary care in clinical guidelines [ 31 ].

The SPICT ™ has been translated, cross culturally adapted and validated to identify patients with palliative care needs in Danish [ 21 ] and German [ 38 ] using the Translation, Review, Adjudication, Pre-testing and Documentation (TRAPD) model. Another study by Afshar, et al. [ 32 ] further established the validity of the SPICT-DE ™ in German in general practice with a patient cohort. In addition the SPICT ™ has been translated from English to Italian [SPICT-IT ™ ] [ 22 ], Spanish [SPICT-ES ™ ] [ 20 ], Swedish [ 24 ] and Japanese [SPICT-J ™ ] [ 23 ] using the Beaton protocol for cross cultural adaptation of health measures [ 45 ]. Farfán-Zuñiga and Zimmerman-Vildoso [ 26 ] established the reliability and validity of the SPICT-ES CHTM after culturally adapting the SPICT-ES ™ using the Beaton protocol. Nurses positively evaluated the feasibility of the tool. In addition Oishi et al. [ 27 ] also performed a translation and cross-cultural adaptation of the SPICT ™ into Japanese using a similar approach. The forward-back translated Indonesian version of the tool was found to be highly reliable and valid and greatly assisted in identifying hospital patients’ unmet palliative care needs [ 41 ].

The SPICT™ for low-income settings (LIS) was translated and cross culturally adapted for use in Thailand [ 25 ]. The interrater reliability of the final SPICT-LIS™ Thai version was high when nurses and GPs used it to ascertain palliative care needs of patients in case vignettes.

A Delphi study was used to develop the SPICT™ for the South African context [ 43 ]. Modifications to the original tool included the addition of haematological and infectious diseases and trauma however the SPICT-SA™ has yet to be validated in these patient cohorts. Although not a validated study per se in research comparing the performance of the Dutch version of the tool (SPICT-NL™) and the SQ in general practice ( n  = 3,640) the SPICT™ was found to be superior to the SQ in identifying patients with palliative care needs particularly younger people [ 37 ].

Of note the SPICT4-ALL™ [ 46 ] is a simplified version of the original SPICT™ developed for family/friends and care staff to identify individual palliative care needs. It is available to download from the SPICT™ website in English, German, Danish and Spanish. Although Sudhakaran, et al. [ 40 ] successfully used it to identify palliative care needs in two communities in rural India. No studies validating or evaluating it were found in our search.

Does use of the SPICT ™ facilitate changes in documented goals of care?

There is evidence that the SPICT™ by virtue of assisting clinicians to discuss end of life care facilitates changes in documented goals of care. Specifically this was demonstrated in a pre-post intervention study in which GPs trained in palliative care and the use of the SPICT-DE™ were requested to use it in their everyday practice for 12 months with every adult patient diagnosed with a chronic, progressive disease [ 30 ]. This occurred concurrently with a public campaign focused on informing health care providers and stake holders in two counties in Germany about end-of-life care. GPs’ documentation improved after the intervention. Records of care planning increased from 33 to 51% and documentation of preferred place of death towards the end-of-life increased from 20 to 33% and patients’ wishes, and spiritual beliefs increased from 18 to 27%. Incidentally GPs’ self-reported quality of end-of-life care increased after the implementation of the SPICT-DE™ and the information campaign [ 30 ].

In a study including 187 residents in an aged care facility in Australia comparing the SPICT™ and SQ, two Directors of Nursing pre-screened residents using the SQ and if the response was ‘yes’ (SQ+) applied the SPICT™ [ 35 ]. Of the 80 (43%) residents who were SQ+, 100% of these showed signs of nearing end of life according to the SPICT™. Of these residents 39 (49%) had some form of palliative care from either GPs, a specialist palliative care physician or palliative care nurse. Nearly all 39 (97%) had a GP management plan, and 67% had an advance care directive and 67% had discussed treatment options with their care provider [ 29 ]. It is unclear whether the SPICT™ affected care planning or documentation as the study involved pre-screening with the SQ and documentation was not assessed before and after this intervention.

Death and dying are taboo in many countries and thus any discussion about end of life is challenging. However, clinicians are morally and ethically obliged to appropriately initiate discussions about advance care planning towards the end of life when patients are ready [ 47 ]. This review found that the SPICT™ may help the clinician with this conversation. Specifically, evidence suggests that the tool may be a useful proforma and a conversation ‘checklist’ to ensure that the priority areas for advance care planning are addressed. Specifically, the tool may enable an assessment of the person’s readiness to have an advance planning conversation and an exploration of their expectations, the diagnosis, what matters to them, treatment options and future plans [ 14 ]. Importantly extensive specialist training is not required to administer it; the studies in this review employed brief information interventions to prepare clinicians to use the SPICT™. Thus, the SPICT™ provides a method of ‘objectively’ assessing palliative care needs, articulating the requirement for a specialist palliative review if required and advance care planning.

This review found that the SPICT™ was used in mainly primary health care settings and predominately in European countries. Of note there were few published records of its use in countries in the Asian and African continents and North America. The tool has been translated into more than eight languages including Spanish (SPICT-ES™) [ 20 ], Italian (SPICT-IT™) [ 22 ], German (SPICT-DE™) [ 38 ] and Japanese (SPICT-J™) [ 23 ] although not all versions have been formally validated [ 25 , 26 , 27 , 33 ].

Furthermore, there is evidence to suggest that the SPICT™ may facilitate changes in the goals of care and documentation of end of life care planning and patient wishes. Incidentally the SPICT™ appears to be positively received by clinicians with some suggesting that the tool provides a common language for clinicians when collaborating to identify palliative care needs and provide palliative care.

Of note the tool did not meet the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria [ 18 ]. However arguably these criteria may not be the most appropriate criteria on which to base an assessment of the SPICT™ given that it was never meant to be used to objectively measure a parameter such as prognosis; Highet et al. [ 2 ] were clear about the remit of the tool i.e., “help clinicians working in primary and secondary care recognise when their patients might be at risk of dying and likely to benefit from supportive and palliative care in parallel with appropriate ongoing management of their advanced conditions.” [ 2 ], p11.

There is an imperative to improve recognition of palliative care needs particularly in fast paced acute care settings. Evidence suggests that a tool such as the SPICT™ is an important adjunct for initiating a conversation about end-of-life care and ensuring that key palliative care needs are identified. Importantly the SPICT™ requires little training and its brevity may be suited to settings in which there is limited opportunity to engage in lengthy conversations and in which clear unambiguous communication is key to timely referral and treatment. Formalising palliative care needs screening in an end-of-life conversation in acute care settings may reduce distress for patients and their informal care givers [ 48 ] and the SPICT™ is a relevant proforma for such a conversation. Furthermore with the increase in the numbers of people living with chronic illness globally [ 49 ] arguably the formal adoption of palliative care needs screening in all health care settings may not only reduce patient distress but may assist health care managers and policy makers to more appropriately plan services [ 50 ]. Identifying needs early in the illness trajectory may allow appropriate personalised care and services to be provided in a timely and cost effective manner thus avoiding health crises at the end of life [ 51 ].

Conversely caution should be exercised when recommending a tool to guide advance care planning and end of life conversations particularly in the setting of low health professional skill level. This was highlighted by experienced GP participants in the study by Afshar et al. [ 33 ]. The GPs did not consider that the SPICT-DE™ made any impact on their practice. A proforma or guideline cannot replace the need for exemplary health care professional communication during advance care planning and end of life conversations particularly as studies reporting the use of the SPICT™ were not specifically focused on testing its efficacy in this regard. Flexibility and sensitivity are required to assess and manage people with life limiting conditions to ensure care is individualised. Thus, a sufficiently trained and resourced workforce is vital in addition to aids such as the SPICT™.

In addition, although not the focus of this review we noticed that there was an apparent lack of attention paid to input from the family and consideration of the family context in the included studies. In practice the advance care planning conversation goes beyond using the family to identify palliative care needs and the requirement for referral. The conversation should include addressing family members’ concerns and emotions and facilitate communication between the person who is the focus of advance care planning and their family members [ 52 ].

There are translations of the English version of the SPICT™ available to download from the SPICT™ website for a number of countries including; Brazil, France, Greece, Portugal and South Africa. However, studies reporting the use of many versions of the SPICT™ indicates that formal validation has not been performed. Further validation may strengthen the efficacy and reputation of these versions of the tool. Further studies are required to establish the validity of translated versions of the SPICT™ in Swedish, Danish, Indonesian and the SPICT-LIS™ (Thai), for everyday use in other patient cohorts.

The SPICT™ has scope to be tested in other patient cohorts. Specifically more work is required to extend and test its use in acute care settings where the demand for palliative care is rising and appropriate timely referrals to specialist palliative care are vital to avoid unnecessary distress [ 51 ]. Similarly, there are research opportunities such as reliability and validity testing in relation to the SPICT-4ALL™ version which has been specifically designed to be used by family and informal carers.

Strengths and limitations

This review has strengths which warrant consideration. For example, a systematic approach based on the PRISMA-SCr methodological framework was used, and the search was extensive including 5 electronic databases and many sources of grey literature. A limitation of this review is that we were unable to access a healthcare librarian to assist with the search thus important records may have been missed. In addition, we did not have funding to arrange the translation of two studies which were identified as potentially eligible. Studies included in this scoping review were not appraised for bias thus the level of evidence for the effectiveness of the SPICT™ was not reported. Of note most studies were descriptive and thus evidence for the effectiveness in relation to review question 1 (how does the tool assist with conversations about advance care planning?) is not available.

The current scoping review aimed to assess the impact and extent of the use of the SPICT™. In summary the SPICT™ appears to enable advance care planning, review of care plans and initiation of palliative care in many settings. Previous research suggests that patients and their families greatly appreciate the opportunity to discuss end of life matters. The SPICT™ provides clinicians a proforma on which to base this conversation and a common language to collaborate for palliative care. Clinicians with advance care planning and end of life communication in all settings should consider using the SPICT™ for this purpose. Future research should focus on further validating the SPICT™ in more patient cohorts and acute care settings. Further testing of the tool beyond Europe in countries in Africa, Asia and North America is also warranted.

Data availability

Not applicable.

Worldwide Palliative Care Alliance. In: Connor S, editor. Global atlas of Palliative Care. 2nd ed. London, UK: World Health Organisation; 2020. p. 120.

Google Scholar  

Highet G, Crawford D, Murray SA, et al. Development and evaluation of the supportive and Palliative Care indicators Tool (SPICT): a mixed-methods study. BMJ Supportive Palliat Care. 2014;4(3):285–90. https://doi.org/10.1136/bmjspcare-2013-000488 .

Article   Google Scholar  

Le B, Mileshkin L, Doan K, Saward D, Sruyt O, Yoong J, Gunawardana D, Conron M, Phillip J. Acceptability of early integration of palliative care in patients with incurable lung cancer. J Palliat Med. 2014;17(5). https://doi.org/10.1089/jpm.2013.0473 .

Perrin KO, Kazanowski M. Overcoming barriers to Palliative Care Consultation. Crit Care Nurse. 2015;35(5):44–52. https://doi.org/10.4037/ccn2015357 .

Article   PubMed   Google Scholar  

Scholz B, Goncharov L, Emmerich N, et al. Clinicians’ accounts of communication with patients in end-of-life care contexts: a systematic review. Patient Educ Couns. 2020;103(10):1913–21. https://doi.org/10.1016/j.pec.2020.06.033 .

Aldridge MD, Hasselaar J, Garralda E, et al. Education, implementation, and policy barriers to greater integration of palliative care: a literature review. Palliat Med. 2016;30(3):224–39. https://doi.org/10.1177/0269216315606645 . [published Online First: 2015/09/26].

Heyland D, Dodek, Rocker G, Groll D, Gafni A, Pachora D, Shortt S, Tranmer J, Lazar N, Kutsogiannis A, Lam M. What matters most in end-of-life care: perceptions of seriously ill patients and their family members. CMAJ. 2006;174(5):627–33.

Article   PubMed   PubMed Central   Google Scholar  

Salins N, Ramanjulu R, Patra L, et al. Integration of early specialist palliative care in cancer care and patient related outcomes: a critical review of evidence. Indian J Palliat care. 2016;22(3). https://doi.org/10.4103/0973-1075.185028 .

Hui D, Nooruddin Z, Didwaneya N, Dev R, De La Cruz M, Kim SH, Kwon JH, Hutchins R, Liem C, Bruera E. Concepts and definitions for actively dying, end of life, terminally ill, terminal care, and transition of care: a systematic review. J Pain Symptom Manag 2014;47(1).

Kelly A, Morrison R. Palliative care for the seriously ill. N Engl J Med. 2015;373. https://doi.org/10.1056/NEJMra1404684 .

Haute autorite de sante. ORGANISATION OF PATHWAYS - essentials of the palliative approach. TOOL TO IMPROVE PROFESSIONAL PRACTICE; 2016.

Sudore RL, Lum HD, You JJ, et al. Defining Advance Care planning for adults: a Consensus Definition from a Multidisciplinary Delphi Panel. J Pain Symptom Manag. 2017;53(5):821–e321. https://doi.org/10.1016/j.jpainsymman.2016.12.331 .

Boyd K, Murray S. Recognising and managing key transitions in end-of-life care. BMJ. 2010;341. https://doi.org/10.1136/bmj.c4863 .

Group PPCR. The SPICT ™ Supportive and palliative care indicators tool Edinburgh, Scotland: University of Edinburgh; 2023. https://www.spict.org.uk/the-spict/ accessed 05 Feb 2024.

Peters MDJ, Godfrey C, McInerney P, et al. In: Munn EAZ, editor. JBI Manual for evidence synthesis. Scoping reviews: Joanna Briggs Institute; 2020. p. 45.

Tricco A, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMAScR): Checklist and explanation. Ann Intern Med. 2018;169:467–73. https://doi.org/10.7326/M18-0850 .

IAHPC. Global Consensus based palliative care definition Houston, TX, USA: The International Association for Hospice and Palliative Care. 2018. https://hospicecare.com/what-we-do/projects/consensus-based-definition-of-palliative-care/definition/ accessed 10 July 2023.

Teike Luthi F, MacDonald I, Rosselet Amoussou J, et al. Instruments for the identification of patients in need of palliative care in the hospital setting: a systematic review of measurement properties. JBI Evid Synth. 2022;20(3):761–87. https://doi.org/10.11124/JBIES-20-00555 .

Lucas PJ, Baird J, Arai L, et al. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Methodol. 2007;7(1):4. https://doi.org/10.1186/1471-2288-7-4 .

Fachado A, Alonso, Martínez NS, Roselló MM, et al. Spanish adaptation and validation of the supportive & palliative care indicators tool – SPICT-ES™. Rev Saúde Pública. 2018;52(0):3. https://doi.org/10.11606/S1518-8787.2018052000398 .

Bergenholtz H, Weibull A, Raunkiaer M. Supportive and palliative care indicators tool (SPICT) in a Danish healthcare context: translation, cross-cultural adaptation, and content validation. BMC Palliat Care. 2022;21(1):41. https://doi.org/10.1186/s12904-022-00931-6 . [published Online First: 2022/03/26].

Casale G, Magnani C, Fanelli R, et al. Supportive and palliative care indicators tool (SPICT): content validity, feasibility and pre-test of the Italian version. BMC Palliat Care. 2020;19(1):79. https://doi.org/10.1186/s12904-020-00584-3 . [published Online First: 2020/06/09].

Hamano J, Oishi A, Kizawa Y. Identified Palliative Care Approach needs with SPICT in Family Practice: a preliminary observational study. J Palliat Med. 2018;21(7):992–98. https://doi.org/10.1089/jpm.2017.0491 . [published Online First: 2018/02/10].

Pham L, Arnby M, Benkel I, et al. Early integration of palliative care: translation, cross-cultural adaptation and content validity of the supportive and Palliative Care indicators Tool in a Swedish healthcare context. Scand J Caring Sci. 2020;34(3):762–71. https://doi.org/10.1111/scs.12781 . [published Online First: 2019/11/02].

Sripaew S, Fumaneeshoat O, Ingviya T. Systematic adaptation of the Thai version of the supportive and palliative care indicators tool for low-income setting (SPICT-LIS). BMC Palliat Care. 2021;20(1):35. https://doi.org/10.1186/s12904-021-00729-y . [published Online First: 2021/02/21].

Farfán-Zuñiga X, Zimmermann-Vildoso M. Cultural adaptation and validation of the SPICT-ES™ instrument to identify palliative care needs in Chilean older adults. BMC Palliat Care. 2022;21(1):223. https://doi.org/10.1186/s12904-022-01111-2 . [published Online First: 2022/12/17].

Oishi A, Hamano J, Boyd K, et al. Translation and cross-cultural adaptation of the supportive and Palliative Care indicators Tool into Japanese: a preliminary Report. Palliat Med Rep. 2022;3(1):1–5. https://doi.org/10.1089/pmr.2021.0083 . [published Online First: 2022/09/06].

Clark NH. Palliative Care: improving Eligible Patient Identification to encourage early intervention [Capstone]. Northern Arizona University; 2019.

Dinega G. Improving palliative care utilization processes: An effort to enhance nurses’ contribution [D.N.P]. The College of St. Scholastica; 2016.

van Baal K, Wiese B, Muller-Mundt G, et al. Quality of end-of-life care in general practice - a pre-post comparison of a two-tiered intervention. BMC Prim Care. 2022;23(1):90. https://doi.org/10.1186/s12875-022-01689-9 . [published Online First: 2022/04/22].

Maas EA, Murray SA, Engels Y, et al. What tools are available to identify patients with palliative care needs in primary care: a systematic literature review and survey of European practice. BMJ Support Palliat Care. 2013;3(4):444–51. https://doi.org/10.1136/bmjspcare-2013-000527 . [published Online First: 2014/06/21].

Afshar K, Wiese B, Schneider N, et al. Systematic identification of critically ill and dying patients in primary care using the German version of the supportive and Palliative Care indicators Tool (SPICT-DE). Ger Med Sci. 2020;18:Doc02. https://doi.org/10.3205/000278 . [published Online First: 2020/02/13].

Afshar K, van Baal K, Wiese B, et al. Structured implementation of the supportive and Palliative Care indicators Tool in general practice – a prospective interventional study with follow-up. BMC Palliat Care. 2022;21(1):214. https://doi.org/10.1186/s12904-022-01107-y .

Lunardi L, Hill K, Crail S, et al. Supportive and Palliative Care indicators Tool (SPICT) improves renal nurses’ confidence in recognising patients approaching end of life. BMJ Support Palliat Care. 2020. https://doi.org/10.1136/bmjspcare-2020-002496 . [published Online First: 2020/11/05].

Liyanage T, Mitchell G, Senior H. Identifying palliative care needs in residential care. Aust J Prim Health. 2018;24(6):524–29. doi: 10.1071/PY17168 [published Online First: 2018/11/14].

Chan AS, Rout A, ’Adamo CRD, et al. Palliative referrals in Advanced Cancer patients: utilizing the supportive and Palliative Care indicators Tool and Rothman Index. Am J Hospice Palliat Medicine®. 2022;39(2):164–68. https://doi.org/10.1177/10499091211017873 .

van Wijmen MPS, Schweitzer BPM, Pasman HR, et al. Identifying patients who could benefit from palliative care by making use of the general practice information system: the Surprise question versus the SPICT. Fam Pract. 2020;37(5):641–47. https://doi.org/10.1093/fampra/cmaa049 . [published Online First: 2020/05/20].

Afshar K, Feichtner A, Boyd K, et al. Systematic development and adjustment of the German version of the supportive and Palliative Care indicators Tool (SPICT-DE). BMC Palliat Care. 2018;17(1):27. https://doi.org/10.1186/s12904-018-0283-7 .

Fumaneeshoat O, Ingviya T, Sripaew S. Prevalence of Cancer Patients Requiring Palliative Care in Outpatient Clinics in a Tertiary Hospital in Southern Thailand. 2021 2021;39(5):11. https://doi.org/10.31584/jhsmr.2021798 [published Online First: 2021-07-02].

Sudhakaran D, Shetty RS, Mallya SD, et al. Screening for palliative care needs in the community using SPICT. Med J Armed Forces India. 2023;79(2):213–19. [published Online First: 2023/03/28].

Effendy C, Silva JFDS, Padmawati RS. Identifying palliative care needs of patients with non-communicable diseases in Indonesia using the SPICT tool: a descriptive cross-sectional study. BMC Palliat Care. 2022;21(1):13. https://doi.org/10.1186/s12904-021-00881-5 .

Pairojkul S, Thongkhamcharoen R, Raksasataya A, et al. Integration of specialist Palliative Care into Tertiary hospitals: a Multicenter Point Prevalence Survey from Thailand. Palliat Med Rep. 2021;2(1):272–79.

Krause R, Barnard A, Burger H, et al. A Delphi study to guide the development of a clinical indicator tool for palliative care in South Africa. Afr J Prim Health Care Fam Med. 2022;14(1):e1–7. https://doi.org/10.4102/phcfm.v14i1.3351 . [published Online First: 2022/06/14].

Pinedo-Torres I, Intimayta-Escalante C, Jara-Cuadros D, et al. Association between the need for palliative care and chronic diseases in patients treated in a Peruvian hospital. Revista peruana de medicina Experimental y Salud pública. 2021;38(4):569–76. https://doi.org/10.17843/rpmesp.2021.384.9288 .

Beaton DE, Bombardier C, Guillemin F, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186–91. https://doi.org/10.1097/00007632-200012150-00014 . [published Online First: 2000/12/22].

Article   CAS   PubMed   Google Scholar  

University of Edinburgh. SPICT-4ALL Edinburgh, Scotland, 2023. https://www.spict.org.uk/spict-4all/ accessed 04 July 2023.

Zwakman M, Jabbarian LJ, van Delden J, et al. Advance care planning: a systematic review about experiences of patients with a life-threatening or life-limiting illness. Palliat Med. 2018;32(8):1305–21. https://doi.org/10.1177/0269216318784474 . [published Online First: 2018/06/30].

Article   CAS   PubMed   PubMed Central   Google Scholar  

Wright AA, Zhang B, Ray A, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665–73. https://doi.org/10.1001/jama.300.14.1665 . [published Online First: 2008/10/09].

Hajat C, Stein E. The global burden of multiple chronic conditions: a narrative review. Prev Med Rep. 2018;12:284–93. https://doi.org/10.1016/j.pmedr.2018.10.008 . [published Online First: 2018/11/09].

Taghavi M, Johnston G, Urquhart R, et al. Workforce planning for community-based Palliative Care specialist teams using Operations Research. J Pain Symptom Manag. 2021;61(5):1012–e224. https://doi.org/10.1016/j.jpainsymman.2020.09.009 .

Hui D, Heung Y, Bruera E. Timely Palliative Care: personalizing the process of Referral. Cancers. 2022;14(4):1047.

Kishino M, Ellis-Smith C, Afolabi O, et al. Family involvement in advance care planning for people living with advanced cancer: a systematic mixed-methods review. Palliat Med. 2022;36(3):462–77. https://doi.org/10.1177/02692163211068282 . [published Online First: 2022/01/07].

Download references

Acknowledgements

The work involved in performing this review was not funded.

Author information

Authors and affiliations.

Avondale University, Wahroonga, Sydney, NSW, Australia

Melanie Mahura

Hammond Care, Greenwich, Sydney, NSW, Australia

Brigitte Karle

Royal North Shore Hospital, St. Leonards, Sydney, NSW, Australia

Louise Sayers, Felicity Dick-Smith & Rosalind Elliott

University of Technology Sydney, Ultimo, Sydney, NSW, Australia

Rosalind Elliott

You can also search for this author in PubMed   Google Scholar

Contributions

We declare that this paper has not been published elsewhere and we have not submitted the paper to any other journal at the time of this submission. In addition, we Rosalind Elliott (RE), Melanie Mahura (MM), Brigitte Karle (BK), Felicity Dick-Smith (F D-S) and Louise Sayers (LS), declare that we have no known conflicts of interest in relation to the review or manuscript.All authors contributed to the research and reporting. MM lead the protocol design with input from all authors (RE, BK, FD-S and LS) and the search. MM and RE screened the search records with BK and LS acting as arbitrator. MM and BK extracted the data. Synthesis and interpretation were performed mainly by MM and BK with input from all authors. The writing was lead by MM with contributions from all authors (RE, BK, FD-S and LS). RE provided mentorship and thoroughly edited the final version. All authors (MM, RE, BK, FD-S and LS) reviewed the final version and gave their final approval for submission of the paper.

Corresponding author

Correspondence to Rosalind Elliott .

Ethics declarations

Consent for publication, ethical approval and consent to participate, ethical guidelines.

The International Medical Journal Editors’ Committee guidelines for authorship were followed.

Author’s information (optional)

Conflict of interest.

The authors know of no conflicts of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mahura, M., Karle, B., Sayers, L. et al. Use of the supportive and palliative care indicators tool (SPICT™) for end-of-life discussions: a scoping review. BMC Palliat Care 23 , 119 (2024). https://doi.org/10.1186/s12904-024-01445-z

Download citation

Received : 17 August 2023

Accepted : 25 April 2024

Published : 16 May 2024

DOI : https://doi.org/10.1186/s12904-024-01445-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Communication
• Documentation
• Palliative Care
• Patient Care Planning
• Terminal Care

BMC Palliative Care

ISSN: 1472-684X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Study and research support
• Academic skills

Dissertation examples

Listed below are some of the best examples of research projects and dissertations from undergraduate and taught postgraduate students at the University of Leeds We have not been able to gather examples from all schools. The module requirements for research projects may have changed since these examples were written. Refer to your module guidelines to make sure that you address all of the current assessment criteria. Some of the examples below are only available to access on campus.

• Undergraduate examples
• Taught Masters examples