what should a systematic literature review include

Reference management. Clean and simple.

How to write a systematic literature review [9 steps]

What is a systematic literature review?

Where are systematic literature reviews used, what types of systematic literature reviews are there, how to write a systematic literature review, 1. decide on your team, 2. formulate your question, 3. plan your research protocol, 4. search for the literature, 5. screen the literature, 6. assess the quality of the studies, 7. extract the data, 8. analyze the results, 9. interpret and present the results, registering your systematic literature review, frequently asked questions about writing a systematic literature review, related articles.

A systematic literature review is a summary, analysis, and evaluation of all the existing research on a well-formulated and specific question.

Put simply, a systematic review is a study of studies that is popular in medical and healthcare research. In this guide, we will cover:

• the definition of a systematic literature review
• the purpose of a systematic literature review
• the different types of systematic reviews
• how to write a systematic literature review

➡️ Visit our guide to the best research databases for medicine and health to find resources for your systematic review.

Systematic literature reviews can be utilized in various contexts, but they’re often relied on in clinical or healthcare settings.

Medical professionals read systematic literature reviews to stay up-to-date in their field, and granting agencies sometimes need them to make sure there’s justification for further research in an area. They can even be used as the starting point for developing clinical practice guidelines.

A classic systematic literature review can take different approaches:

• Effectiveness reviews assess the extent to which a medical intervention or therapy achieves its intended effect. They’re the most common type of systematic literature review.
• Diagnostic test accuracy reviews produce a summary of diagnostic test performance so that their accuracy can be determined before use by healthcare professionals.
• Experiential (qualitative) reviews analyze human experiences in a cultural or social context. They can be used to assess the effectiveness of an intervention from a person-centric perspective.
• Costs/economics evaluation reviews look at the cost implications of an intervention or procedure, to assess the resources needed to implement it.
• Etiology/risk reviews usually try to determine to what degree a relationship exists between an exposure and a health outcome. This can be used to better inform healthcare planning and resource allocation.
• Psychometric reviews assess the quality of health measurement tools so that the best instrument can be selected for use.
• Prevalence/incidence reviews measure both the proportion of a population who have a disease, and how often the disease occurs.
• Prognostic reviews examine the course of a disease and its potential outcomes.
• Expert opinion/policy reviews are based around expert narrative or policy. They’re often used to complement, or in the absence of, quantitative data.
• Methodology systematic reviews can be carried out to analyze any methodological issues in the design, conduct, or review of research studies.

Writing a systematic literature review can feel like an overwhelming undertaking. After all, they can often take 6 to 18 months to complete. Below we’ve prepared a step-by-step guide on how to write a systematic literature review.

• Decide on your team.
• Formulate your question.
• Plan your research protocol.
• Search for the literature.
• Screen the literature.
• Assess the quality of the studies.
• Extract the data.
• Analyze the results.
• Interpret and present the results.

When carrying out a systematic literature review, you should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

You may also need to team up with a librarian to help with the search, literature screeners, a statistician to analyze the data, and the relevant subject experts.

Define your answerable question. Then ask yourself, “has someone written a systematic literature review on my question already?” If so, yours may not be needed. A librarian can help you answer this.

You should formulate a “well-built clinical question.” This is the process of generating a good search question. To do this, run through PICO:

• Patient or Population or Problem/Disease : who or what is the question about? Are there factors about them (e.g. age, race) that could be relevant to the question you’re trying to answer?
• Intervention : which main intervention or treatment are you considering for assessment?
• Comparison(s) or Control : is there an alternative intervention or treatment you’re considering? Your systematic literature review doesn’t have to contain a comparison, but you’ll want to stipulate at this stage, either way.
• Outcome(s) : what are you trying to measure or achieve? What’s the wider goal for the work you’ll be doing?

Now you need a detailed strategy for how you’re going to search for and evaluate the studies relating to your question.

The protocol for your systematic literature review should include:

• the objectives of your project
• the specific methods and processes that you’ll use
• the eligibility criteria of the individual studies
• how you plan to extract data from individual studies
• which analyses you’re going to carry out

For a full guide on how to systematically develop your protocol, take a look at the PRISMA checklist . PRISMA has been designed primarily to improve the reporting of systematic literature reviews and meta-analyses.

When writing a systematic literature review, your goal is to find all of the relevant studies relating to your question, so you need to search thoroughly .

This is where your librarian will come in handy again. They should be able to help you formulate a detailed search strategy, and point you to all of the best databases for your topic.

➡️ Read more on on how to efficiently search research databases .

The places to consider in your search are electronic scientific databases (the most popular are PubMed , MEDLINE , and Embase ), controlled clinical trial registers, non-English literature, raw data from published trials, references listed in primary sources, and unpublished sources known to experts in the field.

➡️ Take a look at our list of the top academic research databases .

Tip: Don’t miss out on “gray literature.” You’ll improve the reliability of your findings by including it.

Don’t miss out on “gray literature” sources: those sources outside of the usual academic publishing environment. They include:

• non-peer-reviewed journals
• pharmaceutical industry files
• conference proceedings
• pharmaceutical company websites
• internal reports

Gray literature sources are more likely to contain negative conclusions, so you’ll improve the reliability of your findings by including it. You should document details such as:

• The databases you search and which years they cover
• The dates you first run the searches, and when they’re updated
• Which strategies you use, including search terms
• The numbers of results obtained

➡️ Read more about gray literature .

This should be performed by your two reviewers, using the criteria documented in your research protocol. The screening is done in two phases:

• Pre-screening of all titles and abstracts, and selecting those appropriate
• Screening of the full-text articles of the selected studies

Make sure reviewers keep a log of which studies they exclude, with reasons why.

➡️ Visit our guide on what is an abstract?

Your reviewers should evaluate the methodological quality of your chosen full-text articles. Make an assessment checklist that closely aligns with your research protocol, including a consistent scoring system, calculations of the quality of each study, and sensitivity analysis.

The kinds of questions you'll come up with are:

• Were the participants really randomly allocated to their groups?
• Were the groups similar in terms of prognostic factors?
• Could the conclusions of the study have been influenced by bias?

Every step of the data extraction must be documented for transparency and replicability. Create a data extraction form and set your reviewers to work extracting data from the qualified studies.

Here’s a free detailed template for recording data extraction, from Dalhousie University. It should be adapted to your specific question.

Establish a standard measure of outcome which can be applied to each study on the basis of its effect size.

Measures of outcome for studies with:

• Binary outcomes (e.g. cured/not cured) are odds ratio and risk ratio
• Continuous outcomes (e.g. blood pressure) are means, difference in means, and standardized difference in means
• Survival or time-to-event data are hazard ratios

Design a table and populate it with your data results. Draw this out into a forest plot , which provides a simple visual representation of variation between the studies.

Then analyze the data for issues. These can include heterogeneity, which is when studies’ lines within the forest plot don’t overlap with any other studies. Again, record any excluded studies here for reference.

Consider different factors when interpreting your results. These include limitations, strength of evidence, biases, applicability, economic effects, and implications for future practice or research.

Apply appropriate grading of your evidence and consider the strength of your recommendations.

It’s best to formulate a detailed plan for how you’ll present your systematic review results. Take a look at these guidelines for interpreting results from the Cochrane Institute.

Before writing your systematic literature review, you can register it with OSF for additional guidance along the way. You could also register your completed work with PROSPERO .

Systematic literature reviews are often found in clinical or healthcare settings. Medical professionals read systematic literature reviews to stay up-to-date in their field and granting agencies sometimes need them to make sure there’s justification for further research in an area.

The first stage in carrying out a systematic literature review is to put together your team. You should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

Your systematic review should include the following details:

A literature review simply provides a summary of the literature available on a topic. A systematic review, on the other hand, is more than just a summary. It also includes an analysis and evaluation of existing research. Put simply, it's a study of studies.

The final stage of conducting a systematic literature review is interpreting and presenting the results. It’s best to formulate a detailed plan for how you’ll present your systematic review results, guidelines can be found for example from the Cochrane institute .

Easy guide to conducting a systematic review

Affiliations.

• 1 Discipline of Child and Adolescent Health, University of Sydney, Sydney, New South Wales, Australia.
• 2 Department of Nephrology, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• 3 Education Department, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
• PMID: 32364273
• DOI: 10.1111/jpc.14853

A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a predefined search strategy to identify and appraise all available published literature on a specific topic. The meticulous nature of the systematic review research methodology differentiates a systematic review from a narrative review (literature review or authoritative review). This paper provides a brief step by step summary of how to conduct a systematic review, which may be of interest for clinicians and researchers.

Keywords: research; research design; systematic review.

© 2020 Paediatrics and Child Health Division (The Royal Australasian College of Physicians).

Publication types

• Systematic Review
• Research Design*

Systematic Reviews and Meta Analysis

• Getting Started
• Guides and Standards
• Review Protocols
• Databases and Sources
• Randomized Controlled Trials
• Controlled Clinical Trials
• Observational Designs
• Tests of Diagnostic Accuracy
• Software and Tools
• Where do I get all those articles?
• Collaborations
• EPI 233/528
• Countway Mediated Search
• Risk of Bias (RoB)

Systematic review Q & A

What is a systematic review.

A systematic review is guided filtering and synthesis of all available evidence addressing a specific, focused research question, generally about a specific intervention or exposure. The use of standardized, systematic methods and pre-selected eligibility criteria reduce the risk of bias in identifying, selecting and analyzing relevant studies. A well-designed systematic review includes clear objectives, pre-selected criteria for identifying eligible studies, an explicit methodology, a thorough and reproducible search of the literature, an assessment of the validity or risk of bias of each included study, and a systematic synthesis, analysis and presentation of the findings of the included studies. A systematic review may include a meta-analysis.

For details about carrying out systematic reviews, see the Guides and Standards section of this guide.

Is my research topic appropriate for systematic review methods?

A systematic review is best deployed to test a specific hypothesis about a healthcare or public health intervention or exposure. By focusing on a single intervention or a few specific interventions for a particular condition, the investigator can ensure a manageable results set. Moreover, examining a single or small set of related interventions, exposures, or outcomes, will simplify the assessment of studies and the synthesis of the findings.

Systematic reviews are poor tools for hypothesis generation: for instance, to determine what interventions have been used to increase the awareness and acceptability of a vaccine or to investigate the ways that predictive analytics have been used in health care management. In the first case, we don't know what interventions to search for and so have to screen all the articles about awareness and acceptability. In the second, there is no agreed on set of methods that make up predictive analytics, and health care management is far too broad. The search will necessarily be incomplete, vague and very large all at the same time. In most cases, reviews without clearly and exactly specified populations, interventions, exposures, and outcomes will produce results sets that quickly outstrip the resources of a small team and offer no consistent way to assess and synthesize findings from the studies that are identified.

If not a systematic review, then what?

You might consider performing a scoping review . This framework allows iterative searching over a reduced number of data sources and no requirement to assess individual studies for risk of bias. The framework includes built-in mechanisms to adjust the analysis as the work progresses and more is learned about the topic. A scoping review won't help you limit the number of records you'll need to screen (broad questions lead to large results sets) but may give you means of dealing with a large set of results.

This tool can help you decide what kind of review is right for your question.

Can my student complete a systematic review during her summer project?

Probably not. Systematic reviews are a lot of work. Including creating the protocol, building and running a quality search, collecting all the papers, evaluating the studies that meet the inclusion criteria and extracting and analyzing the summary data, a well done review can require dozens to hundreds of hours of work that can span several months. Moreover, a systematic review requires subject expertise, statistical support and a librarian to help design and run the search. Be aware that librarians sometimes have queues for their search time. It may take several weeks to complete and run a search. Moreover, all guidelines for carrying out systematic reviews recommend that at least two subject experts screen the studies identified in the search. The first round of screening can consume 1 hour per screener for every 100-200 records. A systematic review is a labor-intensive team effort.

How can I know if my topic has been been reviewed already?

Before starting out on a systematic review, check to see if someone has done it already. In PubMed you can use the systematic review subset to limit to a broad group of papers that is enriched for systematic reviews. You can invoke the subset by selecting if from the Article Types filters to the left of your PubMed results, or you can append AND systematic[sb] to your search. For example:

"neoadjuvant chemotherapy" AND systematic[sb]

The systematic review subset is very noisy, however. To quickly focus on systematic reviews (knowing that you may be missing some), simply search for the word systematic in the title:

"neoadjuvant chemotherapy" AND systematic[ti]

Any PRISMA-compliant systematic review will be captured by this method since including the words "systematic review" in the title is a requirement of the PRISMA checklist. Cochrane systematic reviews do not include 'systematic' in the title, however. It's worth checking the Cochrane Database of Systematic Reviews independently.

You can also search for protocols that will indicate that another group has set out on a similar project. Many investigators will register their protocols in PROSPERO , a registry of review protocols. Other published protocols as well as Cochrane Review protocols appear in the Cochrane Methodology Register, a part of the Cochrane Library .

• Next: Guides and Standards >>
• Last Updated: Feb 26, 2024 3:17 PM
• URL: https://guides.library.harvard.edu/meta-analysis

The Graduate Health & Life Sciences Research Library at Georgetown University Medical Center

Systematic reviews.

• Should I do a systematic review?
• Writing the Protocol
• Building a Systematic Search
• Where to Search
• Managing Project Data
• How can a DML librarian help?

Guides and Standards

• The Cochrane Handbook The Cochrane Handbook has become the de facto standard for planning and carrying out a systematic review. Chapter 6, Searching for Studies, is most helpful in planning your review.
• Finding What Works in Health Care: Standards for Systematic Reviews The IOM standards promote objective, transparent, and scientifically valid systematic reviews. They address the entire systematic review process, from locating, screening, and selecting studies for the review, to synthesizing the findings (including meta-analysis) and assessing the overall quality of the body of evidence, to producing the final review report.
• PRISMA Standards The Preferred Reporting Items for Systematic Reviews and Meta-Analyses is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. A 27-item checklist, PRISMA focuses on randomized trials but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

What is a systematic review?

A systematic literature review is a research methodology designed to answer a focused research question. Authors conduct a methodical and comprehensive literature synthesis focused on a well-formulated research question. Its aim is to identify and synthesize all of the scholarly research on a particular topic, including both published and unpublished studies. Systematic reviews are conducted in an unbiased, reproducible way to provide evidence for practice and policy-making and identify gaps in research.  Every step of the review, including the search, must be documented for reproducibility. 

Researchers in medicine may be most familiar with Cochrane Reviews, which synthesize randomized controlled trials to evaluate specific medical interventions. Systematic reviews are conducted in many other fields, though the type of evidence analyzed varies with the research question. 

When to use systematic review methodology

Systematic reviews require more time and manpower than traditional literature reviews. Before beginning a systematic review, researchers should address these questions:

Is there is enough literature published on the topic to warrant a review? 

Systematic reviews are designed to distill the evidence from many studies into actionable insights. Is there a body of evidence available to analyze, or does more primary research need to be done?

Can your research question be answered by a systematic review?

Systematic review questions should be specific and clearly defined. Questions that fit the PICO (problem/patient, intervention, comparison, outcome) format are usually well-suited for the systematic review methodology. The research question determines the search strategy, inclusion criteria, and data that you extract from the selected studies, so it should be clearly defined at the start of the review process.

Do you have a protocol outlining the review plan?

The protocol is the roadmap for the review project. A good protocol outlines study methodology, includes the rationale for the systematic review, and describes the key question broken into PICO components. It is also a good place to plan out inclusion/exclusion criteria, databases that will be searched, data abstraction and management methods, and how the studies will be assessed for methodological quality.

Do you have a team of experts?

A systematic review is team effort. Having multiple reviewers minimizes bias and strengthens analysis. Teams are often composed of subject experts, two or more literature screeners, a librarian to conduct the search, and a statistician to analyze the data. 

Do you have the time that it takes to properly conduct a systematic review?  

Systematic reviews typically take 12-18 months. 

Do you have a method for discerning bias?  

There are many types of bias, including selection, performance, & reporting bias, and assessing the risk of bias of individual studies is an important part of your study design.

Can you afford to have articles in languages other than English translated?  

You should include all relevant studies in your systematic review, regardless of the language they were published in, so as to avoid language bias. 

Which review is right for you?

If your project does not meet the above criteria, there are many more options for conducting a synthesis of the literature. The chart below highlights several review methodologies. Reproduced from: Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x  . Review. PubMed PMID: 19490148 

• Next: Writing the Protocol >>
• Last Updated: Apr 2, 2024 9:22 AM
• URL: https://guides.dml.georgetown.edu/systematicreviews

The Responsible Use of Electronic Resources policy governs the use of resources provided on these guides. © Dahlgren Memorial Library, Georgetown University Medical Center. Unless otherwise stated, these guides may be used for educational or academic purposes as long as proper attribution is given. Please seek permission for any modifications, adaptations, or for commercial purposes. Email [email protected] to request permission. Proper attribution includes: Written by or adapted from, Dahlgren Memorial Library, URL.

Reviewing systematic literature reviews: ten key questions and criteria for reviewers

• Open access
• Published: 12 July 2021
• Volume 71 , pages 519–524, ( 2021 )

Cite this article

You have full access to this open access article

• Andreas Kuckertz 1 &
• Joern Block 2  

8963 Accesses

37 Citations

14 Altmetric

Explore all metrics

Systematic literature review articles are important for synthesizing knowledge in management and business research. However, to date, we lack clear guidelines how to review such articles. This editorial takes the perspective of the reviewer. It presents ten key questions and criteria that reviewers should ask when reviewing systematic literature reviews.

Avoid common mistakes on your manuscript.

Owing to its focus on systematic literature reviews, meta-analyses, replications, and bibliometric studies, Management Review Quarterly (MRQ) requires submissions to meet specific criteria. Several published MRQ editorials explain those criteria, namely Block and Kuckertz ( 2018 ) for replication studies, Block and Fisch ( 2020 ) for bibliometric studies, and Fisch and Block ( 2018 ) with the extension by Clark ( 2021 ) for systematic literature reviews.

We again focus on systematic literature reviews (hereafter SLRs) in this editorial but shift the perspective. Whereas previous editorials addressed (potential) authors of MRQ submissions, this one addresses MRQ’s reviewers. Reviewers for the journal are often selected for their topical expertise and publication record, allowing them to provide the highest quality feedback to authors of SLRs. However, being an experienced researcher in a particular topical domain does not necessarily come with methodological expertise in SLRs. Hence, in this editorial, we highlight ten key questions and criteria that reviewers should consider when evaluating SLRs. Of course, knowledge of these criteria might be helpful for authors as well, as it will allow them to address issues that might arise during the review process proactively and consequently avoid them.

We consider SLRs (sometimes also called structured literature reviews) to be a variant of literature reviews, primarily aiming at increasing the transparency of the literature selection process underlying the review. Doing so helps to minimize subjectivity concerning which studies are included in the review and reducing what could be called the sampling error of unsystematic literature reviews. The term systematic thus refers in particular to a structured or systematic literature identification and selection. For reviewers of such SLRs, the following ten questions are crucial.

1. Does the SLR have an explicit and well-explained research question that is also well-justified?

Every study needs a research question, and that applies to SLRs too. That research question might be broad and topic-based (e.g., What is state-of-the-art in the respective field or literature stream? What do we know about a particular phenomenon?), but could also be more specific (e.g., What is the empirical evidence regarding a specific research question?). Opportunities to frame and justify research questions are endless, but, in all cases, authors should state a clear research question, and reviewers need to check whether the submission provides one. In addition to establishing an explicit research question, authors must justify the choice clearly and convey the potential value of their answer to the research question. It is then the task of the reviewers to assess whether they support that argumentation. In our own experience as reviewers and editors of MRQ , many initial submissions of SLRs lack sufficient justification. However, a literature review is not an end in itself. It needs to be justified like any other academic study. Possible justifications could be phenomenon-based, theory-driven, or guided by practice.

2. Does the SLR acknowledge previous literature reviews (including meta-analyses)?

Particularly in subject areas with a long history of research, it is to be expected that previous literature reviews exist. Every SLR should acknowledge these previous contributions (and in particular similar SLRs), describe their character (e.g., descriptive, bibliographic, state-of-the-art, or narrative), and explicate why another attempt to synthesize the body of knowledge is necessary. Reasons can be manifold—outdated prior literature reviews might need to be updated, including a different type of publication might potentially allow for new or more complete perspectives (e.g., gray literature vs. journal publications), or perhaps researchers are attempting to answer a hitherto unanswered research question. Reviewers should also check whether the SLR acknowledges published meta-analyses as these share many of the goals of an SLR.

3. Is the research question correctly translated into a (relevant) set of search terms?

An SLR refers to databases to identify relevant studies and queries them with an appropriate set of search terms. Reviewers must check whether the SLR states its search term transparently, accurately, and ascertain if the search term can guarantee to return the relevant literature to answer the research question. Formulating search terms is essentially the operationalization of the research question, and the success of any SLR stands or falls on it. Denyer and Tranfield ( 2009 ) suggest translating a research question with the help of the context-interventions-mechanisms-outcome (CIMO) framework into a set of search terms. Adopting a CIMO framework or a similar alternative also helps to devise the research questions that an SLR could reasonably be expected to answer. The set of search terms should be carefully thought out and not be so narrow that it misses relevant studies but also not so broad that the identified studies do not match the research questions. Reviewers should also check whether the researchers followed an iterative process where the search terms are adapted based on the search process results.

4. Is the database selection explained and based on a clear rationale?

Different databases of academic literature return different results. Therefore, it is good practice to employ more than one database in the literature search. Triangulation helps compensate for particular disadvantages that naturally limit the output of each individual database. At a minimum, a paper can address triangulation simply by adding more databases (e.g., the more databases, the better); an exemplary SLR will provide tangible reasons for including (or excluding) particular databases. Gusenbauer ( 2019 ) and Gusenbauer and Haddaway ( 2020 ) evaluate numerous available databases and provide a good starting point for justifying database selection.

5. Is the literature selection up-to-date?

Authors often submit outdated SLRs to MRQ . Updating a review before the submission is essential, and not something authors should postpone until the later rounds of the review process. Submitting an outdated literature review is usually a reason for a desk rejection, but reviewers should also check that an SLR is not outdated and remains topical throughout the process of revising and resubmitting.

6. Does the SLR clearly express its inclusion and exclusion criteria?

Quite often, a simple database query returns hundreds (or perhaps thousands) of studies with the potential for inclusion in the review. An SLR will usually report the search funnel and the authors’ inclusion and exclusion criteria applied to narrow down a search. Reviewers must check whether those criteria are expressly stated in the paper and examine whether they serve their purpose. The selected inclusion and exclusion criteria depend on the aims and research questions of the SLR: They might be justified in terms of content, method, or publication quality. We recommend using a flow chart to graphically illustrate the selection process. That flow chart should include the initial number of identified studies and the number of studies eliminated at each stage of the literature search process. Further guidance on how to ensure the selection of a final sample is transparent can be found in "Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA)" (Liberati , 2009 ; Moher, 2009 ).

7. Does the SLR include one or more overview tables characterizing each study in the sample?

Another good practice is to include a table summarizing each study from the sample. The table may be incorporated within the paper or, in the case of larger samples, presented as an (online) appendix. Such tables help readers quickly orient themselves in a particular research stream and understand the individual contribution of particular studies they might be interested in. Reviewers will need to check the quality of the information presented in such tables or appendices and whether the authors summarize the right information describing a single study. Standard columns of the table should include a reference to the individual study, a literal quote (or paraphrase) of its research question, the epistemological character of the study (e.g., conceptual, qualitative, quantitative, or mixed), the specific methodological design employed in the study (in the case of empirical studies), and the main result(s), although those columns might vary to reflect the purpose of the SLR. Depending on the number of studies included in the review, splitting this table into several sub-tables based on sub-topics is recommended. It is also possible to deposit a file in a public (data) repository (for instance, that operated by the Open Science Foundation). A repository offers authors greater flexibility regarding the number of columns and lines to be included, and reviewers can point authors to appropriate repositories.

8. Does the SLR employ a particular aggregation and presentation method?

It is essential to understand that an SLR is primarily a method to arrive at a suitable sample of studies to be reviewed, but how the selected literature is then analyzed, synthesized, and presented is a decision for individual authors who can choose from among a plethora of methods (see Booth et al. ( 2012 ) for an extensive overview of options). Far too often, this is the step where authors abandon methodological considerations. Nevertheless, there are many methods available to ensure this step is conducted in a rigorous and method-led manner, and SLRs employing such methods are more convincing. Such methods might range from a thematic analysis and narrative synthesis through meta-ethnographies to bibliometric approaches (e.g., historiographic mapping). Published MRQ papers illustrate the breadth of available methods. Here, the reviewer’s task is to check whether the chosen method was correctly executed and help the authors reach for a more profound and potentially more fruitful analysis. However, the use of particular aggregation and presentation methods should be fully explained and should align with the research question(s) of the SLR.

9. Does the SLR contribute beyond merely offering an analysis of the status quo of the literature?

Synthesis builds on analysis and may come in different forms. Some authors construct integrative models, provide thematic clusters based on bibliometric analyses, or offer creative answers to the research question. Synthesis is an essential step and distinguishes a technically correct but uninformative literature review from an informative one. Literature reviews tend to be among the most often-cited of journal articles, but will only be so if they contribute something unique and provide guidance to the community on what has been done and what is yet to be done.

10. Does the SLR provide implications for (future) research and practice?

A high-quality synthesis provides a basis for assessing the status quo and can set an agenda for future research. For practitioners, an SLR offers a convenient way to understand what is known or not yet known about a particular phenomenon or problem, and hence, the SLR should make evidence-based suggestions for managerial practice. Authors should not leave this task to readers but proactively outline specific (practice) implications that might speak to consumers, employees, managers, investors, firm owners, policymakers, and any other groups interested in management research. Similarly, for a research audience, every well-executed and well-crafted SLR has the potential to set the agenda for further research. Admittedly, that demands some creativity, as the easy solution to simply point the reader to under-researched areas invites encouraging studies based on the infamous has-not-been-done-before argument. The onus is on the researcher to identify and justify important avenues for future research, and reviewers should carefully consider whether they support those choices or should highlight alternatives they consider more promising. Authors should include the part of the SLR reflecting future research opportunities in an initial submission, but be aware that the suggestions will also potentially be subject to considerable change as a result of the review process. The length and nature of the future research section also depend on the goal of the SLR. Broader, state-of-the-art SLRs should provide a broad future research agenda, whereas reviews with a narrow research question should have a more focused and precise future research section.

SLRs have an important role in the accumulation of knowledge in management and business research; however, they can only fulfill that role if they are of high quality. This editorial, through its ten questions and criteria for reviewing SLRs, aims to signpost to reviewers the key issues relevant to evaluating the quality of an SLR. We also hope that it will enhance the quality of SLRs submitted to MRQ and other outlets in management research.

Block JH, Fisch C (2020) Eight tips and questions for your bibliographic study in business and management research. Manag Rev Q 70(3):307–312. https://doi.org/10.1007/s11301-020-00188-4

Article   Google Scholar  

Block J, Kuckertz A (2018) Seven principles of effective replication studies: strengthening the evidence base of management research. Manag Rev Q 68(4):355–359. https://doi.org/10.1007/s11301-018-0149-3

Booth A, Papaioannou D, Sutton A (2012) Systematic Approaches to a Successful Literature Review. Sage Publications, London

Google Scholar  

Clark WR et al (2021) Extending Fisch and Block’s (2018) tips for a systematic review in management and business literature. Manag Rev Q 71:215–231. https://doi.org/10.1007/s11301-020-00184-8

Denyer D, Tranfield D (2009) Producing a Systematic Review. In: Buchanan DA, Bryman A (eds) The Sage Handbook of Organizational Research Methods. Sage Publications, Thousand Oaks, CA, pp 671–689

Fisch C, Block J (2018) Six tips for your (systematic) literature review in business and management research. Manag Rev Q 68(2):103–106. https://doi.org/10.1007/s11301-018-0142-x

Gusenbauer M (2019) Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases. Scientometrics 118:177–214. https://doi.org/10.1007/s11192-018-2958-5

Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods 11(2):181–217. https://doi.org/10.1002/jrsm.1378

Liberati A et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. https://doi.org/10.1371/journal.pmed.1000100

Moher D et al (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097

Download references

Acknowledgements

We are indebted to Christian Fisch, Mirko Hirschmann, Holger Steinmetz and Rolf Wilmes for sharing their perspectives on the challenges of reviewing SLRs.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Entrepreneurship Research Group, University of Hohenheim, Stuttgart, Germany

Andreas Kuckertz

University of Trier, Trier, Germany

Joern Block

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Joern Block .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Kuckertz, A., Block, J. Reviewing systematic literature reviews: ten key questions and criteria for reviewers. Manag Rev Q 71 , 519–524 (2021). https://doi.org/10.1007/s11301-021-00228-7

Download citation

Accepted : 22 June 2021

Published : 12 July 2021

Issue Date : July 2021

DOI : https://doi.org/10.1007/s11301-021-00228-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

JEL Classification

• Find a journal
• Publish with us
• Track your research

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• When and how to update...

When and how to update systematic reviews: consensus and checklist

• Related content

Peer review

This article has a correction. please see:.

• Errata - September 06, 2016
• Paul Garner , professor 1 ,
• Sally Hopewell , associate professor 2 ,
• Jackie Chandler , methods coordinator 3 ,
• Harriet MacLehose , senior editor 3 ,
• Elie A Akl , professor 5 6 ,
• Joseph Beyene , associate professor 7 ,
• Stephanie Chang , director 8 ,
• Rachel Churchill , professor 9 ,
• Karin Dearness , managing editor 10 ,
• Gordon Guyatt , professor 4 ,
• Carol Lefebvre , information consultant 11 ,
• Beth Liles , methodologist 12 ,
• Rachel Marshall , editor 3 ,
• Laura Martínez García , researcher 13 ,
• Chris Mavergames , head 14 ,
• Mona Nasser , clinical lecturer in evidence based dentistry 15 ,
• Amir Qaseem , vice president and chair 16 17 ,
• Margaret Sampson , librarian 18 ,
• Karla Soares-Weiser , deputy editor in chief 3 ,
• Yemisi Takwoingi , senior research fellow in medical statistics 19 ,
• Lehana Thabane , director and professor 4 20 ,
• Marialena Trivella , statistician 21 ,
• Peter Tugwell , professor of medicine, epidemiology, and community medicine 22 ,
• Emma Welsh , managing editor 23 ,
• Ed C Wilson , senior research associate in health economics 24 ,
• Holger J Schünemann , professor 4 5
• 1 Cochrane Infectious Diseases Group, Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool L3 5QA, UK
• 2 Oxford Clinical Trials Research Unit, University of Oxford, Oxford, UK
• 3 Cochrane Editorial Unit, Cochrane Central Executive, London, UK
• 4 Department of Clinical Epidemiology and Biostatistics and Department of Medicine, McMaster University, Hamilton, ON, Canada
• 5 Cochrane GRADEing Methods Group, Ottawa, ON, Canada
• 6 Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
• 7 Department of Mathematics and Statistics, McMaster University
• 8 Evidence-based Practice Center Program, Agency for Healthcare and Research Quality, Rockville, MD, USA
• 9 Centre for Reviews and Dissemination, University of York, York, UK
• 10 Cochrane Upper Gastrointestinal and Pancreatic Diseases Group, Hamilton, ON, Canada
• 11 Lefebvre Associates, Oxford, UK
• 12 Kaiser Permanente National Guideline Program, Portland, OR, USA
• 13 Iberoamerican Cochrane Centre, Barcelona, Spain
• 14 Cochrane Informatics and Knowledge Management, Cochrane Central Executive, Freiburg, Germany
• 15 Plymouth University Peninsula School of Dentistry, Plymouth, UK
• 16 Department of Clinical Policy, American College of Physicians, Philadelphia, PA, USA
• 17 Guidelines International Network, Pitlochry, UK
• 18 Children’s Hospital of Eastern Ontario, Ottawa, ON, Canada
• 19 Institute of Applied Health Research, University of Birmingham, Birmingham, UK
• 20 Biostatistics Unit, Centre for Evaluation, McMaster University, Hamilton, ON, Canada
• 21 Centre for Statistics in Medicine, University of Oxford, Oxford, UK
• 22 University of Ottawa, Ottawa, ON, Canada
• 23 Cochrane Airways Group, Population Health Research Institute, St George’s, University of London, London, UK
• 24 Cambridge Centre for Health Services Research, University of Cambridge, Cambridge, UK
• Correspondence to: P Garner Paul.Garner{at}lstmed.ac.uk
• Accepted 26 May 2016

Updating of systematic reviews is generally more efficient than starting all over again when new evidence emerges, but to date there has been no clear guidance on how to do this. This guidance helps authors of systematic reviews, commissioners, and editors decide when to update a systematic review, and then how to go about updating the review.

Systematic reviews synthesise relevant research around a particular question. Preparing a systematic review is time and resource consuming, and provides a snapshot of knowledge at the time of incorporation of data from studies identified during the latest search. Newly identified studies can change the conclusion of a review. If they have not been included, this threatens the validity of the review, and, at worst, means the review could mislead. For patients and other healthcare consumers, this means that care and policy development might not be fully informed by the latest research; furthermore, researchers could be misled and carry out research in areas where no further research is actually needed. 1 Thus, there are clear benefits to updating reviews, rather than duplicating the entire process as new evidence emerges or new methods develop. Indeed, there is probably added value to updating a review, because this will include taking into account comments and criticisms, and adoption of new methods in an iterative process. 2 3 4 5 6

Cochrane has over 20 years of experience with preparing and updating systematic reviews, with the publication of over 6000 systematic reviews. However, Cochrane’s principle of keeping all reviews up to date has not been possible, and the organisation has had to adapt: from updating when new evidence becomes available, 7 to updating every two years, 8 to updating based on need and priority. 9 This experience has shown that it is not possible, sensible, or feasible to continually update all reviews all the time. Other groups, including guideline developers and journal editors, adopt updating principles (as applied, for example, by the Systematic Reviews journal; https://systematicreviewsjournal.biomedcentral.com/ ).

The panel for updating guidance for systematic reviews (PUGs) group met to draw together experiences and identify a common approach. The PUGs guidance can help individuals or academic teams working outside of a commissioning agency or Cochrane, who are considering writing a systematic review for a journal or to prepare for a research project. The guidance could also help these groups decide whether their effort is worthwhile.

Summary points

Updating systematic reviews is, in general, more efficient than starting afresh when new evidence emerges. The panel for updating guidance for systematic reviews (PUGs; comprising review authors, editors, statisticians, information specialists, related methodologists, and guideline developers) met to develop guidance for people considering updating systematic reviews. The panel proposed the following:

Decisions about whether and when to update a systematic review are judgments made for individual reviews at a particular time. These decisions can be made by agencies responsible for systematic review portfolios, journal editors with systematic review update services, or author teams considering embarking on an update of a review.

The decision needs to take into account whether the review addresses a current question, uses valid methods, and is well conducted; and whether there are new relevant methods, new studies, or new information on existing included studies. Given this information, the agency, editors, or authors need to judge whether the update will influence the review findings or credibility sufficiently to justify the effort in updating it.

Review authors and commissioners can use a decision framework and checklist to navigate and report these decisions with “update status” and rationale for this status. The panel noted that the incorporation of new synthesis methods (such as Grading of Recommendations Assessment, Development and Evaluation (GRADE)) is also often likely to improve the quality of the analysis and the clarity of the findings.

Given a decision to update, the process needs to start with an appraisal and revision of the background, question, inclusion criteria, and methods of the existing review.

Search strategies should be refined, taking into account changes in the question or inclusion criteria. An analysis of yield from the previous edition, in relation to databases searched, terms, and languages can make searches more specific and efficient.

In many instances, an update represents a new edition of the review, and authorship of the new version needs to follow criteria of the International Committee of Medical Journal Editors (ICMJE). New approaches to publishing licences could help new authors build on and re-use the previous edition while giving appropriate credit to the previous authors.

The panel also reflected on this guidance in the context of emerging technological advances in software, information retrieval, and electronic linkage and mining. With good synthesis and technology partnerships, these advances could revolutionise the efficiency of updating in the coming years.

Panel selection and procedures

An international panel of authors, editors, clinicians, statisticians, information specialists, other methodologists, and guideline developers was invited to a two day workshop at McMaster University, Hamilton, Canada, on 26-27 June 2014, organised by Cochrane. The organising committee selected the panel (web appendix 1). The organising committee invited participants, put forward the agenda, collected background materials and literature, and drafted the structure of the report.

The purpose of the workshop was to develop a common approach to updating systematic reviews, drawing on existing strategies, research, and experience of people working in this area. The selection of participants aimed on broad representation of different groups involved in producing systematic reviews (including authors, editors, statisticians, information specialists, and other methodologists), and those using the reviews (guideline developers and clinicians). Participants within these groups were selected on their expertise and experience in updating, in previous work developing methods to assess reviews, and because some were recognised for developing approaches within organisations to manage updating strategically. We sought to identify general approaches in this area, and not be specific to Cochrane; although inevitably most of the panel were somehow engaged in Cochrane.

The workshop structure followed a series of short presentations addressing key questions on whether, when, and how to update systematic reviews. The proceedings included the management of authorship and editorial decisions, and innovative and technological approaches. A series of small group discussions followed each question, deliberating content, and forming recommendations, as well as recognising uncertainties. Large group, round table discussions deliberated further these small group developments. Recommendations were presented to an invited forum of individuals with varying levels of expertise in systematic reviews from McMaster University (of over 40 people), widely known for its contributions to the field of research evidence synthesis. Their comments helped inform the emerging guidance.

The organising committee became the writing committee after the meeting. They developed the guidance arising from the meeting, developed the checklist and diagrams, added examples, and finalised the manuscript. The guidance was circulated to the larger group three times, with the PUGs panel providing extensive feedback. This feedback was all considered and carefully addressed by the writing committee. The writing committee provided the panel with the option of expressing any additional comments from the general or specific guidance in the report, and the option for registering their own view that might differ to the guidance formed and their view would be recorded in an annex. In the event, consensus was reached, and the annex was not required.

Definition of update

The PUGs panel defined an update of a systematic review as a new edition of a published systematic review with changes that can include new data, new methods, or new analyses to the previous edition. This expands on a previous definition of a systematic review update. 10 An update asks a similar question with regard to the participants, intervention, comparisons, and outcomes (PICO) and has similar objectives; thus it has similar inclusion criteria. These inclusion criteria can be modified in the light of developments within the topic area with new interventions, new standards, and new approaches. Updates will include a new search for potentially relevant studies and incorporate any eligible studies or data; and adjust the findings and conclusions as appropriate. Box 1 provides some examples.

Box 1: Examples of what factors might change in an updated systematic review

A systematic review of steroid treatment in tuberculosis meningitis used GRADE methods and split the composite outcome in the original review of death plus disability into its two components. This improved the clarity of the reviews findings in relation to the effects and the importance of the effects of steroids on death and on disability. 11

A systematic review of dihydroartemisinin-piperaquine (DHAP) for treating malaria was updated with much more detailed analysis of the adverse effect data from the existing trials as a result of questions raised by the European Medicines Agency. Because the original review included other comparisons, the update required extracting only the DHAP comparisons from the original review, and a modification of the title and the PICO. 12

A systematic review of atorvastatin was updated with simple uncontrolled studies. 13 This update allowed comparisons with trials and strengthened the review findings. 14

Which systematic reviews should be updated and when?

Any group maintaining a portfolio of systematic reviews as part of their normative work, such as guidelines panels or Cochrane review groups, will need to prioritise which reviews to update. Box 2 presents the approaches used by the Agency for HealthCare Research and Quality (AHRQ) and Cochrane to prioritise which systematic reviews to update and when. Clearly, the responsibility for deciding which systematic reviews should be updated and when they will be updated will vary: it may be centrally organised and resourced, as with the AHRQ scientific resource centre (box 2). In Cochrane, the decision making process is decentralised to the Cochrane Review Group editorial team, with different approaches applied, often informally.

Box 2: Examples of how different organisations decide on updating systematic reviews

Agency for healthcare research and quality (us).

The AHRQ uses a needs based approach; updating systematic reviews depends on an assessment of several criteria:

Stakeholder impact

Interest from stakeholder partners (such as consumers, funders, guideline developers, clinical societies, James Lind Alliance)

Use and uptake (for example, frequency of citations and downloads)

Citation in scientific literature including clinical practice guidelines

Currency and need for update

New research is available

Review conclusions are probably dated

Update decision

Based on the above criteria, the decision is made to either update, archive, or continue surveillance.

Of over 50 Cochrane editorial teams, most but not all have some systems for updating, although this process can be informal and loosely applied. Most editorial teams draw on some or all of the following criteria:

Strategic importance

Is the topic a priority area (for example, in current debates or considered by guidelines groups)?

Is there important new information available?

Practicalities in organising the update that many groups take into account

Size of the task (size and quality of the review, and how many new studies or analyses are needed)

Availability and willingness of the author team

Impact of update

New research impact on findings and credibility

Consider whether new methods will improve review quality

Priority to update, postpone update, class review as no longer requiring an update

The PUGs panel recommended an individualised approach to updating, which used the procedures summarised in figure 1 ⇓ . The figure provides a status category, and some options for classifying reviews into each of these categories, and builds on a previous decision tool and earlier work developing an updating classification system. 15 16 We provide a narrative for each step.

Fig 1 Decision framework to assess systematic reviews for updating, with standard terms to report such decisions

• Download figure
• Open in new tab
• Download powerpoint

Step 1: assess currency

Does the published review still address a current question.

An update is only worthwhile if the question is topical for decision making for practice, policy, or research priorities (fig 1 ⇑ ). For agencies, people responsible for managing a portfolio of systematic reviews, there is a need to use both formal and informal horizon scanning. This type of scanning helps identify questions with currency, and can help identify those reviews that should be updated. The process could include monitoring policy debates around the review, media outlets, scientific (and professional) publications, and linking with guideline developers.

Has the review had good access or use?

Metrics for citations, article access and downloads, and sharing via social or traditional media can be used as proxy or indicators for currency and relevance of the review. Reviews that are widely cited and used could be important to update should the need arise. Comparable reviews that are never cited or rarely downloaded, for example, could indicate that they are not addressing a question that is valued, and might not be worth updating.

In most cases, updated reviews are most useful to stakeholders when there is new information or methods that result in a change in findings. However, there are some circumstances in which an up to date search for information is important for retaining the credibility of the review, regardless of whether the main findings would change or not. For example, key stakeholders would dismiss a review if a study is carried out in a relevant geographical setting but is not included; if a large, high profile study that might not change the findings is not included; or if an up to date search is required for a guideline to achieve credibility. Box 3 provides such examples. If the review does not answer a current question, the intervention has been superseded, then a decision can be made not to update and no further intelligence gathering is required (fig 1 ⇑ ).

Box 3: Examples of a systematic review’s currency

The public is interested in vitamin C for preventing the common cold: the Cochrane review includes over 29 trials with either no or small effects, concluding good evidence of no important effects. 17 Assessment: still a current question for the public.

Low osmolarity oral rehydration salt (ORS) solution versus standard solution for acute diarrhoea in children: the 2001 Cochrane review 18 led the World Health Organization to recommend ORS solution formula worldwide to follow the new ORS solution formula 19 and this has now been accepted globally. Assessment: no longer a current question.

Routine prophylactic antibiotics with caesarean section: the Cochrane review reports clear evidence of maternal benefit from placebo controlled trials but no information on the effects on the baby. 20 Assessment: this is a current question.

A systematic review published in the Lancet examined the effects of artemisinin based combination treatments compared with monotherapy for treating malaria and showed clear benefit. 21 Assessment: this established the treatment globally and is no longer a current question and no update is required.

A Cochrane review of amalgam restorations for dental caries 22 is unlikely to be updated because the use of dental amalgam is declining, and the question is not seen as being important by many dental specialists. Assessment: no longer a current question.

Did the review use valid methods and was it well conducted?

If the question is current and clearly defined, the systematic review needs to have used valid methods and be well conducted. If the review has vague inclusion criteria, poorly articulated outcomes, or inappropriate methods, then updating should not proceed. If the question is current, and the review has been cited or used, then it might be appropriate to simply start with a new protocol. The appraisal should take into account the methods in use when the review was done.

Step 2: identify relevant new methods, studies, and other information

Are there any new relevant methods.

If the question is current, but the review was done some years ago, the quality of the review might not meet current day standards. Methods have advanced quickly, and data extraction and understanding of the review process have become more sophisticated. For example:

Methods for assessing risk of bias of randomised trials, 23 diagnostic test accuracy (QUADAS-2), 24 and observational studies (ROBINS-1). 25

Application of summary of findings, evidence profiles, and related GRADE methods has meant the characteristics of the intervention, characteristics of the participants, and risk of bias are more thoroughly and systematically documented. 26 27

Integration of other study designs containing evidence, such economic evaluation and qualitative research. 28

There are other incremental improvements in a wide range of statistical and methodological areas, for example, in describing and taking into account cluster randomised trials. 29 AMSTAR can assess the overall quality of a systematic review, 30 and the ROBIS tool can provide a more detailed assessment of the potential for bias. 31

Are there any new studies or other information?

If an authoring or commissioning team wants to ensure that a particular review is up to date, there is a need for routine surveillance for new studies that are potentially relevant to the review, by searching and trial register inspection at regular intervals. This process has several approaches, including:

Formal surveillance searching 32

Updating the full search strategies in the original review and running the searches

Tracking studies in clinical trial and other registers

Using literature appraisal services 33

Using a defined abbreviated search strategy for the update 34

Checking studies included in related systematic reviews. 35

How often this surveillance is done, and which approaches to use, depend on the circumstances and the topic. Some topics move quickly, and the definition of “regular intervals” will vary according to the field and according to the state of evidence in the field. For example, early in the life of a new intervention, there might be a plethora of studies, and surveillance would be needed more frequently.

Step 3: assess the effect of updating the review

Will the adoption of new methods change the findings or credibility.

Editors, referees, or experts in the topic area or methodologists can provide an informed view of whether a review can be substantially improved by application of current methodological expectations and new methods (fig 1 ⇑ ). For example, a Cochrane review of iron supplementation in malaria concluded that there was “no significant difference between iron and placebo detected.” 36 An update of the review included a GRADE assessment of the certainty of the evidence, and was able to conclude with a high degree of certainty that iron does not cause an excess of clinical malaria because the upper relative risk confidence intervals of harm was 1.0 with high certainty of evidence. 37

Will the new studies, information, or data change the findings or credibility?

The assessment of new data contained in new studies and how these data might change the review is often used to determine whether an update should go ahead, and the speed with which the update should be conducted. The appraisal of these new data can be carried out in different ways. Initially, methods focused on statistical approaches to predict an overturning of the current review findings in terms of the primary or desired outcome (table 1 ⇓ ). Although this aspect is important, additional studies can add important information to a review, which is more than just changing the primary outcome to a more accurate and reliable estimate. Box 4 gives examples.

Formal prediction tools: how potentially relevant new studies can affect review conclusions

• View inline

Box 4: Examples of new information other than new trials being important

The iconic Cochrane review of steroids in preterm labour was thought to provide evidence of benefit in infants, and this question no longer required new trials. However, a new large trial published in the Lancet in 2015 showed that in low and middle income countries, strategies to promote the uptake of neonatal steroids increased neonatal mortality and suspected maternal infection. 49 This information needs to somehow be incorporated into the review to maintain its credibility.

A Cochrane review of community deworming in developing countries indicates that in recent studies, there is little or no effect. 50 The inclusion of a large trial of two million children confirmed that there was no effect on mortality. Although the incorporation of the trial in the review did not change the review’s conclusions, the trial’s absence would have affected the credibility of the review, so it was therefore updated.

A new paper reporting long term follow-up data on anthracycline chemotherapy as part of cancer treatment was published. Although the effects from the outcomes remained essentially unchanged, apart from this longer follow-up, the paper also included information about the performance bias in the original trial, shifting the risk of bias for several outcomes from “unknown” to “high” in the Cochrane review. 51

Reviews with a high level of certainty in the results (that is, when the GRADE assessment for the body of evidence is high) are less likely to change even with the addition of new studies, information, or data, by definition. GRADE can help guide priorities in whether to update, but it is still important to assess new studies that might meet the inclusion criteria. New studies can show unexpected effects (eg, attenuation of efficacy) or provide new information about the effects seen in different circumstances (eg, groups of patients or locations).

Other tools are specifically designed to help decision making in updating. For example, the Ottawa 39 and RAND 45 methods focus on identification of new evidence, the statistical predication tool 15 calculates the probability of new evidence changing the review conclusion, and the value of information analysis approach 52 calculates the expected health gain (table 1 ⇑ ). As yet, there has been limited external validation of these tools to determine which approach would be most effective and when.

If potentially relevant studies are identified that have not previously been assessed for inclusion, authors or those managing the updating process need to assess whether including them might affect the conclusions of the review. They need to examine the weight and certainty of the new evidence to help determine whether an update is needed and how urgent that update is. The updating team can assess this informally by judging whether new studies or data are likely to substantively affect the review, for example, by altering the certainty in an existing comparison, or by generating new comparisons and analyses in the existing review.

New information can also include fresh follow-up data on existing included studies, or information on how the studies were carried out. These should be assessed in terms of whether they might change the review findings or improve its credibility (fig 1 ⇑ ). Indeed, if any study has been retracted, it is important the authors assess the reasons for its retraction. In the case of data fabrication, the study needs to be removed from the analysis and this recorded. A decision needs to be made as to whether other studies by the same author should be removed from the review and other related reviews. An investigation should also be initiated following guidelines from the Committee on Publication Ethics (COPE). Additional published and unpublished data can become available from a wide range of sources—including study investigators, regulatory agencies and industry—and are important to consider.

Preparing for an update

Refresh background, objectives, inclusion criteria, and methods

Before including new studies in the review, authors need to revisit the background, objectives, inclusion criteria, and methods of the current review. In Cochrane, this is referred to as the protocol, and editors are part of this process. The update could range from simply endorsing the current question and inclusion criteria, through to full rewriting of the question, inclusion criteria and methods, and republishing the protocol. As a field progresses with larger and better quality trials rigorously testing the questions posed, it may be appropriate to exclude weaker study designs (such as quasi-randomised comparisons or very small trials) from the update (table 2 ⇓ ). The PUGs panel recommended that a protocol refresh will require the authors to use the latest accepted methods of synthesis, even if this means repeating data extraction for all studies.

New authors and authorship

Updated systematic reviews are new publications with new citations. An authorship team publishing an update in a scientific or medical journal is likely to manage the new edition of a review in the same way as with any other publication, and follow the ICMJE authorship criteria. 56 If the previous author or author team steps down, then they should be acknowledged in the new version. However, some might perceive that their efforts in the first version warrant continued authorship, which may be valid. The management of authorship between versions can sometimes be complicated. At worst, it delays new authors completing an update and leads to long authorship lists of people from previous versions who probably do not meet ICMJE authorship criteria. One approach with updates including new authors is to have an opt-in policy for the existing authors: they can opt in to the new edition, provided that they make clear their contribution, and this is then agreed with the entire author team.

Although they are new publications, updates will generally include content from the published version. Changing licensing rights around systematic reviews to allow new authors of future updates to remix, tweak, or build on the contributions of the original authors of the published version (similar to the rights available via a Creative Commons licence; https://creativecommons.org ) could be a more sustainable and simpler approach. This approach would allow systematic reviews to continue to evolve and build on the work of a range of authors over time, and for contributors to be given credit for contributions to this previous work.

Efficient searching

In performing an update, a search based on the search conducted for the original review is required. The updated search strategy will need to take into account changes in the review question or inclusion criteria, for example, and might be further adjusted based on knowledge of running the original search strategy. The search strategy for an update need not replicate the original search strategy, but could be refined, for example, based on an analysis of the yield of the original search. These new search approaches are currently undergoing formal empirical evaluation, but they may well provide much more efficient search strategies in the future. Some examples of these possible new methods for review updates are described in web appendix 2.

In reporting the search process for the update, investigators must ensure transparency for any previous versions and the current update, and use an adapted flow diagram based on PRISMA reporting (preferred reporting items for systematic reviews and meta-analyses). 57 The search processes and strategies for the update must be adequately reported such that they could be replicated.

Systematic reviews published for the first time in peer reviewed journals are by definition peer reviewed, but practice for updates remains variable, because an update might have few changes (such as an updated search but no new studies found and therefore included) or many changes (such as revise methods and inclusion of several new studies leading to revised conclusions). Therefore, and to use peer reviewers’ time most effectively, editors need to consider when to peer review an update and the type of peer reviewer most useful for a particular update (for example, topic specialist, methodologist). The decision to use peer review, and the number and expertise of the peer reviewers could depend on the nature of the update and the extent of any changes to the systematic review as part of an editor assessment. A change in the date of the search only (where no new studies were identified) would not require peer review (except, arguably, peer review of the search), but the addition of studies that lead to a change in conclusions or significant changes to the methods would require peer review. The nature of the peer review could be described within the published article.

Reporting changes

Authors should provide a clear description of the changes in approach or methods between different editions of a review. Also, authors need to report the differences in findings between the original and updated edition to help users decide how to use the new edition. The approach or format used to present the differences in findings might vary with the target user group. 58 Publishers need to ensure that all previous versions of the review remain publically accessible.

Updates can range from small adjustments to reviews being completely rewritten, and the PUGs panel spent some time debating whether the term “new edition” would be a better description than “update.” However, the word “update” is now in common parlance and changing the term, the panel judged, could cause confusion. However, the debate does illustrate that an update could represent a review that asks a similar question but has been completely revised.

Technology and innovation

The updating of systematic review is generally done manually and is time consuming. There are opportunities to make better use of technology to streamline the updating process and improve efficiency (table 3 ⇓ ). Some of these tools already exist and are in development or in early use, and some are commercially available or freely available. The AHRQ’s evidence based practice centre team has recently published tools for searching and screening, and will provide an assessment of the use, reliability, and availability of these tools. 63

Technological innovations to improve the efficiency of updating systematic reviews

Other developments, such as targeted updates that are performed rapidly and focus on updating only key components of a review, could provide different approaches to updating in the future and are being piloted and evaluated. 64 With implementation of these various innovations, the longer term goal is for “living” systematic reviews, which identify and incorporate information rapidly as it evolves over time. 60

Concluding remarks

Updating systematic reviews, rather than addressing the same question with a fresh protocol, is generally more efficient and allows incremental improvement over time. Mechanical rules appear unworkable, but there is no clear unified approach on when to update, and how implement this. This PUGs panel of authors, editors, statisticians, information specialists, other methodologists, and guideline developers brought together current thinking and experience in this area to provide guidance.

Decisions about whether and when to update a systematic review are judgments made at a point in time. They depend on the currency of the question asked, the need for updating to maintain credibility, the availability of new evidence, and whether new research or new methods will affect the findings.

Whether the review uses current methodological standards is important in deciding if the update will influence the review findings, quality, reliability, or credibility sufficiently to justify the effort in updating it. Those updating systematic reviews to author clinical practice guidelines might consider the influence of new study results in potentially overturning the conclusions of an existing review. Yet, even in cases where new study findings do not change the primary outcome measure, new studies can carry important information about subgroup effects, duration of treatment effects, and other relevant clinical information, enhancing the currency and breadth of review results.

An update requires appraisal and revision of the background, question, inclusion criteria, and methods of the existing review and the existing certainty in the evidence. In particular, methods might need to be updated, and search strategies reconsidered. Authors of updates need to consider inputs to the current edition, and follow ICMJE criteria regarding authorship. 56

The PUGs panel proposed a decision framework (fig 1 ⇑ ), with terms and categories for reporting the decisions made for updating procedures for adoption by Cochrane and other stakeholders. This framework includes journals publishing systematic review updates and independent authors considering updates of existing published reviews. The panel developed a checklist to help judgements about when and how to update.

The current emphasis of authors, guideline developers, Cochrane, and consequently this guidance has been on effects reviews. The checklists and guidance here still applies to other types of systematic reviews, such as those on diagnostic test accuracy, and this guidance will need adapting. Accumulative experience and methods development in reviews other than those of effects are likely to help refine guidance in the future.

This guidance could help groups identify and prioritise reviews for updating and hence use their finite resources to greatest effect. Software innovation and new management systems are being developed and in early use to help streamline review updates in the coming years.

Contributors: HJS initiated the workshop. JC, SH, PG, HM, and HJS organised the materials and the agenda. SH wrote up the proceedings. PG wrote the paper from the proceedings and coordinated the development of the final guidance; JC, SH, HM, and HJS were active in the finalising of the guidance. All PUGs authors contributed to three rounds of manuscript revision.

Funding: Attendance at this meeting, for those attendees not directly employed by Cochrane, was not funded by Cochrane beyond the reimbursement of out of pocket expenses for those attendees for whom this was appropriate. Expenses were not reimbursed for US federal government attendees, in line with US government policy. Statements in the manuscript should not be construed as endorsement by the US Agency for Healthcare Research and Quality or the US Department of Health and Human Services.

Competing interests: All participants have a direct or indirect interest in systematic reviews and updating as part of their job or academic career. Most participants contribute to Cochrane, whose mission includes a commitment to the updating of its systematic review portfolio. JC, HM, RM, CM, KS-W, and MT are, or were at that time, employed by the Cochrane Central Executive.

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 3.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/3.0/ .

• ↵ Shekelle PG, Ortiz E, Rhodes S, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 2001 ; 286 : 1461 - 7 . doi:10.1001/jama.286.12.1461 pmid:11572738 . OpenUrl CrossRef PubMed Web of Science
• ↵ Claxton K, Cohen JT, Neumann PJ. When is evidence sufficient? Health Aff (Millwood) 2005 ; 24 : 93 - 101 . doi:10.1377/hlthaff.24.1.93 pmid:15647219 . OpenUrl Abstract / FREE Full Text
• ↵ Fenwick E, Claxton K, Sculpher M, et al. Improving the efficiency and relevance of health technology assessment: the role of decision analytic modelling. Paper 179. Centre for Health Economics, University of York, 2000 .
• ↵ Sculpher M, Claxton K. Establishing the cost-effectiveness of new pharmaceuticals under conditions of uncertainty—when is there sufficient evidence? Value Health 2005 ; 8 : 433 - 46 . doi:10.1111/j.1524-4733.2005.00033.x pmid:16091019 . OpenUrl CrossRef PubMed Web of Science
• ↵ Sculpher M, Drummond M, Buxton M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy 1997 ; 2 : 26 - 30 . pmid:10180650 . OpenUrl Abstract / FREE Full Text
• ↵ Wilson E, Abrams K. From evidence based economics to economics based evidence: using systematic review to inform the design of future research. In: Shemilt I, Mugford M, Vale L, et al, eds. Evidence based economics. Blackwell Publishing, 2010 doi:10.1002/9781444320398.ch12 .
• ↵ Chalmers I, Enkin M, Keirse MJ. Preparing and updating systematic reviews of randomized controlled trials of health care. Milbank Q 1993 ; 71 : 411 - 37 . doi:10.2307/3350409 pmid:8413069 . OpenUrl CrossRef PubMed Web of Science
• ↵ Higgins J, Green S, Scholten R. Chapter 3. Maintaining reviews: updates, amendments and feedback: Version 5.1.0 (updated March 2011). Cochrane Collaboration, 2011 .
• ↵ Cochrane. Editorial and publishing policy resource. http://community.cochrane.org/editorial-and-publishing-policy-resource . 2016.
• ↵ Moher D, Tsertsvadze A. Systematic reviews: when is an update an update? Lancet 2006 ; 367 : 881 - 3 . doi:10.1016/S0140-6736(06)68358-X pmid:16546523 . OpenUrl CrossRef PubMed Web of Science
• ↵ Prasad K, Singh MB, Ryan H. Corticosteroids for managing tuberculous meningitis. Cochrane Database Syst Rev 2016 ; 4 : CD002244 . pmid:27121755 . OpenUrl PubMed
• ↵ Zani B, Gathu M, Donegan S, Olliaro PL, Sinclair D. Dihydroartemisinin-piperaquine for treating uncomplicated Plasmodium falciparum malaria. Cochrane Database Syst Rev 2014 ; 1 : CD010927 . pmid:24443033 . OpenUrl CrossRef PubMed
• ↵ Adams SP, Tsang M, Wright JM. Lipid lowering efficacy of atorvastatin. Cochrane Database Syst Rev 2012 ; 12 : CD008226 . pmid:23235655 . OpenUrl PubMed
• ↵ Higgins J. Convincing evidence from controlled and uncontrolled studies on the lipid-lowering effect of a statin. Cochrane Database Syst Rev 2012 ; 12 : ED000049 . pmid:23361645 . OpenUrl PubMed
• ↵ Takwoingi Y, Hopewell S, Tovey D, Sutton AJ. A multicomponent decision tool for prioritising the updating of systematic reviews. BMJ 2013 ; 347 : f7191 . doi:10.1136/bmj.f7191 pmid:24336453 . OpenUrl FREE Full Text
• ↵ MacLehose H, Hilton J, Tovey D, et al. The Cochrane Library: revolution or evolution? Shaping the future of Cochrane content. Background paper for The Cochrane Collaboration’s Strategic Session Paris, France, 18 April 2012. http://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/2012-CC-strategic-session_full-report.pdf .
• ↵ Hemilä H, Chalker E. Vitamin C for preventing and treating the common cold. Cochrane Database Syst Rev 2013 ;( 1 ): CD000980 . pmid:23440782 .
• ↵ Hahn S, Kim S, Garner P. Reduced osmolarity oral rehydration solution for treating dehydration caused by acute diarrhoea in children. Cochrane Database Syst Rev 2002 ;( 1 ): CD002847 . pmid:11869639 .
• ↵ World Health Organization (WHO). Reduced osmolarity oral rehydration salts (ORS) formulation. A report from a meeting of Experts jointly organized by UNICEF and WHO. New York: Child and Adolescent Health and Development, 18 July 2001 http://apps.who.int/iris/bitstream/10665/67322/1/WHO_FCH_CAH_01.22.pdf .
• ↵ Smaill FM, Grivell RM. Antibiotic prophylaxis versus no prophylaxis for preventing infection after cesarean section. Cochrane Database Syst Rev 2014 ;( 10 ): CD007482 . pmid:25350672 .
• ↵ Adjuik M, Babiker A, Garner P, Olliaro P, Taylor W, White N. International Artemisinin Study Group. Artesunate combinations for treatment of malaria: meta-analysis. Lancet 2004 ; 363 : 9 - 17 . doi:10.1016/S0140-6736(03)15162-8 pmid:14723987 . OpenUrl CrossRef PubMed Web of Science
• ↵ Agnihotry A, Fedorowicz Z, Nasser M. Adhesively bonded versus non-bonded amalgam restorations for dental caries. Cochrane Database Syst Rev 2016 ; 3 : CD007517 . pmid:26954446 . OpenUrl PubMed
• ↵ Higgins JP, Altman DG, Gøtzsche PC, et al. Cochrane Bias Methods Group Cochrane Statistical Methods Group. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011 ; 343 : d5928 . doi:10.1136/bmj.d5928 pmid:22008217 . OpenUrl FREE Full Text
• ↵ Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011 ; 155 : 529 - 36 . doi:10.7326/0003-4819-155-8-201110180-00009 pmid:22007046 . OpenUrl CrossRef PubMed Web of Science
• ↵ Sterne JAC, Higgins JPT, Reeves BC; on behalf of the development group for ROBINS-I. A tool for assessing risk of bias in non-randomized studies of interventions, version 7. March 2016. www.riskofbias.info .
• ↵ Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008 ; 336 : 924 - 6 . doi:10.1136/bmj.39489.470347.AD pmid:18436948 . OpenUrl FREE Full Text
• ↵ Schünemann HJ. Interpreting GRADE’s levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? J Clin Epidemiol 2016 ; 75 : 6 - 15 . doi:10.1016/j.jclinepi.2016.03.018 pmid:27063205 . OpenUrl CrossRef PubMed
• ↵ Gough D. Qualitative and mixed methods in systematic reviews. Syst Rev 2015 ; 4 : 181 . doi:10.1186/s13643-015-0151-y pmid:26670769 . OpenUrl CrossRef PubMed
• ↵ Richardson M, Garner P, Donegan S. Cluster randomised trials in Cochrane reviews: evaluation of methodological and reporting practice. PLoS One 2016 ; 11 : e0151818 . doi:10.1371/journal.pone.0151818 pmid:26982697 . OpenUrl CrossRef PubMed
• ↵ Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007 ; 7 : 10 . doi:10.1186/1471-2288-7-10 pmid:17302989 . OpenUrl CrossRef PubMed
• ↵ Whiting P, Savović J, Higgins JP, et al. ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016 ; 69 : 225 - 34 . doi:10.1016/j.jclinepi.2015.06.005 pmid:26092286 . OpenUrl CrossRef PubMed
• ↵ Sampson M, Shojania KG, McGowan J, et al. Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol 2008 ; 61 : 755 - 62 . doi:10.1016/j.jclinepi.2007.10.003 pmid:18586179 . OpenUrl CrossRef PubMed Web of Science
• ↵ Hemens BJ, Haynes RB. McMaster Premium LiteratUre Service (PLUS) performed well for identifying new studies for updated Cochrane reviews. J Clin Epidemiol 2012 ; 65 : 62 - 72.e1 . doi:10.1016/j.jclinepi.2011.02.010 pmid:21856121 . OpenUrl CrossRef PubMed
• ↵ Sagliocca L, De Masi S, Ferrigno L, Mele A, Traversa G. A pragmatic strategy for the review of clinical evidence. J Eval Clin Pract 2013 ; 19 : 689 - 96 . doi:10.1111/jep.12020 pmid:23317014 . OpenUrl CrossRef PubMed
• ↵ Rada G, Peña J, Capurro D, et al. How to create a matrix of evidence in Epistemonikos. Abstracts of the 22nd Cochrane Colloquium; Evidence-informed public health: opportunities and challenges; Hyderabad, India. Cochrane Database Syst Rev 2014 ; suppl 1 : 132 .
• ↵ Okebe JU, Yahav D, Shbita R, Paul M. Oral iron supplements for children in malaria-endemic areas. Cochrane Database Syst Rev 2011 ;( 10 ): CD006589 . pmid:21975754 .
• ↵ Neuberger A, Okebe J, Yahav D, Paul M. Oral iron supplements for children in malaria-endemic areas. Cochrane Database Syst Rev 2016 ; 2 : CD006589 . pmid:26921618 . OpenUrl PubMed
• Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011 ; 64 : 401 - 6 . doi:10.1016/j.jclinepi.2010.07.015 pmid:21208779 . OpenUrl CrossRef PubMed Web of Science
• ↵ Chung M, Newberry SJ, Ansari MT, et al. Two methods provide similar signals for the need to update systematic reviews. J Clin Epidemiol 2012 ; 65 : 660 - 8 . doi:10.1016/j.jclinepi.2011.12.004 pmid:22464414 . OpenUrl CrossRef PubMed
• Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007 ; 147 : 224 - 33 . doi:10.7326/0003-4819-147-4-200708210-00179 pmid:17638714 . OpenUrl CrossRef PubMed Web of Science
• Shojania K, Sampson M, Ansari M, et al. Updating systematic reviews; AHRQ technical reviews; report no 07-0087. Agency for Healthcare Research and Quality, 2007 .
• Pattanittum P, Laopaiboon M, Moher D, Lumbiganon P, Ngamjarus C. A comparison of statistical methods for identifying out-of-date systematic reviews. PLoS One 2012 ; 7 : e48894 . doi:10.1371/journal.pone.0048894 pmid:23185281 . OpenUrl CrossRef PubMed
• Shekelle PG, Motala A, Johnsen B, Newberry SJ. Assessment of a method to detect signals for updating systematic reviews. Syst Rev 2014 ; 3 : 13 . doi:10.1186/2046-4053-3-13 pmid:24529068 . OpenUrl CrossRef PubMed
• Shekelle PG, Newberry SJ, Wu H, et al. Identifying signals for updating systematic reviews: a comparison of two methods; report no 11-EHC042-EF. Agency for Healthcare Research and Quality, 2011 .
• ↵ Shekelle P, Newberry S, Maglione M, et al. Assessment of the need to update comparative effectiveness reviews: report of an initial rapid program assessment (2005-2009). Agency for Healthcare Research and Quality, 2009 .
• Tovey D, Marshall R, Bazian, Hopewell S, Rader T. Fit for purpose: centralised updating support for high-priority Cochrane Reviews; National Institute for Health Research Cochrane-National Health Service Engagement Award Scheme, July 2011. https://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/10_4000_01%20Fit%20for%20purpose%20-%20centralised%20updating%20support%20for%20high%20priority%20Cochrane%20Reviews%20FINAL%20REPORT.pdf .
• Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ 1999 ; 18 : 341 - 64 . doi:10.1016/S0167-6296(98)00039-3 pmid:10537899 . OpenUrl CrossRef PubMed Web of Science
• Wilson EC. A practical guide to value of information analysis. Pharmacoeconomics 2015 ; 33 : 105 - 21 . doi:10.1007/s40273-014-0219-x pmid:25336432 . OpenUrl CrossRef PubMed
• ↵ Althabe F, Belizán JM, McClure EM, et al. A population-based, multifaceted strategy to implement antenatal corticosteroid treatment versus standard care for the reduction of neonatal mortality due to preterm birth in low-income and middle-income countries: the ACT cluster-randomised trial. Lancet 2015 ; 385 : 629 - 39 . doi:10.1016/S0140-6736(14)61651-2 pmid:25458726 . OpenUrl CrossRef PubMed
• ↵ Taylor-Robinson D, Maayan N, Soares-Weiser K, et al. Deworming drugs for soil-transmitted intestinal worms in children: effects on nutritional indicators, haemoglobin and school performance. Cochrane Database Syst Rev 2012 ;( 11 ): CD000371 .
• ↵ van Dalen EC, van der Pal HJ, Kremer LC. Different dosage schedules for reducing cardiotoxicity in people with cancer receiving anthracycline chemotherapy. Cochrane Database Syst Rev 2016 ; 3 : CD005008 . pmid:26938118 . OpenUrl PubMed
• ↵ Wilson E. on behalf of the Cochrane Priority Setting and Campbell & Cochrane Economics Methods Groups. Which study when? Proof of concept of a proposed automated tool to help decision which reviews to update first. Cochrane Database Syst Rev 2014 ; suppl 2 : 29 - 31 .
• Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol 2010 ; 63 : 607 - 19 . doi:10.1016/j.jclinepi.2009.12.013 pmid:20434023 . OpenUrl CrossRef PubMed
• Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol 2010 ; 63 : 620 - 6 . doi:10.1016/j.jclinepi.2009.12.014 pmid:20434024 . OpenUrl CrossRef PubMed Web of Science
• Vandvik PO, Santesso N, Akl EA, et al. Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial. J Clin Epidemiol 2012 ; 65 : 748 - 55 . doi:10.1016/j.jclinepi.2011.11.013 pmid:22564503 . OpenUrl CrossRef PubMed
• ↵ International Committee of Medical Journal Editors (ICMJE). Defining the role of authors and contributors. 2016. www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html .
• ↵ Stovold E, Beecher D, Foxlee R, Noel-Storr A. Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram. Syst Rev 2014 ; 3 : 54 . doi:10.1186/2046-4053-3-54 pmid:24886533 . OpenUrl CrossRef PubMed
• ↵ Newberry SJ, Shekelle PG, Vaiana M, et al. Reporting the findings of updated systematic reviews of comparative effectiveness: how do users want to view new information? report no 13-EHC093-EF. Agency for Healthcare Research and Quality, 2013 .
• Marshall IJ, Kuiper J, Wallace BC. Automating risk of bias assessment for clinical trials BCB’14. Proceedings of the 5th ACM conference on Bioinformatics, computational biology, and health informatics. 2014:88-95. http://thirdworld.nl/automating-risk-of-bias-assessment-for-clinical-trials .
• ↵ Elliott JH, Turner T, Clavisi O, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med 2014 ; 11 : e1001603 . doi:10.1371/journal.pmed.1001603 pmid:24558353 . OpenUrl CrossRef PubMed
• Elliott J, Sim I, Thomas J, et al. #CochraneTech: technology and the future of systematic reviews. Cochrane Database Syst Rev 2014 ;( 9 ): ED000091 . pmid:25288182 .
• Cochrane. Project transform: the Cochrane Collaboration. 2016. http://community.cochrane.org/tools/project-coordination-and-support/transform .
• ↵ Paynter R, Bañez L, Berlinerm E, et al. EPC methods: an exploration of the use of text-mining software in systematic reviews. Research white paper. AHRQ publication 16-EHC023-EF. Agency for Healthcare Research and Quality, April 2016. https://www.effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productID=2214 .
• ↵ Soares-Weiser K, Marshall R, Bergman H, et al. Updating Cochrane Reviews: results of the first pilot of a focused update. Cochrane Database Syst Rev 2014 ; suppl 1 : 31 - 3 .

• Open access
• Published: 01 July 2021

Systematic and other reviews: criteria and complexities

• Robert T. Sataloff 1 , 2 ,
• Matthew L. Bush 3 ,
• Rakesh Chandra 4 ,
• Douglas Chepeha 5 ,
• Brian Rotenberg 6 ,
• Edward W. Fisher 7 ,
• David Goldenberg 8 ,
• Ehab Y. Hanna 9 ,
• Joseph E. Kerschner 10 ,
• Dennis H. Kraus 11 ,
• John H. Krouse 12 , 13 ,
• Daqing Li 14 , 15 ,
• Michael Link 16 ,
• Lawrence R. Lustig 17 ,
• Samuel H. Selesnick 18 ,
• Raj Sindwani 19 ,
• Richard J. Smith 20 ,
• James Tysome 21 ,
• Peter C. Weber 22 &
• D. Bradley Welling 23  

Journal of Otolaryngology - Head & Neck Surgery volume  50 , Article number:  41 ( 2021 ) Cite this article

4997 Accesses

1 Citations

1 Altmetric

Metrics details

Review articles can be extremely valuable. They synthesize information for readers, often provide clarity and valuable insights into a topic; and good review articles tend to be cited frequently. Review articles do not require Institutional Review Board (IRB) approval if the data reviewed are public (including private and government databases) and if the articles reviewed have received IRB approval previously. However, some institutions require IRB review and exemption for review articles. So, authors should be familiar with their institution’s policy. In assessing and interpreting review articles, it is important to understand the article’s methodology, scholarly purpose and credibility. Many readers, and some journal reviewers, are not aware that there are different kinds of review articles with different definitions, criteria and academic impact [ 1 ]. In order to understand the importance and potential application of a review article, it is valuable for readers and reviewers to be able to classify review articles correctly.

Systematic reviews

Authors often submit articles that include the term “systematic” in the title without realizing that that term requires strict adherence to specific criteria. A systematic review follows explicit methodology to answer a well-defined research question by searching the literature comprehensively, evaluating the quantity and quality of research evidence rigorously, and analyzing the evidence to synthesize an answer to the research question. The evidence gathered in systematic reviews can be qualitative or quantitative. However, if adequate and comparable quantitative data are available then a meta-analysis can be performed to assess the weighted and summarized effect size of the studies included. Depending on the research question and the data collected, systematic reviews may or may not include quantitative meta-analyses; however, meta-analyses should be performed in the setting of a systematic review to ensure that all of the appropriate data were accessed. The components of a systematic review can be found in an important article by Moher et al. published in 2009 that defined requirements for systematic reviews and meta-analyses [ 2 ].

In order to optimize reporting of meta-analyses, an international group developed the Quality of Reporting of Meta-Analyses (QUOROM) statement at a meeting in 1996 that led to publication of the QUOROM statement in 1999 [ 3 ]. Moher et al. revised that document and re-named the guidelines the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The PRISMA statement included both meta-analyses and systematic reviews, and the authors incorporated definitions established by the Cochrane Collaboration [ 4 ]. The PRISMA statement established the current standard for systematic reviews. To qualify as a systematic review, the methods section should acknowledge use of the PRISMA guidelines, and all PRISMA components should be incorporated strictly in all facets of the paper from the research question to the discussion. The PRISMA statement includes a checklist of 27 items that must be included when reporting a systematic review or meta-analysis [ 2 ]. A downloadable version of this checklist can be used by authors, reviewers, and journal editorial staff to ensure compliance with recommended components [ 5 ]. All 27 will not be listed in this brief editorial (although authors and reviewers are encouraged to consult the article by Moher et al. and familiarize themselves with all items), but a few will be highlighted.

The research question, as reflected in the title, should be a hypothesis-based specific research inquiry. The introduction must describe the rationale for the review and provide a specific goal or set of goals to be addressed. The type of systematic review, according to the Cochrane Collaboration, is based on the research question being asked and may assess diagnostic test accuracy, review prognostic studies evidence, evaluate intervention effect, scrutinize research methodology, or summarize qualitative evidence [ 6 ].

In the methods section, the participants, interventions, comparisons, outcomes and study design (PICOS) must be put forward. In addition to mentioning compliance with PRISMA, the methods section should state whether a review protocol exists and, if so, where it can be accessed (including a registration number). Systematic reviews are eligible for registration in the International Prospective Register of Systematic Reviews (PROSPERO) as established at the University of York (York, UK). When PROSPERO is used (it is available but not required for systematic reviews), registration should occur at the initial protocol stage of the review, and the final paper should direct to the information in the register. The methods section also must include specific study characteristics including databases used, years considered, languages of articles included, specific inclusion and exclusion criteria for studies; and rationale for each criterion must be included. Which individuals specifically performed searches should be noted. Electronic search strategy (with a full description of at least one electronic search strategy sufficient to allow replication of the search), process for article selection, data variables sought, assumptions and simplifications, methods for assessing bias risk of each individual study (such as selective reporting in individual studies) and utilization of this information in data synthesis, principal summary measures (risk ratio, hazard ratio, difference in means, etc.), methods of data management and combining study results, outcome level assessment, and other information should be reported.

The results section should include the number of studies identified, screened, evaluated for eligibility (including rationale for exclusion), and those included in the final synthesis. A PRISMA flow diagram should be included to provide this information succinctly [ 7 ]. The results also should include the study characteristics, study results, risk of bias within and across studies, and a qualitative or quantitative synthesis of the results of the included studies. This level of rigor in acquiring and evaluating the evidence of each individual study is one of the criteria that distinguishes systematic reviews from other categories. If the systematic review involves studies with paired samples and quantitative data, a summary of data should be provided for each intervention group along with effect estimates and confidence intervals for all outcomes of each study. If a meta-analysis is performed, then synthesized effect size should be reported with confidence intervals and measures of consistency (i.e. – data heterogeneity such as I 2 ) for each meta-analysis, and assessment of bias risk across studies. A forest plot, which provides a graphical presentation of the meta-analysis results, should be included.

The discussion section should summarize the main findings commenting on the strength of evidence for each outcome, as well as relevance to healthcare providers, policymakers and other key stake-holders; limitations of the study and outcomes; and conclusions highlighting the interpretation of results in the context of other research, and implications for future research.

Without adhering to of all of these criteria and the others listed in the PRISMA statement and checklist, the review does not qualify to be classified as “systematic”.

Meta-analyses

Meta-analyses, when feasible based on available and comparable quantitative data, supplement a systematic review evaluation, by adding a secondary statistical analysis of the pooled weighted outcomes of similar studies. This adds a level of objectivity in the synthesis of the review’s findings. Meta-analyses are appropriate when at least 2 individual studies contain paired samples (experimental group and control group) and provide quantitative outcome data and sample size. Studies that lack a control group may over-estimate the effect size of the experimental intervention or condition being studied and are not ideal for meta-analyses [ 8 ]. It also should be remembered that the conclusions of a meta-analysis are only as valid as the data on which the analysis is based. If the articles included are flawed, then the conclusions of the meta-analysis also may be flawed. Systematic reviews and meta-analyses are the most rigorous categories of review.

Other types of reviews

Mixed methods reviews.

Systematic reviews typically contain a single type of data, either qualitative or quantitative; however, mixed methods reviews bring together a combination of data types or study types. This approach may be utilized when quantitative data, in the setting of an intervention study, only provide a narrow perspective of the efficacy or effectiveness of the intervention. The addition of qualitative data or qualitative studies may provide a more complete picture of the knowledge, attitudes, and behaviors of clinicians, patients or researchers regarding that intervention. This type of review could involve collecting either the quantitative or the qualitative data using systematic review methodology, but often the qualitative data are gathered using a convenience sampling. Many qualitative studies provide useful insights into clinical management and/or implementation of research interventions; and incorporating them into a mixed methods review may provide valuable perspective on a wide range of literature. Mixed methods reviews are not necessarily systematic in nature; however, authors conducting mixed methods reviews should follow systematic review methodology, when possible.

Literature and narrative reviews

Literature reviews include peer-reviewed original research, systematic reviews, and meta-analyses, but also may include conference abstracts, books, graduate degree theses, and other non-peer reviewed publications. The methods used to identify and evaluate studies should be specified, but they are less rigorous and comprehensive than those required for systematic reviews. Literature reviews can evaluate a broad topic but do not specifically articulate a specific question, nor do they synthesize the results of included studies rigorously. Like mixed method reviews, they provide an overview of published information on the topic, although they may be less comprehensive than integrative reviews; and, unlike systematic reviews, they do not need to support evidence-based clinical or research practices, or highlight high-quality evidence for the reader. Narrative reviews are similar to literature reviews and evaluate the same scope of literature. The terms sometimes are used interchangeably, and author bias in article selection and data interpretation is a potential concern in literature and narrative reviews.

Umbrella reviews

An umbrella review integrates previously published, high-quality reviews such as systematic reviews and meta-analyses. Its purpose is to synthesize information in previously published systematic reviews and meta-analyses into one convenient paper.

Rapid review

A rapid review uses systematic review methodology to evaluate existing research. It provides a quick synthesis of evidence and is used most commonly to assist in emergent decision-making such as that required to determine whether COVID-19 vaccines should receive emergent approval.

Scoping, mapping, and systematized reviews

If literature has not been reviewed comprehensively in a specific subject that is varied and complex, a mapping review (also called scoping review) may be useful to organize initial understanding of the topic and its available literature. While mapping reviews may be helpful in crystallizing research findings and may be published, they are particularly useful in helping to determine whether a topic is amenable to systematic review, and to help organize and direct the approach of the systematic review or other reviews of the subject. Systematized reviews are used most commonly by students. The systematized review provides initial assessment of a topic that is potentially appropriate for a systematic review, but a systematized review does not meet the rigorous criteria of a systematic review and has substantially more limited value. Additional types of reviews exist including critical review, state-of-the-art review, and others.

Reviews can be invaluable; but they also can be misleading. Systematic reviews and meta-analyses provide readers with the greatest confidence that rigorous efforts have attempted to eliminate bias and ensure validity, but even they have limitations based upon the strengths and weaknesses of the literature that they have assessed (and the skill and objectivity with which the authors have executed the review). Risks of bias, incomplete information and misinformation increase as the rigor of review methodology decreases. While review articles may summarize research related to a topic for readers, non-systematic reviews lack the rigor to answer adequately hypothesis-driven research questions that can influence evidence-based practice. Journal authors, reviewers, editorial staff, and should be cognizant of the strengths and weaknesses of review methodology and should consider them carefully as they assess the value of published review articles, particularly as they determine whether the information presented should alter their patient care.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x .

Article   Google Scholar  

Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097.

Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reporting of meta-analysis of randomized controlled trials: the QUOROM statement. Lancet. 1999;354(9193):1896–900. https://doi.org/10.1016/S0140-6736(99)04149-5 .

Article   CAS   PubMed   Google Scholar  

Green S, Higgins J (eds). Glossary. Cochrane handbook for systematic reviews of interventions 4.2.5. The Cochrane Collaboration. 2005. Available at http://www.cochrane.org/resources/glossary.htm . Accessed on 19 May 2009.

Google Scholar  

PRISMA 2009 Checklist. PRISMA-Statement.org . http://prisma-statement.org/documents/PRISMA%202009%20checklist.pdf . Accessed 1/19/2021.

Chapter I: Introduction. Cochrane Training. https://training.cochrane.org/handbook/archive/v6/chapter-i . Access 1/19/2021.

PRISMA 2009 Flow diagram. PRISMA-Statement.org . http://www.prisma-statement.org/documents/PRISMA%202009%20flow%20diagram.pdf . Accessed 1/19/2021.

Goodacre S. Uncontrolled before-after studies: discouraged by Cochrane and the EMJ. Emerg Med J. 2015;32(7):507–8. https://doi.org/10.1136/emermed-2015-204761 .

Article   PubMed   Google Scholar  

Download references

Author information

Authors and affiliations.

Editor-in-Chief, Journal of Voice, Philadephia, USA

Robert T. Sataloff

Editor Emeritus, Ear, Nose and Throat Journal, Philadephia, USA

Assistant Editor, Otology & Neurotology, Lexington, USA

Matthew L. Bush

Editor-in-Chief, Ear, Ear, Nose and Throat Journal, Nashville, USA

Rakesh Chandra

Editor-in-Chief, Journal of Otolaryngology – Head & Neck Surgery, Toronto, Canada

Douglas Chepeha

Editor-in-Chief, Journal of Otolaryngology – Head & Neck Surgery, London, Canada

Brian Rotenberg

Senior Editor, Journal of Laryngology and Otology, Birmingham, UK

Edward W. Fisher

Editor-in-Chief, Operative Techniques in Otolaryngology – Head and Neck Surgery, Hershey, USA

David Goldenberg

Editor-in-Chief, Head & Neck, Houston, USA

Ehab Y. Hanna

Editor-in-Chief, International Journal of Pediatric Otorhinolaryngology, Milwaukee, USA

Joseph E. Kerschner

Co-Editor-in-Chief, Journal of Neurological Surgery Part B: Skull Base, New York, USA

Dennis H. Kraus

Editor-in-Chief, Otolaryngology – Head and Neck Surgery, Philadelphia, USA

John H. Krouse

Editor-in-Chief, OTO-Open, Philadelphia, USA

Editor-in-Chief, Journal for Oto-Rhino-Laryngology, Head and Neck Surgery, Philadelphia, USA

Editor-in-Chief, World Journal of Otorhinolaryngology – Head and Neck Surgery, Philadelphia, USA

Co-Editor-in-Chief, Journal of Neurological Surgery Part B: Skull Base, Rochester, USA

Michael Link

Editor-in-Chief, Otology & Neurotology, New York, USA

Lawrence R. Lustig

Editor-in-Chief, The Laryngoscope, New York, USA

Samuel H. Selesnick

Editor-in-Chief, American Journal of Rhinology & Allergy, Cleveland, USA

Raj Sindwani

Editor-in-Chief, Annals of Otology, Rhinology & Laryngology, Iowa City, USA

Richard J. Smith

Editor-in-Chief, Clinical Otolaryngology, Cambridge, UK

James Tysome

Editor-in-Chief, American Journal of Otolaryngology, Boston, USA

Peter C. Weber

Editor-in-Chief, Laryngoscope Investigative Otolaryngology, Boston, USA

D. Bradley Welling

You can also search for this author in PubMed   Google Scholar

Contributions

The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Robert T. Sataloff .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is co-published in the following journals: Journal of Voice, Otology & Neurotology, Ear, Nose and Throat Journal, Journal of Laryngology and Otology, Operative Techniques in Otolaryngology – Head and Neck Surgery, Head & Neck, International Journal of Pediatric Otorhinolaryngology, Journal of Neurological Surgery Part B: Skull Base, Otolaryngology – Head and Neck Surgery, World Journal of Otorhinolaryngology – Head and Neck Surgery, The Laryngoscope, American Journal of Rhinology & Allergy, Annals of Otology, Rhinology & Laryngology, Clinical Otolaryngology, American Journal of Otolaryngology, Laryngoscope Investigative Otolaryngology.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sataloff, R.T., Bush, M.L., Chandra, R. et al. Systematic and other reviews: criteria and complexities. J of Otolaryngol - Head & Neck Surg 50 , 41 (2021). https://doi.org/10.1186/s40463-021-00527-9

Download citation

Published : 01 July 2021

DOI : https://doi.org/10.1186/s40463-021-00527-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Journal of Otolaryngology - Head & Neck Surgery

ISSN: 1916-0216

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 13 December 2023

Arts and creativity interventions for improving health and wellbeing in older adults: a systematic literature review of economic evaluation studies

• Grainne Crealey 1 ,
• Laura McQuade 2 ,
• Roger O’Sullivan 2 &
• Ciaran O’Neill 3  

BMC Public Health volume  23 , Article number:  2496 ( 2023 ) Cite this article

1166 Accesses

22 Altmetric

Metrics details

As the population ages, older people account for a larger proportion of the health and social care budget. A significant body of evidence suggests that arts and creativity interventions can improve the physical, mental and social wellbeing of older adults, however the value and/or cost-effectiveness of such interventions remains unclear.

We systematically reviewed the economic evidence relating to such interventions, reporting our findings according to PRISMA guidelines. We searched bibliographic databases (MEDLINE, EMBASE, Econlit and Web of Science and NHSEED), trial registries and grey literature. No language or temporal restrictions were applied. Two screening rounds were conducted independently by health economists experienced in systematic literature review. Methodological quality was assessed, and key information extracted and tabulated to provide an overview of the published literature. A narrative synthesis without meta-analysis was conducted.

Only six studies were identified which provided evidence relating to the value or cost-effectiveness of arts and creativity interventions to improve health and wellbeing in older adults. The evidence which was identified was encouraging, with five out of the six studies reporting an acceptable probability of cost-effectiveness or positive return on investment (ranging from £1.20 to over £8 for every £1 of expenditure). However, considerable heterogeneity was observed with respect to study participants, design, and outcomes assessed. Of particular concern were potential biases inherent in social value analyses.

Conclusions

Despite many studies reporting positive health and wellbeing benefits of arts and creativity interventions in this population, we found meagre evidence on their value or cost-effectiveness. Such evidence is costly and time-consuming to generate, but essential if innovative non-pharmacological interventions are to be introduced to minimise the burden of illness in this population and ensure efficient use of public funds. The findings from this review suggests that capturing data on the value and/or cost-effectiveness of such interventions should be prioritised; furthermore, research effort should be directed to developing evaluative methods which move beyond the confines of current health technology assessment frameworks, to capture a broader picture of ‘value’ more applicable to arts and creativity interventions and public health interventions more generally.

PROSPERO registration

CRD42021267944 (14/07/2021).

Peer Review reports

The number and proportion of older adults in the population has increased in virtually every country in the world over past decades [ 1 ]. In 2015, there were around 901 million people aged 60 years and over worldwide, by 2030, this will have increased to 1.4 billion [ 2 ]. An ageing population is one of the greatest successes of public health but it has implications for economies in numerous ways: slower labour force growth; working-age people will have to make greater provisions in welfare payments for older people who are no longer economically active; provisions for increased long-term care; and, society must adjust to the changing needs, expectations and capabilities of an expanding group of its citizens.

The Covid-19 pandemic shone an uncompromising light on the health and social care sector, highlighting the seriousness of gaps in policies, systems and services. It also focused attention on the physical and mental health consequences of loneliness and social isolation. To foster healthy ageing and improve the lives of older people, their families and communities, sustained and equitable investment in health and wellbeing is required [ 3 ]. The prevailing model of health and social care which is based ostensibly on formal care provision is unlikely to be sustainable over the longer term. New models, which promote healthy ageing and recognise the need for increasing reliance on self-care are required, as will be evidence of their effectiveness, cost-effectiveness and scalability.

Arts and creativity interventions (ACIs) can have positive effects on health and well-being, as several reviews have shown [ 4 , 5 ]. For older people, ACI’s can enhance wellbeing [ 6 , 7 , 8 , 9 ], quality of life [ 10 , 11 ] and cognitive function [ 12 , 13 , 14 , 15 , 16 ]. They can also foster social cohesion [ 17 , 18 , 19 ] and reduce social disparities and injustices [ 20 ]; promote healthy behaviour; prevent ill health (including enhancing well-being and mental health) [ 21 , 22 , 23 , 24 , 25 ], reducing cognitive decline [ 26 , 27 ], frailty [ 28 , 29 , 30 , 31 , 32 , 33 ] and premature mortality [ 34 , 35 , 36 , 37 , 38 ]); support people with stroke [ 39 , 40 , 41 , 42 ]; degenerative neurological disorders and dementias and support end of life care [ 43 , 44 ]. Moreover, ACIs can benefit not only individuals, but also others, such as supporting the well-being of formal and informal carers, enriching our knowledge of health, and improving clinical skills [ 4 , 5 ].

The benefits of ACIs have also been acknowledged at a governmental level by those responsible for delivering health and care services: The UK All-Party Parliamentary Special Interest group on Arts, Health and Wellbeing produced a comprehensive review of creative intervention for health and wellbeing [ 45 ]. This report contained three key messages: that the arts can keep us well, aid recovery and support longer better lived lives; they can help meet major challenges facing health and social care; and that the arts can save money for the health service and social care.

Despite robust scientific evidence and governmental support, no systematic literature review has collated the evidence with respect to the value, cost or cost-effectiveness of such interventions. Our objective was to assess the economic impact of ACIs aimed at improving the health and wellbeing of older adults; to determine the range and quality of available studies; identify gaps in the evidence-base; and guide future research, practice and policy.

A protocol for this review was registered at PROSPERO, an international prospective register of systematic reviews (Registration ID CRD42021267944). We used pre-determined criteria for considering studies to include in the review, in terms of types of studies, participant and intervention characteristics.

The review followed the five-step approach on how to prepare a Systematic Review of Economic Evaluations (SR-EE) for informing evidence-based healthcare decisions [ 46 , 47 , 48 ]. Subsequent to developing and registering the protocol, the International Society for Pharmacoeconomic Outcomes and Research (ISPOR) published a good practice task force report for the critical appraisal of systematic reviews with costs and cost-effectiveness outcomes (SR-CCEOs) [ 49 ]. This was also used to inform the conduct of this review.

Eligibility criteria

Full economic evaluations are regarded as the optimal type of evidence for inclusion in a SR-EE [ 46 ], hence cost-minimisation analyses (CMA), cost-effectiveness analyses (CEA), cost-utility analyses (CUA) and cost–benefit analyses (CBA) were included. Social value analyses were also included as they are frequently used to inform decision-making and commissioning of services within local government. Additionally, they represent an important intermediate stage in our understanding of the costs and consequences of public health interventions, where significant challenges exist with regard to performing full evaluations [ 50 , 51 , 52 , 53 ].

Development of search strategies

The population (P), intervention (I), comparator (C) and outcomes (O) (PICO) tool provided a framework for development of the search strategy. Studies were included if participants were aged 50 years or older (or if the average age of the study population was 50 years or over). Interventions could relate to performance art (dance, singing, theatre, drama etc.), creative and visual arts (painting, sculpture, art making and design), or creative writing (writing narratives, poetry, storytelling). The intervention had to be active (for example, creating art as opposed to viewing art; playing an instrument as opposed to listening to music). The objective of the intervention had to be to improve health and wellbeing; it had to be delivered under the guidance of a professional; delivered in a group setting and delivered on more than one occasion. No restrictions were placed on the type of comparator(s) or the type of outcomes captured in the study. We deliberately limited the study to professionally led activities to provide a sharper distinction between social events where arts and creativity may occur and arts and creativity interventions per se. We set no language restriction nor a restriction on the date from which studies were reported.

Search methods

PRESS (peer-review electronic search strategies) guidelines informed the design our search strategy [ 54 , 55 ] and an information specialist adapted the search terms (outlined in Table S 1 ) for the following electronic bibliographic databases: MEDLINE, PubMed, EMBASE, Econlit and Web of Science and NHSEED. We also inspected references of all relevant studies; and searched trials registers (ClinicalTrials.gov). Search terms used included cost, return on investment, economic, arts, music, storytelling, dancing, writing and older adult as well as social return on investment (SROI). The last search was performed on 09/11/2022. As many economic evaluations of ACIs (especially SROIs) are commissioned by government bodies or charitable organisations, a search of the grey literature was undertaken.

Handling searches

A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow chart was used to document study selection, illustrating the numbers of records retrieved and selection flow through the screening rounds [ 56 , 57 , 58 ]; all excluded records (with rationale for exclusion) were documented.

Selection of studies

Two screening rounds were conducted independently by two health economists experienced in undertaking reviews (GC, CO’N). The first round screened the title and abstract of articles based on the eligibility criteria; those selected at this stage entered a second round of full text screening with eligibility based on the inclusion and exclusion criteria. Any disagreements were discussed among the two reviewers, with access to a third reviewer available to resolve disagreements, though this proved unnecessary.

Data extraction and management

Two reviewers extracted relevant information independently using an proforma developed specifically for the purposes of this study, which included all 35 items suggested by Wijnen et al. (2016) [ 48 ]. Information was extracted in relation to the following factors: (1) general information including study title, author, year, funding source, country, setting and study design; (2) recruitment details, sample size, demographic characteristics (age, gender) and baseline health data (diagnosis, comorbidities); (3) interventions, effectiveness and cost data; (4) type of economic evaluation, perspective, payer, beneficiary, time horizon, measure of benefit and scale of intervention; (5) quality assessment, strength of evidence, any other important information; (6) results; (7) analysis of uncertainty and (8) conclusions. The quality assessment/risk of bias checklists were included in the data extraction proforma, and picklists were used to enhance uniformity of responses. The data extraction form was piloted by two reviewers (GC and CON) on one paper and discussion used to ensure consistent application thereafter.

Assessment of study quality

Two reviewers (GC & CON) independently assessed study quality, with recourse to a third reviewer for resolution of differences though this proved unnecessary. Quality assessment was based on the type of economic evaluation undertaken. Full and partial trial-based economic evaluations were assessed using the CHEC-extended checklist [ 59 ]. SROI analyses were assessed using a SROI-specific quality framework developed for the purpose of systematic review [ 60 ].

Data analysis methods

Due to the small number of evaluations detected, possible sources of heterogeneity and a lack of consensus on appropriate methods for pooling cost-effectiveness estimates [ 61 ] a narrative synthesis analysis was undertaken.

Database searches returned 11,619 records; from this, 402 duplicates were removed leaving 11,214 reports. From these 113 reports were assessment against the inclusion and exclusion criteria resulting in 4 studies for inclusion in the review. Over 40 websites were searched for relevant content returning 2 further studies for inclusion. The PRISMA 2020 diagram is presented in Fig.  1 . A high sensitivity search strategy was adopted to ensure all relevant studies were identified, resulting in a large number of studies being excluded at the first stage of screening.

PRISMA 2020 flow diagram for new systematic reviews which include searches of databases, registers and other sources

A total of six studies were identified; key characteristics are presented in Table 1 . Identified studies were published between 2011 and 2020. Two studies used a health technology assessment (HTA) framework alongside clinical trials [ 62 , 63 ] to assess the cost-effectiveness of community singing interventions. Both evaluations scored highly on the CHEC-extended checklist (Table 2 ), with findings reported in line with the CHEERS (Consolidated Health Economic Estimation Reporting Standards) checklist 2022 [ 64 ].

Four further studies employed an SROI framework to assess art and/or craft interventions: two studies were published in the peer-reviewed literature [ 65 , 66 ] and a further two in the grey literature [ 67 , 68 ]. All four adhered closely to the suggested steps for performing an SROI and consequently secured high scores (Table 3 ). No quality differential was discerned between those studies published in the academic literature when compared with those from the grey literature.

Five of the studies were undertaken in the UK [ 63 , 66 , 67 , 68 , 69 ] and one in the US [ 63 ]. Four of the studies were designed for older adults with no cognitive impairment [ 62 , 63 , 67 , 68 ]; one was designed for participants with or without dementia [ 65 ], and another was specifically for older adults with dementia and their caregivers [ 66 ]. Three of the studies were delivered in a community setting [ 62 , 63 , 67 ], two in care homes [ 65 , 68 ] and one across a range of settings (hospital, community and residential) [ 66 ]. The length and duration of the ACIs varied; some lasted 1–2 h (with multiple classes available to participants) [ 65 ], whereas others were structured programmes with sessions lasting 90 min over a 14-week period [ 62 ]. The number of participants included in studies varied; the largest study contained data from 390 participants [ 63 ], whereas other studies measured engagement using numbers of care homes or housing associations included [ 67 , 68 ].

Costs were captured from a narrower perspective (i.e., the payer—health service) for those economic evaluations which followed a health technology assessment (HTA) framework [ 62 , 63 ]. Costs associated with providing the programme and health and social care utilisation costs were captured using cost diaries. Valuation of resource usage was in line with the reference case specified for each jurisdiction.

Social value analyses included in the review [ 65 , 66 , 67 , 68 ] captured a broader picture of cost; programme provision costs included were similar in nature to those identified using an HTA framework, however, the benefits captured went beyond the individual to capture costs to a wide range of stakeholders such as family members, activity co-ordinations and care home personnel. Costs were apportioned using financial proxies from a range of sources including HACT Social Value Bank [ 69 ] and market-based valuation methods.

The range of outcomes captured and valued across HTAs and SROIs was extensive: including, but not limited to, wellbeing, quality of life, physical health, cognitive functioning, communication, control over daily life choices, engagement and empowerment, social isolation, mobility, community inclusion, depressive symptoms, sadness, anxiety, loneliness, positive affect and interest in daily life. In the programmes assessed using an HTA framework, outcomes were captured using standardised and validated instruments, for both control and intervention groups across multiple time points. Statistical methods were used to assess changes in outcomes over time. Programmes assessed using SROI relied primarily on qualitative methods (such as reflective diaries and in-depth interviews) combined with routinely collected administrative data.

The evidence from the singing interventions was encouraging but not conclusive. The ‘Silver Song Club’ programme [ 62 ] reported a 64% probability of being cost-effective at a willingness-to-pay threshold of £30,000. This study was also included in the Public Health England (PHE) decision tool to support local commissioners in designing and implementing services to support older people’s healthy ageing, reporting a positive societal return on investment [ 70 ]. Evidence from the ‘Community of Voices’ trial [ 63 ] suggested that although intervention group members experienced statistically significant improvements in loneliness and interest in life compared to control participants, no significant group differences were observed for cognitive or physical outcomes or for healthcare costs.

A positive return on investment was reported by all social value analyses undertaken. The ‘Imagine Arts’ programme, reported a positive SROI of £1.20 for every £1 of expenditure [ 65 ]. A higher yield of between £3.20-£6.62 for each £1 invested was reported in the ‘Dementia and Imagination’ programme [ 66 ]. The ‘Craft Café’ programme, reported an SROI of £8.27 per £1 invested [ 68 ], and the ‘Creative Caring’ programme predicted a SROI of between £3 to £4 for every £1 spent [ 67 ]. The time period over which return on investment was calculated differed for each evaluation from less than one year to 4 years.

The primary finding from our review concerns the paucity of evidence relating to the value, cost and/or cost-effectiveness of ACIs aimed at improving health and wellbeing in this population. Despite few restrictions being applied to our search, only six studies were found which met our inclusion criteria. This is not indicative of research into ACIs in this population, as evidenced by the identification of ninety-three studies where arts and creativity interventions were found to support better health and wellbeing outcomes in another recent review [ 5 ]. An alternative explanation is that funders do not see the added value of undertaking such evaluations in this area. That is, for funders, the cost of evaluating an ACIs is likely to be deemed unjustified given the relatively small welfare loss a misallocation of resources to them might produce. While at first glance this may seem reasonable, it disadvantages ACIs in competing with other interventions for funding and arguably exposes an implicit prejudice in the treatment of interventions from which it may be difficult to extract profit in general. That is, the paucity of evidence, may reflect inherent biases within our political economy that favour the generation of marketable solutions to health issues from which value can be appropriated as profit. Pharmaceuticals are an obvious example of such solutions, where the literature is replete with examples of evaluations sponsored by pharmaceutical companies or where public funds are used to test the claims made by pharmaceutical companies in respect of the value of their products. If the potential of ACIs to improve health and well-being is to be robustly established, ACIs must effectively compete for funding with other interventions including those from pharma. This requires a larger, more robust evidence base than is currently available and investment in the creation of such an evidence base. As there is currently no ‘for-profit’ industry to generate such an evidence base, public funding of evaluations will be central to its creation.

Our second finding concerns the values reported in the meagre evidence we did find. In five of the six studies we identified, evidence indicated that ACIs targeted at older people offered value for money [ 62 , 65 , 66 , 67 , 68 ]. One study provided mixed evidence [ 63 ], however, in this study a ‘payer’ perspective was adopted when applying an HTA framework which, by virtue of the perspective adopted, excluded a range of benefits attributable to ACIs and public health interventions more generally. Among the four studies that adopted a SROI approach, estimated returns per £1 invested ranged from £1.20 to £8.27. Given the evident heterogeneity among studies in terms of context and methods, care is warranted in comparing estimates with each other or with other SROIs. Care is also required in accepting at face value the estimates reported given methodological issues that pertain to the current state of the art with respect to SROI. With these caveats in mind noted, the values reported for ACIs using the SROI approach are comparable with those from other SROI studies in other contexts including those as diverse as a first aid intervention [ 71 ], investment in urban greenways [ 72 ] and the provision of refuge services to those experiencing domestic violence [ 73 ] (a return on investment of £3.50-£4, £2.88-£5.81 and £4.94 respectively). Similarly, with respect to the study that adopted a cost-effectiveness approach, Coulton and colleagues (2015) reported a 64% probability of the intervention being cost-effective at a threshold of £30,000 [ 62 ]. Again, it is difficult to compare studies directly, but this is similar to that reported for interventions as diverse as a falls prevention initiative [ 74 ] and the treatment of depression using a collaborative approach [ 75 ] both in the UK. That the evidence base is meagre notwithstanding, there is, in other words, a prima facie case that ACIs are capable of offering value for money when targeted at older persons.

Our third finding relates to the state of the art with respect to SROIs in this area. Over the past 40 years, considerable time, effort and resources have been expended in the development of cost-effectiveness techniques in health and social care. While considerable heterogeneity can exist around their conduct, national guidance exists in many jurisdictions on the conduct of cost-effectiveness analyses (CEA) – such as the NICE reference case in the UK [ 76 ]– as well as in the reporting of these as set out in the CHEERS 2022 guidance [ 64 ]. This has helped raise the quality of published evaluations and the consistency with which they are reported. Despite the existence of a step-by-step guidance document on how to perform SROIs [ 77 ] which outlines how displacement effects, double counting, effect attribution and drop-off should be addressed, a significant body of work still remains to ensure that the methodology addresses a range of known biases in a robust manner. Where there is no comparator to the intervention being evaluated (as was the case in the SROIs reported here) it may be difficult to convince funders that the implicit incremental costs and benefits reported are indeed incremental and attributable to the intervention. Equally, where a comparator is present, greater consensus and standardisation is required regarding the identification, generation and application of, for example, financial proxies. Currently, SROI ratios combine value across a wide range of stakeholders, which is understandable if the objective is to capture all aspects of social benefit generated. This ratio, however, may not reflect the priorities and statutory responsibilities of healthcare funders. Whist all of the aforementioned issues can be addressed, investment is required to develop the SROI methodology further to more closely meet the needs of commissioning bodies.

Notwithstanding these challenges, social value analyses play a pivotal role within the procurement processes employed by government, local authorities and other non-departmental public bodies and should not be dismissed simply because the ‘burden of proof’ falls short of that required to secure remuneration within the health sector. As most SROIs are published in the grey literature, this means they often avoid peer scrutiny prior to publication and the potential quality assurance this can offer. It is noteworthy however that two of the SROIs included in this review [ 65 , 66 ] were published in the academic literature, suggesting that the academic community are engaging with this method which is to be applauded.

Moving forward, it is unlikely we will be able to meet all of the health and wellbeing needs of our ageing population solely in a primary or secondary care setting. New models of care are required, as are new models of funding to support interventions which can be delivered in non-healthcare settings. New hybrid models of evaluation will be required to provide robust economic evidence to assist in the allocation of scarce resources across health and non-healthcare settings; such evaluative frameworks must have robust theoretical underpinnings and be capable of delivering evidence from a non-clinical setting in a timely and cost-effective manner.

In the absence of a definitive evaluation framework for ACIs being currently available, we have a number of recommendations. First, and most importantly, all impact assessments should have a control group or credible counterfactual. This is currently not required when performing an SROI making it difficult to determine if all of the benefits ascribed to an intervention are in fact attributable. This recommendation is in line with the conclusion of a report by the London School of Economics [ 78 ] for the National Audit Office (NAO) which concluded that ‘any impact evaluation (and subsequent value for money calculation) requires construction of a counterfactual’. Second, a detailed technical appendix should accompany all impact assessments to allow independent review by a subject specialist. While this would assist peer review, it would allow providing greater transparency where peer review was not undertaken prior to publication. Furthermore, it would enable recalculation of SROI ratios to exclude ‘value’ attributable to stakeholders which are not relevant to a particular funder. Third, equity considerations should be addressed explicitly in all evaluations (this is currently not required in HTAs). Fourth, both costs and outcomes should be captured from a ‘broad’ perspective (adopting a ‘narrow’ healthcare perspective may underestimate the full economic impact), with non-healthcare sector costs being detailed as part of the analysis. Finally, data should be collected post-implementation to ensure that resources continue to be allocated efficiently.

As with any review, there are limitations which should be noted. A search of the grey literature was included as evaluations of applied public health interventions are not always reported in the academic literature. Systematically identifying grey literature and grey data can be problematic [ 79 , 80 , 81 , 82 , 83 ] as it is not collected, organised or stored in a consistent manner. Hence it is possible that we have not identified all relevant studies. Furthermore, as applied public health interventions can be performed in a non-healthcare setting we included SROIs in our review of economic evaluations. Current guidance on the systematic review of economic evaluations has been developed primarily for review of HTA as opposed to public health interventions and hence SROIs would be excluded, or if included would score poorly due to the inherent biases arising from no comparator or counterfactual being included.

This systematic review found that participation in group-based arts and creativity programmes was generally cost-effective and/or produced a positive return on investment whilst having a positive impact on older people’s physical, psychological, and social health and wellbeing outcomes. Unfortunately, the small number of studies identified, coupled with differences in methods used to assess economic impact hinders our ability to conclusively determine which types of art and creativity-based activities are more cost-effective or represent best value for money.

As well as the need for a greater focus on prevention of poor health as we age, new hybrid models of healthcare delivery are necessary to meet the needs of our ageing population. These models will integrate traditional medical care with other services such as home health aides (some of which may include artificial intelligence), telemedicine and social support networks. Alongside these, ACIs have the potential to provide a low cost, scalable, easily implementable and cost-effective solution to reduce the burden of illness in this age group and support healthy ageing.

Evidence on the cost-effectiveness of a range of ACIs is of utmost importance for policy and decision makers as it can both inform the development of policies that support the provision of ACIs in the context of ageing, but also identify the most cost-effective approaches for delivering such interventions. The development of hybrid models of evaluation, capable of capturing cost-effectiveness and social value, is becoming increasingly necessary as healthcare delivery for this age group moves beyond the realms of primary and secondary care and into the community. The development and refinement of such models will ensure a more comprehensive assessment of the impact of a diverse range of interventions providing a more nuanced understanding of the impact of an intervention. This will help inform decision making and ensure interventions are implemented in a cost-effective and socially beneficial manner.

Availability of data and materials

All data generated or analysed during this study are included in the published article and its supplementary information files.

United Nations, Department of Economic and Social Affairs, Population Division. World Population Prospects: The 2015 Revision, Key Findings and Advance Tables. Working Paper No. ESA/P/WP.241. 2015.

Office for National Statistics. Living longer: how our population is changing and why it matters. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/ageing/articles/livinglongerhowourpopulationischangingandwhyitmatters/2018-08-13#how-do-incomes-of-older-people-compare-with-younger-ages . 2018. Accessed 07/12/2022

Dyakova M, Hamelmann C, Bellis MA, Besnier E, Grey CNB, Ashton K, Schwappach A, Clar C. Investment for health and well-being: a review of the social return on investment from public health policies to support implementing the Sustainable Development Goals by building on Health 2020 [Internet]. Copenhagen: WHO Regional Office for Europe; 2017.

Google Scholar  

Fancourt D, Finn S. What is the evidence on the role of the arts in improving health and well-being? A scoping review. Copenhagen: WHO Regional Office for Europe; 2019.

McQuade L, O’Sullivan R. Examining arts and creativity in later life and its impact on older people’s health and wellbeing: a systematic review of the evidence. Perspect Publ Health. 2023;0(0). https://doi.org/10.1177/17579139231157533

Skingley A, De’Ath S, Napleton L. Evaluation of Edna: arts and dance for older people. Work Older People. 2016;20(1):46–56.

Article   Google Scholar  

Brustio PR, Liubicich ME, Chiabrero M, et al. Dancing in the golden age: a study on physical function, quality of life, and social engagement. Geriatr Nurs. 2018;39(6):635–9.

Article   PubMed   Google Scholar  

Beauchet O, Bastien T, Mittelman M, et al. Participatory art-based activity, community-dwelling older adults and changes in health condition: results from a pre-post intervention, single-arm, prospective and longitudinal study. Maturitas. 2020;134:8–14.

Article   CAS   PubMed   Google Scholar  

Roswiyani R, Hiew CH, Witteman CLM, et al. Art activities and qigong exercise for the well-being of older adults in nursing homes in Indonesia: a randomized controlled trial. Aging Ment Health. 2020;24(10):1569–78.

Shanahan J, Bhriain ON, Morris ME, et al. Irish set dancing classes for people with Parkinson’s disease: the needs of participants and dance teachers. Complement Ther Med. 2016;27:12–7.

Garcia Gouvêa JA, Antunes MD, Bortolozzi F, et al. Impact of senior dance on emotional and motor parameters and quality of life of the elderly. Rev Rene. 2017;18(1):51–8.

Sun J, Zhang N, Buys N, et al. The role of Tai Chi, cultural dancing, playing a musical instrument and singing in the prevention of chronic disease in Chinese older adults: a mind–body meditative approach. Int J Ment Health Pr. 2013;15:227–39.

Fu MC, Belza B, Nguyen H, et al. Impact of group-singing on older adult health in senior living communities: a pilot study. Arch Gerontol Geriatr. 2018;76:138–46.

Feng L, Romero-Garcia R, Suckling J, et al. Effects of choral singing versus health education on cognitive decline and aging: a randomized controlled trial. Aging-us. 2020;12(24):24798–816.

Seinfeld S, Figueroa H, Ortiz-Gil J, et al. Effects of music learning and piano practice on cognitive function, mood and quality of life in older adults. Front Psychol. 2013;4:810.

Article   PubMed   PubMed Central   Google Scholar  

MacRitchie J, Breaden M, Milne AJ, et al. Cognitive, motor and social factors of music instrument training programs for older adults’ improved wellbeing. Front Psychol. 2020;10:2868.

Freeman WJI. A neurobiological role of music in social bonding. In: Wallin N, Merkur B, Brown S, editors. The origins of music. Cambridge: MIT Press; 2000. http://escholarship.org/uc/item/9025x8rt .

Huron D. Is music an evolutionary adaptation? Ann N Y Acad Sci. 2001;930(1):43–61. https://doi.org/10.1111/j.1749-6632.2001.tb05724.x .

Tarr B, Launay J, Dunbar RIM. Music and social bonding: “self–other” merging and neurohormonal mechanisms. Front Psychol. 2014;5:1096. https://doi.org/10.3389/fpsyg.2014.01096 .

Cain M, Lakhani A, Istvandity L. Short and long term outcomes for culturally and linguistically diverse (cald) and at-risk communities in participatory music programs: a systematic review. Arts Health. 2016;8(2):105–24. https://doi.org/10.1080/17533015.2015.1027934 .

Martin L, Oepen R, Bauer K, Nottensteiner A, Mergheim K, Gruber H, et al. Creative arts interventions for stress management and prevention – a systematic review. Behav Sci (Basel). 2018;8(2):pii:E28. https://doi.org/10.3390/bs8020028 .

Linnemann A, Wenzel M, Grammes J, Kubiak T, Nater UM. Music listening and stress in daily life: a matter of timing. Int J Behav Med. 2018;25(2):223–30. https://doi.org/10.1007/s12529-017-9697-5 .

Linnemann A, Strahler J, Nater UM. The stress-reducing effect of music listening varies depending on the social context. Psychoneuroendocrinology. 2016;72:97–105. https://doi.org/10.1016/j.psyneuen.2016.06.003 .

Panteleeva Y, Ceschi G, Glowinski D, Courvoisier DS, Grandjean DM. Music for anxiety? meta-analysis of anxiety reduction in non-clinical samples. Psychol Music. 2017;46(4):473–87. https://doi.org/10.1177/0305735617712424 .

Fancourt D, Tymoszuk U. Cultural engagement and incident depression in older adults: evidence from the English longitudinal study of ageing. Br J Psychiatry. 2018;214(4):225–9. https://doi.org/10.1192/bjp.2018.267 .

Balbag MA, Pedersen NL, Gatz M. Playing a musical instrument as a protective factor against dementia and cognitive impairment: a population-based twin study. Int J Alzheimer’s Dis. 2014;2014:836748. https://doi.org/10.1155/2014/836748 .

Porat S, Goukasian N, Hwang KS, Zanto T, Do T, Pierce J, et al. Dance experience and associations with cortical gray matter thickness in the aging population. Dement Geriatr Cogn Dis Extra. 2016;6(3):508–17. https://doi.org/10.1159/000449130 .

Federici A, Bellagamba S, Rocchi MBL. Does dance-based training improve balance in adult and young old subjects? a pilot randomized controlled trial. Aging Clin Exp Res. 2005;17(5):385–9 PMID: 16392413.

Alpert PT, Miller SK, Wallmann H, Havey R, Cross C, Chevalia T, et al. The effect of modified jazz dance on balance, cognition, and mood in older adults. J Am Acad Nurse Pract. 2009;21(2):108–15. https://doi.org/10.1111/j.1745-7599.2008.00392.x .

Jeon MY, Bark ES, Lee EG, Im JS, Jeong BS, Choe ES. The effects of a Korean traditional dance movement program in elderly women. Taehan Kanho Hakhoe Chi. 2005;35(7):126876 (in Korean). PMID: 16418553.

Eyigor S, Karapolat H, Durmaz B, Ibisoglu U, Cakir S. A randomized controlled trial of Turkish folklore dance on the physical performance, balance, depression and quality of life in older women. Arch Gerontol Geriatr. 2009;48(1):84–8. https://doi.org/10.1016/j.archger.2007.10.008 .

Noopud P, Suputtitada A, Khongprasert S, Kanungsukkasem V. Effects of Thai traditional dance on balance performance in daily life among older women. Aging Clin Exp Res. 2018;31(7):961–7. https://doi.org/10.1007/s40520-018-1040-8 .

Trombetti A, Hars M, Herrmann FR, Kressig RW, Ferrari S, Rizzoli R. Effect of musicbased multitask training on gait, balance, and fall risk in elderly people: a randomized controlled trial. Arch Intern Med. 2011;171(6):525–33. https://doi.org/10.1001/archinternmed.2010.446 .

Hyyppä MT, Mäki J, Impivaara O, Aromaa A. Individual-level measures of social capital as predictors of all-cause and cardiovascular mortality: a population-based prospective study of men and women in Finland. Eur J Epidemiol. 2007;22(9):589–97. https://doi.org/10.1007/s10654-007-9153-y .

Hyyppä MT, Mäki J, Impivaara O, Aromaa A. Leisure participation predicts survival: a population-based study in Finland. Health Promot Int. 2006;21(1):5–12. https://doi.org/10.1093/heapro/dai027 .

Lennartsson C, Silverstein M. Does engagement with life enhance survival of elderly people in Sweden? the role of social and leisure activities. J Gerontol B Psychol Sci Soc Sci. 2001;56(6):S335–42. https://doi.org/10.1093/geronb/56.6.s335 .

Sundquist K, Lindström M, Malmström M, Johansson SE, Sundquist J. Social participation and coronary heart disease: a follow-up study of 6900 women and men in Sweden. Soc Sci Med. 1982;58(3):615–22. https://doi.org/10.1016/s0277-9536(03)00229-6 .

Väänänen A, Murray M, Koskinen A, Vahtera J, Kouvonen A, Kivimäki M. Engagement in cultural activities and cause-specific mortality: prospective cohort study. Prev Med. 2009;49(2–3):142–7. https://doi.org/10.1016/j.ypmed.2009.06.026 .

Särkämö T, Soto D. Music listening after stroke: beneficial effects and potential neural mechanisms. Ann N Y Acad Sci. 2012;1252(1):266–81. https://doi.org/10.1111/j.1749-6632.2011.06405.x .

Särkämö T, Pihko E, Laitinen S, Forsblom A, Soinila S, Mikkonen M, et al. Music and speech listening enhance the recovery of early sensory processing after stroke. J Cogn Neurosci. 2010;22(12):2716–27. https://doi.org/10.1162/jocn.2009.21376 .

Särkämö T, Ripollés P, Vepsäläinen H, Autti T, Silvenno HM, Salli E, et al. Structural changes induced by daily music listening in the recovering brain after middle cerebral artery stroke: a voxel-based morphometry study. Front Hum Neurosci. 2014;8:245. https://doi.org/10.3389/fnhum.2014.00245 .

Särkämö T, Tervaniemi M, Laitinen S, Forsblom A, Soinila S, Mikkonen M, et al. Music listening enhances cognitive recovery and mood after middle cerebral artery stroke. Brain. 2008;131(3):866–76. https://doi.org/10.1093/brain/awn013 .

Fancourt D, Steptoe A, Cadar D. Cultural engagement and cognitive reserve: museum attendance and dementia incidence over a 10-year period. Br J Psychiatry. 2018;213(5):661–3. https://doi.org/10.1192/bjp.2018.129 .

Fancourt D, Steptoe A, Cadar D. Cultural engagement predicts changes in cognitive function in older adults over a 10 year period: findings from the English longitudinal study of ageing. Sci Rep. 2018;8(1):10226. https://doi.org/10.1192/bjp.2018.129 .

All Party Parliamentary group on arts, health and wellbeing. Creative health: the arts for health and wellbeing. 2017.

van Mastrigt GA, Hiligsmann M, Arts JJ, Broos PH, Kleijnen J, Evers SM, Majoie MH. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):689–704. https://doi.org/10.1080/14737167.2016.1246960 . Epub 2016 Nov 2 PMID: 27805469.

Thielen FW, Van Mastrigt G, Burgers LT, Bramer WM, Majoie H, Evers S, Kleijnen J. How to prepare a systematic review of economic evaluations for clinical practice guidelines: database selection and search strategy development (part 2/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):705–21. https://doi.org/10.1080/14737167.2016.1246962 . Epub 2016 Nov 2 PMID: 27805466.

Wijnen B, Van Mastrigt G, Redekop WK, Majoie H, De Kinderen R, Evers S. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: data extraction, risk of bias, and transferability (part 3/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):723–32. https://doi.org/10.1080/14737167.2016.1246961 . Epub 2016 Oct 21 PMID: 27762640.

Mandrik OL, Severens JLH, Bardach A, Ghabri S, Hamel C, Mathes T, Vale L, Wisløff T, Goldhaber-Fiebert JD. Critical appraisal of systematic reviews with costs and cost-effectiveness outcomes: an ISPOR good practices task force report. Value Health. 2021;24(4):463–72. https://doi.org/10.1016/j.jval.2021.01.002 . PMID: 33840423.

Kelly MP, McDaid D, Ludbrook A, Powell J: Economic appraisal of public health interventions. http://www.cawt.com/Site/11/Documents/Publications/Population%20Health/Economics%20of%20Health%20Improvement/Economic_appraisal_of_public_health_interventions.pdf

Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, Rice N, Sculpher M, Sowden A. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy. 2009;93(2–3):85–92. https://doi.org/10.1016/j.healthpol.2009.07.012 . Epub 2009 Aug 25 PMID: 19709773.

Payne K, McAllister M, Davies LM. Valuing the economic benefits of complex interventions: when maximising health is not sufficient. Health Econ. 2012. https://doi.org/10.1002/hec.2795 .

Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health. 2013;24(13):1001. https://doi.org/10.1186/1471-2458-13-1001.PMID:24153037;PMCID:PMC4015185 .

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26. https://doi.org/10.1016/j.jclinepi.2014.11.025 . Epub 2015 Feb 7 PMID: 25766056.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6. https://doi.org/10.1016/j.jclinepi.2016.01.021 . Epub 2016 Mar 19 PMID: 27005575.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. PLoS Med. 2008;6:e1000097. https://doi.org/10.1371/journal.pmed.1000097 .

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;21(339):b2700. https://doi.org/10.1136/bmj.b2700.PMID:19622552;PMCID:PMC2714672 .

Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group. PLoS Med. 2009;6(7):e1000097 Evers S, Goossens M, De Vet H, et al. Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria. Int J Technol Assess Health Care. 2005;21(02):240–245.

Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care. 2005;21(2):240–5 PMID: 15921065.

Hutchinson CL, Berndt A, Gilbert-Hunt S, George S, Ratcliffe J. Valuing the impact of health and social care programmes using social return on investment analysis: how have academics advanced the methodology? A protocol for a systematic review of peer-reviewed literature. BMJ Open. 2018;8(12):e022534. https://doi.org/10.1136/bmjopen-2018-022534 . PMID:30530579;PMCID:PMC6303612.

Higgins J, Green S. Cochrane handbook for systematic reviews of interventions version 5.1. 0. Chichester: The Cochrane Collaboration; 2013.

Coulton S, Clift S, Skingley A, Rodriguez J. Effectiveness and cost-effectiveness of community singing on mental health-related quality of life of older people: randomised controlled trial. Br J Psychiatry. 2015;207(3):250–5. https://doi.org/10.1192/bjp.bp.113.129908 . Epub 2015 Jun 18 PMID: 26089304.

Johnson JK, Stewart AL, Acree M, Nápoles AM, Flatt JD, Max WB, Gregorich SE. A community choir intervention to promote well-being among diverse older adults: results from the community of voices trial. J Gerontol B Psychol Sci Soc Sci. 2020;75(3):549–59. https://doi.org/10.1093/geronb/gby132 . PMID:30412233;PMCID:PMC7328053.

Husereau D, Drummond M, Augustovski F, de Bekker-Grob E, Briggs AH, Carswell C, Caulley L, Chaiyakunapruk N, Greenberg D, Loder E, Mauskopf J, Mullins CD, Petrou S, Pwu RF, Staniszewska S, CHEERS 2022 ISPOR Good Research Practices Task Force. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. Value Health. 2022;25(1):3–9. https://doi.org/10.1016/j.jval.2021.11.1351 . PMID: 35031096.

Bosco A, Schneider J, Broome E. The social value of the arts for care home residents in England: a social return on investment (SROI) analysis of the imagine arts programme. Maturitas. 2019;124:15–24. https://doi.org/10.1016/j.maturitas.2019.02.005 . Epub 2019 Mar 13 PMID: 31097173.

Jones C, Windle G, Edwards RT. Dementia and imagination: a social return on investment analysis framework for art activities for people living with dementia. Gerontologist. 2020;60(1):112–23. https://doi.org/10.1093/geront/gny147 . PMID: 30476114.

Social Value Lab and Impact Arts Craft Café: creative solutions to isolation and loneliness; Social return on investment. 2011. http://www.socialvaluelab.org.uk/wp-content/uploads/2013/05/CraftCafeSROI.pdf

MB associates. Make my day: the impact of Creative Caring in older people’s care homes. 2013. https://www.suffolkartlink.org.uk/wp-content/uploads/2014/10/CreativeCarersSROIReport_Nov2013.pdf

HACT. n.d. UK Social Value Bank. Retrieved December 11, 2023. from https://hact.org.uk/tools-and-services/uk-social-value-bank/ .

The Older Adults’ NHS and social care return on investment tool. Project report. Public health England. December 2019. Last accessed 27/03/2023.

British Red Cross – Valuing First Aid Education. 2018. https://socialvalueuk.org/wp-content/uploads/2018/12/Valuing-First-Aid-Education-Social-Return-on-Investment-Report-on-the-value-of-First-Aid-Education-Assured-Report.pdf . Accessed 17/02/2023

Hunter R, Dallat M, Tully M, O’Neill C, Heron L, Kee F. Social return on investment analysis of an urban greenway. Cities and Health. 2020. https://doi.org/10.1080/23748834.2020.1766783 .

NEF Consulting. Refuge: A social return on investment evaluation. 2016. https://socialvalueuk.org/wp-content/uploads/2017/04/Refuge-SROI-2016.pdf Accessed 17/02/2022

Corbacho B, Cockayne S, Fairhurst C, Hewitt CE, Hicks K, Kenan AM, Lamb SE, MacIntosh C, Menz HB, Redmond AC, Rodgers S, Scantlebury A, Watson J, Torgerson DJ, on behalf of the REFORM study. Cost-Effectiveness of a Multifaceted Podiatry Intervention for the Prevention of Falls in Older People: The REducing Falls with Orthoses and a Multifaceted Podiatry Intervention Trial Findings. Gerontology. 2018;64(5):503–12. https://doi.org/10.1159/000489171 . Epub 2018 Jun 26 PMID: 29945150.

Green C, Richards DA, Hill JJ, Gask L, Lovell K, Chew-Graham C, Bower P, Cape J, Pilling S, Araya R, Kessler D, Bland JM, Gilbody S, Lewis G, Manning C, Hughes-Morley A, Barkham M. Cost-effectiveness of collaborative care for depression in UK primary care: economic evaluation of a randomised controlled trial (CADET). PLoS ONE. 2014;9(8):e104225. https://doi.org/10.1371/journal.pone.0104225.PMID:25121991;PMCID:PMC4133193 .

National Institute for Health and Care Excellence (NICE). NICE health technology evaluations: the manual. 2022. Retrieved 27 March, 2023 from  https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation

NEF Consulting. SSE – Beatrice SROI framework – guidance document. https://www.sse.com/media/svnn5jpk/sroi-methodology-guidance-nef-consulting.pdf . Accessed 17/02/2022

Gibbons S, McNally S, Overman H. Review of Government Evaluations: A report for the NAO. London: National Audit Office; 2013.

Turner AM, Liddy ED, Bradley J, Wheatley JA. Modeling public health interventions for improved access to the gray literature. J Med Libr Assoc. 2005;93(4):487–94 PMID: 16239945; PMCID: PMC1250325.

PubMed   PubMed Central   Google Scholar  

Benzies KM, Premji S, Hayden KA, Serrett K. State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews Evid Based Nurs. 2006;3(2):55–61. https://doi.org/10.1111/j.1741-6787.2006.00051.x . PMID: 17040510.

Franks H, Hardiker NR, McGrath M, McQuarrie C. Public health interventions and behaviour change: reviewing the grey literature. Public Health. 2012;126(1):12–7. https://doi.org/10.1016/j.puhe.2011.09.023 . Epub 2011 Nov 29 PMID: 22130477.

Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synth Methods. 2014;5(3):221–34. https://doi.org/10.1002/jrsm.1106 . Epub 2013 Dec 6 PMID: 26052848.

Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. 2015;22(4):138. https://doi.org/10.1186/s13643-015-0125-0 . PMID:26494010;PMCID:PMC4619264.

Download references

Acknowledgements

We would like to thank Ms. Louise Bradley (Information Resource Officer, Institute of Public Health) for her assistance in refining search strategies and literature search.

This study was supported by the Institute of Public Health (IPH), 200 South Circular Road, Dublin 8, Ireland, D08 NH90. This study was a collaboration between two health economists (GC, CO’N) and two members of staff from the funding organisation (LM, RO’S). Input from IPH staff was fundamental in defining the scope of work and research question, refining search terms and review and editing of the manuscript. Staff from IPH were not involved in quality assurance or review of papers included in the manuscript.

Author information

Authors and affiliations.

Clinical Costing Solutions, Belfast, BT15 4EB, UK

Grainne Crealey

Institute of Public Health, 200 South Circular Road, Dublin 8, D08 NH90, Ireland

Laura McQuade & Roger O’Sullivan

Centre for Public Health, Institute of Clinical Sciences, Royal Victoria Hospital, Belfast, BT12 6BA, UK

Ciaran O’Neill

You can also search for this author in PubMed   Google Scholar

Contributions

LMcQ and ROS were involved in defining the scope of work, refining the research question, provision of subject specific (public health) context, review of search strategy, review & editing of manuscript. CON and GC were involved in refining the research question and search strategy, provision of health economics and systematic reviewing expertise, review of returned reports, original draft preparation, review, editing and submission of manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ciaran O’Neill .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

: Table S1. Search strategy for electronic databases and grey literature.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Crealey, G., McQuade, L., O’Sullivan, R. et al. Arts and creativity interventions for improving health and wellbeing in older adults: a systematic literature review of economic evaluation studies. BMC Public Health 23 , 2496 (2023). https://doi.org/10.1186/s12889-023-17369-x

Download citation

Received : 23 April 2023

Accepted : 28 November 2023

Published : 13 December 2023

DOI : https://doi.org/10.1186/s12889-023-17369-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Economic evaluation
• Older adults

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 15 April 2024

Psychosocial factors associated with overdose subsequent to Illicit Drug use: a systematic review and narrative synthesis

• Christopher J. Byrne 1 , 2 ,
• Fabio Sani 3 ,
• Donna Thain 2 ,
• Emma H. Fletcher 2 &
• Amy Malaguti 3 , 4  

Harm Reduction Journal volume  21 , Article number:  81 ( 2024 ) Cite this article

227 Accesses

Metrics details

Background and aims

Psychological and social status, and environmental context, may mediate the likelihood of experiencing overdose subsequent to illicit drug use. The aim of this systematic review was to identify and synthesise psychosocial factors associated with overdose among people who use drugs.

This review was registered on Prospero (CRD42021242495). Systematic record searches were undertaken in databases of peer-reviewed literature (Medline, Embase, PsycINFO, and Cinahl) and grey literature sources (Google Scholar) for work published up to and including 14 February 2023. Reference lists of selected full-text papers were searched for additional records. Studies were eligible if they included people who use drugs with a focus on relationships between psychosocial factors and overdose subsequent to illicit drug use. Results were tabulated and narratively synthesised.

Twenty-six studies were included in the review, with 150,625 participants: of those 3,383–4072 (3%) experienced overdose. Twenty-one (81%) studies were conducted in North America and 23 (89%) reported polydrug use. Psychosocial factors associated with risk of overdose ( n  = 103) were identified and thematically organised into ten groups. These were: income; housing instability; incarceration; traumatic experiences; overdose risk perception and past experience; healthcare experiences; perception of own drug use and injecting skills; injecting setting; conditions with physical environment; and social network traits.

Conclusions

Global rates of overdose continue to increase, and many guidelines recommend psychosocial interventions for dependent drug use. The factors identified here provide useful targets for practitioners to focus on at the individual level, but many identified will require wider policy changes to affect positive change. Future research should seek to develop and trial interventions targeting factors identified, whilst advocacy for key policy reforms to reduce harm must continue.

Introduction

People Who Use Drugs (PWUD) experience myriad harms which drive substantial morbidity and mortality [ 1 , 2 , 3 , 4 , 5 , 6 ]. In 2019, approximately 6% of the world’s population used illicit drugs at least once – including using illicitly obtained prescription medications in the context of polydrug use – and this is predicted to rise to 11% by 2030 [ 7 , 8 ]. Approximately 21% of PWUD are estimated to have experienced recent non-fatal overdose – known to precipitate future fatal overdose – equating to an estimated 3.2 million people, while approximately 42% have ever experienced overdose [ 2 ]. Internationally, approximately 500,000-600,000 fatalities are attributable to drug use annually, with close to 80% of these related to opioids and 25–30% directly induced by opioid overdose [ 7 , 9 ]. This can include illicit drugs, such as heroin, as well as use of illicitly obtained pharmaceutical opioids, such as morphine, fentanyl, and oxycodone [ 2 , 3 ]. The escalation in drug-related harms and mortality in recent decades has been attributed to a triple-wave epidemic, mediated by supply and demand side drivers, characterised by widespread opioid use; beginning with prescription opioid pills, transitioning through heroin use, and culminating in synthetic opioids – of variable quality and potency – including fentanyl variants, and nitazenes, often combined with or substituted for heroin [ 10 , 11 ].

In North America alone, nearly 600,000 people have died from an opioid-induced overdose in the last two decades with 1.2 million predicted to meet the same fate by 2029 if current trends persist. Elsewhere in the Americas substantial mortality rates have also been recorded [ 12 , 13 ]. In the UK and Western Europe, overdose and mortality rates associated with polydrug use are increasing year-on-year in some nations, with opioids involved in most fatalities [ 14 , 15 , 16 , 17 ]. In Australasia, an estimated 51% of PWUD are reported to have experienced non-fatal overdose, while this is estimated at approximately 34%, 45%, and 50%, in East & Southeast Asia, South Asia, and Central Asia, respectively [ 2 ]. Indeed, Asia, relative to North America, Europe, and Australia, has the highest crude mortality rates among PWUD, with many attributable to fatal overdose [ 3 ]. Although data from African settings is sparse, the available evidence suggests that overdose consequent to illicit drug use, fatal or non-fatal, is increasingly common worldwide, and constitutes a significant threat to public health. Beyond opioids, other central nervous system depressants – benzodiazepines, alcohol – play a critical role contributing to risk, usually in the context of polydrug use [ 17 ]. Similarly, stimulants like cocaine in different forms, and amphetamines, are commonly used together with opioids and elevate risk by artificially masking respiratory depression [ 17 , 18 ].

Responding to these alarming trends, many have endeavoured to improve surveillance and trial interventions to protect people who use drugs from harm. Some existing medicalised interventions include naloxone provision [ 19 , 20 , 21 , 22 ], opioid agonist therapy (OAT) [ 23 ], opioid antagonist therapy [ 24 ], supervised consumption sites [ 25 , 26 , 27 ], related healthcare engagement [ 28 ], detoxification [ 29 ], and integrated prevention activities [ 30 ]. Naloxone provision has gained particular salience due to its efficacy in rapidly reversing opioid-induced overdose symptoms [ 31 ]. Conventionally carried in medical and pre-hospital settings, evidence has shown high willingness among overdose bystanders to administer it [ 20 , 32 , 33 ]. Subsequently, several countries spanning Europe, Australia, and North America, have adopted legislative changes to enable provision without prescription, and protect bystanders who administer it from prosecution [ 34 , 35 , 36 ]. Beyond medicalised interventions, recovery-based approaches which prioritise empowerment, self-determination, and holistic wellbeing, have been widely adopted to underpin recovery journeys with senses of identity, belonging, purpose, and social connection [ 37 ]. Peer outreach and in-reach programmes for overdose reduction, as well as mutual help programmes, have also demonstrated efficacious impacts on recovery [ 38 , 39 , 40 , 41 ]. Such approaches acknowledge that recovery is an ongoing process that requires support, compassion, and dedication, which often extends beyond drug use alone to shifts in identity [ 42 , 43 , 44 ].

It is in the context of the varied approaches to overdose intervention, and the acknowledgement that experiences of drug effects are influenced by psychological characteristics and social processes, that we sought to evaluate the available evidence quantifying the risk of overdose among PWUD associated with psychosocial factors [ 45 , 46 ]. That is, features that pertain to the influence of social factors on an individual’s mind or behaviour, and to the interrelation of behavioural and social factors upon outcomes [ 47 ]. These may relate, for example, to social resources, like healthcare access or income source; psychological resources, such as risk perception; and psychological morbidity. Several guidelines on illicit drug use and dependence recommend psychosocial interventions, often targeting behaviour change through mindfulness, motivational interviewing, cognitive behavioural therapy (CBT) based interventions, and acceptance and commitment therapy [ 48 , 49 , 50 , 51 , 52 ]. These interventions are frequently positioned as adjuncts to overall treatment packages, as they are of uncertain benefit relative to medicalised therapies [ 53 , 54 , 55 ].

Over the years, many risk factors for overdose have been identified, for example: polydrug use; psychiatric comorbidity; unstable housing; witnessing overdose; substance use disorder; prescription of opioids; increasing pharmacy use; increasing opioid prescribers; vulnerability to socio-economic marginalisation; hepatitis C/HIV infection; male gender; rural residence; certain employment types/industries; incarceration; familial distress; disability; detoxification programme experience; the built environment; and suicidality as key factors [ 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 ]. However, despite this expansive evidence base, prior to this review, we were unable to identify any unified work that identified which psychosocial factors are associated with overdose, and therefore best to target with interventions found in prevailing guidelines.

Generating this information is critically important in the current era of increasingly limited public health resource and multiple competing public health priorities. Given their prevalence in clinical guidelines, and the uncertainty around their benefits, we sought to understand which psychosocial factors might impact on risk of overdose, to inform future intervention development and clinical practice. Accordingly, we undertook a systematic review with a narrative synthesis, which aimed to identify which, if any, psychosocial factors are associated with risk of overdose, whether fatal or non-fatal.

This review complied with the updated PRISMA statement checklist for reporting of systematic reviews and meta-analyses [ 68 ] and reporting guidelines for synthesis without meta-analysis in systematic reviews [ 69 ]. The review protocol with methods and inclusion criteria was registered in advance on PROSPERO (CRD42021242495).

Eligibility criteria

Only studies written in English were considered. The search (up to 14 February 2023) was completed with no limitations on publication dates and no geographic restrictions.

Participants

Studies were required to include PWUD as participants.

The exposure in this study was psychosocial factors which are associated with fatal and non-fatal overdose. Psychosocial was defined as pertaining to the influence of social factors on an individual’s mind or behaviour, and to the interrelation of behavioural and social factors on the outcome [ 47 ].

In studies where comparison was undertaken, PWUD who experienced overdose were compared to PWUD who did not.

The primary outcome was overdose (fatal or non-fatal) consequent to use of illicit, or illicitly obtained controlled, drugs. Intentional overdose was excluded where possible, as suicidality constitutes different behavioural characteristics to unintentional overdose. Where it was unclear whether intention was assessed or not, the study was included.

The review included observational studies (cross-sectional, cohort, case-control, and qualitative studies). Case series, case reports, and reviews, were excluded.

Information sources

The following databases were searched via OVID: Medline, Embase and PsycINFO. Cinhal was searched via EBSCOhost. Grey literature was explored by searching with Google Scholar. Reference lists of selected full-text studies were manually screened for further identification of relevant studies.

Search strategy

The search strategy was identical across databases, adjusting for database-specific search requirements. An example of the search strategy is provided in the Supplementary File. Reference lists for manuscripts eligible for full text review were searched manually for relevant titles; whilst Google Scholar was searched with ‘Psychosocial factors AND drug overdose’, and results screened manually. Screening stopped once 100 sequential results did not match search terms, given the results were ordered according to accuracy and relevance. Database searches were saved in an EBSCOhost or OVID account folder. Duplicates were removed.

Study selection and data extraction

Search results were exported from relevant databases into Microsoft Excel 365 spreadsheets for screening, with tables on study characteristics and psychosocial factors created using Microsoft Word 365. One reviewer (AM) screened titles for inclusion. Two reviewers (AM and CJB) screened all abstracts and full texts independently and a third reviewer (FS) arbitrated. Inter-rater agreement, calculated using Cohen’s kappa in Stata 17 BE, indicated high levels of agreement for both abstract (κ = 0.672 [0.565-0.780], p  < .001) and full-text (κ = 0.835 [0.697-0.974], p  < .001) screening. Data were extracted by two reviewers (AM and CJB), and separated into tables. First, data were extracted for study and sample characteristics: author, study design, location and location type, sample size, gender, age, ethnicity, population type, drugs (and other substances) reported, overdose definition, and number who experienced overdose. Second, psychosocial factors associated with overdose identified in each study along with comparators and the estimated effects/description of the association were extracted and tabulated.

Risk of bias assessment

Two reviewers independently assessed risk of bias for all included studies, discussing any discrepancies and mutually agreeing on final assessment; where required, arbitration was conducted by a third person to arrive at a final decision. The National Institutes of Health Study Quality Assessment Tools for quantitative studies, and the Critical Appraisal Skills Programme Qualitative studies checklist for qualitative studies, were used [ 70 , 71 ]. In brief, these prompt quality appraisal by considering clarity of research aims; definition of, and homogeneity of, study populations; participation rates; appropriateness of analytic approaches; clarity of outcomes measured; and ethical conduct.

Effect measures

Effect measures extracted from the studies were tabulated. Given the heterogeneous nature of the studies selected for the review, and the attendant factors examined, results were narratively synthesised; effects were not meta-analysed.

Synthesis procedure

Data were extracted manually and tabulated according to study characteristics and study findings (identified factor, author, effect size, and direction of effect). The tables were used to familiarise the reviewers with the data initially. Once data extraction was complete, the findings were reviewed, and relationships within the data and overlapping themes were annotated throughout the process of narratively synthesising individual data. The themes were discussed among three members of the research team (AM, CJB, FS) and a peer worker with lived experienced of drug use to ensure they were as accurate a reflection of the lived reality of drug use as could feasibly be achieved for a review. Themes were considered against the review question and full dataset to ensure they were focused and addressed the research question. Extracted data within each theme were then inspected to explore differences in effect direction and potential bias introduced by the different study designs included in the review. Where divergences existed, these were considered in light of study design and risk of bias. Following these steps, the manuscript was drafted, which continued the analytical, procedural, and conceptual thinking for the synthesis to be completed.

Study selection

The screening results are illustrated in Fig.  1 . During the search, 2,802 titles were screened: 2,408 were excluded, and 394 were selected for abstract review. After exclusion of duplicates, 187 remained. After further review, 61 were selected for full text assessment. Thirty-five studies were excluded with reason, whilst 26 were selected for quality appraisal and analysis.

Prisma flow chart summarising the screening process

Study characteristics

All studies focussed on overdose, fatal and non-fatal, consequent to illicit drug use as the primary outcome. This was often combined with use of legal substances (e.g. alcohol), and/or illicitly obtained controlled drugs, meaning the cohorts examined were often in the context of polydrug use. One study defined the outcome as death by unintentional overdose, according to post-mortem medical examination records [ 72 ], while one examined people hospitalised with ICD-9 codes for opioid-induced non-fatal overdose [ 73 ]. All other studies relied on self-reported non-fatal overdose disclosure, though outcome timeframes varied. In nine studies, participants self-reported ever experiencing overdose [ 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 ]. For nine other studies, the primary outcome was self-reported overdose in the last six months [ 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 ]. The primary outcome for three studies was experience of overdose in the past 12 months [ 92 , 93 , 94 ]. Riggs et al. defined the primary outcome as self-reported overdose in the last three years, while Argento et al. defined it as self-reported overdose during the study observation period (participants were sampled over nine years and follow-up varied) [ 95 , 96 ]. Lastly, for one study the primary outcome was self-reported overdose in the past five years [ 97 ]. Descriptive characteristics of each study are in Table  1 .

The total sample comprised 150,625 people. Of those, the number of participants who experienced overdose, according to the definitions reported, ranged from 3,383 to 4,072 (3%). A range is provided as one study did not report the number with sufficient clarity [ 87 ].

Most studies were conducted in North America ( n  = 21), three were in Asia, one was in Europe, and one in Australia. Participant ages ranged from 21 to 56 years. Six studies focussed on female and/or gender minority participants [ 75 , 77 , 84 , 88 , 90 , 96 ], and the remainder had a preponderance of male participants (Table  1 ). Twenty-three studies reported polydrug use and, of those, eight specified this was a mixture of prescription and illicit drugs. Three studies did not disclose the specific drugs used [ 73 , 74 , 88 ].

Methodological quality

No methodological concerns were identified which warranted removal of any of the included studies (Supplementary file 1 ).

Psychosocial factors

Factors associated with overdose ( n  = 103) were extracted from each study and structured into ten thematically similar groupings (Table  2 ; Fig.  2 ).

Thematic groups of factors found to impact on experience of overdose in reviewed studies ( n  = 103)

Note : N in each circle is the number of factors within that thematic group. Groups with smaller N are smaller circles, while groups with the same N are the same colour. Groups are randomly scattered as there is no inherent hierarchy or linearity to their impact

Eighteen studies reported odds ratios (OR) as the measure of the association between factors and exposure to overdose [ 73 , 75 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 90 , 91 , 93 , 94 , 95 ]. Two studies reported incidence rate ratios (IRR) [ 74 , 76 ], two reported relative risk (RR) [ 89 , 92 ], and two reported hazard ratios (HR) [ 72 , 96 ]. Two studies were qualitative, so no quantitative estimates were reported [ 77 , 97 ]. Given the heterogeneity of measures and study designs, summary statistics were not calculated, and meta-analysis was not performed [ 98 ]. Despite this heterogeneity, estimates of effects were considered and informed the narrative synthesis.

Eight studies explored the relationship between income source and/or unemployment and odds, or risk, of overdose [ 73 , 75 , 81 , 85 , 87 , 89 , 90 , 94 ]. Winter et al. demonstrated sustained unemployment prior to imprisonment was associated with four-to-five times higher risk of overdose following liberation. Mitra et al. also showed a four-fold increase in odds associated with unemployment. Similarly, Pabayo et al. found 40% and 70% higher odds of overdose among men and women respectively, in receipt of social welfare. Harris et al. showed recent engagement in sex work was associated with 60% higher odds of overdose, while Fairbairn et al. reported ever engaging in sex work was associated with twice the odds. El-Bassel et al. examined compounding effects of sex work and violence, with over ten years sex work experience also associated with twice the odds of overdose, and combined exposure to this with recent violence, including from intimate partners, increasing the odds four-fold. Analysis from Latkin et al. (2019) implied selling drugs in the past 30 days was associated with two-to-three times higher odds of overdose. Finally, work by Silva et al. found identifying as a lower socio-economic status growing up increased odds of overdose by 80%.

Homeless/housing instability

Eight studies explored this theme [ 73 , 81 , 87 , 88 , 89 , 90 , 91 , 95 ]. Unstable housing and lack of accommodation was consistently found to increase the odds and risk of overdose. Mitra et al. observed the largest effect, with housing insecurity increasing the odds of overdose seven-to-eight-fold. Thumath et al. found recent homelessness was associated with 60% higher odds, current homelessness increased odds by 30% according to Riggs et al., while being unhoused in the past six months was associated with 50–70% increased odds in a study by Harris et al. in an all-female sample, and 30% higher odds in Pabayo et al. in a restricted male-only analysis. The highest estimate among examinations of recent homelessness was by Silva et al, who showed past 90-day homelessness increased odds of overdose by close to three-fold, while Tomko et al. estimated a two-fold increase. Ever experiencing homelessness and ever living in a foster home were associated with five-fold and 60% increases in odds of overdose in work by Thumath et al. and Silva et al. respectively. Finally, Winter et al. found experience of unstable accommodation one month prior to incarceration increased risk of overdose three-fold among recently liberated prisoners.

Incarceration

Eight studies explored incarceration-related factors [ 72 , 75 , 77 , 79 , 81 , 86 , 89 ]. Winter et al. estimated any previous incarceration as an adult resulted in five-times higher risk of overdose, while Milloy et al. and El-Bassel et al. estimated a roughly four-fold increase in odds of overdose for participants with similar histories, and Silva et al. estimated a doubling of odds. Harris et al. and Lake et al. found incarceration in the past six months was also associated with twice the odds of overdose, with the effect enduring when adjusted for physical or emotional neglect in the work by Lake et al. El-Bassel et al. estimated a more pronounced effect among those with history of incarceration and intimate partner violence, who experienced five-times higher odds of overdose, with those who experienced non-partner violence having close to four-times higher odds. Recent liberation from prison, coupled with mental ill health, conferred a 50% higher hazard of overdose in work by Pizzicato et al. and Lamonica et al., in their qualitative study, also found that recent liberation from carceral settings increased risk of overdose in a suburban all-female cohort.

Traumatic experiences

Nine studies assessed traumatic experiences [ 75 , 77 , 84 , 86 , 88 , 89 , 90 , 91 , 96 ]. Lamonica et al. found emotional trauma, such as negative life events and consequent depressive states, increased risk of overdose. Various other traumatic experiences were examined, but multiple iterations of physical trauma pre-dominated. Thumath et al. found experience of intimate partner violence doubled the odds of overdose among marginalised women in Canada, Lake et al. found physical abuse and neglect increased odds of overdose by 40% and 30% respectively. Harris et al. found recent physical violence increased overdose odds by 80% in an all-female cohort, with that increasing to close to three-fold among sex workers and adjusted for confounders. Combined physical and sexual workplace violence was associated with twice the odds of overdose among sex workers in Goldenberg et al., while sexual abuse carried a 50% increase in odds in Lake et al., and any physical/sexual violence conferred a 90% increase in hazard in Argento et al. El-Bassel et al. examined multiple type of physical violence, imparted by intimate partners and others, and found consistently elevated odds of overdose, with severe physical violence conferring 30% increased odds in adjusted analysis.

Beyond physical trauma, Tomko et al. identified a 70% increase in odds of overdose among those who experience daily psychological pain in adjusted analysis. Separately, severe emotional abuse conferred a 50% increase in odds in adjusted analysis by Lake et al. Adverse childhood events, such as removal from family as a child, or removal from parental care, were associated with a four-fold increase in odds by Winter et al. and a doubling of odds by Thumath et al., respectively. Similarly, having a child removed from one’s care held a 60% increase in odds in adjusted analysis by Thumath et al., and child custody loss was linked with higher overdose risk in qualitative work by Lamonica et al. Finally, Thumath et al. found food insecurity drove a 90% increased in odds of overdose.

Overdose risk perception and past experience

Risk perception and past experiences with overdose were evaluated in six studies [ 74 , 77 , 80 , 81 , 92 , 95 ]. There were divergent effects between perceived severity of prior overdose experience and participants’ perception of their own susceptibility to overdosing in work by Bonar et al., where higher perceived severity was linked to 40% decreased incidence and higher perceived susceptibility was linked to 50% higher incidence. Vicarious experience, i.e. witnessing an overdose, was associated with two-fold higher odds of subsequent overdose experience in Riggs et al., while ever witnessing a family member overdose conferred 60% higher odds in adjusted analysis by Silva et al. Schiavon et al. estimated that the higher the number of times a participant witnessed another person overdose, odds of subsequent overdose experience increased by 40%, with odds increasing four-fold where the other person was identified as a friend. Prior experience of overdose was also linked to 70% higher risk of subsequent overdose in Grau et al. whereas, in qualitative work by Lamonica et al., being a ‘novice’ to drug use, which may include erroneous polydrug use, was linked to higher risk.

Healthcare experiences

Most healthcare experiences, across eight studies, focused on medicalised addictions treatment [ 76 , 80 , 81 , 84 , 86 , 89 , 91 , 94 ]. Ever experiencing addictions treatment was associated with a 60% increased incidence of overdose in Havens et al., while Latkin et al. estimated a 50% increase in odds. However, when examined by Silva et al., the increase in odds was two-fold, and ever receiving opioid substitution therapy conferred a three-fold increase in relative risk in Winter et al. Schiavon et al. estimated that with increasing number of treatment episodes, the odds of experiencing overdose increased by 60% in adjusted analysis. Conversely, Lake et al. found that being denied access to addictions treatment was associated with close to three-fold odds of overdose. Other studies examined healthcare need, with Goldenberg et al. identifying unmet healthcare need was associated with 70% higher odds of overdose, and Tomko et al. linking unmet mental health care need to a 40% increase in adjusted analysis.

Perception of own drug use and injecting skills

Three studies examined participants’ perceptions of their own drug use, two of which were qualitative [ 77 , 95 , 97 ]. In the quantitative work, Riggs et al. estimated that participants who perceived they had a drug ‘problem’ had five-fold higher odds of subsequent overdose in adjusted analysis. Lamonica et al. found participants who disclosed a lack of knowledge about drug use, a lack of control over the quality of the drugs they were using, or lack of knowledge of their tolerance of those drugs, had higher risk of experiencing overdose. Chang et al. termed similar types of knowledge as ‘opioid expertise’ – this also included perceived self-control over opioid use and one’s bodily response – and identified that participants who felt they possessed a high degree of opioid expertise had increased risk of overdose. Related to the sense of expertise and experience, low injecting skill was examined in two studies [ 86 , 87 ]. Both linked requiring assistance with injecting with increased odds of overdose. Lake et al. found requiring help to inject increased odds by 90%, with adjusted models for physical and sexual abuse yielding 70% higher odds, and adjusted models for physical and emotional neglect yielding 70% and 50% higher odds respectively. Likewise, Pabayo et al., found that, among men, requiring help injecting increased odds of overdose by 74%.

Injecting setting

Injecting setting was assessed in four studies [ 83 , 84 , 85 , 86 ]. Injecting in public spaces in the past six months was consistently linked with higher odds of overdose. Lake et al. found a close to three-fold increase in odds of overdose in a Canadian cohort, which attenuated to 90% when adjusted for experience of emotional abuse, and to 70% when adjusted for experience of emotional neglect. Fairbairn et al. estimated a more pronounced effect, with a close to five-fold increase in odds associated with injecting in public settings. Both cohorts were sampled in Vancouver, Canada. Conversely, these studies found diverging effects for injecting alone in the last six months. Lake et al. estimated an 80% increase in odds, while Fairbairn et al. found the odds of overdose decreased by 60%. Fear of police intervention while injecting in public spaces was associated with a two-fold increase in odds by Bazazi et al., including in adjusted analysis. While ‘rushed’ outdoor drug use in the last six months conferred a 30% increase in odds in work by Goldenberg et al.

Conditions within physical environment

In related analyses, specific conditions within the wider physical environment were found to mediate overdose likelihood in six studies that examined this [ 83 , 84 , 90 , 93 , 94 , 96 ]. Proximity to harm reduction provision was examined in three studies, with somewhat diverging outcomes. First, Bazazi et al., found that among those who reported that a needle and syringe provision (NSP) site was the main source of their injecting equipment acquisition, this was linked to a 60% reduction in odds of overdose. However, Latkin et al. (2019) found that among those who replaced syringes through such a service, there was a three-to-four-fold increase in odds. Vallance et al. also reported a similar finding, where participants that resided in areas of high harm reduction coverage had twice the odds of overdose in adjusted analysis. In further conflicting results, Goldenberg et al. identified police-related barriers to harm reduction access doubled odds of overdose in adjusted analysis.

Similarly, Argento et al., found the same parameter conferred a close to three-fold increase in hazard of overdose in adjusted analysis, while Harris et al. observed that, among women, being stopped, searched, detained, or assaulted by police conferred a 50% increase in odds. This increased to a doubling of odds when stratified for sex workers only. Meanwhile, living in an area characterised by criminalisation, marginalisation, and prevalence of drug use, was associated with 40% higher odds of overdose in the same paper. Somewhat similar to wider drug use prevalence in the area, residing in a neighbourhood with an increasing number of known settings in which to use drugs was associated with 30% increase in odds overdose in adjusted analysis by Latkin et al.

Social network traits

Finally, density of social networks and supports were examined in six studies [ 76 , 77 , 78 , 82 , 84 , 87 ]. Pabayo et al. found three or more social supports was associated with a 50% reduction in odds of overdose among women in adjusted analysis. While, in their study, Tobin et al. found density of social network at baseline, and increases in density reported during follow-up, were associated with 90% and 80% reductions in odds in adjusted analyses. However, among those who reported recent injection drug use, Tobin et al. found increasing density in social network conferred a 20% increase in overdose odds in adjusted analysis, while Latkin et al. (2004) identified that reporting increasing numbers of people who inject heroin in one’s social network was associated with 20% higher odds of past overdose, and 30% higher odds of recent overdose. Conversely, in the same study, increasing numbers of contacts who snort heroin, rather than inject, was associated with a 20% reduction in odds of overdose.

Conflicting somewhat with these findings, Tobin et al. also found that, among those who reported recent injection drug use, an increasing number of people who inject drugs in participants’ social networks was associated with 80% reduced odds of overdose in adjusted analysis. Similarly, Havens et al. found increasing numbers of support members in one’s social network was linked to a 20% increased in incidence of overdose in adjusted analysis. Latkin et al. found increasing levels of conflict within a participant’s social network conferred a 30% increase in odds, whilst other studies examined intimate partnerships. In their qualitative study, Lamonica et al., found being friends, or in an intimate partnership, with someone who uses drugs increased participants’ risk of overdose. Similarly, Goldenberg et al., reported that providing drugs for an intimate partner (who was male) was associated with a 40% increase in odds of overdose.

This review is the first to our knowledge which specifically evaluated psychosocial factors associated with unintentional overdose consequent to illicit drug use, with many reviewed studies documenting polydrug use. Prior research suggests the majority of serious overdoses are unintentional, implying our findings are pertinent to the experiences of many people who use drugs [ 99 ]. While existing review evidence has elucidated many important factors, as noted in the Introduction, none highlighted the important connections between sex work, violence, or social networks, and overdose risk that we identified [ 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 ]. Twenty-six studies from seven countries were reviewed, only two of which were qualitative, with the vast majority conducted in North America. Most participants were male, though several studies examined female-only cohorts. The overall proportion estimated to have experienced overdose was 3%, contrasting sharply with global estimates of 21% (15-26%) of PWUD reported to have recently experienced overdose [ 2 ]. Sample sizes varied widely, with two registry studies reporting disproportionately large samples relative to other reviewed studies, and low relative overdose prevalence [ 72 , 73 ]. Excluding these from the estimate would bring the overall prevalence closer to 16%. Thus we believe most studies reviewed are representative of the at-risk population.

Identified factors were structured into ten overarching groups, with some thematically similar correlates yielding conflicting results. Factors varied from the individual (e.g. risk perception) to the structural (e.g. housing) in a manner which illustrates the synergies between biological factors, psychological traits, and social processes, both at micro and macro levels, which influence an individual’s likelihood of experiencing overdose [ 45 , 46 , 100 , 101 ].

For example, income played an important role in mediating risk, with experience of sex work, unemployment, drug selling, social welfare receipt, and lower socio-economic status, all associated with increased reports of overdose. The relationship between income and health may be explained by subjective psychosocial experiences mediated by work environments and exposure to unemployment [ 102 , 103 ]. However, the correlates reported are characterised by socioeconomic marginalisation, which speaks to the economic and political frameworks which worsen health outcomes for people who use drugs within the model of interdiction which predominates globally. For instance, at the micro level, while the individual acts involved in drug use may have shaped sex worker/client interactions and were important in moderating overdose risk, the ultimate harm induced by that behaviour was enabled by the fact sex workers were reticent to report overdose due to criminalisation and structural stigmatisation, both of their drug use but also their method of income generation [ 104 ]. The risk environment for sex workers was elucidated further by El-Bassel et al. who demonstrated the compounding impact of violence and sex work on overdose risk [ 75 ]. The context may then be at least partially characterised by risky drug use and frequent violence at the micro level – a common experience among sex workers operating in a social environment of gendered norms and unequal power dynamics – which is enabled by public policy at the macro level which marginalises sex workers and leaves them vulnerable to harms related to drug use [ 105 , 106 ]. These findings speak to the urgent need to cease using criminal law to enforce morals upon income generation and strengthen the previously elucidated case for this as the best strategy to reduce harms experienced by sex workers [ 107 ].

At the individual level, there is little evidence to support the use of psychosocial interventions to improve health and well-being among sex workers, perhaps due to the structural factors at play [ 108 ]. Separate to this, unemployment was generally associated with higher risk than sex work and other income factors such as social welfare receipt, participation in the illicit drug trade, and lower socio-economic status, and it is important to note that the relationship between these factors and overdose may be mediated by social capital and isolation [ 59 , 62 , 109 ]. These, in turn, drive worse psychosocial outcomes, which are enabled by prevailing policies of state-imposed methods of control (social welfare) of non-conforming behaviour (non-participation in ‘normative’ modes of economic activity), and intentional criminalisation of drug use which erodes drug supply quality and increases overdose risk [ 10 ].

In a similar vein, housing instability was consistently linked with increased odds of overdose, similar to prior research which observed this [ 110 ]. Among vulnerable adults experiencing homelessness, psychological and social issues at the micro level, such as self-esteem, social support, coping mechanisms, and emotional distress, have been associated with increased substance use [ 111 ]. Further, people facing homelessness experience frequent stigmatisation which negatively impacts mental health and well-being, and wider social interactions. Whilst drug use in this context of unstable housing will be influenced by immediate social norms of the situation, there is an overarching synergy between housing and drug use which has driven opioid-overdose to be a leading cause of death among people experiencing it [ 112 , 113 , 114 ]. Research suggests this synergy confers 38% higher odds of overdose [ 115 ]. These issues are likely manifestations of both immediate social interactions in the context of insecure housing, and macro housing policy which inhibits the social environments which vulnerable individuals are enabled to access. Recent work has reported positive effects for psychosocial interventions in reducing psychological morbidity among people experiencing homelessness [ 116 ], but these will not negate the risks which require wider policy reform around housing programmes [ 112 ]. For example, many housing programmes restrict PWUD accessing their services as a matter of policy, despite housing being linked with harm reduction impacts and improved psychosocial measures which may facilitate recovery-based approaches [ 117 , 118 , 119 ]. The results illustrate a need for supportive and stable housing – a fundamental requirement to establish a sense of safety and stability – to be viewed as a critical intervention which policy makers and public health practitioners should seek to deliver to moderate prevalence of overdose.

The likelihood of becoming homeless may be mediated by history of incarceration [ 120 ]. Incarceration was consistently linked to higher risk of overdose in reviewed studies, and other work not reviewed here [ 115 ]. The circumstances surrounding the first two weeks post liberation have been demonstrated to induce an up to eight-fold increase in risk of fatal overdose relative to subsequent weeks and, furthermore, all-cause mortality is up to 12.7 times higher than that of the general population among those recently liberated, with most attributable to fatal overdose [ 121 , 122 ]. While mental health difficulties, victimisation, and feeling unsafe during incarceration, have been linked to poorer psychosocial adjustment upon liberation (which psychosocial interventions may help address), these findings emphasise the inadequacy of efforts by health and welfare services, and carceral establishments, to assist people in the vulnerable period following liberation with transitional social and medical supports [ 123 , 124 , 125 ].

Research has shown relapse to drug use in this window occurs in the context of poor social support, situational stressors (violence, poverty, isolation, availability), and decreased tolerance [ 125 ]. Conversely, exposure to factors which address these, such as housing, social supports (including avoiding old social networks), mutual help programmes, and spiritual services, have been cited as protective [ 125 ]. Overdose risk caused by liberation to environments that trigger drug use may be somewhat ameliorated by provision of take-home naloxone, but research has shown people in prison may not be receptive to training and carriage of naloxone, and motivation to carry it is complicated by desires to remain abstinent [ 126 , 127 ]. Beyond individual factors, useful conceptual frameworks have been posited to frame the multilevel nature of the determinants involved in overdose risk upon liberation, which suggest researchers shift the lens through which this issue framed from the individual to the socio-structural [ 128 , 129 ]. Our findings highlight the harms conferred by structural control mechanisms which reinforce criminalisation of drug use and compound inequalities experienced by people who use drugs in health outcomes.

There were additive effects for incarceration with physical neglect and recent experience of violence. Intimate partner violence (IPV) was among the traumatic experiences linked to higher risk, alongside multiple types of intimate partner and non-partner violence, including sexual abuse and neglect. It was unclear from the results whether IPV, abuse, and neglect experienced were reciprocal/bidirectional, however all but one study examining these experiences were in female cohorts. So the relationship between overdose risk and these factors may be understood as the confluence of the drug effects, the norms and boundaries concerning gender-based violence within the immediate social context, and wider cultural and systemic factors which perpetuate gender-based violence. At the individual level, psychosocial interventions, with advocacy and psychological components, can reduce depressive symptomology and post-traumatic stress among IPV survivors, which may ameliorate overdose risk [ 130 ]. However, they do not mitigate against re-experience and therefore policy changes which address the physical, social, and economic circumstances that manifest in the macro environment, and perpetuate gender-based violence, are critical to reducing risk, alongside individual interventions. One relevant example is the ongoing pilot of discreet payments to women availing of aid services in Scotland to abscond from circumstances of abuse [ 131 ].

In studies which examined experiences of healthcare, unmet needs and denied care were important in elevating overdose risk. PWUD are less likely to be able to avail of preventive healthcare to screen and manage conditions due to frequent experiences of stigma, distrust, and frustration in health environments; with those same people often blamed for the stigma they experience [ 132 , 133 , 134 , 135 , 136 ]. Unmet health needs have been linked to increased depression, with 29% (21-37%) of PWUD meeting the threshold for clinical depression diagnosis, and consequent self-harm and post-traumatic stress common [ 1 , 137 ]. There were also associations between experience of addictions treatment and overdose which were unexpected, given OAT is known to be protective against drug-related mortality [ 138 ]. This association may be explained by severity of dependence (and related suboptimal dosage); changes in tolerance whilst engaging with treatment; those who engaged with treatment having a higher likelihood of follow-up for overdose; those with past overdose experience being more likely to be referred for treatment; OAT discontinuity and re-entry; and transferring between OAT providers [ 139 , 140 ]. It should further be acknowledged that, though it is an established harm reduction tool, OAT can (and has) been interpreted as a mechanism of control through which moral discipline is inculcated in people who participate in drug use [ 141 , 142 ]. Through this lens, OAT engagement is necessitated only by ongoing interdiction and the intersecting inequalities and harms this produces. Safer supply and decriminalisation of drug use present reasonable (structural) approaches relative to individual interventions such as OAT, which may aid in mitigating overdose risk at the population level, whilst simultaneously mitigating against negative effects of interventions premised on ill-conceived moral frameworks [ 143 , 144 ].

Some environmental factors linked to overdose included experience of police-related interventions such as blocking access to harm reduction, stopping, arresting, and detaining people. All of which are more likely to occur in areas characterised by socio-economic marginalisation and prevalent drug use. Policing of drug use is characterised by violence which drives increased psychological distress among PWUD [ 145 , 146 ]. Similarly, rushed and public injecting, often accompanied by punitive policing, drove increased risk, as demonstrated in previous work [ 115 ]. Social-ecological frameworks have been proposed to articulate a means of addressing such factors, as it is unlikely individual-level interventions will modify these risks [ 147 , 148 ]. It is likely public health approaches which account for the societal, communal, and interpersonal factors, which drive these risks will be required to mitigate against the high likelihood of overdose they confer. These approaches require policy change – particularly regarding criminalisation of drug use and associated policing – while educational campaigns and clear service pathways to harm reduction are also critical.

At a more individual level, perception and social issues noted highlight the interconnectedness between drug use, individual psychology, and social processes. Social support systems impact psychological and physical wellbeing, and the interplay of social networks with environmental and individual factors can differentially impact upon psychological stressors [ 149 ]. This was apparent in the results, with contrasting effects observed. Higher density of social networks of varying degrees were protective against overdose in one study [ 82 ], while others which examined social networks characterised by conflict, ongoing injecting, and exposure to recent overdose among peers, signalled harmful impacts. Individually, peer social support may reduce psychological distress which in turn reduces overdose risk [ 150 , 151 ], and interventions which target social connectedness may be beneficial in this context [ 152 ]. More broadly, these results may be viewed through the Social Identity Model of Recovery, which proposes that recovery from drug use relies on a shift in identity wherein individuals reshape their social network to one wherein drug use is uncommon [ 43 , 44 , 153 ]. Reviewed studies which signalled harmful impacts studied social networks characterised by ongoing risks, whilst one might infer that those which examined network density where actually examining surrogates of networks wherein use of drugs was less prevalent. Where recovery from drug use is sought, peer support can be critical. One form which this takes is in mutual aid groups, which have been shown to catalyse changes in social networks, increase recovery capital, and enhance commitment to sobriety, through community reinforcement [ 154 , 155 ]. Additionally, alternative unstructured peer support strategies, such as recovery cafes, can also be enabling, whilst strategies like ‘spotting’ can help to enhance overdose response in the context of ongoing drug use [ 156 , 157 ].

Furthering the consideration of social context, witnessing overdose is deleterious to psychological wellbeing, causes post-traumatic stress, and can drive people to engage in risky drug use behaviours to manage feelings of bereavement and trauma [ 158 , 159 ]. Psychological distress has itself been independently associated with close to ten-times higher odds of overdose in young people [ 110 ]. Therefore trauma-informed psychosocial interventions for post-traumatic stress – which have been demonstrated as effective, particularly CBT-based therapies – may be important to integrate into existing harm reduction services [ 160 , 161 ]. Particularly when prefaced by safety and stabilisation work within a phased interventional model, to establish safety and create coping mechanisms before trauma reprocessing occurs [ 162 ]. However, an increase in psychological wellbeing may not mitigate against social factors such as requiring injecting assistance – shown previously to increase risk by approximately 58% – and risk conferred by one’s perception of their drug use [ 115 ]. Factors which implied low injecting skill were associated with increased risk – psychosocial interventions may improve injecting skills among PWUD [ 163 ] – alongside identifying as an expert in drug use. This contrasts with research among people who use new psychoactive substances, where expertise has been linked to higher risk perception and greater control in exposure to risk [ 164 ]. Individual-level interventions which assess and affect changes to psychological mechanisms that relate risk perception to overdose risk may therefore also be appropriate to explore.

Limitations

There are several limitations to this review. First, we did not undertake a meta-analysis due to the heterogeneity in effect estimates and study designs, instead opting for narrative review of the effects. Although appropriate for the heterogeneous study types and factors examined, this provides limited information for decision making relative to meta-analysis and risks emphasising the results of some studies erroneously and potentially misrepresenting the evidence [ 165 ]. Second, reviewed studies were concentrated in high-income countries, mostly in North America, significantly limiting the generalisability of the work. No work from African settings was identified, which is a critical limitation given the ongoing epidemic of extra-medical use of opioids (tramadol) and expansion of cocaine markets in recent years into African and Near and Middle Eastern settings, beyond conventional markets in Europe and North America [ 166 ]. This likely means PWUD in these settings will be disproportionately impacted by associated harms in coming years, with little representation in research. Third, our search strategy included terms for ‘psychosocial’, ‘psychological’, ‘social’, or ‘behavioural’, which was intended to be comprehensive. Nonetheless, some relevant research may have been omitted unintentionally due to the search design and/or interpretation of the results by the reviewers, given the broad scope and interpretability of the term ‘psychosocial’; we mitigated against this by referencing a recognised definition when interpreting and extracting results, and citing works thought to be relevant in the Discussion [ 47 ]. Finally, only two studies reviewed were qualitative in nature. This suggests the findings may omit relevant work documenting subjective experience, not captured in the quantitative studies. We suggest two reasons for this: our search strategy did not include terms for methodology like ‘quantitative’ or ‘qualitative’ which may have resulted in more results returned for relevant qualitative work; and much qualitative work proximal to overdose which we reviewed for inclusion concurrently examined factors which made them ineligible on the basis of our criteria (e.g. suicidal ideation; relationships).

Globally, rates of fatal and non-fatal overdose continue to increase, alongside many cognate harms, consequent to illicit drug use [ 1 , 2 , 167 ]. This review identified many psychosocial correlates of overdose which spoke to the interdependencies between drug use, psychological traits, and social processes, alongside the overlapping structural, societal, and environmental inequities which govern harms related to drug use, and therefore frame the risks related to overdose. Existing harm reduction interventions are insufficient to resolve the crisis of overdose and avoidable fatalities consequent to the opioid epidemic [ 168 ]. To date, many national drug policies are premised more on ideology than evidence, and our findings support the view that punitive approaches are not just ineffective in reducing prevalence of overdose, but actually contribute to the risk environment which increases it [ 144 ]. Where we believe this review adds value for the harm reduction movement is in elucidating several themes not previously identified in existing review evidence, which may be helpful in policy work concerning drug use, and clarifying the factors which practitioners may seek to engage at the individual level when exploring psychosocial interventions in harm reduction services, to facilitate therapeutic response. For example: mechanisms underlying risk perception, social connectedness, coping mechanisms, and screening and management of IPV [ 50 , 51 , 52 , 55 ].

Data availability

As this was a systematic review, no original data was generated in completing the work. All cited works are listed in the References for consultation.

Colledge S, Larney S, Peacock A, Leung J, Hickman M, Grebely J, et al. Depression, post-traumatic stress disorder, suicidality and self-harm among people who inject drugs: a systematic review and meta-analysis. Drug Alcohol Depend. 2020;207:107793.

Article   PubMed   Google Scholar  

Colledge S, Peacock A, Leung J, Larney S, Grebely J, Hickman M, et al. The prevalence of non-fatal overdose among people who inject drugs: a multi-stage systematic review and meta-analysis. Int J Drug Policy. 2019;73:172–84.

Degenhardt L, Bucello C, Mathers B, Briegleb C, Ali H, Hickman M, et al. Mortality among regular or dependent users of heroin and other opioids: a systematic review and meta-analysis of cohort studies. Addiction. 2011;106(1):32–51.

Degenhardt L, Charlson F, Stanaway J, Larney S, Alexander LT, Hickman M, et al. Estimating the burden of disease attributable to injecting drug use as a risk factor for HIV, Hepatitis C, and hepatitis B: findings from the global burden of Disease Study 2013. Lancet Infect Dis. 2016;16(12):1385–98.

Larney S, Peacock A, Mathers BM, Hickman M, Degenhardt L. A systematic review of injecting-related injury and disease among people who inject drugs. Drug Alcohol Depend. 2017;171:39–49.

Larney S, Tran LT, Leung J, Santo T Jr., Santomauro D, Hickman M, et al. All-cause and cause-Specific Mortality among people using Extramedical opioids: a systematic review and Meta-analysis. JAMA Psychiatry. 2020;77(5):493–502.

United Nations Office on Drugs and Crime. World Drug Report 2021. United Nations Office on Drugs and Crime; 2021. 24/06/2021.

Conway KP, Vullo GC, Nichter B, Wang J, Compton WM, Iannotti RJ, et al. Prevalence and patterns of polysubstance use in a nationally representative sample of 10th graders in the United States. J Adolesc Health. 2013;52(6):716–23.

Article   PubMed   PubMed Central   Google Scholar  

World Health Organization. Opioid overdose 2023 [updated 29/08/2023. www.who.int/news-room/fact-sheets/detail/opioid-overdose .

Ciccarone D. The triple wave epidemic: supply and demand drivers of the US opioid overdose crisis. Int J Drug Policy. 2019;71:183–8.

Alcohol and Drug Foundation. Nitazenes 2023 [updated 22/11/2023. https://adf.org.au/drug-facts/nitazenes/ .

United Nations Office on Drugs and Crime. Drug-related deaths and mortality rates in Americas 2018 [updated 21/06/2018. https://dataunodc.un.org/drugs/mortality/americas .

The Lancet Public Health. Opioid overdose crisis: time for a radical rethink. Lancet Public Health. 2022;7(3):e195.

Article   CAS   PubMed   Google Scholar  

ONS. Deaths related to drug poisoning in England and Wales. 2020 registrations: Office of National Statistics; 2021 [ https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsrelatedtodrugpoisoninginenglandandwales/2020/pdf .

EMCDDA. European Drug Report. Trends and developments: European Monitoring Centre for Drugs and Drug Addiction; 2021 [updated 09/06/2021. https://www.emcdda.europa.eu/system/files/publications/13838/TDAT21001ENN.pdf .

McAuley A, Matheson C, Robertson JR. From the clinic to the street: the changing role of benzodiazepines in the Scottish overdose epidemic. Int J Drug Policy. 2022;100:103512.

European Monitoring Centre for Drugs and Drug Addiction (EMCDDA). Polydrug use: health and social responses. 2021.

Degenhardt L, Grebely J, Stone J, Hickman M, Vickerman P, Marshall BDL, et al. Global patterns of opioid use and dependence: harms to populations, interventions, and future action. Lancet. 2019;394(10208):1560–79.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bird SM, McAuley A, Perry S, Hunter C. Effectiveness of Scotland’s National Naloxone Programme for reducing opioid-related deaths: a before (2006-10) versus after (2011-13) comparison. Addiction. 2016;111(5):883–91.

Clark AK, Wilder CM, Winstanley EL. A Systematic Review of Community Opioid Overdose Prevention and Naloxone Distribution Programs. J Addict Med. 2014;8(3):153–63.

McDonald R, Strang J. Are take-home naloxone programmes effective? Systematic review utilizing application of the Bradford Hill criteria. Addiction. 2016;111(7):1177–87.

Bird SM, McAuley A. Scotland’s National Naloxone Programme. Lancet. 2019;393(10169):316–8.

NICE. Methadone and buprenorphine for the management of opioid dependence (TA114). London: National Institute for Health and Care Excellence; 2007. 24/01/2007.

Google Scholar  

NICE. Naltrexone for the management of opioid dependence (TA115). London: National Insitute for Health and Care Excellence; 2007. 24/01/2007.

EMCDDA. Drug consumption rooms: an overview of provision and evidence. Lisbon; 2016 07/06/2018.

Kerr T, Mitra S, Kennedy MC, McNeil R. Supervised injection facilities in Canada: past, present, and future. Harm Reduct J. 2017;14.

Shorter GW, Harris M, McAuley A, Trayner KM, Stevens A. The United Kingdom?s first unsanctioned overdose prevention site; a proof-of-concept evaluation. Int J Drug Policy. 2022;104.

Caven M, Robinson EM, Eriksen AJ, Fletcher EH, Dillon JF. Hepatitis C diagnosis and treatment, impact on engagement and behaviour of people who inject drugs, a service evaluation, the hooked C project. J Viral Hepat. 2020;27(6):576–84.

NICE. Drug misuse in over 16s: opioid detoxification (CG52). London: National Institute for Health and Care Excellence; 2007. 25/07/2007.

NICE. Drug misuse prevention: targeted interventions (NG64). London: National Institute for Health and Care Excellence; 2017. 22/02/2017.

Wermeling DP. Review of naloxone safety for opioid overdose: practical considerations for new technology and expanded public access. Ther Adv Drug Saf. 2015;6(1):20–31.

McDonald R, Breidahl S, Abel-Ollo K, Akhtar S, Clausen T, Day E, et al. Take-Home Naloxone kits: attitudes and Likelihood-Of-Use outcomes from a European survey of potential overdose witnesses. Eur Addict Res. 2022;28(3):220–5.

World Health Organization and United Nations Office on Drugs and Crime. UNODC-WHO stop-overdose-safely (S-O-S) project implementation in Kazakhstan, Kyrgyzstan, Tajikistan and Ukraine: summary report. Geneva: World Health Organization; 2021.

Government of Canada. About the Good Samaritan Drug Overdose Act 2021 [updated 23/11/2021. https://www.canada.ca/en/health-canada/services/opioids/about-good-samaritan-drug-overdose-act.html .

The Network for Public Health Law. Legal Interventions to Reduce Overdose Mortality: Overdose Good Samaritan Laws. 2023 17/07/2023.

EMCDDA, Take-. home Naloxone 2019 [updated 30/08/2019. https://www.emcdda.europa.eu/publications/topic-overviews/take-home-naloxone_en .

National Association for Alcoholism and Drug Abuse Counselors (NAADAC). Using recovery-oriented principles in addiction counselling practice. 2014 23/07/2014.

Waye KM, Goyer J, Dettor D, Mahoney L, Samuels EA, Yedinak JL, et al. Implementing peer recovery services for overdose prevention in Rhode Island: an examination of two outreach-based approaches. Addict Behav. 2019;89:85–91.

Scott CK, Dennis ML, Grella CE, Nicholson L, Sumpter J, Kurz R, et al. Findings from the recovery initiation and management after overdose (RIMO) pilot study experiment. J Subst Abuse Treat. 2020;108:65–74.

Welch AE, Jeffers A, Allen B, Paone D, Kunins HV. Relay: a peer-delivered Emergency Department-based response to nonfatal opioid overdose. Am J Public Health. 2019;109(10):1392–5.

Weiss RD, Griffin ML, Marcovitz DE, Hilton BT, Fitzmaurice GM, McHugh RK et al. Correlates of opioid abstinence in a 42-Month Posttreatment Naturalistic Follow-Up study of prescription opioid dependence. J Clin Psychiatry. 2019;80(2).

Laudet AB. What does recovery mean to you? Lessons from the recovery experience for research and practice. J Subst Abuse Treat. 2007;33(3):243–56.

Best D, Beckwith M, Haslam C, Alexander Haslam S, Jetten J, Mawson E, et al. Overcoming alcohol and other drug addiction as a process of social identity transition: the social identity model of recovery (SIMOR). Addict Res Theory. 2016;24(2):111–23.

Article   Google Scholar  

Dingle GA, Cruwys T, Frings D. Social identities as pathways into and out of addiction. Front Psychol. 2015;6:1795.

Moore D. Beyond Zinberg’s ‘social setting’: a processural view of illicit drug use. Drug Alcohol Rev. 1993;12(4):413–21.

Zinberg NE, Drug. Set and setting: basis for Controlled Intoxicant Use. London: Yale University Press; 1986.

Oxford English Dictionary. Psychosocial Oxford: Oxford University Press. 2022 [updated 03/2022. www.oed.com/view/Entry/153937 .

NICE. Drug misuse in over 16s: psychosocial interventions (CG51). London: National Institute for Health and Care Excellence; 2007.

Donroe JH, Tetrault JM. Narrowing the treatment gap in managing opioid use disorder. Can Med Assoc J. 2018;190(9):E236–7.

NSW Department of Health. Drug and Alcohol Psychosocial interventions Professional Practice guidelines. Gladesville NSW 2111; 2008 06/05/2008.

Fisher A, Nepal S, Harvey L, Peach N, Marel C, Kay-Lambkin F, et al. Drug and alcohol psychosocial interventions. The Sax Institute; 2020.

American Psychiatric Association. Practice guideline for the treatment of patients with substance use disorders. 2 ed. American Psychiatric Association; 2006.

Amato L, Minozzi S, Davoli M, Vecchi S. Psychosocial combined with agonist maintenance treatments versus agonist maintenance treatments alone for treatment of opioid dependence. Cochrane Db Syst Rev. 2011(10).

Goldberg SB, Pace B, Griskaitis M, Willutzki R, Skoetz N, Thoenes S et al. Mindfulness-based interventions for substance use disorders. Cochrane Db Syst Rev. 2021(10).

Darker CD, Sweeney BP, Barry JM, Farrell MF, Donnelly-Swift E. Psychosocial interventions for benzodiazepine harmful use, abuse or dependence. Cochrane Db Syst Rev. 2015(5).

Lyons RM, Yule AM, Schiff D, Bagley SM, Wilens TE. Risk factors for drug overdose in Young people: a systematic review of the literature. J Child Adolesc Psychopharmacol. 2019;29(7):487–97.

van Draanen J, Tsang C, Mitra S, Phuong V, Murakami A, Karamouzian M, et al. Mental Disorder and opioid overdose: a systematic review. Soc Psychiatry Psychiatr Epidemiol. 2022;57(4):647–71.

Wang L, Hong PJ, Jiang W, Rehman Y, Hong BY, Couban RJ, et al. Predictors of fatal and nonfatal overdose after prescription of opioids for chronic pain: a systematic review and meta-analysis of observational studies. CMAJ. 2023;195(41):E1399–411.

van Draanen J, Tsang C, Mitra S, Karamouzian M, Richardson L. Socioeconomic marginalization and opioid-related overdose: a systematic review. Drug Alcohol Depend. 2020;214:108127.

Green TC, McGowan SK, Yokell MA, Pouget ER, Rich JD. HIV infection and risk of overdose: a systematic review and meta-analysis. AIDS. 2012;26(4):403–17.

King NB, Fraser V, Boikos C, Richardson R, Harper S. Determinants of increased opioid-related mortality in the United States and Canada, 1990–2013: a systematic review. Am J Public Health. 2014;104(8):e32–42.

Cano M, Oh S, Osborn P, Olowolaju SA, Sanchez A, Kim Y, et al. County-level predictors of US drug overdose mortality: a systematic review. Drug Alcohol Depend. 2023;242:109714.

Mital S, Wolff J, Carroll JJ. The relationship between incarceration history and overdose in North America: a scoping review of the evidence. Drug Alcohol Depend. 2020;213:108088.

Clingan SE, Woodruff SI, Gaines TL, Davidson PJ. Detoxification, 12-step meeting attendance, and non-fatal opioid overdoses among a suburban/exurban population with opioid use disorder. J Addict Dis. 2023;41(4):266–73.

Morgan JR, Wang J, Barocas JA, Jaeger JL, Durham NN, Babakhanlou-Chase H, et al. Opioid overdose and inpatient care for substance use disorder care in Massachusetts. J Subst Abuse Treat. 2020;112:42–8.

Bohnert AS, Roeder K, Ilgen MA. Unintentional overdose and suicide among substance users: a review of overlap and risk factors. Drug Alcohol Depend. 2010;110(3):183–92.

Martins SS, Sampson L, Cerda M, Galea S. Worldwide Prevalence and trends in Unintentional Drug Overdose: a systematic review of the literature. Am J Public Health. 2015;105(11):e29–49.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Bmj-Brit Med J. 2021;372.

Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. Bmj-Brit Med J. 2020;368.

National Institutes of Health. National Institutes of Health Study. Quality Assessment Tools 2021 [updated 15/07/2021. https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools .

Critical Appraisal Skills Programme. CASP Qualitative Studies Checklist Oxford2016 [updated 06/12/2016. https://casp-uk.net/casp-tools-checklists/ .

Pizzicato LN, Drake R, Domer-Shank R, Johnson CC, Viner KM. Beyond the walls: risk factors for overdose mortality following release from the Philadelphia Department of Prisons. Drug Alcohol Depen. 2018;189:108–15.

Mitra A, Ahsan H, Li WJ, Liu WS, Kerns RD, Tsai J et al. Risk factors Associated with Nonfatal Opioid Overdose leading to Intensive Care Unit Admission: a cross-sectional study. Jmir Med Inf. 2021;9(11).

Bonar EE, Bohnert ASB. Perceived severity of and susceptibility to overdose among injection drug users: relationships with Overdose History. Subst Use Misuse. 2016;51(10):1379–83.

El-Bassel N, Pala AN, Mukherjee TI, McCrimmon T, Mergenova G, Terlikbayeva A et al. Association of Violence against Female Sex Workers who use drugs with Nonfatal Drug Overdose in Kazakhstan. Jama Netw Open. 2020;3(10).

Havens JR, Oser CB, Knudsen HK, Lofwall M, Stoops WW, Walsh SL, et al. Individual and network factors associated with non-fatal overdose among rural Appalachian drug users. Drug Alcohol Depen. 2011;115(1–2):107–12.

Lamonica AK, Boeri M, Turner J. Circumstances of overdose among suburban women who use opioids: extending an urban analysis informed by drug, set, and setting. Int J Drug Policy. 2021;90.

Latkin CA, Hua W, Tobin K. Social network correlates of self-reported non-fatal overdose. Drug Alcohol Depen. 2004;73(1):61–7.

Milloy MJ, Fairbairn N, Hayashi K, Suwannawong P, Kaplan K, Wood E et al. Overdose experiences among injection drug users in Bangkok, Thailand. Harm Reduct J. 2010;7.

Schiavon S, Hodgin K, Sellers A, Word M, Galbraith JW, Dantzler J, et al. Medical, psychosocial, and treatment predictors of opioid overdose among high risk opioid users. Addict Behav. 2018;86:51–5.

Silva K, Schrager SM, Kecojevic A, Lankenau SE. Factors associated with history of non-fatal overdose among young nonmedical users of prescription drugs. Drug Alcohol Depen. 2013;128(1–2):104–10.

Tobin KE, Hua W, Costenbader EC, Latkin CA. The association between change in social network characteristics and non-fatal overdose: results from the SHIELD study in Baltimore, MD, USA. Drug Alcohol Depen. 2007;87(1):63–8.

Bazazi AR, Zelenev A, Fu JJ, Yee I, Kamarulzaman A, Altice FL. High prevalence of non-fatal overdose among people who inject drugs in Malaysia: correlates of overdose and implications for overdose prevention from a cross-sectional study. Int J Drug Policy. 2015;26(7):675–81.

Goldenberg S, Watt S, Braschel M, Hayashi K, Moreheart S, Shannon K. Police-related barriers to harm reduction linked to non-fatal overdose amongst sex workers who use drugs: results of a community-based cohort in Metro Vancouver, Canada. Int J Drug Policy. 2020;76.

Fairbairn N, Wood E, Stoltz JA, Li K, Montaner J, Kerr T. Crystal methamphetamine use associated with non-fatal overdose among a cohort of injection drug users in Vancouver. Public Health. 2008;122(1):70–8.

Lake S, Hayashi K, Milloy MJ, Wood E, Dong HR, Montaner J, et al. Associations between childhood trauma and non-fatal overdose among people who inject drugs. Addict Behav. 2015;43:83–8.

Pabayo R, Alcantara C, Kawachi I, Wood E, Kerr T. The role of depression and social support in non-fatal drug overdose among a cohort of injection drug users in a Canadian setting. Drug Alcohol Depen. 2013;132(3):603–9.

Thumath M, Humphreys D, Barlow J, Duff P, Braschel M, Bingham B et al. Overdose among mothers: the association between child removal and unintentional drug overdose in a longitudinal cohort of marginalised women in Canada. Int J Drug Policy. 2021;91.

Winter RJ, Stoove M, Degenhardt L, Hellard ME, Spelman T, Jenkinson R, et al. Incidence and predictors of non-fatal drug overdose after release from prison among people who inject drugs in Queensland, Australia. Drug Alcohol Depen. 2015;153:43–9.

Article   CAS   Google Scholar  

Harris MTH, Goldenberg S, Cui ZS, Fairbairn N, Milloy MJS, Hayashi K et al. Association of sex work and social-structural factors with non-fatal overdose among women who use drugs in Vancouver, Canada. Int J Drug Policy. 2023;112.

Tomko C, Schneider KE, Rouhani S, Urquhart GJ, Park JN, Morris M et al. Identifying pathways to recent non-fatal overdose among people who use opioids non-medically: how do psychological pain and unmet mental health need contribute to overdose risk? Addict Behav. 2022;127.

Grau LE, Green TC, Torban M, Blinnikova K, Krupitsky E, Ilyuk R et al. Psychosocial and contextual correlates of opioid overdose risk among drug users in St. Petersburg, Russia. Harm Reduct J. 2009;6.

Vallance K, Pauly B, Wallace B, Chow C, Perkin K, Martin G, et al. Factors associated with public injection and nonfatal overdose among people who inject drugs in street-based settings. Drug-Educ Prev Polic. 2018;25(1):38–46.

Latkin CA, Gicquelais RE, Clyde C, Dayton L, Davey-Rothwell M, German D, et al. Stigma and drug use settings as correlates of self-reported, non-fatal overdose among people who use drugs in Baltimore, Maryland. Int J Drug Policy. 2019;68:86–92.

Riggs KR, Hoge AE, DeRussy AJ, Montgomery AE, Holmes SK, Austin EL et al. Prevalence of and risk factors Associated with Nonfatal Overdose among veterans who have experienced homelessness. Jama Netw Open. 2020;3(3).

Argento E, Shannon K, Fairbairn N, Moreheart S, Braschel M, Goldenberg S. Increasing trends and incidence of nonfatal overdose among women sex workers who use drugs in British Columbia: the role of criminalization-related barriers to harm reduction. Drug Alcohol Depen. 2023;244.

Chang JS, Behar E, Coffin PO. Narratives of people who inject drugs on factors contributing to opioid overdose. Int J Drug Policy. 2019;74:26–32.

Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 2022.

Bohnert ASB, Walton MA, Cunningham RM, Ilgen MA, Barry K, Chermack ST, et al. Overdose and adverse drug event experiences among adult patients in the emergency department. Addict Behav. 2018;86:66–72.

Maher L, Dertadian G. Qualitative research. Addiction. 2018;113(1):167–72.

Rhodes T. The ‘risk environment’: a framework for understanding and reducing drug-related harm. Int J Drug Policy. 2002;13(2):85–94.

Hounkpatin HO, Wood AM, Dunn G. Does income relate to health due to psychosocial or material factors? Consistent support for the psychosocial hypothesis requires operationalization with income rank not the Yitzhaki Index. Soc Sci Med. 2016;150:76–84.

Thomson RM, Igelstrom E, Purba AK, Shimonovich M, Thomson H, McCartney G, et al. How do income changes impact on mental health and wellbeing for working-age adults? A systematic review and meta-analysis. Lancet Public Health. 2022;7(6):e515–28.

Lavalley J, Collins AB, Mayer S, Gaudette L, Krusi A, McNeil R, et al. Negotiating sex work and client interactions in the context of a fentanyl-related overdose epidemic. Cult Health Sex. 2021;23(10):1390–405.

Buttram ME, Surratt HL, Kurtz SP. Resilience and syndemic risk factors among African-American female sex workers. Psychol Health Med. 2014;19(4):442–52.

Footer KHA, Park JN, Allen ST, Decker MR, Silberzahn BE, Huettner S, et al. Police-related correlates of client-perpetrated violence among female sex workers in Baltimore City, Maryland. Am J Public Health. 2019;109(2):289–95.

Brooks-Gordon B, Morris M, Sanders T. Harm reduction and decriminalization of sex work: introduction to the Special section. Sex Res Social Policy. 2021;18(4):809–18.

Turner K, Meyrick J, Miller D, Stopgate L. Which psychosocial interventions improve sex worker well-being? A systematic review of evidence from resource-rich countries. BMJ Sex Reprod Health. 2022;48(e1):e88–100.

Navarro-Carrillo G, Alonso-Ferres M, Moya M, Valor-Segura I. Socioeconomic status and Psychological Well-Being: revisiting the role of subjective socioeconomic status. Front Psychol. 2020;11:1303.

Calvo M, MacFarlane J, Zaccaro H, Curtis M, Caban M, Favaro J, et al. Young people who use drugs engaged in harm reduction programs in New York City: overdose and other risks. Drug Alcohol Depend. 2017;178:106–14.

Stein JA, Dixon EL, Nyamathi AM. Effects of psychosocial and situational variables on substance abuse among homeless adults. Psychol Addict Behav. 2008;22(3):410–6.

Doran KM, Fockele CE, Maguire M. Overdose and homelessness-why we need to talk about housing. Jama Netw Open. 2022;5(1):e2142685.

Fine DR, Dickins KA, Adams LD, De Las Nueces D, Weinstock K, Wright J, et al. Drug overdose mortality among people experiencing homelessness, 2003 to 2018. Jama Netw Open. 2022;5(1):e2142676.

Reilly J, Ho I, Williamson A. A systematic review of the effect of stigma on the health of people experiencing homelessness. Health Soc Care Community. 2022;30(6):2128–41.

Kerr T, Fairbairn N, Tyndall M, Marsh D, Li K, Montaner J, et al. Predictors of non-fatal overdose among a cohort of polysubstance-using injection drug users. Drug Alcohol Depend. 2007;87(1):39–45.

Hyun M, Bae SH, Noh D. Systematic review and meta-analyses of randomized control trials of the effectiveness of psychosocial interventions for homeless adults. J Adv Nurs. 2020;76(3):773–86.

Friedman MR, Kay ES, Maclin BJ, Hawk ME. Housing is harm reduction. AIDS. 2023;37(9):1477–9.

Kerman N, Polillo A, Bardwell G, Gran-Ruaz S, Savage C, Felteau C, et al. Harm reduction outcomes and practices in Housing First: a mixed-methods systematic review. Drug Alcohol Depend. 2021;228:109052.

Nixon LL, Burns VF. Exploring harm reduction in supportive housing for formerly homeless older adults. Can Geriatr J. 2022;25(3):285–94.

Moschion J, Johnson G. Homelessness and incarceration: a reciprocal relationship? J Quant Criminol. 2019;35(4):855–87.

Merrall EL, Kariminia A, Binswanger IA, Hobbs MS, Farrell M, Marsden J, et al. Meta-analysis of drug-related deaths soon after release from prison. Addiction. 2010;105(9):1545–54.

Binswanger IA, Stern MF, Deyo RA, Heagerty PJ, Cheadle A, Elmore JG, et al. Release from prison–a high risk of death for former inmates. N Engl J Med. 2007;356(2):157–65.

Schappell A, Docherty M, Boxer P. Violence and victimization during incarceration: relations to Psychosocial Adjustment during Reentry to the community. Violence Vict. 2016;31(2):361–78.

Hechanova MRM, Reyes JC, Acosta AC, Tuliao AP. Psychosocial treatment for incarcerated methamphetamine users: the Philippines experience. Int J Prison Health. 2020;16(4):343–58.

Binswanger IA, Nowels C, Corsi KF, Glanz J, Long J, Booth RE, et al. Return to drug use and overdose after release from prison: a qualitative study of risk and protective factors. Addict Sci Clin Pract. 2012;7(1):3.

Sondhi AC. Addressing perceptions of opiate-using prisoners to take-home naloxone: findings from one English region. Drugs Alcohol Today. 2016;16(2):124–30.

Pearce LA, Mathany L, Rothon D, Kuo M, Buxton JA. An evaluation of take Home Naloxone program implementation in British columbian correctional facilities. Int J Prison Health. 2019;15(1):46–57.

Joudrey PJ, Khan MR, Wang EA, Scheidell JD, Edelman EJ, McInnes DK, et al. A conceptual model for understanding post-release opioid-related overdose risk. Addict Sci Clin Pract. 2019;14(1):17.

Flam-Ross JM, Lown J, Patil P, White LF, Wang J, Perry A, et al. Factors associated with opioid-involved overdose among previously incarcerated people in the U.S.: a community engaged narrative review. Int J Drug Policy. 2022;100:103534.

Micklitz HM, Glass CM, Bengel J, Sander LB. Efficacy of Psychosocial Interventions for Survivors of Intimate Partner Violence: a systematic review and Meta-analysis. Trauma Violence Abuse. 2023:15248380231169481.

Scottish Government. Support to leave an abusive relationship 2023 [updated 17/10/2023. https://www.gov.scot/news/support-to-leave-an-abusive-relationship/ .

Nambiar D, Stoove M, Dietze P. A cross-sectional study describing factors associated with utilisation of GP services by a cohort of people who inject drugs. BMC Health Serv Res. 2014;14:308.

Chan Carusone S, Guta A, Robinson S, Tan DH, Cooper C, O’Leary B, et al. Maybe if I stop the drugs, then maybe they’d care?-hospital care experiences of people who use drugs. Harm Reduct J. 2019;16(1):16.

Lloyd C. The stigmatization of problem drug users: a narrative literature review. Drugs: Educ Prev Policy. 2013;20(2):85–95.

Muncan B, Walters SM, Ezell J, Ompad DC. They look at us like junkies: influences of drug use stigma on the healthcare engagement of people who inject drugs in New York City. Harm Reduct J. 2020;17(1):53.

Treloar C, Rance J, Yates K, Mao L. Trust and people who inject drugs: the perspectives of clients and staff of needle Syringe Programs. Int J Drug Policy. 2016;27:138–45.

Eimontas J, Gegieckaite G, Zamalijeva O, Pakalniskiene V. Unmet Healthcare needs Predict Depression symptoms among older adults. Int J Environ Res Public Health. 2022;19(15).

McAuley A, Fraser R, Glancy M, Yeung A, Jones HE, Vickerman P, et al. Mortality among individuals prescribed opioid-agonist therapy in Scotland, UK, 2011-20: a national retrospective cohort study. Lancet Public Health. 2023;8(7):e484–93.

Bogdanowicz KM, Stewart R, Chang CK, Shetty H, Khondoker M, Day E, et al. Excess overdose mortality immediately following transfer of patients and their care as well as after cessation of opioid substitution therapy. Addiction. 2018;113(5):946–51.

Clausen T, Waal H, Thoresen M, Gossop M. Mortality among opiate users: opioid maintenance therapy, age and causes of death. Addiction. 2009;104(8):1356–62.

Bourgois P. Disciplining addictions: the bio-politics of methadone and heroin in the United States. Cult Med Psychiatry. 2000;24(2):165–95.

Moore D. Governing street-based injecting drug users: a critique of heroin overdose prevention in Australia. Soc Sci Med. 2004;59(7):1547–57.

Gagnon M, Rudzinski K, Guta A, Schmidt RA, Kryszajtys DT, Kolla G, et al. Impact of safer supply programs on injection practices: client and provider experiences in Ontario, Canada. Harm Reduct J. 2023;20(1):81.

The Lancet. Drug decriminalisation: grounding policy in evidence. Lancet. 2023;402(10416):1941.

Cooper HL. War on drugs Policing and Police Brutality. Subst Use Misuse. 2015;50(8–9):1188–94.

Mattingly DT, Howard LC, Krueger EA, Fleischer NL, Hughes-Halbert C, Leventhal AM. Change in distress about police brutality and substance use among young people, 2017–2020. Drug Alcohol Depend. 2022;237:109530.

Jalali MS, Botticelli M, Hwang RC, Koh HK, McHugh RK. The opioid crisis: need for systems science research. Health Res Policy Syst. 2020;18(1):88.

Jalali MS, Botticelli M, Hwang RC, Koh HK, McHugh RK. The opioid crisis: a contextual, social-ecological framework. Health Res Policy Syst. 2020;18(1):87.

Ozbay F, Johnson DC, Dimoulas E, Morgan CA, Charney D, Southwick S. Social support and resilience to stress: from neurobiology to clinical practice. Psychiatry (Edgmont). 2007;4(5):35–40.

PubMed   Google Scholar  

Li F, Luo S, Mu W, Li Y, Ye L, Zheng X, et al. Effects of sources of social support and resilience on the mental health of different age groups during the COVID-19 pandemic. BMC Psychiatry. 2021;21(1):16.

Aram JW, Spencer MRT, Garnett MF, Hedegaard HB. Psychological distress and the risk of drug overdose death. J Affect Disord. 2022;318:16–21.

Malaguti A, Byrne CJ, Sani F, Power K, Eriksen A, Dillon JF. Drug network identification predicts injecting risk behavior among people who inject drugs on hepatitis C virus treatment in Tayside, Scotland. Behav Med. 2022:1–11.

Donaldson SR, Radley A, Dillon JF. Transformation of identity in substance use as a pathway to recovery and the potential of treatment for hepatitis C: a systematic review protocol. BMJ Open. 2022;12(2):e049713.

Martinelli TF, van de Mheen D, Best D, Vanderplasschen W, Nagelhout GE. Are members of mutual aid groups better equipped for addiction recovery? European cross-sectional study into recovery capital, social networks, and commitment to sobriety. Drugs: Educ Prev Policy. 2021;28(5):389–98.

Meyers RJ, Miller WR, Hill DE, Tonigan JS. Community reinforcement and family training (CRAFT): engaging unmotivated drug users in treatment. J Subst Abuse. 1998;10(3):291–308.

UK Addiction Treatment Centres. Five Reasons Why Recovery Cafes Are Becoming Popular in the UK 2018 [ https://www.ukat.co.uk/blog/society/5-reasons-recovery-cafes-becoming-popular-uk/ .

Perri M, Kaminski N, Bonn M, Kolla G, Guta A, Bayoumi AM, et al. A qualitative study on overdose response in the era of COVID-19 and beyond: how to spot someone so they never have to use alone. Harm Reduct J. 2021;18(1):85.

Schneider KE, Tomko C, Nestadt DF, Silberzahn BE, White RH, Sherman SG. Conceptualizing overdose trauma: the relationships between experiencing and witnessing overdoses with PTSD symptoms among street-recruited female sex workers in Baltimore, Maryland. Int J Drug Policy. 2021;92:102859.

Macmadu A, Frueh L, Collins AB, Newman R, Barnett NP, Rich JD, et al. Drug use behaviors, trauma, and emotional affect following the overdose of a social network member: a qualitative investigation. Int J Drug Policy. 2022;107:103792.

Gehringer R, Freytag A, Krause M, Schlattmann P, Schmidt K, Schulz S, et al. Psychological interventions for posttraumatic stress disorder involving primary care physicians: systematic review and Meta-analysis of randomized controlled trials. BMC Fam Pract. 2020;21(1):176.

Stein MB. Posttraumatic stress disorder in adults: Psychotherapy and psychosocial interventions: Wolters Kluwer; 2021 [updated 31/07/2023. www.uptodate.com/contents/posttraumatic-stress-disorder-in-adults-psychotherapy-and-psychosocial-interventions .

Herman JL. Trauma and recovery: the aftermath of violence - from domestic abuse to political terror. 2022 ed. New York: Basic Books; 2022. 469 p.

Gilchrist G, Swan D, Shaw A, Keding A, Towers S, Craine N, et al. The acceptability and feasibility of a brief psychosocial intervention to reduce blood-borne virus risk behaviours among people who inject drugs: a randomised control feasibility trial of a psychosocial intervention (the PROTECT study) versus treatment as usual. Harm Reduct J. 2017;14(1):14.

Bujalski M, Wieczorek L, Sieroslawski J. Risk perception and risk aversion among people who use new psychoactive substances. Int J Drug Policy. 2021;97:103326.

Cumpston MS, Brennan SE, Ryan R, McKenzie JE. Synthesis methods other than meta-analysis were commonly used but seldom specified: survey of systematic reviews. J Clin Epidemiol. 2023;156:42–52.

United Nations Office on Drugs and Crime. World Drug Report (Booklet 1 - executive Summary/Policy implications). Vienna: United Nations Office on Drugs and Crime; 2022. 26/06/2022.

Degenhardt L, Webb P, Colledge-Frisby S, Ireland J, Wheeler A, Ottaviano S, et al. Epidemiology of injecting drug use, prevalence of injecting-related harm, and exposure to behavioural and environmental risks among people who inject drugs: a systematic review. Lancet Glob Health. 2023;11(5):e659–72.

Greer A, Ritter A. Harm reduction and the opioid crisis: emerging policy challenges. Int J Drug Policy. 2019;71:139–41.

Download references

Acknowledgements

We wish to acknowledge and thank Teresa Flynn, Tammie Brown, Ann Eriksen, Dr Jennifer Breen, and Dr Fiona Cowden, for their contributions to this review and our wider research programme.

This study was funded by the Scottish Drug Death Taskforce (grant number: DDTFRF16). The funder was not involved in collection, analysis, and/or interpretation of data, in the writing of the report, or in the decision to submit the manuscript for publication.

Author information

Authors and affiliations.

Division of Molecular and Clinical Medicine, School of Medicine, Ninewells Hospital and Medical School, University of Dundee, Dundee, UK

Christopher J. Byrne

Directorate of Public Health, NHS Tayside, Kings Cross Hospital, Dundee, UK

Christopher J. Byrne, Donna Thain & Emma H. Fletcher

Division of Psychology, School of Humanities, Social Sciences and Law, University of Dundee, Scrymgeour Building, Dundee, UK

Fabio Sani & Amy Malaguti

Tayside Drug and Alcohol Recovery Psychology Service, NHS Tayside, Dundee, UK

Amy Malaguti

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualisation: AM. Methodology: AM, CJB, FS. Software: Not applicable. Validation: Not applicable. Formal analysis: AM, CJB, FS. Investigation: AM, CJB, FS. Resources: Not applicable. Data curation: AM, CJB, FS. Writing – Original Draft: CJB, AM. Writing – Review & Editing: All authors. Visualisation: AM, CJB. Supervision: AM, CJB. Project administration: AM. Funding acquisition: AM.

Corresponding author

Correspondence to Christopher J. Byrne .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

CJB has received honoraria from the International Network for Health and Hepatitis in Substance Users (INHSU), and grant funding from the Scottish Society of Physicians, unrelated to the submitted work. FS received funding from the Scottish Drug Deaths Taskforce related to the submitted work. AM has received funding from the Scottish Drug Deaths Taskforce related to the submitted work, and funding from the British Psychological Society unrelated to the submitted work. EF and DT report no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Byrne, C.J., Sani, F., Thain, D. et al. Psychosocial factors associated with overdose subsequent to Illicit Drug use: a systematic review and narrative synthesis. Harm Reduct J 21 , 81 (2024). https://doi.org/10.1186/s12954-024-00999-8

Download citation

Received : 18 September 2023

Accepted : 03 April 2024

Published : 15 April 2024

DOI : https://doi.org/10.1186/s12954-024-00999-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Psychosocial
• Narrative synthesis
• Systematic review
• Drug-related death

Harm Reduction Journal

ISSN: 1477-7517

• Submission enquiries: [email protected]

SYSTEMATIC REVIEW article

This article is part of the research topic.

Assessment and Monitoring of Human Movement

Electromechanical efficiency index of skeletal muscle and its applicability: a systematic review Provisionally Accepted

• 1 Institute of Kinesiology, Faculty of Sport, University of Ljubljana, Slovenia
• 2 Institute of Sport, Faculty of Sport, University of Ljubljana, Slovenia

The final, formatted version of the article will be published soon.

The electromechanical efficiency of skeletal muscle represents the dissociation between electrical and mechanical events within a muscle. It has been widely studied, with varying methods for its measurement and calculation. For this reason, the purpose of this literature review was to integrate the available research to date and provide more insights about this measure. A systematic search of the literature was performed across three online databases: PubMed, ScienceDirect, and SPORTDiscus. This yielded 1284 reports, of which 10 met the inclusion criteria. Included studies have used different methods to measure the electromechanical efficiency (EME) index, including electromyography (EMG), mechanomyography (MMG) and tensiomyography (TMG). The EME index was used to assess muscle conditions such as muscle atrophy, pain syndromes, or to monitor rehabilitation in patients with knee problems, fatigue and the effects of exercise and rehabilitation. TMG has been shown to be one of the most reliable methods to obtain the EME index, but its use precludes obtaining the index during voluntary muscle contractions. Standardizing the EME index is crucial for its diverse applications in clinical, sport, and rehabilitation contexts. Future research should prioritize standardization of measurement protocols for establishing the most repeatable, and reliable approach that can be used for inter-individual comparisons or for assessing an individual for multiple times over a longer period.

Keywords: EME index, muscle function, Tensiomyography, Electromyography, Rehabilitation, Athletic Performance

Received: 08 Mar 2024; Accepted: 17 Apr 2024.

Copyright: © 2024 Paravlic and Turnsek. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mx. Armin H. Paravlic, Institute of Kinesiology, Faculty of Sport, University of Ljubljana, Ljubljana, Slovenia

People also looked at

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley-Blackwell Online Open

An overview of methodological approaches in systematic reviews

Prabhakar veginadu.

1 Department of Rural Clinical Sciences, La Trobe Rural Health School, La Trobe University, Bendigo Victoria, Australia

Hanny Calache

2 Lincoln International Institute for Rural Health, University of Lincoln, Brayford Pool, Lincoln UK

Akshaya Pandian

3 Department of Orthodontics, Saveetha Dental College, Chennai Tamil Nadu, India

Mohd Masood

Associated data.

APPENDIX B: List of excluded studies with detailed reasons for exclusion

APPENDIX C: Quality assessment of included reviews using AMSTAR 2

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full‐text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non‐English, gray, and unpublished literature, and (c) use of text‐mining approaches during title and abstract screening.

The overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

1. INTRODUCTION

Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the “gold standard” of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search, appraise, and synthesize the available evidence. 3 Several guidelines, developed by various organizations, are available for the conduct of an SR; 4 , 5 , 6 , 7 among these, Cochrane is considered a pioneer in developing rigorous and highly structured methodology for the conduct of SRs. 8 The guidelines developed by these organizations outline seven fundamental steps required in SR process: defining the scope of the review and eligibility criteria, literature searching and retrieval, selecting eligible studies, extracting relevant data, assessing risk of bias (RoB) in included studies, synthesizing results, and assessing certainty of evidence (CoE) and presenting findings. 4 , 5 , 6 , 7

The methodological rigor involved in an SR can require a significant amount of time and resource, which may not always be available. 9 As a result, there has been a proliferation of modifications made to the traditional SR process, such as refining, shortening, bypassing, or omitting one or more steps, 10 , 11 for example, limits on the number and type of databases searched, limits on publication date, language, and types of studies included, and limiting to one reviewer for screening and selection of studies, as opposed to two or more reviewers. 10 , 11 These methodological modifications are made to accommodate the needs of and resource constraints of the reviewers and stakeholders (e.g., organizations, policymakers, health care professionals, and other knowledge users). While such modifications are considered time and resource efficient, they may introduce bias in the review process reducing their usefulness. 5

Substantial research has been conducted examining various approaches used in the standardized SR methodology and their impact on the validity of SR results. There are a number of published reviews examining the approaches or modifications corresponding to single 12 , 13 or multiple steps 14 involved in an SR. However, there is yet to be a comprehensive summary of the SR‐level evidence for all the seven fundamental steps in an SR. Such a holistic evidence synthesis will provide an empirical basis to confirm the validity of current accepted practices in the conduct of SRs. Furthermore, sometimes there is a balance that needs to be achieved between the resource availability and the need to synthesize the evidence in the best way possible, given the constraints. This evidence base will also inform the choice of modifications to be made to the SR methods, as well as the potential impact of these modifications on the SR results. An overview is considered the choice of approach for summarizing existing evidence on a broad topic, directing the reader to evidence, or highlighting the gaps in evidence, where the evidence is derived exclusively from SRs. 15 Therefore, for this review, an overview approach was used to (a) identify and collate evidence from existing published SR articles evaluating various methodological approaches employed in each of the seven fundamental steps of an SR and (b) highlight both the gaps in the current research and the potential areas for future research on the methods employed in SRs.

An a priori protocol was developed for this overview but was not registered with the International Prospective Register of Systematic Reviews (PROSPERO), as the review was primarily methodological in nature and did not meet PROSPERO eligibility criteria for registration. The protocol is available from the corresponding author upon reasonable request. This overview was conducted based on the guidelines for the conduct of overviews as outlined in The Cochrane Handbook. 15 Reporting followed the Preferred Reporting Items for Systematic reviews and Meta‐analyses (PRISMA) statement. 3

2.1. Eligibility criteria

Only published SRs, with or without associated MA, were included in this overview. We adopted the defining characteristics of SRs from The Cochrane Handbook. 5 According to The Cochrane Handbook, a review was considered systematic if it satisfied the following criteria: (a) clearly states the objectives and eligibility criteria for study inclusion; (b) provides reproducible methodology; (c) includes a systematic search to identify all eligible studies; (d) reports assessment of validity of findings of included studies (e.g., RoB assessment of the included studies); (e) systematically presents all the characteristics or findings of the included studies. 5 Reviews that did not meet all of the above criteria were not considered a SR for this study and were excluded. MA‐only articles were included if it was mentioned that the MA was based on an SR.

SRs and/or MA of primary studies evaluating methodological approaches used in defining review scope and study eligibility, literature search, study selection, data extraction, RoB assessment, data synthesis, and CoE assessment and reporting were included. The methodological approaches examined in these SRs and/or MA can also be related to the substeps or elements of these steps; for example, applying limits on date or type of publication are the elements of literature search. Included SRs examined or compared various aspects of a method or methods, and the associated factors, including but not limited to: precision or effectiveness; accuracy or reliability; impact on the SR and/or MA results; reproducibility of an SR steps or bias occurred; time and/or resource efficiency. SRs assessing the methodological quality of SRs (e.g., adherence to reporting guidelines), evaluating techniques for building search strategies or the use of specific database filters (e.g., use of Boolean operators or search filters for randomized controlled trials), examining various tools used for RoB or CoE assessment (e.g., ROBINS vs. Cochrane RoB tool), or evaluating statistical techniques used in meta‐analyses were excluded. 14

2.2. Search

The search for published SRs was performed on the following scientific databases initially from inception to third week of November 2020 and updated in the last week of February 2022: MEDLINE (via Ovid), Embase (via Ovid), Web of Science Core Collection, Cochrane Database of Systematic Reviews, and American Psychological Association (APA) PsycINFO. Search was restricted to English language publications. Following the objectives of this study, study design filters within databases were used to restrict the search to SRs and MA, where available. The reference lists of included SRs were also searched for potentially relevant publications.

The search terms included keywords, truncations, and subject headings for the key concepts in the review question: SRs and/or MA, methods, and evaluation. Some of the terms were adopted from the search strategy used in a previous review by Robson et al., which reviewed primary studies on methodological approaches used in study selection, data extraction, and quality appraisal steps of SR process. 14 Individual search strategies were developed for respective databases by combining the search terms using appropriate proximity and Boolean operators, along with the related subject headings in order to identify SRs and/or MA. 16 , 17 A senior librarian was consulted in the design of the search terms and strategy. Appendix A presents the detailed search strategies for all five databases.

2.3. Study selection and data extraction

Title and abstract screening of references were performed in three steps. First, one reviewer (PV) screened all the titles and excluded obviously irrelevant citations, for example, articles on topics not related to SRs, non‐SR publications (such as randomized controlled trials, observational studies, scoping reviews, etc.). Next, from the remaining citations, a random sample of 200 titles and abstracts were screened against the predefined eligibility criteria by two reviewers (PV and MM), independently, in duplicate. Discrepancies were discussed and resolved by consensus. This step ensured that the responses of the two reviewers were calibrated for consistency in the application of the eligibility criteria in the screening process. Finally, all the remaining titles and abstracts were reviewed by a single “calibrated” reviewer (PV) to identify potential full‐text records. Full‐text screening was performed by at least two authors independently (PV screened all the records, and duplicate assessment was conducted by MM, HC, or MG), with discrepancies resolved via discussions or by consulting a third reviewer.

Data related to review characteristics, results, key findings, and conclusions were extracted by at least two reviewers independently (PV performed data extraction for all the reviews and duplicate extraction was performed by AP, HC, or MG).

2.4. Quality assessment of included reviews

The quality assessment of the included SRs was performed using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). The tool consists of a 16‐item checklist addressing critical and noncritical domains. 18 For the purpose of this study, the domain related to MA was reclassified from critical to noncritical, as SRs with and without MA were included. The other six critical domains were used according to the tool guidelines. 18 Two reviewers (PV and AP) independently responded to each of the 16 items in the checklist with either “yes,” “partial yes,” or “no.” Based on the interpretations of the critical and noncritical domains, the overall quality of the review was rated as high, moderate, low, or critically low. 18 Disagreements were resolved through discussion or by consulting a third reviewer.

2.5. Data synthesis

To provide an understandable summary of existing evidence syntheses, characteristics of the methods evaluated in the included SRs were examined and key findings were categorized and presented based on the corresponding step in the SR process. The categories of key elements within each step were discussed and agreed by the authors. Results of the included reviews were tabulated and summarized descriptively, along with a discussion on any overlap in the primary studies. 15 No quantitative analyses of the data were performed.

From 41,556 unique citations identified through literature search, 50 full‐text records were reviewed, and nine systematic reviews 14 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 were deemed eligible for inclusion. The flow of studies through the screening process is presented in Figure  1 . A list of excluded studies with reasons can be found in Appendix B .

Study selection flowchart

3.1. Characteristics of included reviews

Table  1 summarizes the characteristics of included SRs. The majority of the included reviews (six of nine) were published after 2010. 14 , 22 , 23 , 24 , 25 , 26 Four of the nine included SRs were Cochrane reviews. 20 , 21 , 22 , 23 The number of databases searched in the reviews ranged from 2 to 14, 2 reviews searched gray literature sources, 24 , 25 and 7 reviews included a supplementary search strategy to identify relevant literature. 14 , 19 , 20 , 21 , 22 , 23 , 26 Three of the included SRs (all Cochrane reviews) included an integrated MA. 20 , 21 , 23

Characteristics of included studies

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.

The included SRs evaluated 24 unique methodological approaches (26 in total) used across five steps in the SR process; 8 SRs evaluated 6 approaches, 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 while 1 review evaluated 18 approaches. 14 Exclusion of gray or unpublished literature 21 , 26 and blinding of reviewers for RoB assessment 14 , 23 were evaluated in two reviews each. Included SRs evaluated methods used in five different steps in the SR process, including methods used in defining the scope of review ( n  = 3), literature search ( n  = 3), study selection ( n  = 2), data extraction ( n  = 1), and RoB assessment ( n  = 2) (Table  2 ).

Summary of findings from review evaluating systematic review methods

There was some overlap in the primary studies evaluated in the included SRs on the same topics: Schmucker et al. 26 and Hopewell et al. 21 ( n  = 4), Hopewell et al. 20 and Crumley et al. 19 ( n  = 30), and Robson et al. 14 and Morissette et al. 23 ( n  = 4). There were no conflicting results between any of the identified SRs on the same topic.

3.2. Methodological quality of included reviews

Overall, the quality of the included reviews was assessed as moderate at best (Table  2 ). The most common critical weakness in the reviews was failure to provide justification for excluding individual studies (four reviews). Detailed quality assessment is provided in Appendix C .

3.3. Evidence on systematic review methods

3.3.1. methods for defining review scope and eligibility.

Two SRs investigated the effect of excluding data obtained from gray or unpublished sources on the pooled effect estimates of MA. 21 , 26 Hopewell et al. 21 reviewed five studies that compared the impact of gray literature on the results of a cohort of MA of RCTs in health care interventions. Gray literature was defined as information published in “print or electronic sources not controlled by commercial or academic publishers.” Findings showed an overall greater treatment effect for published trials than trials reported in gray literature. In a more recent review, Schmucker et al. 26 addressed similar objectives, by investigating gray and unpublished data in medicine. In addition to gray literature, defined similar to the previous review by Hopewell et al., the authors also evaluated unpublished data—defined as “supplemental unpublished data related to published trials, data obtained from the Food and Drug Administration  or other regulatory websites or postmarketing analyses hidden from the public.” The review found that in majority of the MA, excluding gray literature had little or no effect on the pooled effect estimates. The evidence was limited to conclude if the data from gray and unpublished literature had an impact on the conclusions of MA. 26

Morrison et al. 24 examined five studies measuring the effect of excluding non‐English language RCTs on the summary treatment effects of SR‐based MA in various fields of conventional medicine. Although none of the included studies reported major difference in the treatment effect estimates between English only and non‐English inclusive MA, the review found inconsistent evidence regarding the methodological and reporting quality of English and non‐English trials. 24 As such, there might be a risk of introducing “language bias” when excluding non‐English language RCTs. The authors also noted that the numbers of non‐English trials vary across medical specialties, as does the impact of these trials on MA results. Based on these findings, Morrison et al. 24 conclude that literature searches must include non‐English studies when resources and time are available to minimize the risk of introducing “language bias.”

3.3.2. Methods for searching studies

Crumley et al. 19 analyzed recall (also referred to as “sensitivity” by some researchers; defined as “percentage of relevant studies identified by the search”) and precision (defined as “percentage of studies identified by the search that were relevant”) when searching a single resource to identify randomized controlled trials and controlled clinical trials, as opposed to searching multiple resources. The studies included in their review frequently compared a MEDLINE only search with the search involving a combination of other resources. The review found low median recall estimates (median values between 24% and 92%) and very low median precisions (median values between 0% and 49%) for most of the electronic databases when searched singularly. 19 A between‐database comparison, based on the type of search strategy used, showed better recall and precision for complex and Cochrane Highly Sensitive search strategies (CHSSS). In conclusion, the authors emphasize that literature searches for trials in SRs must include multiple sources. 19

In an SR comparing handsearching and electronic database searching, Hopewell et al. 20 found that handsearching retrieved more relevant RCTs (retrieval rate of 92%−100%) than searching in a single electronic database (retrieval rates of 67% for PsycINFO/PsycLIT, 55% for MEDLINE, and 49% for Embase). The retrieval rates varied depending on the quality of handsearching, type of electronic search strategy used (e.g., simple, complex or CHSSS), and type of trial reports searched (e.g., full reports, conference abstracts, etc.). The authors concluded that handsearching was particularly important in identifying full trials published in nonindexed journals and in languages other than English, as well as those published as abstracts and letters. 20

The effectiveness of checking reference lists to retrieve additional relevant studies for an SR was investigated by Horsley et al. 22 The review reported that checking reference lists yielded 2.5%–40% more studies depending on the quality and comprehensiveness of the electronic search used. The authors conclude that there is some evidence, although from poor quality studies, to support use of checking reference lists to supplement database searching. 22

3.3.3. Methods for selecting studies

Three approaches relevant to reviewer characteristics, including number, experience, and blinding of reviewers involved in the screening process were highlighted in an SR by Robson et al. 14 Based on the retrieved evidence, the authors recommended that two independent, experienced, and unblinded reviewers be involved in study selection. 14 A modified approach has also been suggested by the review authors, where one reviewer screens and the other reviewer verifies the list of excluded studies, when the resources are limited. It should be noted however this suggestion is likely based on the authors’ opinion, as there was no evidence related to this from the studies included in the review.

Robson et al. 14 also reported two methods describing the use of technology for screening studies: use of Google Translate for translating languages (for example, German language articles to English) to facilitate screening was considered a viable method, while using two computer monitors for screening did not increase the screening efficiency in SR. Title‐first screening was found to be more efficient than simultaneous screening of titles and abstracts, although the gain in time with the former method was lesser than the latter. Therefore, considering that the search results are routinely exported as titles and abstracts, Robson et al. 14 recommend screening titles and abstracts simultaneously. However, the authors note that these conclusions were based on very limited number (in most instances one study per method) of low‐quality studies. 14

3.3.4. Methods for data extraction

Robson et al. 14 examined three approaches for data extraction relevant to reviewer characteristics, including number, experience, and blinding of reviewers (similar to the study selection step). Although based on limited evidence from a small number of studies, the authors recommended use of two experienced and unblinded reviewers for data extraction. The experience of the reviewers was suggested to be especially important when extracting continuous outcomes (or quantitative) data. However, when the resources are limited, data extraction by one reviewer and a verification of the outcomes data by a second reviewer was recommended.

As for the methods involving use of technology, Robson et al. 14 identified limited evidence on the use of two monitors to improve the data extraction efficiency and computer‐assisted programs for graphical data extraction. However, use of Google Translate for data extraction in non‐English articles was not considered to be viable. 14 In the same review, Robson et al. 14 identified evidence supporting contacting authors for obtaining additional relevant data.

3.3.5. Methods for RoB assessment

Two SRs examined the impact of blinding of reviewers for RoB assessments. 14 , 23 Morissette et al. 23 investigated the mean differences between the blinded and unblinded RoB assessment scores and found inconsistent differences among the included studies providing no definitive conclusions. Similar conclusions were drawn in a more recent review by Robson et al., 14 which included four studies on reviewer blinding for RoB assessment that completely overlapped with Morissette et al. 23

Use of experienced reviewers and provision of additional guidance for RoB assessment were examined by Robson et al. 14 The review concluded that providing intensive training and guidance on assessing studies reporting insufficient data to the reviewers improves RoB assessments. 14 Obtaining additional data related to quality assessment by contacting study authors was also found to help the RoB assessments, although based on limited evidence. When assessing the qualitative or mixed method reviews, Robson et al. 14 recommends the use of a structured RoB tool as opposed to an unstructured tool. No SRs were identified on data synthesis and CoE assessment and reporting steps.

4. DISCUSSION

4.1. summary of findings.

Nine SRs examining 24 unique methods used across five steps in the SR process were identified in this overview. The collective evidence supports some current traditional and modified SR practices, while challenging other approaches. However, the quality of the included reviews was assessed to be moderate at best and in the majority of the included SRs, evidence related to the evaluated methods was obtained from very limited numbers of primary studies. As such, the interpretations from these SRs should be made cautiously.

The evidence gathered from the included SRs corroborate a few current SR approaches. 5 For example, it is important to search multiple resources for identifying relevant trials (RCTs and/or CCTs). The resources must include a combination of electronic database searching, handsearching, and reference lists of retrieved articles. 5 However, no SRs have been identified that evaluated the impact of the number of electronic databases searched. A recent study by Halladay et al. 27 found that articles on therapeutic intervention, retrieved by searching databases other than PubMed (including Embase), contributed only a small amount of information to the MA and also had a minimal impact on the MA results. The authors concluded that when the resources are limited and when large number of studies are expected to be retrieved for the SR or MA, PubMed‐only search can yield reliable results. 27

Findings from the included SRs also reiterate some methodological modifications currently employed to “expedite” the SR process. 10 , 11 For example, excluding non‐English language trials and gray/unpublished trials from MA have been shown to have minimal or no impact on the results of MA. 24 , 26 However, the efficiency of these SR methods, in terms of time and the resources used, have not been evaluated in the included SRs. 24 , 26 Of the SRs included, only two have focused on the aspect of efficiency 14 , 25 ; O'Mara‐Eves et al. 25 report some evidence to support the use of text‐mining approaches for title and abstract screening in order to increase the rate of screening. Moreover, only one included SR 14 considered primary studies that evaluated reliability (inter‐ or intra‐reviewer consistency) and accuracy (validity when compared against a “gold standard” method) of the SR methods. This can be attributed to the limited number of primary studies that evaluated these outcomes when evaluating the SR methods. 14 Lack of outcome measures related to reliability, accuracy, and efficiency precludes making definitive recommendations on the use of these methods/modifications. Future research studies must focus on these outcomes.

Some evaluated methods may be relevant to multiple steps; for example, exclusions based on publication status (gray/unpublished literature) and language of publication (non‐English language studies) can be outlined in the a priori eligibility criteria or can be incorporated as search limits in the search strategy. SRs included in this overview focused on the effect of study exclusions on pooled treatment effect estimates or MA conclusions. Excluding studies from the search results, after conducting a comprehensive search, based on different eligibility criteria may yield different results when compared to the results obtained when limiting the search itself. 28 Further studies are required to examine this aspect.

Although we acknowledge the lack of standardized quality assessment tools for methodological study designs, we adhered to the Cochrane criteria for identifying SRs in this overview. This was done to ensure consistency in the quality of the included evidence. As a result, we excluded three reviews that did not provide any form of discussion on the quality of the included studies. The methods investigated in these reviews concern supplementary search, 29 data extraction, 12 and screening. 13 However, methods reported in two of these three reviews, by Mathes et al. 12 and Waffenschmidt et al., 13 have also been examined in the SR by Robson et al., 14 which was included in this overview; in most instances (with the exception of one study included in Mathes et al. 12 and Waffenschmidt et al. 13 each), the studies examined in these excluded reviews overlapped with those in the SR by Robson et al. 14

One of the key gaps in the knowledge observed in this overview was the dearth of SRs on the methods used in the data synthesis component of SR. Narrative and quantitative syntheses are the two most commonly used approaches for synthesizing data in evidence synthesis. 5 There are some published studies on the proposed indications and implications of these two approaches. 30 , 31 These studies found that both data synthesis methods produced comparable results and have their own advantages, suggesting that the choice of the method must be based on the purpose of the review. 31 With increasing number of “expedited” SR approaches (so called “rapid reviews”) avoiding MA, 10 , 11 further research studies are warranted in this area to determine the impact of the type of data synthesis on the results of the SR.

4.2. Implications for future research

The findings of this overview highlight several areas of paucity in primary research and evidence synthesis on SR methods. First, no SRs were identified on methods used in two important components of the SR process, including data synthesis and CoE and reporting. As for the included SRs, a limited number of evaluation studies have been identified for several methods. This indicates that further research is required to corroborate many of the methods recommended in current SR guidelines. 4 , 5 , 6 , 7 Second, some SRs evaluated the impact of methods on the results of quantitative synthesis and MA conclusions. Future research studies must also focus on the interpretations of SR results. 28 , 32 Finally, most of the included SRs were conducted on specific topics related to the field of health care, limiting the generalizability of the findings to other areas. It is important that future research studies evaluating evidence syntheses broaden the objectives and include studies on different topics within the field of health care.

4.3. Strengths and limitations

To our knowledge, this is the first overview summarizing current evidence from SRs and MA on different methodological approaches used in several fundamental steps in SR conduct. The overview methodology followed well established guidelines and strict criteria defined for the inclusion of SRs.

There are several limitations related to the nature of the included reviews. Evidence for most of the methods investigated in the included reviews was derived from a limited number of primary studies. Also, the majority of the included SRs may be considered outdated as they were published (or last updated) more than 5 years ago 33 ; only three of the nine SRs have been published in the last 5 years. 14 , 25 , 26 Therefore, important and recent evidence related to these topics may not have been included. Substantial numbers of included SRs were conducted in the field of health, which may limit the generalizability of the findings. Some method evaluations in the included SRs focused on quantitative analyses components and MA conclusions only. As such, the applicability of these findings to SR more broadly is still unclear. 28 Considering the methodological nature of our overview, limiting the inclusion of SRs according to the Cochrane criteria might have resulted in missing some relevant evidence from those reviews without a quality assessment component. 12 , 13 , 29 Although the included SRs performed some form of quality appraisal of the included studies, most of them did not use a standardized RoB tool, which may impact the confidence in their conclusions. Due to the type of outcome measures used for the method evaluations in the primary studies and the included SRs, some of the identified methods have not been validated against a reference standard.

Some limitations in the overview process must be noted. While our literature search was exhaustive covering five bibliographic databases and supplementary search of reference lists, no gray sources or other evidence resources were searched. Also, the search was primarily conducted in health databases, which might have resulted in missing SRs published in other fields. Moreover, only English language SRs were included for feasibility. As the literature search retrieved large number of citations (i.e., 41,556), the title and abstract screening was performed by a single reviewer, calibrated for consistency in the screening process by another reviewer, owing to time and resource limitations. These might have potentially resulted in some errors when retrieving and selecting relevant SRs. The SR methods were grouped based on key elements of each recommended SR step, as agreed by the authors. This categorization pertains to the identified set of methods and should be considered subjective.

5. CONCLUSIONS

This overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process. Limited evidence was also identified on some methodological modifications currently used to expedite the SR process. Overall, findings highlight the dearth of SRs on SR methodologies, warranting further work to confirm several current recommendations on conventional and expedited SR processes.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

APPENDIX A: Detailed search strategies

ACKNOWLEDGMENTS

The first author is supported by a La Trobe University Full Fee Research Scholarship and a Graduate Research Scholarship.

Open Access Funding provided by La Trobe University.

Veginadu P, Calache H, Gussy M, Pandian A, Masood M. An overview of methodological approaches in systematic reviews . J Evid Based Med . 2022; 15 :39–54. 10.1111/jebm.12468 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]