database searching for literature review

An official website of the United States government

Here's how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Literature Search: Databases and Gray Literature

The literature search.

• A systematic review search includes a search of databases, gray literature, personal communications, and a handsearch of high impact journals in the related field.  See our list of recommended databases and gray literature sources on this page.
• a comprehensive literature search can not be dependent on a single database, nor on bibliographic databases only.
• inclusion of multiple databases helps avoid publication bias (georaphic bias or bias against publication of negative results).
• The Cochrane Collaboration recommends PubMed, Embase and the Cochrane Central Register of Controlled Trials (CENTRAL) at a minimum.     
• NOTE:  The Cochrane Collaboration and the IOM recommend that the literature search be conducted by librarians or persons with extensive literature search experience. Please contact the NIH Librarians for assistance with the literature search component of your systematic review. 

Cochrane Library

A collection of six databases that contain different types of high-quality, independent evidence to inform healthcare decision-making. Search the Cochrane Central Register of Controlled Trials here.

European database of biomedical and pharmacologic literature.

PubMed comprises more than 21 million citations for biomedical literature from MEDLINE, life science journals, and online books.

Largest abstract and citation database of peer-reviewed literature and quality web sources. Contains conference papers.

Web of Science

World's leading citation databases. Covers over 12,000 of the highest impact journals worldwide, including Open Access journals and over 150,000 conference proceedings. Coverage in the sciences, social sciences, arts, and humanities, with coverage to 1900.

Subject Specific Databases

APA PsycINFO

Over 4.5 million abstracts of peer-reviewed literature in the behavioral and social sciences. Includes conference papers, book chapters, psychological tests, scales and measurement tools.

CINAHL Plus

Comprehensive journal index to nursing and allied health literature, includes books, nursing dissertations, conference proceedings, practice standards and book chapters.

Latin American and Caribbean health sciences literature database

Gray Literature

• Gray Literature  is the term for information that falls outside the mainstream of published journal and mongraph literature, not controlled by commercial publishers
• hard to find studies, reports, or dissertations
• conference abstracts or papers
• governmental or private sector research
• clinical trials - ongoing or unpublished
• experts and researchers in the field     
• Library catalogs
• Professional association websites
• Google Scholar  - Search scholarly literature across many disciplines and sources, including theses, books, abstracts and articles.
• Dissertation Abstracts - dissertation and theses database - NIH Library biomedical librarians can access and search for you.
• NTIS  - central resource for government-funded scientific, technical, engineering, and business related information.
• AHRQ  - agency for healthcare research and quality
• Open Grey  - system for information on grey literature in Europe. Open access to 700,000 references to the grey literature.
• World Health Organization  - providing leadership on global health matters, shaping the health research agenda, setting norms and standards, articulating evidence-based policy options, providing technical support to countries and monitoring and assessing health trends.
• New York Academy of Medicine Grey Literature Report  - a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new gray literature publications in health services research and selected public health topics. NOTE: Discontinued as of Jan 2017, but resources are still accessible.
• Gray Source Index
• OpenDOAR - directory of academic repositories
• International Clinical Trials Registery Platform  - from the World Health Organization
• Australian New Zealand Clinical Trials Registry
• Brazilian Clinical Trials Registry
• Chinese Clinical Trial Registry - 
• ClinicalTrials.gov   - U.S.  and international federally and privately supported clinical trials registry and results database
• Clinical Trials Registry  - India
• EU clinical Trials Register
• Japan Primary Registries Network  
• Pan African Clinical Trials Registry

• Open access
• Published: 06 December 2017

Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study

• Wichor M. Bramer 1 ,
• Melissa L. Rethlefsen 2 ,
• Jos Kleijnen 3 , 4 &
• Oscar H. Franco 5  

Systematic Reviews volume  6 , Article number:  245 ( 2017 ) Cite this article

667 Citations

88 Altmetric

Metrics details

Within systematic reviews, when searching for relevant references, it is advisable to use multiple databases. However, searching databases is laborious and time-consuming, as syntax of search strategies are database specific. We aimed to determine the optimal combination of databases needed to conduct efficient searches in systematic reviews and whether the current practice in published reviews is appropriate. While previous studies determined the coverage of databases, we analyzed the actual retrieval from the original searches for systematic reviews.

Since May 2013, the first author prospectively recorded results from systematic review searches that he performed at his institution. PubMed was used to identify systematic reviews published using our search strategy results. For each published systematic review, we extracted the references of the included studies. Using the prospectively recorded results and the studies included in the publications, we calculated recall, precision, and number needed to read for single databases and databases in combination. We assessed the frequency at which databases and combinations would achieve varying levels of recall (i.e., 95%). For a sample of 200 recently published systematic reviews, we calculated how many had used enough databases to ensure 95% recall.

A total of 58 published systematic reviews were included, totaling 1746 relevant references identified by our database searches, while 84 included references had been retrieved by other search methods. Sixteen percent of the included references (291 articles) were only found in a single database; Embase produced the most unique references ( n  = 132). The combination of Embase, MEDLINE, Web of Science Core Collection, and Google Scholar performed best, achieving an overall recall of 98.3 and 100% recall in 72% of systematic reviews. We estimate that 60% of published systematic reviews do not retrieve 95% of all available relevant references as many fail to search important databases. Other specialized databases, such as CINAHL or PsycINFO, add unique references to some reviews where the topic of the review is related to the focus of the database.

Conclusions

Optimal searches in systematic reviews should search at least Embase, MEDLINE, Web of Science, and Google Scholar as a minimum requirement to guarantee adequate and efficient coverage.

Peer Review reports

Investigators and information specialists searching for relevant references for a systematic review (SR) are generally advised to search multiple databases and to use additional methods to be able to adequately identify all literature related to the topic of interest [ 1 , 2 , 3 , 4 , 5 , 6 ]. The Cochrane Handbook, for example, recommends the use of at least MEDLINE and Cochrane Central and, when available, Embase for identifying reports of randomized controlled trials [ 7 ]. There are disadvantages to using multiple databases. It is laborious for searchers to translate a search strategy into multiple interfaces and search syntaxes, as field codes and proximity operators differ between interfaces. Differences in thesaurus terms between databases add another significant burden for translation. Furthermore, it is time-consuming for reviewers who have to screen more, and likely irrelevant, titles and abstracts. Lastly, access to databases is often limited and only available on subscription basis.

Previous studies have investigated the added value of different databases on different topics [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Some concluded that searching only one database can be sufficient as searching other databases has no effect on the outcome [ 16 , 17 ]. Nevertheless others have concluded that a single database is not sufficient to retrieve all references for systematic reviews [ 18 , 19 ]. Most articles on this topic draw their conclusions based on the coverage of databases [ 14 ]. A recent paper tried to find an acceptable number needed to read for adding an additional database; sadly, however, no true conclusion could be drawn [ 20 ]. However, whether an article is present in a database may not translate to being found by a search in that database. Because of this major limitation, the question of which databases are necessary to retrieve all relevant references for a systematic review remains unanswered. Therefore, we research the probability that single or various combinations of databases retrieve the most relevant references in a systematic review by studying actual retrieval in various databases.

The aim of our research is to determine the combination of databases needed for systematic review searches to provide efficient results (i.e., to minimize the burden for the investigators without reducing the validity of the research by missing relevant references). A secondary aim is to investigate the current practice of databases searched for published reviews. Are included references being missed because the review authors failed to search a certain database?

Development of search strategies

At Erasmus MC, search strategies for systematic reviews are often designed via a librarian-mediated search service. The information specialists of Erasmus MC developed an efficient method that helps them perform searches in many databases in a much shorter time than other methods. This method of literature searching and a pragmatic evaluation thereof are published in separate journal articles [ 21 , 22 ]. In short, the method consists of an efficient way to combine thesaurus terms and title/abstract terms into a single line search strategy. This search is then optimized. Articles that are indexed with a set of identified thesaurus terms, but do not contain the current search terms in title or abstract, are screened to discover potential new terms. New candidate terms are added to the basic search and evaluated. Once optimal recall is achieved, macros are used to translate the search syntaxes between databases, though manual adaptation of the thesaurus terms is still necessary.

Review projects at Erasmus MC cover a wide range of medical topics, from therapeutic effectiveness and diagnostic accuracy to ethics and public health. In general, searches are developed in MEDLINE in Ovid (Ovid MEDLINE® In-Process & Other Non-Indexed Citations, Ovid MEDLINE® Daily and Ovid MEDLINE®, from 1946); Embase.com (searching both Embase and MEDLINE records, with full coverage including Embase Classic); the Cochrane Central Register of Controlled Trials (CENTRAL) via the Wiley Interface; Web of Science Core Collection (hereafter called Web of Science); PubMed restricting to records in the subset “as supplied by publisher” to find references that not yet indexed in MEDLINE (using the syntax publisher [sb]); and Google Scholar. In general, we use the first 200 references as sorted in the relevance ranking of Google Scholar. When the number of references from other databases was low, we expected the total number of potential relevant references to be low. In this case, the number of hits from Google Scholar was limited to 100. When the overall number of hits was low, we additionally searched Scopus, and when appropriate for the topic, we included CINAHL (EBSCOhost), PsycINFO (Ovid), and SportDiscus (EBSCOhost) in our search.

Beginning in May 2013, the number of records retrieved from each search for each database was recorded at the moment of searching. The complete results from all databases used for each of the systematic reviews were imported into a unique EndNote library upon search completion and saved without deduplication for this research. The researchers that requested the search received a deduplicated EndNote file from which they selected the references relevant for inclusion in their systematic review. All searches in this study were developed and executed by W.M.B.

Determining relevant references of published reviews

We searched PubMed in July 2016 for all reviews published since 2014 where first authors were affiliated to Erasmus MC, Rotterdam, the Netherlands, and matched those with search registrations performed by the medical library of Erasmus MC. This search was used in earlier research [ 21 ]. Published reviews were included if the search strategies and results had been documented at the time of the last update and if, at minimum, the databases Embase, MEDLINE, Cochrane CENTRAL, Web of Science, and Google Scholar had been used in the review. From the published journal article, we extracted the list of final included references. We documented the department of the first author. To categorize the types of patient/population and intervention, we identified broad MeSH terms relating to the most important disease and intervention discussed in the article. We copied from the MeSH tree the top MeSH term directly below the disease category or, in to case of the intervention, directly below the therapeutics MeSH term. We selected the domain from a pre-defined set of broad domains, including therapy, etiology, epidemiology, diagnosis, management, and prognosis. Lastly, we checked whether the reviews described limiting their included references to a particular study design.

To identify whether our searches had found the included references, and if so, from which database(s) that citation was retrieved, each included reference was located in the original corresponding EndNote library using the first author name combined with the publication year as a search term for each specific relevant publication. If this resulted in extraneous results, the search was subsequently limited using a distinct part of the title or a second author name. Based on the record numbers of the search results in EndNote, we determined from which database these references came. If an included reference was not found in the EndNote file, we presumed the authors used an alternative method of identifying the reference (e.g., examining cited references, contacting prominent authors, or searching gray literature), and we did not include it in our analysis.

Data analysis

We determined the databases that contributed most to the reviews by the number of unique references retrieved by each database used in the reviews. Unique references were included articles that had been found by only one database search. Those databases that contributed the most unique included references were then considered candidate databases to determine the most optimal combination of databases in the further analyses.

In Excel, we calculated the performance of each individual database and various combinations. Performance was measured using recall, precision, and number needed to read. See Table  1 for definitions of these measures. These values were calculated both for all reviews combined and per individual review.

Performance of a search can be expressed in different ways. Depending on the goal of the search, different measures may be optimized. In the case of a clinical question, precision is most important, as a practicing clinician does not have a lot of time to read through many articles in a clinical setting. When searching for a systematic review, recall is the most important aspect, as the researcher does not want to miss any relevant references. As our research is performed on systematic reviews, the main performance measure is recall.

We identified all included references that were uniquely identified by a single database. For the databases that retrieved the most unique included references, we calculated the number of references retrieved (after deduplication) and the number of included references that had been retrieved by all possible combinations of these databases, in total and per review. For all individual reviews, we determined the median recall, the minimum recall, and the percentage of reviews for which each single database or combination retrieved 100% recall.

For each review that we investigated, we determined what the recall was for all possible different database combinations of the most important databases. Based on these, we determined the percentage of reviews where that database combination had achieved 100% recall, more than 95%, more than 90%, and more than 80%. Based on the number of results per database both before and after deduplication as recorded at the time of searching, we calculated the ratio between the total number of results and the number of results for each database and combination.

Improvement of precision was calculated as the ratio between the original precision from the searches in all databases and the precision for each database and combination.

To compare our practice of database usage in systematic reviews against current practice as evidenced in the literature, we analyzed a set of 200 recent systematic reviews from PubMed. On 5 January 2017, we searched PubMed for articles with the phrase “systematic review” in the title. Starting with the most recent articles, we determined the databases searched either from the abstract or from the full text until we had data for 200 reviews. For the individual databases and combinations that were used in those reviews, we multiplied the frequency of occurrence in that set of 200 with the probability that the database or combination would lead to an acceptable recall (which we defined at 95%) that we had measured in our own data.

Our earlier research had resulted in 206 systematic reviews published between 2014 and July 2016, in which the first author was affiliated with Erasmus MC [ 21 ]. In 73 of these, the searches and results had been documented by the first author of this article at the time of the last search. Of those, 15 could not be included in this research, since they had not searched all databases we investigated here. Therefore, for this research, a total of 58 systematic reviews were analyzed. The references to these reviews can be found in Additional file 1 . An overview of the broad topical categories covered in these reviews is given in Table  2 . Many of the reviews were initiated by members of the departments of surgery and epidemiology. The reviews covered a wide variety of disease, none of which was present in more than 12% of the reviews. The interventions were mostly from the chemicals and drugs category, or surgical procedures. Over a third of the reviews were therapeutic, while slightly under a quarter answered an etiological question. Most reviews did not limit to certain study designs, 9% limited to RCTs only, and another 9% limited to other study types.

Together, these reviews included a total of 1830 references. Of these, 84 references (4.6%) had not been retrieved by our database searches and were not included in our analysis, leaving in total 1746 references. In our analyses, we combined the results from MEDLINE in Ovid and PubMed (the subset as supplied by publisher) into one database labeled MEDLINE.

Unique references per database

A total of 292 (17%) references were found by only one database. Table  3 displays the number of unique results retrieved for each single database. Embase retrieved the most unique included references, followed by MEDLINE, Web of Science, and Google Scholar. Cochrane CENTRAL is absent from the table, as for the five reviews limited to randomized trials, it did not add any unique included references. Subject-specific databases such as CINAHL, PsycINFO, and SportDiscus only retrieved additional included references when the topic of the review was directly related to their special content, respectively nursing, psychiatry, and sports medicine.

Overall performance

The four databases that had retrieved the most unique references (Embase, MEDLINE, Web of Science, and Google Scholar) were investigated individually and in all possible combinations (see Table  4 ). Of the individual databases, Embase had the highest overall recall (85.9%). Of the combinations of two databases, Embase and MEDLINE had the best results (92.8%). Embase and MEDLINE combined with either Google Scholar or Web of Science scored similarly well on overall recall (95.9%). However, the combination with Google Scholar had a higher precision and higher median recall, a higher minimum recall, and a higher proportion of reviews that retrieved all included references. Using both Web of Science and Google Scholar in addition to MEDLINE and Embase increased the overall recall to 98.3%. The higher recall from adding extra databases came at a cost in number needed to read (NNR). Searching only Embase produced an NNR of 57 on average, whereas, for the optimal combination of four databases, the NNR was 73.

Probability of appropriate recall

We calculated the recall for individual databases and databases in all possible combination for all reviews included in the research. Figure  1 shows the percentages of reviews where a certain database combination led to a certain recall. For example, in 48% of all systematic reviews, the combination of Embase and MEDLINE (with or without Cochrane CENTRAL; Cochrane CENTRAL did not add unique relevant references) reaches a recall of at least 95%. In 72% of studied systematic reviews, the combination of Embase, MEDLINE, Web of Science, and Google Scholar retrieved all included references. In the top bar, we present the results of the complete database searches relative to the total number of included references. This shows that many database searches missed relevant references.

Percentage of systematic reviews for which a certain database combination reached a certain recall. The X -axis represents the percentage of reviews for which a specific combination of databases, as shown on the y -axis, reached a certain recall (represented with bar colors). Abbreviations: EM Embase, ML MEDLINE, WoS Web of Science, GS Google Scholar. Asterisk indicates that the recall of all databases has been calculated over all included references. The recall of the database combinations was calculated over all included references retrieved by any database

Differences between domains of reviews

We analyzed whether the added value of Web of Science and Google Scholar was dependent of the domain of the review. For 55 reviews, we determined the domain. See Fig.  2 for the comparison of the recall of Embase, MEDLINE, and Cochrane CENTRAL per review for all identified domains. For all but one domain, the traditional combination of Embase, MEDLINE, and Cochrane CENTRAL did not retrieve enough included references. For four out of five systematic reviews that limited to randomized controlled trials (RCTs) only, the traditional combination retrieved 100% of all included references. However, for one review of this domain, the recall was 82%. Of the 11 references included in this review, one was found only in Google Scholar and one only in Web of Science.

Percentage of systematic reviews of a certain domain for which the combination Embase, MEDLINE and Cochrane CENTRAL reached a certain recall

Reduction in number of results

We calculated the ratio between the number of results found when searching all databases, including databases not included in our analyses, such as Scopus, PsycINFO, and CINAHL, and the number of results found searching a selection of databases. See Fig.  3 for the legend of the plots in Figs.  4 and 5 . Figure  4 shows the distribution of this value for individual reviews. The database combinations with the highest recall did not reduce the total number of results by large margins. Moreover, in combinations where the number of results was greatly reduced, the recall of included references was lower.

Legend of Figs. 3 and 4

The ratio between number of results per database combination and the total number of results for all databases

The ratio between precision per database combination and the total precision for all databases

Improvement of precision

To determine how searching multiple databases affected precision, we calculated for each combination the ratio between the original precision, observed when all databases were searched, and the precision calculated for different database combinations. Figure  5 shows the improvement of precision for 15 databases and database combinations. Because precision is defined as the number of relevant references divided by the number of total results, we see a strong correlation with the total number of results.

Status of current practice of database selection

From a set of 200 recent SRs identified via PubMed, we analyzed the databases that had been searched. Almost all reviews (97%) reported a search in MEDLINE. Other databases that we identified as essential for good recall were searched much less frequently; Embase was searched in 61% and Web of Science in 35%, and Google Scholar was only used in 10% of all reviews. For all individual databases or combinations of the four important databases from our research (MEDLINE, Embase, Web of Science, and Google Scholar), we multiplied the frequency of occurrence of that combination in the random set, with the probability we found in our research that this combination would lead to an acceptable recall of 95%. The calculation is shown in Table  5 . For example, around a third of the reviews (37%) relied on the combination of MEDLINE and Embase. Based on our findings, this combination achieves acceptable recall about half the time (47%). This implies that 17% of the reviews in the PubMed sample would have achieved an acceptable recall of 95%. The sum of all these values is the total probability of acceptable recall in the random sample. Based on these calculations, we estimate that the probability that this random set of reviews retrieved more than 95% of all possible included references was 40%. Using similar calculations, also shown in Table  5 , we estimated the probability that 100% of relevant references were retrieved is 23%.

Our study shows that, to reach maximum recall, searches in systematic reviews ought to include a combination of databases. To ensure adequate performance in searches (i.e., recall, precision, and number needed to read), we find that literature searches for a systematic review should, at minimum, be performed in the combination of the following four databases: Embase, MEDLINE (including Epub ahead of print), Web of Science Core Collection, and Google Scholar. Using that combination, 93% of the systematic reviews in our study obtained levels of recall that could be considered acceptable (> 95%). Unique results from specialized databases that closely match systematic review topics, such as PsycINFO for reviews in the fields of behavioral sciences and mental health or CINAHL for reviews on the topics of nursing or allied health, indicate that specialized databases should be used additionally when appropriate.

We find that Embase is critical for acceptable recall in a review and should always be searched for medically oriented systematic reviews. However, Embase is only accessible via a paid subscription, which generally makes it challenging for review teams not affiliated with academic medical centers to access. The highest scoring database combination without Embase is a combination of MEDLINE, Web of Science, and Google Scholar, but that reaches satisfactory recall for only 39% of all investigated systematic reviews, while still requiring a paid subscription to Web of Science. Of the five reviews that included only RCTs, four reached 100% recall if MEDLINE, Web of Science, and Google Scholar combined were complemented with Cochrane CENTRAL.

The Cochrane Handbook recommends searching MEDLINE, Cochrane CENTRAL, and Embase for systematic reviews of RCTs. For reviews in our study that included RCTs only, indeed, this recommendation was sufficient for four (80%) of the reviews. The one review where it was insufficient was about alternative medicine, specifically meditation and relaxation therapy, where one of the missed studies was published in the Indian Journal of Positive Psychology . The other study from the Journal of Advanced Nursing is indexed in MEDLINE and Embase but was only retrieved because of the addition of KeyWords Plus in Web of Science. We estimate more than 50% of reviews that include more study types than RCTs would miss more than 5% of included references if only traditional combination of MEDLINE, Embase, and Cochrane CENTAL is searched.

We are aware that the Cochrane Handbook [ 7 ] recommends more than only these databases, but further recommendations focus on regional and specialized databases. Though we occasionally used the regional databases LILACS and SciELO in our reviews, they did not provide unique references in our study. Subject-specific databases like PsycINFO only added unique references to a small percentage of systematic reviews when they had been used for the search. The third key database we identified in this research, Web of Science, is only mentioned as a citation index in the Cochrane Handbook, not as a bibliographic database. To our surprise, Cochrane CENTRAL did not identify any unique included studies that had not been retrieved by the other databases, not even for the five reviews focusing entirely on RCTs. If Erasmus MC authors had conducted more reviews that included only RCTs, Cochrane CENTRAL might have added more unique references.

MEDLINE did find unique references that had not been found in Embase, although our searches in Embase included all MEDLINE records. It is likely caused by difference in thesaurus terms that were added, but further analysis would be required to determine reasons for not finding the MEDLINE records in Embase. Although Embase covers MEDLINE, it apparently does not index every article from MEDLINE. Thirty-seven references were found in MEDLINE (Ovid) but were not available in Embase.com . These are mostly unique PubMed references, which are not assigned MeSH terms, and are often freely available via PubMed Central.

Google Scholar adds relevant articles not found in the other databases, possibly because it indexes the full text of all articles. It therefore finds articles in which the topic of research is not mentioned in title, abstract, or thesaurus terms, but where the concepts are only discussed in the full text. Searching Google Scholar is challenging as it lacks basic functionality of traditional bibliographic databases, such as truncation (word stemming), proximity operators, the use of parentheses, and a search history. Additionally, search strategies are limited to a maximum of 256 characters, which means that creating a thorough search strategy can be laborious.

Whether Embase and Web of Science can be replaced by Scopus remains uncertain. We have not yet gathered enough data to be able to make a full comparison between Embase and Scopus. In 23 reviews included in this research, Scopus was searched. In 12 reviews (52%), Scopus retrieved 100% of all included references retrieved by Embase or Web of Science. In the other 48%, the recall by Scopus was suboptimal, in one occasion as low as 38%.

Of all reviews in which we searched CINAHL and PsycINFO, respectively, for 6 and 9% of the reviews, unique references were found. For CINAHL and PsycINFO, in one case each, unique relevant references were found. In both these reviews, the topic was highly related to the topic of the database. Although we did not use these special topic databases in all of our reviews, given the low number of reviews where these databases added relevant references, and observing the special topics of those reviews, we suggest that these subject databases will only add value if the topic is related to the topic of the database.

Many articles written on this topic have calculated overall recall of several reviews, instead of the effects on all individual reviews. Researchers planning a systematic review generally perform one review, and they need to estimate the probability that they may miss relevant articles in their search. When looking at the overall recall, the combination of Embase and MEDLINE and either Google Scholar or Web of Science could be regarded sufficient with 96% recall. This number however is not an answer to the question of a researcher performing a systematic review, regarding which databases should be searched. A researcher wants to be able to estimate the chances that his or her current project will miss a relevant reference. However, when looking at individual reviews, the probability of missing more than 5% of included references found through database searching is 33% when Google Scholar is used together with Embase and MEDLINE and 30% for the Web of Science, Embase, and MEDLINE combination. What is considered acceptable recall for systematic review searches is open for debate and can differ between individuals and groups. Some reviewers might accept a potential loss of 5% of relevant references; others would want to pursue 100% recall, no matter what cost. Using the results in this research, review teams can decide, based on their idea of acceptable recall and the desired probability which databases to include in their searches.

Strengths and limitations

We did not investigate whether the loss of certain references had resulted in changes to the conclusion of the reviews. Of course, the loss of a minor non-randomized included study that follows the systematic review’s conclusions would not be as problematic as losing a major included randomized controlled trial with contradictory results. However, the wide range of scope, topic, and criteria between systematic reviews and their related review types make it very hard to answer this question.

We found that two databases previously not recommended as essential for systematic review searching, Web of Science and Google Scholar, were key to improving recall in the reviews we investigated. Because this is a novel finding, we cannot conclude whether it is due to our dataset or to a generalizable principle. It is likely that topical differences in systematic reviews may impact whether databases such as Web of Science and Google Scholar add value to the review. One explanation for our finding may be that if the research question is very specific, the topic of research might not always be mentioned in the title and/or abstract. In that case, Google Scholar might add value by searching the full text of articles. If the research question is more interdisciplinary, a broader science database such as Web of Science is likely to add value. The topics of the reviews studied here may simply have fallen into those categories, though the diversity of the included reviews may point to a more universal applicability.

Although we searched PubMed as supplied by publisher separately from MEDLINE in Ovid, we combined the included references of these databases into one measurement in our analysis. Until 2016, the most complete MEDLINE selection in Ovid still lacked the electronic publications that were already available in PubMed. These could be retrieved by searching PubMed with the subset as supplied by publisher. Since the introduction of the more complete MEDLINE collection Epub Ahead of Print , In-Process & Other Non-Indexed Citations , and Ovid MEDLINE® , the need to separately search PubMed as supplied by publisher has disappeared. According to our data, PubMed’s “as supplied by publisher” subset retrieved 12 unique included references, and it was the most important addition in terms of relevant references to the four major databases. It is therefore important to search MEDLINE including the “Epub Ahead of Print, In-Process, and Other Non-Indexed Citations” references.

These results may not be generalizable to other studies for other reasons. The skills and experience of the searcher are one of the most important aspects in the effectiveness of systematic review search strategies [ 23 , 24 , 25 ]. The searcher in the case of all 58 systematic reviews is an experienced biomedical information specialist. Though we suspect that searchers who are not information specialists or librarians would have a higher possibility of less well-constructed searches and searches with lower recall, even highly trained searchers differ in their approaches to searching. For this study, we searched to achieve as high a recall as possible, though our search strategies, like any other search strategy, still missed some relevant references because relevant terms had not been used in the search. We are not implying that a combined search of the four recommended databases will never result in relevant references being missed, rather that failure to search any one of these four databases will likely lead to relevant references being missed. Our experience in this study shows that additional efforts, such as hand searching, reference checking, and contacting key players, should be made to retrieve extra possible includes.

Based on our calculations made by looking at random systematic reviews in PubMed, we estimate that 60% of these reviews are likely to have missed more than 5% of relevant references only because of the combinations of databases that were used. That is with the generous assumption that the searches in those databases had been designed sensitively enough. Even when taking into account that many searchers consider the use of Scopus as a replacement of Embase, plus taking into account the large overlap of Scopus and Web of Science, this estimate remains similar. Also, while the Scopus and Web of Science assumptions we made might be true for coverage, they are likely very different when looking at recall, as Scopus does not allow the use of the full features of a thesaurus. We see that reviewers rarely use Web of Science and especially Google Scholar in their searches, though they retrieve a great deal of unique references in our reviews. Systematic review searchers should consider using these databases if they are available to them, and if their institution lacks availability, they should ask other institutes to cooperate on their systematic review searches.

The major strength of our paper is that it is the first large-scale study we know of to assess database performance for systematic reviews using prospectively collected data. Prior research on database importance for systematic reviews has looked primarily at whether included references could have theoretically been found in a certain database, but most have been unable to ascertain whether the researchers actually found the articles in those databases [ 10 , 12 , 16 , 17 , 26 ]. Whether a reference is available in a database is important, but whether the article can be found in a precise search with reasonable recall is not only impacted by the database’s coverage. Our experience has shown us that it is also impacted by the ability of the searcher, the accuracy of indexing of the database, and the complexity of terminology in a particular field. Because these studies based on retrospective analysis of database coverage do not account for the searchers’ abilities, the actual findings from the searches performed, and the indexing for particular articles, their conclusions lack immediate translatability into practice. This research goes beyond retrospectively assessed coverage to investigate real search performance in databases. Many of the articles reporting on previous research concluded that one database was able to retrieve most included references. Halladay et al. [ 10 ] and van Enst et al. [ 16 ] concluded that databases other than MEDLINE/PubMed did not change the outcomes of the review, while Rice et al. [ 17 ] found the added value of other databases only for newer, non-indexed references. In addition, Michaleff et al. [ 26 ] found that Cochrane CENTRAL included 95% of all RCTs included in the reviews investigated. Our conclusion that Web of Science and Google Scholar are needed for completeness has not been shared by previous research. Most of the previous studies did not include these two databases in their research.

We recommend that, regardless of their topic, searches for biomedical systematic reviews should combine Embase, MEDLINE (including electronic publications ahead of print), Web of Science (Core Collection), and Google Scholar (the 200 first relevant references) at minimum. Special topics databases such as CINAHL and PsycINFO should be added if the topic of the review directly touches the primary focus of a specialized subject database, like CINAHL for focus on nursing and allied health or PsycINFO for behavioral sciences and mental health. For reviews where RCTs are the desired study design, Cochrane CENTRAL may be similarly useful. Ignoring one or more of the databases that we identified as the four key databases will result in more precise searches with a lower number of results, but the researchers should decide whether that is worth the >increased probability of losing relevant references. This study also highlights once more that searching databases alone is, nevertheless, not enough to retrieve all relevant references.

Future research should continue to investigate recall of actual searches beyond coverage of databases and should consider focusing on the most optimal database combinations, not on single databases.

Levay P, Raynor M, Tuvey D. The contributions of MEDLINE, other bibliographic databases and various search techniques to NICE public health guidance. Evid Based Libr Inf Pract. 2015;10:50–68.

Article   Google Scholar  

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complement Ther Med. 2004;12:228–32.

Article   CAS   PubMed   Google Scholar  

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Inj Prev. 2008;14:401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58:867–73.

Article   PubMed   Google Scholar  

Zheng MH, Zhang X, Ye Q, Chen YP. Searching additional databases except PubMed are necessary for a systematic review. Stroke. 2008;39:e139. author reply e140

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Inf Libr J. 2013;30:49–58.

Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions: The Cochrane Collaboration, London, United Kingdom. 2011.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Article   PubMed   PubMed Central   Google Scholar  

Wilkins T, Gillies RA, Davies K. EMBASE versus MEDLINE for family medicine searches: can MEDLINE searches find the forest or a tree? Can Fam Physician. 2005;51:848–9.

PubMed   Google Scholar  

Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68:1076–84.

Ahmadi M, Ershad-Sarabi R, Jamshidiorak R, Bahaodini K. Comparison of bibliographic databases in retrieving information on telemedicine. J Kerman Univ Med Sci. 2014;21:343–54.

Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than MEDLINE for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30:173–8.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, Alderson P. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66:1051–7.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16:1–13.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews—are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16:161.

van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol. 2014;67:1192–9.

Rice DB, Kloda LA, Levis B, Qi B, Kingsland E, Thombs BD. Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. J Psychosom Res. 2016;87:7–13.

Bramer WM, Giustini D, Kramer BM, Anderson PF. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst Rev. 2013;2:115.

Bramer WM, Giustini D, Kramer BMR. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst Rev. 2016;5:39.

Ross-White A, Godfrey C. Is there an optimum number needed to retrieve to justify inclusion of a database in a systematic review search? Health Inf Libr J. 2017;33:217–24.

Bramer WM, Rethlefsen ML, Mast F, Kleijnen J. A pragmatic evaluation of a new method for librarian-mediated literature searches for systematic reviews. Res Synth Methods. 2017. doi: 10.1002/jrsm.1279 .

Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: how to perform high quality literature searches more efficiently. J Med Libr Assoc. 2018.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68:617–26.

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93:74–80.

PubMed   PubMed Central   Google Scholar  

McKibbon KA, Haynes RB, Dilks CJW, Ramsden MF, Ryan NC, Baker L, Flemming T, Fitzgerald D. How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. Comput Biomed Res. 1990;23:583–93.

Michaleff ZA, Costa LO, Moseley AM, Maher CG, Elkins MR, Herbert RD, Sherrington C. CENTRAL, PEDro, PubMed, and EMBASE are the most comprehensive databases indexing randomized controlled trials of physical therapy interventions. Phys Ther. 2011;91:190–7.

Download references

Acknowledgements

Not applicable

Melissa Rethlefsen receives funding in part from the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR001067. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available from the corresponding author on a reasonable request.

Author information

Authors and affiliations.

Medical Library, Erasmus MC, Erasmus University Medical Centre Rotterdam, 3000 CS, Rotterdam, the Netherlands

Wichor M. Bramer

Spencer S. Eccles Health Sciences Library, University of Utah, Salt Lake City, Utah, USA

Melissa L. Rethlefsen

Kleijnen Systematic Reviews Ltd., York, UK

Jos Kleijnen

School for Public Health and Primary Care (CAPHRI), Maastricht University, Maastricht, the Netherlands

Department of Epidemiology, Erasmus MC, Erasmus University Medical Centre Rotterdam, Rotterdam, the Netherlands

Oscar H. Franco

You can also search for this author in PubMed   Google Scholar

Contributions

WB, JK, and OF designed the study. WB designed the searches used in this study and gathered the data. WB and ML analyzed the data. WB drafted the first manuscript, which was revised critically by the other authors. All authors have approved the final manuscript.

Corresponding author

Correspondence to Wichor M. Bramer .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

WB has received travel allowance from Embase for giving a presentation at a conference. The other authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:.

Reviews included in the research . References to the systematic reviews published by Erasmus MC authors that were included in the research. (DOCX 19 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Bramer, W.M., Rethlefsen, M.L., Kleijnen, J. et al. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 6 , 245 (2017). https://doi.org/10.1186/s13643-017-0644-y

Download citation

Received : 21 August 2017

Accepted : 24 November 2017

Published : 06 December 2017

DOI : https://doi.org/10.1186/s13643-017-0644-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Databases, bibliographic
• Review literature as topic
• Sensitivity and specificity
• Information storage and retrieval

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

How to undertake a literature search: a step-by-step guide

Affiliation.

• 1 Literature Search Specialist, Library and Archive Service, Royal College of Nursing, London.
• PMID: 32279549
• DOI: 10.12968/bjon.2020.29.7.431

Undertaking a literature search can be a daunting prospect. Breaking the exercise down into smaller steps will make the process more manageable. This article suggests 10 steps that will help readers complete this task, from identifying key concepts to choosing databases for the search and saving the results and search strategy. It discusses each of the steps in a little more detail, with examples and suggestions on where to get help. This structured approach will help readers obtain a more focused set of results and, ultimately, save time and effort.

Keywords: Databases; Literature review; Literature search; Reference management software; Research questions; Search strategy.

• Databases, Bibliographic*
• Information Storage and Retrieval / methods*
• Nursing Research
• Review Literature as Topic*

Systematic Reviews: Medical Literature Databases to search

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

How to document your literature search

You should always  document how you have searched each database, what keywords or index terms were used, the date on which the search was performed, how many results you retrieved, and if you use RefWorks to deduplicate results record how many were removed as duplicates and the final number of discrete studies you subjected to your first sift through of study selection.  Here is an example of how to document a literature search on an Excel spreadsheet , this example records a search of the hematology literature for articles about sickle cell disease. Here is another example of  how to document a literature search, this time on one page of a Word document , this example records a search of the medical literature for a poster on Emergency Department throughput.  The numbers recorded can then be used to populate the PRISMA flow diagram summarizing the literature search.

In the final report add as an appendix the full electronic search strategy for each database searched for the literature review e.g. MEDLINE with MeSH terms, keywords & limits

In the final report in the methods section:

PRISMA checklist Item 7 information sources will be reported as:

• What databases/websites you searched, the name of the database search platform and the start/end dates the index covers if relevant e.g. OVID MEDLINE (1950-present, or just PubMed
• Who developed & conducted the searches
• Date each database/website was last searched
• Supplementary sources - what other websites did you search? What journal titles were hand searched, whether reference lists were checked, what trial registries or regulatory agency websites were searched, were manufacturers or other authors contacted to obtain unpublished or missing information on study methods or results.

PRISMA checklist Item 8 search will be reported as:

• In text: describe the principal keywords used to search databases, websites & trials registers

What databases/indexes should you search?

At a minimum you need to search MEDLINE ,  EMBASE , and the  Cochrane CENTRAL  trials register .  This is the recommendation of three medical and public health research organizations: the U.S.  Agency for Healthcare Research and Quality ( AHRQ ), the U.K. Centre for Reviews and Dissemination ( CRD ), and the International Cochrane Collaboration (Source:  Institute of Medicine (2011) Finding What Works in Healthcare: Standards for Systematic Reviews  Table E-1, page 267).  Some databases have an alternate version, linked in parentheses below, that search the same records sets, ie the content of MEDLINE is in PubMed and Scopus, while the content of EMBASE is in Scopus. You should reformat your search for each database as appropriate, contact your librarian if you want help on how to search each database.  

Begin by searching:

1.        MEDLINE  (or  PubMed )

2.       EMBASE (or  Scopus )  Please note Himmelfarb Library does not have a subscription to EMBASE. The content is in the Scopus  database that you can search using keywords, but it is not possible to perform an EMTREE theasaurus search in Scopus.

3.        Cochrane Central Trials Register  (or  Cochrane Library ). In addition Cochrane researchers recommend you search the clinicaltrials.gov and ICTRP clinical trial registries due to the low sensitivity of the Cochrane CENTRAL index because according to Hunter et al (2022) "register records as they appear in CENTRAL are less comprehensive than the original register entry, and thus are at a greater risk than other systems of being missed in a search."

The Polyglot Search Translator is a very useful tool for translating search strings from PubMed or Medline via Ovid across multiple databases, developed by the Institute for Evidence-Based Healthcare at Bond University. But please note Polyglot does not automatically map subject terms across databases (e.g. MeSH terms to Emtree terms) so you will need to manually edit the search syntax in a text editor to change to the actual subject terms used by another database.

The Yale Mesh Analyzer is another very useful tool you can copy and paste in a list of up to 20 PMID numbers for records in the PubMed database, the Yale Mesh Analyzer will then display the Mesh Medical Subject Headings for those 20 articles as a table so you can identify and compare what Mesh headings they have in common, this can suggest additional search terms for your PubMed search.

The MedSyntax tool is another useful tool, for parsing out very long searches with many levels of brackets. This would be useful if you are trying to edit a pre-existing search strategy with many levels of parentheses.

Some sources for pre-existing database search filters or "hedges" include:

• CADTH Search Filters Database ,
• McMaster University Health Information Research Unit ,
• University of York Centre for Reviews and Dissemination InterTASC Information Specialists' Sub-Group ,
• InterTASC Population Specific search filters  (particularly useful for identifying Latinx, Indigenous people's, LGBTQ, Black & Minority ethnic)
• CareSearch Palliative Care PubMed search filters  (bereavement, dementia, heart failure, lung cancer, cost of care, and Palliative Care)
• Low and Middle Income countries filter at https://epoc.cochrane.org/lmic-filters . 
• Search Pubmed for another validated search filter using some variation of a search like this, possibly adding your discipline or search topic keywords: ("Databases, Bibliographic"[Mesh] OR "Search Engine"[Mesh]) AND ("Reproducibility of Results"[Mesh] OR "Sensitivity and Specificity"[Mesh] OR validat*) AND (filter OR hedge) .
• Search MEDLINE (or PubMed), preferably using a peer reviewed search strategy per protocol and apply any relevant methodology filters.
• Search EMBASE (or Scopus) and the Cochrane Central trials register using appropriately reformatted search versions for those databases, and any other online resources. 
• You should also search other subject specific databases that index the literature in your field.  Use our Himmelfarb Library  research guides  to identify other  subject specific databases . 
• Save citations in Covidence to deduplicate citations prior to screening.
• After screening export citations to  RefWorks database when you are ready to write up your manuscript. The Covidence and Refworks databases should be shared with all members of the investigative team.

Supplementary resources to search

Other member of your investigative team may have ideas about databases, websites, and journals they think you should search. Searching these sources is not required to perform a systematic review. You may need to reformat your search keywords.

Researchers at GW should check our subject research guides for suggestions, or check the libguides community for a guide on your subject.

In addition you may wish to search one or more of the following resources:

• Google Scholar
• BASE  academic search engine is useful for searching in University Institutional Repositories
• Cochrane Database of Systematic Reviews  to search for a pre-existing systematic review on your topic
• Epistemonikos database, has a matrix of evidence table so you can see what citations are shared in common across existing systematic reviews of the same topic. This feature might help identify sentinel or 'don't miss' articles.

You might also consider searching one or more of the following websites depending on your topic:

Clinical trial registers. The Cochrane Collaboration recommends for a systematic review to search both clinicaltrials.gov and the WHO ICTRP (See http://handbook.cochrane.org/ section 4.3):

• ClinicalTrials.gov  - also contains study population characteristics and results data of FDA regulated drugs and medical devices in NIH funded studies produced after January 18, 2017.
• WHO ICTRP  - trials register
• TRIP  - searchable index of clinical trials, guidelines,and regulatory guidance
• CenterWatch
• Current Controlled Trials
• European Clinical Trials Register
• ISRCTN Register
• COMPARE - tracks outcome switching in clinical trials
• OpenTrials - aims to match published trials with the underlying data where this is publicly available in an open source 
• ECRI Guidelines Trust

Grey literature resources:

• WONDER - CDC data and reports
• FDSys - search federal government publications
• Science.gov
• NRR Archive
• NIH Reporter
• re3data registry of data repositories
• Data Repositories (listed by the Simmons Open Access Directory)
• OpenDOAR  search academic open access research repositories
• f1000research search open access repositories of articles, slides, and research posters, in the life sciences, public health, education, and communication.
• RAND Health Reports
• National Academy of Medicine Publications
• Kaiser Family Foundation 
• Robert Wood Johnson Foundation health and medical care data archive
• Milbank Memorial Fund reports and issue briefs
• Also search the resources listed in the CADTH (2019) Grey Matters checklist.

Preprints 

• See our Himmelfarb preprints guide page on finding preprints , a useful database for searching Health Sciences preprints is  Europe PMC

Dissertations and Theses:

• Proquest Dissertations and Theses Online 
• Networked Digital Library of Theses and Dissertations
• Open Access Theses and Dissertations
• WorldCat and change Content: from Any Content to Thesis/dissertations

Conference proceedings:

Most conference proceedings are difficult to find because they may or may not be published. Only select individual papers may be made available in print as a book, journal, or series, rather than all of the presented items. Societies and Associations may only publish abstracts, or extended abstracts, from a conference, often in an annual supplement to an issue of the journal of record of that professional society.  Often posters are not published, if they are they may be made available only to other conference registrants at that meeting or online. Authors may "publish" their conference papers or posters on personal or institutional websites.  A limited set of conference proceedings databases include the following:

• BASE  academic search engine, has an Advanced Search feature with a Limit by Type to 'Conference Objects', this is useful for searching for conference posters and submissions stored in University Institutional Repositories.
• Web of Science - click All Databases and select Core Collection - under More Settings limit to the Conference Proceedings Citation Index (CPCI) - searches a limited set of conferences on Science, Social Science and Humanities from 1990-present.
• Scopus - Limit Document Type to Conference Paper or Conference Review.
• Proquest  - Limit search results to conference papers &/or proceedings under Advanced Search.
• BioMed Central Proceedings  - searches a limited set of biomedical conference proceedings, including bioinformatics, genetics, medical students, and data visualization.
• F1000 Research - browse by subject and click the tabs for articles, posters, and slides - which searches a limited number of biology and medical society meetings/conferences. This is a voluntary self-archive repository.

Individual Journals 

• You may choose to "hand search" select journals where the research team reads the Table of Contents of each issue for a chosen period of time.  You can look for the names of high impact journal titles in a particular field indexed in Journal Citation Reports  (JCR). Please note as of August 2021 ISI are linking to a new version of JCR that currently does not have the particularly helpful 'Browse by Category' link working, so I recommend you click the Products link in the top right corner and select Journal Citation Reports (Classic) to switch back to the old version to get that functionality back.
• The AllTrials petition aims to motivate health care researchers to petition regulators and research bodies to require the results and data of all clinical trials be published.
• << Previous: Search strategy
• Next: Study selection and appraisal >>

• Last Updated: May 8, 2024 11:07 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

Help us improve our Library guides with this 5 minute survey . We appreciate your feedback!

• UOW Library
• Key guides for students

Literature Review

Where to search when doing a literature review.

• Find examples of literature reviews
• How to write a literature review
• How to search effectively
• Grey literature

Aim to be as comprehensive as possible when conducting a literature review. Knowing exactly where to search for information is important.

Work through the steps to find out the best databases to search for information on your research topic.

1. Start with research databases

Scopus and Web of Science are good databases to start with for any research topic and literature review.

• Scopus Scopus is a large multidisciplinary database covering published material in the humanities and sciences. It also provides citation analysis of authors and subject areas. Searching Scopus tutorial - Includes access to Scival via expanded top menu (Elsevier personal registration required).
• Web of Science - Core Collection The leading citation index' of scholarly literature, chemical reactions and author information. Includes citation databases: Sciences Expanded (1965+), Social Sciences (1965+), Arts & Humanities (1975+). Conference Proceedings (1990+), Emerging Sources Citation (2005+) , Current Chemical Reactions (1985+) and Index Chemicus (1993+) Access InCites benchmarking & analytics tools via the menu bar at the top of the screen.

2. Focus your search with specific databases

Select two or three discipline/specialist databases to conduct your search for comprehensive results.

Our subject guides will help you find databases relevant to major subject areas in each discipline and specific materials relevant to your research.

• Discipline subject guides
• News sources

3. Find books, theses and more

If you're looking for a specific medium (book, thesis, journal, etc.) for your research, try the following:

• Finding Theses Help finding theses at UOW, Australia and around the world and how to access them
• Previous: How to search effectively
• Next: Grey literature
• Last Updated: May 28, 2024 9:42 AM
• URL: https://uow.libguides.com/literaturereview

Insert research help text here

LIBRARY RESOURCES

Library homepage

Library SEARCH

A-Z Databases

STUDY SUPPORT

Academic Skills Centre

Referencing and citing

Digital Skills Hub

MORE UOW SERVICES

UOW homepage

Student support and wellbeing

IT Services

On the lands that we study, we walk, and we live, we acknowledge and respect the traditional custodians and cultural knowledge holders of these lands.

Copyright & disclaimer | Privacy & cookie usage

• Open access
• Published: 14 August 2018

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

• Chris Cooper   ORCID: orcid.org/0000-0003-0864-5607 1 ,
• Andrew Booth 2 ,
• Jo Varley-Campbell 1 ,
• Nicky Britten 3 &
• Ruth Garside 4  

BMC Medical Research Methodology volume  18 , Article number:  85 ( 2018 ) Cite this article

202 Citations

117 Altmetric

Metrics details

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before.

The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

A literature review.

Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through ‘pearl growing’, citation chasing, a search of PubMed using the systematic review methods filter, and the authors’ topic knowledge.

The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of ‘key stages’ in the process of literature searching.

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

Conclusions

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Peer Review reports

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving review stakeholders clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. This is in contrast to the information science literature, which has developed information processing models as an explicit basis for dialogue and empirical testing. Without an explicit model, research in the process of systematic literature searching will remain immature and potentially uneven, and the development of shared information models will be assumed but never articulated.

One way of developing such a conceptual model is by formally examining the implicit “programme theory” as embodied in key methodological texts. The aim of this review is therefore to determine if a shared model of the literature searching process in systematic reviews can be detected across guidance documents and, if so, how this process is reported and supported.

Identifying guidance

Key texts (henceforth referred to as “guidance”) were identified based upon their accessibility to, and prominence within, United Kingdom systematic reviewing practice. The United Kingdom occupies a prominent position in the science of health information retrieval, as quantified by such objective measures as the authorship of papers, the number of Cochrane groups based in the UK, membership and leadership of groups such as the Cochrane Information Retrieval Methods Group, the HTA-I Information Specialists’ Group and historic association with such centres as the UK Cochrane Centre, the NHS Centre for Reviews and Dissemination, the Centre for Evidence Based Medicine and the National Institute for Clinical Excellence (NICE). Coupled with the linguistic dominance of English within medical and health science and the science of systematic reviews more generally, this offers a justification for a purposive sample that favours UK, European and Australian guidance documents.

Nine guidance documents were identified. These documents provide guidance for different types of reviews, namely: reviews of interventions, reviews of health technologies, reviews of qualitative research studies, reviews of social science topics, and reviews to inform guidance.

Whilst these guidance documents occasionally offer additional guidance on other types of systematic reviews, we have focused on the core and stated aims of these documents as they relate to literature searching. Table  1 sets out: the guidance document, the version audited, their core stated focus, and a bibliographical pointer to the main guidance relating to literature searching.

Once a list of key guidance documents was determined, it was checked by six senior information professionals based in the UK for relevance to current literature searching in systematic reviews.

Identifying supporting studies

In addition to identifying guidance, the authors sought to populate an evidence base of supporting studies (henceforth referred to as “studies”) that contribute to existing search practice. Studies were first identified by the authors from their knowledge on this topic area and, subsequently, through systematic citation chasing key studies (‘pearls’ [ 1 ]) located within each key stage of the search process. These studies are identified in Additional file  1 : Appendix Table 1. Citation chasing was conducted by analysing the bibliography of references for each study (backwards citation chasing) and through Google Scholar (forward citation chasing). A search of PubMed using the systematic review methods filter was undertaken in August 2017 (see Additional file 1 ). The search terms used were: (literature search*[Title/Abstract]) AND sysrev_methods[sb] and 586 results were returned. These results were sifted for relevance to the key stages in Fig.  1 by CC.

The key stages of literature search guidance as identified from nine key texts

Extracting the data

To reveal the implicit process of literature searching within each guidance document, the relevant sections (chapters) on literature searching were read and re-read, with the aim of determining key methodological stages. We defined a key methodological stage as a distinct step in the overall process for which specific guidance is reported, and action is taken, that collectively would result in a completed literature search.

The chapter or section sub-heading for each methodological stage was extracted into a table using the exact language as reported in each guidance document. The lead author (CC) then read and re-read these data, and the paragraphs of the document to which the headings referred, summarising section details. This table was then reviewed, using comparison and contrast to identify agreements and areas of unique guidance. Consensus across multiple guidelines was used to inform selection of ‘key stages’ in the process of literature searching.

Having determined the key stages to literature searching, we then read and re-read the sections relating to literature searching again, extracting specific detail relating to the methodological process of literature searching within each key stage. Again, the guidance was then read and re-read, first on a document-by-document-basis and, secondly, across all the documents above, to identify both commonalities and areas of unique guidance.

Results and discussion

Our findings.

We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community. Whilst the structure of the guidance varies between documents, the same key stages are reported, even where the core focus of each document is different. We were able to identify specific areas of unique guidance, where a document reported guidance not summarised in other documents, together with areas of consensus across guidance.

Unique guidance

Only one document provided guidance on the topic of when to stop searching [ 2 ]. This guidance from 2005 anticipates a topic of increasing importance with the current interest in time-limited (i.e. “rapid”) reviews. Quality assurance (or peer review) of literature searches was only covered in two guidance documents [ 3 , 4 ]. This topic has emerged as increasingly important as indicated by the development of the PRESS instrument [ 5 ]. Text mining was discussed in four guidance documents [ 4 , 6 , 7 , 8 ] where the automation of some manual review work may offer efficiencies in literature searching [ 8 ].

Agreement between guidance: Defining the key stages of literature searching

Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews.

From the guidance, we determined eight key stages that relate specifically to literature searching in systematic reviews. These are summarised at Fig. 1 . The data extraction table to inform Fig. 1 is reported in Table  2 . Table 2 reports the areas of common agreement and it demonstrates that the language used to describe key stages and processes varies significantly between guidance documents.

For each key stage, we set out the specific guidance, followed by discussion on how this guidance is situated within the wider literature.

Key stage one: Deciding who should undertake the literature search

The guidance.

Eight documents provided guidance on who should undertake literature searching in systematic reviews [ 2 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ]. The guidance affirms that people with relevant expertise of literature searching should ‘ideally’ be included within the review team [ 6 ]. Information specialists (or information scientists), librarians or trial search co-ordinators (TSCs) are indicated as appropriate researchers in six guidance documents [ 2 , 7 , 8 , 9 , 10 , 11 ].

How the guidance corresponds to the published studies

The guidance is consistent with studies that call for the involvement of information specialists and librarians in systematic reviews [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] and which demonstrate how their training as ‘expert searchers’ and ‘analysers and organisers of data’ can be put to good use [ 13 ] in a variety of roles [ 12 , 16 , 20 , 21 , 24 , 25 , 26 ]. These arguments make sense in the context of the aims and purposes of literature searching in systematic reviews, explored below. The need for ‘thorough’ and ‘replicable’ literature searches was fundamental to the guidance and recurs in key stage two. Studies have found poor reporting, and a lack of replicable literature searches, to be a weakness in systematic reviews [ 17 , 18 , 27 , 28 ] and they argue that involvement of information specialists/ librarians would be associated with better reporting and better quality literature searching. Indeed, Meert et al. [ 29 ] demonstrated that involving a librarian as a co-author to a systematic review correlated with a higher score in the literature searching component of a systematic review [ 29 ]. As ‘new styles’ of rapid and scoping reviews emerge, where decisions on how to search are more iterative and creative, a clear role is made here too [ 30 ].

Knowing where to search for studies was noted as important in the guidance, with no agreement as to the appropriate number of databases to be searched [ 2 , 6 ]. Database (and resource selection more broadly) is acknowledged as a relevant key skill of information specialists and librarians [ 12 , 15 , 16 , 31 ].

Whilst arguments for including information specialists and librarians in the process of systematic review might be considered self-evident, Koffel and Rethlefsen [ 31 ] have questioned if the necessary involvement is actually happening [ 31 ].

Key stage two: Determining the aim and purpose of a literature search

The aim: Five of the nine guidance documents use adjectives such as ‘thorough’, ‘comprehensive’, ‘transparent’ and ‘reproducible’ to define the aim of literature searching [ 6 , 7 , 8 , 9 , 10 ]. Analogous phrases were present in a further three guidance documents, namely: ‘to identify the best available evidence’ [ 4 ] or ‘the aim of the literature search is not to retrieve everything. It is to retrieve everything of relevance’ [ 2 ] or ‘A systematic literature search aims to identify all publications relevant to the particular research question’ [ 3 ]. The Joanna Briggs Institute reviewers’ manual was the only guidance document where a clear statement on the aim of literature searching could not be identified. The purpose of literature searching was defined in three guidance documents, namely to minimise bias in the resultant review [ 6 , 8 , 10 ]. Accordingly, eight of nine documents clearly asserted that thorough and comprehensive literature searches are required as a potential mechanism for minimising bias.

The need for thorough and comprehensive literature searches appears as uniform within the eight guidance documents that describe approaches to literature searching in systematic reviews of effectiveness. Reviews of effectiveness (of intervention or cost), accuracy and prognosis, require thorough and comprehensive literature searches to transparently produce a reliable estimate of intervention effect. The belief that all relevant studies have been ‘comprehensively’ identified, and that this process has been ‘transparently’ reported, increases confidence in the estimate of effect and the conclusions that can be drawn [ 32 ]. The supporting literature exploring the need for comprehensive literature searches focuses almost exclusively on reviews of intervention effectiveness and meta-analysis. Different ‘styles’ of review may have different standards however; the alternative, offered by purposive sampling, has been suggested in the specific context of qualitative evidence syntheses [ 33 ].

What is a comprehensive literature search?

Whilst the guidance calls for thorough and comprehensive literature searches, it lacks clarity on what constitutes a thorough and comprehensive literature search, beyond the implication that all of the literature search methods in Table 2 should be used to identify studies. Egger et al. [ 34 ], in an empirical study evaluating the importance of comprehensive literature searches for trials in systematic reviews, defined a comprehensive search for trials as:

a search not restricted to English language;

where Cochrane CENTRAL or at least two other electronic databases had been searched (such as MEDLINE or EMBASE); and

at least one of the following search methods has been used to identify unpublished trials: searches for (I) conference abstracts, (ii) theses, (iii) trials registers; and (iv) contacts with experts in the field [ 34 ].

Tricco et al. (2008) used a similar threshold of bibliographic database searching AND a supplementary search method in a review when examining the risk of bias in systematic reviews. Their criteria were: one database (limited using the Cochrane Highly Sensitive Search Strategy (HSSS)) and handsearching [ 35 ].

Together with the guidance, this would suggest that comprehensive literature searching requires the use of BOTH bibliographic database searching AND supplementary search methods.

Comprehensiveness in literature searching, in the sense of how much searching should be undertaken, remains unclear. Egger et al. recommend that ‘investigators should consider the type of literature search and degree of comprehension that is appropriate for the review in question, taking into account budget and time constraints’ [ 34 ]. This view tallies with the Cochrane Handbook, which stipulates clearly, that study identification should be undertaken ‘within resource limits’ [ 9 ]. This would suggest that the limitations to comprehension are recognised but it raises questions on how this is decided and reported [ 36 ].

What is the point of comprehensive literature searching?

The purpose of thorough and comprehensive literature searches is to avoid missing key studies and to minimize bias [ 6 , 8 , 10 , 34 , 37 , 38 , 39 ] since a systematic review based only on published (or easily accessible) studies may have an exaggerated effect size [ 35 ]. Felson (1992) sets out potential biases that could affect the estimate of effect in a meta-analysis [ 40 ] and Tricco et al. summarize the evidence concerning bias and confounding in systematic reviews [ 35 ]. Egger et al. point to non-publication of studies, publication bias, language bias and MEDLINE bias, as key biases [ 34 , 35 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ]. Comprehensive searches are not the sole factor to mitigate these biases but their contribution is thought to be significant [ 2 , 32 , 34 ]. Fehrmann (2011) suggests that ‘the search process being described in detail’ and that, where standard comprehensive search techniques have been applied, increases confidence in the search results [ 32 ].

Does comprehensive literature searching work?

Egger et al., and other study authors, have demonstrated a change in the estimate of intervention effectiveness where relevant studies were excluded from meta-analysis [ 34 , 47 ]. This would suggest that missing studies in literature searching alters the reliability of effectiveness estimates. This is an argument for comprehensive literature searching. Conversely, Egger et al. found that ‘comprehensive’ searches still missed studies and that comprehensive searches could, in fact, introduce bias into a review rather than preventing it, through the identification of low quality studies then being included in the meta-analysis [ 34 ]. Studies query if identifying and including low quality or grey literature studies changes the estimate of effect [ 43 , 48 ] and question if time is better invested updating systematic reviews rather than searching for unpublished studies [ 49 ], or mapping studies for review as opposed to aiming for high sensitivity in literature searching [ 50 ].

Aim and purpose beyond reviews of effectiveness

The need for comprehensive literature searches is less certain in reviews of qualitative studies, and for reviews where a comprehensive identification of studies is difficult to achieve (for example, in Public health) [ 33 , 51 , 52 , 53 , 54 , 55 ]. Literature searching for qualitative studies, and in public health topics, typically generates a greater number of studies to sift than in reviews of effectiveness [ 39 ] and demonstrating the ‘value’ of studies identified or missed is harder [ 56 ], since the study data do not typically support meta-analysis. Nussbaumer-Streit et al. (2016) have registered a review protocol to assess whether abbreviated literature searches (as opposed to comprehensive literature searches) has an impact on conclusions across multiple bodies of evidence, not only on effect estimates [ 57 ] which may develop this understanding. It may be that decision makers and users of systematic reviews are willing to trade the certainty from a comprehensive literature search and systematic review in exchange for different approaches to evidence synthesis [ 58 ], and that comprehensive literature searches are not necessarily a marker of literature search quality, as previously thought [ 36 ]. Different approaches to literature searching [ 37 , 38 , 59 , 60 , 61 , 62 ] and developing the concept of when to stop searching are important areas for further study [ 36 , 59 ].

The study by Nussbaumer-Streit et al. has been published since the submission of this literature review [ 63 ]. Nussbaumer-Streit et al. (2018) conclude that abbreviated literature searches are viable options for rapid evidence syntheses, if decision-makers are willing to trade the certainty from a comprehensive literature search and systematic review, but that decision-making which demands detailed scrutiny should still be based on comprehensive literature searches [ 63 ].

Key stage three: Preparing for the literature search

Six documents provided guidance on preparing for a literature search [ 2 , 3 , 6 , 7 , 9 , 10 ]. The Cochrane Handbook clearly stated that Cochrane authors (i.e. researchers) should seek advice from a trial search co-ordinator (i.e. a person with specific skills in literature searching) ‘before’ starting a literature search [ 9 ].

Two key tasks were perceptible in preparing for a literature searching [ 2 , 6 , 7 , 10 , 11 ]. First, to determine if there are any existing or on-going reviews, or if a new review is justified [ 6 , 11 ]; and, secondly, to develop an initial literature search strategy to estimate the volume of relevant literature (and quality of a small sample of relevant studies [ 10 ]) and indicate the resources required for literature searching and the review of the studies that follows [ 7 , 10 ].

Three documents summarised guidance on where to search to determine if a new review was justified [ 2 , 6 , 11 ]. These focused on searching databases of systematic reviews (The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE)), institutional registries (including PROSPERO), and MEDLINE [ 6 , 11 ]. It is worth noting, however, that as of 2015, DARE (and NHS EEDs) are no longer being updated and so the relevance of this (these) resource(s) will diminish over-time [ 64 ]. One guidance document, ‘Systematic reviews in the Social Sciences’, noted, however, that databases are not the only source of information and unpublished reports, conference proceeding and grey literature may also be required, depending on the nature of the review question [ 2 ].

Two documents reported clearly that this preparation (or ‘scoping’) exercise should be undertaken before the actual search strategy is developed [ 7 , 10 ]).

The guidance offers the best available source on preparing the literature search with the published studies not typically reporting how their scoping informed the development of their search strategies nor how their search approaches were developed. Text mining has been proposed as a technique to develop search strategies in the scoping stages of a review although this work is still exploratory [ 65 ]. ‘Clustering documents’ and word frequency analysis have also been tested to identify search terms and studies for review [ 66 , 67 ]. Preparing for literature searches and scoping constitutes an area for future research.

Key stage four: Designing the search strategy

The Population, Intervention, Comparator, Outcome (PICO) structure was the commonly reported structure promoted to design a literature search strategy. Five documents suggested that the eligibility criteria or review question will determine which concepts of PICO will be populated to develop the search strategy [ 1 , 4 , 7 , 8 , 9 ]. The NICE handbook promoted multiple structures, namely PICO, SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) and multi-stranded approaches [ 4 ].

With the exclusion of The Joanna Briggs Institute reviewers’ manual, the guidance offered detail on selecting key search terms, synonyms, Boolean language, selecting database indexing terms and combining search terms. The CEE handbook suggested that ‘search terms may be compiled with the help of the commissioning organisation and stakeholders’ [ 10 ].

The use of limits, such as language or date limits, were discussed in all documents [ 2 , 3 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ].

Search strategy structure

The guidance typically relates to reviews of intervention effectiveness so PICO – with its focus on intervention and comparator - is the dominant model used to structure literature search strategies [ 68 ]. PICOs – where the S denotes study design - is also commonly used in effectiveness reviews [ 6 , 68 ]. As the NICE handbook notes, alternative models to structure literature search strategies have been developed and tested. Booth provides an overview on formulating questions for evidence based practice [ 69 ] and has developed a number of alternatives to the PICO structure, namely: BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory [ 55 ]; SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) for identification of social science and evaluation studies [ 69 ] and, working with Cooke and colleagues, SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 70 ]. SPIDER has been compared to PICO and PICOs in a study by Methley et al. [ 68 ].

The NICE handbook also suggests the use of multi-stranded approaches to developing literature search strategies [ 4 ]. Glanville developed this idea in a study by Whitting et al. [ 71 ] and a worked example of this approach is included in the development of a search filter by Cooper et al. [ 72 ].

Writing search strategies: Conceptual and objective approaches

Hausner et al. [ 73 ] provide guidance on writing literature search strategies, delineating between conceptually and objectively derived approaches. The conceptual approach, advocated by and explained in the guidance documents, relies on the expertise of the literature searcher to identify key search terms and then develop key terms to include synonyms and controlled syntax. Hausner and colleagues set out the objective approach [ 73 ] and describe what may be done to validate it [ 74 ].

The use of limits

The guidance documents offer direction on the use of limits within a literature search. Limits can be used to focus literature searching to specific study designs or by other markers (such as by date) which limits the number of studies returned by a literature search. The use of limits should be described and the implications explored [ 34 ] since limiting literature searching can introduce bias (explored above). Craven et al. have suggested the use of a supporting narrative to explain decisions made in the process of developing literature searches and this advice would usefully capture decisions on the use of search limits [ 75 ].

Key stage five: Determining the process of literature searching and deciding where to search (bibliographic database searching)

Table 2 summarises the process of literature searching as reported in each guidance document. Searching bibliographic databases was consistently reported as the ‘first step’ to literature searching in all nine guidance documents.

Three documents reported specific guidance on where to search, in each case specific to the type of review their guidance informed, and as a minimum requirement [ 4 , 9 , 11 ]. Seven of the key guidance documents suggest that the selection of bibliographic databases depends on the topic of review [ 2 , 3 , 4 , 6 , 7 , 8 , 10 ], with two documents noting the absence of an agreed standard on what constitutes an acceptable number of databases searched [ 2 , 6 ].

The guidance documents summarise ‘how to’ search bibliographic databases in detail and this guidance is further contextualised above in terms of developing the search strategy. The documents provide guidance of selecting bibliographic databases, in some cases stating acceptable minima (i.e. The Cochrane Handbook states Cochrane CENTRAL, MEDLINE and EMBASE), and in other cases simply listing bibliographic database available to search. Studies have explored the value in searching specific bibliographic databases, with Wright et al. (2015) noting the contribution of CINAHL in identifying qualitative studies [ 76 ], Beckles et al. (2013) questioning the contribution of CINAHL to identifying clinical studies for guideline development [ 77 ], and Cooper et al. (2015) exploring the role of UK-focused bibliographic databases to identify UK-relevant studies [ 78 ]. The host of the database (e.g. OVID or ProQuest) has been shown to alter the search returns offered. Younger and Boddy [ 79 ] report differing search returns from the same database (AMED) but where the ‘host’ was different [ 79 ].

The average number of bibliographic database searched in systematic reviews has risen in the period 1994–2014 (from 1 to 4) [ 80 ] but there remains (as attested to by the guidance) no consensus on what constitutes an acceptable number of databases searched [ 48 ]. This is perhaps because thinking about the number of databases searched is the wrong question, researchers should be focused on which databases were searched and why, and which databases were not searched and why. The discussion should re-orientate to the differential value of sources but researchers need to think about how to report this in studies to allow findings to be generalised. Bethel (2017) has proposed ‘search summaries’, completed by the literature searcher, to record where included studies were identified, whether from database (and which databases specifically) or supplementary search methods [ 81 ]. Search summaries document both yield and accuracy of searches, which could prospectively inform resource use and decisions to search or not to search specific databases in topic areas. The prospective use of such data presupposes, however, that past searches are a potential predictor of future search performance (i.e. that each topic is to be considered representative and not unique). In offering a body of practice, this data would be of greater practicable use than current studies which are considered as little more than individual case studies [ 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 ].

When to database search is another question posed in the literature. Beyer et al. [ 91 ] report that databases can be prioritised for literature searching which, whilst not addressing the question of which databases to search, may at least bring clarity as to which databases to search first [ 91 ]. Paradoxically, this links to studies that suggest PubMed should be searched in addition to MEDLINE (OVID interface) since this improves the currency of systematic reviews [ 92 , 93 ]. Cooper et al. (2017) have tested the idea of database searching not as a primary search method (as suggested in the guidance) but as a supplementary search method in order to manage the volume of studies identified for an environmental effectiveness systematic review. Their case study compared the effectiveness of database searching versus a protocol using supplementary search methods and found that the latter identified more relevant studies for review than searching bibliographic databases [ 94 ].

Key stage six: Determining the process of literature searching and deciding where to search (supplementary search methods)

Table 2 also summaries the process of literature searching which follows bibliographic database searching. As Table 2 sets out, guidance that supplementary literature search methods should be used in systematic reviews recurs across documents, but the order in which these methods are used, and the extent to which they are used, varies. We noted inconsistency in the labelling of supplementary search methods between guidance documents.

Rather than focus on the guidance on how to use the methods (which has been summarised in a recent review [ 95 ]), we focus on the aim or purpose of supplementary search methods.

The Cochrane Handbook reported that ‘efforts’ to identify unpublished studies should be made [ 9 ]. Four guidance documents [ 2 , 3 , 6 , 9 ] acknowledged that searching beyond bibliographic databases was necessary since ‘databases are not the only source of literature’ [ 2 ]. Only one document reported any guidance on determining when to use supplementary methods. The IQWiG handbook reported that the use of handsearching (in their example) could be determined on a ‘case-by-case basis’ which implies that the use of these methods is optional rather than mandatory. This is in contrast to the guidance (above) on bibliographic database searching.

The issue for supplementary search methods is similar in many ways to the issue of searching bibliographic databases: demonstrating value. The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged [ 37 , 61 , 62 , 96 , 97 , 98 , 99 , 100 , 101 ] but understanding the value of the search methods to identify studies and data is unclear. In a recently published review, Cooper et al. (2017) reviewed the literature on supplementary search methods looking to determine the advantages, disadvantages and resource implications of using supplementary search methods [ 95 ]. This review also summarises the key guidance and empirical studies and seeks to address the question on when to use these search methods and when not to [ 95 ]. The guidance is limited in this regard and, as Table 2 demonstrates, offers conflicting advice on the order of searching, and the extent to which these search methods should be used in systematic reviews.

Key stage seven: Managing the references

Five of the documents provided guidance on managing references, for example downloading, de-duplicating and managing the output of literature searches [ 2 , 4 , 6 , 8 , 10 ]. This guidance typically itemised available bibliographic management tools rather than offering guidance on how to use them specifically [ 2 , 4 , 6 , 8 ]. The CEE handbook provided guidance on importing data where no direct export option is available (e.g. web-searching) [ 10 ].

The literature on using bibliographic management tools is not large relative to the number of ‘how to’ videos on platforms such as YouTube (see for example [ 102 ]). These YouTube videos confirm the overall lack of ‘how to’ guidance identified in this study and offer useful instruction on managing references. Bramer et al. set out methods for de-duplicating data and reviewing references in Endnote [ 103 , 104 ] and Gall tests the direct search function within Endnote to access databases such as PubMed, finding a number of limitations [ 105 ]. Coar et al. and Ahmed et al. consider the role of the free-source tool, Zotero [ 106 , 107 ]. Managing references is a key administrative function in the process of review particularly for documenting searches in PRISMA guidance.

Key stage eight: Documenting the search

The Cochrane Handbook was the only guidance document to recommend a specific reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 9 ]. Six documents provided guidance on reporting the process of literature searching with specific criteria to report [ 3 , 4 , 6 , 8 , 9 , 10 ]. There was consensus on reporting: the databases searched (and the host searched by), the search strategies used, and any use of limits (e.g. date, language, search filters (The CRD handbook called for these limits to be justified [ 6 ])). Three guidance documents reported that the number of studies identified should be recorded [ 3 , 6 , 10 ]. The number of duplicates identified [ 10 ], the screening decisions [ 3 ], a comprehensive list of grey literature sources searched (and full detail for other supplementary search methods) [ 8 ], and an annotation of search terms tested but not used [ 4 ] were identified as unique items in four documents.

The Cochrane Handbook was the only guidance document to note that the full search strategies for each database should be included in the Additional file 1 of the review [ 9 ].

All guidance documents should ultimately deliver completed systematic reviews that fulfil the requirements of the PRISMA reporting guidelines [ 108 ]. The guidance broadly requires the reporting of data that corresponds with the requirements of the PRISMA statement although documents typically ask for diverse and additional items [ 108 ]. In 2008, Sampson et al. observed a lack of consensus on reporting search methods in systematic reviews [ 109 ] and this remains the case as of 2017, as evidenced in the guidance documents, and in spite of the publication of the PRISMA guidelines in 2009 [ 110 ]. It is unclear why the collective guidance does not more explicitly endorse adherence to the PRISMA guidance.

Reporting of literature searching is a key area in systematic reviews since it sets out clearly what was done and how the conclusions of the review can be believed [ 52 , 109 ]. Despite strong endorsement in the guidance documents, specifically supported in PRISMA guidance, and other related reporting standards too (such as ENTREQ for qualitative evidence synthesis, STROBE for reviews of observational studies), authors still highlight the prevalence of poor standards of literature search reporting [ 31 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ]. To explore issues experienced by authors in reporting literature searches, and look at uptake of PRISMA, Radar et al. [ 120 ] surveyed over 260 review authors to determine common problems and their work summaries the practical aspects of reporting literature searching [ 120 ]. Atkinson et al. [ 121 ] have also analysed reporting standards for literature searching, summarising recommendations and gaps for reporting search strategies [ 121 ].

One area that is less well covered by the guidance, but nevertheless appears in this literature, is the quality appraisal or peer review of literature search strategies. The PRESS checklist is the most prominent and it aims to develop evidence-based guidelines to peer review of electronic search strategies [ 5 , 122 , 123 ]. A corresponding guideline for documentation of supplementary search methods does not yet exist although this idea is currently being explored.

How the reporting of the literature searching process corresponds to critical appraisal tools is an area for further research. In the survey undertaken by Radar et al. (2014), 86% of survey respondents (153/178) identified a need for further guidance on what aspects of the literature search process to report [ 120 ]. The PRISMA statement offers a brief summary of what to report but little practical guidance on how to report it [ 108 ]. Critical appraisal tools for systematic reviews, such as AMSTAR 2 (Shea et al. [ 124 ]) and ROBIS (Whiting et al. [ 125 ]), can usefully be read alongside PRISMA guidance, since they offer greater detail on how the reporting of the literature search will be appraised and, therefore, they offer a proxy on what to report [ 124 , 125 ]. Further research in the form of a study which undertakes a comparison between PRISMA and quality appraisal checklists for systematic reviews would seem to begin addressing the call, identified by Radar et al., for further guidance on what to report [ 120 ].

Limitations

Other handbooks exist.

A potential limitation of this literature review is the focus on guidance produced in Europe (the UK specifically) and Australia. We justify the decision for our selection of the nine guidance documents reviewed in this literature review in section “ Identifying guidance ”. In brief, these nine guidance documents were selected as the most relevant health care guidance that inform UK systematic reviewing practice, given that the UK occupies a prominent position in the science of health information retrieval. We acknowledge the existence of other guidance documents, such as those from North America (e.g. the Agency for Healthcare Research and Quality (AHRQ) [ 126 ], The Institute of Medicine [ 127 ] and the guidance and resources produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) [ 128 ]). We comment further on this directly below.

The handbooks are potentially linked to one another

What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely. The Cochrane Handbook, first published in 1994, is notably a key source of reference in guidance and systematic reviews beyond Cochrane reviews. It is not clear to what extent broadening the sample of guidance handbooks to include North American handbooks, and guidance handbooks from other relevant countries too, would alter the findings of this literature review or develop further support for the process model. Since we cannot be clear, we raise this as a potential limitation of this literature review. On our initial review of a sample of North American, and other, guidance documents (before selecting the guidance documents considered in this review), however, we do not consider that the inclusion of these further handbooks would alter significantly the findings of this literature review.

This is a literature review

A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. It is possible that other relevant studies could help contribute to the exploration and development of the key stages identified in this review.

This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews. We call this model ‘the conventional approach’, since it appears to be common convention in nine different guidance documents.

The findings reported above reveal eight key stages in the process of literature searching for systematic reviews. These key stages are consistently reported in the nine guidance documents which suggests consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews.

In Table 2 , we demonstrate consensus regarding the application of literature search methods. All guidance documents distinguish between primary and supplementary search methods. Bibliographic database searching is consistently the first method of literature searching referenced in each guidance document. Whilst the guidance uniformly supports the use of supplementary search methods, there is little evidence for a consistent process with diverse guidance across documents. This may reflect differences in the core focus across each document, linked to differences in identifying effectiveness studies or qualitative studies, for instance.

Eight of the nine guidance documents reported on the aims of literature searching. The shared understanding was that literature searching should be thorough and comprehensive in its aim and that this process should be reported transparently so that that it could be reproduced. Whilst only three documents explicitly link this understanding to minimising bias, it is clear that comprehensive literature searching is implicitly linked to ‘not missing relevant studies’ which is approximately the same point.

Defining the key stages in this review helps categorise the scholarship available, and it prioritises areas for development or further study. The supporting studies on preparing for literature searching (key stage three, ‘preparation’) were, for example, comparatively few, and yet this key stage represents a decisive moment in literature searching for systematic reviews. It is where search strategy structure is determined, search terms are chosen or discarded, and the resources to be searched are selected. Information specialists, librarians and researchers, are well placed to develop these and other areas within the key stages we identify.

This review calls for further research to determine the suitability of using the conventional approach. The publication dates of the guidance documents which underpin the conventional approach may raise questions as to whether the process which they each report remains valid for current systematic literature searching. In addition, it may be useful to test whether it is desirable to use the same process model of literature searching for qualitative evidence synthesis as that for reviews of intervention effectiveness, which this literature review demonstrates is presently recommended best practice.

Abbreviations

Behaviour of interest; Health context; Exclusions; Models or Theories

Cochrane Database of Systematic Reviews

The Cochrane Central Register of Controlled Trials

Database of Abstracts of Reviews of Effects

Enhancing transparency in reporting the synthesis of qualitative research

Institute for Quality and Efficiency in Healthcare

National Institute for Clinical Excellence

Population, Intervention, Comparator, Outcome

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Setting, Perspective, Intervention, Comparison, Evaluation

Sample, Phenomenon of Interest, Design, Evaluation, Research type

STrengthening the Reporting of OBservational studies in Epidemiology

Trial Search Co-ordinators

Booth A. Unpacking your literature search toolbox: on search styles and tactics. Health Information & Libraries Journal. 2008;25(4):313–7.

Article   Google Scholar  

Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.

Book   Google Scholar  

Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG Methods Resources. 7 Information retrieval 2014 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK385787/ .

NICE: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual 2014. Available from: https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf .

Sampson M. MJ, Lefebvre C, Moher D, Grimshaw J. Peer Review of Electronic Search Strategies: PRESS; 2008.

Google Scholar  

Centre for Reviews & Dissemination. Systematic reviews – CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009.

eunetha: European Network for Health Technology Assesment Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness 2016. Available from: http://www.eunethta.eu/sites/default/files/Guideline_Information_Retrieval_V1-1.pdf .

Kugley SWA, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell systematic reviews. Oslo: Campbell Collaboration. 2017; Available from: https://www.campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html

Lefebvre C, Manheimer E, Glanville J. Chapter 6: searching for studies. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions; 2011.

Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management.: Environmental Evidence:; 2013. Available from: http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf .

The Joanna Briggs Institute. Joanna Briggs institute reviewers’ manual. 2014th ed: the Joanna Briggs institute; 2014. Available from: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf

Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Inf Libr J. 2003;20(2):65–74.

Article   CAS   Google Scholar  

Harris MR. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. 2005;93(1):81–7.

PubMed   PubMed Central   Google Scholar  

Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One. 2015;10(5):e0125931.

Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

Article   PubMed   Google Scholar  

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. J Med Libr Assoc. 2004;92(2):163–4.

Swinkels A, Briddon J, Hall J. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. 2006;23(4):248–56.

Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager. EAHIL. 2015;11(3):3–7.

Lawson L. OPERATING OUTSIDE LIBRARY WALLS 2004.

Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, et al. Database selection in systematic reviews: an insight through clinical neurology. Health Inf Libr J. 2017;34(2):156–64.

Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, et al. A competency framework for librarians involved in systematic reviews. Journal of the Medical Library Association : JMLA. 2017;105(3):268–75.

Cooper ID, Crum JA. New activities and changing roles of health sciences librarians: a systematic review, 1990-2012. Journal of the Medical Library Association : JMLA. 2013;101(4):268–77.

Crum JA, Cooper ID. Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. Journal of the Medical Library Association : JMLA. 2013;101(4):278–86.

Dudden RF, Protzko SL. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. 2011;30(3):301–15.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008;61(5):440–8.

Maggio LA, Tannery NH, Kanter SL. Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(8):1049–54.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA. 2016;104(4):267–77.

Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. Journal of the Medical Library Association : JMLA. 2016;104(4):346–54.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016;11(9):e0163309.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Fehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15–32.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. 2016;5(1):74.

Article   PubMed   PubMed Central   Google Scholar  

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment (Winchester, England). 2003;7(1):1–76.

Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.

Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews. 2015;4(1):36.

Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol. 2005;5

Felson DT. Bias in meta-analytic research. J Clin Epidemiol. 1992;45(8):885–92.

Article   PubMed   CAS   Google Scholar  

Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–5.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, et al. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One. 2017;12(4):e0176210.

Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet (London, England). 1997;350(9074):326–9.

Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health technology assessment (Winchester, England). 2003;7(41):1–90.

Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol. 2005;58(8):769–76.

Mills EJ, Kanters S, Thorlund K, Chaimani A, Veroniki A-A, Ioannidis JPA. The effects of excluding treatments from network meta-analyses: survey. BMJ : British Medical Journal. 2013;347

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.

van Driel ML, De Sutter A, De Maeseneer J, Christiaens T. Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009;62(8):838–44.e3.

Buchberger B, Krabbe L, Lux B, Mattivi JT. Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. 2016;14:Doc09.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods. 2012;3(1):1–10.

Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22(2):213–28.

Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153–78.

Britten N, Garside R, Pope C, Frost J, Cooper C. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. 2017;27(9):1370–6.

Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J. 2015;32(3):220–35.

Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014;3:135.

Nussbaumer-Streit B, Klerings I, Wagner G, Titscher V, Gartlehner G. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Systematic Reviews. 2016;5:197.

Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017;17(1):121.

Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297–308.

Levay P, Raynor M, Tuvey D. The Contributions of MEDLINE, Other Bibliographic Databases and Various Search Techniques to NICE Public Health Guidance. 2015. 2015;10(1):19.

Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018;102:1–11.

Briscoe S, Cooper C, Glanville J, Lefebvre C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8(3):256–7.

Stansfield C, O'Mara-Eves A, Thomas J. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges. Research Synthesis Methods.n/a-n/a.

Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(3):479–88.

Stansfield C, Thomas J, Kavanagh J. 'Clustering' documents automatically to support scoping reviews of research: a case study. Res Synth Methods. 2013;4(3):230–41.

PubMed   Google Scholar  

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355–68.

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–43.

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health technology assessment (Winchester, England). 2006;10(36):iii-iv, xi-xiii, 1–154.

Cooper C, Levay P, Lorenc T, Craig GM. A population search filter for hard-to-reach populations increased search efficiency for a systematic review. J Clin Epidemiol. 2014;67(5):554–9.

Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):19.

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016;77:118–24.

Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists? A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. 2011;6(4):72–87.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, et al. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

Cooper C, Rogers M, Bethel A, Briscoe S, Lowe J. A mapping review of the literature on UK-focused health and social care databases. Health Inf Libr J. 2015;32(1):5–22.

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Inf Libr J. 2009;26(2):126–35.

Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association : JMLA. 2016;104(4):284–9.

Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference 2017a.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews - are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16(1):161.

Adams CE, Frederick K. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

Kelly L, St Pierre-Hansen N. So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. 2008;54(11):1572–3.

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Injury Prevention. 2008;14(6):401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004;12(4):228–32.

Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

Taylor B, Wylie E, Dempster M, Donnelly M. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. 2007;17(6):697–706.

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

Duffy S, de Kock S, Misso K, Noake C, Ross J, Stirk L. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE in-process searches via Ovid. Journal of the Medical Library Association : JMLA. 2016;104(4):309–12.

Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement (in Press). Research Synthesis Methods. 2017;

Cooper C, Booth, A., Britten, N., Garside, R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. (In Press). BMC Systematic Reviews. 2017.

Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ (Clinical research ed). 2005;331(7524):1064–5.

Article   PubMed Central   Google Scholar  

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5–11.

Levay P, Ainsworth N, Kettle R, Morgan A. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Res Synth Methods. 2016;7(1):34–45.

McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ (Clinical research ed). 1998;317(7172):1562–3.

Westphal A, Kriston L, Holzel LP, Harter M, von Wolff A. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of public health research. 2014;3(2):177.

Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, et al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

Bethel A. Endnote Training (YouTube Videos) 2017b [Available from: http://medicine.exeter.ac.uk/esmi/workstreams/informationscience/is_resources,_guidance_&_advice/ .

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240–3.

Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. Journal of the Medical Library Association : JMLA. 2017;105(1):84–7.

Gall C, Brahmi FA. Retrieval comparison of EndNote to search MEDLINE (Ovid and PubMed) versus searching them directly. Medical reference services quarterly. 2004;23(3):25–32.

Ahmed KK, Al Dhubaib BE. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. 2011;2(4):303–5.

Coar JT, Sewell JP. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. 2010;35(5):205–7.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (PRISMA) literature search reporting guidelines. Journal of the Medical Library Association : JMLA. 2017;105(3):233–9.

Booth A. "brimful of STARLITE": toward standards for reporting literature searches. Journal of the Medical Library Association : JMLA. 2006;94(4):421–9. e205

Faggion CM Jr, Wu YC, Tu YK, Wasiak J. Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. 2016;89(1062):20150878.

Mullins MM, DeLuca JB, Crepaz N, Lyles CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000–2010): are the searches clearly explained, systematic and reproducible? Research Synthesis Methods. 2014;5(2):116–30.

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association : JMLA. 2009;97(1):21–9.

Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev. 2016;5(1):174.

Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia. 2017;39:45–8.

Tao KM, Li XQ, Zhou QH, Moher D, Ling CQ, Yu WF. From QUOROM to PRISMA: a survey of high-impact medical journals' instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011;6(11):e27611.

Wasiak J, Tyack Z, Ware R. Goodwin N. Jr. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; 2016.

Tam WW, Lo KK, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017;7(2):e013905.

Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014;5(2):98–115.

Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77.

Medicine Io. Standards for Systematic Reviews 2011 [Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx .

CADTH: Resources 2018.

Download references

Acknowledgements

CC acknowledges the supervision offered by Professor Chris Hyde.

This publication forms a part of CC’s PhD. CC’s PhD was funded through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (Project Number 16/54/11). The open access fee for this publication was paid for by Exeter Medical School.

RG and NB were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Institute of Health Research, University of Exeter Medical School, Exeter, UK

Chris Cooper & Jo Varley-Campbell

HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Andrew Booth

Nicky Britten

European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK

Ruth Garside

You can also search for this author in PubMed   Google Scholar

Contributions

CC conceived the idea for this study and wrote the first draft of the manuscript. CC discussed this publication in PhD supervision with AB and separately with JVC. CC revised the publication with input and comments from AB, JVC, RG and NB. All authors revised the manuscript prior to submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Cooper .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cooper, C., Booth, A., Varley-Campbell, J. et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 18 , 85 (2018). https://doi.org/10.1186/s12874-018-0545-3

Download citation

Received : 20 September 2017

Accepted : 06 August 2018

Published : 14 August 2018

DOI : https://doi.org/10.1186/s12874-018-0545-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Literature Search Process
• Citation Chasing
• Tacit Models
• Unique Guidance
• Information Specialists

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• Searching Article Databases
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles

Searching in Google vs. Library Databases

Searching databases (video tutorial), what's in a database (infographic).

• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate
• 6. Synthesize
• 7. Write a Literature Review

Check out the videos below to learn more or contact a librarian for some search tips!

Photo Credit: UO Libraries

Watch this video from  Yavapai College Library to learn how to search library databases.

( Click to Enlarge Image )

Long Description of "What's in a Database?" for web accessibility

• << Previous: 3. Search the Literature
• Next: Finding Full-Text of an Article >>
• Last Updated: May 3, 2024 5:17 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

Literature Reviews & Search Strategies

• Defining the Literature Review
• Types of Literature Reviews

Use Multiple Databases

• Search Strategies
• Organizing Your Literature
• Books: Research Design & Scholarly Writing
• Recommended Tutorials

While not every literature search you undertake will be for a systematic review, the Cochrane Handbook's statement that "a search of MEDLINE alone is not considered adequate" holds true for almost all literature reviews. You need to go beyond one database to get a more comprehensive picture of your topic and to minimize selection bias. 

There are A LOT of databases that you could potential search for academic/scholarly articles to use in your literature review. We recommend focusing on resources that specializes in academic sources (ie databases), rather than a general search tool like Google because a lot of scholarly literature is still not discoverable on the open web and when it is you'll often hit a paywall and have to head to a subscription database available through the library to read the full article any way.

All our databases are listed on the A-Z Databases List , these are a few, often recommended, examples:

• << Previous: Types of Literature Reviews
• Next: Search Strategies >>
• Last Updated: Jun 14, 2023 11:18 AM
• URL: https://mcphs.libguides.com/litreviews

• Northeastern University Library
• Research Subject Guides
• Guides for Library Services
• Systematic Reviews and Evidence Syntheses
• Evidence Synthesis Service
• Types of Systematic Reviews in the Health Sciences
• Beginning Your Project
• Standards & Guidance
• Critical Appraisal
• Evidence-Based Assignments
• Tips for a Successful Review Team
• Training and Tutorials

Systematic Reviews and Evidence Syntheses : Databases

You will want to search at least three databases for your systematic review. Three databases alone does not complete the search standards for systematic review requirements. You will also have to complete a search of the grey literature and complete additional hand searches. Which databases you should search is highly dependent on your systematic review topic, so it is recommended you  meet with a librarian . 

Commonly Used Health Sciences Databases

Commonly used social sciences databases, commonly used education databases.

• Resources for Finding Systematic Reviews

You will want to search at least three databases for your systematic review. Three databases alone does not complete the search standards for systematic review requirements as you will also have additional searches of the grey literature and hand searches to complete.  Which databases you search is highly dependent on your systematic review topic, so it is recommended you  meet with a librarian . 

Cochrane, which is considered the gold standard for clinical systematic reviews, recommends searching the following three databases, at a minimum: PubMed, Embase, and Cochrane Central Register of Controlled Trials (CENTRAL).

• ERIC (Education Resources Institute) This link opens in a new window Citations to education information, including scholarly articles, professional literature, education dissertations, and books, plus grey literature such as curriculum guides, conference proceedings, government publications, and white papers. Covers 1966 to the present. more... less... Sponsored by the U.S. Department of Education.

Looking to Find Systematic Reviews?

There are a number of places to look for systematic reviews, including within the commonly used databases listed on this page. Some other resources to consider are:

• Systematic Review Repository - International Initiative for Impact Evaluation The systematic review repository from International Initiative for Impact Evaluation is an essential resource for policymakers and researchers who are looking for synthesized evidence on the effects of social and economic interventions in low- and middle- income countries.
• Epistemonikos Epistemonikos is a collaborative, multilingual database of health evidence. It is the largest source of systematic reviews relevant for health-decision making, and a large source of other types of scientific evidence. PLEASE NOTE: Epistemonikos is a systematic reviews focused database. It pulls in systematic reviews from a number of different international sources and pulls in the studies those reviews. While you will find randomized controlled trials and other primary studies in this database, they are only added in because of their association with a systematic review. Therefore, searching here for randomized controlled trials or other primary studies would NOT be considered a comprehensive search.
• << Previous: Types of Systematic Reviews in the Health Sciences
• Next: Resources for Completing Evidence Syntheses >>
• Ask a Librarian
• Last Updated: May 16, 2024 7:56 AM
• URL: https://subjectguides.lib.neu.edu/systematicreview

• En español – ExME
• Em português – EME

Literature searches: what databases are available?

Posted on 6th April 2021 by Izabel de Oliveira

Many types of research require a search of the medical literature as part of the process of understanding the current evidence or knowledge base. This can be done using one or more biomedical bibliographic databases. [1]

Bibliographic databases make the information contained in the papers more visible to the scientific community and facilitate locating the desired literature.

This blog describes some of the main bibliographic databases which index medical journals.

PubMed was launched in 1996 and, since June 1997, provides free and unlimited access for all users through the internet. PubMed database contains more than 30 million references of biomedical literature from approximately 7,000 journals. The largest percentage of records in PubMed comes from MEDLINE (95%), which contains 25 million records from over 5,600 journals. Other records derive from other sources such as In-process citations, ‘Ahead of Print’ citations, NCBI Bookshelf, etc.

The second largest component of PubMed is PubMed Central (PMC) . Launched in 2000, PMC is a permanent collection of full-text life sciences and biomedical journal articles. PMC also includes articles deposited by journal publishers and author manuscripts, published articles that are submitted in compliance with the public access policies of the National Institutes of Health (NIH) and other research funding agencies. PMC contains approximately 4.5 million articles.

Some National Library of Medicine (NLM) resources associated with PubMed are the NLM Catalog and MedlinePlus. The NLM Catalog contains bibliographic records for over 1.4 million journals, books, audiovisuals, electronic resources, and other materials. It also includes detailed indexing information for journals in PubMed and other NCBI databases, although not all materials in the NLM Catalog are part of NLM’s collection. MedlinePlus is a consumer health website providing information on various health topics, drugs, dietary supplements, and health tools.

MeSH (Medical Subject Headings) is the NLM controlled vocabulary used for indexing articles in PubMed. It is used by indexers who analyze and maintain the PubMed database to reflect the subject content of journal articles as they are published. Indexers typically select 10–12 MeSH terms to describe every paper.

Embase is considered the second most popular database after MEDLINE. More than 32 million records from over 8,200 journals from more than 95 countries, and ‘grey literature’ from over 2.4 million conference abstracts, are estimated to be in the Embase content.

Embase contains subtopics in health care such as complementary and alternative medicine, prognostic studies, telemedicine, psychiatry, and health technology. Besides that, it is also widely used for research on drug-related topics as it offers better coverage than MEDLINE on pharmaceutics-related literature.

In 2010, Embase began to include all MEDLINE citations. MEDLINE records are delivered to Elsevier daily and are incorporated into Embase after de-duplication with records already indexed by Elsevier to produce ‘MEDLINE-unique’ records. These MEDLINE-unique records are not re-indexed by Elsevier. However, their indexing is mapped to Emtree terms used in Embase to ensure that Emtree terminology can be used to search all Embase records, including those originally derived from MEDLINE.

Since this coverage expansion—at least in theory and without taking into consideration the different indexing practices of the two databases—a search in Embase alone should cover every record in both Embase and MEDLINE, making Embase a possible “one-stop” search engine for medical research [1].

Emtree is a hierarchically structured, controlled vocabulary for biomedicine and the related life sciences. It includes a whole range of terms for drugs, diseases, medical devices, and essential life science concepts. Emtree is used to index all of the Embase content. This process includes full-text indexing of journal articles, which is done by experts.

The most important index of the technical-scientific literature in Latin America and the Caribbean, LILACS , was created in 1985 to record scientific and technical production in health. It has been maintained and updated by a network of more than 600 institutions of education, government, and health research and coordinated by Latin America and Caribbean Center on Health Sciences Information (BIREME), Pan American Health Organization (PAHO), and World Health Organization (WHO).

LILACS contains scientific and technical literature from over 908 journals from 26 countries in Latin America and the Caribbean, with free access. About 900,000 records from articles with peer review, theses and dissertations, government documents, conference proceedings, and books; more than 480,000 of them are available with the full-text link in open access.

The LILACS Methodology is a set of standards, manuals, guides, and applications in continuous development, intended for the collection, selection, description, indexing of documents, and generation of databases. This centralised methodology enables the cooperation between Latin American and Caribbean countries to create local and national databases, all feeding into the LILACS database.  Currently, the databases LILACS, BBO, BDENF, MEDCARIB, and national databases of the countries of Latin America are part of the LILACS System.

Health Sciences Descriptors (DeCS) is the multilingual and structured vocabulary created by BIREME to serve as a unique language in indexing articles from scientific journals, books, congress proceedings, technical reports, and other types of materials, and also for searching and retrieving subjects from scientific literature from information sources available on the Virtual Health Library (VHL) such as LILACS, MEDLINE, and others. It was developed from the MeSH with the purpose of permitting the use of common terminology for searching in multiple languages, and providing a consistent and unique environment for the retrieval of information. DeCS vocabulary is dynamic and totals 34,118 descriptors and qualifiers, of which 29,716 come from MeSH, and 4,402 are exclusive.

Cochrane CENTRAL

The Cochrane Central Register of Controlled Trials (CENTRAL) is a database of reports of randomized and quasi-randomized controlled trials. Most records are obtained from the bibliographic databases PubMed and Embase, with additional records from the published and unpublished sources of CINAHL, ClinicalTrials.gov, and the WHO’s International Clinical Trials Registry Platform.

Although CENTRAL first began publication in 1996, records are included irrespective of the date of publication, and the language of publication is also not a restriction to being included in the database.  You won’t find the full text to the article on CENTRAL but there is often a summary of the article, in addition to the standard details of author, source, and year.

Within CENTRAL, there are ‘Specialized Registers’ which are collected and maintained by Cochrane Review Groups (plus a few Cochrane Fields), which include reports of controlled trials relevant to their area of interest. Some Cochrane Centres search the general healthcare literature of their countries or regions in order to contribute records to CENTRAL.

ScienceDirect

ScienceDirect i s Elsevier’s most important peer-reviewed academic literature platform. It was launched in 1997 and contains 16 million records from over 2,500 journals, including over 250 Open Access publications, such as Cell Reports and The Lancet Global Health, as well as 39,000 eBooks.

ScienceDirect topics include:

• health sciences;
• life sciences;
• physical sciences;
• engineering;
• social sciences; and
• humanities.

Web of Science

Web of Science (previously Web of Knowledge) is an online scientific citation indexing service created in 1997 by the Institute for Scientific Information (ISI), and currently maintained by Clarivate Analytics.

Web of Science covers several fields of the sciences, social sciences, and arts and humanities. Its main resource is the Web of Science Core Collection which includes over 1 billion cited references dating back to 1900, indexed from 21,100 peer-reviewed journals, including Open Access journals, books and proceedings.

Web of Science also offers regional databases which cover:

• Latin America (SciELO Citation Index);
• China (Chinese Science Citation Database);
• Korea (Korea Citation Index);
• Russia (Russian Science Citation Index).

Boolean operators

To make the search more precise, we can use boolean operators in databases between our keywords.

We use boolean operators to focus on a topic, particularly when this topic contains multiple search terms, and to connect various pieces of information in order to find exactly what we are looking for.

Boolean operators connect the search words to either narrow or broaden the set of results. The three basic boolean operators are: AND, OR, and NOT.

• AND narrows a search by telling the database that all keywords used must be found in the article in order for it to appear in our results.
• OR broadens a search by telling the database that any of the words it connects are acceptable (this is useful when we are searching for synonymous words).
• NOT narrows the search by telling the database to eliminate all terms that follow it from our search results (this is helpful when we are interested in a specific aspect of a topic or when we want to exclude a type of article.

References (pdf)

You may also be interested in the following blogs for further reading:

Conducting a systematic literature search

Reviewing the evidence: what method should I use?

Cochrane Crowd for students: what’s in it for you?

Izabel de Oliveira

Leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.

Related Articles

Epistemonikos: the world’s largest repository of healthcare systematic reviews

Learn more about the Epistemonikos Foundation and its repository of healthcare systematic reviews. The first in a series of three blogs.

How do you use the Epistemonikos database?

Learn how to use the Epistemonikos database, the world’s largest multilingual repository of healthcare evidence. The second in a series of three blogs.

Epistemonikos: All you need is L·OVE

Discover more about the ‘Living OVerview of Evidence’ platform from Epistemonikos, which maps the best evidence relevant for making health decisions. The final blog in a series of three focusing on the Epistemonikos Foundation.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Evid Based Spine Care J
• v.1(1); 2010 May

Conducting a winning literature search

Erika d. ecker.

1 Spectrum Research, Inc., Tacoma, Washington, USA

Andrea C. Skelly

So what is a “winning literature search”? Simply put, it is one that provides you with the information you need to find the types of articles that will help you with clinical practice or research. Literature searching is a combination of an art and a science. Understanding the basic anatomy and physiology of searching can get you started on finding the information you need.

I. Anatomy of a Literature Search—The Skeleton

A. constructing an appropriate question.

• Asking the right question is the primary key to creating a winning search. Your questions must be answerable. If your question is too broad, your search will yield more information than you can possibly look through.

Similarly, if your study was prognostic rather than therapeutic, a PPO table would be used instead of the PICO table in order to help formulate your question. Thus, the categories would change to Patients, Prognostic factors, and Outcome.

⇛ Using treatment studies as an example, the PICO concept can really help you create an answerable question since, as you will see below, it will help you create a search strategy.

B. Using the appropriate database(s)

Now that you have honed your question, it is time to focus on efficient article retrieval. What type of information is needed and what type of articles do you need? What will you do with the information you gather? This will determine the type of information you need and types of articles that may provide that information. It also may influence the type of database you search for that information.

• Bibliographic databases contain references to published literature, such as journals and newspaper articles, conference proceedings and papers, reports, government and legal publications, patents, and books.
• Peer-reviewed literature is scholarly work that generally represents the latest original research in a field. These articles undergo expert screening before publication to ensure meaningfulness within the context of other research in the discipline and, at least in theory, sound methodology.
• “Gray” literature refers to material that is not formally published by commercial publishers or peer-reviewed journals, including reports, fact sheets, white papers, conference proceedings, and other documents from various organizations and government agencies.

The table below provides brief descriptions of common databases and sources to search both peer-reviewed and gray literature.

For example, continuing with our question regarding complications after ADR versus fusion, which type of database listed in the table makes the most sense to search? Indexed peer-reviewed articles will give us the best available and most current data and MEDLINE, which includes millions of citations for biomedical articles and can be accessed using PubMed for free, seems like a great starting place. Generally speaking, PubMed will be the best place to begin your search and there are various ways, as you will see below, to refine and limit your search in order to find exactly what you need.

II. The Physiology of Literature Searching—How It Works

Now that you have an answerable question and an idea of what type of database you need to search (at least to start), let's talk about the nuts and bolts of searching. For the purposes of this paper, we will use PubMed as the search engine.

A. Getting specific—the basics of “how to”

1. Quickstart:

• Type a word or phrase into the query box, including subject, author, and/or journal
• Click on the search button or press the “enter” key

• To retrieve more information about the search results, use the display settings menu (upper left corner) to view the abstract or MEDLINE formats, change the number of items that appear per page, and sort by recently added, publication date, first author, last author, journal, or title.
• PubMed also contains links to full-text articles (appears in upper right corner of page) at participation publishers' web sites as well as links to other third party sites such as libraries and sequencing centers.

⇛ Anything which appears in blue and is underlined is a link that reveals more information. Clicking on the title would bring up the abstract (Abstract format). Clicking on “Related articles” would provide a link to other similar articles that might be of interest.

2. Advanced searching in PubMed—MeSH terms and the MeSH database:

a. Medical subject headings (MeSH)

It is important to understand that PubMed uses a controlled vocabulary to index journal articles called MeSH and uses “automatic term mapping” to find MeSH terms when you search. MeSH terms are organized in a hierarchy called a tree, with more specific (narrower) terms arranged beneath broader terms. By default, PubMed includes in the search all narrower terms; this is called “exploding” the MeSH term. Inclusion of MeSH terms enhances and optimizes the search strategy. For example, if you looked up the term “Spine” in the MeSH database you would see ​ see :

Therefore, PubMed would retrieve every article containing any of the terms located under Spine in the hierarchy.

b. MeSH database features

MeSH vocabulary contains over 25,000 descriptors and is updated weekly and reviewed annually. You can only search citations that have been indexed for MEDLINE (92% of the PubMed database) using MeSH terms. Features include:

• Allows you to identify and select appropriate MeSH terms for a search and to see their definitions
• Builds a PubMed search strategy
• Displays MeSH terms in the hierarchy (MeSH tree) allowing you to broaden/narrow a search
• Limits MeSH terms to a major concept/topic heading for a search
• Allows you to broaden your search by choosing not to explode a term
• The list of subheadings includes terms paired at least once with a given heading in MEDLINE.
• Focuses searches using other types of MeSH terms including publication types [pt], substance names [nm] or registry numbers [rn], and pharmaceutical actions [pa]
• MeSH Brower for access to annotations: http://www.nlm.nih.gov/mesh/

To access MeSH from PubMed, click on MeSH Database on the PubMed homepage or click MeSH under “more resources” in “advanced search.”

Once in the MeSH database, if you entered cancer into the search bar and clicked Go (or hit Enter) you would see ​ see :

Clicking on “neoplasms” will bring up the page where you have the option of selecting any of the features listed above to help you refine your search.

Also, clicking “links” adjacent to the MeSH term desired, will give you a drop-down menu which offers several options:

• PubMed: search PubMed with the term
• PubMed—Major topic: search PubMed with the MeSH term, retrieving only citations where the term is a major focus
• Clinical queries: put the MeSH term into the Clinical Queries box where the search may be further refined
• NLM MeSH browser: show the MeSH browser descriptor data for this term including scope note, allowable qualifiers, and the MeSH tree

⇛ The Mesh database homepage includes three brief tutorials on how to search with the MeSH database, combine MeSH terms, and apply subheadings and other features of the MeSH database.

B. Too much information! Refining your search

• Replace general search terms with more specific terms (the MeSH database would be a great resource for this)
• AND between terms returns only records that contain all of the search terms
• OR between terms returns all records that contain any of the search terms
• NOT between search terms returns only records that contain the first term and not the second
• Example: mimic* will find all terms that begin with the letters m-i-m-i-c-; eg, mimic, mimics, mimicking
• Example: behavio?r will find behaviour or behavior
• Use the “limit” option in PubMed to limit citations by age group, language, publication type, date, human studies, etc.
• Use the “advanced search” option to look up a term as it is indexed in PubMed
• Use the MeSH database features

Let's say we are interested in what the best surgical treatment is for osteoporotic spine fractures. Using some of the tips above, the chart below shows how a typical search might go: ​ go:

By combining terms (using Boolean logic), truncating a term, and using the limits option we were able to narrow our search down from 16,023 articles to a more manageable and relevant 54 articles. The “details” tab in the PubMed search window shows the complete search expression (ie, query translation) employed by PubMed, similar to what is represented in the table above. ​ above.

One of the best resources that PubMed provides for users new to the database is the online tutorials. They are brief but informative and because they are interactive you are guided step-by-step through each process. Perhaps consulting the online tutorials and the fact sheets on PubMed would be a next step for you. Give it a try! In fact, here is the link to the PubMed Tutorial homepage created by the National Library of Medicine: http://www.nlm.nih.gov/bsd/disted/pubmedtutorial/ . Also, check-out the PubMed help page which contains a plethora of information regarding all aspects of PubMed: http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=helppubmed&part=pubmedhelp

III. Closing Thoughts

Remember, literature searching is a combination of an art and a science. It requires practice, intuition, and some trial and error. While there is a basic structure, a set of guidelines and many tools for assisting one with basic searches, there are a variety of nuances and advanced techniques that may be required for more specialized searches. For systematic reviews as an example, extensive searches are required and may take numerous hours, involving many databases (including those for gray literature), and a combination of advanced search strategies in order to be methodologically sound. Use of personnel with specialized expertise in conducting such searches may provide the best results and be the most resource effective.

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

• What is a Literature Review?
• Developing a Research Question
• Selection Criteria

Choosing Search Terms

Filtering your search, translating between databases.

• Documenting Your Search
• Organize Key Findings
• Reference Management

Boolean Operators

Connecting terms with AND requires all terms to appear in the same article.

Connecting terms with OR results in articles that include one term, a combination of terms, or all of the terms.

Search Tip!

Keep in mind that search is not a linear process -- you will need to test and revise as you go. Terms that seem good may not find good results, or your topic may be too broad or too narrow.

If you would like assistance with your search, contact a librarian .

Before starting your database search, think about terms that can be used to describe the key concepts in your research question. Start your search with terms that you think make sense.  When you find citations that are highly relevant to your research, take a closer look at those records. Examine those records for two types of terms  that you can use in your search : subject headings  and keywords.

• Subject Heading : A single, assigned term that stands for a concept. For example, in PubMed, any paper that discusses acetylsalicylic  acid  would be assigned the Medical Subject Heading (MeSH) term aspirin . A search for the MeSH term  Aspirin  in PubMed should find papers written about aspirin  whether or not the word actually appears in the title or abstract.
• Keyword : Term used for a concept in everyday language. For example, if you need to find articles written about bedpans , the  Medical Subject Heading (MeSH)  term Toilet Facilities  in PubMed may be too broad. Just searching for bedpan OR bedpans  by typing this directly into the search box might work better.

Subject headings and keywords have different advantages and disadvantages. Keywords can retrieve new articles that do not yet have  subject headings assigned to them. You can also use keywords to capture alternative spellings. Subject headings, however, will help you find highly relevant articles, and may mitigate the need to search for synonyms.

When you conduct your search, consider whether it makes sense to use keywords, subject headings, or both.

See "Documenting Your Search"  to learn how to keep track of useful terms.

Many databases allow you to filter your search. You can usually find filters are on the left-hand side of your results page . Based on your selection criteria , you may want to filter your results based on:

• Publication date range (e.g. last 10 years)
• Source (e.g. journal name)
• Article type (e.g. review, research report, etc.)
• Study type (e.g. randomized-control trial, cohort, etc.)

If you're searching different databases for information, keep in mind that you may need to adjust your search terms for each database. For instance, the equivalent subject heading for  " Heart, Artificial" in  PubMed is "Heart, Mechanical" in CINAHL. Additionally, because CINAHL is an allied health and nursing database, you will find specialized subject headings such as "Toileting" in CINAHL that you won't find in PubMed.

Keywords are more likely to stay consistent across databases.

• << Previous: Databases
• Next: Documenting Your Search >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

Conducting a Literature Review

• Literature Review
• Developing a Topic
• Planning Your Literature Review
• Developing a Search Strategy
• Managing Citations
• Critical Appraisal Tools
• Writing a Literature Review

Selecting Sources to Search

A good literature review should be as comprehensive as necessary to identify all of the major works and debates on your research subject.  

Subject-specific Databases - search in databases specific to your discipline of study to find more sources in your field. For example, Sociological Abstracts specializes in Sociology and will have more coverage of the sociology literature than an interdisciplinary, all-purpose database such as ProQuest.  You should also search in more than one database/catalog since no one search tool covers everything.  For example, if your topic involves education, consider also searching an education database, such as ERIC.

Here are ideas about finding places to search.

• Cochrane Library

Databases A-Z  - Once you've identified disciplines or information types, consult the Databases A-Z list by subject.  

Google Scholar   -  also search for your topic in Google Scholar.  If you have a relevant source, consider searching the title in Google Scholar and using the “Cited By” link and the “Related Articles” to locate more literature.  

Library Catalog  - The Library Catalog also searches a variety of databases for books and journal articles at once. However, it does not search everything, so be sure to also look at disciplinary databases

Database Searching Tips

The search terms or  keywords  you use to search are what determine the results you get.  

1.  Express your topic in a topic sentence

2. Generate keyword search terms by  identifying the main ideas or concepts  within that topic sentence:

3. Expand your search terms by  brainstorming related terms or synonyms  that describe your main ideas

Boolean Operators or Combine Search Terms; AND, OR, NOT

Boolean logic is a building block of many computer applications and is an important concept in database searching.  Using the correct Boolean operator can make all the difference in a successful search.For example, if your search terms are Shingles  and vaccines and aged .

• AND  searches find all of the search terms.  For example, searching on shingles  AND  vaccine  AND aged  returns only results that contain all three search terms.  Very limited results.
• OR  searches find one term or the other.  Searching on Shingles  OR  Varicella-Zoster  OR  chickenpox returns all items that contain any of the three search terms.  Returns a large number of results.
• NOT  eliminates items that contain the specified term.  Searching on Shingles  NOT  chickenpox returns items that are about Shingles, but will specifically  NOT  return items that contain the word chickenpox.  This is a way to fine-tune results. Note:  sometimes  AND NOT  is used; serves the same function as  NOT .

Using Boolean Search with Exact Phrases:

If you're searching for a phrase rather than just a single word, you can group the words together with quotation marks.  Searching on "Varicella Zoster" will return only items with that exact phrase.  

Phrase Searching:

If you are searching for a phrase, keep in mind that not all databases will search multiple words automatically as a phrase.  Check the database Help pages to be sure how that database handles multiple works.

• Some assume that words typed next to each other should be searched as phrases.
• Others automatically put a Boolean AND between your search terms, requiring that all the words be present, but not necessarily adjacent to each other.
• Some databases such as PubMed (and Google) use  " "  around multiple words to designate a phrase search.  For example:  " Varicella-Zoster Virus "

When to use parentheses:

Think of your search in concepts, then put those concepts inside parentheses.  Different databases have different rules about combining searches.  To make sure you get the search you want, use parentheses - every database follows those rules.

Truncation:

Using truncation symbols allows you to expand your results by including various endings for a search term.  Most databases will designate a non-alphabetical symbol -- like   ! , *, or ?  -- as a truncation symbol; check the database Help screens to find out the specific symbol.             PubMed uses an Asterisk ( * ) .   Using the truncation symbol at the end of the root word will bring back results that include any ending of that root word.  For example, if the truncation symbol was a  ?  , then:

• child* = child, child's, children, childish, childlike, childhood
• transplant*  = transplants, transplantation, transplanted

Cited Reference Searching

Cited Reference searching  is the ability to search the list of references (or footnotes) found in journal articles, books, dissertations, websites, etc. It is based on the premise that you have a scholarly work in-hand that you really like and you want to see who else has used that work in their research - therefore, have included it in their list of references (or footnotes). Typically, cited reference searching involves looking for works by a particular author or for a specific piece. The best database for cited reference searching are Scopus and Google Scholar.

A large database containing scholarly journal articles and conference papers on any topic in science, technology, medicine, social sciences, and arts & humanities. Includes cited references and h-index information.

Google Schola r  provides a simple way to broadly search for scholarly literature. It is designed very similar to the main  Google  search, but the results are limited to just scholarly sources: papers, theses, books, abstracts and articles, from academic publishers, professional societies, preprint repositories, universities and other scholarly organizations. One of the key features is that it will show how many times a piece has been cited within Google Scholar.

Save Your Search

Save Your Search Results

Save copies of the useful records you find and where possible save a copy of your search strategy. This will ensure that you don’t have to repeat work.

Save the articles you find:

Always save or print the useful article records you find. Most databases give you a few options, such as:

• save – usually as a text file or an RIS file
• direct export to reference software such as Zotero.

Generally you will not be able to download the full text of the documents directly from the database. In many databases you will have to follow the "full text" links. If the Library has a subscription, you will be able to download the article.

Save your search strategy

The database may have a free personal account feature that allows you to save a copy of your search strategy. Saving your strategy means your search can be re-run without you having to re-enter details.

Search Activity Log

Be sure to keep a search activity log, which documents where you searched, search terms and number of results. This can be done very easily in Excel or Sheets:

Beyond the IHS Library

InterLibrary Loan/Document Delivery - (ILL)

Interlibrary services are available to students, faculty, administrators, and staff of SHU. We can help you borrow articles or books not held by the Seton Hall University Libraries through the use of  Document Delivery/Interlibrary Loan (ILL)  to get it. Articles and chapters can usually be scanned and sent electronically, but books must be mailed and typically arrive in 1-2 weeks, so plan ahead. 

The CINAHL database covers nearly 1000 English-language nursing and allied health journals. It also includes references to book chapters, pamphlets, audiovisual materials, software, dissertations, standards of professional practice and nurse practice acts.

HCUPnet is a free, on-line query system based on data from the Healthcare Cost and Utilization Project (HCUP). The system provides health care statistics and information for hospital inpatient, emergency department, and ambulatory settings, as well as population-based health care data on counties

PolicyMap is a web-based, GIS-lite mapping tool. It allows you to create shaded maps with data points as well as tables with U.S. data. Community and Community Health profiles are also available for states, counties and towns.

PubMed comprises more than 22 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full-text content from PubMed Central and publisher web sites.

PsycINFO is the largest resource devoted to peer-reviewed literature in behavioral science and mental health. Journal coverage, which spans from the 1800s to the present, includes international material. 

• Scopus 

International coverage of journal articles, conference proceedings, selected web sites, and patents in the sciences, social sciences, and humanities; provides citation tracking for 1996+ (select coverage for earlier periods) and cited reference searches back to 1970.

Subject Headings

Subject Headings:

Subject Headings, also called descriptors, these terms are assigned to items to describe their content, or what they are about. Subject headings often facilitate more precise searching as they eliminate the need to search multiple phrases and synonyms for the same concept.  Look for subject headings on items in the library catalog and in databases of journal articles.  Many databases also provide a thesaurus, or index, of the subject headings used

 MeSH:

Medical Subject Headings  ( MeSH ) is a comprehensive controlled vocabulary for the purpose of indexing journal articles and books in the  life sciences. It serves as a thesaurus that facilitates searching. Created and updated by the United States National Library of Medicine (NLM), it is used by the  PubMed  article database and by NLM's catalog of book holdings. The 2021 Medical Subject Headings are available at:  https://meshb.nlm.nih.gov/search .

In PubMed, every journal article is indexed with about 10–15 subject headings, subheadings and supplementary concept records, with some of them designated as  major  and marked with an asterisk, indicating the article's major topics. When performing a MEDLINE search via PubMed, entry terms are automatically translated into (i.e. mapped to) the corresponding descriptors with a good degree of reliability; it is recommended to check the 'Details tab' in PubMed to see how a search formulation was translated. 

Author Search

Author Search  - many researchers will write about the same topic for their entire career. Searching by an author's name in a subject database and/or Google Scholar  PubMed, Scopus may garner additional relevant articles related to your topic.

Bibliography Mining

Bibliography Mining  - use the list of works cited from a relevant source to locate additional related sources.  This is a way to look for relevant sources published  prior to  the one in hand.  A database may have a direct link to all the works cited, such as the example below; otherwise, you may need to look at each work cited manually.

Show Scopus

• The Interprofessional Health Sciences Library
• 123 Metro Boulevard
• Nutley, NJ 07110
• [email protected]
• Visiting Campus
• News and Events
• Parents and Families
• Web Accessibility
• Career Center
• Public Safety
• Accountability
• Privacy Statements
• Report a Problem
• Login to LibApps

Literature Reviews: 2. Identify Databases & Resources to Search

• Library Basics
• 1. Choose Your Topic
• How to Find Books
• Types of Clinical Study Designs
• Types of Literature
• 3. Search the Literature
• 4. Read & Analyze the Literature
• 5. Write the Review
• Keeping Track of Information
• Style Guides
• Books, Tutorials & Examples

Where to Look?

Key databases.

• Business Source Complete
• CINAHL (Cumulative Index to Nursing and Allied Health Literature)
• ERIC (Education Resources Information Center)
• Sociological Collection

Browse our main Databases page here .

• Google Scholar

Grey Literature

There are many definitions of grey literature, but it is usually understood to mean literature that is not formally published in sources such as books or journal articles.   It may be described as ephemeral, invisible, informal, underground, etc., that is, literature that may be unevaluated, not peer-reviewed.

Grey Literature exists in many formats: reports-including preprints; preliminary progress and advanced reports; institutional, internal, technical, and statistical reports; research memoranda; state-of-the-art reports; market research reports; reports of commissions and study groups; as well as * theses * conference proceedings * technical specifications and standards * translations (not distributed commercially) * bibliographies * technical and commercial documentation * official documents (issued in limited numbers)

• GSU Theses and Dissertations

About finding articles

Articles are typically the most important type of source for many types of research.  The most efficient way to find articles on a topic is to search a database, which allows you to search for articles from hundreds of journals at once.  Each database searches different sets of journals, so usually you'll want to search several databases.All of the databases listed on this page are accessible from off campus. Off-campus users will be prompted for their Campus ID/password.

Getting the full article

• << Previous: 1. Choose Your Topic
• Next: How to Find Books >>
• Last Updated: Dec 29, 2023 11:41 AM
• URL: https://research.library.gsu.edu/litrev

• UNC Libraries
• HSL Academic Process
• Creating a PRISMA flow diagram
• PRISMA 2020

Creating a PRISMA flow diagram: PRISMA 2020

Created by health science librarians.

What is PRISMA?

Which prisma 2020 flow diagram should i use, step-by-step: prisma 2020 flow diagram, using the covidence prisma diagram, documenting your grey literature search, updating a systematic review with prisma 2020, citing prisma 2020, for more information, prisma 2020 checklist.

• PRISMA 2020 Checklist (.doc)
• PRISMA 2020 Checklist (.pdf)
• PRISMA 2020 Expanded Checklist

PRISMA 2020 Flow Diagram Templates

• PRISMA 2020 V1- New Reviews with Databases and Registers only PRISMA 2020 flow diagram for new systematic reviews which included searches of databases and registers only
• PRISMA 2020 V2 - New Reviews with Databases, Registers, and Other Sources PRISMA 2020 flow diagram for new systematic reviews which included searches of databases, registers and other sources

The format of the PRISMA Step-By-Step was first developed by Glasgow Caledonian University https://www.gcu.ac.uk/library

"PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses. We have focused on randomized trials, but PRISMA can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions. PRISMA may also be useful for critical appraisal of published systematic reviews, although it is not a quality assessment instrument to gauge the quality of a systematic review. The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram ."

"The PRISMA Explanation and Elaboration document explains and illustrates the principles underlying the PRISMA Statement. It is strongly recommended that it be used in conjunction with the PRISMA Statement.

PRISMA is part of a broader effort, to improve the reporting of different types of health research, and in turn to improve the quality of research used in decision-making in healthcare."

From prisma-statement.org

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  J Clin Epidemiol . 2009;62(10):e1-e34. doi:10.1016/j.jclinepi.2009.06.006

Page MJ, Moher D, Bossuyt P, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. doi:10.31222/osf.io/gwdhk.

Rethlefsen M, Kirtley S, Waffenschmidt S, et al. PRISMA-S: An Extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. doi:10.31219/osf.io/sfc38.

In PRISMA 2020, there are now expanded options depending on where you search and whether you are updating a review. Version 1 of PRISMA 2020 includes databases and clinical trial or preprint registers.  Version 2 includes additional sections for elaborating on your grey literature search, such as searches on websites or in citation lists.  Both versions are available for new and updated reviews from the Equator Network's PRISMA Flow Diagram page .

Templates for New Reviews

Step 1: Preparation To complete the the PRISMA diagram, save a copy of the diagram to use alongside your searches. It can be downloaded from the PRISMA website . 

Step 2: Doing the Database Search Run the search for each database individually, including ALL your search terms, any MeSH or other subject headings, truncation (like hemipleg * ), and/or wildcards (like sul ? ur). Apply all your limits (such as years of search, English language only, and so on). Once all search terms have been combined and you have applied all relevant limits, you should have a final number of records or articles for each database. Enter this information in the top left box of the PRISMA flow chart. You should add the total number of combined results from all databases (including duplicates) after the equal sign where it says Databases (n=) . Many researchers also add notations in the box for the number of results from each database search, for example, Pubmed (n=335), Embase (n= 600), and so on.  If you search trial registers, such as ClinicalTrials.gov , CENTRAL , ICTRP , or others, you should enter that number after the equal sign in Registers (n=) .

NOTE: Some citation managers automatically remove duplicates with each file you import.  Be sure to capture the number of articles from your database searches before any duplicates are removed.

Step 3: Remove All Duplicates To avoid reviewing duplicate articles, you need to remove any articles that appear more than once in your results. You may want to export the entire list of articles from each database to a citation manager such as EndNote, Sciwheel, Zotero, or Mendeley (including both citation and abstract in your file) and remove the duplicates there. If you are using Covidence for your review, you should also add the duplicate articles identified in Covidence to the citation manager number.  Enter the number of records removed as duplicates in the second box on your PRISMA template.  If you are using automation tools to help evaluate the relevance of citations in your results, you would also enter that number here.

NOTE: If you are using Covidence to screen your articles , you can copy the numbers from the PRISMA diagram in your Covidence review into the boxes mentioned below.  Covidence does not include the number of results from each database, so you will need to keep track of that  number yourself.

Step 4: Records Screened- Title/Abstract Screening The next step is to add the number of articles that you will screen. This should be the number of records identified minus the number from the duplicates removed box.

Step 5: Records Excluded- Title/Abstract Screening You will need to screen the titles and abstracts for articles which are relevant to your research question. Any articles that appear to help you provide an answer to your research question should be included. Record the number of articles excluded through title/abstract screening in the box to the right titled "Records excluded."  You can optionally add exclusion reasons at this level, but they are not required until full text screening.

Step 6: Reports Sought for Retrieval This is the number of articles you obtain in preparation for full text screening.  Subtract the number of excluded records (Step 5) from the total number screened (Step 4) and this will be your number sought for retrieval.

Step 7: Reports Not Retrieved List the number of articles for which you are unable to find the full text.  Remember to use Find@UNC and Interlibrary Loan to request articles to see if we can order them from other libraries before automatically excluding them.

Step 8: Reports Assessed for Eligibility- Full Text Screening   This should be the number of reports sought for retrieval (Step 6) minus the number of reports not retrieved (Step 7). Review the full text for these articles to assess their eligibility for inclusion in your systematic review. 

Step 9: Reports Excluded After reviewing all articles in the full-text screening stage for eligibility, enter the total number of articles you exclude in the box titled "Reports excluded," and then list your reasons for excluding the articles as well as the number of records excluded for each reason.  Examples include wrong setting, wrong patient population, wrong intervention, wrong dosage, etc.  You should only count an excluded article once in your list even if if meets multiple exclusion criteria.

Step 10: Included Studies The final step is to subtract the number of records excluded during the eligibility review of full-texts (Step 9) from the total number of articles reviewed for eligibility (Step 8). Enter this number in the box labeled "Studies included in review," combining numbers with your grey literature search results in this box if needed.  You have now completed your PRISMA flow diagram, unless you have also performed searches in non-database sources.

To view the PRISMA diagram created after using Covidence to screen references for your review, click the PRISMA button on the main menu of your review in Covidence.

If you listed your sources when importing citations, your PRISMA diagram will include the list of databases you used and the number of references from each.

If you imported references from a citation manager, your PRISMA diagram starts with duplicate removal.  To have a complete PRISMA diagram, you will need to add the number of results from each database you searched, as well as the number of additional sources you found. 

Once you have finished title/abstract and full text screening (and data extraction or quality assessment if applicable), click Download DOCX to download your flow diagram as a Word document, or click View as text to copy and paste the PRISMA data or into an editable template for PRISMA and fill in the numbers.

There are many places articles can get lost in the review process. Remember to make sure your PRISMA numbers add up correctly!

Step 6: Included Studies The final step is to subtract the number of excluded articles or records during the eligibility review of full-texts from the total number of articles reviewed for eligibility. Enter this number in the box labeled "Studies included in review," combining numbers with your database search results in this box if needed.  You have now completed your PRISMA flow diagram, which you can now include in the results section of your article or assignment.

If you are updating an existing review, use one of these PRISMA 2020 Updated Review templates, which feature an additional box for the number of studies and reports of studies included in the previous search iterations.

• PRISMA 2020 flow diagram for updated systematic reviews- databases and registers only
• PRISMA 2020 flow diagram for updated systematic reviews- databases, registers and other sources

When referring to PRISMA 2020, The Equator Network recommends using journal article citations (such as those in our For More Information box ) rather than referring to the PRISMA website. If you are not already using a journal article citation, they recommend that you cite one of the original publications of the PRISMA Statement or PRISMA Explanation and Elaboration .

Related HSL Guides

• Systematic Reviews

Additional Readings

• Page MJ, McKenzie JE, Bossuyt PM, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement . J Clin Epidemiol. 2021;134:103-112.
• Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews . Bmj. 2021;372:n160.
• Radua J. PRISMA 2020 - An updated checklist for systematic reviews and meta-analyses. Neurosci Biobehav Rev. 2021;124:324-325.
• Sarkis-Onofre R, Catalá-López F, Aromataris E, Lockwood C. How to properly use the PRISMA Statement . Systematic reviews. 2021;10(1):117-117.
• Sohrabi C, Franchi T, Mathew G, et al. PRISMA 2020 statement: What's new and the importance of reporting guidelines. Int J Surg. 2021;88:105918.
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews . Int J Surg. 2021;88:105906.
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. J Clin Epidemiol. 2021.
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews . Bmj. 2021;372:n71.
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews . PLoS Med. 2021;18(3):e1003583.
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews . Syst Rev. 2021;10(1):89.
• Last Updated: May 14, 2024 12:49 PM
• URL: https://guides.lib.unc.edu/prisma

10 Of The Best Databases for Research

Declan Gessel

May 26, 2024

When conducting research, finding reliable sources can be challenging. Luckily, there are databases available to help you find credible information. By using databases, you can save time and ensure that your sources pass the CRAAP test . 

In this guide, I’ll introduce you to some of the best databases for research. Everyone can benefit from learning about these tools whether you're a student, a professional, or a hobbyist.

Table of Contents

What is a research database, why you should use a research database, frequently asked questions about 10 of the best databases for research.

A research database is a specialized collection of data relevant to academic or scholarly pursuits. Unlike general databases, which may contain a wide range of information, research databases focus on providing resources explicitly tailored for educational research. These resources may include scholarly articles, research papers, books, conference proceedings, and other types of publications that interest researchers and scholars in different disciplines. Research databases are precious tools for finding scholarly information. They provide access to a wide range of high-quality, reliable sources vetted by experts in the field. This curated collection of resources makes it easier for researchers to find the information they need quickly and efficiently. 

Accessing credible and current information is crucial in academic research. While general web searches can yield some results, they often lack the depth, accuracy, and reliability needed for scholarly work. Research databases address this by providing curated collections of academic articles, books, and other publications, ensuring the quality and validity of your research foundation.

Related Reading

10 Examples of Reliable Sources

Academic Search Engines

how to know if an article is peer reviewed

semantic scholar

what are scholarly sources

credible sources for research

how to use google scholar

craap method

evaluating sources

Research databases provide a curated and efficient way to locate credible and relevant sources, saving you time and effort while ensuring the quality of your research. Research databases offer powerful search functionalities that allow you to target your research precisely. You can use advanced search operators, filters, and subject headings to refine your results and quickly locate relevant articles, books, or datasets.

Efficient Searching

Unlike a general web search that might include personal blogs, opinion pieces, or outdated information, research databases focus on scholarly sources. These sources are typically peer-reviewed, meaning experts in the field have rigorously evaluated them to ensure their accuracy, methodology, and contribution to existing knowledge. This vetting process minimizes the risk of encountering misleading or inaccurate information.

Discovering Hidden Information

General search engines often prioritize popular content and websites. Research databases, however, can introduce you to a broader range of academic publications, including niche journals or conference proceedings you might not have found otherwise. This can broaden your understanding and expose you to new perspectives on your research topic.

Additional Features

Many research databases offer valuable features beyond just searching for sources. Citation analysis tools can help you identify highly influential articles in your field or track the impact of previous research. Additionally, some databases provide access to primary source materials like historical documents, datasets, or government reports, enriching your research with firsthand evidence.

Write smarter, not harder with Jotbot. Start writing for free with Jotbot today — sign in with Google and get started in seconds.

Jotbot is your personal document assistant. Jotbot does AI note-taking , AI video summarizing, AI citation/source finder, writes AI outlines for essays, and even writes entire essays with Jotbot’s AI essay writer. Join 500,000+ writers, students, teams, and researchers worldwide to write more, write better, and write faster with Jotbot. Write smarter, not harder with Jotbot. Start writing for free with Jotbot today — sign in with Google and get started in seconds.

2. Scopus & Web of Science

Citation-tracking databases with extensive coverage across numerous academic disciplines. They offer advanced search features and citation analysis tools valuable for comprehensive research.

A trusted archive providing full-text and bibliographic content in humanities, social sciences, and select scientific journals. JSTOR is particularly helpful for researchers seeking historical context in their studies.

4. PubMed/MEDLINE

Maintained by the National Institutes of Health (NIH) , PubMed/MEDLINE is the leading resource for biomedical and life sciences research. It offers unparalleled depth and breadth of medical literature, including peer-reviewed journals, clinical trials, and pre-clinical studies.

Sponsored by the U.S. Department of Education, ERIC is the go-to resource for education research. This database comprehensively indexes journals, reports, and other materials related to all aspects of education.

6. IEEE Xplore

The Institute of Electrical and Electronics Engineers (IEEE) hosts IEEE Xplore, the leading digital library for electrical engineering, computer science, and related technologies. IEEE Xplore offers a vast collection of publications, conference proceedings, and technical standards in these fields.

7. ScienceDirect

A gateway to millions of academic articles published by Elsevier, with a significant portion being open access. ScienceDirect is a valuable resource for researchers across many scientific disciplines.

8. Business Source Complete

This comprehensive database is a great starting point for business and management research. It offers various scholarly publications, news sources, and company reports.

9. ProQuest Research Library

ProQuest Research Library indexes over 5,000 journals and magazines across various disciplines, making it a versatile tool for researchers in many fields.

10. Academic Search Complete

This versatile database is a good starting point for research on various topics. Academic Search Complete offers scholarly articles across multiple subjects, from science and history to literature and business.

How do I know which database is right for my research?

The best database for your research depends on your topic and field of study. This list provides a good starting point, but it's essential to consider your research needs.

Focus on Multidisciplinary Databases

If your research topic is broad or cuts across disciplines, start with options like Scopus, Web of Science, or JSTOR.

Explore Subject-Specific Gems

Once you have a narrower focus, delve into databases catering to your field, such as PubMed/MEDLINE for medicine, ERIC for education, or IEEE Xplore for engineering and computer science.

Consult Your Librarian

Librarians are experts at navigating research resources. Don't hesitate to contact your librarian or research advisor for personalized recommendations based on your specific research needs.

ChatPDF Alternative

How to Find Peer Reviewed Articles on Google Scholar

Google Scholar Alternative

How to Find Research Papers

How to Tell if a Source Is Scholarly

Best AI for Writing Research Papers

Scholarly vs Popular Sources

Peer-Reviewed Sources

How to Find Sources for a Research Paper

Best Websites for Research Papers Free

Databases Like JSTOR

AI That Cites Sources

Are there any free options available? (h3)

Yes, there are several free options available for research databases. Here are a few to explore:

Directory of Open Access Journals (DOAJ)

This searchable directory indexes millions of open-access scholarly journals across all disciplines.

Many university libraries offer free access to various research databases for their students and faculty. Check with your university library to see what resources they provide.

Government websites

Some government agencies offer free access to databases containing valuable information relevant to their specific areas of expertise.

Write Smarter With Jotbot — Start Writing for Free Today

I am impressed by the capabilities of Jotbot, especially its AI note-taking feature. Using AI to take notes, Jotbot can automatically transcribe meetings, lectures, and interviews. Moreover, the note-taking feature can be accessed from any device, increasing flexibility and convenience. 

AI Video Summarizing  

Jotbot’s AI video summarizing tool is equally impressive. This tool can automatically summarize lengthy videos into bite-sized notes. What I particularly like about this feature is its ability to condense information effectively without losing crucial details. 

AI Citation/Source Finder 

Another commendable feature of Jotbot is its AI citation and source-finding tool. This tool scans through multiple sources to find relevant information and citations, making it easier for users to create accurate and well-researched content. 

AI Outlines for Essays 

Jotbot’s AI outline generator provides a structured framework for essays, making the writing process more organized and efficient. This feature can save time and help consistently produce high-quality essays. 

AI Essay Writer

Jotbot’s AI essay writer is a game-changer. This tool can write entire essays based on user input and requirements. The AI’s ability to understand context and generate coherent content is remarkable. With this tool, writing essays has become a breeze.

Vetting Sources

Are Blogs Scholarly Sources

How to Find Sources for an Essay

How to Find Scientific Articles

Finding Sources

Types of Scholarly Sources

Sourcely AI

Trusted by 500,000+ Students

Your documents, supercharged with AI.

Once you write with JotBot, you'll never want to write without it.

Start writing for free

Your personal document assistant.

Start for free

Press enquiries

Influencer Program

Affiliate Program

Terms & Conditions

Privacy policy

AI Source Finder

AI Outline Generator

How to Use JotBot AI

AI Note Taker

AI Video Summarizer

AI YouTube Video Summarizer

© 2023 JotBot AI by SLAM Ventures, LLC all rights reserved

© 2023 SLAM Ventures, LLC

• Systematic Review
• Open access
• Published: 23 May 2024

Systematic literature review of real-world evidence for treatments in HR+/HER2- second-line LABC/mBC after first-line treatment with CDK4/6i

• Veronique Lambert   ORCID: orcid.org/0000-0002-6984-0038 1 ,
• Sarah Kane   ORCID: orcid.org/0009-0006-9341-4836 2   na1 ,
• Belal Howidi   ORCID: orcid.org/0000-0002-1166-7631 2   na1 ,
• Bao-Ngoc Nguyen   ORCID: orcid.org/0000-0001-6026-2270 2   na1 ,
• David Chandiwana   ORCID: orcid.org/0009-0002-3499-2565 3 ,
• Yan Wu   ORCID: orcid.org/0009-0008-3348-9232 1 ,
• Michelle Edwards   ORCID: orcid.org/0009-0001-4292-3140 3 &
• Imtiaz A. Samjoo   ORCID: orcid.org/0000-0003-1415-8055 2   na1  

BMC Cancer volume  24 , Article number:  631 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Cyclin-dependent kinase 4 and 6 inhibitors (CDK4/6i) combined with endocrine therapy (ET) are currently recommended by the National Comprehensive Cancer Network (NCCN) guidelines and the European Society for Medical Oncology (ESMO) guidelines as the first-line (1 L) treatment for patients with hormone receptor-positive, human epidermal growth factor receptor 2-negative, locally advanced/metastatic breast cancer (HR+/HER2- LABC/mBC). Although there are many treatment options, there is no clear standard of care for patients following 1 L CDK4/6i. Understanding the real-world effectiveness of subsequent therapies may help to identify an unmet need in this patient population. This systematic literature review qualitatively synthesized effectiveness and safety outcomes for treatments received in the real-world setting after 1 L CDK4/6i therapy in patients with HR+/ HER2- LABC/mBC.

MEDLINE®, Embase, and Cochrane were searched using the Ovid® platform for real-world evidence studies published between 2015 and 2022. Grey literature was searched to identify relevant conference abstracts published from 2019 to 2022. The review was conducted in accordance with PRISMA guidelines (PROSPERO registration: CRD42023383914). Data were qualitatively synthesized and weighted average median real-world progression-free survival (rwPFS) was calculated for NCCN/ESMO-recommended post-1 L CDK4/6i treatment regimens.

Twenty records (9 full-text articles and 11 conference abstracts) encompassing 18 unique studies met the eligibility criteria and reported outcomes for second-line (2 L) treatments after 1 L CDK4/6i; no studies reported disaggregated outcomes in the third-line setting or beyond. Sixteen studies included NCCN/ESMO guideline-recommended treatments with the majority evaluating endocrine-based therapy; five studies on single-agent ET, six studies on mammalian target of rapamycin inhibitors (mTORi) ± ET, and three studies with a mix of ET and/or mTORi. Chemotherapy outcomes were reported in 11 studies. The most assessed outcome was median rwPFS; the weighted average median rwPFS was calculated as 3.9 months (3.3-6.0 months) for single-agent ET, 3.6 months (2.5–4.9 months) for mTORi ± ET, 3.7 months for a mix of ET and/or mTORi (3.0–4.0 months), and 6.1 months (3.7–9.7 months) for chemotherapy. Very few studies reported other effectiveness outcomes and only two studies reported safety outcomes. Most studies had heterogeneity in patient- and disease-related characteristics.

Conclusions

The real-world effectiveness of current 2 L treatments post-1 L CDK4/6i are suboptimal, highlighting an unmet need for this patient population.

Peer Review reports

Introduction

Breast cancer (BC) is the most diagnosed form of cancer in women with an estimated 2.3 million new cases diagnosed worldwide each year [ 1 ]. BC is the second leading cause of cancer death, accounting for 685,000 deaths worldwide per year [ 2 ]. By 2040, the global burden associated with BC is expected to surpass three million new cases and one million deaths annually (due to population growth and aging) [ 3 ]. Numerous factors contribute to global disparities in BC-related mortality rates, including delayed diagnosis, resulting in a high number of BC cases that have progressed to locally advanced BC (LABC) or metastatic BC (mBC) [ 4 , 5 , 6 ]. In the United States (US), the five-year survival rate for patients who progress to mBC is three times lower (31%) than the overall five-year survival rate for all stages (91%) [ 6 , 7 ].

Hormone receptor (HR) positive (i.e., estrogen receptor and/or progesterone receptor positive) coupled with negative human epidermal growth factor 2 (HER2) expression is the most common subtype of BC, accounting for ∼ 60–70% of all BC cases [ 8 , 9 ]. Historically, endocrine therapy (ET) through estrogen receptor modulation and/or estrogen deprivation has been the standard of care for first-line (1 L) treatment of HR-positive/HER2-negative (HR+/HER2-) mBC [ 10 ]. However, with the approval of the cyclin-dependent kinase 4/6 inhibitor (CDK4/6i) palbociclib in combination with the aromatase inhibitor (AI) letrozole in 2015 by the US Food and Drug Administration (FDA), 1 L treatment practice patterns have evolved such that CDK4/6i (either in combination with AIs or with fulvestrant) are currently considered the standard of care [ 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. Other CDK4/6i (ribociclib and abemaciclib) in combination with ET are approved for the treatment of HR+/HER2- LABC/mBC; 1 L use of ribociclib in combination with an AI was granted FDA approval in March 2017 for postmenopausal women (with expanded approval in July 2018 for pre/perimenopausal women and for use in 1 L with fulvestrant for patients with disease progression on ET as well as for postmenopausal women), and abemaciclib in combination with fulvestrant was granted FDA approval in September 2017 for patients with disease progression following ET and as monotherapy in cases where disease progression occurs following ET and prior chemotherapy in mBC (with expanded approval in February 2018 for use in 1 L in combination with an AI for postmenopausal women) [ 18 , 19 , 20 , 21 ].

Clinical trials investigating the addition of CDK4/6i to ET have demonstrated significant improvement in progression-free survival (PFS) and significant (ribociclib) or numerical (palbociclib and abemaciclib) improvement in overall survival (OS) compared to ET alone in patients with HR+/HER2- advanced or mBC, making this combination treatment the recommended option in the 1 L setting [ 22 , 23 , 24 , 25 , 26 , 27 ]. However, disease progression occurs in a significant portion of patients after 1 L CDK4/6i treatment [ 28 ] and the optimal treatment sequence after progression on CDK4/6i remains unclear [ 29 ]. At the time of this review (literature search conducted December 14, 2022), guidelines by the National Comprehensive Cancer Network (NCCN) and the European Society for Medical Oncology (ESMO) recommend various options for the treatment of HR+/HER2- advanced BC in the second-line (2 L) setting, including fulvestrant monotherapy, mammalian target of rapamycin inhibitors (mTORi; e.g., everolimus) ± ET, alpelisib + fulvestrant (if phosphatidylinositol-4,5-bisphosphate 3-kinase catalytic subunit alpha mutation positive [PIK3CA-m+]), poly-ADP ribose polymerase inhibitors (PARPi) including olaparib or talazoparib (if breast cancer gene/partner and localizer of BRCA2 positive [BRCA/PALB2m+]), and chemotherapy (in cases when a visceral crisis is present) [ 15 , 16 ]. CDK4/6i can also be used in 2 L [ 16 , 30 ]; however, limited data are available to support CDK4/6i rechallenge after its use in the 1 L setting [ 15 ]. Depending on treatments used in the 1 L and 2 L settings, treatment in the third-line setting is individualized based on the patient’s response to prior treatments, tumor load, duration of response, and patient preference [ 9 , 15 ]. Understanding subsequent treatments after 1 L CDK4/6i, and their associated effectiveness, is an important focus in BC research.

Treatment options for HR+/HER2- LABC/mBC continue to evolve, with ongoing research in both clinical trials and in the real-world setting. Real-world evidence (RWE) offers important insights into novel therapeutic regimens and the effectiveness of treatments for HR+/HER2- LABC/mBC. The effectiveness of the current treatment options following 1 L CDK4/6i therapy in the real-world setting highlights the unmet need in this patient population and may help to drive further research and drug development. In this study, we conducted a systematic literature review (SLR) to qualitatively summarize the effectiveness and safety of treatment regimens in the real-world setting after 1 L treatment with CDK4/6i in patients with HR+/HER2- LABC/mBC.

Literature search

An SLR was performed in accordance with the Cochrane Handbook for Systematic Reviews of Interventions [ 31 ] and reported in alignment with the Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses (PRISMA) statement [ 32 ] to identify all RWE studies assessing the effectiveness and safety of treatments used for patients with HR+/HER2- LABC/mBC following 1 L CDK4/6i therapy and received subsequent treatment in 2 L and beyond (2 L+). The Ovid® platform was used to search MEDLINE® (including Epub Ahead of Print and In-Process, In-Data-Review & Other Non-Indexed Citations), Ovid MEDLINE® Daily, Embase, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews by an experienced medical information specialist. The MEDLINE® search strategy was peer-reviewed independently by a senior medical information specialist before execution using the Peer Review of Electronic Search Strategies (PRESS) checklist [ 33 ]. Searches were conducted on December 14, 2022. The review protocol was developed a priori and registered with the International Prospective Register of Systematic Review (PROSPERO; CRD42023383914) which outlined the population, intervention, comparator, outcome, and study design (PICOS) criteria and methodology used to conduct the review (Table  1 ).

Search strategies utilized a combination of controlled vocabulary (e.g., “HER2 Breast Cancer” or “HR Breast Cancer”) and keywords (e.g., “Retrospective studies”). Vocabulary and syntax were adjusted across databases. Published and validated filters were used to select for study design and were supplemented using additional medical subject headings (MeSH) terms and keywords to select for RWE and nonrandomized studies [ 34 ]. No language restrictions were included in the search strategy. Animal-only and opinion pieces were removed from the results. The search was limited to studies published between January 2015 and December 2022 to reflect the time at which FDA approval was granted for the first CDK4/6i agent (palbociclib) in combination with AI for the treatment of LABC/mBC [ 35 ]. Further search details are presented in Supplementary Material 1 .

Grey literature sources were also searched to identify relevant abstracts and posters published from January 2019 to December 2022 for prespecified relevant conferences including ESMO, San Antonio Breast Cancer Symposium (SABCS), American Society of Clinical Oncology (ASCO), the International Society for Pharmacoeconomics and Outcomes Research (ISPOR US), and the American Association for Cancer Research (AACR). A search of ClinicalTrials.gov was conducted to validate the findings from the database and grey literature searches.

Study selection, data extraction & weighted average calculation

Studies were screened for inclusion using DistillerSR Version 2.35 and 2.41 (DistillerSR Inc. 2021, Ottawa, Canada) by two independent reviewers based on the prespecified PICOS criteria (Table  1 ). A third reviewer was consulted to resolve any discrepancies during the screening process. Studies were included if they reported RWE on patients aged ≥ 18 years with HR+/HER2- LABC/mBC who received 1 L CDK4/6i treatment and received subsequent treatment in 2 L+. Studies were excluded if they reported the results of clinical trials (i.e., non-RWE), were published in any language other than English, and/or were published prior to 2015 (or prior to 2019 for conference abstracts and posters). For studies that met the eligibility criteria, data relating to study design and methodology, details of interventions, patient eligibility criteria and baseline characteristics, and outcome measures such as efficacy, safety, tolerability, and patient-reported outcomes (PROs), were extracted (as available) using a Microsoft Excel®-based data extraction form (Microsoft Corporation, WA, USA). Data extraction was performed by a single reviewer and was confirmed by a second reviewer. Multiple publications identified for the same RWE study, patient population, and setting that reported data for the same intervention were linked and extracted as a single publication. Weighted average median real-world progression-free survival (rwPFS) values were calculated by considering the contribution to the median rwPFS of each study proportional to its respective sample size. These weighted values were then used to compute the overall median rwPFS estimate.

Quality assessment

The Newcastle-Ottawa scale (NOS) for nonrandomized (cohort) studies was used to assess the risk of bias for published, full-text studies [ 36 ]. The NOS allocates a maximum of nine points for the least risk of bias across three domains: (1) Formation of study groups (four points), (2) Comparability between study groups (two points), (3) Outcome ascertainment (three points). NOS scores can be categorized in three groups: very high risk of bias (0 to 3 points), high risk of bias (4 to 6), and low risk of bias (7 to 9) [ 37 ]. Risk of bias assessment was performed by one reviewer and validated by a second independent reviewer to verify accuracy. Due to limited methodological data by which to assess study quality, risk of bias assessment was not performed on conference abstracts or posters. An amendment to the PROSPERO record (CRD42023383914) for this study was submitted in relation to the quality assessment method (specifying usage of the NOS).

The database search identified 3,377 records; after removal of duplicates, 2,759 were screened at the title and abstract stage of which 2,553 were excluded. Out of the 206 reports retrieved and assessed for eligibility, an additional 187 records were excluded after full-text review; most of these studies were excluded for having patients with mixed lines of CDK4/6i treatment (i.e., did not receive CDK4/6i exclusively in 1 L) (Fig.  1 and Table S1 ). The grey literature search identified 753 records which were assessed for eligibility; of which 752 were excluded mainly due to the population not meeting the eligibility criteria (Fig.  1 ). In total, the literature searches identified 20 records (9 published full-text articles and 11 conference abstracts/posters) representing 18 unique RWE studies that met the inclusion criteria. The NOS quality scores for the included full-text articles are provided in Table S2 . The scores ranged from four to six points (out of a total score of nine) and the median score was five, indicating that all the studies suffered from a high risk of bias [ 37 ].

Most studies were retrospective analyses of chart reviews or medical registries, and all studies were published between 2017 and 2022 (Table S3 ). Nearly half of the RWE studies (8 out of 18 studies) were conducted in the US [ 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ], while the remaining studies included sites in Canada, China, Germany, Italy, Japan, and the United Kingdom [ 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 ]. Sample sizes ranged from as few as 4 to as many as 839 patients across included studies, with patient age ranging from 26 to 86 years old.

Although treatment characteristics in the 1 L setting were not the focus of the present review, these details are captured in Table S3 . Briefly, several RWE studies reported 1 L CDK4/6i use in combination with ET (8 out of 18 studies) or as monotherapy (2 out of 18 studies) (Table S3 ). Treatments used in combination with 1 L CDK4/6i included letrozole, fulvestrant, exemestane, and anastrozole. Where reported (4 out of 18 studies), palbociclib was the most common 1 L CDK4/6i treatment. Many studies (8 out of 18 studies) did not report which specific CDK4/6i treatment(s) were used in 1 L or if its administration was in combination or monotherapy.

Characteristics of treatments after 1 L CDK4/6i therapy

Across all studies included in this review, effectiveness and safety data were only available for treatments administered in the 2 L setting after 1 L CDK4/6i treatment. No studies were identified that reported outcomes for patients treated in the third-line setting or beyond after 1 L CDK4/6i treatment. All 18 studies reported effectiveness outcomes in 2 L, with only two of these studies also describing 2 L safety outcomes. The distribution of outcomes reported in these studies is provided in Table S4 . Studies varied in their reporting of outcomes for 2 L treatments; some studies reported outcomes for a group of 2 L treatments while others described independent outcomes for specific 2 L treatments (i.e., everolimus, fulvestrant, or chemotherapy agents such as eribulin mesylate) [ 42 , 45 , 50 , 54 , 55 ]. Due to the heterogeneity in treatment classes reported in these studies, this data was categorized (as described below) to align with the guidelines provided by NCCN and ESMO [ 15 , 16 ]. The treatment class categorizations for the purpose of this review are: single-agent ET (patients who exclusively received a single-agent ET after 1 L CDK4/6i treatment), mTORi ± ET (patients who exclusively received an mTORi with or without ET after 1 L CDK4/6i treatment), mix of ET and/or mTORi (patients who may have received only ET, only mTORi, and/or both treatments but the studies in this group lacked sufficient information to categorize these patients in the “single-agent ET” or “mTOR ± ET” categories), and chemotherapy (patients who exclusively received chemotherapy after 1 L CDK4/6i treatment). Despite ESMO and NCCN guidelines indicating that limited evidence exists to support rechallenge with CDK4/6i after 1 L CDK4/6i treatment [ 15 , 16 ], two studies reported outcomes for this treatment approach. Data for such patients were categorized as “ CDK4/6i ± ET ” as it was unclear how many patients receiving CDK4/6i rechallenge received concurrent ET. All other patient groups that lacked sufficient information or did not report outcome/safety data independently (i.e., grouped patients with mixed treatments) to categorize as one of the treatment classes described above were grouped as “ other ”.

The majority of studies reported effectiveness outcomes for endocrine-based therapy after 1 L CDK4/6i treatment; five studies for single-agent ET, six studies for mTORi ± ET, and three studies for a mix of ET and/or mTORi (Fig.  2 ). Eleven studies reported effectiveness outcomes for chemotherapy after 1 L CDK4/6i treatment, and only two studies reported effectiveness outcomes for CDK4/6i rechallenge ± ET. Eight studies that described effectiveness outcomes were grouped into the “other” category. Safety data was only reported in two studies: one study evaluating the chemotherapy agent eribulin mesylate and one evaluating the mTORi everolimus.

Effectiveness outcomes

Real-world progression-free survival

Median rwPFS was described in 13 studies (Tables  2 and Table S5 ). Across the 13 studies, the median rwPFS ranged from 2.5 months [ 49 ] to 17.3 months [ 39 ]. Out of the 13 studies reporting median rwPFS, 10 studies reported median rwPFS for a 2 L treatment recommended by ESMO and NCCN guidelines, which ranged from 2.5 months [ 49 ] to 9.7 months [ 45 ].

Weighted average median rwPFS was calculated for 2 L treatments recommended by both ESMO and NCCN guidelines (Fig.  3 ). The weighted average median rwPFS for single-agent ET was 3.9 months ( n  = 92 total patients) and was derived using data from two studies reporting median rwPFS values of 3.3 months ( n  = 70) [ 38 ] and 6.0 months ( n  = 22) [ 40 ]. For one study ( n  = 7) that reported outcomes for single agent ET, median rwPFS was not reached during the follow-up period; as such, this study was excluded from the weighted average median rwPFS calculation [ 49 ].

The weighted average median rwPFS for mTORi ± ET was 3.6 months ( n  = 128 total patients) and was derived based on data from 3 studies with median rwPFS ranging from 2.5 months ( n  = 4) [ 49 ] to 4.9 months ( n  = 25) [ 54 ] (Fig.  3 ). For patients who received a mix of ET and/or mTORi but could not be classified into the single-agent ET or mTORi ± ET treatment classes, the weighted average median rwPFS was calculated to be 3.7 months ( n  = 17 total patients). This was calculated based on data from two studies reporting median rwPFS values of 3.0 months ( n  = 5) [ 46 ] and 4.0 months ( n  = 12) [ 49 ]. Notably, one study of patients receiving ET and/or everolimus reported a median rwPFS duration of 3.0 months; however, this study was excluded from the weighted average median rwPFS calculation for the ET and/or mTORi class as the sample size was not reported [ 53 ].

The weighted average median rwPFS for chemotherapy was 6.1 months ( n  = 499 total patients), calculated using data from 7 studies reporting median rwPFS values ranging from 3.7 months ( n  = 249) [ 38 ] to 9.7 months ( n  = 121) [ 45 ] (Fig.  3 ). One study with a median rwPFS duration of 5.6 months was not included in the weighted average median rwPFS calculation as the study did not report the sample size [ 53 ]. A second study was excluded from the calculation since the reported median rwPFS was not reached during the study period ( n  = 7) [ 41 ].

Although 2 L CDK4/6i ± ET rechallenge lacks sufficient information to support recommendation by ESMO and NCCN guidelines, the limited data currently available for this treatment have shown promising results. Briefly, two studies reported median rwPFS for CDK4/6i ± ET with values of 8.3 months ( n  = 302) [ 38 ] and 17.3 months ( n  = 165) (Table  2 ) [ 39 ]. The remaining median rwPFS studies reported data for patients classified as “Other” (Table S5 ). The “Other” category included median rwPFS outcomes from seven studies, and included a myriad of treatments (e.g., ET, mTOR + ET, chemotherapy, CDK4/6i + ET, alpelisib + fulvestrant, chidamide + ET) for which disaggregated median rwPFS values were not reported.

Overall survival

Median OS for 2 L treatment was reported in only three studies (Table  2 ) [ 38 , 42 , 43 ]. Across the three studies, the 2 L median OS ranged from 5.2 months ( n  = 3) [ 43 ] to 35.7 months ( n  = 302) [ 38 ]. Due to the lack of OS data in most of the studies, weighted averages could not be calculated. No median OS data was reported for the single-agent ET treatment class whereas two studies reported median OS for the mTORi ± ET treatment class, ranging from 5.2 months ( n  = 3) [ 43 ] to 21.8 months ( n  = 54) [ 42 ]. One study reported 2 L median OS of 24.8 months for a single patient treated with chemotherapy [ 43 ]. The median OS data in the CDK4/6i ± ET rechallenge group was 35.7 months ( n  = 302) [ 38 ].

Patient mortality was reported in three studies [ 43 , 44 , 45 ]. No studies reported mortality for the single-agent ET treatment class and only one study reported this outcome for the mTORi ± ET treatment class, where 100% of patients died ( n  = 3) as a result of rapid disease progression [ 43 ]. For the chemotherapy class, one study reported mortality for one patient receiving 2 L capecitabine [ 43 ]. An additional study reported eight deaths (21.7%) following 1 L CDK4/6i treatment; however, this study did not disclose the 2 L treatments administered to these patients [ 44 ].

Other clinical endpoints

The studies included limited information on additional clinical endpoints; two studies reported on time-to-discontinuation (TTD), two reported on duration of response (DOR), and one each on time-to-next-treatment (TTNT), time-to-progression (TTP), objective response rate (ORR), clinical benefit rate (CBR), and stable disease (Tables  2 and Table S5 ).

Safety, tolerability, and patient-reported outcomes

Safety and tolerability data were reported in two studies [ 40 , 45 ]. One study investigating 2 L administration of the chemotherapy agent eribulin mesylate reported 27 patients (22.3%) with neutropenia, 3 patients (2.5%) with febrile neutropenia, 10 patients (8.3%) with peripheral neuropathy, and 14 patients (11.6%) with diarrhea [ 45 ]. Of these, neutropenia of grade 3–4 severity occurred in 9 patients (33.3%) [ 45 ]. A total of 55 patients (45.5%) discontinued eribulin mesylate treatment; 1 patient (0.83%) discontinued treatment due to adverse events [ 45 ]. Another study reported that 5 out of the 22 patients receiving the mTORi everolimus combined with ET in 2 L (22.7%) discontinued treatment due to toxicity [ 40 ]. PROs were not reported in any of the studies included in the SLR.

The objective of this study was to summarize the existing RWE on the effectiveness and safety of therapies for patients with HR+/HER2- LABC/mBC after 1 L CDK4/6i treatment. We identified 18 unique studies reporting specifically on 2 L treatment regimens after 1 L CDK4/6i treatment. The weighted average median rwPFS for NCCN- and ESMO- guideline recommended 2 L treatments ranged from 3.6 to 3.9 months for ET-based treatments and was 6.1 months when including chemotherapy-based regimens. Treatment selection following 1 L CDK4/6i therapy remains challenging primarily due to the suboptimal effectiveness or significant toxicities (e.g., chemotherapy) associated with currently available options [ 56 ]. These results highlight that currently available 2 L treatments for patients with HR+/HER2- LABC/mBC who have received 1 L CDK4/6i are suboptimal, as evidenced by the brief median rwPFS duration associated with ET-based treatments, or notable side effects and toxicity linked to chemotherapy. This conclusion is aligned with a recent review highlighting the limited effectiveness of treatment options for HR+/HER2- LABC/mBC patients post-CDK4/6i treatment [ 56 , 57 ]. Registrational trials which have also shed light on the short median PFS of 2–3 months achieved by ET (i.e., fulvestrant) after 1 L CDK4/6i therapy emphasize the need to develop improved treatment strategies aimed at prolonging the duration of effective ET-based treatment [ 56 ].

The results of this review reveal a paucity of additional real-world effectiveness and safety evidence after 1 L CDK4/6i treatment in HR+/HER2- LABC/mBC. OS and DOR were only reported in two studies while other clinical endpoints (i.e., TTD, TTNT, TTP, ORR, CBR, and stable disease) were only reported in one study each. Similarly, safety and tolerability data were only reported in two studies each, and PROs were not reported in any study. This hindered our ability to provide a comprehensive assessment of real-world treatment effectiveness and safety following 1 L CDK4/6i treatment. The limited evidence may be due to the relatively short period of time that has elapsed since CDK4/6i first received US FDA approval for 1 L treatment of HR+/HER2- LABC/mBC (2015) [ 35 ]. As such, almost half of our evidence was informed by conference abstracts. Similarly, no real-world studies were identified in our review that reported outcomes for treatments in the third- or later-lines of therapy after 1 L CDK4/6i treatment. The lack of data in this patient population highlights a significant gap which limits our understanding of the effectiveness and safety for patients receiving later lines of therapy. As more patients receive CDK4/6i therapy in the 1 L setting, the number of patients requiring subsequent lines of therapy will continue to grow. Addressing this data gap over time will be critical to improve outcomes for patients with HR+/HER2- LABC/mBC following 1 L CDK4/6i therapy.

There are several strengths of this study, including adherence to the guidelines outlined in the Cochrane Handbook to ensure a standardized and reliable approach to the SLR [ 58 ] and reporting of the SLR following PRISMA guidelines to ensure transparency and reproducibility [ 59 ]. Furthermore, the inclusion of only RWE studies allowed us to assess the effectiveness of current standard of care treatments outside of a controlled environment and enabled us to identify an unmet need in this patient population.

This study had some notable limitations, including the lack of safety and additional effectiveness outcomes reported. In addition, the dearth of studies reporting PROs is a limitation, as PROs provide valuable insight into the patient experience and are an important aspect of assessing the impact of 2 L treatments on patients’ quality of life. The studies included in this review also lacked consistent reporting of clinical characteristics (e.g., menopausal status, sites of metastasis, prior surgery) making it challenging to draw comprehensive conclusions or comparisons based on these factors across the studies. Taken together, there exists an important gap in our understanding of the long-term management of patients with HR+/HER2- LABC/mBC. Additionally, the effectiveness results reported in our evidence base were informed by small sample sizes; many of the included studies reported median rwPFS based on less than 30 patients [ 39 , 40 , 41 , 46 , 49 , 51 , 60 ], with two studies not reporting the sample size at all [ 47 , 53 ]. This may impact the generalizability and robustness of the results. Relatedly, the SLR database search was conducted in December 2022; as such, novel agents (e.g., elacestrant and capivasertib + fulvestrant) that have since received FDA approval for the treatment of HR+/HER2- LABC/mBC may impact current 2 L rwPFS outcomes [ 61 , 62 ]. Finally, relative to the number of peer-reviewed full-text articles, this SLR identified eight abstracts and one poster presentation, comprising half (50%) of the included unique studies. As conference abstracts are inherently limited by how much content that can be described due to word limit constraints, this likely had implications on the present synthesis whereby we identified a dearth of real-world effectiveness outcomes in patients with HR+/HER2- LABC/mBC treated with 1 L CDK4/6i therapy.

Future research in this area should aim to address the limitations of the current literature and provide a more comprehensive understanding of optimal sequencing of effective and safe treatment for patients following 1 L CDK4/6i therapy. Specifically, future studies should strive to report robust data related to effectiveness, safety, and PROs for patients receiving 2 L treatment after 1 L CDK4/6i therapy. Future studies should also aim to understand the mechanism underlying CDK4/6i resistance. Addressing these gaps in knowledge may improve the long-term real-world management of patients with HR+/HER2- LABC/mBC. A future update of this synthesis may serve to capture a wider breadth of full-text, peer-reviewed articles to gain a more robust understanding of the safety, effectiveness, and real-world treatment patterns for patients with HR+/HER2- LABC/mBC. This SLR underscores the necessity for ongoing investigation and the development of innovative therapeutic approaches to address these gaps and improve patient outcomes.

This SLR qualitatively summarized the existing real-world effectiveness data for patients with HR+/HER2- LABC/mBC after 1 L CDK4/6i treatment. Results of this study highlight the limited available data and the suboptimal effectiveness of treatments employed in the 2 L setting and underscore the unmet need in this patient population. Additional studies reporting effectiveness and safety outcomes, in addition to PROs, for this patient population are necessary and should be the focus of future research.

PRISMA flow diagram. *Two included conference abstracts reported the same information as already included full-text reports, hence both conference abstracts were not identified as unique. Abbreviations: 1 L = first-line; AACR = American Association of Cancer Research; ASCO = American Society of Clinical Oncology; CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ESMO = European Society for Medical Oncology; ISPOR = Professional Society for Health Economics and Outcomes Research; n = number of studies; NMA = network meta-analysis; pts = participants; SABCS = San Antonio Breast Cancer Symposium; SLR = systematic literature review.

Number of studies reporting effectiveness outcomes exclusively for each treatment class. *Studies that lack sufficient information on effectiveness outcomes to classify based on the treatment classes outlined in the legend above. Abbreviations: CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ET = endocrine therapy; mTORi = mammalian target of rapamycin inhibitor.

Weighted average median rwPFS for 2 L treatments (recommended in ESMO/NCCN guidelines) after 1 L CDK4/6i treatment. Circular dot represents weighted average median across studies. Horizontal bars represent the range of values reported in these studies. Abbreviations: CDK4/6i = cyclin-dependent kinase 4/6 inhibitor; ESMO = European Society for Medical Oncology; ET = endocrine therapy, mTORi = mammalian target of rapamycin inhibitor; n = number of patients; NCCN = National Comprehensive Cancer Network; rwPFS = real-world progression-free survival.

Data availability

All data generated or analyzed during this study are included in this published article [and its supplementary information files]. This study is registered with PROSPERO (CRD42023383914).

Abbreviations

Second-line

Second-line treatment setting and beyond

American Association of Cancer Research

Aromatase inhibitor

American Society of Clinical Oncology

• Breast cancer

breast cancer gene/partner and localizer of BRCA2 positive

Clinical benefit rate

Cyclin-dependent kinase 4/6 inhibitor

Complete response

Duration of response

European Society for Medical Oncology

Food and Drug Administration

Human epidermal growth factor receptor 2

Human epidermal growth factor receptor 2 negative

Hormone receptor

Hormone receptor positive

Professional Society for Health Economics and Outcomes Research

Locally advanced breast cancer

Metastatic breast cancer

Medical Literature Analysis and Retrieval System Online

Medical subject headings

Mammalian target of rapamycin inhibitor

National Comprehensive Cancer Network

Newcastle Ottawa Scale

Objective response rate

Poly-ADP ribose polymerase inhibitor

Progression-free survival

Population, Intervention, Comparator, Outcome, Study Design

Partial response

Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses

Patient-reported outcomes

• Real-world evidence

San Antonio Breast Cancer Symposium

• Systematic literature review

Time-to-discontinuation

Time-to-next-treatment

Time-to-progression

United States

Łukasiewicz S, Czeczelewski M, Forma A, Baj J, Sitarz R, Stanisławek A, Breast, Cancer—Epidemiology. Risk factors, classification, prognostic markers, and current treatment Strategies—An. Updated Rev Cancers. 2021;13(17):4287.

Google Scholar  

World Health Organization (WHO). Breast Cancer Facts Sheet [updated July 12 2023. https://www.who.int/news-room/fact-sheets/detail/breast-cancer .

Arnold M, Morgan E, Rumgay H, Mafra A, Singh D, Laversanne M, et al. Current and future burden of breast cancer: global statistics for 2020 and 2040. Breast. 2022;66:15–23.

Article   PubMed   PubMed Central   Google Scholar  

Wilkinson L, Gathani T. Understanding breast cancer as a global health concern. Br J Radiol. 2022;95(1130):20211033.

Article   PubMed   Google Scholar  

Giaquinto AN, Sung H, Miller KD, Kramer JL, Newman LA, Minihan A et al. Breast Cancer Statistics, 2022. CA: A Cancer Journal for Clinicians. 2022;72(6):524– 41.

National Cancer Institute (NIH). Cancer Stat Facts: Female Breast Cancer [updated 2020. https://seer.cancer.gov/statfacts/html/breast.html .

American Cancer Society. Key Statistics for Breast Cancer [ https://www.cancer.org/cancer/types/breast-cancer/about/how-common-is-breast-cancer.html .

Zagami P, Carey LA. Triple negative breast cancer: pitfalls and progress. npj Breast Cancer. 2022;8(1):95.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Matutino A, Joy AA, Brezden-Masley C, Chia S, Verma S. Hormone receptor-positive, HER2-negative metastatic breast cancer: redrawing the lines. Curr Oncol. 2018;25(Suppl 1):S131–41.

Lloyd MR, Wander SA, Hamilton E, Razavi P, Bardia A. Next-generation selective estrogen receptor degraders and other novel endocrine therapies for management of metastatic hormone receptor-positive breast cancer: current and emerging role. Ther Adv Med Oncol. 2022;14:17588359221113694.

Cardoso F, Senkus E, Costa A, Papadopoulos E, Aapro M, André F, et al. 4th ESO-ESMO International Consensus guidelines for advanced breast Cancer (ABC 4)†. Ann Oncol. 2018;29(8):1634–57.

Article   CAS   PubMed   Google Scholar  

US Food Drug Administration. Palbociclib (Ibrance) 2017 [updated March 31, 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/palbociclib-ibrance .

US Food Drug Administration. FDA expands ribociclib indication in HR-positive, HER2-negative advanced or metastatic breast cancer 2018 [updated July 18. 2018. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-expands-ribociclib-indication-hr-positive-her2-negative-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves abemaciclib for HR positive, HER2-negative breast cancer 2017 [updated Sept 28. 2017. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abemaciclib-hr-positive-her2-negative-breast-cancer .

NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®). Breast Cancer 2022 [ https://www.nccn.org/professionals/physician_gls/pdf/breast.pdf .

Gennari A, André F, Barrios CH, Cortés J, de Azambuja E, DeMichele A, et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Ann Oncol. 2021;32(12):1475–95.

Beaver JA, Amiri-Kordestani L, Charlab R, Chen W, Palmby T, Tilley A, et al. FDA approval: Palbociclib for the Treatment of Postmenopausal Patients with estrogen Receptor-Positive, HER2-Negative metastatic breast Cancer. Clin Cancer Res. 2015;21(21):4760–6.

US Food Drug Administration. Ribociclib (Kisqali) [ https://www.fda.gov/drugs/resources-information-approved-drugs/ribociclib-kisqali#:~:text=On%20March%2013%2C%202017%2C%20the,hormone%20receptor%20(HR)%2Dpositive%2C .

US Food Drug Administration. FDA approves new treatment for certain advanced or metastatic breast cancers [ https://www.fda.gov/news-events/press-announcements/fda-approves-new-treatment-certain-advanced-or-metastatic-breast-cancers .

US Food Drug Administration. FDA expands ribociclib indication in HR-positive, HER2-negative advanced or metastatic breast cancer. 2018 [ https://www.fda.gov/drugs/resources-information-approved-drugs/fda-expands-ribociclib-indication-hr-positive-her2-negative-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves abemaciclib as initial therapy for HR-positive, HER2-negative metastatic breast cancer [ https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abemaciclib-initial-therapy-hr-positive-her2-negative-metastatic-breast-cancer .

Turner NC, Slamon DJ, Ro J, Bondarenko I, Im S-A, Masuda N, et al. Overall survival with Palbociclib and fulvestrant in advanced breast Cancer. N Engl J Med. 2018;379(20):1926–36.

Slamon DJ, Neven P, Chia S, Fasching PA, De Laurentiis M, Im SA, et al. Phase III randomized study of Ribociclib and Fulvestrant in hormone Receptor-Positive, human epidermal growth factor receptor 2-Negative advanced breast Cancer: MONALEESA-3. J Clin Oncol. 2018;36(24):2465–72.

Goetz MP, Toi M, Campone M, Sohn J, Paluch-Shimon S, Huober J, et al. MONARCH 3: Abemaciclib as initial therapy for advanced breast Cancer. J Clin Oncol. 2017;35(32):3638–46.

Gopalan PK, Villegas AG, Cao C, Pinder-Schenck M, Chiappori A, Hou W, et al. CDK4/6 inhibition stabilizes disease in patients with p16-null non-small cell lung cancer and is synergistic with mTOR inhibition. Oncotarget. 2018;9(100):37352–66.

Watt AC, Goel S. Cellular mechanisms underlying response and resistance to CDK4/6 inhibitors in the treatment of hormone receptor-positive breast cancer. Breast Cancer Res. 2022;24(1):17.

Goetz M. MONARCH 3: final overall survival results of abemaciclib plus a nonsteroidal aromatase inhibitor as first-line therapy for HR+, HER2- advanced breast cancer. SABCS; 2023.

Munzone E, Pagan E, Bagnardi V, Montagna E, Cancello G, Dellapasqua S, et al. Systematic review and meta-analysis of post-progression outcomes in ER+/HER2– metastatic breast cancer after CDK4/6 inhibitors within randomized clinical trials. ESMO Open. 2021;6(6):100332.

Gennari A, André F, Barrios CH, Cortés J, de Azambuja E, DeMichele A, et al. ESMO Clinical Practice Guideline for the diagnosis, staging and treatment of patients with metastatic breast cancer. Annals of Oncology. 2021;32(12):1475-95.

European Society for Medical Oncology (ESMO). ESMO ​Metastatic Breast Cancer Living Guideline: ER-positive HER2-negative​ Breast Cancer​ [updated May 2023. https://www.esmo.org/living-guidelines/esmo-metastatic-breast-cancer-living-guideline/er-positive-her2-negative-breast-cancer .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Welch PM VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). www.training.cochrane.org/handbook : Cochrane; 2021.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

Fraser C, Murray A, Burr J. Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Med Res Methodol. 2006;6(1):41.

US Food Drug Administration. Palbociclib (Ibrance). Silver Spring, MD: US Food and Drug Administration; 2017.

Book   Google Scholar  

GA Wells BS, D O’Connell J, Peterson V, Welch M, Losos PT. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [ https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp .

Lo CK-L, Mertz D, Loeb M. Newcastle-Ottawa Scale: comparing reviewers’ to authors’ assessments. BMC Med Res Methodol. 2014;14(1):45.

Martin JM, Handorf EA, Montero AJ, Goldstein LJ. Systemic therapies following progression on first-line CDK4/6-inhibitor treatment: analysis of real-world data. Oncologist. 2022;27(6):441–6.

Kalinsky KM, Kruse M, Smyth EN, Guimaraes CM, Gautam S, Nisbett AR et al. Abstract P1-18-37: Treatment patterns and outcomes associated with sequential and non-sequential use of CDK4 and 6i for HR+, HER2- MBC in the real world. Cancer Research. 2022;82(4_Supplement):P1-18-37-P1-18-37.

Choong GM, Liddell S, Ferre RAL, O’Sullivan CC, Ruddy KJ, Haddad TC, et al. Clinical management of metastatic hormone receptor-positive, HER2-negative breast cancer (MBC) after CDK 4/6 inhibitors: a retrospective single-institution study. Breast Cancer Res Treat. 2022;196(1):229–37.

Xi J, Oza A, Thomas S, Ademuyiwa F, Weilbaecher K, Suresh R, et al. Retrospective Analysis of Treatment Patterns and effectiveness of Palbociclib and subsequent regimens in metastatic breast Cancer. J Natl Compr Canc Netw. 2019;17(2):141–7.

Rozenblit M, Mun S, Soulos P, Adelson K, Pusztai L, Mougalian S. Patterns of treatment with everolimus exemestane in hormone receptor-positive HER2-negative metastatic breast cancer in the era of targeted therapy. Breast Cancer Res. 2021;23(1):14.

Bashour SI, Doostan I, Keyomarsi K, Valero V, Ueno NT, Brown PH, et al. Rapid breast Cancer Disease Progression following cyclin dependent kinase 4 and 6 inhibitor discontinuation. J Cancer. 2017;8(11):2004–9.

Giridhar KV, Choong GM, Leon-Ferre R, O’Sullivan CC, Ruddy K, Haddad T, et al. Abstract P6-18-09: clinical management of metastatic breast cancer (MBC) after CDK 4/6 inhibitors: a retrospective single-institution study. Cancer Res. 2019;79:P6–18.

Article   Google Scholar  

Mougalian SS, Feinberg BA, Wang E, Alexis K, Chatterjee D, Knoth RL, et al. Observational study of clinical outcomes of eribulin mesylate in metastatic breast cancer after cyclin-dependent kinase 4/6 inhibitor therapy. Future Oncol. 2019;15(34):3935–44.

Moscetti LML, Riggi L, Sperduti I, Piacentini FOC, Toss A, Barbieri E, Cortesi L, Canino FMA, Zoppoli G, Frassoldati A, Schirone A, Dominici MECF. SEQUENCE OF TREATMENTS AFTER CDK4/6 THERAPY IN ADVANCED BREAST CANCER (ABC), A GOIRC MULTICENTER RETRO/ PROSPECTIVE STUDY. PRELIMINARY RESULTS IN THE RETROSPECTIVE SERIES OF 116 PATIENTS. Tumori. 2022;108(4S):80.

Menichetti AZE, Giorgi CA, Bottosso M, Leporati R, Giarratano T, Barbieri C, Ligorio F, Mioranza E, Miglietta F, Lobefaro R, Faggioni G, Falci C, Vernaci G, Di Liso E, Girardi F, Griguolo G, Vernieri C, Guarneri V, Dieci MV. CDK 4/6 INHIBITORS FOR METASTATIC BREAST CANCER: A MULTICENTER REALWORLD STUDY. Tumori. 2022;108(4S):70.

Marschner NW, Harbeck N, Thill M, Stickeler E, Zaiss M, Nusch A, et al. 232P Second-line therapies of patients with early progression under CDK4/6-inhibitor in first-line– data from the registry platform OPAL. Annals of Oncology. 2022;33:S643-S4

Gousis C, Lowe KMH, Kapiris M. V. Angelis. Beyond First Line CDK4/6 Inhibitors (CDK4/6i) and Aromatase Inhibitors (AI) in Patients with Oestrogen Receptor Positive Metastatic Breast Cancer (ERD MBC): The Guy’s Cancer Centre Experience. Clinical Oncology2022. p. e178.

Endo Y, Yoshimura A, Sawaki M, Hattori M, Kotani H, Kataoka A, et al. Time to chemotherapy for patients with estrogen receptor-positive breast Cancer and cyclin-dependent kinase 4 and 6 inhibitor use. J Breast Cancer. 2022;25(4):296–306.

Li Y, Li W, Gong C, Zheng Y, Ouyang Q, Xie N, et al. A multicenter analysis of treatment patterns and clinical outcomes of subsequent therapies after progression on palbociclib in HR+/HER2- metastatic breast cancer. Ther Adv Med Oncol. 2021;13:17588359211022890.

Amaro CP, Batra A, Lupichuk S. First-line treatment with a cyclin-dependent kinase 4/6 inhibitor plus an aromatase inhibitor for metastatic breast Cancer in Alberta. Curr Oncol. 2021;28(3):2270–80.

Crocetti SPM, Tassone L, Marcantognini G, Bastianelli L, Della Mora A, Merloni F, Cantini L, Scortichini L, Agostinelli V, Ballatore Z, Savini A, Maccaroni E. Berardi R. What is the best therapeutic sequence for ER-Positive/HER2- Negative metastatic breast cancer in the era of CDK4/6 inhibitors? A single center experience. Tumori. 2020;106(2S).

Nichetti F, Marra A, Giorgi CA, Randon G, Scagnoli S, De Angelis C, et al. 337P Efficacy of everolimus plus exemestane in CDK 4/6 inhibitors-pretreated or naïve HR-positive/HER2-negative breast cancer patients: A secondary analysis of the EVERMET study. Annals of Oncology. 2020;31:S382

Luhn P, O’Hear C, Ton T, Sanglier T, Hsieh A, Oliveri D, et al. Abstract P4-13-08: time to treatment discontinuation of second-line fulvestrant monotherapy for HR+/HER2– metastatic breast cancer in the real-world setting. Cancer Res. 2019;79(4Supplement):P4–13.

Mittal A, Molto Valiente C, Tamimi F, Schlam I, Sammons S, Tolaney SM et al. Filling the gap after CDK4/6 inhibitors: Novel Endocrine and Biologic Treatment options for metastatic hormone receptor positive breast Cancer. Cancers (Basel). 2023;15(7).

Ashai N, Swain SM. Post-CDK 4/6 inhibitor therapy: current agents and novel targets. Cancers (Basel). 2023;15(6).

Higgins JPTTJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). www.training.cochrane.org/handbook : Cochrane; 2022.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Serdar CC, Cihan M, Yücel D, Serdar MA. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies. Biochem Med (Zagreb). 2021;31(1):010502.

US Food Drug Administration. FDA approves elacestrant for ER-positive, HER2-negative, ESR1-mutated advanced or metastatic breast cancer [updated January 27 2023. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-elacestrant-er-positive-her2-negative-esr1-mutated-advanced-or-metastatic-breast-cancer .

US Food Drug Administration. FDA approves capivasertib with fulvestrant for breast cancer [updated November 16 2023. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-capivasertib-fulvestrant-breast-cancer .

Download references

Acknowledgements

The authors would like to acknowledge Joanna Bielecki who developed, conducted, and documented the database searches.

This study was funded by Pfizer Inc. (New York, NY, USA) and Arvinas (New Haven, CT, USA).

Author information

Sarah Kane, Belal Howidi, Bao-Ngoc Nguyen and Imtiaz A. Samjoo contributed equally to this work.

Authors and Affiliations

Pfizer, 10017, New York, NY, USA

Veronique Lambert & Yan Wu

EVERSANA, Burlington, ON, Canada

Sarah Kane, Belal Howidi, Bao-Ngoc Nguyen & Imtiaz A. Samjoo

Arvinas, 06511, New Haven, CT, USA

David Chandiwana & Michelle Edwards

You can also search for this author in PubMed   Google Scholar

Contributions

VL, IAS, SK, BH, BN, DC, YW, and ME participated in the conception and design of the study. IAS, SK, BH and BN contributed to the literature review, data collection, analysis, and interpretation of the data. VL, IAS, SK, BH, BN, DC, YW, and ME contributed to the interpretation of the data and critically reviewed for the importance of intellectual content for the work. VL, IAS, SK, BH, BN, DC, YW, and ME were responsible for drafting or reviewing the manuscript and for providing final approval. VL, IAS, SK, BH, BN, DC, YW, and ME meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work, and have given their approval for this version to be published.

Corresponding author

Correspondence to Imtiaz A. Samjoo .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors of this manuscript declare that the research presented was funded by Pfizer Inc. and Arvinas. While the support from Pfizer Inc. and Arvinas was instrumental in facilitating this research, the authors affirm that their interpretation of the data and the content of this manuscript were conducted independently and without bias to maintain the transparency and integrity of the research. IAS, SK, BH, and BN are employees of EVERSANA, Canada, which was a paid consultant to Pfizer in connection with the development of this manuscript.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lambert, V., Kane, S., Howidi, B. et al. Systematic literature review of real-world evidence for treatments in HR+/HER2- second-line LABC/mBC after first-line treatment with CDK4/6i. BMC Cancer 24 , 631 (2024). https://doi.org/10.1186/s12885-024-12269-8

Download citation

Received : 26 January 2024

Accepted : 16 April 2024

Published : 23 May 2024

DOI : https://doi.org/10.1186/s12885-024-12269-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• First-line CDK4/6i

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

5.5: Annotated List of Useful Databases

• Last updated
• Save as PDF
• Page ID 58152

• Anne Eidenmuller
• Columbia Basin College

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

While internet search engines have made locating sources online easier, there are still many digital sources beyond websites. Databases contract with publishers and other content providers to package access to articles, reports, conference proceedings, ebooks, films, images, and other material. Using databases and having access to such a variety of source material is an important part of the research process.

Search Engines vs. Databases vs. Catalogs

Most libraries provide links to different types of search systems, which contain the information, data, and search interfaces used to locate sources. The most common types of search systems are internet search engines, databases, and catalogs. They each search different types of information in different ways.

• How Databases and Search Engines Differ . Created by Undergraduate Library, University Library, University of Illinois at Urbana-Champaign.
• How Databases and Online Catalogs Differ . Created by Undergraduate Library, University Library, University of Illinois at Urbana-Champaign.

Libraries subscribe to the databases to provide access to their users (students, faculty, staff, etc.), and because libraries subscribe to dozens (if not hundreds) of databases, it’s often helpful to evaluate both the database and the sources within the database . The following annotated list of databases helps with preliminary evaluation by describing the types of sources with the databases. Some of the databases are broad-based, interdisciplinary systems that can be used to search any topic or subject. Other databases provide sources according to specific type (such as newspaper or video) or specific subjects (such as literature or science).

Most academic libraries should subscribe to these databases. Please be sure to check your library’s database aggregator or list of databases for availability and access.

Broad-Based Databases

Academic Search™ Complete . EBSCO Publishing .

Academic Search Complete™ is a multidisciplinary database that offers indexing and abstracts as well as full-text and scholarly sources. It also contains popular sources, such as magazines and newspapers. The variety of subjects includes “anthropology, astronomy, biology, chemistry, civil engineering, engineering, ethnic & multicultural studies, geology, law, materials science, mathematics, music, pharmaceutical sciences, physics, psychology, religion & theology, veterinary science, women’s studies, zoology, and many other fields.” This database works well for interdisciplinary searches as well as being a starting place for subject-specific searches.

JSTOR . ITHAKA .

JSTOR is a multidisciplinary database that offers indexing and abstracts as well as full-text and scholarly sources. It also contains books and primary sources. The database offers both the usual search options along with a subject browse tool for the following areas: area studies, arts, business and economics, history, humanities, law, medicine and allied health, science and mathematics, and social sciences. This database works well for interdisciplinary searches, but may be better for students with more experience searching databases. For example, it has a more unique search interface that does not allow for subject heading (also known as subject term) searches. However, it contains high-quality sources that are worth the effort of searching.

ProQuest Research Library™ . ProQuest .

ProQuest Research Library™ is a multidisciplinary database that offers indexing and abstracts as well as full-text and scholarly sources. It also contains popular sources, such as magazines and newspapers. Specific subjects range from business to education to humanities to sciences, plus many more. This database supports interdisciplinary searches while still providing a good starting point for subject-specific searches. (ProQuest developed a LibGuide tutorial for this database, which is located here .)

Web of Science® . Thomson Reuters .

Web of Science® is a multidisciplinary database that offers indexing and abstracts for sources. It offers links to full-text availability from other sources—either directly from the publishers (at cost) or through a library’s OpenURL connector to other databases. The database’s interdisciplinary nature comes from the combination of its indexes: Science Citation Index Expanded®, Social Sciences Citation Index®, Arts & Humanities Citation Index®, Conference Proceedings Citation Index, Index Chemicus®, and Current Chemical Reactions®. It also integrates EndNote Web® for managing articles and references online. This database works well for interdisciplinary searches as well as citation mapping, which is a form of search that locates articles based on their citations in other articles.

Source-Type Databases

ARTstor Digital Library . ARTstor .

“ The ARTstor Digital Library is a nonprofit resource that provides more than one million digital images in the arts, architecture, humanities, and sciences with an accessible suite of software tools for teaching and research.” It allows users to search for, view, and download images related to a variety of topics, such as: art, architecture, religion, anthropology, history, and literature. The database also provides image credit information for properly citing the images. Students can use these images in papers, presentations, and other assignments.

CQ Researcher . CQ Press , SAGE Publications .

CQ Researcher is a topics-focused database that “covers a wide range of social, economic, political, and environmental issues.” The database’s standardized reports review current events as researched by journalists. The reports include twelve sections, some of which are: an overview, background, outlook, pro/con, and bibliography. Students will be able to use these reports to research current events and controversial issues. The reports also provide broad background information to aid students in developing the foundations for their research.

Films on Demand . Films Media Group .

Films on Demand is a database of streaming videos. It includes both a search function and browsing ability by subject and collection. The collections come from a variety of well-known film production companies, such as the BBC, PBS, and other news organizations, as well as National Geographic and TED. Subject browsing begins with broad categories, such as biology and political science, and narrows down to more specific subtopics, such as genetics and political institutions. This database is a good option for visual learners and researchers who want a broad range of source types. The videos can be embedded into presentations, and many videos include transcripts and closed captioning, which helps for quoting material.

ProQuest Newsstand™ . ProQuest .

ProQuest Newsstand™ focuses on news sources, including newspapers and wire services. It offers indexing, abstracts, and full-text availability. Its newspaper coverage “includes international, national and regional papers.” This database would be useful for researching current events or opinions, such as controversial topics. It also assists with historical, human interest, and genealogical searches.

Subject-Specific Databases

ABI/INFORM® . ProQuest .

ABI/INFORM® is a business research database that provides indexing and abstracts, full text, images, and graphics. Sources include articles from journals and conference proceedings, market reports, business news, business cases, and dissertations. This would be a good database choice for students researching topics in business, economics, corporate strategies, management, business trends, accounting, finance, etc.

ACM Digital Library . Association for Computing Machinery .

The ACM Digital Library focuses on computing, computer systems, and related subjects. The database contains “full text of every article ever published by ACM and bibliographic citations from major publishers in computing.” This means that it is both an indexing and abstracting database (for non-ACM publishers) and a full-text database (for ACM-published sources). Sources include articles, conference proceedings, books, interviews, and other sources. Students should be able to locate sources to support research involving information technology, computers, software, computer and/or software engineering, programming, technical communication, telecommunications, and other related fields.

History Reference Center® . EBSCO Publishing .

History Reference Center® is a history research database. It is created specifically for researchers ranging from high school students to undergraduates. Sources include reference materials, biographies, documents, photos, maps, videos, and scholarly articles. Students should be able to locate sources to support research involving historical events, specific time periods, figures, military history, and other related historical topics.

IEEE Xplore® Digital Library . IEEE [Institute of Electrical and Electronics Engineers] .

IEEE Xplore® is a subject-specific database for “scientific and technical content published by the IEEE (Institute of Electrical and Electronics Engineers) and its publishing partners.” Sources include journals, conference proceedings, reports, standards, and electronic books. Students should be able to locate sources to support research involving engineering, computer science, electronics, and related fields. For a better idea of the range of related fields, review the IEEE Society Memberships page, which includes: IEEE Broadcast Technology Society , IEEE Education Society , IEEE Oceanic Engineering Society , IEEE Professional Communications Society , etc. All of these groups have publications included in the database.

Literature Resource Center . Gale , Cengage Learning .

Literature Resource Center is a subject-specific database for literature, literary criticism, and the humanities. It offers a variety of types of searches for both specific works and authors. Sources include critical essays, interviews, reviews, and other scholarly articles. This database would be a good choice for students researching literature, literary theory, poetry, and drama.

MathSciNet® . American Mathematical Society .

MathSciNet® is a mathematical literature review database. It includes the literature reviews along with abstracts, indexing, and links to articles. This database would be most useful for students researching the mathematical sciences: algebra, trigonometry, geometry, calculus, etc.

PsycINFO® . American Psychological Association .

PsycINFO® is a subject-specific database that focuses on behavioral sciences and mental health studies (such as psychology, neuroscience, social work, medicine, nursing, etc.) along with related fields (such as forensics, business, engineering, etc.). It is an indexing and abstracting database that also provides either links to full text or the actual full text, depending on the platform. (The database is available from several vendors, including the APA, EBSCO, and ProQuest.) This database would be a good source for students studying behaviors, social sciences, psychology, medicine, and related fields.

ScienceDirect . Elsevier .

ScienceDirect is a science and technology database that provides indexing and abstracts along with full-text access to journal articles and book chapters published by Elsevier and its imprints . It carries very few third-party sources. The majority of the information contained in the database focuses on the sciences—physical sciences and engineering (such as chemistry, computer science, engineering, mathematics), life sciences (such as agriculture, biology, neuroscience), and health sciences (medicine, pharmacology, nursing, veterinary science). It also includes some articles from related fields in the social sciences and humanities, but in general, it’s not a social science or humanities database. Students should be able to locate sources in the sciences, technical fields, and related subjects.

Further Reading

Please view these links for related search, database, and library information:

• Boolean Operators from Database Search Tips . MIT Libraries.
• Web Search Strategies
• Using Databases
• Search the Library Catalog
• Seek Help from Librarians
• Understanding Library Resources
• Annotated List of Useful Databases. Authored by : Amy Coughenour. Provided by : Writing Commons. Located at : http://writingcommons.org/open-text/information-literacy/library-and-internet-research/using-databases/848-annotated-list-of-useful-library-databases . License : CC BY-NC-ND: Attribution-NonCommercial-NoDerivatives

• Open access
• Published: 21 May 2024

Health profession education hackathons: a scoping review of current trends and best practices

• Azadeh Rooholamini   ORCID: orcid.org/0000-0002-9638-7953 1 &
• Mahla Salajegheh   ORCID: orcid.org/0000-0003-0651-3467 1  

BMC Medical Education volume  24 , Article number:  554 ( 2024 ) Cite this article

Metrics details

While the concept of hacking in education has gained traction in recent years, there is still much uncertainty surrounding this approach. As such, this scoping review seeks to provide a detailed overview of the existing literature on hacking in health profession education and to explore what we know (and do not know) about this emerging trend.

This was a scoping review study using specific keywords conducted on 8 databases (PubMed, Embase, Scopus, Web of Science, ERIC, PsycINFO, Education Source, CINAHL) with no time limitation. To find additional relevant studies, we conducted a forward and backward searching strategy by checking the reference lists and citations of the included articles. Studies reporting the concept and application of hacking in education and those articles published in English were included. Titles, abstracts, and full texts were screened and the data were extracted by 2 authors.

Twenty-two articles were included. The findings are organized into two main categories, including (a) a Description of the interventions and expected outcomes and (b) Aspects of hacking in health profession education.

Hacking in health profession education refers to a positive application that has not been explored before as discovering creative and innovative solutions to enhance teaching and learning. This includes implementing new instructional methods, fostering collaboration, and critical thinking to utilize unconventional approaches.

Peer Review reports

Introduction

Health professions education is a vital component of healthcare systems to provide students with the knowledge, skills, and attitudes necessary to provide high-quality care to patients [ 1 ]. However, with the advent of innovative technologies and changing global dynamics, there is a growing need to incorporate new educational methods to prepare medical science students for the future [ 2 ].

Although traditional methods can be effective for certain learning objectives and in specific contexts and may create a stable and predictable learning environment, beneficial for introducing foundational concepts, memorization, and repetition, however, they may not fully address the diverse needs and preferences of today’s learners [ 3 ]. Some of their limitations may be limited engagement, passive learning, lack of personalization, and limited creativity and critical thinking [ 4 ].

As Du et al. (2022) revealed the traditional teaching model fails to capture the complex needs of today’s students who require practical and collaborative learning experiences. Students nowadays crave interactive learning methods that enable them to apply theoretical knowledge in real-world situations [ 5 ].

To achieve innovation in health professions education, engaging students and helping them learn, educators should use diverse and new educational methods [ 6 ]. Leary et al. (2022) described how schools of nursing can integrate innovation into their mission and expressed that education officials must think strategically about the knowledge and skills the next generation of students will need to learn, to build an infrastructure that supports innovation in education, research, and practice, and provide meaningful collaboration with other disciplines to solve challenging problems. Such efforts should be structured and built on a deliberate plan and include curricular innovations, and experiential learning in the classroom, as well as in practice and research [ 7 ].

The incorporation of technology in education is another aspect that cannot be ignored. Technology has revolutionized the way we communicate and learn, providing opportunities for students to access information and resources beyond the traditional education setting. According to the advancement of technology in education, hacking in education is an important concept in this field [ 8 ].

Hack has become an increasingly popular term in recent years, with its roots in the world of computer programming and technology [ 9 ]. However, the term “hack” is not limited solely to the realm of computers and technology. It can also refer to a creative approach to problem-solving, a willingness to challenge established norms, and a desire to find new and innovative ways to accomplish tasks [ 10 ]. At its core, hacking involves exploring and manipulating technology systems to gain a deeper understanding of how they work. This process of experimentation and discovery can be applied to many different fields, including education [ 11 ].

In education, the concept of “hack” has become popular as educators seek innovative ways to engage students and improve learning outcomes. As Wizel (2019) described “hack in education” involves applying hacker mentality and techniques, such as using technology creatively and challenging traditional structures, to promote innovation within the educational system [ 12 ]. These hacking techniques encompass various strategies like gamification, hackathons, creating new tools and resources for education, use of multimedia presentations, online forums, and educational apps for project-based learning [ 9 ]. Butt et al. (2020) demonstrated the effectiveness of hack in education in promoting cross-disciplinary learning in medical education [ 13 ]. However, concerns exist about the negative connotations and ethical implications of hacking in education, with some educators hesitant to embrace these techniques in their classrooms [ 7 , 14 ].

However, while the concept of hack in education has gained traction in recent years, there is still a great deal of uncertainty surrounding its implementation and efficacy. As such, this scoping review seeks to provide a comprehensive overview of the existing literature on hacking in health profession education (HPE), to explore what we know (and do not know) about this emerging trend. To answer this research question, this study provided a comprehensive review of the literature related to hacking in HPE. Specifically, it explored the various ways in which educators are using hack techniques to improve learning outcomes, increase student engagement, and promote creativity in the classroom.

Methods and materials

This scoping review was performed based on the Arksey and O’Malley Framework [ 15 ] and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement to answer some questions about the hacking approach in health professions education [ 16 ].

Search strategies

The research question was “What are the aspects of hacking in education?“. We used the PCC framework which is commonly used in scoping reviews to develop the research question [ 17 ]. In such a way the Population assumed as learners, the Concept supposed as aspects of hacking in education, and the Context is considered to be the health profession education.

A systematic literature search was conducted on June 2023, using the following terms and their combinations: hack OR hacking OR hackathon AND education, professional OR “medical education” OR “medical training” OR “nursing education” OR “dental education” OR “pharmacy education” OR “health professions education” OR “health professional education” OR “higher education” OR “healthcare education” OR “health care education” OR “students, health occupations” OR “medical student” OR “nursing student” OR “dental student” OR “pharmacy student” OR “schools, health occupations” OR “medical school” OR “nursing school” OR “dental school” OR “pharmacy school”) in 8 databases (PubMed, Embase, Scopus, Web of Science, ERIC, PsycINFO, Education Source, CINAHL) with no time limitation. (A copy of the search strategy is included in Appendix 1 ). To find additional relevant studies, we conducted a forward and backward searching strategy by checking the reference lists and citations of the included articles.

Inclusion and exclusion criteria

Original research reporting the different aspects of hacking in health professions education and published in English was included. We excluded commentaries, editorials, opinion pieces, perspectives, reviews, calls for change, needs assessments, and other studies in which no real interventions had been employed.

Study identification

After removing the duplicates, each study potentially meeting the inclusion criteria was independently screened by 2 authors (A.R. and M.S.). Then, the full texts of relevant papers were assessed independently by the 2 authors for relevance and inclusion. Disagreements at either step were resolved when needed until a consensus was reached.

Quality assessment of the studies

We used the BEME checklist [ 18 ], consisting of 11 indicators, to assess the quality of studies. Each indicator was rated as “met,” “unmet,” or “unclear.” To be deemed of high quality, articles should meet at least 7 indicators. The quality of the full text of potentially relevant studies was assessed by 2 authors (A.R. and M.S.). Disagreements were resolved through discussion. No study was removed based on the results of the quality assessment.

Data extraction and synthesis

To extract the data from the studies, a data extraction form was designed based on the results of the entered studies. A narrative synthesis was applied as a method for comparing, contrasting, synthesizing, and interpreting the results of the selected papers. All outcomes relevant to the review question were reported. The two authors reviewed and coded each included study using the data extraction form independently.

A total of 645 titles were found, with a further four titles identified through the hand-searching of reference lists of all reviewed articles. After removing the duplicate references, 422 references remained. After title screening, 250 studies were considered for abstract screening, and 172 studies were excluded. After the abstract screening, 73 studies were considered for full-text screening, and 177 studies were excluded due to reasons such as:1. being irrelevant, 2. loss of data, and 3. language limitation. 22 studies were included in the final analysis. The 2020 PRISMA diagram for the included studies is shown in Fig.  1 . The quality was evaluated as “high” in 12 studies, “moderate” in 7 studies, and “low” in 3 studies.

PRISMA flow diagram for included studies

The review findings are organized into two main categories: (a) Description of the interventions and expected outcomes and (b) Aspects of hacking in health profession education.

Description of the interventions and expected outcomes

The description of the studies included the geographical context of the interventions, type, and number of participants, focus of the intervention, evaluation methodology, and outcomes. Table  1 displays a summary of these features.

Geographical context

Of the 22 papers reviewed, 11 studies (45.4%) took place in the United States of America [ 7 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ], two studies in Pakistan [ 13 , 29 ], one study performed in international locations [ 30 ], and the remainder being in the United Kingdom [ 31 ], Germany [ 32 ], Finland [ 33 ], Australia [ 34 ], Austria [ 35 ], Thailand [ 36 ], Africa [ 37 ], and Canada [ 38 ].

Type and number of participants

Hacking in HPE interventions covered a wide range and multiple audiences. The majority of interventions targeted students (17 studies, 77.2%) [ 7 , 13 , 20 , 21 , 23 , 24 , 25 , 26 , 27 , 29 , 30 , 31 , 32 , 33 , 36 , 37 , 38 ]. Their field of education was reported differently including medicine, nursing, engineering, design, business, kinesiology, and computer sciences. Also, they were undergraduates, postgraduates, residents, and post-docs. Ten interventions (45.4%) were designed for physicians [ 13 , 19 , 21 , 24 , 25 , 26 , 28 , 29 , 33 , 35 ]. Their field of practice was reported diverse including psychology, radiology, surgery, and in some cases not specified. Eight (36.3%) studies focused on staff which included healthcare staff, employees of the university, nurses, care experts, and public health specialists [ 13 , 22 , 26 , 28 , 29 , 30 , 32 , 35 ]. Interestingly, nine of the hacking in HPE interventions (40.9%) welcomed specialists from other fields outside of health sciences and medicine [ 13 , 19 , 22 , 25 , 26 , 28 , 29 , 33 , 35 ]. Their field of practice was very diverse including engineers, theologians, artists, entrepreneurs, designers, informaticists, IT professionals, business professionals, industry members, data scientists, and user interface designers. The next group of participants was faculty with 5 studies (22.7%) [ 7 , 23 , 32 , 34 , 36 ]. An intervention (4.5%) targeted the researchers [ 27 ]. The number of participants in the interventions ranged from 12 to 396. Three studies did not specify the number of their participants.

The focus of the intervention

The half of interventions aimed to improve HPE (12 studies, 54.5%) [ 7 , 13 , 21 , 23 , 24 , 26 , 28 , 30 , 31 , 32 , 34 , 38 ], with a secondary emphasis on enhancing clinical or health care [ 19 , 22 , 25 , 29 , 33 , 35 , 36 , 37 ]. Two studies highlighted the improvement in entrepreneurship skills of health professions [ 19 , 20 ]. One study aimed to improve the research skills of health professionals [ 27 ].

Evaluation methodology

Methods to evaluate hacking in HPE interventions included end-of-program questionnaires, pre-and post-test measures to assess attitudinal or cognitive change, self-assessment of post-training performance, project-based assessment through expert judgment and feedback, interviews with participants, and direct observations of behavior.

Hacking in HPE interventions has resulted in positive outcomes for participants. Five studies found high levels of satisfaction for participants with the intervention [ 21 , 31 , 32 , 33 , 37 ]. Some studies evaluated learning, which included changes in attitudes, knowledge, and skills. In most studies, participants demonstrated a gain in knowledge regarding awareness of education’s strengths and problems, in the desire to improve education by enhancement of awareness for technological possibilities [ 7 , 13 , 19 , 21 , 23 , 30 , 32 , 33 , 34 , 35 , 38 ]. Some studies found improving participant familiarity with healthcare innovation [ 19 , 22 , 24 , 25 , 26 , 33 , 36 , 37 ]. Some participants reported a positive change in attitudes towards HPE as a result of their involvement in hacking interventions. They cited a greater awareness of personal strengths and limitations, increased motivation, more confidence, and a notable appreciation of the benefits of professional development [ 20 , 21 , 29 , 34 ]. Some studies also demonstrated behavioral change. In one study, changes were noted in developing a successful proof-of-concept of a radiology training module with elements of gamification, enhancement engagement, and learning outcomes in radiology training [ 28 ]. In a study, participants reported building relationships when working with other members which may be students, faculty, or healthcare professionals [ 7 ]. Five studies found a high impact on participant perceptions and attitudes toward interdisciplinary collaboration [ 22 , 26 , 27 , 36 , 38 ].

Aspects of hacking in health profession education

The special insights of hacking in HPE included the adaptations considered in the interventions, the challenges of interventions, the suggestions for future interventions, and Lessons learned.

Adaptations

The adaptations are considered to improve the efficacy of hacking in HPE interventions. We found that 21 interventions were described as hackathons. Out of this number, some were only hackathons, and some others had benefited from hackathons besides other implications of hacking in education. Therefore, most of the details in this part of the findings are presented with a focus on hackathons. The hackathon concept has been limited to the industry and has not been existing much in education [ 39 , 40 ]. In the context of healthcare, hackathons are events exposing healthcare professionals to innovative methodologies while working with interdisciplinary teams to co-create solutions to the problems they see in their practice [ 19 , 22 , 24 , 25 , 30 , 41 , 42 ].

Some hackathons used various technologies for internal and external interactions during the hackathon including Zoom, Gmail, WhatsApp, Google Meet, etc [ 37 ]. . . Almost all hackathons were planned and performed in the following steps including team formation, team working around the challenges, finding innovative solutions collaboratively, presenting the solutions and being evaluating based on some criteria including whether they work, are good ideas with a suitable problem/solution fit, how a well-designed experience and execution, etc. For example, in the hackathon conducted by Pathanasethpong et al. (2017), the judging criteria included innovativeness, feasibility, and value of the projects [ 36 ]. Also, they managed the cultural differences between the participants through strong support of leadership, commitment, flexibility, respect for culture, and willingness to understand each other’s needs [ 36 ].

Despite valuable adaptations, several challenges were reported. The hackathons faced some challenges such as limited internet connectivity, time limitations, limited study sample, power supply, associated costs, lack of diversity among participants, start-up culture, and lack of organizational support [ 13 , 19 , 25 , 28 , 30 , 34 , 37 ]. Some interventions reported the duration of the hackathon was deemed too short to develop comprehensive solutions [ 37 ]. One study identified that encouraging experienced physicians and other healthcare experts to participate in healthcare hackathons is an important challenge [ 26 ].

Suggestions for the future

Future hackathons should provide internet support for participants and judges, invite investors and philanthropists to provide seed funding for winning teams, and enable equal engagement of all participants to foster interdisciplinary collaboration [ 37 ]. Subsequent hackathons have to evaluate the effect of implementation or durability of the new knowledge in practice [ 19 , 28 ]. Wang et al. (2018) performed a hackathon to bring together interdisciplinary teams of students and professionals to collaborate, brainstorm, and build solutions to unmet clinical needs. They suggested that future healthcare hackathon organizers a balanced distribution of participants and mentors, publicize the event to diverse clinical specialties, provide monetary prizes and investor networking opportunities for post-hackathon development, and establish a formal vetting process for submitted needs that incorporates faculty review and well-defined evaluation criteria [ 22 ]. Most interventions had an overreliance on self-assessments to assess their effectiveness. To move forward, we should consider the use of novel assessment methods [ 30 ].

Lessons learned

Based on the findings of hackathons, they have developed efficient solutions to different problems related to public health and medical education. Some of these solutions included developing novel computer algorithms, designing and building model imaging devices, designing more approachable online patient user websites, developing initial prototypes, developing or optimizing data analysis tools, and creating a mobile app to optimize hospital logistics [ 25 , 26 , 27 , 36 ]. Staziaki et al. (2022) performed an intervention to develop a radiology curriculum. Their strategies were creating new tools and resources, gamification, and conducting a hackathon with colleagues from five different countries. They revealed a radiology training module that utilized gamification elements, including experience points and a leaderboard, for annotation of chest radiographs of patients with tuberculosis [ 28 ].

Most hackathons provide an opportunity for medical health professionals to inter-professional and inter-university collaboration and use technology to produce innovative solutions to public health and medical education [ 7 , 23 , 26 , 30 , 37 , 38 ]. For example, one study discussed that hackathons allowed industry experts and mentors to connect with students [ 37 ]. In the study by Mosene et al. (2023), results offer an insight into the possibilities of hackathons as a teaching/learning event for educational development and thus can be used for large-scale-assessments and qualitative interviews for motivational aspects to participate in hackathons, development of social skills and impact on job orientation [ 32 ].

The participants’ willingness to continue working on the projects after the hackathons was also reported in some papers [ 13 , 29 , 33 ]. One study highlights the potential of hackathons to address unmet workforce needs and the preference of female surgeons for small-group discussions and workshops [ 24 ]. Craddock et al. (2016) discussed that their intervention provided a unique opportunity for junior researchers and those from developing economies who have limited opportunities to interact with peers and senior scientists outside their home institution [ 27 ].

Dameff et al. (2019) developed and evaluated a novel high-fidelity simulation-based cybersecurity training program for healthcare providers. They found significant improvements in the knowledge and confidence of participants related to clinical cybersecurity after completing the simulation exercise. They also reported high levels of satisfaction with the training program [ 21 ].

This scoping review provided a detailed overview of the existing literature on hacking in health profession education and explored what we know (and do not know) about this emerging trend. Our results emphasized the increasing pattern of utilizing hacking in HPE for enhancing teaching and learning, problem-solving, and product generation. Our findings revealed that elements of hacking in HPE can include; innovation, creativity, critical thinking, and collaboration. Innovation is a critical element of hacking in education that holds different meanings for different disciplines. Those involved in HPE consider innovation to create new tools and resources [ 7 , 28 ], hackathons [ 13 , 19 , 20 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ], gamification [ 28 ], and simulation-based training [ 21 ].

This study by introducing a different perspective or a new application of hacking that has not been explored before allows for a broader understanding of hacking and its potential positive applications in HPE. Although it does mention “hacking,” it does not refer to the malicious or illegal activities often associated with the term [ 43 , 44 ]. The results of this study indicate incorporating hacking into HPE aimed at improving education and enhancing clinical or healthcare had positive outcomes in learning, attitudes, knowledge, and skills. Embracing hacking in HPE revolutionizes traditional teaching methods, promotes interdisciplinary collaboration, leverages cutting-edge technologies, and cultivates a culture of lifelong learning, ultimately enhancing clinical outcomes and the healthcare system as a whole [ 13 , 20 , 21 , 22 , 26 , 27 , 28 , 30 , 31 , 32 , 33 , 34 , 36 , 37 , 38 ].

This study reveals that hackathons are more prominent in the United States of America (USA) education system compared to other countries due to the culture of innovation and entrepreneurship [ 7 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ]. It is important to note that while hackathons are more prominent in the USA, they are also gaining popularity in other countries [ 13 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ]. This mindset directly contributes to designing effective interventions and driving innovation across different countries and regions around the world. In comparison to other educational interventions, in hacking within education studies, the geographical context, the focus of the intervention, and outcomes can play a significant role in shaping the educational intervention. The relationship between them can be explained through Socio-cultural theory which emphasizes the influence of social interactions and cultural factors in learning and development [ 45 ]. According to this theory, factors such as cultural values, societal norms, availability of technological resources, access to educational opportunities, and collaboration with local communities all play a role in shaping the outcomes of hacking in education. In light of the findings, creating a positive impact on education through “hacking” as innovation requires adaptations and overcoming challenges. Adaptations could involve modifying traditional teaching methods, incorporating new technologies into the learning process, or adopting new pedagogical approaches, such as project-based learning or blended learning [ 40 ]. Adapting education through hacking means finding innovative solutions to improve teaching methods, student engagement, and overall learning outcomes [ 46 ]. Challenges refer to the obstacles or barriers that educators, leaders, or organizations may face when trying to implement innovative changes in education could be related to resistance to change, lack of resources or funding, bureaucratic hurdles, or simply the complexities of navigating a rapidly changing educational landscape [ 47 ]. Therefore, driving positive change requires leading with creativity, perseverance, and collaboration [ 48 ]. In this way, different leadership and management approaches and models can help to create change. For example, studies show that Kotter’s 8-Step Change theory can be considered a guide for educators to lead innovation in education through hacking [ 49 ].

With a clear definition of innovation, the next is to consider how to systematize and embed a culture of innovation within the educational organization. An important component of this strategy is tying innovation to professional, school, and university priorities. Innovation is a human-centered endeavor and requires key stakeholders’ engagement to identify challenges and opportunities. Our findings emphasized that while meeting with multiple stakeholders is critical, developing other champions of an innovation focus is essential. Consider resources available in developing internal and external advisory members, local entrepreneurs, or leaders in innovation roles. Other strategies can be used to guide the design and development of innovation programs including co-design sessions, focus groups, and the use of external consultants.

Faculty members are the main actors of change and the most effective source of creativity in education. They have a significant role to play in driving change in education by preparing the ground for creativity, adapting to new changes, and stimulating change within the classroom. They can create a positive and innovative learning environment that benefits both students and the entire organization [ 50 , 51 ].

For many faculty members, innovation will be a new area of inquiry. Hence, based on our findings we recommend to the planners and organizers of faculty development programs to design and implement some programs about innovation in the teaching and learning process considering these three key elements: building knowledge, acquiring skills in applying rigorous innovation methodologies to identifying and solving problems, and generating opportunities to participate in innovation activities can way to develop an interest in innovation and elevate it as a school goal and priority [ 51 , 52 ].

Overall, these findings demonstrate that the hackathon effectively met its objectives in the case of HPE by promoting interdisciplinary collaboration, building relationships, facilitating learning, developing innovation, knowledge acquisition, practical problem-solving skills, cross-disciplinary tools for teaching and learning, and inquiry-based learning. In addition, findings reveal the positive outcomes of hackathons in HPE including increasing confidence levels as innovators, enhancing awareness of technological possibilities for future healthcare givers, improved familiarity with healthcare innovation and teaching entrepreneurship, improving engagement, and learning outcomes in training, high participant satisfaction, and increased motivation with the program. Also, Hackathon in HPE emphasizes the role of multidisciplinary teams and technology in solving medical education problems and encourages disciplinary collaborations to improve data collection and analysis [ 7 , 13 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ]. A potential gap of knowledge in this study is the lack of research on the long-term impact and sustainability of hacking in HPE. While the study highlights the positive outcomes of incorporating hacking into education, it does not delve into the long-term effects or address the potential challenges in maintaining and sustaining these innovative practices. Additionally, there is limited mention of the assessment methods used to measure the effectiveness of hacking in education, which could be an area for further investigation.

Some limitations of this study are including, this comprehensive study includes a straightforward research question, a predefined search strategy, and inclusion and exclusion criteria for studies that summarize all relevant studies, allowing for a detailed understanding of the available evidence. This had some limitations when it came to collecting eligible articles. Since this review extracted only published research, there are educational interventions that are reported at conferences but have not yet been published in the literature. The moderate quality of full-text studies is indeed a limitation of this study. Future research should consider including higher-quality full-text studies to enhance the robustness of the findings.

Although we searched for articles using general keywords, these were limited to hackathon keywords. Further research is needed to conduct hackathons in HPE to drive sustained innovation and crowd-source solutions. First, research should investigate how to enhance faculty and student engagement and retention to foster hackathons in HPE. Second, a multidisciplinary study is crucial to strike a balance between embracing innovation and evaluating its impact to ensure its successful integration into the education system. Third, future research could focus on exploring the long-term impact, sustainability, and assessment methods of incorporating hackathons in HPE.

Hacking in the health profession educational context refers to the positive applications in teaching and learning that have not been explored before. Embracing hacking requires adaptations, overcoming challenges, and driving change through creativity, perseverance, and collaboration. The goal of hacking in health profession education is to create a more dynamic, adaptable, and effective educational system that meets the needs of all learners and prepares them for success in the rapidly evolving 21st-century economy.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Van Schalkwyk SC, Hafler J, Brewer TF, Maley MA, Margolis C, McNamee L, et al. Transformative learning as pedagogy for the health professions: a scoping review. Med Educ. 2019;53(6):547–58.

Article   Google Scholar  

Green M, Wayne DB, Neilson EG. Medical education 2020—charting a path forward. JAMA. 2019;322(10):934–5.

Koolivand H, Shooreshi MM, Safari-Faramani R, Borji M, Mansoory MS, Moradpoor H, et al. Comparison of the effectiveness of virtual reality-based education and conventional teaching methods in dental education: a systematic review. BMC Med Educ. 2024;24(1):8.

Saini S, Kamath G, Mathew M, DSouza D. Case based interprofessional learning versus traditional teaching methods for medical, nursing, and physiotherapy students. Internet J Allied Health Sci Pract. 2024;22(1):27.

Google Scholar  

Du L, Zhao L, Xu T, Wang Y, Zu W, Huang X et al. Blended learning vs traditional teaching: the potential of a novel teaching strategy in nursing education-a systematic review and meta-analysis. Nurse Educ Pract. 2022:103354.

Gyimah N. Assessing technological innovation on education in the world of coronavirus (COVID-19). Annal Immunol Immunotherapy. 2022;4(1):000158.

Leary M, Villarruel AM, Richmond TS. Creating an innovation infrastructure in academic nursing. J Prof Nurs. 2022;38:83–8.

Záhorec J, Nagyová A, Hašková A. Teachers’ Attitudes to Incorporation Digital Means in Teaching Process in Relation to the Subjects they Teach. Int J Eng Pedagogy. 2019;9(4).

Barpi F, Dalmazzo D, De Blasio A, Vinci F. Hacking higher education: rethinking the EduHack course. Educ Sci. 2021;11(2):40.

Kim HJ, Jang HY. Sustainable technology integration in underserved area schools: the impact of perceived student change on teacher continuance intention. Sustainability. 2020;12(12):4802.

Iglesias-Sánchez PP, Jambrino-Maldonado C, de las Heras-Pedrosa C. Training entrepreneurial competences with open innovation paradigm in higher education. Sustainability. 2019;11(17):4689.

Wizel M, editor. Teachers as Hackers: Implications for 21st Century Teacher Education. 7th Teaching & Education Conference; 2019; London: Lesley University.

Butt WA, Shahood Q, Farooqi WH, Ghias K, Sabzwari S, Mian A. Healthcare hackathons: fostering medical education through innovation in a developing country: a case study from Pakistan. BMJ Innovations. 2020;7(1):1–6.

Maimon D, Louderback ER. Cyber-dependent crimes: an interdisciplinary review. Annual Rev Criminol. 2019;2:191–216.

Arksey H, O’malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

Pollock D, Peters MD, Khalil H, McInerney P, Alexander L, Tricco AC, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evid Synthesis. 2023;21(3):520–32.

Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S, et al. The educational effects of portfolios on undergraduate student learning: a best evidence Medical Education (BEME) systematic review. BEME Guide 11 Med Teacher. 2009;31(4):282–98.

Preiksaitis C, Dayton JR, Kabeer R, Bunney G, Boukhman M. Teaching principles of Medical Innovation and Entrepreneurship through hackathons: Case Study and qualitative analysis. JMIR Med Educ. 2023;9(1):e43916.

Kagan O, Sciasci NG, Koszalinski RS, Kagan DH, Leary M, Nadel H. Nurses’ confidence in starting a new venture, startup or project in the context of nurse-led hackathons: results of prehackathon survey. Nurs Outlook. 2023;71(3):101961.

Dameff CJ, Selzer JA, Fisher J, Killeen JP, Tully JL. Clinical cybersecurity training through novel high-fidelity simulations. J Emerg Med. 2019;56(2):233–8.

Wang JK, Roy SK, Barry M, Chang RT, Bhatt AS. Institutionalizing healthcare hackathons to promote diversity in collaboration in medicine. BMC Med Educ. 2018;18(1):1–9.

Saffari SS, Frederick Lambert R, Dang L, Pagni S, Dragan IF. Integrating student feedback during Dental Curriculum Hack-A-thon. BMC Med Educ. 2018;18:1–6.

Ruzgar NM, Ahuja C, Kristin EY, Sallam A, Rosenthal R, Killelea B. How we do it: creation of a workforce development-focused track at a surgical hackathon. J Surg Educ. 2020;77(5):1028–32.

Cooper K, Siefert A, Weinreb J. Skills beyond the Reading room: training in innovation and collaboration at a radiology hackathon. J Am Coll Radiol. 2018;15(3):466–8.

Silver JK, Binder DS, Zubcevik N, Zafonte RD. Healthcare hackathons provide educational and innovation opportunities: a case study and best practice recommendations. J Med Syst. 2016;40:1–7.

Cameron Craddock R, Margulies S, Bellec D, Nolan Nichols P, Alcauter B, Barrios SA et al. F,. Brainhack: a collaborative workshop for the open neuroscience community. GigaScience. 2016;5(1):s13742-016-0121-x.

Staziaki PV, Santinha JA, Coelho MO, Angulo D, Hussain M, Folio LR. Gamification in radiology training module developed during the society for imaging informatics in medicine annual meeting hackathon. J Digit Imaging. 2022;35(3):714–22.

Butt WA, Shariff A, Khan S, Mian AI. Global surgery hackathons: a Case Study from Pakistan. Surg Innov. 2021;28(4):496–501.

Wang JK, Pamnani RD, Capasso R, Chang RT. An extended hackathon model for collaborative education in medical innovation. J Med Syst. 2018;42:1–8.

Kienzler H, Fontanesi C. Learning through inquiry: a global health hackathon. Teach High Educ. 2017;22(2):129–42.

Mosene K, Kleinesper C, Prokop G, Caroli F, Teufel D, Berberat PO et al. OPEN Hackathon at the TUM School of Medicine, Germany. GMS J Med Educ. 2023;40(2).

Kolog EA, Sutinen E, Nygren E. Hackathon for learning digital theology in computer science. Int J Mod Educ Comput Sci. 2016;8(6):1.

Brand G, Collins J, Bedi G, Bonnamy J, Barbour L, Ilangakoon C, et al. I teach it because it is the biggest threat to health: integrating sustainable healthcare into health professions education. Med Teach. 2021;43(3):325–33.

Ströckl DE, Perchtaler M, Oberzaucher J, editors. Interdisciplinary Hackathons-A method to embed digitization in Healthcare Education. dHealth; 2022.

Pathanasethpong A, Soomlek C, Morley K, Morley M, Polpinit P, Dagan A, et al. Tackling regional public health issues using mobile health technology: event report of an mHealth hackathon in Thailand. JMIR mHealth uHealth. 2017;5(10):e8259.

Babatunde AO, Brimmo FO, Arama UO, Onyinyechi MG, Josephat KA, Osiene AO. A Public Health Hackathon for Medical Students in Africa: Process, Outcome and Recommendations. medRxiv. 2023:2023.01. 28.23284802.

Muñoz-Leija MA, Paul BR, Shi G, Dixit I, Quiroga-Garza A, Elizondo-Omaña RE, et al. THE HIVE: a multidisciplinary approach to medical education. Eur J Anat. 2021;25(1):101–6.

Oyetade KE, Zuva T, Harmse A. Factors influencing Hackathon Adoption for Learning Information Technology (IT) programming modules. TEM J. 2022;11(3).

Horton PA, Jordan SS, Weiner S, Lande M, editors. Project-based learning among engineering students during short-form hackathon events. 2018 ASEE Annual Conference & Exposition; 2018.

Walker A, Ko N. Bringing medicine to the digital age via hackathons and beyond. J Med Syst. 2016;40:1–3.

Ahmed R, Mian AI. A case for global surgery in Pakistan: implementation through Multi-disciplinary Engagement. JPMA J Pakistan Med Association. 2019;69(1):S98–100.

Clarke R, Youngstein T. Cyberattack on Britain’s National Health Service—a wake-up call for modern medicine. N Engl J Med. 2017;377(5):409–11.

Grimes S, Wirth A. Holding the line: events that shaped healthcare cybersecurity. Biomedical Instrum Technol. 2017;51(s6):30–2.

Mahn H, John-Steiner V. Vygotsky and sociocultural approaches to teaching and learning. Handbook of Psychology, Second Edition. 2012;7.

Wizel M. Preparing educational hackers. Contemporary pedagogies in teacher education and development. IntechOpen; 2018.

Ávila LV, Leal Filho W, Brandli L, Macgregor CJ, Molthan-Hill P, Özuyar PG, et al. Barriers to innovation and sustainability at universities around the world. J Clean Prod. 2017;164:1268–78.

Matthew CT, Sternberg RJ. Leading innovation through collaboration. Innovation through collaboration. Volume 12. Emerald Group Publishing Limited; 2006. pp. 27–52.

Kotter J. The 8-step process for leading change. Kotter Int. 2012.

Potter EM. Perceptions of Creativity among Faculty in Higher Education. 2013.

Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide 8. Med Teach. 2006;28(6):497–526.

Steinert Y. Perspectives on faculty development: aiming for 6/6 by 2020. Perspect Med Educ. 2012;1:31–42.

Download references

Acknowledgements

Not applicable.

This study was conducted with the financial support of the Vice-Chancellor for Research and Technology of Kerman University of Medical Sciences (project number: 402000210).The role of the funding body was to provide support for data collection and analysis.

Author information

Authors and affiliations.

Department of Medical Education, Medical Education Development Center, Kerman University of Medical Sciences, Kerman, Iran

Azadeh Rooholamini & Mahla Salajegheh

You can also search for this author in PubMed   Google Scholar

Contributions

AR and MS formulated the research idea, extracted data, and performed the analysis of the data, wrote the manuscript, and edited the draft of the paper. All authors approved the final manuscript.

Corresponding author

Correspondence to Mahla Salajegheh .

Ethics declarations

Ethical approval and consent to participate.

The Ethics Committee of Kerman University of Medical Sciences approved the study (No: IR.KMU.REC.1402.251).

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rooholamini, A., Salajegheh, M. Health profession education hackathons: a scoping review of current trends and best practices. BMC Med Educ 24 , 554 (2024). https://doi.org/10.1186/s12909-024-05519-7

Download citation

Received : 05 December 2023

Accepted : 06 May 2024

Published : 21 May 2024

DOI : https://doi.org/10.1186/s12909-024-05519-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical education
• Health profession education

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Case report

• Open access
• Published: 19 May 2024

Fistulising skin metastases in Crohn’s disease: a case report and review of the literature

• Tanja Elger 1 ,
• Johanna Loibl 1 ,
• Christa Buechler 1 ,
• Sebastian Haferkamp 3 ,
• Jens Werner 2 ,
• Konstantin Drexler 3 ,
• Ulrich Hohenleutner 3 ,
• Karsten Guelow 1 ,
• Claudia Kunst 1 ,
• Arne Kandulski 1 ,
• Pia Goeggelmann 1 ,
• Martina Mueller 1 &
• Hauke Christian Tews 1  

Journal of Medical Case Reports volume  18 , Article number:  252 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Metastatic Crohn’s disease is a rare disorder characterized by various granulomatous skin lesions that occur independently of gastrointestinal tract involvement. However, currently there is no standardized care or specific treatment. Therapeutic approaches include immunosuppressive agents, such as corticosteroids, azathioprine, and monoclonal antibodies targeting inflammatory cytokines like tumor necrosis factor (TNF).

Case presentation

We present a case of a 29-year-old western European woman with significant blind ending abdominal subcutaneous fistulas and abscesses, who sought evaluation in the dermatology department. Histological examination revealed multiple epithelioid cell granulomas. There was no evidence of infectious or rheumatologic diseases such as sarcoidosis. The tentative diagnosis was metastatic Crohn’s disease, which was not related to an intestinal manifestation of the disease. The patient responded to infliximab but had to discontinue it due to an allergic reaction. Subsequent adalimumab treatment failed to induce clinical remission; thus, therapy was switched to ustekinumab, resulting in a positive response. Written informed consent for publication of their clinical details and clinical images was obtained from the patient.

For our study more than 1600 publications were screened for cases of metastatic Crohn’s disease on PubMed database. 59 case reports with 171 patients were included in the analysis and evaluated for localization, diagnostic and therapeutic approaches, and complications and were summarized in this review.

The successful ustekinumab treatment of a patient with metastatic Crohn's disease underscores the potential of this minimally investigated therapeutic option, highlighting the need for future treatment guidelines given the increasing prevalence of such cases.

Peer Review reports

Metastatic Crohn`s disease is a rare disease that primarily affects patients diagnosed with inflammatory bowel diseases (IBD). In rare cases, cutaneous manifestations precede gastrointestinal involvement. It most commonly occurs in the genital region, but can also affect every other part of the body. The lesions present as plaques, ulcers, fissures or papules [ 1 ] and, in rare cases, as fistulas. They are often asymptomatic but may also cause pain or itching. Despite the misleading wording, metastatic Crohn’s disease is not considered an oncologic entity. Instead, the term “metastatic” refers to the involvement of sites with no physical connection to the GI-tract. Metastatic Crohn’s disease must not be mixed up with extraintestinal manifestations of Crohn´s disease. Whereas extraintestinal manifestations such as pyoderma gangraenosum and erythema nodosum represent distinct immunologic phenomena, metastatic Crohn’s disease exhibits the same histological findings as intestinal Crohn’s disease, but on other organ sites.

Usually, a biopsy from the involved site is required for the diagnosis of metastatic Crohn’s disease. Histological findings show non-caseating, sarcoid-like granulomas, Langerhans giant cells and foreign body giant cells surrounded by inflammatory histiocytes, plasma cells and lymphocytes [ 2 ]. For a confirmed diagnosis of metastatic Crohn’s disease, other causes for granulomatous disorders have to be excluded, especially cutaneous sarcoidosis, tuberculosis, syphilis, mycobacterial infections, actinomycosis, deep fungal infections, lymphogranuloma venereum and granuloma inguinale. Also non-granulomatous skin lesions such as hidradenitis suppurativa, pyoderma gangrenosum, impetigo, erythema nodosum, factitial dermatitis from factitial injection of foreign substances, schistosomiasis, chronic lymphedema resulting from obstruction, erysipelas, chronic cellulitis and foreign body reaction need to be ruled out [ 3 ].

To objectively assess intestinal involvement, endoscopy of the upper and lower GI tract along with an abdominal magnetic resonance imaging (MRI) scan should be performed.

Currently, there is no standardized treatment of metastatic Crohn’s disease and no German or European guideline, especially for cases without GI involvement. Only individual case reports exist regarding the therapeutic use of approved medication for intestinal Crohn’s disease in metastatic conditions, including steroids, anti-TNF antibodies, azathioprine, and antibiotic therapy [ 4 , 5 , 6 , 7 ].

Ustekinumab is a human monoclonal IgG1κ-antibody, which binds specifically to the p40-subunit of interleucin 12 (IL-12) and interleucin 23 (IL-23). The bioactivity of IL-12 and IL-23 is inhibited by ustekinumab by preventing the p40 subunit from binding to IL-12Rß1-receptorprotein on the surface of immune cells. It is assumed that hereby the cytokine pathways of Th1- and Th17-cells are interrupted, which both play an important role in the pathogenesis of Crohn’s disease [ 8 ].

We here report a case of metastatic Crohn’s disease successfully treated with ustekinumab at our university hospital and provide a literature review on the current therapeutic options for metastatic Crohn’s disease.

In 2020, a 29-year-old western European woman presented to the dermatology department of our university hospital with pronounced abdominal blind-ending fistulas. The initial patient contact was documented in 2019 when she sought care at the emergency department with multiple recurrent abscesses of the abdominal skin that first appeared after a tick bite a few weeks prior.

Between January 2019 and July 2020, multiple incisions of recurrent abscesses were performed in combination with antibiotic treatment. However, complete healing was not achieved. Instead, the abscess cavities expanded beneath the skin, subsequently forming a system of connected fistulas (Fig.  1 ), as visualized in MRI enterography (Fig.  2 ).

Large subcutaneous fistula, 2020

MRI enterography of the large subcutaneous abdominal fistula (red circle), April 2021

When the patient presented to the dermatology department in 2020, the fistulas showed active inflammation with secretion and were almost 2 cm in diameter. In addition, a blind ending subcutaneous fistula of the thigh could be detected, which also showed active inflammation and secretion.

Histologic examination revealed a granulomatous infiltrate with histiocytes and multinuclear giant cells, forming granulomas. In addition, lymphocytes, plasma cells, neutrophils, and eosinophils, and a granulomatous perivasculitis were observed (Fig.  3 ). Differential diagnoses of granulomatous diseases such as sarcoidosis, tuberculosis and immunodeficiencies were ruled out, confirming the diagnosis of metastatic Crohn’s disease.

Histologic examination of a fistula biopsy showing granulomas with histiocytes and multinuclear giant cells, July 2020

To investigate intestinal involvement, abdominal MRI, gastroscopy, and colonoscopy were performed. However, these examinations did not reveal any gastrointestinal involvement at this time. Moreover, fecal calprotectin levels were normal. There was no history of inflammatory bowel disease in the patient’s family.

Antibiotic therapy had failed in the past, but the Patient responded well to prednisolone. As maintenance therapy for metastatic Crohn’s disease, the patient initially received infliximab at a dosage of 5 mg/kg. The patient responded well to this therapy, leading to the cessation of fluid secretion by the fistulas. However, an allergic reaction with dyspnea and rash occurred after the 10th dose of infliximab in October 2021, necessitating the discontinuation of treatment.

To evaluate further treatment options, the patient was referred to the department of Gastroenterology in November 2021. Meanwhile, she had developed diarrhea and abdominal pain. MRI revealed a mild ileitis, which, however, could not be validated by colonoscopy.

Therapy was switched to adalimumab in December 2021, to which the patient did not respond. Instead, a new fistula ostium developed.

Despite the lack of evidence for ustekinumab therapy in metastatic Crohn’s disease, and given its established efficacy only for intestinal manifestation, treatment with ustekinumab was initiated in May 2022 with an initial dose of 390 mg intravenous, followed by 90 mg subcutaneous every 8 weeks. This decision was based on the suspicion that the inflammatory processes in the abdominal fistula mirrored those seen in intestinal inflammation. The patient responded well and inflammation decreased within a few weeks. However, fistulas persisted, albeit with reduced secretion.

In June 2022, the blind-ending subcutaneous fistula of the thigh could be successfully treated by surgery after the active inflammation resolved. In April 2023, the abdominal fistula also showed no remaining inflammation, so a complete excision of the abdominal fistulas was performed. At the patients last visit in August 2023, no new fistula or abscesses were detected, but ustekinumab was continued due to the long and complicated clinical history. Written informed consent for publication of their clinical details and clinical images was obtained from the patient.

Review of the current literature

Material and methods.

A data base literature search was performed using the keywords “metastatic” and “Crohn’s” and “disease”. 1,875 reports published from January 2012 to November 2023 were found. The number of papers meeting the search criteria steadily increased, highlighting the clinical relevance of this topic.

So far, 59 case reports including 171 patients and 12 reviews about clinical presentation, diagnostic approach and therapeutic options have been published. However, no statistical analysis of patient characteristics and treatments is available. Therefore, our aim was to objectively assess these items. Moreover, we intended to discuss ustekinumab as a novel but successful therapeutic approach for metastatic Crohn’s disease.

Of the patients with metastatic Crohn’s disease, 74% were female, and 38% of the cases involved individuals under 18 years old. From this cohort, we assessed diagnostic and therapeutic approaches for metastatic Crohn’s disease.

80% of the patients were diagnosed with intestinal Crohn’s disease before or at the time skin lesions appearance, while 10% exhibited gastrointestinal symptoms such as diarrhea or abdominal pain without IBD [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. 7% had isolated extraintestinal manifestations as illustrated in our presented case. Fistulas were described in 13% of all cases [ 4 , 20 , 21 , 22 ], highlighting the rarity of this condition. However, fistula presence or absence was not explicitly mentioned in 58% of all published cases.

At the time of diagnosing metastatic Crohn’s disease, usually skin biopsies are taken and examined. Granulomas were present in 58% of all cases [ 3 , 4 , 5 , 6 , 7 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ], with some studies not commenting on histological findings.

(Fig.  4 ).

Proportion of patients with GI-Involvement, granuloma and fistula ( n  = 171)

The most common localization of metastatic Crohn’s disease was the genital region, including groin, vulva and penis/scrotum (Fig.  5 ). Notably, our case report highlights that metastatic Crohn’s disease can affect almost every part of the body. This is in accordance with other reports describing affected skin at various sites of the body including extremities, trunk or head/face [ 4 , 5 , 16 , 22 , 25 , 30 , 31 , 32 , 36 , 37 , 38 , 41 , 44 , 47 , 48 ].

Localisation of lesions in patients with metastatic Crohn’s disease ( n  = 159)

Less common sites of involvement included intraabdominal abscesses [ 31 , 49 ] and necrotizing endocarditis of the aortic valve with granuloma formation [ 26 ]. Some patients also exhibited lesions on more than one site. They were counted for each site in Fig.  5 . For 12 patients, the site of involvement was not mentioned.

There are no general recommendations for the treatment of metastatic Crohn’s disease, but similar pathophysiological processes are suspected. Hence, established therapeutic regimens for the treatment of intestinal Crohn’s disease are employed. The most common therapeutic approach involves corticosteroid admission (Figure  6 ), often resulting in a lesion reduction. Anti-TNF, especially infliximab, was effective and utilized in 41.6% of cases. Some patients received multiple therapies and were counted accordingly (Fig.  6 ).

Therapeutic approaches for metastatic Crohn’s disease ( n  = 219)

Steroid treatment led to remission or improvement in 30.8% of all described cases (Table  1 ). Excluding unknown cases, steroids induced remission (17.2%) or improvement (55.2%) in 72.4% of patients.

Anti-TNF, especially infliximab, less common also adalimumab and certolizumab, only had an effect (improvement or remission) in 15.6% of the cases. However, excluding unknown cases, remission was induced in 13.3% and improvement in 53.3% of patients (Table  1 ).

Antibiotics, azathioprine and topical tacrolimus also showed positive effects, although azathioprine did not induce remission in any patient. Data regarding surgical interventions, mesalazine, anti-TNF plus azathioprine, MTX/6-MP, ustekinumab, and hyperbaric oxygen therapy are limited and inconclusive based on the literature. Notably and in line with the presented case, two out of three patients achieved remission after treatment with ustekinumab.

Our examination summarizes patient characteristics in metastatic Crohn’s disease and outlines therapeutic options based on the presented case. However, there are some limitations.

We established the diagnosis of metastatic Crohn’s diseases based on clinical and histological findings. Other granulomatous diseases such as sarcoidosis, tuberculosis and immunodeficiencies were ruled out, and histological findings were typical for Crohn’s lesions despite there was no intestinal involvement at the time of diagnosis. However, there is no possibility of completely ruling out any other underlying immunological condition causing similar symptoms. The analyzed case reports mostly also based their diagnosis on histological findings fitting Crohn’s criteria, and often on a known underlying intestinal Crohn’s disease, but in some publications, diagnostic criteria were not discussed in detail.

Our analysis is a summary of published cases and no randomized trial, making it impossible to compare the efficiencies of different therapies. This article outlines various treatment approaches and their success rates according to literature data. However, assessing the patient's condition before and after treatment from literature data can be challenging.

Several patients responded well to antibiotic treatment (see Table  1 ). This was also effective in some patients with GI involvement. Nevertheless, it remains unclear if lesions described as metastatic Crohn’s disease are infectious or a manifestation of Crohn’s disease.

Despite these limitations, treatments for metastatic Crohn`s disease could be evaluated. The most effective treatments, according to the current literature, include steroids and anti-TNF-antibodies. Despite the small number of cases, also azathioprine showed good clinical results. Mesalazine also appeared to positively impact skin lesions in patients with metastatic Crohn’s disease (see Table  1 ).

Conclusively, mesalazine, azathioprine, steroids, and anti-TNF antibodies should be considered as first line therapy for metastatic Crohn´s disease.

In case of treatment failure, there are less common therapeutic options such as ustekinumab, surgical intervention or hyperbaric oxygen therapy, which can be offered to the patient. Although the efficacy cannot be evaluated based on the limited amount of available data, the existing literature as well as our case report suggests a positive effect of ustekinumab in patients with metastatic Crohn’s disease.

Topical tacrolimus can improve the lesions, but there is no case report where topical tacrolimus could induce remission [ 3 , 5 , 9 , 24 , 39 , 57 , 61 ], and can currently not be recommended as a single therapeutic option. It might be useful to support other therapies.

Metastatic Crohn’s disease can affect patients with or without GI involvement. Diagnostics include anamnesis, inspection and biopsies of the involved site as well as endoscopy and MRI.

Currently, with no German, European or American guidelines available, approved therapies for intestinal Crohn’s disease are employed, based on the suspected similar inflammatory pathophysiology in intestinal and extraintestinal sites: Steroids, anti-TNF antibodies, and antibiotics were the primary and most potent agents. Mesalazine and azathioprine as well as less common treatment options such as surgical intervention or hyperbaric oxygen therapy may also be considered in case of treatment failure.

With the increasing number of reported cases of metastatic Crohn’s disease, the need for future guidelines for treating these patients becomes apparent. Our case demonstrates successful ustekinumab treatment for metastatic Crohn’s disease, suggesting a potential new therapeutic option.

Availability of data and materials

Laboratory results, histological findings, imaging and other diagnostic results are available if necessary.

Ickrath F, Stoevesandt J, Schulmeyer L, Glatzel C, Goebeler M, Kerstan A. Metastatic Crohn’s disease: an underestimated entity. J Dtsch Dermatol Ges. 2021;19(7):973–82.

PubMed   Google Scholar  

Kurtzman DJB, Jones T, Lian F, Peng LS. Metastatic Crohn’s disease: a review and approach to therapy. J Am Acad Dermatol. 2014;71(4):804–13.

Article   PubMed   Google Scholar  

Aberumand B, Howard J, Howard J. Metastatic Crohn’s disease: an approach to an uncommon but important cutaneous disorder. Biomed Res Int. 2017;2017:8192150.

Article   PubMed   PubMed Central   Google Scholar  

Ballester Sánchez R, Sanchís Sánchez C, Rodrigo Nicolás B, Valcuende CF. Enfermedad de Crohn metastásica tratada con ustekinumab. Actas Dermosifiliogr. 2021;112(2):182–3.

Blasco Alonso J, Girón Fernández-Crehuet F, Lendínez Ramírez MA, Gallego Gutiérrez S, Luque Pérez S, Serrano Nieto J, et al . Metastatic Crohn’s disease in pediatrics. Rev Esp Enferm Dig. 2016;108(9):598–603.

Kapur SV, Oswal JS, Viswanathan V. Vulval edema as a manifestation of childhood metastatic Crohn’s disease. Indian J Pediatr. 2021;88(2):182. https://doi.org/10.1007/s12098-020-03419-4.pdf .

Kyriakou G, Gkermpesi M, Thomopoulos K, Marangos M, Georgiou S. Metastatic vulvar Crohn’s disease preceding intestinal manifestations: a case report and short review. Acta Dermatovenerol Alp Pannonica Adriat. 2019;28(3):131–3.

Janssen Cilag International, editor. Fachinformation Ustekinumab. 2023.

Barrick BJ, Tollefson MM, Schoch JJ, McEvoy MT, Hand JL, Wieland CN, et al . Penile and scrotal swelling: an underrecognized presentation of Crohn’s disease. Pediatr Dermatol. 2016;33(2):172–7.

Gulseren D, Ersoy-Evans S. Metastatic Crohn disease with groin localization in an adult patient. Dermatol Pract Concept. 2022;12(2): e2022056.

Kim SSY, Flannigan RK, Samarasekera D, Terry J, Macneily AE. Case: metastatic Crohn’s disease of the genitalia in a prepubescent male: an illustrative case of an uncommon diagnosis. Can Urol Assoc J. 2017;11(9):E379–81.

Kim S, Won YB, Seo SK, Cho S, Choi YS, Lee BS, et al . Vulvar Crohn’s disease in an adolescent diagnosed after unsuccessful surgical treatment. BMC Womens Health. 2021;21(1):316.

Lanka P, Lanka LR, Sylvester N, Lakshmi MD, Ethirajan N. Metastatic Crohn’s disease. Indian Dermatol Online J. 2014;5(1):41–3.

Mazzoni D, Bodapati S, Mortimore R, Davies J, Wheller L. Metastatic Crohn’s disease in a boy presenting with genital swelling. Pediatr Dermatol. 2020;37(6):1165–6.

Mirheydar HS, Friedlander SF, Kaplan GW. Prepubertal male genitourinary metastatic Crohn’s disease: report of a case and review of literature. Urology. 2014;83(5):1165–9.

Pérez-Santiago L, García-Vázquez A, Martín-Arévalo J. A diagnostic challenge: chronic cutaneous ulcers as an isolated clinical finding in metastatic Crohn’s disease. Rev Esp Enferm Dig. 2022. https://doi.org/10.17235/reed.2022.8995/2022 .

Article   Google Scholar  

Saito K, Iwata Y, Nakajima Y, Numata S, Sugiura K. Metastatic Crohn’s disease in childhood: a case report. J Dermatol. 2018;45(7):e199–200.

Schneider SL, Foster K, Patel D, Shwayder T. Cutaneous manifestations of metastatic Crohn’s disease. Pediatr Dermatol. 2018. https://doi.org/10.1111/pde.13565 .

Vahabi-Amlashi S, Molkara S, Shahrokhi Y. Cutaneous Crohn disease without intestinal manifestations. Adv Biomed Res. 2021;10:39.

Rani U, Russell A, Tanaka S, Correa H, Nicholson MR. Urogenital manifestations of metastatic Crohn’s disease in children: case series and review of the literature. Urology. 2016;92:117–21.

Batra J, Goraya SK, Grewal S, Singh A. Metastatic Crohn’s Disease of the Vulva: A Rare Presentation. Indian Dermatol Online J. 2020;11(3):416–8.

Farkas K, Nagy F, Kovács L, Wittmann T, Molnár T. Anti-tumor necrosis factor-α induced systemic lupus erythematosus in a patient with metastatic Crohn’s disease–what is the role of anti-TNF antibody? J Crohns Colitis. 2013;7(4):e143–5.

Adachi A, Komine M, Murata S, Ohtsuki M. Japanese case of metastatic cutaneous Crohn’s disease of the groin. J Dermatol. 2015;42(2):224–5.

AlAsmari A, AlEssa R, AlAjroush W, AlKhodair R, AlHaddad S. Novel association of metastatic Crohn’s disease and Wolman disease. JAAD Case Rep. 2022;20:40–3.

Campos S, Coutinho I, Cardoso JC, Portela F. Metastatic Crohn’s disease despite infliximab therapy. An Bras Dermatol. 2017;92(5 Suppl 1):104–6.

Gettigan NM, Sheehan M, Slattery E. Metastatic Crohn’s disease of the aortic valve resulting in severe aortic insufficiency, non-infective endocarditis, and pericarditis. Inflamm Bowel Dis. 2019;25(10):e117–8.

Guglielmetti A, Gompertz M, Jahr C, Silva T, González S. Remission of refractory metastatic Crohn’s disease achieved with dapsone. Int J Dermatol. 2018;57(4):467–9.

Hogan N, Byrnes V, Hussey A, Joyce M. Surgical management of gluteal metastatic cutaneous Crohn’s disease. J Crohns Colitis. 2012;6(5):617–20.

Ishida M, Iwai M, Yoshida K, Kagotani A, Okabe H. Metastatic Crohn’s disease accompanying granulomatous vasculitis and lymphangitis in the vulva. Int J Clin Exp Pathol. 2013;6(10):2263–6.

PubMed   PubMed Central   Google Scholar  

Kiuru M, Camp B, Adhami K, Jacob V, Magro C, Wildman H. Treatment of metastatic cutaneous Crohn disease with certolizumab. Dermatol Online J. 2015;21(11):4.

Labidi A, Karoui S, Ben Mustapha N, Serghini M, Azzouz H, Allouche W, et al . Cutaneous metastatic Crohn’s disease of the abdominal wall: an exceptional location. Tunis Med. 2013;91(6):424–5.

Lang N, Hartschuh W, Enk A, Toberer F. Metastatic Crohn’s disease: a diagnostic and therapeutic challenge. J Dtsch Dermatol Ges. 2015;13(6):571–4.

Lansdorp CA, Buskens CJ, Gecse KB, D’Haens GR, van Hulst RA. Wound healing of metastatic perineal Crohn’s disease using hyperbaric oxygen therapy: a case series. United Eur Gastroenterol J. 2020;8(7):820–7.

Article   CAS   Google Scholar  

Lim Y, Singh M. Metastatic vulval crohn’s disease with good outcome on Ustekinumab. Cureus. 2021;13(7): e16252.

Math CJ, George A. Vegetating plaques in the groin: a manifestation of metastatic Crohn’s disease. Indian J Dermatol. 2018;63(4):338–41.

Misago N, Narisawa Y. Erythema induratum (nodular vasculitis) associated with Crohn’s disease: a rare type of metastatic Crohn’s disease. Am J Dermatopathol. 2012;34(3):325–9.

Park HC, Kim HW, Park CG, Ko JY. Metastatic Crohn disease clinically reminiscent of erythema nodosum on the right leg. Cutis. 2016;98(3):E11–5.

Patel AV, Jones DM, Hill JC, MacDermott RP. Development of metastatic Crohn’s disease of the skin while on anti-TNF biologics. Inflamm Bowel Dis. 2012;18(6):1188–90.

Pérez-Belmonte LM, Gómez-Moyano E. Metastatic Crohn’s disease. Indian J Med Res. 2015;142(4):497.

Pousa-Martínez M, Alfageme F, de Domingo González MA, Suárez-Masa D, Calvo M, Roustán G. Vulvar metastatic Crohn disease: clinical, histopathological and ultrasonographic findings. Dermatol Online J. 2017;23(11):21.

Ruiz-Villaverde R, Sánchez-Cano D, Perez-Lopez I, Aneiros-Fernández J. Enfermedad de Crohn metastásica. Actas Dermosifiliogr. 2017;108(2):171–2.

Article   CAS   PubMed   Google Scholar  

Sabbadini C, Banzato C, Schena D, Peroni D, Girolomoni G. Metastatic Crohn’s disease in childhood. J Deutsche Derma Gesell. 2016. https://doi.org/10.1111/ddg.12861 .

Sackett DD, Meshekow JS, Figueroa TE, Napoli JA. Isolated penile lymphedema in an adolescent male: a case of metastatic Crohn’s disease. J Pediatr Urol. 2012;8(5):e55–8.

Streight KL, Braun TL, Lowe N, Kim SJ. A rare clinical presentation of metastatic Crohn’s disease. Cureus. 2020;12(5): e8285.

Vint R, Husain E, Hussain F, McClinton S, Ormerod A. Metastatic Crohn’s disease of the penis: two cases. Int Urol Nephrol. 2012;44(1):45–9.

Wylomanski S, Bouquin R, Dréno B, Quéreux G. Spectacular response of metastatic vulval Crohn’s disease to infliximab treatment. Int J Dermatol. 2016;55(10):1146–8.

Gontijo JRV, Leidenz FAB, de Sousa MSLA. Case for diagnosis. Metastatic Crohn’s disease. An Bras Dermatol. 2016;91(4):531–3.

Her Y, Park SE, Kim SS, Kim CW. Metastatic cutaneous Crohn’s disease in a child. Eur J Dermatol. 2015;25(1):74–5.

Amarapurkar DN, Sonavane A, Amarapurkar AD. Metastatic Crohn’s Disease. J Assoc Phys India. 2017;65(4):86–8.

Google Scholar  

Alotaibi HM, Fathaddin AA, AlMutairi HM, Barakeh MM. Metastatic Crohn’s disease in external genitalia with good outcome on adalimumab: a rare case of a Saudi female and a short review. Cureus. 2023;15(8): e43380.

Bhoyrul B, Lyon C. Crohn’s disease of the vulva: a prospective study. J Gastroenterol Hepatol. 2018;33(12):1969–74.

Chatterjee D, Bhattacharjee R, Khullar G, Kumaran S, De D, Saikia UN, et al . Metastatic Crohn disease: a clinicohistological appraisal from a tertiary care center in India. Am J Dermatopathol. 2020;42(7):506–12.

Costa Blasco M, McFeely O, Doyle C, Wolinska A, Andrawis M, Murphy L, et al . Metastatic Crohn disease improving with vedolizumab. Br J Dermatol. 2023;189(2): e35.

Das D, Gupta B, Saha M. Metastatic vulvar Crohn’s disease-a rare case report and short review of literature. Indian J Dermatol. 2016;61(1):70–4.

Dederichs F, Iesalnieks I, Sladek M, Tzivinikos C, Hansen R, Muñoz C, et al . Genital granulomatosis in male and female patients with Crohn’s disease: clinical presentation and treatment outcomes. J Crohns Colitis. 2018;12(2):197–203.

Ferreira de Castro L, Gomes R, Sá DC, Fernandes E, Lima R, Tavares M. Metastatic Crohn’s disease in paediatrics: Vulvar lesions as the first clinical presentation. J Paediatr Child Health. 2023;59(9):1092–4.

García-Delgado R, Escario-Travesedo E, Sánchez-Romero A. Absorción sistémica de tacrolimus tópico en enfermedad de Crohn metastásica con úlceras cutáneas. Actas Dermosifiliogr. 2016;107(10):866–7.

Pampín A, Gamo R, Andreu-Barasoain M, Pinedo F, La Gómez-de FE, López-Estebaranz JL. Metastatic Crohn’s disease in genital and perianal area preceding 11 years intestinal Crohn’s disease. Int J Dermatol. 2014;53(3):e176–7.

Pereira AS, Coutinho I. A challenging case of metastatic Crohn’s disease without gastrointestinal manifestations. Cureus. 2023;15(9): e45791.

Raman LG, AbdullGaffar B. Metastatic Crohn disease revealing vascular embolization: report of two cases. J Cutan Pathol. 2023;50(11):929–32.

Shields BE, Richardson C, Arkin L, Kornik R. Vulvar Crohn disease: diagnostic challenges and approach to therapy. Int J Womens Dermatol. 2020;6(5):390–4.

Xiao TL, Ezenwa E, Ruiz de Luzuriaga A, Hoffman MD. Refractory metastatic Crohn’s disease responsive to ustekinumab dose intensification. JAAD Case Rep. 2023;32:65–7.

Woody MM, Holliday AC, Gavino ACP, McReynolds A, Soldano AC. Metastatic vulvovaginal Crohn disease in the setting of well-controlled intestinal disease. Cutis. 2018;102(2):E16–8.

Download references

Acknowledgements

Not applicable.

Open Access funding enabled and organized by Projekt DEAL. This study received no funding.

Author information

Authors and affiliations.

Department of Internal Medicine I, Gastroenterology, Hepatology, Endocrinology, Rheumatology and Infectious Diseases, University Hospital Regensburg, Franz-Josef-Strauß-Allee 11, 93053, Regensburg, Germany

Tanja Elger, Johanna Loibl, Christa Buechler, Karsten Guelow, Claudia Kunst, Arne Kandulski, Pia Goeggelmann, Martina Mueller & Hauke Christian Tews

Department of Visceral Surgery, University Hospital Regensburg, Regensburg, Germany

Jens Werner

Department of Dermatology, University Hospital Regensburg, Regensburg, Germany

Sebastian Haferkamp, Konstantin Drexler & Ulrich Hohenleutner

You can also search for this author in PubMed   Google Scholar

Contributions

HCT initiated the project and designed the work; TE and HCT were responsible for case preparation; TE drafted the manuscript; TE, JL, CB, SH, JW, KD, UH, AK, PG, and HCT analyzed results; TE performed literature search; TE prepared figures; TE, CK, KG, MMS, and HCT wrote and edited the manuscript. All authors approved the final version of the manuscript.

Corresponding author

Correspondence to Tanja Elger .

Ethics declarations

Ethical approval and consent to participate.

See section below.

Consent for publication

Written informed consent for publication of their clinical details and clinical images was obtained from the patient. A copy of the written consent is available for review by the Editor-in-Chief of this journal.

Competing interests

All authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Elger, T., Loibl, J., Buechler, C. et al. Fistulising skin metastases in Crohn’s disease: a case report and review of the literature. J Med Case Reports 18 , 252 (2024). https://doi.org/10.1186/s13256-024-04569-1

Download citation

Received : 02 February 2024

Accepted : 10 April 2024

Published : 19 May 2024

DOI : https://doi.org/10.1186/s13256-024-04569-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Metastatic Crohn’s disease
• Skin fistulae
• Ustekinumab
• Inflammatory bowel disease
• Extraintestinal manifestation

Journal of Medical Case Reports

ISSN: 1752-1947

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 15 May 2024

The association between increased fetal movements in the third trimester and perinatal outcomes; a systematic review and meta-analysis

• Sedigheh Hantoushzadeh 1 ,
• Omid Kohandel Gargari 2 ,
• Marzieh Jamali 3 ,
• Fatemeh Farrokh 3 ,
• Nasim Eshraghi 1 ,
• Fatemeh Asadi 1 ,
• Masoumeh Mirzamoradi 4 ,
• Seyed Jafar Razavi 1 ,
• Marjan Ghaemi 1 ,
• Sudabeh Kazemi Aski 1 ,
• Zahra Panhi 1 &
• Gholam Reza Habibi 1  

BMC Pregnancy and Childbirth volume  24 , Article number:  365 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Fetal movement monitoring is one of the strategies used to assess the fetus’s health. Until now, most studies focused on the decreased fetal movement and neonatal outcome, although this systematic review and meta-analysis is designed to assess the association between increased fetal movements (IFM) with perinatal outcomes.

The electronic databases including PubMed, Scopus, Web of Science, and EMBASE were systematically searched for studies investigating the perinatal outcome of women with increased fetal movements from inception to July 2023. Following that, a random-effect meta-analysis model was used to obtain the combined diagnostic and predictive parameters including perinatal mortality (still birth and early neonatal mortality), operative delivery, Apgar score, neonatal resuscitation at birth and NICU Admission.

After the initial screening, seven studies examining the association between increased third trimester fetal movement and various perinatal outcomes were included. Meta-analysis revealed a significant reduction in the risk of cesarean delivery among patients with IFM compared to controls, suggesting a potential protective effect during childbirth. However, no statistically significant difference was observed in birth weight, small or large for gestational age births, neonatal intensive care unit admission, maternal age, umbilical cord around the neck, gestational diabetes mellitus, and hypertension, indicating that IFM may not be a major predictor of adverse perinatal outcomes or maternal conditions. Notably, IFM was significantly associated with a higher likelihood of labor induction.

The findings suggest that IFM may have a protective effect against cesarean delivery. Additionally, IFM does not appear to be significantly associated with maternal age, umbilical cord around the neck, gestational diabetes mellitus and hypertension. However, the observed significant association with labor induction warrants further investigation.

Peer Review reports

Adverse perinatal outcomes are the main causes of neonatal mortality and morbidity. It is estimated that 2.5 million late gestation stillbirths and 3 million neonatal deaths occur globally each year [ 1 ]. Moreover, over 40% of all stillbirths that happen during labor and delivery could have been prevented with improved fetal monitoring and access to emergency obstetric care when needed [ 2 ]. Various factors may contribute to stillbirth, including maternal health and obstetric condition, sociodemographic and economic status, congenital abnormalities, pregnancy disorder, and placental dysfunction [ 3 , 4 ]. It seems crucial to determine the modifiable risk factors to develop prevention strategies and reduce the risk of stillbirth [ 5 ].

Fetal well-being can be monitored by the number of fetal movements being perceived by the mother [ 6 ]. Reduced fetal movement (RFM) is typically defined as a decrease in the mother’s perception of fetal movements or a change in the pattern of normal movement [ 7 ]. A great number of studies that evaluated the relationship between maternal understanding of fetal activity and pregnancy outcomes, were focused RFM. There is some evidence to support that RFM is associated with poor pregnancy outcomes [ 8 ]. It was estimated that more than half of the pregnant women who face stillbirth understood the reduction in fetal movements before the diagnosis. The likely mechanism behind RFM was explained as impaired placental function, which led to increased inflammation and limited blood supply in fetuses experiencing RFM [ 9 ]. So, guidelines have been developed based on the best recent findings, to provide recommendations for clinicians to manage women with reduced fetal movement [ 10 , 11 ].

Many studies have been conducted to identify modifiable factors associated with stillbirth, which could lead to direct interventions to reduce the incidence. These studies have examined various factors, encompassing both maternal and fetal aspects, that may contribute to the occurrence of stillbirth. Some of the modifiable factors that have been explored include maternal weight, illicit drug use, the quality of obstetrical care during pregnancy, sleeping conditions, and the condition of being small for gestational age (SGA) [ 12 , 13 , 14 ].

Regarding the maternal perception of fetal movements, some clinical data indicated that increased movements are associated with adverse perinatal outcomes [ 15 , 16 ]. But these data are currently insufficient and sparse [ 17 ]. In addition, there is no systematic review that evaluated the risk of adverse outcomes such as stillbirth among women reporting this symptom, and also, it is still unclear how to reduce the risk. The objective of this study is to assess if there is any association between increased fetal movements and perinatal outcomes (included: neonatal mortality, neonatal intensive care unit (NICU) admission, fetal distress, low Apgar score). As a result of this information, medical professionals would be able to provide pregnant women with more comprehensive care, and women would also be able to receive more detailed information to reduce their anxiety.

Search strategy

We conducted this systematic review while fully adhering to the guidelines available at the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). To identify the published studies of interest, we prepared a search strategy comprising strings of keywords related to our study’s objectives from inception to July 2023, provided as supplementary file S1. PubMed, SCOPUS, Embase, and Web of Science databases were systematically searched for record identification.

Selection criteria

To appropriately investigate the identified studies for eligibility, we considered a framework for the investigation of risk of exposure with health outcomes in studies, known as PICO (Population, Intervention, Comparator, Outcomes). We only considered English observational studies (i.e., case-control studies, cohorts, and cross-sectional studies) that investigated perinatal outcomes among women with increased fetal movements (IFM). Therefore, all interventional studies, case series, case reports letters to the editors, meetings, and conference abstracts or proceedings, surveys, editorials, and reviews were excluded.

IFM is diagnosed using assessment surveys, to avoid missing studies we also included studies recruiting mothers with self-reported IFM

The population in our study were pregnant women with an age > 18 years old, gestational age over 20 weeks, and a singleton, non-anomalous pregnancy. Moreover, pregnant women under age 18, gestational age under 20 weeks, Multiple pregnancies, any established fetal anomalies, and any maternal medical conditions demanding medications that can affect fetal movement were excluded.

Patients should have learned how to measure fetal movements and report them in a specific time interval planned by researchers in each article, or their fetal movements should be measured by questionnaire. Either IFM sensation, surveys and interviews were considered acceptable for IFM assessment. The control groups should be selected from healthy population pregnancies with gestational age-matched (± 7 days) with the intervention group and normal pregnancy outcome. All articles must measure the pregnancy outcomes concerning Stillbirth.

Inclusion criteria:

English observation studies (Cohort, cross-sectional and case-control).

pregnant women with an age > 18 years old, gestational > 20 weeks,

Singleton and non-anomalous pregnancy.

Included patients with IFM.

Reported perinatal outcomes.

Data collection and analysis

The study selection, quality assessment, and data extraction were carried out under the supervision of the senior author. We initially collected the identified records from the four mentioned databases and checked for duplicates using the 20th version of the Endnote software package. Then using the duplicate removal tool provided by Rayyan Incorporation10, any remaining duplicate records were manually removed. Next, two authors independently screened the resulting studies based on their titles and abstracts, removing those deemed irrelevant. They screened the records passing through the first round based on their full texts, excluding the ineligible studies. If discrepancies occur in the study selection stages, they are resolved by senior author recommendations.

two authors independently extracted the data from the eligible studies using a pre-specified flexible data extraction form in an Excel Microsoft office spreadsheet. These data included the study’s first author, country, the year it was conducted, study type, method assessment time, fetal movement assessment period estimated outcome, total sample size, case group size, control group size, age case (mean/SD), age control (mean/SD), gestational age range, method of assessment, number of women who experienced increased fetal movements, the study hypothesis and the outcome based on the method of assessment.

Outcomes include type of delivery (cesarean delivery, induced or natural vaginal delivery), birth weight, small for gestational age (SGA), large for gestational age (LGA), neonatal intensive care unit (NICU) admission, maternal age, umbilical cord around the neck (UCNA), gestational diabetes mellitus (GDM), hypertension (HTN), and labor induction.

Adverse perinatal outcomes were defined as perinatal mortality (still birth and early neonatal mortality defined as newborn death within the first 7 days), operative delivery (cesarean section or vacuum) due to fetal distress, Apgar min 5 < 7, neonatal resuscitation at birth (including both invasive ventilation such as mechanical ventilation; non-invasive ventilation such as oxygen therapy, nasal CPAP, high flow), and NICU Admission. In case of any discrepancies, it was resolved by senior third authors. The authors of included articles were contacted to provide additional information.

For outcomes with two or more studies random effect model was used for data pooling and calculating summary estimates. Risk ratio and odds ratio would be used for dichotomous outcomes and the standard mean difference would be used for continuous variables. According to the Cochrane handbook, two studies could be used to perform meta-analysis if their results are sufficiently similar [ 18 ]. The quality of the included studies was assessed by utilizing the tools recommended by the Joanna Briggs Institute (JBI) [ 19 , 20 ]. The quality assessment table is provided in supplementary file S2.

Study selection

Database search resulted in 6292 studies and after duplicate removal 3505 studies underwent title and abstract screening. 98 studies were selected for eligibility assessment and full-text screening. Finally, 7 studies were selected to be included in this study. Two studies, although reported fetal movement, did not have a control group and were excluded [ 21 , 22 ], one study was excluded because it did not report neonatal outcomes [ 23 ] (Fig.  1 ).

PRISMA 2020 flow diagram for new systematic reviews which included searches of databases and registers only

Flow diagram of the recruiting studies according to PRISMA

Study characteristics

Four out of seven included studies were cohort studies designed to compare patients with increased fetal movement with regular pregnancies [ 24 , 25 , 26 , 27 ]. Three studies were case-control studies which compared patients with adverse neonatal outcomes with controls and reported increased fetal movement as one of their outcomes [ 28 , 29 , 30 ]. Table  1 summarizes characteristics of included studies.

Risk of bias in studies

No significant bias was identified among included studies (Supplementary file).

Cohort studies

Type of delivery.

Pooled risk ratio of cesarean delivery was calculated among 422 patients with IFM and 17,649 controls in three studies. Risk of cesarean delivery was significantly lower among patients with IFM. Overall estimates are the following: Random effect model, RR = 0.82, 95%CI; [0.69-0,97], P  = 0.02, Fig.  2 . No significant heterogeneity was found among included studies: I 2  = 0%, tau 2  = 0.00.

Forest plot for random-effect meta-analysis comparing risk of cesarean delivery between increased fetal movement patients and control group

Birth weight

Mean birth weights of 640 IFM patients and 44,009 controls were reported in three studies, pooled standard mean difference of birthweight was calculated via random effect model. Birth weight was not significantly different between two groups. Overall estimates are the following: SMD = 0.09, 95%CI; [-0.15, 0.32], P  = 0.46, Fig.  3 . There was high heterogeneity among findings of included studies. I 2  = 84%, tau 2  = 0.04.

Forest plot for random-effect meta-analysis comparing standard birth weight mean difference between increased fetal movement patients and control group

Small for gestational age

Number of neonates small for their gestational age was reported in three studies including 640 IFM patients and 44,009 controls. Pooled risk ratio of SGA birth was not significantly different between two groups. Pooled estimates are the following: RR = 0.98, 95%CI [0.72, 1.33], P  = 0.9, Fig.  4 . No significant heterogeneity was found among studies: I 2  = 0, tau 2   = 0.00.

Forest plot for random-effect meta-analysis comparing risk of small for gestational age (SGA) between increased fetal movement patients and control group

Large for gestational age

LGA was reported in three studies including overall of 44,009 IFM cases and 640 controls. Similar to SGA, IFM was not significantly associated with higher risk of LGA birth. Summary estimates are the following: Random effect model, RR = 1.01, 95%CI; [0.63, 1.60], P  = 0.97, Fig.  5 . there was moderate heterogeneity among findings of these studies: I 2  = 48%, tau 2  = 0.08.

Forest plot for random-effect meta-analysis comparing risk of large for gestational age (LGA) between increased fetal movement patients and control group

NICU admission

Number of NICU admitted neonates were reported in three studies including 565 cases of IFM and 60,782 controls. Risk of NICU admission was not significantly different between two groups. Summary estimates are the following: Random effect model, RR = 1.02, 95%CI; [0.62, 1.68], P  = 0.99, Fig.  6 . No significant heterogeneity was present; I 2  = 0, tau 2  = 0.00.

Forest plot for random-effect meta-analysis comparing risk of NICU admission between increased fetal movement patients and control group

Maternal age

Maternal age was reported in two studies, but Sharp et al. study only reported maternal age of IFM cases and did not report mean maternal age of controls. Standard mean difference of maternal age was not significantly different between two groups. Summary estimates are the following: SMD= -0.08, 95%CI; [-0.24-0.07], P  = 0.28, Fig.  7 .

Forest plot for random-effect meta-analysis comparing maternal age between increased fetal movement patients and control group

Umbilical cord around neck

Two studies including 502 IFM cases and 43,710 controls reported occurrence rate of umbilical cord around neck. Pooling via random effect model showed that IFM is not associated with higher or lower risk of umbilical cord around neck. Summary estimates are the following: RR = 1.06, 95%CI; [0.89, 1.25], P  = 0.51, Fig.  8 .

Forest plot for random-effect meta-analysis comparing rate of umbilical cord around neck between increased fetal movement patients and control group

Two studies including 372 cases of IFM and 577 controls reported GDM prevalence among included mothers. Pooling showed no significant association GDM and IFM: OR = 1.27, 95%CI; [0.71, 2.29], P  = 0.78, I 2  = 0, tau 2   = 0.00 , Fig.  9 .

Forest plot for random-effect meta-analysis comparing prevalence of GDM between increased fetal movement patients and control group

Two studies including 372 cases of IFM and 577 controls reported HTN prevalence among included mothers. Pooling showed no significant association HTN and IFM: Fixed effect, OR = 1.01, 95%CI; [0.53, 1.92], P  = 0.98, I 2  = 0, tau 2   = 0.00 , Fig.  10 .

Forest plot for random-effect meta-analysis comparing prevalence of HTN between increased fetal movement patients and control group

Two studies including 203 cases of IFM and 8,430 controls reported number of cases underwenting labor induction. Pooling via fixed effect model showed that IFM is significantly associated with higher risk of labor induction; RR = 1.25, 95%CI; [1.01, 1.55], P  = 0.04, I 2  = 0, tau 2   = 0.00 ,, Fig.  11 .

Forest plot for random-effect meta-analysis comparing number of cases underwenting labor induction between increased fetal movement patients and control group

Case-control studies

Adverse perinatal outcomes.

Three case-control studies including an overall of 374 cases with adverse neonatal outcomes and 871 controls reported IFM as one their outcomes. Pooled Odds of increased fetal movement was higher among cases compared to controls but it was not significant, random effect model, OR = 3.09, 95%CI; [0.94, 10.16], P  = 0.06, Fig.  12 .

Forest plot for random-effect meta-analysis comparing odds of increased fetal movement between cases with adverse perinatal outcomes and regular pregnancy controls. Adverse perinatal events were defined as perinatal mortality (still birth and early neonatal mortality), operative delivery (cesarean section or vacuum) due to fetal distress, Apgar min 5 < 7, neonatal resuscitation at birth (including both invasive ventilation such as mechanical ventilation; non-invasive ventilation such as oxygen therapy, nasal CPAP, high flow), and NICU Admission

The present study aimed to investigate the association between increased fetal movement (IFM) and various perinatal outcomes. To achieve this, a systematic review was conducted, resulting in the inclusion of 7 studies. The selected studies were analyzed to determine the relationship between IFM and outcomes such as type of delivery (cesarean delivery, induced or natural vaginal delivery), birth weight, small for gestational age (SGA), large for gestational age (LGA), neonatal intensive care unit (NICU) admission, maternal age, umbilical cord around the neck (UCNA), gestational diabetes mellitus (GDM), hypertension (HTN), and labor induction.

In terms of the type of delivery, the pooled analysis indicated that patients with IFM had a significantly lower risk of cesarean delivery compared to controls. It has been indicated that women with a gestational age of more than 37 weeks tend to experience a higher prevalence of IFM. Additionally, primigravid women have been found to have more episodes of excessive fetal movements compared to multiparous women [ 31 ]. This finding could potentially be attributed to a lower rate of cesarean sections in primigravid women and term pregnancies, or it could be an incidental observation.

However, when examining birth weight, no significant difference was observed between infants with IFM and controls. This implies that IFM does not seem to have a substantial impact on the birth weight of infants. Similarly, no significant difference in the risk of infants being born small for gestational age (SGA) or large for gestational age (LGA) was found between the IFM group and controls. These results suggest that IFM may not be a strong predictor of abnormal fetal growth. The analysis of NICU admission rates revealed no significant difference between neonates with IFM and controls. This suggests that IFM does not appear to be associated with an increased risk of neonatal complications requiring admission to the neonatal intensive care unit. Regarding maternal age, the pooled analysis did not find a significant difference between the IFM group and controls, indicating that increased fetal movement is not likely to be influenced by maternal age.

The occurrence rate of umbilical cord around the neck (UCNA) was also examined, and the results showed no significant association with IFM. This indicates that IFM is not a major factor contributing to the presence of UCNA during delivery. The prevalence of gestational diabetes mellitus (GDM) and hypertension (HTN) among mothers with IFM was also investigated. The analysis did not find a significant association between IFM and either GDM or HTN. However, it is worth noting that in the context of labor induction, the pooled analysis showed a significant association between IFM and a higher risk of undergoing labor induction. This finding implies that IFM may be a relevant factor in determining the need for labor induction.

In the case-control studies examining adverse perinatal outcomes including still birth, the pooled odds of increased fetal movement were higher among cases with adverse neonatal outcomes compared to controls. However, this result was not statistically significant, indicating that the relationship between IFM and adverse perinatal outcomes requires further investigation.

Consistent with the results of this study, a cohort study found a significant increase in vaginal delivery in the IFM group compared to the control group [ 31 ]. Similarly, another study reported a higher rate of induction of labor in women who experienced changes in fetal movement or reduced fetal movement compared to women with normal fetal movements [ 32 ].

In 1977, Sadowsky et al. [ 33 ], conducted a study that initially identified the sudden excessive movement of a fetus as a potential sign of acute fetal distress. Since then, numerous studies have been conducted to explore and better understand this phenomenon. Most of them suggested that increased fetal movements might be associated with Stillbirth or poor perinatal outcomes. As part of the STARS cohort, a web-based survey was used to study 1714 women who had experienced a singleton stillbirth at > 28 weeks gestation. Among them, 39% experienced unusual fetal movements, with 30.5% reporting significantly less movement and 8.5% reporting significantly more movement. In addition to the group that described only increased fetal movements, the number of women described increased movements before decreased movements and fetal deaths [ 21 ]. A similar finding was found in a Swedish study, where 10% of women described abnormally vigorous activity before stillbirth. A sudden increase in the movement was followed by limited or no movement and then fetal death [ 22 ].

Based on the Auckland Stillbirth Study, women who experienced a single episode of more vigorous movements had a seven-fold increased risk of stillbirth. Conversely, women who reported repeated episodes of the increased fetal movement were protected against stillbirth [ 28 ]. The findings of this study were replicated in the UK MiNES Study, in which women with a single episode of increased fetal movements had a two-fold risk of Stillbirth but a reduced risk if the episodes recurred [ 16 , 29 ]. This team also demonstrated that women who reported increased strength of movements in the last 2 weeks had decreased risk of late stillbirth compared with those whose movements were not changed. Moreover, another study in New Zealand showed that maternal perceptions of more vigorous than usual fetal movements were associated with lower risks of late stillbirth [ 23 ].

Except for the increased or excessive fetal movements, fetal hiccups perception, duration, and frequency were also assessed to see the outcomes. Hiccup perception was assessed in 4 papers, reporting the negative effect of maternal perception of fetal hiccups on Stillbirth [ 16 , 23 , 28 , 29 ].

Hazell et al. stated that in unadjusted analysis, daily hiccups or prolonged episodes of hiccups for more than 5 min can reduce stillbirth, while in adjusted analyses, it is not significant anymore. They also reported that the presence of hiccups did not make difference in the pregnancy outcomes between the case and control groups. Bradford et al. have demonstrated that increases in strength and frequency, and fetal hiccups are associated with a decrease in the incidence of stillbirth [ 13 ].

It was reported that women should expect the fetal movement to at least remain as strong or increase in late pregnancy. Women perceive these changes in strength differently, and some may not feel a stronger movement. The perception of the increased strength of movement may simply be due to the increased size of the fetus and the relatively limited space that makes movement more noticeable [ 23 , 31 ].

Indeed, maternal perception of fetal hiccups is common and is associated with a reduced risk of late stillbirth. Fetal hiccups were first reported by Ferroni and are considered to be a normal part of fetal development [ 34 , 35 ]. Increased maternal perception of fetal hiccups near term may result from greater fetal size, changes in fetal breathing, or neurological development. It may also result from increased recognition of fetal hiccups by the mother. Therefore, we can conclude that fetal hiccups are a normal part of pregnancy and are not associated with an increased risk of Stillbirth [ 5 ].

On the other hand, it is still unclear what underlying mechanisms lead to the excessive movements in the fetus; it may be caused by asphyxia, infection, an attempt to release cord entanglement, or a change in fetal behavior (causing signs of distress) in response to noxious stimuli [ 16 ]. Additionally, increased maternal anxiety may lead to an increased perception of fetal activity [ 36 ]. The evidence regarding excessive fetal movements is sparse; there is no clinical guidance regarding reporting this symptom and reducing the risk of subsequent Stillbirth. Cardiotocography and ultrasonography of the fetus and cord could be utilized at the presentation time to evaluate fetal seizures or umbilical cord entanglement [ 37 ].

It is possible to determine whether a mother has been exposed to an infection or noxious stimuli by examining her history and measuring the level of inflammatory markers or toxins in her blood [ 5 , 38 ]. It is possible to assess maternal anxiety using validated anxiety scores [ 39 ]. In excessive fetal movements, fetal outcomes can be recorded after birth. Apgar scores, fetal acidaemia, or stress-related factors in umbilical cord blood can be used to diagnose perinatal asphyxia [ 40 ]. Placentas and cords can be systematically examined for signs of hypoxia, infection, or compression of the umbilical cord. Such studies would provide further evidence regarding the underlying cause of excessive fetal movement and how this symptom might relate to in-utero compromise and Stillbirth. As a result of this approach, we will be able to determine whether excessive fetal movements can be used alongside reduced fetal movements to reduce the risk of perinatal mortality.

The main strength of this study is an extensive collection of pregnancy-related variables from several countries and inclusion of studies with large number of participants. This study has some limitations; the main limitation is low number combinable studies although as mentioned in the methods section doing a meta-analysis is not impossible and included studies have large number patients, our study serves as a motivation for further research. case-control studies may be prone to recall bias due to their nature. However, these studies included interviewer-administered questionnaires, so they did not include hypotheses about the potential association of various patterns of movements that reduced this risk. Selection bias is also possible; however, the reasons for this would likely vary across countries, but the prevalence of fetal movement variables was relatively consistent. It is also noteworthy that we included different types of IFM assessment methods which could lead to bias.

Conclusions

In conclusion, this systematic review and meta-analysis, which included seven studies, investigated the association between increased fetal movement (IFM) and various perinatal outcomes. The findings suggest that no statistically significant difference was found in birth weight, small or large for gestational age births, neonatal intensive care unit admission, maternal age, umbilical cord around the neck, gestational diabetes mellitus, or hypertension, implying that IFM may not be a major predictor of adverse perinatal outcomes or maternal conditions. Nevertheless, the significant association with increased labor induction warrants attention and further investigation. The study highlights the need for future research with larger sample sizes and standardized protocols to validate these associations and enhance our understanding of the impact of increased fetal movement on perinatal outcomes. Due to the limited number of studies included, the current findings should be interpreted with caution, and additional research is crucial to strengthen the evidence base in this area.

Acknowledgments .

Data availability

The dataset supporting the conclusions of this article (i.e., data extracted from included studies) is available upon request to the corresponding author.

Flenady V, Sexton J. Epidemiology of fetal and neonatal death. Keeling’s fetal neonatal Pathol, 2022: p. 131–57.

Ouyang F, et al. Recurrence of adverse perinatal outcomes in developing countries. Bull World Health Organ. 2013;91(5):357–67.

Article   PubMed   PubMed Central   Google Scholar  

Sanchez TE, Meaney S, O’Donoghue K. Modifiable risk factors for stillbirth: a literature review. Midwifery. 2019;79:102539.

Article   Google Scholar  

Flenady V, et al. Major risk factors for stillbirth in high-income countries: a systematic review and meta-analysis. Lancet. 2011;377(9774):1331–40.

Article   PubMed   Google Scholar  

Heazell AEP, et al. Alterations in maternally perceived fetal movement and their association with late stillbirth: findings from the Midland and North of England Stillbirth case-control study. BMJ Open. 2018;8(7):e020031.

Frøen JF. A kick from within – fetal movement counting and the cancelled progress. Antenatal care. 2004;32(1):13–24.

Google Scholar  

Heazell AEP, Frøen JF. Methods of fetal movement counting and the detection of fetal compromise. J Obstet Gynaecol. 2008;28(2):147–54.

Article   CAS   PubMed   Google Scholar  

Bekiou A, Gourounti K. Reduced fetal movements and Perinatal Mortality. Mater Sociomed. 2020;32(3):227–34.

Bradford BF et al. Decreased fetal movements: Report from the International Stillbirth Alliance conference workshop International Journal of Gynecology & Obstetrics. n/a(n/a).

No G-tG. Reduced fetal movements . 2011, February.

Tillett A. Reduced Fetal movements 2019.

Woods JR, Heazell AEP. Stillbirth: is it preventable? Gynecol Reproductive Med. 2018;28(5):148–54. Obstetrics.

Escañuela Sánchez T, Meaney S, O’Donoghue K. Modifiable risk factors for stillbirth: a literature review. Midwifery. 2019;79:102539.

Tshibumbu DD, Blitz J. Modifiable antenatal risk factors for stillbirth amongst pregnant women in the Omusati region, Namibia. Afr J Prim Health Care Fam Med. 2016;8(1):e1–6.

Whitehead CL, et al. Are increased fetal movements always reassuring? J Maternal-Fetal Neonatal Med. 2020;33(21):3713–8.

Heazell AE, et al. Excessive fetal movements are a sign of fetal compromise which merits further examination. Med Hypotheses. 2018;111:19–23.

Hantoushzadeh S, et al. Could the increase in fetal Movement be a sign of the imminent fetal demise? A Case Report. Fertil Gynecol Androl. 2022;2(1):e132085.

R, R., Cochrane Consumers and Communication Group reviews : Meta-analysis . 2016, Cochrane Consumers and Communication Review Group.

Barker TH, et al. Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process. JBI Evid Synthesis. 2023;21(3):478–93.

Munn Z, et al. Assessing the risk of bias of quantitative analytical studies: introducing the vision for critical appraisal within JBI systematic reviews. JBI Evid Synthesis. 2023;21(3):467–71.

Warland J, et al. STARS (study of trends and Associated risks for Stillbirth) consortium 2015, an international internet survey of the experiences of 1,714 mothers with a late stillbirth: the STARS cohort study. BMC Pregnancy Childbirth. 2015;15:172.

Linde A, Pettersson K, Rådestad I. Women’s experiences of fetal movements before the confirmation of fetal death—contractions misinterpreted as fetal movement. Birth. 2015;42(2):189–94.

Bradford B, Maude R. Maternal perception of fetal movements in the third trimester: a qualitative description. Women Birth. 2018;31(5):e287–93.

Huang C, Han W, Fan Y. Correlation study between increased fetal movement during the third trimester and neonatal outcome. BMC Pregnancy Childbirth. 2019;19(1):1–6.

Sharp I, et al. Investigation of the outcome of pregnancies complicated by increased fetal movements and their relation to underlying causes–A prospective cohort study. Acta Obstet Gynecol Scand. 2021;100(1):91–100.

Cohen G, et al. Are increased fetal movements during pregnancy a predictor of neonatal adverse outcomes? Am J Obstet Gynecol. 2022;227(2):349–51.

Avraham S, et al. The association between increased subjective sensation of fetal movements and pregnancy outcome-a prospective cohort and a retrospective comparative analysis. J Matern Fetal Neonatal Med. 2023;36(1):2184224.

Stacey T, et al. The Auckland Stillbirth study, a case–control study exploring modifiable risk factors for third trimester stillbirth: methods and rationale. Aust N Z J Obstet Gynaecol. 2011;51(1):3–8.

Heazell AE, et al. Stillbirth is associated with perceived alterations in fetal activity–findings from an international case control study. BMC Pregnancy Childbirth. 2017;17(1):1–11.

Monari F, et al. Women’s perception of fetal movements and perinatal outcomes: results of a prospective cohort study. J Matern Fetal Neonatal Med. 2023;36(1):2193664.

Huang C, Han W, Fan Y. Correlation study between increased fetal movement during the third trimester and neonatal outcome. BMC Pregnancy Childbirth. 2019;19(1):467.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Akselsson A, et al. Increased labor induction and women presenting with decreased or altered fetal movements - a population-based survey. PLoS ONE. 2019;14(5):e0216216.

Sadovsky E, Polishuk WZ. Fetal movements in utero: nature, assessment, prognostic value, timing of delivery. Obstet Gynecol. 1977;50(1):49–55.

CAS   PubMed   Google Scholar  

Ferroni E. Osservazioni E Ricerche Sui movementi ritmici fetali intrauterini. Ann Ostet Ginecol. 1899;21:897.

Liley AW. The foetus as a personality. Australian New Z J Psychiatry. 1972;6(2):99–105.

Article   CAS   Google Scholar  

Kinsella MT, Monk C. Impact of maternal stress, depression & anxiety on fetal neurobehavioral development. Clin Obstet Gynecol. 2009;52(3):425.

Moshiri M, et al. Comprehensive imaging review of abnormalities of the umbilical cord. Radiographics. 2014;34(1):179–96.

Padula AM, et al. A review of maternal prenatal exposures to environmental chemicals and psychosocial stressors-implications for research on perinatal outcomes in the ECHO program. J Perinatol. 2020;40(1):10–24.

Sinesi A, et al. Anxiety scales used in pregnancy: systematic review. BJPsych Open. 2019;5(1):e5.

Cai Y, et al. The value of umbilical artery blood gas analysis in the diagnosis and prognosis evaluation of fetal distress. Am J Transl Res. 2022;14(7):4821–9.

CAS   PubMed   PubMed Central   Google Scholar  

Download references

No funding or sponsorship was received for the conduct of this study.

Author information

Authors and affiliations.

Student Research Committee, School of Medicine, Alborz University of Medical Sciences, Karaj, Iran

Sedigheh Hantoushzadeh, Nasim Eshraghi, Fatemeh Asadi, Seyed Jafar Razavi, Marjan Ghaemi, Sudabeh Kazemi Aski, Zahra Panhi & Gholam Reza Habibi

Gene Therapy Research Center, Digestive Diseases Research Institute, Tehran University of Medical Sciences, Tehran, Iran

Omid Kohandel Gargari

Department of Gynecology and Obstetrics, Mahdiyeh Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Marzieh Jamali & Fatemeh Farrokh

Vali-E-Asr Reproductive Health Research Center, Family Health Research Institute, Tehran University of Medical Sciences, Tehran, Iran

Masoumeh Mirzamoradi

You can also search for this author in PubMed   Google Scholar

Contributions

S.H.: Study’s design MG and N.E.: Conceptualization and supervision literature search. M.J. and F.F. and G.R.H: Selected the studies, extracted the relevant information, and then assessed and confirmed by the senior author O.K.G and S.K.A.: performed meta-analysisF.A. and M.M and S.J.R. and Z.P.: Edited the draft. All authors reviewed the manuscript.

Corresponding author

Correspondence to Marjan Ghaemi .

Ethics declarations

Ethics approval.

No ethical approval was required as this manuscript is a review article with no original research data.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hantoushzadeh, S., Gargari, O.K., Jamali, M. et al. The association between increased fetal movements in the third trimester and perinatal outcomes; a systematic review and meta-analysis. BMC Pregnancy Childbirth 24 , 365 (2024). https://doi.org/10.1186/s12884-024-06547-3

Download citation

Received : 27 September 2023

Accepted : 28 April 2024

Published : 15 May 2024

DOI : https://doi.org/10.1186/s12884-024-06547-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Fetal movement
• Fetal demise
• Systematic review

BMC Pregnancy and Childbirth

ISSN: 1471-2393

• Submission enquiries: [email protected]
• General enquiries: [email protected]