literature types in research

Literature Reviews: Types of Literature

• Library Basics
• 1. Choose Your Topic
• How to Find Books
• Types of Clinical Study Designs

Types of Literature

• 3. Search the Literature
• 4. Read & Analyze the Literature
• 5. Write the Review
• Keeping Track of Information
• Style Guides
• Books, Tutorials & Examples

Different types of publications have different characteristics.

Primary Literature Primary sources means original studies, based on direct observation, use of statistical records, interviews, or experimental methods, of actual practices or the actual impact of practices or policies. They are authored by researchers, contains original research data, and are usually published in a peer-reviewed journal. Primary literature may also include conference papers, pre-prints, or preliminary reports. Also called empirical research .

Secondary Literature Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature. Examples include review articles (such as meta-analysis and systematic reviews) and reference works. Professionals within each discipline take the primary literature and synthesize, generalize, and integrate new research.

Tertiary Literature Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.

Adapted from the Information Services Department of the Library of the Health Sciences-Chicago , University of Illinois at Chicago.

Types of Scientific Publications

These examples and descriptions of publication types will give you an idea of how to use various works and why you would want to write a particular kind of paper.

• Scholarly article aka empirical article
• Review article
• Conference paper

Scholarly (aka empirical) article -- example

Empirical studies use data derived from observation or experiment. Original research papers (also called primary research articles) that describe empirical studies and their results are published in academic journals.  Articles that report empirical research contain different sections which relate to the steps of the scientific method.

      Abstract - The abstract provides a very brief summary of the research.

     Introduction - The introduction sets the research in a context, which provides a review of related research and develops the hypotheses for the research.

     Method - The method section describes how the research was conducted.

     Results - The results section describes the outcomes of the study.

     Discussion - The discussion section contains the interpretations and implications of the study.

     References - A references section lists the articles, books, and other material cited in the report.

Review article -- example

A review article summarizes a particular field of study and places the recent research in context. It provides an overview and is an excellent introduction to a subject area. The references used in a review article are helpful as they lead to more in-depth research.

Many databases have limits or filters to search for review articles. You can also search by keywords like review article, survey, overview, summary, etc.

Conference proceedings, abstracts and reports -- example

Conference proceedings, abstracts and reports are not usually peer-reviewed.  A conference article is similar to a scholarly article insofar as it is academic. Conference articles are published much more quickly than scholarly articles. You can find conference papers in many of the same places as scholarly articles.

How Do You Identify Empirical Articles?

To identify an article based on empirical research, look for the following characteristics:

     The article is published in a peer-reviewed journal .

     The article includes charts, graphs, or statistical analysis .

     The article is substantial in size , likely to be more than 5 pages long.

     The article contains the following parts (the exact terms may vary): abstract, introduction, method, results, discussion, references .

• << Previous: Types of Clinical Study Designs
• Next: 3. Search the Literature >>
• Last Updated: Dec 29, 2023 11:41 AM
• URL: https://research.library.gsu.edu/litrev

Systematic Reviews: Types of literature review, methods, & resources

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

Analytical reviews

GUIDELINES FOR HOW TO CARRY OUT AN ANALYTICAL REVIEW OF QUANTITATIVE RESEARCH

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Randomised trials, Systematic reviews, Study protocols, Diagnostic/prognostic studies, Case reports, Clinical practice guidelines, Animal pre-clinical studies, etc). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

When comparing therapies :

PRISMA (Guideline on how to perform and write-up a systematic review and/or meta-analysis of the outcomes reported in multiple clinical trials of therapeutic interventions. PRISMA  replaces the previous QUORUM statement guidelines ):  Liberati, A,, Altman, D,, Moher, D, et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration.  Plos Medicine, 6 (7):e1000100. doi:10.1371/journal.pmed.1000100 

When comparing diagnostic methods :

Checklist for Artificial Intelligence in Medical Imaging (CLAIM). CLAIM is modeled after the STARD guideline and has been extended to address applications of AI in medical imaging that include classification, image reconstruction, text analysis, and workflow optimization. The elements described here should be viewed as a “best practice” to guide authors in presenting their research. Reported in Mongan, J., Moy, L., & Kahn, C. E., Jr (2020). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers.  Radiology. Artificial intelligence ,  2 (2), e200029. https://doi.org/10.1148/ryai.2020200029

STAndards for the Reporting of Diagnostic accuracy studies (STARD) Statement. (Reporting guidelines for writing up a study comparing the accuracy of competing diagnostic methods)  http://www.stard-statement.org/

When evaluating clinical practice guidelines :

AGREE Research Trust (ART) (2013).  Appraisal of Guidelines for Research & Evaluation (AGREE-II) . (A 23-item instrument for as sessing th e quality of Clinical Practice Guidelines. Used internationally for evaluating or deciding which guidelines could be recommended for use in practice or to inform health policy decisions.)

National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) Instrument (2019). (A 15-item instrument using scales of 1-5 to evaluate a guideline's adherence to the Institute of Medicine's standard for trustworthy guidelines. It has good external validity among guideline developers and good interrater reliability across trained reviewers.)

When reviewing genetics studies

Human genetics review reporting guidelines.  Little J, Higgins JPT (eds.). The HuGENet™ HuGE Review Handbook, version 1.0 . 

When you need to re-analyze individual participant data

If you wish to collect, check, and re-analyze individual participant data (IPD) from clinical trials addressing a particular research question, you should follow the  PRISMA-IPD  guidelines as reported in  Stewart, L.A., Clarke, M., Rovers, M., et al. (2015). Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement. JAMA, 313(16):1657-1665. doi:10.1001/jama.2015.3656 .

When comparing Randomized studies involving animals, livestock, or food:

O’Connor AM, et al. (2010).  The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement.  Zoonoses Public Health. 57(2):95-104. Epub 2010/01/15. doi: 10.1111/j.1863-2378.2009.01311.x. PubMed PMID: 20070653.

Sargeant JM, et al. (2010).  The REFLECT Statement: Reporting Guidelines for Randomized Controlled Trials in Livestock and Food Safety: Explanation and Elaboration.  Zoonoses Public Health. 57(2):105-36. Epub 2010/01/15. doi: JVB1312 [pii] 10.1111/j.1863-2378.2009.01312.x. PubMed PMID: 20070652.

GUIDELINES FOR HOW TO WRITE UP FOR PUBLICATION THE RESULTS OF ONE QUANTITATIVE CLINICAL TRIAL

When reporting the results of a Randomized Controlled Trial :

Consolidated Standards of Reporting Trials (CONSORT) Statement. (2010 reporting guideline for writing up a Randomized Controlled Clinical Trial).  http://www.consort-statement.org . Since updated in 2022, see Butcher, M. A., et al. (2022). Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension . JAMA : the Journal of the American Medical Association, 328(22), 2252–2264. https://doi.org/10.1001/jama.2022.21022

Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010). Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biology, 8(6), e1000412–e1000412. https://doi.org/10.1371/journal.pbio.1000412 (A 20-item checklist, following the CONSORT approach, listing the information that published articles reporting research using animals should include, such as the number and specific characteristics of animals used; details of housing and husbandry; and the experimental, statistical, and analytical methods used to reduce bias.)

Narrative reviews

GUIDELINES  FOR HOW TO CARRY OUT  A  NARRATIVE REVIEW / QUALITATIVE RESEARCH /  OBSERVATIONAL STUDIES

Campbell, M. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ, 368. doi: https://doi.org/10.1136/bmj.l6890  (guideline on how to analyse evidence for a narrative review, to provide a recommendation based on heterogenous study types).

Community Preventive Services Task Force (2021).  The Methods Manual for Community Guide Systematic Reviews . (Public Health Prevention systematic review guidelines)

Enhancing the QUAlity and Transparency Of health Research (EQUATOR) network. (Tracking and listing over 550 reporting guidelines for various different study types including Observational studies, Qualitative research, Quality improvement studies, and Economic evaluations). http://www.equator-network.org/resource-centre/library-of-health-research-reporting/

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from  https://methods.cochrane.org/qi/training-resources . (Training materials for how to do a meta-synthesis, or qualitative evidence synthesis). 

Cornell University Library (2019). Planning worksheet for structured literature reviews. Retrieved 4/8/22 from  https://osf.io/tnfm7/  (offers a framework for a narrative literature review).

Green, B. N., Johnson, C. D., & Adams, A. (2006).  Writing narrative literature reviews for peer-reviewed journals: secrets of the trade . Journal of Chiropractic Medicine, 5(3): 101-117. DOI: 10.1016/ S0899-3467 (07)60142-6.  This is a very good article about what to take into consideration when writing any type of narrative review.

When reviewing observational studies/qualitative research :

STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement. (Reporting guidelines for various types of health sciences observational studies).  http://www.strobe-statement.org 

Meta-analysis of Observational Studies in Epidemiology (MOOSE)  http://jama.jamanetwork.com/article.aspx?articleid=192614

RATS Qualitative research systematic review guidelines.  https://www.equator-network.org/reporting-guidelines/qualitative-research-review-guidelines-rats/

Methods/Guidance

Right Review , this decision support website provides an algorithm to help reviewers choose a review methodology from among 41 knowledge synthesis methods.

The Systematic Review Toolbox , an online catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Maintained by the UK University of York Health Economics Consortium, Newcastle University NIHR Innovation Observatory, and University of Sheffield School of Health and Related Research.

Institute of Medicine. (2011).  Finding What Works in Health Care: Standards for Systematic Reviews . Washington, DC: National Academies  (Systematic review guidelines from the Health and Medicine Division (HMD) of the U.S. National Academies of Sciences, Engineering, and Medicine (formerly called the Institute of Medicine)).

International Committee of Medical Journal Editors (2022).  Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals . Guidance on how to prepare a manuscript for submission to a Medical journal.

Cochrane Handbook of Systematic Reviews of Interventions (International Cochrane Collaboration systematic review guidelines). The various Cochrane review groups comporise around 30,000 physicians around the world working in the disciplines on reviews of interventions with very detailed methods for verifying the validity of the research methods and analysis performed in screened-in Randmized Controlled Clinical Trials. Typically published Cochrane Reviews are the most exhaustive review of the evidence of effectiveness of a particular drug or intervention, and include a statistical meta-analysis. Similar to practice guidelines, Cochrane reviews are periodically revised and updated.

Joanna Briggs Institute (JBI) Manual of Evidence Synthesis . (International systematic review guidelines). Based at the University of Adelaide, South Australia, and collaborating with around 80 academic and medical entities around the world. Unlike Cochrane Reviews that strictly focus on efficacy of interventions, JBI offers a broader, inclusive approach to evidence, to accommodate a range of diverse questions and study designs. The JBI manual provides guidance on how to analyse and include both quantitative and qualitative research.

Cochrane Methods Support Unit, webinar recordings on methodological support questions 

Cochrane Qualitative & Implementation Methods Group. (2019). Training resources. Retrieved from https://methods.cochrane.org/qi/training-resources . (How to do a meta-synthesis, or qualitative evidence synthesis). 

Center for Reviews and Dissemination (University of York, England) (2009).  Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care . (British systematic review guidelines). 

Agency for Health Research & Quality (AHRQ) (2013). Methods guide for effectiveness and comparative effectiveness reviews . (U.S. comparative effectiveness review guidelines)

Hunter, K. E., et al. (2022). Searching clinical trials registers: guide for systematic reviewers.  BMJ (Clinical research ed.) ,  377 , e068791. https://doi.org/10.1136/bmj-2021-068791

Patient-Centered Outcomes Research Institute (PCORI).  The PCORI Methodology Report . (A 47-item methodology checklist for U.S. patient-centered outcomes research. Established under the Patient Protection and Affordable Care Act, PCORI funds the development of guidance on the comparative effectivess of clinical healthcare, similar to the UK National Institute for Clinical Evidence but without reporting cost-effectiveness QALY metrics). 

Canadian Agency for Drugs and Technologies in Health (CADTH) (2019). Grey Matters: a practical tool for searching health-related grey literature. Retrieved from https://www.cadth.ca/resources/finding-evidence/grey-matters . A checklist of N American & international online databases and websites you can use to search for unpublished reports, posters, and policy briefs, on topics including general medicine and nursing, public and mental health, health technology assessment, drug and device regulatory, approvals, warnings, and advisories.

Hempel, S., Xenakis, L., & Danz, M. (2016). Systematic Reviews for Occupational Safety and Health Questions: Resources for Evidence Synthesis. Retrieved 8/15/16 from http://www.rand.org/pubs/research_reports/RR1463.html . NIOSH guidelines for how to carry out a systematic review in the occupational safety and health domain.

A good source for reporting guidelines is the  NLM's  Research Reporting Guidelines and Initiatives .

Grading of Recommendations Assessment, Development and Evaluation (GRADE). (An international group of academics/clinicians working to promote a common approach to grading the quality of evidence and strength of recommendations.) 

Phillips, B., Ball, C., Sackett, D., et al. (2009). Oxford Centre for Evidence Based Medicine: Levels of Evidence. Retrieved 3/20/17 from https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf . (Another commonly used criteria for grading the quality of evidence and strength of recommendations, developed in part by EBM guru David Sackett.) 

Systematic Reviews for Animals & Food  (guidelines including the REFLECT statement for carrying out a systematic review on animal health, animal welfare, food safety, livestock, and agriculture)

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies . Health Information & Libraries Journal, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x. (Describes 14 different types of literature and systematic review, useful for thinking at the outset about what sort of literature review you want to do.)

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements . Health information and libraries journal, 36(3), 202–222. doi:10.1111/hir.12276  (An updated look at different types of literature review, expands on the Grant & Booth 2009 article listed above).

Garrard, J. (2007).  Health Sciences Literature Review Made Easy: The Matrix Method  (2nd Ed.).   Sudbury, MA:  Jones & Bartlett Publishers. (Textbook of health sciences literature search methods).

Zilberberg, M. (2012).  Between the lines: Finding the truth in medical literature . Goshen, MA: Evimed Research Press. (Concise book on foundational concepts of evidence-based medicine).

Lang, T. (2009). The Value of Systematic Reviews as Research Activities in Medical Education . In: Lang, T. How to write, publish, & present in the health sciences : a guide for clinicians & laboratory researchers. Philadelphia : American College of Physicians.  (This book chapter has a helpful bibliography on systematic review and meta-analysis methods)

Brown, S., Martin, E., Garcia, T., Winter, M., García, A., Brown, A., Cuevas H.,  & Sumlin, L. (2013). Managing complex research datasets using electronic tools: a meta-analysis exemplar . Computers, Informatics, Nursing: CIN, 31(6), 257-265. doi:10.1097/NXN.0b013e318295e69c. (This article advocates for the programming of electronic fillable forms in Adobe Acrobat Pro to feed data into Excel or SPSS for analysis, and to use cloud based file sharing systems such as Blackboard, RefWorks, or EverNote to facilitate sharing knowledge about the decision-making process and keep data secure. Of particular note are the flowchart describing this process, and their example screening form used for the initial screening of abstracts).

Brown, S., Upchurch, S., & Acton, G. (2003). A framework for developing a coding scheme for meta-analysis . Western Journal Of Nursing Research, 25(2), 205-222. (This article describes the process of how to design a coded data extraction form and codebook, Table 1 is an example of a coded data extraction form that can then be used to program a fillable form in Adobe Acrobat or Microsoft Access).

Elamin, M. B., Flynn, D. N., Bassler, D., Briel, M., Alonso-Coello, P., Karanicolas, P., & ... Montori, V. M. (2009). Choice of data extraction tools for systematic reviews depends on resources and review complexity .  Journal Of Clinical Epidemiology ,  62 (5), 506-510. doi:10.1016/j.jclinepi.2008.10.016  (This article offers advice on how to decide what tools to use to extract data for analytical systematic reviews).

Riegelman R.   Studying a Study and Testing a Test: Reading Evidence-based Health Research , 6th Edition.  Lippincott Williams & Wilkins, 2012. (Textbook of quantitative statistical methods used in health sciences research).

Rathbone, J., Hoffmann, T., & Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 480. doi:10.1186/s13643-015-0067-6

Guyatt, G., Rennie, D., Meade, M., & Cook, D. (2015). Users' guides to the medical literature (3rd ed.). New York: McGraw-Hill Education Medical.  (This is a foundational textbook on evidence-based medicine and of particular use to the reviewer who wants to learn about the different types of published research article e.g. "what is a case report?" and to understand what types of study design best answer what types of clinical question).

Glanville, J., Duffy, S., Mccool, R., & Varley, D. (2014). Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? Journal of the Medical Library Association : JMLA, 102(3), 177–183. https://doi.org/10.3163/1536-5050.102.3.007

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan a web and mobile app for systematic reviews.  Systematic Reviews, 5 : 210, DOI: 10.1186/s13643-016-0384-4. http://rdcu.be/nzDM

Kwon Y, Lemieux M, McTavish J, Wathen N. (2015). Identifying and removing duplicate records from systematic review searches. J Med Libr Assoc. 103 (4): 184-8. doi: 10.3163/1536-5050.103.4.004. https://www.ncbi.nlm.nih.gov/pubmed/26512216

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. (2016). De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 104 (3):240-3. doi: 10.3163/1536-5050.104.3.014. Erratum in: J Med Libr Assoc. 2017 Jan;105(1):111. https://www.ncbi.nlm.nih.gov/pubmed/27366130

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–46. doi: 10.1016/j.jclinepi.2016.01.021 . PRESS is a guideline with a checklist for librarians to critically appraise the search strategy for a systematic review literature search.

Clark, JM, Sanders, S, Carter, M, Honeyman, D, Cleo, G, Auld, Y, Booth, D, Condron, P, Dalais, C, Bateup, S, Linthwaite, B, May, N, Munn, J, Ramsay, L, Rickett, K, Rutter, C, Smith, A, Sondergeld, P, Wallin, M, Jones, M & Beller, E 2020, 'Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial',  Journal of the Medical Library Association , vol. 108, no. 2, pp. 195-207.

Journal articles describing systematic review methods can be searched for in PubMed using this search string in the PubMed search box: sysrev_methods [sb] . 

Software tools for systematic reviews

• Covidence GW in 2019 has bought a subscription to this Cloud based tool for facilitating screening decisions, used by the Cochrane Collaboration. Register for an account.
• NVIVO for analysis of qualitative research NVIVO is used for coding interview data to identify common themes emerging from interviews with several participants. GW faculty, staff, and students may download NVIVO software.
• RedCAP RedCAP is software that can be used to create survey forms for research or data collection or data extraction. It has very detailed functionality to enable data exchange with Electronic Health Record Systems, and to integrate with study workflow such as scheduling follow up reminders for study participants.
• SRDR tool from AHRQ Free, web-based and has a training environment, tutorials, and example templates of systematic review data extraction forms
• RevMan 5 RevMan 5 is the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews
• Rayyan Free, web-based tool for collecting and screening citations. It has options to screen with multiple people, masking each other.
• GradePro Free, web application to create, manage and share summaries of research evidence (called Evidence Profiles and Summary of Findings Tables) for reviews or guidelines, uses the GRADE criteria to evaluate each paper under review.
• DistillerSR Needs subscription. Create coded data extraction forms from templates.
• EPPI Reviewer Needs subscription. Like DistillerSR, tool for text mining, data clustering, classification and term extraction
• SUMARI Needs subscription. Qualitative data analysis.
• Dedoose Needs subscription. Qualitative data analysis, similar to NVIVO in that it can be used to code interview transcripts, identify word co-occurence, cloud based.
• Meta-analysis software for statistical analysis of data for quantitative reviews SPSS, SAS, and STATA are popular analytical statistical software that include macros for carrying out meta-analysis. Himmelfarb has SPSS on some 3rd floor computers, and GW affiliates may download SAS to your own laptop from the Division of IT website. To perform mathematical analysis of big data sets there are statistical analysis software libraries in the R programming language available through GitHub and RStudio, but this requires advanced knowledge of the R and Python computer languages and data wrangling/cleaning.
• PRISMA 2020 flow diagram generator The PRISMA Statement website has a page listing example flow diagram templates and a link to software for creating PRISMA 2020 flow diagrams using R software.

GW researchers may want to consider using Refworks to manage citations, and GW Box to store the full text PDF's of review articles. You can also use online survey forms such as Qualtrics, RedCAP, or Survey Monkey, to design and create your own coded fillable forms, and export the data to Excel or one of the qualitative analytical software tools listed above.

Forest Plot Generators

• RevMan 5 the desktop version of the software used by Cochrane systematic review teams. RevMan 5 is free for academic use and can be downloaded and configured to run as stand alone software that does not connect with the Cochrane server if you follow the instructions at https://training.cochrane.org/online-learning/core-software-cochrane-reviews/revman/revman-5-download/non-cochrane-reviews.
• Meta-Essentials a free set of workbooks designed for Microsoft Excel that, based on your input, automatically produce meta-analyses including Forest Plots. Produced for Erasmus University Rotterdam joint research institute.
• Neyeloff, Fuchs & Moreira Another set of Excel worksheets and instructions to generate a Forest Plot. Published as Neyeloff, J.L., Fuchs, S.C. & Moreira, L.B. Meta-analyses and Forest plots using a microsoft excel spreadsheet: step-by-step guide focusing on descriptive data analysis. BMC Res Notes 5, 52 (2012). https://doi-org.proxygw.wrlc.org/10.1186/1756-0500-5-52
• For R programmers instructions are at https://cran.r-project.org/web/packages/forestplot/vignettes/forestplot.html and you can download the R code package from https://github.com/gforge/forestplot
• << Previous: Home
• Next: Protocol and registration >>

• Last Updated: May 8, 2024 11:07 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: May 2, 2024 10:39 AM
• URL: https://libguides.northwestern.edu/literaturereviews

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

Tutorial: Evaluating Information: Scholarly Literature Types

• Evaluating Information
• Scholarly Literature Types
• Primary vs. Secondary Articles
• Peer Review
• Systematic Reviews & Meta-Analysis
• Gray Literature
• Evaluating Like a Boss
• Evaluating AV

Types of scholarly literature

You will encounter many types of articles and it is important to distinguish between these different categories of scholarly literature. Keep in mind the following definitions.

Peer-reviewed (or refereed):  Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal. This can include empirical studies, review articles, meta-analyses among others.

Empirical study (or primary article): An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research. These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections: Introduction, Methods, Results, and Discussion.

Review article:  In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic. These are useful when you want to get an idea of a body of research that you are not yet familiar with. It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

Systematic review:  This is a methodical and thorough literature review focused on a particular research question. It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making. It may involve a meta-analysis (see below). 

Meta-analysis:  This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic. There are many methods, some complex, applied to performing this type of analysis.

Types of non-formally published scholarly literature

What is gray literature.

Gray (or grey) literature is literature produced by individuals or organizations outside of commercial and/or academic publishers. This type of non-formally published substantive information (often not formally peer-reviewed; especially important in all kinds of sciences) can include information such:

• theses and dissertations
• technical reports  
• working papers 
• government reports
• evaluation and think tank reports and resources
• conference proceedings, papers and posters
• publications from NGOs, INGOs, think tanks and policy institutes
• unpublished clinical trials
• and much more

The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations.

Why Search the Gray Literature?

Most of gray literature is considered less prestigious, reliable, and "official" than publication in a peer-reviewed journal. But they are still fully legitimate avenues of publication. Often they are used to publicize early findings, before a study is entirely complete. Or, in the case of theses, they are published as a condition of receiving an advanced degree. Government technical reports are issued either by agencies that do scientific research themselves or else by a lab that has received government funding. Increasingly, such labs may be required to publish technical reports as a condition of receiving such funding. Gray literature may be cited like any other paper although with the caveat mentioned before that it is considered less "official" and reliable than peer-reviewed scientific papers.

When doing evidence synthesiis, it's important because the intent is to synthesize  all available evidence  that is applicable to your research question. There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect. Meanwhile, many studies and trials that show no effect end up going unpublished. But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making. While not peer-reviewed, gray literature represents a valuable body of information that is critical to consider when synthesizing and evaluating all available evidence.

• << Previous: Evaluating Information
• Next: Primary vs. Secondary Articles >>
• Last Updated: Oct 20, 2021 11:11 AM
• URL: https://guides.library.cornell.edu/evaluate

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 9, 2024 11:05 AM
• URL: https://libguides.usc.edu/writingguide

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Apr 17, 2024 2:02 PM
• URL: https://guides.library.ucla.edu/systematicreviews

Types of Literature Review

There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:

Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review

Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.

When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.

Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.

Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.

Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.

Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.

Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.

At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods.  Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.

[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology

  John Dudovskiy

Research Methods: Literature Reviews

• Annotated Bibliographies
• Literature Reviews
• Scoping Reviews
• Systematic Reviews
• Scholarship of Teaching and Learning
• Persuasive Arguments
• Subject Specific Methodology

A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. A literature review helps the author learn about the history and nature of their topic, and identify research gaps and problems.

Steps & Elements

Problem formulation

• Determine your topic and its components by asking a question
• Research: locate literature related to your topic to identify the gap(s) that can be addressed
• Read: read the articles or other sources of information
• Analyze: assess the findings for relevancy
• Evaluating: determine how the article are relevant to your research and what are the key findings
• Synthesis: write about the key findings and how it is relevant to your research

Elements of a Literature Review

• Summarize subject, issue or theory under consideration, along with objectives of the review
• Divide works under review into categories (e.g. those in support of a particular position, those against, those offering alternative theories entirely)
• Explain how each work is similar to and how it varies from the others
• Conclude which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of an area of research

Writing a Literature Review Resources

• How to Write a Literature Review From the Wesleyan University Library
• Write a Literature Review From the University of California Santa Cruz Library. A Brief overview of a literature review, includes a list of stages for writing a lit review.
• Literature Reviews From the University of North Carolina Writing Center. Detailed information about writing a literature review.
• Undertaking a literature review: a step-by-step approach Cronin, P., Ryan, F., & Coughan, M. (2008). Undertaking a literature review: A step-by-step approach. British Journal of Nursing, 17(1), p.38-43

Literature Review Tutorial

• << Previous: Annotated Bibliographies
• Next: Scoping Reviews >>
• Last Updated: Feb 29, 2024 12:00 PM
• URL: https://guides.auraria.edu/researchmethods

1100 Lawrence Street Denver, CO 80204 303-315-7700 Ask Us Directions

Scholarly Literature Types

Types of scholarly literature, non-formally published substantive literature.

• Acknowledgements
• Peer Review This link opens in a new window
• Examples of Academic Sources
• Qualitative vs Quantitative Research
• Anatomy of a Scholarly Article

You will encounter many types of articles and it is important to distinguish between these different categories of scholarly literature. Keep in mind the following definitions.

Peer-reviewed (or refereed):  Refers to articles that have undergone a rigorous review process, often including revisions to the original manuscript, by peers in their discipline, before publication in a scholarly journal. This can include empirical studies, review articles, meta-analyses among others.

Empirical study (or primary article): An empirical study is one that aims to gain new knowledge on a topic through direct or indirect observation and research. These include quantitative or qualitative data and analysis. In science, an empirical article will often include the following sections: Introduction, Methods, Results, and Discussion.

Review article:  In the scientific literature, this is a type of article that provides a synthesis of existing research on a particular topic. These are useful when you want to get an idea of a body of research that you are not yet familiar with. It differs from a systematic review in that it does not aim to capture ALL of the research on a particular topic.

Systematic review:  This is a methodical and thorough literature review focused on a particular research question. It's aim is to identify and synthesize all of the scholarly research on a particular topic in an unbiased, reproducible way to provide evidence for practice and policy-making. It may involve a meta-analysis (see below). 

Meta-analysis:  This is a type of research study that combines or contrasts data from different independent studies in a new analysis in order to strengthen the understanding of a particular topic. There are many methods, some complex, applied to performing this type of analysis.

What is Grey Literature?

Grey literature is literature produced by individuals or organizations outside of commercial and/or academic publishers. This type of non-formally published substantive information (often not formally peer-reviewed; especially important in all kinds of sciences) can include information such:

• theses and dissertations
• technical reports 
• working papers 
• government reports
• evaluation and think tank reports and resources
• conference proceedings, papers and posters
• publications from NGOs, INGOs, think tanks and policy institutes
• unpublished clinical trials
• and much more

The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations.

Why Search the Gray Literature?

Most of gray literature is considered less prestigious, reliable, and "official" than publication in a peer-reviewed journal. But they are still fully legitimate avenues of publication. Often they are used to publicize early findings, before a study is entirely complete. Or, in the case of theses, they are published as a condition of receiving an advanced degree. Government technical reports are issued either by agencies that do scientific research themselves or else by a lab that has received government funding. Increasingly, such labs may be required to publish technical reports as a condition of receiving such funding. Gray literature may be cited like any other paper although with the caveat mentioned before that it is considered less "official" and reliable than peer-reviewed scientific papers.

When doing evidence synthesis, it's important because the intent is to synthesize  all available evidence  that is applicable to your research question. There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect. Meanwhile, many studies and trials that show no effect end up going unpublished. But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making. While not peer-reviewed, gray literature represents a valuable body of information that is critical to consider when synthesizing and evaluating all available evidence.

The guide is based on the Cornell University Library Tutorial: Scholarly Literature Types.

• Next: Peer Review >>
• Last Updated: Sep 13, 2023 5:23 PM
• URL: https://mcphs.libguides.com/ScholarlySources

• Privacy Policy

Home » What is Literature – Definition, Types, Examples

What is Literature – Definition, Types, Examples

Table of Contents

Definition:

Literature refers to written works of imaginative, artistic, or intellectual value, typically characterized by the use of language to convey ideas, emotions, and experiences. It encompasses various forms of written expression, such as novels, poems, plays, essays, short stories, and other literary works.

History of Literature

The history of literature spans thousands of years and includes works from many different cultures and languages. Here is a brief overview of some of the major periods and movements in the history of literature:

Ancient Literature (3000 BCE – 500 CE)

• Ancient Mesopotamian Literature (3000 BCE – 2000 BCE): This period includes the earliest known writings, such as the Epic of Gilgamesh, a Sumerian epic poem that explores themes of friendship, mortality, and the search for immortality.
• Ancient Greek Literature (800 BCE – 200 BCE): This era produced works by legendary writers such as Homer, known for the Iliad and the Odyssey, and playwrights like Sophocles, Aeschylus, and Euripides, who wrote tragic plays exploring human nature and the conflicts between gods and mortals.
• Ancient Roman Literature (200 BCE – 500 CE): Roman literature included works by poets like Virgil (known for the Aeneid) and historians like Livy and Tacitus, who chronicled the rise and fall of the Roman Empire.

Medieval Literature (500 CE – 1500 CE)

• Early Medieval Literature (500 CE – 1000 CE): During this period, literature was mainly religious and included works such as Beowulf, an Old English epic poem, and The Divine Comedy by Dante Alighieri, an Italian epic poem that describes the journey through Hell, Purgatory, and Heaven.
• High Medieval Literature (1000 CE – 1300 CE): This era saw the emergence of troubadour poetry in Provence, France, which celebrated courtly love, as well as the works of Geoffrey Chaucer, such as The Canterbury Tales, which combined diverse stories and social commentary.
• Late Medieval Literature (1300 CE – 1500 CE): Notable works from this period include Dante’s Divine Comedy, Petrarch’s sonnets, and the works of Christine de Pizan, an early feminist writer.

Renaissance Literature (14th – 17th centuries)

• Italian Renaissance Literature (14th – 16th centuries): This period witnessed the flourishing of humanism and produced works by authors such as Francesco Petrarch and Giovanni Boccaccio, who emphasized the individual, the secular, and the revival of classical themes and styles.
• English Renaissance Literature (16th – 17th centuries): This era saw the works of William Shakespeare, including his plays such as Hamlet and Macbeth, which explored complex human emotions and the human condition. Other notable writers include Christopher Marlowe and Edmund Spenser.

Enlightenment Literature (17th – 18th centuries)

• This period marked a shift towards reason, rationality, and the questioning of established beliefs and systems. Influential writers during this time included René Descartes, John Locke, Voltaire, Jean-Jacques Rousseau, and Denis Diderot.

Romanticism (late 18th – mid-19th centuries)

• Romantic literature emphasized individual emotion, imagination, and nature. Key figures include William Wordsworth, Samuel Taylor Coleridge, Lord Byron, Percy Bysshe Shelley, and John Keats.

Victorian Literature (19th century)

• This era was characterized by the reign of Queen Victoria and featured writers such as Charles Dickens, Jane Austen, Charlotte and Emily Brontë, and Oscar Wilde.

Modernist Literature (late 19th – early 20th centuries)

• Modernist literature emerged as a response to the social, political, and technological changes of the time. It is characterized by experimentation with narrative structure, language, and perspective. Notable modernist writers include T.S. Eliot, Virginia Woolf, James Joyce, and Marcel Proust.

Postmodern Literature (mid-20th century – present)

• Postmodern literature challenges traditional notions of narrative and reality. It often incorporates elements of metafiction, intertextuality, and fragmented narratives. Prominent postmodern authors include Jorge Luis Borges, Italo Calvino, Salman Rushdie, and Margaret Atwood.

Contemporary Literature (late 20th century – present)

• Contemporary literature encompasses a wide range of diverse voices and styles. It explores various themes and addresses contemporary issues, reflecting the cultural, social, and political contexts of the present time. Notable contemporary authors include Toni Morrison, J.K. Rowling, Haruki Murakami, Chimamanda Ngozi Adichie, and Zadie Smith.

Types of Literature

Types of Literature are as follows:

Short story

Graphic novel, electronic literature.

Poetry is a form of literature that uses language to convey emotions or ideas in a concise and often rhythmic manner. Poetry has been around for centuries, with many different cultures creating their own unique styles. While some people may view poetry as difficult to understand, there is often great beauty in its simplicity. Whether you are looking to read poems for enjoyment or to better analyze literary works, understanding the basics of poetry can be very helpful.

Examples of Poetry in Literature

There are countless examples of poetry in literature, ranging from ancient works to contemporary masterpieces. Here are just a few examples:

• “ The Love Song of J. Alfred Prufrock ” by T.S. Eliot (1915): This modernist poem explores themes of alienation, identity, and the human condition.
• “ Do not go gentle into that good night ” by Dylan Thomas (1951): This villanelle is a powerful meditation on death and the struggle for survival.
• “ The Waste Land” by T.S. Eliot (1922) : This epic poem is a complex and multi-layered exploration of the modern world and its spiritual emptiness.
• “ The Raven” by Edgar Allan Poe (1845) : This famous poem is a haunting and macabre exploration of grief, loss, and the supernatural.
• “ Sonnet 18″ by William Shakespeare (1609) : This classic sonnet is a beautiful and romantic tribute to the beauty of the beloved.
• “ Ode to a Nightingale” by John Keats (1819) : This ode is a sublime exploration of the power of beauty and the transcendent experience of art.
• “ The Road Not Taken” by Robert Frost (1916) : This famous poem is a contemplative meditation on choices, regrets, and the uncertainties of life.

These are just a few examples of the many works of poetry that exist in literature. Poetry can explore a wide range of themes and emotions, using language and imagery to create powerful and moving works of art.

Prose is a type of written language that typically contains dialogue and narration. In literature, prose is the most common form of writing. Prose can be found in novels, short stories, plays, and essays.

Examples of Prose in Literature

“ The Essays” by Michel de Montaigne (1580) – This collection of prose is a seminal work of the French Renaissance and is credited with popularizing the use of personal reflections in prose literature. Montaigne’s writing style in these works is informal and conversational, and covers a vast range of topics including morality, philosophy, religion, and politics. The prose is notable for its intimacy and personal nature, as Montaigne often uses his own experiences and thoughts to illustrate his ideas.

A novel is a fictional book that is typically longer than 300 pages. It tells a story, usually in chronological order, and has characters and settings that are developed over the course of the story. Novels are often divided into chapters, which help to break up the story and make it easier to read.

Novels are one of the most popular genres of literature, and there are many different types of novels that you can read. Whether you’re looking for a romance novel, a mystery novel, or a historical fiction novel, there’s sure to be a book out there that you’ll love.

Examples of Novels in Literature

• “Don Quixote” by Miguel de Cervantes (1605) – This novel is considered one of the greatest works of Spanish literature and is a satirical take on chivalric romance. It follows the adventures of a delusional knight, Don Quixote, and his loyal squire, Sancho Panza.
• “Robinson Crusoe” by Daniel Defoe (1719) – This novel is considered one of the earliest examples of the English novel and is a tale of survival and self-reliance. It follows the story of a man named Robinson Crusoe, who is stranded on a deserted island for 28 years.
• “Pride and Prejudice” by Jane Austen (1813) – This novel is considered one of the greatest works of English literature and is a romantic comedy of manners. It follows the story of Elizabeth Bennet and her complicated relationship with Mr. Darcy, a wealthy landowner.
• “To Kill a Mockingbird” by Harper Lee (1960) – This novel is a classic of American literature and deals with issues of race, class, and justice in the American South during the 1930s. It follows the story of a young girl named Scout and her experiences with racism and prejudice.
• “The Great Gatsby” by F. Scott Fitzgerald (1925) – This novel is considered a masterpiece of American literature and is a social commentary on the decadence and excess of the Roaring Twenties. It follows the story of Jay Gatsby, a wealthy and mysterious man, and his obsession with a woman named Daisy Buchanan.

A novella is a work of fiction that is shorter than a novel but longer than a short story. The word “novella” comes from the Italian word for “new”, which is fitting because this type of story is often seen as being between the old and the new. In terms of length, a novella typically has about 20,000 to 40,000 words.

While novels are usually about one main plot with several subplots, novellas are usually focused on one central conflict. This conflict is usually resolved by the end of the story. However, because novellas are longer than short stories, there is more room to develop characters and explore themes in depth.

Examples of Novella in Literature

• “Heart of Darkness” by Joseph Conrad (1899) – This novella is a powerful and haunting portrayal of European imperialism in Africa. It follows the journey of a steamboat captain named Marlow, who is sent to find a man named Kurtz deep in the Congo.
• “The Old Man and the Sea” by Ernest Hemingway (1952) – This novella is a Pulitzer Prize-winning story of an aging Cuban fisherman named Santiago and his epic struggle to catch a giant marlin. It is a testament to the resilience and determination of the human spirit.
• “The Metamorphosis” by Franz Kafka (1915) – This novella is a surreal and disturbing tale of a man named Gregor Samsa, who wakes up one morning to find himself transformed into a giant insect. It explores themes of isolation, identity, and the human condition.
• “Of Mice and Men” by John Steinbeck (1937) – This novella is a tragic story of two migrant workers, George and Lennie, who dream of owning their own farm but are thwarted by their own limitations and the harsh realities of the Great Depression. It is a powerful commentary on the American Dream and the plight of the working class.
• “Animal Farm” by George Orwell (1945) – This novella is a satirical allegory of the Russian Revolution and the rise of Stalinism. It follows the story of a group of farm animals who overthrow their human owner and create their own society, only to be corrupted by their own leaders. It is a cautionary tale about the dangers of totalitarianism and propaganda.

A short story is a work of fiction that typically can be read in one sitting and focuses on a self-contained incident or series of linked incidents.

The short story is one of the oldest forms of literature and has been found in oral cultures as well as in written form. In terms of length, it is much shorter than the novel, typically ranging from 1,000 to 20,000 words.

The short story has often been described as a “perfect form” because it allows for greater compression and variety than either the novel or poem. It also allows writers to experiment with different styles and genres.

Examples of Short Story in Literature

• “The Tell-Tale Heart” by Edgar Allan Poe (1843) – This classic horror story is a chilling portrayal of a murderer who is haunted by the sound of his victim’s heartbeat. It is a masterful example of Poe’s psychological and suspenseful writing style.
• “The Lottery” by Shirley Jackson (1948) – This controversial short story is a commentary on the dark side of human nature and the dangers of blind adherence to tradition. It follows the annual tradition of a small town that holds a lottery, with a surprising and shocking ending.
• “The Gift of the Magi” by O. Henry (1905) – This heartwarming story is a classic example of a holiday tale of selflessness and sacrifice. It follows the story of a young couple who each give up their most prized possession to buy a gift for the other.
• “A Clean, Well-Lighted Place” by Ernest Hemingway (1933) – This minimalist story is a reflection on the existential angst and loneliness of modern life. It takes place in a cafe late at night and explores the relationships between the patrons and the waiter.
• “The Yellow Wallpaper” by Charlotte Perkins Gilman (1892) – This feminist short story is a powerful critique of the medical establishment and the treatment of women’s mental health. It follows the story of a woman who is confined to her bedroom and becomes obsessed with the yellow wallpaper on the walls.

A graphic novel is a book that tells a story through the use of illustrations and text. Graphic novels can be based on true stories, or they can be fictional. They are usually longer than traditional books, and they often have more complex plots.

Graphic novels first gained popularity in the 1970s, when publishers began releasing collections of comics that had been previously published in magazines. Since then, the genre has grown to include original works, as well as adaptations ofexisting stories.

Graphic novels are now widely respected as a form of literature, and they have been adapted into many different mediums, including movies, television shows, and stage plays.

Examples of Graphic Novels in Literature

• “ Watchmen” by Alan Moore and Dave Gibbons (1986-1987) – This graphic novel is considered one of the greatest works of the medium and is a deconstruction of the superhero genre. It follows a group of retired superheroes who come out of retirement to investigate the murder of one of their own.
• “ Maus” by Art Spiegelman (1980-1991) – This Pulitzer Prize-winning graphic novel is a harrowing and poignant account of a Jewish survivor of the Holocaust and his strained relationship with his son. The characters are depicted as animals, with the Jews as mice and the Nazis as cats.
• “ Persepolis” by Marjane Satrapi (2000-2003) – This autobiographical graphic novel is a coming-of-age story set against the backdrop of the Iranian Revolution. It follows the author’s experiences growing up in Iran and then moving to Europe as a teenager.
• “Sandman” by Neil Gaiman (1989-1996) – This epic fantasy series is a masterful exploration of mythology, literature, and human nature. It follows the story of Morpheus, the Lord of Dreams, as he navigates through the world of dreams and interacts with characters from across time and space.
• “Batman: The Dark Knight Returns” by Frank Miller (1986) – This influential graphic novel is a gritty and realistic portrayal of an aging Batman who comes out of retirement to fight crime in a dystopian future. It is credited with revolutionizing the Batman character and inspiring a new era of darker and more mature superhero stories.

Electronic literature, also known as e-literature, is a genre of writing that uses electronic media to create works of art. This type of literature often includes elements of interactivity, hypertextuality, and multimedia.

E-literature has its roots in early computer games and interactive fiction. These early works were created using simple text-based programming languages like BASIC and HTML. Today, e-literature has evolved into a complex form of art that incorporates multimedia elements such as audio and video.

Examples of Electronic Literature in Literature

• “ Afternoon: A Story” by Michael Joyce (1987) – This hypertext fiction is considered one of the earliest examples of electronic literature. It is a nonlinear narrative that can be read in multiple paths and contains multimedia elements like images and sound.
• “ Patchwork Girl” by Shelley Jackson (1995) – This hypertext novel is a retelling of Mary Shelley’s “Frankenstein” that uses digital media to explore the themes of identity, gender, and creation. It contains animated graphics, video, and sound.
• “ The Dreamlife of Letters” by Brian Kim Stefans (2000) – This work of interactive poetry uses computer algorithms to generate new poems based on the user’s input. It combines traditional poetic forms with digital technologies to create a unique reading experience.
• “ Flight Paths” by Kate Pullinger and Chris Joseph (2007) – This work of electronic literature is a collaborative multimedia project that explores the lives of immigrants and refugees. It combines text, video, and audio to create an immersive and interactive experience.
• “Inanimate Alice” by Kate Pullinger and Chris Joseph (2005-2016) – This interactive digital novel follows the story of a young girl named Alice as she grows up in a world of technology and media. It uses a combination of text, video, animation, and sound to create a unique and engaging narrative.

Non-fiction

Non-fiction in literature is defined as prose writings that are based on real events, people, or places. Non-fiction is often divided into categories such as biography, history, and essay.

Examples of Non-fiction in Literature

• “ The Origin of Species” by Charles Darwin (1859) – This landmark book is one of the most influential works in the history of science. It lays out Darwin’s theory of evolution by natural selection and provides evidence for the descent of all living things from a common ancestor.
• “The Autobiography of Malcolm X” by Malcolm X and Alex Haley (1965) – This autobiography is a candid and powerful account of Malcolm X’s life as an African American civil rights leader. It explores his journey from a troubled youth to a powerful orator and activist, and provides insights into the social and political climate of the time.
• “ The Feminine Mystique” by Betty Friedan (1963) – This groundbreaking book is a seminal work of feminist literature. It critiques the idea of the “happy housewife” and argues that women’s social roles and expectations are limiting and oppressive.
• “The New Jim Crow” by Michelle Alexander (2010) – This book is a powerful critique of the criminal justice system and its impact on communities of color. It argues that the system perpetuates racial inequality and provides a call to action for reform.

Drama is a genre of literature that tells a story through the use of dialogue and action. It often has a strong plot and characters who undergo change or development over the course of the story. Drama can be divided into several subgenres, such as tragedy, comedy, and farce.

Examples of Drama in Literature

• “ Hamlet” by William Shakespeare (1603) – This tragedy is considered one of the greatest plays ever written. It tells the story of Prince Hamlet of Denmark and his quest for revenge against his uncle, who murdered his father and married his mother.
• “ A Doll’s House” by Henrik Ibsen (1879) – This play is a landmark work of modern drama. It explores themes of gender roles, marriage, and personal identity through the story of a married woman who decides to leave her husband and children in order to discover herself.
• “ Death of a Salesman” by Arthur Miller (1949) – This play is a powerful critique of the American Dream and the pressures of modern society. It tells the story of a salesman named Willy Loman and his family, as they struggle to come to terms with the realities of their lives.
• “ Fences” by August Wilson (1985) – This play is part of Wilson’s “Pittsburgh Cycle,” a series of ten plays that explore the African American experience in the 20th century. It tells the story of a former Negro League baseball player named Troy Maxson and his relationship with his family.

Also see Literature Review

Examples of Literature

Examples of Literature are as follows:

• “The Silent Patient” by Alex Michaelides
• “Normal People” by Sally Rooney
• “Where the Crawdads Sing” by Delia Owens
• “The Water Dancer” by Ta-Nehisi Coates
• “Harry Potter and the Cursed Child” by J.K. Rowling, Jack Thorne, and John Tiffany
• “The Ferryman” by Jez Butterworth
• “The Inheritance” by Matthew Lopez
• “Sweat” by Lynn Nottage
• “The Hill We Climb” by Amanda Gorman (inaugural poem at the 2021 U.S. presidential inauguration)
• “The Tradition” by Jericho Brown
• “Homie” by Danez Smith
• “The Carrying” by Ada Limón
• “Call Me by Your Name” (2017) directed by Luca Guadagnino (based on the novel by André Aciman)
• “The Great Gatsby” (2013) directed by Baz Luhrmann (based on the novel by F. Scott Fitzgerald)
• “The Lord of the Rings” trilogy (2001-2003) directed by Peter Jackson (based on the novels by J.R.R. Tolkien)
• “The Handmaiden” (2016) directed by Park Chan-wook (based on the novel “Fingersmith” by Sarah Waters)
• “Lemonade” (2016) by Beyoncé (visual album with accompanying poetry and prose)
• “To Pimp a Butterfly” (2015) by Kendrick Lamar (rap album with dense lyrical storytelling)
• “I See You” (2017) by The xx (album inspired by themes of love and connection)
• “Carrie & Lowell” (2015) by Sufjan Stevens (folk album exploring personal and familial themes)
• Blogs and online articles that discuss literary analysis, book reviews, and creative writing
• Online literary magazines and journals publishing contemporary works of fiction, poetry, and essays
• E-books and audiobooks available on platforms like Kindle, Audible, and Scribd
• Social media platforms where writers share their works and engage with readers, such as Twitter and Instagram

Purpose of Literature

The purpose of literature is multifaceted and can vary depending on the author, genre, and intended audience. However, some common purposes of literature include:

Entertainment

Literature can provide enjoyment and pleasure to readers through engaging stories, complex characters, and beautiful language.

Literature can teach readers about different cultures, time periods, and perspectives, expanding their knowledge and understanding of the world.

Reflection and introspection

Literature can encourage readers to reflect on their own experiences and beliefs, prompting self-discovery and personal growth.

Social commentary

Literature can serve as a medium for social criticism, addressing issues such as inequality, injustice, and oppression.

Historical and cultural preservation

Literature can document and preserve the history, traditions, and values of different cultures and societies, providing insight into the past.

Aesthetic appreciation:

literature can be appreciated for its beauty and artistic value, inspiring readers with its language, imagery, and symbolism.

The Significance of Literature

Literature holds immense significance in various aspects of human life and society. It serves as a powerful tool for communication, expression, and exploration of ideas. Here are some of the key significances of literature:

Communication and Expression

Literature allows individuals to communicate their thoughts, emotions, and experiences across time and space. Through various literary forms such as novels, poems, plays, and essays, writers can convey their ideas and perspectives to readers, fostering understanding and empathy.

Cultural Reflection

Literature often reflects the values, beliefs, and experiences of a particular culture or society. It provides insights into different historical periods, social structures, and cultural practices, offering a glimpse into the diversity and richness of human experiences.

Knowledge and Education

Literature is a valuable source of knowledge, as it presents ideas, concepts, and information in an engaging and accessible manner. It introduces readers to different subjects, such as history, science, philosophy, psychology, and more, allowing them to expand their understanding and broaden their intellectual horizons.

Emotional and Intellectual Development

Literature has the power to evoke emotions and provoke critical thinking. By immersing oneself in literary works, readers can develop a deeper understanding of complex emotions, empathy for diverse perspectives, and the ability to think critically and analytically.

Preservation of Cultural Heritage

Literature acts as a repository of a society’s cultural heritage. It preserves the history, traditions, myths, and folklore of a particular community, ensuring that future generations can connect with their roots and learn from the experiences of the past.

Social Commentary and Critique

Literature often serves as a platform for social commentary and critique. Writers use their works to shed light on social issues, challenge societal norms, and promote positive change. By addressing controversial topics and presenting alternative viewpoints, literature can spark discussions and inspire activism.

Entertainment and Escapism

Literature offers a means of entertainment and escapism from the realities of everyday life. Engaging narratives, compelling characters, and vivid descriptions transport readers to different worlds, allowing them to experience joy, excitement, and adventure through the pages of a book.

Imagination and Creativity

Literature fuels the human imagination and nurtures creativity. It encourages readers to think beyond the boundaries of their own experiences, envision new possibilities, and explore alternative realities. Literature inspires writers to craft unique stories and ideas, contributing to the expansion of artistic expression.

Personal Growth and Self-Reflection

Reading literature can have a profound impact on personal growth and self-reflection. It provides opportunities for introspection, introspection, and self-discovery, as readers identify with characters, grapple with moral dilemmas, and contemplate the deeper meaning of life and existence.

The Enduring Impact of Literature

Literature has an enduring impact that transcends time and continues to influence individuals and societies long after it is written. Here are some ways in which literature leaves a lasting impression:

Cultural Legacy:

Literary works become part of a society’s cultural legacy. They shape and reflect the values, beliefs, and traditions of a particular era or community. Classic works of literature, such as Shakespeare’s plays or the novels of Jane Austen, continue to be studied, performed, and celebrated, preserving their impact across generations.

Influence on Other Art Forms:

Literature has a profound influence on other art forms, such as film, theater, music, and visual arts. Many famous literary works have been adapted into films or stage productions, reaching new audiences and extending their influence beyond the written word. Artists and musicians often draw inspiration from literary themes, characters, and narratives, further amplifying their impact.

Shaping Worldviews:

Literature has the power to shape and challenge worldviews. Through stories, ideas, and perspectives presented in literary works, readers are exposed to different cultures, experiences, and ideologies. This exposure fosters empathy, broadens perspectives, and encourages critical thinking, ultimately influencing how individuals perceive and understand the world around them.

Inspirational Source:

Literature serves as an inspirational source for individuals in various fields. Writers, artists, scientists, and thinkers often draw inspiration from the works of literary giants who have explored the depths of human emotions, grappled with existential questions, or challenged societal norms. Literature provides a wellspring of ideas and creativity that continues to fuel innovation and intellectual discourse.

Social and Political Change:

Literature has played a significant role in driving social and political change throughout history. Many literary works have addressed pressing social issues, advocated for human rights, and challenged oppressive systems. By shedding light on societal injustices and encouraging readers to question the status quo, literature has been instrumental in inspiring activism and fostering social progress.

Universal Themes and Human Experience:

Literature explores universal themes and the complexities of the human experience. Whether it’s love, loss, identity, or the pursuit of meaning, these themes resonate with readers across time and cultures. Literary works offer insights into the depths of human emotions, dilemmas, and aspirations, creating a shared understanding and connecting individuals across generations.

Intellectual and Personal Development:

Reading literature stimulates intellectual growth and personal development. It encourages critical thinking, analytical skills, and the ability to empathize with diverse perspectives. Literary works challenge readers to reflect on their own lives, values, and beliefs, promoting self-discovery and personal growth.

Enduring Literary Characters:

Iconic literary characters have a lasting impact on popular culture and the collective imagination. Characters like Sherlock Holmes, Hamlet, or Elizabeth Bennet have become archetypes, influencing the portrayal of similar characters in other works and becoming a part of our cultural lexicon.

Preservation of History and Memory:

Literature plays a crucial role in preserving historical events, experiences, and cultural memories. Historical novels, memoirs, and eyewitness accounts provide valuable insights into past eras, allowing future generations to learn from and connect with the past.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

What is Art – Definition, Types, Examples

What is Anthropology – Definition and Overview

Economist – Definition, Types, Work Area

Anthropologist – Definition, Types, Work Area

What is History – Definitions, Periods, Methods

Philosopher – Definition, Types and Work Area

• Open access
• Published: 01 May 2024

Hospital performance evaluation indicators: a scoping review

• Shirin Alsadat Hadian   ORCID: orcid.org/0000-0002-1443-1990 1 ,
• Reza Rezayatmand   ORCID: orcid.org/0000-0002-9907-3597 2 ,
• Nasrin Shaarbafchizadeh   ORCID: orcid.org/0000-0001-7104-2214 3 ,
• Saeedeh Ketabi   ORCID: orcid.org/0000-0002-6778-5645 4 &
• Ahmad Reza Pourghaderi   ORCID: orcid.org/0000-0003-2682-2160 5  

BMC Health Services Research volume  24 , Article number:  561 ( 2024 ) Cite this article

319 Accesses

Metrics details

Hospitals are the biggest consumers of health system budgets and hence measuring hospital performance by quantitative or qualitative accessible and reliable indicators is crucial. This review aimed to categorize and present a set of indicators for evaluating overall hospital performance.

We conducted a literature search across three databases, i.e., PubMed, Scopus, and Web of Science, using possible keyword combinations. We included studies that explored hospital performance evaluation indicators from different dimensions.

We included 91 English language studies published in the past 10 years. In total, 1161 indicators were extracted from the included studies. We classified the extracted indicators into 3 categories, 14 subcategories, 21 performance dimensions, and 110 main indicators. Finally, we presented a comprehensive set of indicators with regard to different performance dimensions and classified them based on what they indicate in the production process, i.e., input, process, output, outcome and impact.

The findings provide a comprehensive set of indicators at different levels that can be used for hospital performance evaluation. Future studies can be conducted to validate and apply these indicators in different contexts. It seems that, depending on the specific conditions of each country, an appropriate set of indicators can be selected from this comprehensive list of indicators for use in the performance evaluation of hospitals in different settings.

Peer Review reports

Healthcare is complex [ 1 ] and a key sector [ 2 ] that is now globally faced with problems of rising costs, lack of service efficiency, competition, and equity as well as responsiveness to users [ 3 ]. One estimate by the WHO has shown a yearly waste of approximately 20–40% of total healthcare resources because of inefficiency [ 4 ]. European countries have spent on average 9.6% of their gross domestic product (GDP) on healthcare in 2017 and 9.92% in 2019. Germany, France, and Sweden reported the highest healthcare expenditures in Europe in 2018 (between 10.9% and 11.5% of GDP) [ 5 ]. In the U.S., healthcare spending consumes 18% of the GDP, which is likely to eclipse $6 trillion by 2027 [ 6 ].

Hospitals, as the biggest consumers of health system budgets [ 7 ], are the major part of the health system [ 8 ]. In many countries 50–80% of the health sector budget is dedicated to hospitals [ 8 , 9 ]. As a result, hospital performance analysis is becoming a routine task for every hospital manager. On the one hand, hospital managers worldwide are faced with difficult decisions regarding cost reduction, increasing service efficiency, and equity [ 10 ]. On the other hand, measuring hospital efficiency is an issue of interest among researchers because patients demand high-quality care at lower expenses [ 11 ].

To address the above mentioned need to measure hospital performance, implementing an appropriate hospital performance evaluation system is crucial in any hospital. In doing so, hospital administrators use various tools to analyse and monitor hospital activities [ 1 ], which need well-defined objectives, standards and quantitative indicators [ 12 ]. The latter are used to evaluate care provided to patients both quantitatively and qualitatively and are often related to input, output, processes, and outcomes. These indicators can be used for continuous quality improvement by monitoring, benchmarking, and prioritizing activities [ 13 ]. These parameters are developed to improve health outcomes and to provide comparative information for monitoring and managing and formulating policy objectives within and across health services [ 12 ]. Studies thus far have used their own set of indicators while evaluating hospital performance, which could be context dependent. In addition, those studies have mostly used a limited set of indicators that focus on few dimensions (2–6 dimensions) of hospital performance [ 14 , 15 , 16 , 17 , 18 ].

Therefore, comprehensive knowledge of potential indicators that can be used for hospital performance evaluation is necessary. It would help choose appropriate indicators when evaluating hospital performance in different contexts. It would also help researchers extend the range of analysis to evaluate performance from a wider perspective by considering more dimensions of performance. Although performance is a very commonly used term, it has several definitions [ 19 , 20 ], yet, it is often misunderstood [ 21 ]. Therefore, some researchers have expressed confusion about the related terms and considered them interchangeable. These terms are effectiveness, efficiency, productivity, quality, flexibility, creativity, sustainability, evaluation, and piloting [ 21 , 22 , 23 ]. Thus, this scoping review aimed to categorize and present a comprehensive set of indicators that can be used as a suitable set for hospital performance evaluation at any needed level of analysis, i.e., clinical, para-clinical, logistical, or departmental, and relate those indicators to the appropriate performance dimensions. The uniqueness of this paper is that it provides its readers with a comprehensive collection of indicators that have been used in different performance analysis studies.

Materials and methods

We conducted a scoping review of a body of literature. The scoping review can be of particular use when the topic has not yet been extensively reviewed or has a complex or heterogeneous nature. This type of review is commonly undertaken to examine the extent, range, and nature of research activity in a topic area; determine the value and potential scope and cost of undertaking a full systematic review; summarize and disseminate research findings; and identify research gaps in the existing literature. As a scoping review provides a rigorous and transparent method for mapping areas of research, it can be used as a standalone project or as a preliminary step to a systematic review [ 24 ]. While a systematic review (qualitative or quantitative) usually addresses a narrow topic/scope and is a method for integrating or comparing findings from previous studies [ 25 ].

In our study, we used the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist following the methods outlined by Arksey and O’Malley [ 26 ] and Tricco [ 27 ]. A systematic search for published and English-language literature on hospital performance evaluation models was conducted, using three databases, i.e., PubMed, Scopus, and Web of Science, from 2013 to January 2023. Initially, the identified keywords were refined and validated by a team of experts. Then, a combination of vocabularies was identified by the authors through a brainstorming process. The search strategy was formulated using Boolean operators. The title and abstract of the formulas were searched in the online databases. The search query for each database is presented in Table  1 .

In the screening process, relevant references related to hospital performance evaluation were screened and abstracted into researcher-developed Microsoft® Excel forms by dual independent reviewers and conflicting information was provided by other reviewers.

The inclusion criteria were as follows: focused only on the hospital setting, available full text and written in English. We excluded studies that focused on health organization indicators, not specifically on hospital indicators; articles without appropriate data (only focused on models and not indicators; or qualitative checklist questionnaires); and articles that focused only on clinical or disease-related indicators, not hospital performance dimensions, and provided very general items as indicators, not the domains of the indicators themselves. Then, a PRISMA-ScR Checklist was used to improve transparency in our review [ 28 ].

To extract the data, researcher-developed Microsoft® Excel forms (data tables) were designed. The following data were subsequently extracted into Microsoft®Excel for synthesis and evaluation: title, author, article year, country, indicator category, study environment (number of hospitals studied), study time frame, indicator name, number of indicators, indicator level (hospital level, department level), evaluation perspective (performance, productivity, efficiency, effectiveness, quality, cost, safety, satisfaction, etc. ) , study type (quantitative or qualitative), indicator subtype (input (structure), process, output (result), outcome and impact), and other explanations. To create a descriptive summary of the results that address the objectives of this scoping review, numerical summarization was also used.

The purpose of creating the main category and the evaluation perspective section was to develop them and create new categories, which focused on the type of indicators related to the performance term. For example, in the “Category” section, the names of the departments or wards of the hospital (such as hospital laboratories, pharmacies, clinical departments, and warehouses) and in the “Evaluation perspective” section, various terms related to the evaluation of hospital performance were extracted. These two types were used after extracting their information under the title “performance dimension”.

The indicators’ levels were collected to determine the level of performance evaluation with the relevant index. Some indicators were used to evaluate the performance of the entire hospital, some were used to evaluate the performance of hospital departments, and some were used to evaluate the performance at the level of a specific project. For example, several indicators (such as bed occupancy ratio, length of stay, and waiting time) were used to evaluate the performance of the entire hospital, and other indicators (such as laboratory department indicators, energy consumption indicators, and neonatal department indicators) were used only to measure the performance of specific departments. This sections were used under the title “category”. The “category” and “indicator’s name” sections were defined according to the results of the “subcategory” section.

The subtypes of indicators (input (structure), process, output(result), outcome and impact) were defined based on the chain model, and each of the selected indicators was linked to it (Appendix 1 ). As a result of the chain model, inputs were used to carry out activities, activities led to the delivery of services or products (outputs). The outputs started to bring about change (outcomes), and eventually, this (hopefully) contributed to the impact [ 29 ]. The classification of the set of input, process, output, outcome and impact indicators was such that readers could access these categories if necessary according to their chosen evaluation models. The term was used under the title “Indicators by types”.

The type of study was considered quantitative or qualitative for determining whether an indicator was able to perform calculations. In this way, readers can choose articles that use quantitative or qualitative indicators to evaluate hospital performance.

We included 91 full-text studies (out of 7475) in English published between 2013 and January 2023 (Fig.  1 ), approximately 40% of which were published between 2020 and 2023. More than 20% of the retrieved studies were conducted in Iran and USA.

Study selection and data abstraction

Study characteristic

As shown in Table  2 , in 85% of the reviewed studies, a number of hospitals (1 to 3828 hospitals, 13,221 hospitals in total) were evaluated. More than 90% of the studies used a quantitative approach. In more than 70% of the studies, hospital evaluation occurred at the department level, which can also be divided into three levels: administrative, clinical ward, and paramedical department. In addition, the administrative departments consist of 13 departments, including financial management [ 48 , 55 , 61 , 67 , 68 , 80 , 83 , 109 , 113 ], supply chain management and warehouse [ 15 , 43 , 84 ], value-based purchasing [ 33 , 85 ], human resource management [ 97 , 101 ], medical equipment [ 32 , 87 ], health information management department [ 90 ], information systems [ 106 ], nutritional assessment [ 93 ], energy management [ 30 , 45 , 92 ], facility management [ 52 , 53 ], building sustainability and resilience [ 35 ], research activities [ 44 ], and education [ 107 ].

The clinical wards consisted of 8 wards, namely, emergency departments (EDs) [ 16 , 39 , 56 , 57 , 69 , 70 , 89 ], surgery departments [ 58 , 62 , 63 , 91 , 102 ], intensive care units (ICUs) [ 47 , 64 , 65 ], operating rooms (ORs) [ 38 , 88 , 108 ], surgical intensive care units (SICUs) [ 111 ], obstetrics and gynecology department [ 59 ], neonatal intensive care units (NICUs) [ 74 , 103 ] and quality of care [ 18 , 31 , 40 , 50 , 72 , 92 , 95 , 112 ] indicators. The paramedical departments consisted of 3 departments, pharmacy [ 60 , 76 , 98 ], laboratory and blood bank [ 37 , 42 , 43 , 49 ], and outpatient assessment [ 86 ] indicators.

With regard to data categorization, firstly, a total of 1204 indicators in 91 studies were extracted and after detailed examination, 43 indices (such as hospital ownership, level of care, admission process, and personal discipline) were removed due to their generality and impossibility of calculation in the hospital environment. Then, 1161 performance indicators were entered in this research and were categorized based on the performance criteria (more details about the indicators can be found in Appendix 1 ). Secondly, 145 functional dimensions, including divisions based on different departments and units of the hospital, were defined according to several focus group discussions with 5 health experts. Then, re-categorization and functional summarization were performed, after which 21 performance dimensions were finalized.

As shown in Table  4 , the 21 performance dimensions were divided into three parts: category, subcategory, and related indicators. Additionally, according to the hospital levels, there were three categories: ‘organizational management’, ‘clinical management’, and ‘administrative management’. Then, according to the type of indicators, fifteen subcategories were defined for the 110 selected main indicators.

Performance dimensions

The ‘productivity’ dimension focuses on indicators reflecting the macro-performance of the hospital, considering that this index is more effective and efficient. The ‘efficiency’ dimension focuses on general performance indicators for the optimal use of resources to create optimal output in the hospital. The ‘effectiveness’ dimension is a general performance indicator with an outcome view. The ‘speed’ dimension focuses on the indicators that show attention to the service delivery time and the speed of the procedures. The ‘development’ dimension focuses on matters related to employees’ and students’ training and related training courses. In terms of ‘safety’ dimension, there were issues related to patient safety, unwanted and harmful events, and hospital infections.

The “quality of work life” dimension emphasizes matters related to personnel volume and work conditions. The ‘quality’ dimension is related to the quality of service provided in different parts of the hospital and possible complications in improving the quality of services. The ‘satisfaction’ dimension focuses on the satisfaction of patients, employees, and their complaints. The ‘innovation’ dimension relates to the research process and its output. The ‘appropriateness’ dimension involves proper service from clinical departments, pharmaceutical services, and patient treatment. The ‘evaluation’ dimension focuses on the indicators related to the assessment scores of the para-clinical departments of the hospital.

The ‘profitability’ dimension focuses on the overall output indicators for income and profitability. The ‘cost’ dimension focuses on indicators related to general expenditures and the average cost per bed and patient and budgeting. The ‘economy’ dimension is related to financial rates and their indicators. The ‘coherence’ dimension emphasizes the indicators related to the continuity of the service delivery process. The ‘patient-centeredness’ dimension focuses on the indicators related to the patient’s experience of the facility, environment, treatment processes, communications, and relevant support for the patient. The ‘equity’ dimension studies indicators related to social and financial justice and life expectancy. The ‘relationship’ dimension evaluates the process of consultations and discussions required during the patients’ care provided by the treatment team. The ‘sustainability’ dimension focuses on indicators related to energy standards. The ‘flexibility’ dimension focuses on the hospital’s response to the crisis.

According to Table  4 , most studies focused on ‘efficiency’, ‘productivity’, ‘safety’ and ‘effectiveness’ as performance dimensions in 54, 53, 38 and 37 studies, respectively (40–70% of studies). In the ‘efficiency’ subcategory, resource management, supportive unit assessment, and human resource management indicators were the first to third most common indicators used in 26, 23 and 22 studies, respectively (approximately 25% of the studies).

In addition, for the ‘efficiency’ dimension, ‘medical staff numbers’, ‘emergency department bed numbers’, and ‘nonmedical staff numbers’ were reported in 16, 13, and 11 studies, respectively (between 20 and 30% of the studies). For the ‘productivity’ subcategory, ‘bed utilization rate’ and ‘service delivery and treatment’ were reported in 50% and 20% of the studies, respectively (46 and 19 out of 91).

Additionally, for the ‘productivity’ dimension, the ‘length of stay’ indicator was used more than others and reported in approximately 80% of the studies (43 out of 53), followed by the ‘bed occupancy rate’ in approximately 40% of the studies (21 out of 53). The ‘bed turnover ratio’ and ‘hospitalization rate’ were also reported in 12 studies. Furthermore, for ‘safety’ dimensions, all indicators were in the ‘patient safety’ subcategory, which has been reported in 38 studies, and ‘complications’, ‘accidents or adverse events’, and ‘incidents or errors rates’ were the most concentrated indicators by researchers in 13, 12, and 11 studies, respectively. The performance dimension of ‘effectiveness’ was presented in 37 studies (40%), with only two indicators, ‘mortality rate’ in 29 studies and ‘readmission rate’ in 23 studies.

Performance categories

Considering the three categories shown in Table  4 , ‘organizational management’ indicators were more commonly used among the other two categories (‘clinical’ and ‘administrative’) and were present in more than 85% of the studies (78 out of 91). Two categories, ‘clinical management’ and ‘administrative management’, were reported in 62 and 51 studies, respectively.

Performance subcategories

Considering the 14 subcategories shown in Table  4 , both the ‘bed utilization rate’ and ‘patient safety’ indicators were mentioned in 46 studies and were more common among the other subcategories. The second most common indicator of the ‘financial management’ subcategory was reported in 38 studies. At the third level, both the ‘human resource management’ and ‘time management’ indicators were presented in 31 studies. The ‘paramedical’ subcategory indicators were presented in less than 10% of the studies [ 60 , 96 , 97 , 98 , 106 , 113 ].

Performance indicators

According to the indicator columns in Table  3 , the most used indicators in reviewed studies were the length of stay, mortality rate, and readmission rate in 47%, 32%, and 25% of studies, respectively. Bed occupancy rate and non-personnel costs were reported in 23% of studies. Additionally, among the 110 indicators, 16 indicators, namely, the lab cancellation rate, exam-physician ratios, number of coded diagnoses, number of medical records, laboratory sample/report intervals, medical information request time, safety standards in the archives, nutritional risk screening, imaging quality control failures, errors in medical reports, average impact factor, nutritional measures, laboratory scoring, imaging inspection, discharge process and emergency response rate, were reported in less than 1% of the studies.

The classification of the indicators in Table  4 was performed based on the chain model, which included the input, process, output, outcome and impact. The assignment of the indicators to each category was performed according to the experts’ opinions. For instance, the number of publications by academic member of an academic hospital and the average impact factor of those publications were considered outcome indicators. As depicted in the Table  4 , most studies (80%) focused more on output indicators. Additionally, fifteen studies focused on introducing and extracting some of the input, process, output, outcome and impact indicators; among those, only one study [ 96 ] has examined the input, process, output and impact indicators simultaneously.

Additionally, in approximately 42% (36 out of 91) of the studies, the indicators’ definitions, formulas, or descriptions have been illustrated, while less than 10% of the studies have defined measuring units, standard or benchmark units for all studied indicators [ 15 , 43 , 45 , 51 , 52 , 57 , 67 ].

Overall, nine studies related to hospital performance evaluation were conducted using systematic review methodologies (five systematic reviews [ 16 , 29 , 30 , 56 , 113 ], two literature reviews [ 79 , 80 ], one narrative review [ 98 ] and one brief review [ 92 ]). Most of these studies focused on extracting performance indicators from one or more hospital departments (e.g., the emergency department) [ 16 , 56 ], hospital laboratory and radiology information systems [ 106 ], supply chain performance [ 29 ], resources and financial results and activity [ 113 ], hospital water consumption [ 30 ], and the pharmaceutical sector [ 98 ]. Other reviews included a three-step process to review, evaluate and rank these hospital indicators in a systematic approach [ 16 ], or to evaluate performance indicator models to create an interactive network and visualize the causal relationships between performance indicators [ 79 ]; moreover, some have focused on the importance of indicators to ensure adequate coverage of the relevant areas of health care services to be evaluated [ 92 ].

Only one scoping review aimed to identify current assessments of hospital performance and compared quality measures from each method in the context of the six qualitative domains of STEEEP (safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness) of the Institute of Medicine (IOM) in accordance with Donabedian’s framework and formulating policy recommendations [ 115 ].

In addition, 21 studies divided performance indicators into 2 to 6 dimensions of performance. Also, the reviewed studies included 2–40 indicators in zero [ 29 , 30 , 98 ] to 6 domains [ 34 ]. Moreover, none of the studies have tried to comprehensively summarize and categorize the performance indicators in several categories, focusing on all the indicators reflecting the performance of the entire hospital organization, or the indicators of administrative units or clinical departments.

In this scoping review, a unique set of hospital performance evaluation indicators related to the various performance dimensions was categorized from 91 studies over the past ten years.

Similarly, in a study, 19 performance dimensions, 32 sub-dimensions, and 138 indicators were extracted from only six studies. Those dimensions were described by all studies included in the review, but only three studies specified the relevant indicators, and the list provided for all possible indicators was not comprehensive. Also, despite current review, there was no classification of indicators based on the hospital levels: managerial, clinical, or organizational levels [ 116 ]. Another study has similarly investigated the performance evaluation indicators of the hospital in such a way that among 42 studies, 111 indicators were presented in the four categories: input, output, outcome, and impact. But, there was no classification of indicators based on performance dimensions and hospital levels [ 117 ].

In this study, the importance of categorized indicators, for the first time to our knowledge, was determined based on their frequency of use in the published literature (Appendix 2 ). The ‘Organizational management’ indicators were the most common compared with the other two categories (‘clinical’ and ‘administrative’). It could be because of the fact that the indicators such as ‘bed occupancy rate’, ‘average length of stay’, ‘mortality rate’, ‘hospital infection rate’, and ‘patient safety’ are easier to be registered in hospital software compared to other indicators, and also they better reflect the overall performance of hospital. Thus, researchers are more interested in using these indicators.

Considering 14 subcategories, indicators related to three subcategories i.e. bed utilization, patient safety and financial management are the most frequent used indicators for hospital performance evaluation. It reflects the need of hospital managers to increase the profitability of hospital in one hand, and to control cost on the other hand. As a results, researchers have paid special attention to ‘cost income’, ‘profitability’, ‘economic’, etc., as indicators for evaluating hospital performance.

When considering indicators by type, more studies have focused on output indicators, while input indicators were the least common used. This might be because of the fact that at hospital level, it is difficult for managers to change those inputs such as ‘beds’, ‘human resources’, ‘equipment and facilities’. In addition, due to the complexity of interdepartmental relationships in hospitals, process indicators seemed to provide more variety for analysis than input indicators, so they were more often used. As mentioned above, output indicators were the most used indicators for hospital performance evaluation due to their ease of calculation and interpretation.

The main purpose of this paper was to identify a comprehensive set of indicators that can be used to evaluate hospital performance in various hospital settings by being distilled into a smaller and more related set of indicators for every hospital or department setting. future studies could be designed to validate each set of indicators in any specific context. In addition, they could investigate the relationship between the indicators and their outcomes of interest and the performance dimension each could address. This will enable hospital managers to build their own set of indicators for performance evaluation both at organization or at department level. Also it should be mentioned that.

Although some previous studies have provided definitions for each indicator and determined the standard criteria for them, this was not done in this study because the focus of this study was to provide a collection of all the indicators used in hospital performance evaluation, which resulted in the identification of more than a thousand indicators without limiting to specific country or context. So while preparing a smaller set of indicators, specific conditions of each country, such as the type of health system and its policy, the type of financing system, and the structure of services, should be taken into account to select appropriate indicators.

In addition, although it is important to examine the scope of each article to compare the list of indicators and the relationships between the dimensions of the hospital in terms of size and type and between the number and type of selected indicators, this was considered beyond the scope of this review due to the high number of indicators, which made the abovementioned investigations impossible. Future studies could do that while working with a smaller set of indicators.

This review aimed to categorize and present a comprehensive set of indicators for evaluating overall hospital performance in a systematic way. 1161 hospital performance indicators were drawn from 91 studies over the past ten years. They then were summarized into 110 main indicators, and categorized into three categories: 14 subcategories, and 21 performance dimensions This scoping review also highlighted the most frequent used indicators in performance evaluation studies which could reflect their importance for that purpose. The results of this review help hospital managers to build their own set of indicators for performance evaluation both at organization or at department level with regard to various performance dimensions.

As the results of this review was not limited to any specific country or context, specific conditions of each country, such as the type of health system and its policy, the type of financing system, and the structure of services, should be taken into account while selecting appropriate indicators as a smaller set of indicators for hospital performance evaluation in specific context.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Gross domestic product

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews

Emergency departments

Intensive care unit

Operating room

Surgical intensive care unit

Neonatal intensive care unit

Readmission rate

Quality Control

Medication use evaluation

safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness

Institute of Medicine

Abdullah A, Ahmad S, Athar MA, Rajpoot N, Talib F. Healthcare performance management using integrated FUCOM-MARCOS approach: the case of India. Int J Health Plann Manage. 2022;37(5):2635–68.

Article   PubMed   Google Scholar  

Pestana M, Pereira R, Moro S. Improving health care management in hospitals through a productivity dashboard. J Med Syst. 2020;44(4):87.

Amos D. A practical framework for performance measurement of facilities management services in developing countries’ public hospitals. J Facil Manag. 2022;20(5):713–31.

Article   Google Scholar  

Ahmed S, Hasan MZ, MacLennan M, Dorin F, Ahmed MW, Hasan MM, et al. Measuring the efficiency of health systems in Asia: a data envelopment analysis. BMJ Open. 2019;9(3):e022155.

Article   PubMed   PubMed Central   Google Scholar  

Mitkova Z, Doneva M, Gerasimov N, Tachkov K, Dimitrova M, Kamusheva M, et al. Analysis of healthcare expenditures in Bulgaria. Healthc. 2022;10(2):274.

Patrinely JR, Walker SH, Glassman GE, Davis MJ, Abu-Ghname A, Khan U, et al. The importance of financial metrics in physician funding and performance evaluation. Plast Reconstr Surg. 2021;147:1213–8.

Article   CAS   PubMed   Google Scholar  

Buathong S, Bangchokdee S. The use of the performance measures in Thai public hospitals. ASIAN Rev Acc. 2017;25(4):472–85.

Google Scholar  

Imani A, Alibabayee R, Golestani M, Dalal K. Key indicators affecting hospital efficiency: a systematic review. Front Public Heal. 2022;10:830102.

Mahdiyan S, Dehghani A, Tafti AD, Pakdaman M, Askari R. Hospitals’ efficiency in Iran: a systematic review and meta-analysis. J Educ Health Promot. 2019;8(1):126.

PubMed   PubMed Central   Google Scholar  

Amos D, Musa ZN, Au-Yong CP. Performance measurement of facilities management services in Ghana’s public hospitals. Build Res Inf. 2020;48(2):218–38.

Feibert DC, Andersen B, Jacobsen P. Benchmarking healthcare logistics processes–a comparative case study of Danish and US hospitals. Total Qual Manag Bus Excell. 2019;30(1–2):108–34.

Gün I, Yilmaz F, Şenel IK. Efficiency analysis of health systems in world bank countries. Arch Heal Sci Res. 2021;8(2):147–52.

Breyer JZ, Giacomazzi J, Kuhmmer R, Lima KM, Hammes LS, Ribeiro RA, et al. Hospital quality indicators: a systematic review. Int J Health Care Qual Assur. 2019;32(2):474–87.

Regragui H, Sefiani N, Azzouzi H. Improving performance through measurement: the application of BSC and AHP in healthcare organization. In: Equipe De Recherche, Ingénierie, Innovation Et Management Des Systèmes Industriels, Université Abdelmalek Saadi. Tanger, Morocco: Institute of Electrical and Electronics Engineers Inc; 2018. p. 51–6.

Ghozali MT, Latifah DN, Darayani A. Analysis of Drug Supply Management of the Pharmacy Warehouse of Prof. Dr. Soerojo Mental Health Hospital, Magelang, Indonesia. Clin Schizophr Relat Psychoses. 2021;15:1–6.

Etu EE, Monplaisir L, Aguwa C, Arslanturk S, Masoud S, Markevych I, et al. Identifying indicators influencing emergency department performance during a medical surge: a consensus-based modified fuzzy Delphi approach. PLoS ONE. 2022;17(4 April):e0265101.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lin C-Y, Shih F-C, Ho Y-H. Applying the balanced scorecard to build service performance measurements of medical institutions: An AHP-DEMATEL approach. Int J Environ Res Public Health. 2023;20(2):1022.

Backman C, Vanderloo S, Forster AJ. Measuring and improving quality in university hospitals in Canada: the collaborative for excellence in healthcare quality. Health Policy (New York). 2016;120(9):982–6.

Ghalem Â, Okar C, Chroqui R, Semma E. Performance: A concept to define. In: Performance: A concept to define. LOGISTIQUA 2016; 2016. p. 1–13.

Sonnentag S, Frese M. Performance Concepts and Performance Theory. In 2005. p. 1–25.

Tangen S. Demystifying productivity and performance. Int J Prod Perform Manag. 2005;54:34–46.

Elena-Iuliana I, Maria C. Organizational Performance – A Concept That Self-Seeks To Find Itself. Ann - Econ Ser Constantin Brancusi Univ Fac Econ. 2016;4(4):179–83.

Riratanaphong C, Van der Voordt T, Sarasoja A. Performance Measurement in the context of CREM and FM. In: Per Anker Jensen, Theo Van der Voordt CC, editor. The added value of facilities management: concepts, findings and perspectives. Lyngby Denmark: Polyteknisk Forlag; 2012. p. 1–21.

Pham M, Rajić A, Greig J, Sargeant J, Papadopoulos A, Mcewen S. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85.

Chaney M. So you want to write a narrative review article? J Cardiothorac Vasc Anesth. 2021;35:3045–9.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Tricco A, Lillie E, Zarin W, O’Brien K, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Dolatabad AH, Mahdiraji HA, Babgohari AZ, Garza-Reyes JA, Ai A. Analyzing the key performance indicators of circular supply chains by hybrid fuzzy cognitive mapping and Fuzzy DEMATEL: evidence from healthcare sector. Environ Dev Sustain. 2022;1–27.

Batista KJM, da Silva SR, Rabbani ERK, Zlatar T. Systematic review of indicators for the assessment of water consumption rates at hospitals. Water Supply. 2020;20(2):373–82.

Beta G, Role D, Berloviene D, Balkena Z. PATIENTS’ SATISFICATION AS THE QUALITY INDICATOR OF NURSING. In: Lubkina V, Kaupuzs A, Znotina D, editors. SOCIETY INTEGRATION EDUCATION, VOL VI: PUBLIC HEALTH AND SPORT, RESEARCHES IN ECONOMICS AND MANAGEMENT FOR SUSTAINABLE EDUCATION. 2020. p. 79–88.

Bhardwaj P, Joshi NK, Singh P, Suthar P, Joshi V, Jain YK, et al. Competence-based assessment of biomedical equipment management and maintenance system (e-Upkaran) using benefit evaluation framework. CUREUS J Med Sci. 2022;14(10):e30579.

Cheon O, Song M, Mccrea AM, Meier KJ. Health care in America: the relationship between subjective and objective assessments of hospitals. Int PUBLIC Manag J. 2021;24(5):596–622.

Craig KJT, McKillop MM, Huang HT, George J, Punwani ES, Rhee KB. US hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm. BMC Health Serv Res. 2020;20(1):640.

Cristiano S, Ulgiati S, Gonella F. Systemic sustainability and resilience assessment of health systems, addressing global societal priorities: Learnings from a top nonprofit hospital in a bioclimatic building in Africa. Renew Sustain ENERGY Rev. 2021;141:110765.

Dadi D, Introna V, Santolamazza A, Salvio M, Martini C, Pastura T, et al. Private Hospital Energy Performance Benchmarking Using Energy Audit Data: An Italian Case Study. Energies. 2022;15(3):1–16.

Dawande PP, Wankhade RS, Akhtar FI, Noman O. Turnaround time: an efficacy measure for medical laboratories. CUREUS J Med Sci. 2022;14(9):e28824.

De Sousa LR, Mazzo A, De Almeida ACF, Tonello C, Lourençone LFM. Evaluation of quality indicators in the management of an operating room at a tertiary-level hospital. Med. 2022;55(1):1–8.

Drynda S, Schindler W, Slagman A, Pollmanns J, Horenkamp-Sonntag D, Schirrmeister W, et al. Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study. BMJ Open. 2020;10(9):e038776.

Fekri O, Manukyan E, Klazinga N. Appropriateness, effectiveness and safety of care delivered in Canadian hospitals: a longitudinal assessment on the utility of publicly reported performance trend data between 2012–2013 and 2016–2017. BMJ Open. 2020;10(6):e035447.

Galloa AJO, Ramírez CA. Evaluating Colombian public hospitals productivity during 2004–2015. A luenberger-indicator approach. Rev Gerenc Y Polit Salud. 2020;19:1–23.

Gebreyes M, Sisay A, Tegen D, Asnake A, Wolde M. Evaluation of laboratory performance, associated factors and staff awareness towards achieving turnaround time in tertiary hospitals, Ethiopia. Ethiop J Health Sci. 2020;30(5):767–76.

Gnanaraj J, Kulkarni RG, Sahoo D, Abhishekh B. Assessment of the Key Performance Indicator Proposed by NABH in the Blood Centre of a Tertiary Health Care Hospital in Southern India. Indian J Hematol Blood Transfus. 2022;39:308–16.

Horenberg F, Lungu DA, Nuti S. Measuring research in the big data era: the evolution of performance measurement systems in the Italian teaching hospitals. Health Policy (New York). 2020;124(12):1387–94.

Hwang DK, Cho J, Moon J. Feasibility study on energy audit and data driven analysis procedure for building energy efficiency: bench-marking in Korean hospital buildings. Energies. 2019;14(15):3006.

Jaskova D. Efficiency of management, processes in a private hospital. Entrep Sustain Issues. 2021;9(1):436–46.

Jebraeily M, Valizadeh MA, Rahimi B, Saeidi S. The development of a clinical dashboard for monitoring of key performance indicators in ICU. J Iran Med Counc. 2022;5(2):308–17.

Kang Y, Kim M, Jung K. The equity of health care spending in South Korea: testing the impact of publicness. Int J Environ Res Public Health. 2020;17(5):1775.

Abou Tarieh RR, Zayyat R, Naoufal RN, Samaha HR. A case study exploring the impact of JCI standards implementation on staff productivity and motivation at the laboratory and blood bank. Heal Sci Rep. 2022;5(1):e497.

Kadoic N, Simic D, Mesaric J, Redep NB. Measuring quality of public hospitals in croatia using a multi-criteria Approach. Int J Environ Res Public Health. 2021;18:19.

Khalilabad T, Amir N, Asl P, Raeissi Shali M, Niknam N. Assessment of clinical and paraclinical departments of military hospitals based on the Pabon Lasso Model. J Educ Health Promot. 2020;9:1–6.

Lai JHK, Hou H, Edwards DJ, Yuen PL. An analytic network process model for hospital facilities management performance evaluation. Facilities. 2022;40(5–6):333–52.

Lai J, Yuen PL. Identification, classification and shortlisting of performance indicators for hospital facilities management. Facilities. 2021;39(1–2):4–18.

Lin CS, Chiu CM, Huang YC, Lang HC, Chen MS. Evaluating the operational efficiency and quality of Tertiary hospitals in Taiwan: the application of the EBITDA Indicator to the DEA Method and TOBIT Regression. Healthcare. 2022;10(1):58.

Matos R, Ferreira D, Pedro MI. Economic analysis of portuguese public hospitals through the construction of quality, efficiency, access, and financial related composite indicators. Soc Indic Res. 2021;157(1):361–92.

Morisod K, Luta X, Marti J, Spycher J, Malebranche M, Bodenmann P. Measuring health equity in emergency care using routinely collected data: a systematic review. Heal Equity. 2021;5(1):801–17.

Nik Hisamuddin R, Tuan Hairulnizam TK. Developing key performance indicators for emergency department of teaching hospitals: a mixed fuzzy Delphi and nominal group technique approach. Malays J Med Sci. 2022;29(2):114–25.

Ramírez Calazans A, Paredes Esteban RM, Grijalva Estrada OB, Ibarra Rodríguez MR. Assessment of quality indicators in pediatric major outpatient surgery. Influence of the COVID-19 pandemic. Cir Pediatr. 2023;36(1):17–21.

PubMed   Google Scholar  

Shaqura II, Gholami M, Akbari Sari A. Assessment of public hospitals performance in Gaza governorates using the Pabón Lasso Model. Int J Health Plann Manage. 2021;36(4):1223–35.

Al-Jazairi AS, Alnakhli AO. Quantifying clinical pharmacist activities in a tertiary care hospital using key performance indicators. Hosp Pharm. 2021;56(4):321–7.

Aloh HE, Onwujekwe OE, Aloh OG, Nweke CJ. Is bed turnover rate a good metric for hospital scale efficiency? A measure of resource utilization rate for hospitals in Southeast Nigeria. Cost Eff Resour Alloc. 2020;18(1):1–8.

Bari S, Incorvia J, Ahearn O, Dara L, Sharma S, Varallo J, et al. Building safe surgery knowledge and capacity in Cambodia: a mixed-methods evaluation of an innovative training and mentorship intervention. Glob Health Action. 2021;14(1):1998996.

Bari S, Incorvia J, Iverson KR, Bekele A, Garringer K, Ahearn O, et al. Surgical data strengthening in Ethiopia: results of a Kirkpatrick framework evaluation of a data quality intervention. Glob Health Action. 2021;14(1):1–11.

Bastos LSL, Hamacher S, Zampieri FG, Cavalcanti AB, Salluh JIF, Bozza FA. Structure and process associated with the efficiency of intensive care units in low-resource settings: an analysis of the CHECKLIST-ICU trial database. J Crit Care. 2020;59:118–23.

Bastos LSL, Wortel SA, de Keizer NF, Bakhshi-Raiez F, Salluh JIF, Dongelmans DA, et al. Comparing continuous versus categorical measures to assess and benchmark intensive care unit performance. J Crit Care. 2022;70:154063.

Kocisova K, Hass-Symotiuk M, Kludacz-Alessandri M. Use of the dea method to verify the performance model for hospitals. E M Ekon A Manag. 2018;21(4):125–40.

Lee D, Yu S, Yoon SN. Analysis of hospital management based on the characteristics of hospitals: focusing on financial indicators. Glob Bus Financ Rev. 2019;24(3):1–13.

Mirzaei A, Tabibi SJ, Nasiripour AA, Riahi L. Evaluating the feasibility of financial variables of health: A hospital administrator’s viewpoint. Galen Med J. 2016;5(1):25–30.

Middleton S, Gardner G, Gardner A, Considine J, FitzGerald G, Christofis L, et al. Are service and patient indicators different in the presence or absence of nurse practitioners? The EDPRAC cohort study of Australian emergency departments. BMJ Open. 2019;9(7):e024529.

Nobakht S, Jahangiri K, Hajinabi K. Correlation of performance indicators and productivity: A cross sectional study of emergency departments in Tehran, Iran during year 2016. Trauma Mon. 2018;23(5):1–6.

Nuti S, Grillo Ruggieri T, Podetti S. Do university hospitals perform better than general hospitals? A comparative analysis among Italian regions. BMJ Open. 2016;6(8):e011426.

Petrovic GM, Vukovic M, Vranes AJ. The impact of accreditation on health care quality in hospitals. Vojnosanit Pregl. 2018;75(8):803–8.

Pirani N, Zahiri M, Engali KA, Torabipour A. Hospital efficiency measurement before and after health sector evolution plan in Southwest of Iran: a DEA-panel data study. Acta Inf Med. 2018;26(2):106–10.

Profit J, Gould JB, Bennett M, Goldstein BA, Draper D, Phibbs CS, et al. The association of level of care with NICU quality. Pediatrics. 2016;137(3):44–51.

Rahimi H, Bahmaei J, Shojaei P, Kavosi Z, Khavasi M. Developing a strategy map to improve public hospitals performance with balanced scorecard and dematel approach. Shiraz E Med J. 2018;19(7):1–12.

Ahmed S, Hasan MZ, Laokri S, Jannat Z, Ahmed MW, Dorin F, et al. Technical efficiency of public district hospitals in Bangladesh: a data envelopment analysis. COST Eff Resour Alloc. 2019;17:17.

Rahman MH, Tumpa TJ, Ali SM, Paul SK. A grey approach to predicting healthcare performance. Meas J Int Meas Confed. 2019;134:307–25.

Sajadi HS, Sajadi ZS, Sajadi FA, Hadi M, Zahmatkesh M. The comparison of hospitals’ performance indicators before and after the Iran’s hospital care transformations plan. J Educ Health Promot. 2017;6:89.

Si S-L, You X-Y, Liu H-C, Huang J. Identifying key performance indicators for holistic hospital management with a modified DEMATEL approach. Int J Environ Res Public Health. 2017;14(8): 934.

Váchová L, Hajdíková T. Evaluation of Czech hospitals performance using MCDM methods. In: A SI, G WS, C D, editors. Department of exact methods, faculty of management, university of economics, Prague, Jarošovská 1117, Jindřichuv Hradec, vol. 37701. Czech Republic: Newswood Limited; 2017. p. 732–5.

Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. Cost Eff Resour Alloc. 2017;15(1):6.

Zhang L, Liu R, Jiang S, Luo G, Liu H-C. Identification of key performance indicators for hospital management using an extended hesitant linguistic DEMATEL Approach. Healthc (Basel Switzerland). 2019;8(1):7.

Aksezer CS. A nonparametric approach for optimal reliability allocation in health services. Int J Qual Reliab Manag. 2016;33(2):284–94.

Cagliano AC, Grimaldi S, Rafele C. Assessing warehouse centralization and outsourcing in the healthcare sector: an Italian case study. In: Department of Management and Production Engineering, Politecnico Di Torino, Corso Duca Degli Abruzzi 24, Torino, 10129. Italy: AIDI - Italian Association of Industrial Operations Professors; 2017. p. 244–50.

Cefalu MS, Elliott MN, Setodji CM, Cleary PD, Hays RD. Hospital quality indicators are not unidimensional: a reanalysis of Lieberthal and Comer. Health Serv Res. 2019;54(2):502–8.

Gao H, Chen H, Feng J, Qin X, Wang X, Liang S, et al. Balanced scorecard-based performance evaluation of Chinese county hospitals in underdeveloped areas. J Int Med Res. 2018;46(5):1947–62.

Gonnelli V, Satta F, Frosini F, Iadanza E. Evidence-based approach to medical equipment maintenance monitoring. In: V HEO, V J, editors. University of Florence, Dept. of Information Engineering. Florence, Italy: Springer; 2017. p. 258–61.

Helkio P, Aantaa R, Virolainen P, Tuominen R. Productivity benchmarks for operative service units. ACTA Anaesthesiol Scand. 2016;60(4):450–6.

Khalifa M, Zabani I. Developing emergency room key performance indicators: What to measure and why should we measure it? J. M, A. H, P. G, A. K, M.S. H, editors. Vol. 226. King Faisal Specialist Hospital and Research Center, Jeddah, Saudi Arabia: IOS Press BV; 2016. p. 179–182.

Ajami S, Ebadsichani A, Tofighi S, Tavakoli N. Medical records department and balanced scorecard approach. J Educ Health Promot. 2013;2:7.

Bosse G, Mtatifikolo F, Abels W, Strosing C, Breuer J-P, Spies C. Immediate outcome indicators in perioperative care: a controlled intervention study on quality improvement in hospitals in Tanzania. PLoS One. 2013;8(6):e65428.

Hung K-Y, Jerng J-S. Time to have a paradigm shift in health care quality measurement. J Formos Med Assoc. 2014;113(10):673–9.

Jeejeebhoy KN, Keller H, Gramlich L, Allard JP, Laporte M, Duerksen DR, et al. Nutritional assessment: comparison of clinical assessment and objective variables for the prediction of length of hospital stay and readmission. Am J Clin Nutr. 2015;101(5):956–65.

Kittelsen SAC, Anthun KS, Goude F, Huitfeldt IMS, Häkkinen U, Kruse M, et al. Costs and quality at the hospital level in the nordic countries. Heal Econ (United Kingdom). 2015;24:140–63.

Koné Péfoyo AJ, Wodchis WP. Organizational performance impacting patient satisfaction in Ontario hospitals: a multilevel analysis. BMC Res Notes. 2013;6: 509.

Li CH, Yu CH. Performance evaluation of public non-profit hospitals using a BP Artificial neural network: the case of Hubei Province in China. Int J Environ Res Public Health. 2013;10(8):3619–33.

Liu K, Jain S, Shi J. Physician performance assessment using a composite quality index. Stat Med. 2013;32(15):2661–80.

Lloyd GF, Bajorek B, Barclay P, Goh S. Narrative review: Status of key performance indicators in contemporary hospital pharmacy practice. J Pharm Pract Res. 2015;45(4):396–403.

Mehrtak M, Yusefzadeh H, Jaafaripooyan E. Pabon Lasso and data envelopment analysis: a complementary approach to hospital performance measurement. Glob J Health Sci. 2014;6(4):107–16.

Mohammadi M, Ziapoor A, Mahboubi M, Faroukhi A, Amani N, Pour FH, et al. Performance evaluation of hospitals under supervision of Kermanshah medical sciences using pabonlasoty diagram of a five-year period (2008–2012). Life Sci J. 2014;11:77–81 ( 1 SPECL. ISSUE) ).

Niaksu O, Zaptorius J. Applying operational research and data mining to performance based medical personnel motivation system. In: Vilnius University, Institute of Mathematics and Informatics. Lithuania: IOS; 2014. p. 63–70.

Córdoba S, Caballero I, Navalón R, Martínez-Sánchez D, Martínez-Morán C, Borbujo J. Analysis of the surgical activity in the dermatology department of Fuenlabrada University Hospital, Madrid, Spain, between 2005 and 2010: determination of the standard operating times. Actas Dermosifiliogr. 2013;104(2):141–7.

Profit J, Kowalkowski MA, Zupancic JAF, Pietz K, Richardson P, Draper D, et al. Baby-MONITOR: a composite indicator of NICU Quality. Pediatrics. 2014;134(1):74–82.

Rabar D, Pap N. Evaluation of crotia’s regional hospital effiency: an application of data envelopment analysis . Bacher U, Barkovic D, Dernoscheg KH, LamzaMaronic M, Matic B, Runzheimer B, editors. Interdisciplinary Management Research IX. 2013;9:649–59.

Ramos MCA, da Cruz LP, Kishima VC, Pollara WM, de Lira ACO, Couttolenc BF. Performance evaluation of hospitals that provide care in the public health system, Brazil. Rev Saude Publica. 2015;49:1–9.

Schuers M, Joulakian MB, Griffon N, Pachéco J, Périgard C, Lepage E, et al. In: S IN, de PM AM, editors. Quality indicators from laboratory and radiology information systems. A. G. Volume 216. France: IOS; 2015. pp. 212–6. Department of Biomedical Informatics, Rouen University Hospital, Rouen Cedex, 76031,.

Tabrizi JS, Saadati M, Sadeghi-Bazargani H, Ebadi A, Golzari SEJ. Developing indicators to improve educational governance in hospitals. Clin Gov. 2014;19(2):117–25.

Costa A Jr, aS., Leão LE, Novais MA, Zucchi P. An assessment of the quality indicators of operative and non-operative times in a public university hospital. Einstein (Sao Paulo). 2015;13(4):594–9.

Coyne JS, Helton J. How prepared are US hospitals for the affordable care act? A financial condition analysis of US hospitals in 2011. J Health Care Finance. 2015;41(3).

Davis P, Milne B, Parker K, Hider P, Lay-Yee R, Cumming J, et al. Efficiency, effectiveness, equity (E-3). Evaluating hospital performance in three dimensions. Health Policy (New York). 2013;112(1–2):19–27.

Flatow VH, Ibragimova N, Divino CM, Eshak DSA, Twohig BC, Bassily-Marcus AM, et al. Quality outcomes in the surgical intensive care unit after electronic health record implementation. Appl Clin Inf. 2015;6(4):611–8.

Article   CAS   Google Scholar  

Fonseca JRS, Ramos RMP, Santos AMP, Fonseca APSS. Policy effects on the quality of public health care: evaluating Portuguese public hospitals’ quality through customers’ views. Cent Eur J Public Policy. 2015;9(2):122–40.

Hadji B, Meyer R, Melikeche S, Escalon S, Degoulet P. Assessing the Relationships Between Hospital Resources and Activities: A Systematic Review. J Med Syst. 2014;38(10):1–21.

Hajduová Z, Herbrik G, Beslerová S. Application of DEA in the environment of Slovak hospitals. Invest Manag Financ Innov. 2015;12(4):148–53.

Thomas Craig KJ, McKillop MM, Huang HT, George J, Punwani ES, Rhee KB. U.S. hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm. BMC Health Serv Res. 2020;20(1):640.

Carini E, Gabutti I, Frisicale EM, Di Pilla A, Pezzullo AM, de Waure C, et al. Assessing hospital performance indicators. What dimensions? Evidence from an umbrella review. BMC Health Serv Res. 2020;20(1):1038.

Rasi V, Delgoshaee B, Maleki M. Identification of common indicators of hospital performance evaluation models: a scoping review. J Educ Health Promot. 2020;9(1):63.

Xenos P, Yfantopoulos J, Nektarios M, Polyzos N, Tinios P, Constantopoulos A. Efficiency and productivity assessment of public hospitals in Greece during the crisis period 2009–2012. COST Eff Resour Alloc. 2017;15:15.

Shaqura II, Gholami M, Sari AA. Evaluation of performance at Palestinian public hospitals using Pabon Lasso model. Int J Health Plann Manage. 2021;36(3):896–910.

Li J, Seale H, Ray P, Wang Q, Yang P, Li S, et al. E-Health preparedness assessment in the context of an influenza pandemic: a qualitative study in China. BMJ Open. 2013;3(3):e002293.

Huang C-Y, Lai C-H. Effects of internal branding management in a hospital context. Serv Ind J. 2021;41(15–16):985–1006.

Download references

Acknowledgements

The authors are grateful for the support of the Vice Chancellor for Research of Isfahan University of Medical Sciences.

The present article is part of the result of a doctoral thesis approved by Isfahan University of Medical Sciences with code 55657 (IR.MUI.NUREMA.REC.1401.005), without financial source.

Author information

Authors and affiliations.

Student Research Committee, School of Management and Medical Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Shirin Alsadat Hadian

Health Management and Economics Research Center, Isfahan University of Medical Sciences, Isfahan, Iran

Reza Rezayatmand

Hospital Management Research Center, Health Management Research Institute, Iran University of Medical Sciences, Tehran, Iran

Nasrin Shaarbafchizadeh

Department of Management, Faculty of Administrative Sciences and Economics, University of Isfahan, Isfahan, Iran

Saeedeh Ketabi

School of Public Health and Preventive Medicine, Monash University, Victoria, Australia

Ahmad Reza Pourghaderi

You can also search for this author in PubMed   Google Scholar

Contributions

Shirin Alsadat Hadian and Reza Rezayatmans and Saeedeh Ketabi: Study conceptualization and design. Acquisition of data: Shirin Alsadat Hadian, Reza Rezayatmand. Analysis and interpretation of data: Shirin Alsadat Hadian, Reza Rezayatmand, Nasrin Shaarbafchizadeh, Saeedeh Ketabi. Drafting of the manuscript: Shirin Alsadat Hadian, Reza Rezayatmand. Critical revision of the manuscript for important intellectual content: Reza Rezayatmand, Nasrin Shaarbafchizadeh, Saeedeh Ketabi, Ahmad Reza Pourghaderi.

Corresponding author

Correspondence to Reza Rezayatmand .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hadian, S.A., Rezayatmand, R., Shaarbafchizadeh, N. et al. Hospital performance evaluation indicators: a scoping review. BMC Health Serv Res 24 , 561 (2024). https://doi.org/10.1186/s12913-024-10940-1

Download citation

Received : 03 January 2024

Accepted : 02 April 2024

Published : 01 May 2024

DOI : https://doi.org/10.1186/s12913-024-10940-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Performance evaluation

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 10 May 2024

Happiness amidst the COVID-19 pandemic in Indonesia: exploring gender, residence type, and pandemic severity

• Indera Ratna Irawati Pattinasarany   ORCID: orcid.org/0009-0008-1529-2751 1  

Humanities and Social Sciences Communications volume  11 , Article number:  609 ( 2024 ) Cite this article

Metrics details

This study delves into the dynamics shaping happiness levels in Indonesia before and during the COVID-19 pandemic, specifically emphasizing gender and residence-type disparities. Using data from the 2017 and 2021 Happiness Level Measurement Survey, it offers insights into how different population segments were affected. The analysis employs a multilevel mixed-effects ordered logistic model, considering individuals nested within provinces, and measures pandemic severity using positive COVID-19 cases per 100,000 residents. This study evaluates pandemic-related happiness shifts using nationwide cross-sectional survey data from two timeframes. It derives substantial statistical strength from data involving 137,000+ respondents gathered through comprehensive face-to-face interviews. It mitigates recall bias by capturing happiness at two distinct time points, avoiding retrospective measures. The study examines and validates four research questions. First, higher COVID-19 cases in provinces correlate with lower happiness. Second, though women were happier than men, the pandemic reduced this gender-based gap. Third, urban residents were generally happier than rural residents, but the pandemic narrowed this difference. All the estimates exhibit statistical significance at the 1 percent level. Finally, while provincial poverty showed minimal happiness impact, a negative association between unequal per capita expenditure and happiness emerged, providing partial backing for investigating the role of macroeconomic conditions. This study reveals that the COVID-19 pandemic altered happiness dynamics in Indonesia, narrowing gender and residence-based gaps. It also emphasizes the role of socioeconomic factors, particularly unequal per capita expenditure, in influencing individual happiness, highlighting implications for targeted policy interventions.

Similar content being viewed by others

Happiness inequality has a Kuznets-style relation with economic growth in China

Why has the COVID-19 pandemic increased support for Universal Basic Income?

Socioeconomic status and social capital as predictors of happiness: evidence and gender differences

Introduction.

Studying factors influencing our happiness has been a persistent and important topic of investigation over the years. Happiness holds significant implications for our lives, serving not only as a personal aspiration but also as a societal objective (Petrovič et al. 2021 ; Veenhoven 2012 ). Scholars and policymakers have been paying growing attention to subjective well-being (SWB) measures in recent decades. These measures have been sought as alternative ways to gauge economic and social progress, addressing concerns with traditional welfare indicators (Ahmadiani et al. 2022 ; Deaton and Stone 2013 ; Delhey and Kroll 2013 ). Notably, Oishi and Diener’s ( 2014 ) study revealed that self-reported happiness and life satisfaction could effectively reflect objective societal and economic conditions, quantify individuals’ hardships, and evaluate the effectiveness of specific public policies.

The impact of COVID-19 on SWB presents various perspectives. Firstly, a global decline in SWB is evident across studies, including those in China (Yang and Ma, 2020 ), Germany (Bittmann, 2022a ; Möhring et al. 2021 ), and a multi-country study encompassing China, Japan, South Korea, Italy, the United Kingdom, and the United States (Nguyen 2021 ). Secondly, the World Happiness Report (WHR) 2021 indicates a non-significant increase in global life evaluation indicators from 2017–2019 to 2020 (Helliwell et al. 2021 ), similarly reflected in Rajkumar’s ( 2023 ) research across 78 countries. Thirdly, French researchers discovered improved self-reported health and well-being during lockdown compared to previous years (Recchi et al. 2020 ). These diverse outcomes underscore the complex link between the pandemic and individuals’ SWB, arising from individual and household differences, contextual factors, and varying COVID-19 severity across regions.

As the world’s fourth most populous nation, Indonesia has confronted profound repercussions from the pandemic, ranking 20th worldwide in total reported COVID-19 cases and 11th in COVID-19-related fatalities (Worldometer 2023 ). Moreover, the variability in COVID-19 exposure across provinces and the distinction between urban and rural areas within Indonesia is noteworthy. Footnote 1 In light of these circumstances, it becomes essential to undertake an exhaustive study of how the pandemic’s severity has uniquely influenced the happiness of Indonesians.

This study aims to empirically examine the factors influencing shifts in happiness levels before and during the COVID-19 pandemic in Indonesian society. Given the indications from prior research that the pandemic affects women (Dang and Nguyen 2020 ; Fortier 2020 ; Gausman and Langer 2020 ; Giurge et al. 2021 ) and urban dwellers (López-Ruiz et al. 2021 ; Shams and Kadow 2022 ) disproportionately compared to other their respected counterparts, our investigation will primarily focus on comprehending the distinct contributions of gender and residency to the observed changes in happiness levels. By exploring how being male or female and where people live affect changes in happiness during the pandemic, we can better understand the different experiences and difficulties faced by different population segments. Significantly, this study stands as a pioneering effort to investigate the changes in happiness levels stemming from the COVID-19 pandemic among the broader populace of Indonesia.

This study addresses several limitations of existing literature on changes in happiness during the COVID-19 pandemic. Many of these previous investigations have not effectively addressed the following limitations: concentration on specific population segments (e.g., healthcare workers, students), employment of single-point-in-time data collection, dependence on convenience sampling for participant recruitment, administration of online surveys, limited observation durations, and reliance on participants’ retrospective reports of pre-pandemic circumstances.

We overcome these limitations because we use national-level cross-sectional survey data for two different points in time. First, our survey data covers the period before and during the pandemic, enabling us to examine changes in self-reported happiness levels associated with the pandemic’s impact. Using survey data from over 137,000 respondents provides this study with robust statistical power, enhancing the precision of our analysis of happiness level changes over time. Second, our survey data was collected through face-to-face interviews, employing a rigorous sampling method. This approach ensures a more representative sample distribution, avoiding biases from self-selection in online surveys (Andrade 2020 ).

Third, our study evaluates happiness at multiple time points. This method acts as a temporal anchor, assisting respondents in recalling and distinguishing their experiences more accurately. Given that respondents often generalize or simplify their experiences when recalling over an extended timeframe, evaluating happiness at different times enables a comprehensive capture of fluctuations and variations in individuals’ emotional states. In this study, assessing happiness at two distinct time points, before and during the COVID-19 pandemic, guarantees a more accurate portrayal of an individual’s SWB and alleviates recall bias (Hyman 2013 ; Tadic et al. 2014 ).

This study consists of six sections. In Section 2, we offer a summary of pertinent prior studies, followed by an investigation into the research questions posed in this study. Section 3 explains the methodologies and models used and outlines the data sources. Section 4 examines and analyzes the outcomes from the estimations, while Section 5 discusses the results. Finally, Section 6 summarizes the findings and offers policy recommendations based on the results.

Literature review and research questions

Theoretical background.

The reactivity theory , embraced by social scientists, including economists and sociologists, asserts that SWB, particularly happiness, is influenced by objective external conditions at both the individual and social levels (Lee 2022 ). These objective conditions encompass various factors such as income, age, gender, marital status, occupation, family structure, geographic region, and government policies (Diener 1984 ). According to the reactivity theory, individuals’ perceptions and assessments of their happiness primarily stem from their passive responses to these objective conditions. In simpler terms, individuals tend to react to the circumstances and external factors surrounding them, significantly impacting their SWB. Within the framework of our study, positive events like economic improvements or technological advancements consistently raise happiness levels. In contrast, adverse events such as natural disasters (Calvo et al. 2015 ; Rehdanz et al. 2015 ; Sekulova and van den Bergh 2016 ) or the COVID-19 pandemic tend to decrease happiness.

The impact of the COVID-19 pandemic on happiness in Indonesia

Before the pandemic, numerous studies in Indonesia explored factors influencing happiness across various scopes. These studies encompassed general population happiness levels (Aryogi and Wulansari 2016 ; Landiyanto et al. 2011 ; Sohn 2013 ; Sujarwoto et al. 2017 ) and specific demographic segments (Anna et al. 2019 on fishermen; Sollis et al. 2023 on native-immigrant). Regional studies (Firmansyah et al. 2017 ; Nandini and Afiatno 2020 ) shed light on context-specific happiness factors. Specific topics like religiosity (Kurniawati and Pierewan 2020 ), height (Sohn 2014 ), decentralization (Sujarwoto and Tampubolon 2015 ), and income inequality (Furwanti et al. 2021 ) were examined, providing valuable insights. Furthermore, Pattinasarany ( 2018 ) conducted a cross-national analysis exploring happiness and life satisfaction determinants in Indonesia, Thailand, Japan, and South Korea.

In both pre-pandemic and pandemic contexts in Indonesia, the World Happiness Report (WHR) and the Happiness Index are commonly used measures of happiness. Footnote 2 However, these two references provide contradictory information regarding the impact of the COVID-19 pandemic on the happiness levels of individuals in Indonesia. The WHR indicates a decrease in the happiness level of Indonesian people from 5.345 from 2018 to 2020 to 5.240 from 2019 to 2021 (Helliwell et al. 2020 ; 2021 ; 2022 ). In contrast, the Happiness Index shows an increase from 70.69 in 2017 to 71.49 in 2021 (Badan Pusat Statistik 2021a ).

Multiple studies have explored the effects of the pandemic on SWB in Indonesia. Tjahjana et al. ( 2021 ) conducted an online survey a month after the pandemic, indicating that 41% of respondents reported decreased happiness. Rahmanita et al. ( 2021 ) collected data 1–3 months post-pandemic, revealing that 59% of respondents expressed happiness in staying at home. Iskandarsyah et al. ( 2022 ) explored the effects of COVID-19 information and behaviors on anxiety and happiness a month post-outbreak, noting increased information searches linked to higher anxiety but more testing and treatment information tied to less anxiety and greater happiness. Dwidienawati et al. ( 2021 ) found ongoing pandemic adaptation challenges, with no improvement in happiness or life satisfaction reported after a year. Halimatussadiah et al. ( 2021 ) conducted two cross-sectional online surveys in 2020 and 2021, revealing a trend towards heightened happiness. In a separate study, Borualogo and Casas ( 2022 ) collected data during the same period, discovering higher SWB and positive affect among boys during the pandemic and improved satisfaction in friend interactions.

The following are overviews of studies using general population survey data to understand the pandemic’s impact on SWB in Indonesia’s neighboring countries. Tambyah et al. ( 2023 ) found a significant decrease in life satisfaction among Singaporeans, dropping from 4.51 in 2016 to 4.18 in 2022 on a scale of 1–6. The study highlighted health risks and job security as primary concerns during the COVID-19 pandemic. Phulkerd et al. ( 2023 ) reported that Thai adults had an average life satisfaction score of 22.4 during the 2021 COVID-19 epidemic, down from 25.5 before the pandemic in 2019 on a 5–35-point scale.

Research questions

This study investigates four specific research questions (RQs) to elucidate and support the study objectives within the broader context of the Indonesian population. Limited research has explored the impact of COVID-19 severity on self-reported happiness at subnational levels due to a lack of reliable data. However, some exception studies exist (Bittmann 2022a ; Le and Nguyen 2021 ). In Indonesia, the impact of the pandemic varies across provinces and districts, each of which implemented unique policies to curb the spread of the pandemic and cope with its consequences (Arifin et al. 2022 ). This study examines a connection between the severity of COVID-19 and self-reported happiness, anticipating that increased severity will correspond to decreased reported happiness.

RQ1: To what extent does the severity of COVID-19 contribute to a reduction in individuals’ happiness levels?

Global research suggests women typically report higher life evaluations than men (Blanchflower and Bryson 2022 ; Blanchflower and Oswald 2011 ; Fortin et al. 2015 ). However, women worldwide bear a disproportionate burden of socio-economic challenges during crises like natural disasters, economic downturns, and pandemics. Such inequity stems from gender roles and undervaluation of women’s work, leading to increased caregiving responsibilities and exposing women to short-term economic instability and long-term welfare declines (Dinella et al. 2023 ; Fortier 2020 ; Langer et al. 2015 ). This study investigates whether the severity of COVID-19 has narrowed the gap in self-reported happiness between women and men.

RQ2: To what extent does the severity of the COVID-19 pandemic lessen women’s happiness advantage over men?

International evidence indicates that, at low levels of economic development, substantial gaps favor urban over rural areas in income, education, and occupational structure, resulting in higher SWB for urban residents than for rural residents. Such higher life satisfaction holds despite urban challenges like pollution and congestion. However, these economic disparities diminish as development progresses, enabling rural areas to close the gap and even surpass urban life satisfaction (Burger et al. 2020 ; Easterlin et al. 2011 ). In Indonesia, Sohn ( 2013 ) identified a positive association between living in urban areas and happiness. Additionally, Sujarwoto ( 2021 ) observed that individuals residing in rural settings expressed lower life satisfaction than their urban counterparts. Given the COVID-19 pandemic’s disproportionate impact on urban areas compared to rural regions, an intriguing query arises: How did the severity of the pandemic influence the link between urban living and self-reported happiness?

RQ3: To what extent does the severity of the COVID-19 pandemic diminish the happiness advantage of urban residents compared to rural residents?

Incorporating contextual variables in measuring self-reported happiness in a multilevel framework is crucial for more accurate analyses and informed policymaking (Ballas and Tranmer 2012 ; Gómez-Balcácer et al. 2023 ). Analytically, incorporating contextual variables like macroeconomic and socio-economic conditions enhances research depth and accuracy. From a policy standpoint, this approach provides a robust foundation for informed decision-making, resulting in more effective and targeted policies. This study utilizes three provincial-level contextual variables: COVID-19 severity (as discussed in RQ1), poverty incidence, and income inequality.

RQ4: To what extent do provincial macroeconomic conditions, specifically poverty and income inequality, impact individuals’ happiness levels?

These research questions delve into diverse facets of the pandemic’s influence on happiness levels within Indonesian society. They examine consequences such as health risks, economic disruptions, and social isolation (RQ1). Furthermore, they investigate the role of societal norms, gender roles, and structural inequalities in women’s experiences during the pandemic (RQ2) and assess potential challenges in urban areas (RQ3). Finally, the study evaluates the impact of macroeconomic factors, specifically poverty and income disparities, on happiness levels during the pandemic (RQ4).

Materials and methods

Multilevel mixed-effects ordered logistic model.

In this study, we estimate a multilevel mixed-effects ordered logistic model that incorporates nesting while considering the dependent variables’ categorical nature and providing adjusted standard errors that add precision to the coefficients (Rabe-Hesketh and Skrondal 2022 ). By using multilevel models, we can control for individual and province variables, isolating the impact of pandemic severity on self-reported happiness levels (Mehmetoglu and Jakobsen 2017 ; Snijders and Bosker 2012 ). Observations in our study comprise individuals (level 1) nested within provinces (level 2). Our multilevel regressions are computed with random intercepts for each province to account for the fact that provinces are affected differently by the pandemic and that respondents in one province might be more similar than respondents in another. Finally, we used an ordered logistic model due to the ordered nature of the dependent variable.

We postulate a latent variable (y*) representing an individual’s underlying happiness. In this study, we will estimate two models: the ‘main’ (hereafter: Main Model) and the ‘with interaction terms’ (hereafter: Interaction Model) models. The Main Model’s latent variable is associated with individual traits, household attributes, and provincial-level contextual variables. Individual traits encompass gender, age along with its squared term, marital status, highest education level attained, and employment status. Household-level attributes include residence type and household income. Three contextual variables at the provincial level consist of the poverty rate, income inequality, and the count of COVID-19-infected individuals per 100,000 population, reflecting COVID-19 severity. In contrast, the Interaction Model encompasses the Main Model and incorporates additional interaction variables between gender and residence-type covariates with the severity of the pandemic measure. Footnote 3 We assume that individuals residing in provinces hardest hit by the pandemic will experience a more significant decline in happiness than those in the less affected provinces.

The Main Model is specified as follows:

while the Interaction Model is specified as follows:

where: \({y}_{{ij}}^{* }\) is the unobserved happiness for individual i who resides in province j (latent variable); \({x1}_{{ij}}\) is the individual and household characteristics for individual i living in province j; \({x2}_{j}\) is the provincial contextual variables for province j; \({{COVID}}_{j}\) is the COVID-19 pandemic severity measure for province j; \({{x3}_{{ij}}* {COVID}}_{j}\) is the interaction terms of gender and type of residence covariates with COVID-19 severity measure; this study assesses three specifications incorporating interaction terms: one specific to women, another specific to urban settings, and a third encompassing both women and urban factors; \({z}_{{ij}}\) is the covariates corresponding to the random effects; as this model follows a random-intercept model, \({z}_{{ij}}\) is simply the scalar 1; \({u}_{j}\) is the random effects; and \({\epsilon }_{{ij}}\) is the errors, distributed as logistic with mean 0 and variance π 2 /3 and are independent of \({u}_{j}\) .

This model, \({x1}_{{ij}}\) and \({x2}_{j}\) do not contain a constant term because its effect is absorbed into the cutpoints (κ).

Table 1 illustrates the estimation strategies employed in this study, encompassing three distinct approaches presented in 12 specifications. First, the Main Model uses all observations to illustrate the relationship between happiness levels and each covariate. Second, the Interaction Model examines how COVID-19 severity affects the connection between being female, living in urban areas, and happiness levels. The second approach investigates moderation effects. Lastly, the third approach delves into the factors impacting happiness across specific subgroups based on gender, residence type, and region. This granular analysis offers insights into potential differences or similarities in the determinants of happiness among these subgroups, aiming to unravel complex relationships among predictors in understanding SWB across diverse contexts.

Model estimation is performed using the meologit procedure in Stata 17.0 (StataCorp 2021 ). The meologit procedure estimates ordered logistic regression containing both fixed effects (in this study: \({x1}_{{ij}}\) and \({x2}_{j}\) along with their interaction terms) and random effects ( \({u}_{j}\) ).

The Happiness Level Measurement Survey (SPTK)

This study relies on the Happiness Level Measurement Survey (SPTK) from 2017 and 2021, administered by the Central Statistics Agency of Indonesia (Badan Pusat Statistik; BPS) (Badan Pusat Statistik 2017 ; 2021a ). Footnote 4 The 2021 wave of SPTK fieldwork took place from July 1 to August 27, 2021, during Indonesia’s peak of the COVID-19 pandemic. The data relating to COVID-19 exposure, i.e., total positive cases of COVID-19, was taken from KawalCOVID19, who collected data primarily from the Ministry of Health. The macroeconomic data on poverty levels and inequality of per capita expenditures (Gini coefficient) are all sourced from the BPS.

SPTK extends across every province and district in Indonesia, where districts consist of kabupaten (regencies) and kota (municipalities). Within each district, the BPS has established a master sampling frame comprising Census Blocks (BS) for the periodic implementation of various surveys. A BS constitutes a designated enumeration zone within a village locality consisting of 80 to 120 residential, non-residential, or household census buildings with distinct boundaries identifiable in the field. BS selection for SPTK is selected probabilistically from the master sampling frame. Household updating takes place at each selected BS, with the selection of household respondents based on updated listings that are stratified according to factors such as the household head’s education and the household’s structure.

The data collection involves conducting direct interviews with respondents utilizing structured questionnaires and computer-assisted personal interviewing applications. Footnote 5 The unit of analysis is a randomly selected household. In each sampled household, the head of the household or the spouse of the head of the household (wife/husband) is selected as the respondent to represent the household. This study focuses on 137,958 respondents aged 25–80 years who are working or spend most of their time taking care of the household. Footnote 6 Apart from the level of happiness, SPTK contributed data at the individual and household levels.

Level of happiness

The level of happiness is evaluated using the so-called Cantril ladder (Cantril 1965 ; Levin and Currie 2014 ). The SPTK employs a ladder diagram to measure happiness, prompting respondents to visualize themselves on a scale with steps numbered from zero at the bottom to ten at the top. Respondents are asked to evaluate their happiness using the question, “How happy are you with life as a whole?” The answer ranges from 0 (very unhappy) to 10 (very happy).

Figure 1 shows that the distributions of happiness are skewed to the left. Most respondents evaluate their happiness on the eighth rung (34.1 percent in 2017 and 35.6 percent in 2022). The national average was calculated at 7.78 in 2017, while for 2021, it will be slightly lower at 7.76.

Source: Calculated from SPTK.

For a comparative analysis of self-reported happiness in this study with neighboring nations, Pattinasarany ( 2018 ) investigated happiness and life satisfaction in Indonesia, Thailand, Japan, and South Korea to compare self-reported happiness with neighboring nations. The study used collected data to explore lifestyles and values related to social well-being in seven Asian countries, including the Philippines, Taiwan, and Vietnam. Results revealed similar happiness distribution, with Indonesia and Thailand displaying a left-skewed pattern, indicating majority contentment. Indonesian adults reported slightly higher average happiness (7.68) than their Thai counterparts (7.65). In Japan (6.25) and Korea (5.93), happiness levels exhibited a more normal distribution, with averages not reaching the same highs as observed in Indonesia and Thailand.

Analyzing happiness at the provincial level indicates that Gorontalo and North Maluku reported the highest average levels in 2017 (8.43) and 2021 (8.54), respectively (Fig. 2 ). In contrast, the lowest averages were recorded in East Nusa Tenggara in 2017 (7.32) and Bali in 2021 (7.26). While the national average in 2017 and 2021 remains relatively unchanged, significant differences emerge at the provincial level between the two years. Providing context, half of the 34 provinces saw an increase in their average happiness levels from 2017 to 2021, while the remaining provinces experienced a decline. Central Sulawesi notably showed the most substantial surge, with an increase of 0.347 points, while Bengkulu province witnessed the most significant decrease, dropping by 0.387 points. Recognizing the nested nature of individuals within provinces, the variance in average happiness levels between years at the provincial level becomes a crucial consideration.

In our examination of gender and residence type on changes in SWB during the pandemic, Fig. 3 illustrates average happiness levels categorized by gender and residence type. The left panel reveals that, on average, women reported higher happiness levels than men. However, there was a slight increase in men’s average happiness during the pandemic (+0.03 points), while women experienced a decrease (−0.06 points). In the right panel, it is evident that individuals residing in urban areas typically demonstrated higher average happiness levels than those in rural settings. Interestingly, individuals in rural areas reported higher happiness levels in 2021 compared to 2017 (+0.08 points). In contrast, those living in urban areas displayed the opposite trend, experiencing a decline in happiness levels over the same period (−0.15 points).

Given the limited number of respondents rating their happiness level between zero and five, these five responses were aggregated to achieve a more balanced distribution. Furthermore, data recoding follows the ordered logistic method, requiring each cell to include at least three percent of observations.

Total COVID-19 cases per 100,000 population

In this study, the evaluation of the severity of the COVID-19 pandemic relies on the total population with confirmed exposure to COVID-19. Although daily data has been available since March 2, 2020, the SPTK data lacks specific interview date information. A cut-off point, set on June 30, 2021, was established to determine COVID-19 severity for all survey respondents, conveniently aligning with the day preceding the start of SPTK face-to-face interviews. We used a normalization process to enable meaningful province-to-province comparisons, specifically normalizing the data per 100,000 population.

Figure 4 illustrates the unequal distribution of confirmed COVID-19 cases among provinces. DKI Jakarta records the highest incidence of COVID-19 cases, reaching 5210 per 100,000 population. Conversely, North Sumatera reports the lowest number of cases, only 246 per 100,000 population. These findings underscore the diverse impact and transmission rates of COVID-19 observed across different provinces.

Source: Calculated from KawalCOVID-19.

Concluding the data discussion, Table 2 displays the mean and standard deviation of all variables used in this study, categorized by year.

Estimation results

Table 3 displays happiness level estimates from a multilevel mixed-effects ordered logistic analysis covering the Main and Interaction Models. The Main Model serves as the baseline, while the Interaction Model estimates examine potential changes in gender and type of residence covariates influenced by the COVID-19 pandemic.

We begin by discussing the results of the Null Model, which incorporates no predictors (Table 3 , column [1]). The Intraclass Correlation Coefficient (ICC) for the Null Model is 0.038 (second row from the bottom), indicating that approximately 3.8 percent of the variability in an underlying response is associated with differences between provinces. Footnote 7 Sommet and Morselli ( 2017 ) noted that many authors argue that an ICC below 5 percent, considered insignificant and negligible, leads them to treat the individual as a single unit of analysis, hence opting for a single-level analysis. Nevertheless, we persist with multilevel modeling, recognizing that the minimal ICC (except when zero) does not signify the absence of variation in respondents’ happiness levels between provinces. Moreover, disregarding this variation can lead to inaccurate estimates and potentially result in inappropriate policy decisions. The ICCs for the Main and Interaction Models are modest, ranging between 0.037 and 0.041.

The Likelihood Ratio (LR) test, located in the third row from the bottom, compares the multilevel mixed-effects ordered logistic model with the standard (single-level) ordered logistic model, favoring the former. A p-value of 0.000 for the LR test signifies significant variation in self-reported happiness levels between provinces. The “Variances: Province (constant)” estimates in the fourth row from the bottom indicate the variation in self-reported happiness levels attributed to differences between provinces after accounting for fixed effects and other covariates in the model. This information clarifies how the province-level factor (in our case, poverty rates, Gini coefficient of per capita expenditures, and severity of the pandemic measure) contributes to the overall variability in the outcome. A higher estimated variance suggests a more significant variation in the outcome between provinces.

The severity of the COVID-19 pandemic

The estimation results indicate that individuals in provinces with more COVID-19 cases per 100,000 population tended to assign lower ratings to their happiness (Table 3 , column [2]). Footnote 8 Our findings align with international research. A study across China, Japan, South Korea, Italy, the United Kingdom, and the United States found that individuals in areas with elevated COVID-19 rates are more likely to report lower happiness levels (Nguyen 2021 ). Similarly, a German study using panel data during the initial COVID-19 wave observed a decline in life satisfaction in regions with higher infection rates (Bittmann 2022a ).

Concerns about the robustness of conclusions drawn from estimations using the entire dataset when examining specific characteristics are typical. Table 4 provides Main Model estimates disaggregated by gender (assessing whether estimation results differ for male or female respondents), type of residence (rural versus urban), and major regions in Indonesia (Sumatera, Java-Bali, and Other regions). Table 5 facilitates a comparison of the three primary correlates: gender (women), residence type (urban), and the severity of the COVID-19 pandemic.

These findings indicate that the detrimental impact of the pandemic’s severity on happiness levels is observable for both men and women, as well as for residents in rural areas and the Java-Bali and Other regions of Indonesia. However, the absence of statistical significance for urban residents may be attributed to the predominant concentration of the COVID-19 pandemic in urban areas of Indonesia. Similarly, the lack of statistical significance for the Sumatera region is associated with the lower pandemic severity observed in that region. Despite variations across different samples, these consistent findings underscore the negative association between the severity of the COVID-19 pandemic and individuals’ happiness levels.

In Indonesia, on average, women reported higher happiness levels than men (Table 3 , column [2]). Upon analyzing a disaggregated sample by residence type, the results indicate that women exhibit higher happiness levels than men in both rural and urban areas (Table 4 , columns [8] and [9]). Moreover, women consistently report higher happiness levels than men across all three regions (Sumatera, Java, and others) (Table 4 , columns [10], [11], and [12]).

A noteworthy observation is the degree to which women in the Java-Bali region experience a smaller happiness advantage over men compared to their counterparts in Sumatera and other regions. One potential explanation is the Java-Bali region’s reputation for embracing a more egalitarian gender culture than other parts of Indonesia, suggesting that gender-based disparities in happiness might be comparatively smaller in the Java-Bali region than in other regions (Hayati et al. 2014 ; Utomo 2012 ). Moreover, the Java-Bali region’s higher level of development compared to other parts of Indonesia contributes to enhanced gender equality across various facets, including well-being and happiness.

The Interaction Model estimates reveal that in 2021, the severity of the pandemic led to a decline in women’s happiness relative to men’s (Table 3 , columns [3] and [5]). These results indicate that the pandemic’s effect diminishes the relative advantage of being female in terms of happiness levels. Our findings align with several studies (Blanchflower and Bryson 2022 ; Nguyen 2021 ), all reporting a decrease in women’s life satisfaction and happiness compared to men during the pandemic.

Type of residence

Individuals residing in urban areas generally experience higher levels of happiness than their rural counterparts (Table 3 , column [2]). Easterlin et al. ( 2011 ) provided a comprehensive explanation for such findings, highlighting that the availability of material goods like food, clothing, and shelter in urban areas contributes to higher happiness. However, they also caution that urban life comes with challenges, including traffic congestion, pollution, and feelings of alienation, which can negatively impact happiness.

The difference in happiness levels between urban and rural residents remains consistent across diverse demographics (Table 4 , columns [6], [7], [10], [11], and [12]). Particularly noteworthy is the narrower happiness gap between urban and rural residents in the Java-Bali region (Table 4 , column [11]), indicating that rural areas in Java-Bali may benefit from enhanced public services and infrastructure compared to other regions. This improved availability of resources in rural Java-Bali contributes to a more equitable distribution of opportunities and resources between urban and rural residents.

Nevertheless, as per the Interaction Model, the pandemic’s severity has weakened the traditional happiness advantage of individuals in urban areas compared to their rural counterparts (Table 2 , columns [4] and [5]). Our observation finds backing in urban Pakistan, where Shams and Kadow (2020) documented a decrease in socio-economic satisfaction amid the pandemic, particularly noticeable among unemployed individuals, married couples, men, and older demographics.

Contextual characteristics

The association between poverty levels and happiness lacked statistical significance, suggesting that the poverty rates in a respondent’s province do not influence their happiness. One possible explanation is the substantial variation in poverty rates among districts within a province. For example, in 2021, East Java Province exhibited a poverty rate of 11.4 percent, yet the rates across its 38 kabupaten / kota ranged from 4.1 to 23.8 percent (Badan Pusat Statistik 2021b ). Nevertheless, a deviation from the typical trend is evident in the Java-Bali region, exposing a negative correlation between higher poverty levels and happiness among respondents (Table 4 , column [11]). This finding aligns with the higher poverty population in the Java-Bali region compared other regions in Indonesia (Badan Pusat Statistik 2021b ).

Muthia and Isbah’s ( 2022 ) study sheds light on the lack of a correlation between poverty and happiness, particularly within the impoverished community of DI Yogyakarta Province, Indonesia. The authors argue that impoverished individuals may not find happiness in their economic situation but discover contentment. This occurrence is ascribed to the prevailing belief system and local culture, heavily influenced by the nerimo attitude, emphasizing the acceptance of one’s circumstances. By adopting this mindset, impoverished individuals improve their psychological well-being, regardless of their difficulties.

Regarding inequality, the estimation results reveal an inverse connection between per capita expenditure inequality at the provincial level and self-reported happiness levels. In another study, Furwanti et al. ( 2021 ) utilized cross-sectional data from all Indonesian provinces and a path analysis model, revealing that income inequality significantly and negatively influences happiness in Indonesia.

The findings of this study align with several international reviews exploring the relationship between inequality and happiness. For instance, a review by Ferrer-i-Carbonell and Ramos ( 2014 ) demonstrates a negative correlation between income inequality and happiness in Western countries. However, the connection in non-Western countries is diverse and less conclusive. In addition, Schroder ( 2018 ) discovered that individuals perceive their SWB as lower when inequality within their own country increases over time, but not when it is higher compared to another country.

Individual characteristics

Following is a concise discussion of individual characteristics that fall outside the scope of the four research questions outlined in this study.

Our model incorporates respondents’ age in quadratic terms, revealing a U-shaped pattern in happiness assessment (Easterlin 2004 ; Blanchflower 2021 ; Bittmann 2022b ; Toshkov 2022 ). Generally, happiness levels decline with age until reaching a certain point, after which they begin to rise. In the Main Model, this turning point is identified at 49. The U-shaped pattern corresponds to the “midlife dip” phenomenon, wherein individuals often undergo a decline in happiness during midlife before it subsequently increases later in life, as discussed by Blanchflower and Graham ( 2020 ). Factors such as heightened responsibilities, financial pressures, and changes in personal and professional circumstances can influence this midlife dip.

Individuals in a marital union tend to experience higher happiness levels than unmarried or divorced individuals. This observation is supported by Frey’s ( 2018 ) comprehensive review, affirming that married individuals generally express higher happiness levels than those living alone or in unmarried partnerships. The author highlights the role of marriage or a stable partnership in mitigating loneliness, thereby assisting in alleviating stress related to work life. Various studies (Addai et al. 2014 ; Tambyah et al. 2023 ; Wu and Zhu 2016 ) have also identified the positive influence of being in a marital relationship.

A positive correlation is evident between education and happiness. This finding indicates that higher educational attainment aligns with higher self-reported happiness levels. As noted by Frey ( 2018 ), individuals with advanced education tend to enhance their abilities and gain increased access to opportunities, resulting in heightened life satisfaction. The association between education and happiness has been thoroughly examined, including within Indonesia (Landiyanto et al. 2011 ; Sujarwoto and Tampubolon 2015 ; Rahayu 2016 ). These investigations consistently affirm a positive association between education and happiness within the Indonesian context.

In general, employed respondents report lower happiness levels, although differences exist between men and women. Among male respondents, those actively engaged in work display higher happiness levels than those who are not. This positive correlation between working and happiness among men corresponds with findings from various international studies (Clark and Oswald 1994 ; Di Tella et al. 2001 ; Winkelmann and Winkelmann 1998 ). Conversely, employed individuals report lower happiness within the female sample than those unemployed. To the extent that the SPTK dataset defines those not employed as spending most of their time taking care of the household, the negative association between employment and happiness among women can be interpreted as women who are employed facing a double burden of responsibilities at work and home (Chen et al. 2018 ).

Individuals reporting higher household earnings exhibit higher happiness levels. However, the ongoing debate on whether income contributes to increased happiness encompasses diverse viewpoints. Some studies advocate for a positive correlation between income and self-reported happiness and, therefore, in line with our findings (Diener and Biswas-Diener 2002 ; Frey and Stutzer 2002 ; Lim et al. 2020 ; Yiengprugsawan et al. 2011 ; Yu et al. 2019 ). Conversely, other studies propose that the impact of income on happiness becomes negligible once a certain income threshold is reached (Kahneman and Deaton 2010 ; Muresan et al. 2020 ).

Discussions

Our analysis reveals a significant decline in self-reported happiness among Indonesians due to the severity of the COVID-19 pandemic, addressing RQ1. The pandemic severity measure has eroded the longstanding happiness advantage for women and urban residents, addressing RQ2 and RQ3. A concerning negative correlation between income inequality and happiness is evident, addressing RQ4. These findings emphasize the urgent need for targeted interventions to mitigate these effects on the Indonesian populace’s well-being.

COVID-19 severity reduces happiness

The decrease in self-reported happiness among Indonesians amid the severity of the COVID-19 pandemic arises from various factors. First, increased vulnerability to COVID-19 elevates health apprehensions and anxiety, giving rise to concerns about the risk of infection for both oneself and loved ones. Consequently, this anxiety diminishes overall well-being (Cleofas and Oducado 2022 ; Demirbas and Kutlu 2021 ; van der Vegt and Kleinberg 2020 ). Second, provinces with higher COVID-19 cases face significant economic disruptions, including business closures, job losses, and reduced economic activity, resulting in financial stress, insecurity, and an overall happiness decline (Cheng et al. 2020 ; Greyling et al. 2021 ; Kuhn et al. 2020 ). Third, residents in heavily affected provinces may encounter challenges such as limited social support networks, reduced opportunities for social engagement, and feelings of loneliness or disconnection, significantly impacting their happiness levels (Lepinteur et al. 2022 ; Nguyen 2021 ). Lastly, the increased prevalence of anxiety, depression, or emotional distress among individuals in provinces with higher COVID-19 exposure further contributes to lower self-reported happiness levels (Iskandarsyah et al. 2022 ).

This study underscores the assessment of the COVID-19 pandemic’s impact on individuals’ happiness, specifically through a severity measure focusing on the number of affected individuals per 100,000 population. This choice differs from using time dummy variables, assigning 1 for 2021 survey data (during the pandemic) and 0 for 2017 survey data (pre-pandemic). The severity measure directly reflects the impact of the COVID-19 pandemic on the population, offering a tangible and quantifiable indicator of its scale within a region. This approach is especially appropriate given the considerable variation in pandemic severity across provinces in Indonesia. Nevertheless, we recognize that relying solely on the severity measure may oversimplify the complex dynamics of the pandemic’s impact. Furthermore, Bittmann, ( 2022a ) explores the functional relationship between the severity measure and self-reported happiness, considering alternatives such as linearity (as employed in this paper), quadratic, and others. This exploration opens up possibilities for future studies.

COVID-19 severity moderates gender-residence type association with happiness

The negative and statistically significant interaction terms between COVID-19 severity and gender (being female) indicate that the pandemic’s severity affects the relationship between gender and self-reported happiness. In periods of intensified pandemic severity, the conventional gender gap in happiness, where women usually report higher levels, is disturbed. The negative moderation implies that the pandemic has a more detrimental impact on women’s happiness levels than men.

Research conducted by Alon et al. ( 2020 ), Blanchflower and Bryson ( 2022 ), and Hansen et al. ( 2022 ) underscore that the decline in happiness levels among women can be attributed to heightened caregiving responsibilities, especially as primary caregivers for children. Transitioning to remote learning for children has introduced additional challenges and demands for women. Additionally, as frontline workers, women face elevated stress levels in their roles and are vulnerable to potential job layoffs and disruptions in their participation in the labor market. Conversely, a study by Choi et al. ( 2021 ) concluded that even before the onset of COVID-19, Korean women demonstrated lower levels of SWB compared to men. Therefore, the well-being disparities observed among Korean women are more likely rooted in pre-pandemic variations rather than directly caused by the effects of the pandemic.

Similarly, the adverse and statistically significant interaction terms between COVID-19 severity and residence type (urban) indicate that the severity of the pandemic influences the connection between living in urban areas and self-reported happiness. During periods of heightened pandemic severity, the typical gap in happiness based on residence type, where individuals in urban areas usually report higher levels, ceased to hold. This adverse moderation implies that the pandemic has a more harmful effect on the happiness levels of individuals in urban residences than those in rural areas.

Mayuzumi’s ( 2022 ) research provides valuable insights into the effects of the COVID-19 pandemic on the happiness of urban and rural communities in Bali, Indonesia. The results indicate that individuals in subsistence farming villages, heavily dependent on agriculture, witnessed minimal changes in their livelihoods, suggesting little impact from the pandemic. In contrast, urban residents, primarily reliant on tourism, experienced significant job losses and food accessibility challenges due to government curfews and economic stagnation. On the contrary, Nguyen ( 2021 ) introduces an alternative perspective by proposing that the pandemic has a more noticeable impact on the unhappiness levels of individuals residing in rural areas than those living in urban settings.

Inequality is a catalyst for diminishing happiness

Examining contextual characteristics unveils that, excluding the Java-Bali region, provincial poverty levels have negligible effects on happiness levels. Nonetheless, there is a discernible negative correlation between inequality in per capita expenditure and happiness.

An important observation from the analysis using region-specific breakdowns is the unexpected positive association between the Gini coefficient and happiness in the Sumatera region. The uniqueness of this result in Sumatera may be ascribed to distinct factors inherent to the region, such as particular social structures, values, or expectations. These regional peculiarities in Sumatra could influence individuals’ perspectives on happiness differently than in other locales. A more thorough investigation into the specific factors contributing to these anomalies across regions is necessary to grasp the patterns observed fully.

Study limitations

The research employed a single-question methodology using a 0–10 point Likert scale to assess individual happiness. Although this approach offers a valuable metric, we acknowledged that happiness is a complex concept with multiple dimensions that a single question may need to be more comprehensive. Consequently, the study recognizes the importance of incorporating additional aspects and nuances to understand better individuals’ well-being, including factors like self-evaluated life satisfaction, positive affect, and negative affect.

Moreover, it is essential to consider two significant data constraints when interpreting the findings. First, the SPTK datasets utilized in the study lack precise location information, restricting the analysis to the provincial level and hindering a more detailed examination of the impact of COVID-19 on specific regions or communities within a province. For instance, while information on the poverty rate is accessible at the district level, the unavailability of district codes necessitates using provincial poverty rates.

Second, the datasets do not incorporate information about the interview dates for respondents, which would have facilitated a more precise correlation with the daily severity rate of COVID-19 at the provincial level. Access to interview date information could have offered valuable insights into the temporal relationship between individuals’ experiences and the evolving severity of the pandemic in their respective provinces.

The global repercussions of COVID-19 on individuals’ lives and well-being are profound. In Indonesia, there is a pressing need for more research on the correlation between happiness and pandemic severity across the population. This study addresses this gap by examining the factors influencing happiness levels before and during the pandemic, specifically focusing on gender and residence type. By posing and answering four research questions (RQs), the study provides valuable insights into the intricate dynamics of happiness during the pandemic in Indonesia.

This study employed data from the 2017 and 2021 Happiness Level Measurement Survey (SPTK) to represent pre-pandemic and during-pandemic conditions, respectively. The data analysis involved using a multilevel mixed-effects ordered logistic model, with individuals nested within provinces as the analytical framework. The severity of the pandemic was proxied using the incidence of positive COVID-19 cases per 100,000 residents.

Our analysis underscores a statistically significant decline in self-reported happiness levels among Indonesians attributable to the severity of the COVID-19 pandemic, directly addressing RQ1. Notably, this severity measure has diminished the longstanding happiness advantage previously experienced by women and urban residents, aligning with the inquiries of RQ2 and RQ3. Additionally, our study highlights a negative correlation between income inequality and happiness, illuminating the intricate interplay of socioeconomic dynamics influencing individual well-being as per RQ4. The robust support for our research questions highlights the multifaceted impact of the pandemic on happiness levels in Indonesia.

Immediate policy interventions are required to tackle these findings, encompassing targeted mental health support to aid individuals in overcoming the challenges of lockdown restrictions and the loss of loved ones; economic assistance to support families facing sudden job loss and economic downturn; reinforced public health initiatives to curb the spread of the virus and mitigate the health impact of the pandemic; educational campaigns to inform the public about necessary health protocols; and community-based social support programs to lighten the overall burden faced by communities in dealing with the pandemic. These measures aim to alleviate the negative impact of the pandemic and socioeconomic disparities on the happiness and overall welfare of the Indonesian population.

In light of the adverse effects of COVID-19 on the happiness of women and urban residents, it is important to implement proactive government programs and policies. To address women’s heightened responsibilities, especially in home-based teaching, effective communication, and support between teachers and students, such as regular home visits, are essential. Providing physical visits and care for vulnerable populations, including the elderly, chronically ill, and disabled individuals, can help alleviate some of the burdens on women. Additionally, supporting urban residents involves reinforcing community associations, particularly within neighborhood and religious networks, through collaborative efforts between the Central Government and local administrations.

The future research agenda aims to enhance the comprehensiveness of this study by incorporating field visits that include in-depth interviews and focus group discussions. Validating the findings, gaining deeper insights into individual experiences amidst the challenges posed by COVID-19, and investigating the impact of government assistance are deemed crucial. Complementing the measurement of SWB by incorporating self-evaluated life satisfaction, positive affect, and negative affect will improve our knowledge of the well-being of Indonesians. Furthermore, expanding the study by incorporating subsequent SPTK data will allow for assessing happiness before, during, and after the pandemic.

Data availability

The primary datasets analyzed in this study, the Happiness Level Measurement Survey (SPTK) 2017 and 2021, are not accessible to the public. The author is contractually prohibited from granting access to the SPTK data, as specified in the agreement with the Badan Pusat Statistik (BPS). However, the datasets are available for purchase through the BPS ( https://www.bps.go.id/ ).

The BPS defines an urban area by its primary non-agricultural activities, a functional layout that accommodates urban settlements, and the concentration and distribution of government services, social services, and economic activities. In contrast, rural areas primarily involve agricultural activities, including managing natural resources, and have a functional arrangement that supports rural settlements, government services, social services, and economic activities. In 2022, the urban areas of Indonesia were home to 56.4 percent of the population, while 43.6 percent lived in rural areas.

The WHR, an annual report comparing happiness levels across countries, relies on three well-being indicators: life evaluation, positive affect, and negative affect (Helliwell et al. 2020 ). The Happiness Index, developed by the Central Statistics Agency of Indonesia (Badan Pusat Statistik; BPS), incorporates nineteen indicators that assess dimensions such as life satisfaction, affection, and the meaning of life ( eudaimonia ) (Badan Pusat Statistik 2021a ). It is important to acknowledge that these two measures evaluate distinct aspects. Hence, direct comparison between them is inappropriate, given their representation of separate entities.

These interaction terms capture the moderating effect of the severity of the COVID-19 pandemic on the relationships of interest.

The SPTK is cross-sectional and was conducted in 2012, 2013, 2014, 2017, and 2021. The SPTK has undergone conceptual and methodological improvements (Badan Pusat Statistik 2021a ). For comparability purposes, we will use the last two batches. We need to emphasize that the 2021 SPTK does not aim to study the pandemic’s effect on the happiness level.

Amid the COVID-19 pandemic, the 2021 SPTK data collection encountered many hurdles (Badan Pusat Statistik 2021a ). Originally scheduled for July 1–31, 2021, the fieldwork encountered setbacks due to local lockdowns and the emergence of the Delta variant. Consequently, the 2021 SPTK initiatives necessitated a two-phase extension, extending field activities to two months. Field enumerators grappled with significant challenges, especially in conducting face-to-face surveys amidst stringent health protocols. Setbacks were further compounded as certain respondents refrained from participation due to concerns about infection and the extent to which the virus infected some enumerators. Additionally, due to lockdown restrictions, some survey locations had to be substituted following a month-long delay.

This study includes 67,450 participants from the SPTK 2017 dataset and 70,508 from the SPTK 2021 dataset.

The ICC (Intra-Class Correlation) scale spans from 0 to 1. An ICC value of 0 signifies complete independence of residuals, indicating that the assessment of happiness by individuals does not differ across provinces. Conversely, an ICC value of 1 indicates perfect interdependence of residuals, suggesting that variations in individual happiness levels occur exclusively between provinces.

We also conducted a comparable analysis using the overall count of COVID-19-related deaths to indicate the pandemic’s severity. The results reflected similar patterns: Individuals residing in provinces with higher COVID-19 death tolls generally reported lower levels of happiness. Nevertheless, we opted to omit these findings from our report due to the intricacies associated with attributing a death specifically to COVID-19. Determining the precise cause of death poses challenges, as some individuals might have succumbed to the disease while others had concurrent comorbidities. Consequently, this indicator may be susceptible to inaccuracies, making it a relatively less reliable measure (Bittmann 2022a ).

Addai I, Opoku-Agyeman C, Amanfu SK (2014) Exploring Predictors of Subjective Well-Being in Ghana: A Micro-Level Study. J Happiness Stud 15(4):869–90. https://doi.org/10.1007/s10902-013-9454-7

Article   Google Scholar  

Ahmadiani M, Ferreira S, Kessler J (2022) What Makes People Happy? Evidence from International Data. J Happiness Stud 23:2083–2111. https://doi.org/10.1007/s10902-021-00478-y

Alon TM, Doepke M, Olmstead-Rumsey J, Tertilt M (2020) The impact of COVID 19 on gender equality. National Bureau of Economic Research. https://doi.org/10.3386/w26947

Andrade C (2020) The Limitations of Online Surveys. Indian J Psychological Med 42(6):575–576. https://doi.org/10.1177/0253717620957496

Anna Z, Yusuf AA, Alisjahbana AS, Ghina AA, Rahma (2019) Are fishermen happier? Evidence from a large-scale subjective well-being survey in a lower-middle-income country. Mar Policy 106:103559. https://doi.org/10.1016/j.marpol.2019.103559

Arifin B, Anas T, Wicaksono E, Maududy I, Kusumaningsih N (2022) Ensuring Indonesians Are Safe from COVID-19. In: Indrawati SM, Nazara S, Anas T, Ananda CF, Verico K (eds) Keeping Indonesia Safe from the COVID-19 Pandemic: Lessons Learnt from the National Economic Recovery Programme. ISEAS Publishing and Ministry of Finance of the Republic of Indonesia, Jakarta, p 27–63

Chapter   Google Scholar  

Aryogi I, Wulansari D (2016) Subjective Well-being Individu dalam Rumah Tangga Di Indonesia (Subjective Well-being of Individuals in Households in Indonesia). J Ilmu Ekon Terap 1(1):1–10. https://doi.org/10.20473/jiet.v1i1.1900

Badan Pusat Statistik (2021a) Indeks Kebahagiaan 2021 (Happiness Index 2021). Badan Pusat Statistik, Jakarta

Google Scholar  

Badan Pusat Statistik (2021b) Data dan Informasi Kemiskinan Kabupaten/Kota 2021 (Regency/Municipality Poverty Data and Information 2021). Badan Pusat Statistik, Jakarta

Badan Pusat Statistik (2017) Indeks Kebahagiaan 2017 (Happiness Index 2021). Badan Pusat Statistik, Jakarta

Ballas D, Tranmer M (2012) Happy people or happy places? A multilevel modeling approach to the analysis of happiness and well-being. Int Regional Sci Rev 35(1):70–102. https://doi.org/10.1177/0160017611403737

Bittmann F (2022a) Is there a dose-response relationship? Investigating the functional form between COVID-19 incidence rates and life satisfaction in a multilevel framework. J Happiness Stud 23(7):3315–3330. https://doi.org/10.1007/s10902-022-00542-1

Article   PubMed   PubMed Central   Google Scholar  

Bittmann F (2022b) Beyond the U‑Shape: Mapping the Functional Form Between Age and Life Satisfaction for 81 Countries Utilizing a Cluster Procedure. J Happiness Stud 22:2343–2359. https://doi.org/10.1007/s10902-020-00316-7

Blanchflower DG (2021) Is happiness U-shaped everywhere? Age and subjective well-being in 145 countries. J Popul Econ 34:575–624. https://doi.org/10.1007/s00148-020-00797-z

Article   PubMed   Google Scholar  

Blanchflower DG, Bryson AJ (2022) The Female Happiness Paradox. National Bureau of Economic Research. https://doi.org/10.3386/w29893

Blanchflower DG, Graham C (2020) The Mid-Life Dip in Well-Being: Economists (Who Find It) Versus Psychologists (Who Don’t)! National Bureau of Economic Research. https://doi.org/10.3386/w26888

Blanchflower DG, Oswald AJ (2011) International happiness: A new view on the measure of performance. Acad Manag Perspect 25(1):6–22. http://www.jstor.org/stable/23045031

Borualogo IS, Casas F (2022) Subjective well-being of children and adolescents during the COVID-19 pandemic in Indonesia: two data collections. Curr Psychol https://doi.org/10.1007/s12144-022-03346-x

Burger MJ, Morrison PS, Hendriks M, Hoogerbrugge MM (2020) Urban-Rural Happiness Differentials across the World. In: Helliwell JF, Layard R, Sachs JD, De Neve J-E (eds) World Happiness Report 2020. Sustainable Development Solutions Network, New York, p 67–93

Calvo R, Arcaya M, Baum CF, Lowe SR, Waters MC (2015) Happily ever after? Pre-and-post disaster determinants of happiness among survivors of hurricane Katrina. J Happiness Stud 16(2):427–442. https://doi.org/10.1007/s10902-014-9516-5

Cantril H (1965) Patterns of Human Concerns. Rutgers University Press, New Brunswick

Chen F, Bao L, Lin Z, Zimmer Z, Gultiano S, Borja JB (2018) Double burden for women in mid- and later life: evidence from time-use profiles in Cebu, the Philippines. Ageing Soc 38(11):2325–2355. https://doi.org/10.1017/s0144686x17000599

Cheng TC, Kim S, Koh, K (2020) The Impact of COVID-19 on Subjective Well-Being: Evidence from Singapore. IZA–Institute of Labor Economics

Choi I, Kim JH, Kim N, Choi E, Choi J, Suk HW, Na J (2021) How COVID-19 affected mental well-being: An 11-week trajectories of daily well-being of Koreans amidst COVID-19 by age, gender and region. PLOS One 16(4):e0250252. https://doi.org/10.1371/journal.pone.0250252

Article   CAS   PubMed   PubMed Central   Google Scholar  

Clark AE, Oswald AJ (1994) Unhappiness and Unemployment. Econ J 104(424):648–659. https://doi.org/10.2307/2234639

Cleofas JV, Oducado RMF (2022) COVID-19 Death Occurrences, Pandemic Fatigue, and Well-Being. J Loss Trauma 27(7):679–682. https://doi.org/10.1080/15325024.2021.1971423

Dang H-AH, Nguyen CV (2020) Gender Inequality during the COVID-19 Pandemic: Income, Expenditure, Savings, and Job Loss. World Dev 140:105296. https://doi.org/10.1016/j.worlddev.2020.105296

Deaton A, Stone AA (2013) Two Happiness Puzzles. Am Econ Rev 103(3):591–597. https://doi.org/10.1257/aer.103.3.591

Delhey J, Kroll C (2013) A “Happiness Test” for the New Measures of National Well-Being: How Much Better than GDP are They? In: Brockmann H, Delhey J (eds) Human Happiness and the Pursuit of Maximization. Happiness Studies Book Series. Springer, Dordrecht, p. 191–210. https://doi.org/10.1007/978-94-007-6609-9_14

Demirbas N, Kutlu R (2021) Effects of COVID-19 Fear on Society’s Quality of Life. Int J Ment Health Addiction 20:2813–2822. https://doi.org/10.1007/s11469-021-00550-x

Diener E (1984) Subjective Well-Being. Psychol Bull 95(3):542–575

Article   CAS   PubMed   Google Scholar  

Diener E, Biswas-Diener R (2002) Will money increase subjective well-being? A literature review and guide to needed research. Soc Indic Res 95(2):543–575. https://doi.org/10.1023/A:1014411319119

Dinella LM, Evans K, Levinson JA, Gagnon S (2023) Women disproportionately shoulder burdens imposed by the global COVID‐19 pandemic. J Soc Issues 79(3):1057–1087. https://doi.org/10.1111/josi.12591

Di Tella R, MacCulloch RJ, Oswald A (2001) Preferences over Inflation and Unemployment: Evidence from Surveys of Happiness. Am Econ Rev 91(1):335–341

Dwidienawati D, Tjahjana D, Gandasari D, Faisal M (2021) Happiness and Satisfaction after 1 year of the COVID-19 Pandemic. J Southwest Jiaotong Univ 56(2):111–123. https://doi.org/10.35741/issn.0258-2724.56.2.10

Easterlin RA (2004) The economics of happiness. Daedalus 133(2):26–32

Easterlin RA, Angelescu L, Zweig JS (2011) The Impact of Modern Economic Growth on Urban–Rural Differences in Subjective Well-Being. World Dev 39(12):2187–2198. https://doi.org/10.1016/j.worlddev.2011.04.015

Ferrer-i-Carbonell A, Ramos X (2014) Inequality and Happiness. J Econ Surv 28(5):1016–1027. https://doi.org/10.1111/joes.12049

Firmansyah, Purwanti EY, Susanto H, Oktavilia S, Makhluf HM (2017) Economic of Happiness in Wonosobo Regency, Indonesia. Man India 97(5):289–298

Fortier N (2020) COVID-19, gender inequality, and the responsibility of the state. Int J Wellbeing 10(3):77–93. https://doi.org/10.5502/ijw.v10i3.1305

Frey BS (2018) Economics of Happiness. SpringerBriefs in Economics. Springer, International Publishing, Cham, 10.1007/978-3-319-75807-7

Book   Google Scholar  

Frey BS, Stutzer A (2002) What can economists learn from happiness research? J Econ Lit 40(2):402–435

Fortin N, Helliwell JF, Wang S (2015) How does subjective well-being vary around the world by gender and age. In: Helliwell JF, Layard R, Sachs J (eds) World Happiness Report 2015. Sustainable Development Solutions Network, New York, p 42–74

Furwanti R, Lestari DM, Muflikha M, Wibowo MG (2021) Determinant of Macro-Economics: Does Income Inequality Influence Happiness? Evidence From Indonesia. JEJAK 14(1):146–156. https://doi.org/10.15294/jejak.v14i1.28278

Gausman J, Langer A (2020) Sex and Gender Disparities in the COVID-19 Pandemic. J Womens Health 29(4):465–466. https://doi.org/10.1089/jwh.2020.8472

Giurge LM, Whillans AV, Yemiscigil A (2021) A multicountry perspective on gender differences in time use during COVID-19. Proc Natl Acad Sci 118(12). https://doi.org/10.1073/pnas.2018494118

Gómez-Balcácer L, Arechavala NS, Gómez-Costilla P (2023) The importance of different forms of social capital for happiness in Europe: A multilevel structural equation model (GSEM). Appl Res Qual Life 18(1):601–624. https://doi.org/10.1007/s11482-022-10097-1

Greyling T, Rossouw S, Adhikari T (2021) The good, the bad and the ugly of lockdowns during Covid-19. PLOS One 16(1):e0245546. https://doi.org/10.1371/journal.pone.0245546

Halimatussadiah A, Siregar CH, Bintara H, Rezki JF, Putri LAM, Aufari MR, Sholihah NK, Fadilla RR, Al Kautsar W (2021) Buku Statistik Survei Dampak Pandemi COVID 19 (Statistical Book of Survey of the Impact of the COVID-19 Pandemic). Institute for Economic and Social Research, Faculty of Economics and Business, Universitas Indonesia (LPEM-FEB UI), Jakarta, https://www.lpem.org/wp-content/uploads/2021/12/BUKU_STATISTIK_v8_opt.pdf

Hansen B, Sabia JJ, Schaller J (2022) Schools, job flexibility and married women’s labor supply: evidence from the COVID-19 pandemic. National Bureau of Economic Research. http://www.nber.org/papers/w29660

Hayati EN, Emmelin M, Eriksson M (2014) “We no longer live in the old days”: a qualitative study on the role of masculinity and religion for men’s views on violence within marriage in rural Java, Indonesia. BMC Womens Health 14(1). https://doi.org/10.1186/1472-6874-14-58

Helliwell JF, Layard R, Sachs JD, De Neve J-E, Aknin LB, Wang S (eds) (2022) World Happiness Report 2022. Sustainable Development Solutions Network, New York

Helliwell JF, Layard R, Sachs JD, De Neve J-E (eds) (2021) World Happiness Report 2021. Sustainable Development Solutions Network, New York

Helliwell JF, Layard R, Sachs JD, De Neve J-E (eds) (2020) World Happiness Report 2020. Sustainable Development Solutions Network, New York

Hyman L (2013) Happiness and Memory: Some Sociological Reflections. Sociological Res Online 19(2):1–9. https://doi.org/10.5153/sro.3268

Iskandarsyah A, Yudiana W, Shabrina A, Passchier J (2022) Perception of information about COVID-19 and protective behaviours in relation to feelings of anxiety and happiness. Int J Public Health Sci. 11(1):8–19. https://doi.org/10.11591/ijphs.v11i1.21018

Kahneman D, Deaton A (2010) High income improves evaluation of life but not emotional well-being. Proc Natl Acad Sci 107(38):16489–16493. https://doi.org/10.1073/pnas.1011492107

Article   ADS   PubMed   PubMed Central   Google Scholar  

Kuhn U, Klaas HS, Antal E, Dasoki N, Lebert F, Lipps O, Monsch G-A, Refle J-E, Ryser V-A, Tillmann R, Voorpostel M (2020) Who is most affected by the Corona crisis? An analysis of changes in stress and well-being in Switzerland. Eur Soc 23(sup1):S942–S956. https://doi.org/10.1080/14616696.2020.1839671

Kurniawati N, Pierewan AC (2020) The Determinants Of Women’s Happiness in Indonesia. Int J Sci Technol Res 9(4):3630–3635

Landiyanto EA, Ling J, Puspitasari M, Irianti SE (2011) Wealth and Happiness: Empirical Evidence from Indonesia. Chulalongkorn J Econ 23:1–17

Langer A, Meleis A, Knaul FM, Atun R, Aran M, Arreola-Ornelas H, Bhutta ZA, Binagwaho A, Bonita R, Caglia JM, Claeson M, Davies J, Donnay FA, Gausman JM, Glickman C, Kearns AD, Kendall T, Lozano R, Seboni N, Frenk J (2015) Women and Health: the key for sustainable development. Lancet 386(9999):1165–1210. https://doi.org/10.1016/s0140-6736(15)60497-4

Le K, Nguyen M (2021) The psychological burden of the COVID-19 pandemic severity. Econ Hum Biol 41:100979. https://doi.org/10.1016/j.ehb.2021.100979

Lee SJ (2022) Public Happiness. Springer, Cham, 10.1007/978-3-030-89643-0

Lepinteur A, Clark AE, Ferrer-i-Carbonell A, Piper A, Schröder C, D’Ambrosio C (2022) Gender, loneliness and happiness during COVID-19. J Behav Exp Econ 101:101952. https://doi.org/10.1016/j.socec.2022.101952

Levin KA, Currie C (2014) Reliability and validity of an adapted version of the Cantril ladder for use with adolescent samples. Soc Indic Res 119(2):1047–1063. https://doi.org/10.1007/s11205-013-0507-4

Lim HE, Shaw D, Liao PS, Duan H (2020) The Effects of Income on Happiness in East and South Asia: Societal Values Matter? J Happiness Stud 21(2):391–415. https://doi.org/10.1007/s10902-019-00088-9

López-Ruiz V-R, Huete-Alcocer N, Alfaro-Navarro J-L, Nevado-Peña D (2021) The relationship between happiness and quality of life: A model for Spanish society. PLOS One 16(11):e0259528. https://doi.org/10.1371/journal.pone.0259528

Mayuzumi Y (2022) Survey of rural and urban happiness in Indonesia during the corona crisis. Asia Pac J Region Sci 7:29–67. https://doi.org/10.1007/s41685-022-00265-4

Mehmetoglu M, Jakobsen TG (2017) Applied Statistics Using Stata: A Guide for the Social Sciences. SAGE Publications Asia-Pacific Pte Ltd, Singapore

Möhring K, Naumann E, Reifenscheid M, Wenz A, Rettig T, Krieger U, Friedel S, Finkel M, Cornesse C, Blom AG (2021) The COVID-19 pandemic and subjective well-being: longitudinal evidence on satisfaction with work and family. Eur Soc 23(S1):S601–S617. https://doi.org/10.1080/14616696.2020.1833066

Muresan GM, Ciumas C, Achim MV (2020) Can Money Buy Happiness? Evidence for European Countries. Appl Res Qual Life 15. https://doi.org/10.1007/s11482-019-09714-3

Muthia LH, Isbah MF (2022) “Poor but Happy”: Life Struggle and the Meaning of Happiness among the Poor in Yogyakarta. J Sosiol Walisongo 6(2):117–128. https://doi.org/10.21580/jsw.2022.6.2.8902

Nandini D, Afiatno BE (2020) The Determinants Of Happiness: Empirical Evidence Of Java Island. Ekonika 5(2):123. https://doi.org/10.30737/ekonika.v5i2.713

Nguyen CV (2021) Does the COVID-19 Pandemic Cause People to Be Unhappy? Evidence from a Six-Country Survey. GLO Discussion Paper No.768. Global Labor Organization (GLO), Essen, http://hdl.handle.net/10419/228738

Oishi S, Diener E (2014) Can and Should Happiness Be a Policy Goal? Policy Insights Behav Brain Sci 1(1):195–203. https://doi.org/10.1177/2372732214548427

Pattinasarany IRI (2018) Happiness and Life Satisfaction Among East and Southeast Asian Countries. Paper presented at the Fourth Conference of International Consortium for Social Well-Being Studies. Seoul National University, Seoul

Petrovič F, Murgaš F, Králik R (2021) Happiness in Czechia during the COVID-19 Pandemic. Sustainability 13(19):10826. https://doi.org/10.3390/su131910826

Article   CAS   Google Scholar  

Phulkerd S, Thapsuwan S, Soottipong GR, Chamratrithirong A, Pattaravanich U, Ungchusak, C & Saonuam P (2023) Life Satisfaction before and during COVID-19 pandemic in Thailand. Int J Public Health 68. https://doi.org/10.3389/ijph.2023.1605483

Rabe-Hesketh S, Skrondal A (2022) Multilevel and Longitudinal Modeling Using Stata, Fourth Edition. Stata Press Publication, College Station

Rajkumar RP (2023) Cultural values and changes in happiness in 78 countries during the COVID-19 pandemic: An analysis of data from the World Happiness Reports. Front Psychol 14. https://doi.org/10.3389/fpsyg.2023.1090340

Rahayu TP (2016) The Determinants of Happiness in Indonesia. Mediterr J Soc Sci 7(2):393–404

Rahmanita M, Nurbaeti, Asmaniati F, Dewi TR, Widyastuti N (2021) COVID-19 Pandemic: Happiness Revisited through Work and Leisure during the Stay at Home Period. J Hunan Univ Nat Sci 48(4):100–107

Recchi E, Ferragina E, Helmeid E, Pauly S, Safi M, Sauger N, Schradie J (2020) The “Eye of the Hurricane” Paradox: An Unexpected and Unequal Rise of Well-Being During the Covid-19 Lockdown in France. Res Soc Stratification Mobil 68:100508. https://doi.org/10.1016/j.rssm.2020.100508

Rehdanz K, Welsch H, Narita D, Okubo T (2015) Well-being effects of a major natural disaster: The case of Fukushima. J Econ Behav Organ 116:500–517. https://doi.org/10.1016/j.jebo.2015.05.014

Schroder M (2018) Income Inequality and Life Satisfaction: Unrelated Between Countries, Associated Within Countries Over Time. J Happiness Stud 19:1021–1043. https://doi.org/10.1007/s10902-017-9860-3

Sekulova F, van den Bergh JCJM (2016) Floods and happiness: Empirical evidence from Bulgaria. Ecol Econ 126:51–57. https://doi.org/10.1016/j.ecolecon.2016.02.014

Shams K, Kadow A (2022) COVID-19 and Subjective Well-Being in Urban Pakistan in the Beginning of the Pandemic: A Socio-Economic Analysis. Appl Res Qual Life 18:93–113. https://doi.org/10.1007/s11482-022-10114-3

Snijders TA, Bosker RJ (2012) Multilevel analysis: an introduction to basic and advanced multilevel modeling, Second Edition. Sage Publications Asia-Pacific, Singapore

Sohn K (2014) Height and Happiness in a Developing Country. J Happiness Stud 17(1):1–23. https://doi.org/10.1007/s10902-014-9566-8

Sohn K (2013) Sources of Happiness in Indonesia. Singap Econ Rev 58(02):1350014. https://doi.org/10.1142/s0217590813500148

Sollis K, Resosudarmo BP, Witoelar F, Riswandi R, Mollet JA (2023) Migrant Status and the Wellbeing Gap: The Case of an Ethnically Diverse, High-Conflict Area in Indonesia. J Happiness Stud. https://doi.org/10.1007/s10902-023-00659-x

Sommet N, Morselli D (2017) Keep Calm and Learn Multilevel Logistic Modeling: A Simplified Three-Step Procedure Using Stata, R, Mplus, and SPSS. Int Rev Soc Psychol 30(1):203–218. https://doi.org/10.5334/irsp.90

StataCorp (2021) Stata Multilevel Mixed-Effects Reference Manual Release 17. Stata Press Publication, College Station

Sujarwoto S (2021) Development as happiness: A multidimensional analysis of subjective well-being in Indonesia. Econ Sociol 14(2):274–293. https://doi.org/10.14254/2071-789x.2021/14-2/15

Sujarwoto S, Tampubolon G, Pierewan AC (2017) Individual and Contextual Factors of Happiness and Life Satisfaction in a Low Middle Income Country. Appl Res Qual Life 13(4):927–945. https://doi.org/10.1007/s11482-017-9567-y

Sujarwoto S, Tampubolon G (2015) Decentralisation and Citizen Happiness: A Multilevel Analysis of Self-rated Happiness in Indonesia. J Happiness Stud 16:455–475. https://doi.org/10.1007/s10902-014-9518-3

Tadic M, Braam H, van Vliet K, Veenhoven R (2014) Memory-Experience Gap in Early Adolescents’ Happiness Reports. Child Indic Res 7:21–40. https://doi.org/10.1007/s12187-013-9194-6

Tambyah SK, Tan SJ, Lun YW (2023) Happiness and wellbeing in Singapore: Beyond economic prosperity. Routledge, New York

Tjahjana D, Dwidienawati D, Manurung AH, Gandasari D (2021) Does People’s Well-being Get Impacted by COVID-19 Pandemic Measure in Indonesia? Stud App Econ 39(4). https://doi.org/10.25115/eea.v39i4.4873

Toshkov D (2022) The Relationship Between Age and Happiness Varies by Income. J Happiness Stud 23:1169–1188. https://doi.org/10.1007/s10902-021-00445-7

Utomo AJ (2012) Women as Secondary Earners. Asian Popul Stud 8(1):65–85. https://doi.org/10.1080/17441730.2012.646841

van der Vegt I, Kleinberg B (2020) Women Worry About Family, Men About the Economy: Gender Differences in Emotional Responses to COVID-19. Lect Notes Comput Sci 12467:397–409. https://doi.org/10.1007/978-3-030-60975-7_29

Veenhoven R (2012) Happiness: Also Known as “Life Satisfaction” and “Subjective Well-Being”. In: Land K, Michalos A, Sirgy M (eds) Handbook of Social Indicators and Quality of Life Research. Springer, Dordrecht, p 63–77. 10.1007/978-94-007-2421-1_3

Winkelmann L, Winkelmann R (1998) Why are the unemployed so unhappy? Evidence from panel data. Economica 65(257):1–15. https://doi.org/10.1111/1468-0335.00111

Worldometer (2023) Countries where Coronavirus has spread - Worldometer. https://www.worldometers.info/coronavirus/countries-where-coronavirus-has-spread/ . Accessed on February 23, 2023

Wu Y, Zhu J (2016) When Are People Unhappy? Corruption Experience, Environment, and Life Satisfaction in Mainland China. J Happiness Stud 17(3):1125–47. https://doi.org/10.1007/s10902-015-9635-7

Yang H, Ma J (2020) How an epidemic outbreak impacts happiness: Factors that worsen (vs. protect) emotional well-being during the coronavirus pandemic. Psychiatry Res 289:113045289. https://doi.org/10.1016/j.psychres.2020.113045

Yiengprugsawan V, Somboonsook B, Seubsman S, Sleigh AC (2011) Happiness, Mental Health, and Socio-Demographic Associations Among a National Cohort of Thai Adults. J Happiness Stud 13(6):1019–1029. https://doi.org/10.1007/s10902-011-9304-4

Yu GB, Lee D-J, Sirgy MJ, Bosnjak M (2019) Household Income, Satisfaction with Standard of Living, and Subjective Well-Being: The Moderating Role of Happiness Materialism. J Happiness Stud. https://doi.org/10.1007/s10902-019-00202-x

Download references

Acknowledgements

This study was supported by Grant No. NKB-1211/UN2.RST/HKP.05.00/2022 from the Publikasi Terindeks Internasional (PUTI) Q1, Directorate of Research and Development (Risbang), Universitas Indonesia. The author is grateful for the constructive inputs and discussions throughout the preparation of this study from Professor Masayuki Kanai from the School of Human Sciences, Senshu University, and Professor Iwan Gardono Sudjatmiko from the Department of Sociology, Universitas Indonesia. In addition, Peter Morley from the Australian Volunteers Indonesia assisted in shaping the report and editorial services.

Author information

Authors and affiliations.

Department of Sociology, Faculty of Social and Political Sciences, Universitas Indonesia, Depok, Indonesia

Indera Ratna Irawati Pattinasarany

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Indera Ratna Irawati Pattinasarany .

Ethics declarations

Competing interests.

The author declares no competing interests.

Ethical approval

Ethical approval was not required as the study did not involve human participants performed by the author.

Informed consent

Informed consent was not required as the study did not involve human participants performed by the author.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Pattinasarany, I.R.I. Happiness amidst the COVID-19 pandemic in Indonesia: exploring gender, residence type, and pandemic severity. Humanit Soc Sci Commun 11 , 609 (2024). https://doi.org/10.1057/s41599-024-03131-0

Download citation

Received : 13 July 2023

Accepted : 25 April 2024

Published : 10 May 2024

DOI : https://doi.org/10.1057/s41599-024-03131-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Cement production and CO 2 emission cycles in the USA: evidence from MS-ARDL and MS-VARDL causality methods with century-long data

• Research Article
• Open access
• Published: 10 May 2024

Cite this article

You have full access to this open access article

• Melike E. Bildirici 1 &
• Özgür Ömer Ersin   ORCID: orcid.org/0000-0002-9177-2780 2  

233 Accesses

Explore all metrics

The cement industry is among the top three polluters among all industries and the examination of the nonlinear and cointegration dynamics between cement production and CO 2 emissions has not been explored. Focusing on this research gap, the study employs a novel Markov-switching autoregressive distributed lag (MS-ARDL) model and its generalization to vector error correction, the MS-VARDL model, for regime-dependent causality testing. The new method allows the determination of nonlinear long-run and short-run relations, regime duration, and cement-induced-CO 2 emission cycles in the USA for a historically long dataset covering 1900–2021. Empirical findings point to nonlinearity in all series and nonlinear cointegration between cement production and cement-induced CO 2 emissions. The phases of regimes coincide closely with NBER’s official economic cycles for the USA. The second regime, characterized by expansions, lasts twice as long relative to the first, the contractionary regime, which contains severe economic recessions, as well as economic crises, the 1929 Great Depression, the 1973 Oil Crisis, the 2009 Great Recession, and the COVID-19 Shutdown and Wars, including WWI and II. In both regimes, the adverse effects of cement production on CO 2 emissions cannot be rejected with varying degrees both in the long and the short run. Markov regime-switching vector autoregressive distributed lag (MS-VARDL) causality tests confirm unidirectional causality from cement production to CO 2 emissions in both regimes. The traditional Granger causality test produces an over-acceptance of causality in a discussed set of cases. Industry-level policy recommendations include investments to help with the shift to green kiln technologies and energy efficiency. National-level policies on renewable energy and carbon capture are also vital considering the energy consumption of cement production.

Graphical Abstract

Avoid common mistakes on your manuscript.

Introduction

The issue of environmental pollution has significant effects on global warming. The achievement of sustainability in economic development cannot be achieved without environmental sustainability. As a greenhouse gas, carbon dioxide (CO 2 ) is among the most important sources of global warming and climate change. In the last century, CO 2 emissions have risen to unprecedented levels on which industrial production has strong effects. Compared to the pre-industrial levels (1850–1900), the mean temperatures on Earth have been 1.53 °C higher in the last decades and global warming is affecting life on globe through shifts of climate zones, extreme weather events, alterations in the functioning, and structure of climate including the carbon-cycle feedbacks of Earth (Alkama & Cescatti 2016 ; Forzieri et al. 2017 ; Hoffman et al. 2014 ; Richardson et al. 2013 ). Consequently, if human-induced climate change is not controlled, climate change becomes irreversible. Recent reports from the International Energy Agency (IEA) projects that CO 2 emissions will further reach a peak of 37 billion tons (Gt) in 2025 (IEA 2022a ) and if serious action is not taken soon, global warming will reach an irreversible level in the next 75 years (IEA 2022c ) leading to UN Climate Change Report noted the seriousness of the issue and emphasized the insufficiency in the political commitment already existent (UNCC 2022 ). The report noted that if current policies were to be maintained in the future, CO 2 emissions would reach a 10.6% increase in 2030, just in 8 years; however, to reverse global warming, the opposite, a cut of 45% is necessary before year 2030 (UNCC 2022 ).

If all industries in the world are ranked according to the amount of CO 2 emissions they yield, the cement industry is the top third (The Guardian 2019 ). If the cement industry were a country itself, it would be the third top CO 2 emitter after China and the USA. Worrell et al. stressed that cement production was responsible for 5% of the anthropogenic CO 2 emissions in the early 2000s (Worrell et al. 2001 ). In 2020, its contribution to global CO 2 emissions reached 8–10% (Wu et al. 2022 ). Following these concerns, in the UN’s COP24 meeting that took place in Poland in 2018, cement’s CO 2 emissions were taken into the goals to revert climate change, with a target of 16% reduction in cement production-induced CO 2 emissions by 2030 (Rodgers 2018 ). As a result, the cement sector is one of the most important contributors to CO 2 emissions in the globe with direct and indirect channels.

The direct channel of the cement-induced CO 2 emissions is due to the emissions released during the processes in production. There are three main sources of anthropogenic emissions of CO 2 , i.e., fossil fuel oxidation, land-use change (including deforestation), and decomposition of carbonates, and cement production is considered an important emitter mainly in terms of the third (Andrew 2018 ). CO 2 is emitted from the calcination process of limestone and the combustion of fuels in the kiln during cement production (Costa and Ribeiro  2020 ). Reducing the CO 2 from cement production processes is of great importance. CO 2 emissions are also released due to the utilization of high levels of energy during production. The CO 2 statistics stated in the previous paragraph for cement production avoid indirect releases due to the high levels of energy consumed. As shown by Worrell et al. ( 2001 ) the total amount of CO 2 emissions from processing cement and from the energy it necessitates and the average intensity of CO 2 emissions from global cement production is 222 kg per ton of cement produced. Nagi and Jang stress that the amount is four times higher for Portland cement; each ton of cement produced releases an equal amount of CO 2 emissions (Naqi & Jang 2019 ). The CO 2 emissions of cement accelerate as the share of fossil fuel or nonrenewable energy consumption in the total energy mix is not low and depending on the country and the energy policy followed, the CO 2 mitigation effect would be altered. In the context of Industry 4.0, the nonrenewable energy consumption share of the USA in its total energy use is shown to be one of the highest (M. Bildirici & Ersin 2023 ). Footnote 1 As a result, the rigorous commitment to green energy and a large share of renewable energy in the energy mix would help in the reduction of CO 2 emissions in addition to energy-efficient cement production technologies. As shown in the discussion section, the major cement industries are in China, India, and the USA, and these countries are also among the top countries with high shares of fossil-fuel energy in their energy mixes. As a result, cement industries have not only a national level but also a global effect on CO 2 emissions.

In addition to its role in environmental pollution, cement is an essential product closely linked to economic development policies and various sectors. Economic development projects are generally coupled with construction projects. The nexus between cement production and economic growth has significant connections to business cycles in the economy; cement production is also subject to interconnected fluctuations in the economy and to the influence of fluctuations in the GDP. Business cycles, which include expansionary and contractionary phases, govern economic activity and construction investments which rise during periods of economic development and growth, which fuel cement production.

The business cycle in the USA is shown to be subject to nonlinearity with the expansionary and recessionary periods with asymmetry in characteristics and durations (Hamilton 1989 ). These cycles are frequently influenced by economic crises and deep recessions which also bring about fiscal and monetary interventions of the policymakers to bring the economy back to the track of economic growth. It is clear that the economic policy interventions that favor economic expansions had significant and nonlinear effects on environmental sustainability. The production patterns for cement are expected to be highly nonlinear possessing a cyclical tendency that is also connected to economic activity, not to mention, important historical events such as World Wars or deep recessions. Cement production directly emits CO 2 emissions as a characteristic of cement production and kiln that requires reaching a heat level of 1200 °C. Cement is responsible for 8% of global CO 2 emissions. In addition to its direct effects, the production requires excessive use of energy that further contributes to CO 2 emissions. Indirect effects include the CO 2 emissions geared by construction. Therefore, it is of crucial importance to examine cement production and CO 2 emissions historically by putting forth the cement-induced CO 2 emission cycles and their relation to economic cycles in the USA. In addition to these effects, CO 2 emissions resulting from cement production are expected to be nonlinear and have asymmetric effects that differ in size under distinct regimes with different durations.

With this motivation, the investigation of nonlinear long-run relations and nonlinear causality among cement production and CO 2 emissions will provide vital information regarding the environmental effects of the cement industry from an empirical perspective. For this purpose, the study employs a long sample starting from 1900 to provide a historical perspective. The sample covers economic contractions, deep crises, and abrupt changes caused by World War I and II, the Great Depression of 1929, the Oil Crisis of 1973, the 2009 Great Recession, and 2020 COVID-19. Therefore, the sample provides a laboratory to examine the CO 2 and cement production nexus and the cement-induced CO 2 cycles. The overlook is that these cycles have a relation with economic recessions in the USA; however, the type of recession has a strong influence. As shown in the empirical and discussion sections, cement production and cement-induced CO 2 emissions fluctuate sharply with deep recessions as well as economic crises and abrupt shocks exampled above. The cycles in the cement-induced CO 2 emissions and cement production are in close synchronization with the economic business cycles. Given the size of the cement sector among all sectors, its strong influence on the overall CO 2 emissions of the USA could not be rejected. In addition to the relation of the cement industry with economic cycles in the USA, the relation is not constant, or is linear. The type of the recession matters. We argue that the sector is not affected by short-lasting economic recessions, but has strong relations with longer-lasting and deep recessions, crises, and abrupt changes. The influence of cement on CO 2 emissions differs in size under expansionary and contractionary cement production regimes. Further, cement manufacture is strongly encouraged by the policymakers during periods of recessions and crises for recovery, in addition to economic expansion periods, to contribute to economic growth or to achieve back its track. As a result, contraction in cement production is not a common situation for all recessions and crises, depending on the type of recession. In many cases, especially for long periods of deep recessions or periods of contractions geared by wars, cement is an important sector with inclines in cement production, which also yields cement-induced CO 2 emissions.

In light of the discussion above, the goal of the study is to design a nonlinear method to examine the long-run relation between cement production and environmental pollution in the USA with historically long data covering 1900–2021. The reason for choosing the USA is its significant cement production. In fact, in 2015, the cement industry in the USA yielded 82.8 million tons (81,500,000 long tons; 91,300,000 short tons) of cement, valued at US$9.8 billion (DATIS 2020 ). The USA was ranked as the world’s third-largest cement producer in 2019, trailing behind China and India (USGS 2024 ). By the end of 2022, cement production in the USA had reached around 95 kilometric tons, placing the nation as the fourth-largest cement producer globally after China, India, and Vietnam Footnote 2 (WPR 2024 ). On the other hand, there is no long-term data available for China and India, the top two countries for cement production, and the econometric methods employed in this study require data over a long period.

The study suggests a novel approach, the Markov-switching autoregressive distributed lag (MS-ARDL) model by integrating two seminal methods. The MS-ARDL allows modeling regime dynamics and business-cycle modeling benefiting from the dynamic Markov-switching regressions (MSR) of Hamilton (Hamilton 1989 ). The MS-ARDL approach merges the linear ARDL approach for bound testing and cointegration modeling (Pesaran et al. 2001 ) with the MSR to obtain a unique approach that captures regime-dependent cointegrated long-run relations and short-run relations with different error correction dynamics to the long-run equilibrium under each regime. The MS-ARDL follows single-step modeling of long- and short-run dynamics similar to the ARDL (Pesaran et al. 2001 ), which generalizes the well-known two-stage long-run cointegration methodology (Engle & Granger 1987 ). The proposed model is further generalized to vector autoregressive (VAR) models to obtain the MS-VARDL model in this study. Both MS-ARDL and MS-VARDL models provide insightful information concerning regime durations, cycle dating, and regime-dependent Granger causality investigation for the cement production and cement-induced-CO 2 emission relation. The contribution of this study to the literature is twofold. Firstly, the study proposes the MS-ARDL and MS-VARDL models, which are expected to provide significant contributions to the empirical analyses, especially in energy and environmental research. Secondly, the contribution of the article to the environmental literature is emphasized by analyzing 123 years of data, highlighting the impact of long-term data usage in this literature.

The paper is structured as follows. The literature review is given in the “ Literature review ” section, where a discussion of cement-CO 2 emission relation is evaluated. The econometric methodology for the MS-ARDL model is given in the “ Econometric methodology ” section. The empirical results are given in the “ Econometric results ” section. The discussion, policy recommendations, and conclusion are given in the “ Conclusion ” section.

Literature review

If the literature on industrial production and emissions is investigated, a large body of research focuses on the positive effects of production on emissions at low levels of production and the relation being reversed at high industrial production levels, the so-called environmental Kuznets curve (EKC). Further, we noted that the empirical literature on the cement and emissions nexus is very limited, especially concerning econometric findings at the national level. The existing recent research focuses mainly on China, and as of our literature search, only a few papers discuss the relation of cement industry emissions in the context of other countries, especially the USA.

The empirical research on emission-gross domestic product (GDP) has gained significant pace following the seminal findings (Grossman & Krueger 1991 ; Selden & Song 1994 ; Stern 1994 ). Grossman and Krueger’s empirical results related the levels of two main pollutants by signifying an inverted- U -shaped relation, i.e., environmental pollution increasing (decreasing) at low (high) levels of per capita (Grossman & Krueger 1991 ), and Selden and Song underlined declining hazardous emissions at high levels of economic development (Selden & Song 1994 ). The cause of the decline in emissions at high GDP levels was considered as decentralization of industrial production, and the reversal of the positive trend in population growth at high-income levels (Stern 1994 ). To overcome the impossibility of a negative effect of industrial production on emissions, Lopez recommends internalization of emissions and feedback effects at the industry level and emissions should be taken as a factor of production (Lopez 1994 ). Convergence of carbon emissions at high GDP levels is an important factor and several empirical findings stressed sigma, stochastic, and beta convergence in addition to the existence of the environmental Kuznets curve (EKC) (Anjum et al. 2014 ; Pettersson et al. 2014 ). The existence of a decline in emissions at high industrial production levels is rejected empirically after omitting the bias caused by beta convergence on the empirical methods (Stern et al. 2017 ).

The long-run and causal effects between energy consumption, growth, and CO 2 emissions also found significant applications and the importance of energy efficiency and renewable energies were documented (Ozturk & Acaravci 2013 ). Our findings indicated the close relations of these factors to the cycles in the production of cement; however, these relations are strongly nonlinear both in the short and in the long run, and in addition, our findings suggest the advocation of energy efficiency and green energy policies in the cement industry, which has strong ties with the business cycles of economic growth with differentiated dynamics in the expansionary and recessionary regimes. By investigating the environmental and health effects of the construction industry within a comparative perspective with various sectors, the negative effects of cement production on health and air quality are documented (Bildirici 2020 ).

By following nonlinear regime switching neural network models and by calculating the sensitivity of CO 2 growth rates to fossil fuel and economic growth, Bildirici and Ersin emphasize the questionability of linear in parameter-type EKC formulations, in addition to stressing the role of transfer of industrial production to newly industrializing other countries from already industrialized nations (M. Bildirici & Ersin 2018a ). Bildirici and Ersin suggest a novel nonlinear STARDL cointegration model, with which important deviations from the EKC are obtained compared to linear ARDL, and it is suggested that CO 2 and economic growth have nonlinear characteristics due to business cycles, crises, and structural changes in production historically for 1800–2014 period in the USA (M. Bildirici & Ersin 2018b ). Using a panel of countries including the OECD countries with the nonlinear Panel STAR model, the EKC relation is strongly rejected in both regimes for the panel of countries (Ersin 2016 ). Ersin stresses that the turning point threshold is determined by CO 2 emission growth rates, not the economic growth rates; after the turning point, evidence is against the reversal from environmental degradation under nonlinearity and threshold effects (Ersin 2016 ).

The consensus in the literature that focuses on the cement industry and its impacts on the environment relates emissions to energy levels needed in production and a common policy recommendation is to increase energy efficiency. However, concerns were also raised about how energy efficiency would slow down the emissions of the cement industry. Accordingly, clinker production activity is identified as the central polluter in the industry (Wang et al. 2013 ), and estimates show that the cement industry is the highest emitter industry both in China and in the world (Teller et al. 2000 ). Empirical findings determine labor productivity and energy intensity as major determinants of CO 2 emissions in the cement sector (Lin & Zhang 2016 ). Ke et al. confirm the carbon emissions due to the energy intensity of cement production and advocate energy efficiency to lessen emissions (Ke et al. 2012 ). Xu et al. ( 2012 ) distinguish among four features of cement manufacture, overall output, ratio of clinker, processing technique, and type of energy used up per kiln type (J. H. Xu et al. 2012 ). They identify the link between growth in cement production and economic growth coupled with infrastructure and construction sectors (J. H. Xu et al. 2012 ).

Specific investigation of the cement industry and its effects on emissions has gained increasing attention and led to important findings (Bekun et al. 2022 ; Cai et al. 2016 ; Gao et al. 2017 ; Ren et al. 2023 ; Supino et al. 2016 ; Tan et al. 2022 ), Further, the majority of empirical research on national data is centered on China with few exceptions. Various studies are investigated which focus on different sectors and among these, some have ties with the cement industry. Regarding important mitigating effects concerning emissions, the emphasized sectors in the literature, other than cement, include petrochemical (Xin et al. 2022 ), mining (Chen & Yan 2022 ; Li et al. 2023 ), logistics (Liang et al. 2022 ; B. Xu & Xu 2022 ), transportation (M. Liu et al. 2021 ; H. Xu et al. 2022 ), steel and nonferrous metal (J. Zhang et al. 2023 ), foundry (Zheng et al. 2022 ), manufacturing industry chains (Lin & Teng 2022 ), and coal industry (Xia & Zhang 2022 ). In addition, the construction sector, as a sector related directly to cement consumption, also is among the strong emitters of CO 2 emissions (Y. Liu et al. 2022 ; Zhao et al. 2022 ). The construction sector is followed by sectors of steel, nonferrous sectors, and a fraction of chemical industries as sectors with relations to cement consumption. Concerning the effects of mining (Chen & Yan 2022 ; Li et al. 2023 ), the main findings advocate carbon-neutrality policies in the sector to reduce high levels of emissions (Chen & Yan 2022 ). The nonferrous metal and steel sectors are directly related sectors to the construction sector and have relations to cement consumption. The nonferrous sector has strong effects on CO 2 mitigation and emission reduction strategies are presented (Cao et al. 2022 ; Y. Zhang et al. 2022 ).

Production techniques are criticized in terms of their environmental impacts and alternate techniques are advocated. As an example, the replacement of clinker as the binding material in cement production with recycled material is suggested (Costa and Ribeiro 2020 ). Footnote 3 Martins et al. study the emissions due to solid, construction, and demolition wastes in addition to the energy consumption created through construction contributing to climate change (Martins et al. 2023 ). Karlsson et al. calculate a potential 40% reduction in construction-embodied CO 2 by realizing material efficiency, recycling, and construction supply chains (Karlsson et al. 2021 ). Though zero-emission is advocated through transforming transportation to electric vehicles (EV), if the very large share of fossil-fuel energy in total energy consumption is taken into consideration, as a typical, over 80% in the USA such an EV policy would have little effect without transformation of energy production from nonrenewables which also has strong emission potential in the installment and maintenance (M. Bildirici & Ersin 2023 ).

Studies investigated cement production and emissions in selected countries. Hanle et. al is among the few studies, which emphasize the USA’s cement production highlighting the level of CO 2 emissions it generates (Hanle et al. 2004 ). Footnote 4 The Dutch construction industry is emphasized in terms of recycled concrete materials to achieve circular economy objectives (Yu et al. 2021 ). Pakdel et al. investigate the energy-induced CO 2 mitigation effects of traditional and contemporary methods in the Iranian construction industry (Pakdel et al. 2021 ). Karlsson et al. explore the Swedish road construction industry through the role of supply chains to achieve net-zero CO 2 (Karlsson et al. 2020 ). Huang et al. empirically analyze the nexus between emissions and energy embodied in the construction of buildings in Taipei (Huang et al. 2019 ). Vorayos and Jaitiang ( 2020 ) analyze the relationship between the environment and energy performance of Thailand’s cement industry (Vorayos & Jaitiang 2020 ). Oke et al. ( 2017 ) investigate carbon emission trading in the construction industry in South Africa (Oke et al. 2017 ). The regional dataset for 2000–2005 and data envelopment techniques for India are used to determine the state-level inefficiency levels of the cement sector and the consequences of CO 2 emissions (Kumar Mandal & Madheswaran 2010 ). Turkey’s cement industry is investigated with data envelopment for 51 cement factories in 2016 and CO 2 emission externality in the cement production process is highlighted (Dirik et al. 2019 ). These empirical results revealed that only 16% of all integrated cement factories were efficient leading to inclined environmental worsening (Dirik et al. 2019 ). Belbute and Pereira utilize time-series models with fractional integration to obtain CO 2 emission forecasts from fossil-fuel consumption and cement production in Portugal and their findings could be interpreted as showing the importance of lowering cement production in achieving carbon emission targets (Belbute & Pereira 2020 ). By providing a comparative analysis of China and the USA’s cement industry with nonlinear models and Granger causality among cement production, economic growth, and environmental pollution, Bildirici ( 2019 ) stresses that if nonlinear relations are ignored, policy recommendations would lead to incorrect results which hamper environmental sustainability (M. E. Bildirici 2019 ). It is also shown that the effects of cement production and its effects on environmental degradation and health would be insufficiently identified (M. E. Bildirici 2020 ). Footnote 5

Econometric methodology

Single-regime ardl approach.

Cointegration is a seminal technique that allows the researcher to model long-run and short-run dynamics, adjustment towards the long-run equilibrium following shocks, and the length of adjustment in linear relations (Engle & Granger 1987 ). The ARDL method of Pesaran-Shin-Smith (PSS) generalizes the cointegration method to ARDL methodology with generated testing method of bound tests (Pesaran et al. 2001 ). The paper aims to generalize the linear, i.e., single-regime, ARDL approach to Markov-switching (MS) to account for nonlinear dynamics in long-run relations.

A single-regime long-run linear regression form is

assuming Y t dependent variable being modeled with k number of X 1,t ,…, X k,t independent variables, and the long-run form consisting of k  + 1 number of parameters including the intercept \(\left\{{\delta }_{0},{\delta }_{1},{\delta }_{2},...,{\delta }_{k}\right\}\) . PSS also allows exogenous variables such as a linear trend, or dummy variable, \(D_{t}\) , to be included in the long-run form:

In the Engle-Granger methodology, the variables have a long-run relation if integrated of a common order d and if their linear combination is stationary so that the residuals are stationary (Engle & Granger 1987 ). In the PSS methodology, the ARDL model allows the combination of I (1) and I (0) variables; however, to eliminate degenerate cases and loss in power of the ARDL cointegration test, the dependent variable should be I (1). The short-run model in which the error-correction presentation is embedded is achieved as

where \(\omega\) is the error correction parameter and the speed of transition to the long-run equilibrium is \(1/\omega\) ; for the mechanism to work, it necessitates an estimate of \(\omega\) such that \(-1<\widehat{\omega }<0\) similar to the Engle-Granger cointegration model (Engle & Granger 1987 ). For simplicity, in Eq. ( 3 ),  \(\left\{{Y}_{t}, \, {X}_{1,t}, \, \dots ,{X}_{k,t}\right\}\sim I\left(1\right)\) , so that \({\Delta }^{d}=\Delta\) . However, the Engle-Granger approach is a two-step model in nature given Eqs. ( 1 ) and ( 3 ). The ARDL model of PSS allows long-run and short-run dynamics to be modeled simultaneously within a single-step estimation,

and further, in Eq. ( 4 ), the integration properties of variables are defined as in Pesaran et al. ( 2001 ) so that the series is allowed to be I (1) or I (0) processes or a combination of both (Pesaran et al. 2001 ). The bound test statistic of Pesaran et. al. (2001), F PSS , is calculated by restricting \({\lambda }_{1}\) ,  \({\lambda }_{2}\) ,…,  \({\lambda }_{k+1}\)  = 0 under \({H}_{0}:{\lambda }_{1}=0,{\lambda }_{2}=0,...,{\lambda }_{k+1}=0\) , i.e., no cointegration, to be tested against \({H}_{1}:{\lambda }_{1}\ne 0,{\lambda }_{2}\ne 0,...,{\lambda }_{k+1}\ne 0\) . If F PSS  >  F PSS,Upper and F PSS  >  F PSS,Lower , Pesaran et al. ( 2001 )’s upper and lower bounds, the result would favor cointegration and long-run association (Pesaran et al. 2001 ). However, confirmation of the existence of a single cointegration vector is necessary (Narayan 2014 ). The error-correction form is a restricted form as

where \(\omega {\eta }_{t-1}\) defines the error-correction mechanism and the previous definitions of \({\eta }_{t}\) and \(\omega\) hold. For simplicity, assume k  = 1. Single-regime ARDL model of Eq. ( 4 ) becomes

and the restricted ARDL representation in Eq. ( 5 ):

In Eq. ( 6 ), single-regime and linear ARDL-type cointegration test hypotheses are \({H}_{0}:{\lambda }_{1}=0,{\lambda }_{2}=0\) , \({H}_{1}:{\lambda }_{1}\ne 0,{\lambda }_{2}\ne 0\) and if statistically F PSS  >  F PSS,Upper and F PSS  >  F PSS,Lower , the long-run linear association is accepted. If cointegration is established, a confirmatory test is \({H}_{0}:\omega =0\) and \({H}_{1}:\omega \ne 0\) in Eq. ( 7 ); the former suggests no linear cointegration, by assuming only a linear form of a long-run association. The above-mentioned ARDL methodology has been challenged and criticized for various aspects: (i) Over-acceptance of cointegration, Narayan’s critical values should be preferred (Narayan 2014 ), especially for small samples. Further confirmation of the existence of a single cointegration vector is necessary (Narayan 2014 ). (ii) PSS requires dependent variable to follow \({Y}_{t}\sim I\left(1\right)\) to avoid power loss in the test procedure and to avoid degenerate case-1 (McNown et al. 2018 ). Under such cases, the F or t-tests of ARDL cointegration become inconclusive (McNown et al. 2018 ). Footnote 6 (iii) Bildirici and Ersin ( 2018a , b ) noted ignoring nonlinearity would lead to incorrect policy recommendations and introduce smooth transition ARDL (STARDL) models, by generalizing the single-regime ARDL to smooth transition autoregression (STAR) type nonlinear processes to nonlinear cointegration. Footnote 7

(iv) Banerjee et al. ( 2017 ) show the loss of power of the ARDL test under structural breaks and integrate Fourier terms into the ARDL model. Bildirici and Ersin ( 2023 ) generalize the proposed Fourier ARDL model to bootstrapping ARDL model to achieve Panel Fourier BARDL to control inefficiency under structural change and nonlinearity (M. Bildirici & Ersin 2023 ). Banerjee et al. ( 2017 ) argue that the Fourier functions with different dimensions could capture various forms and numbers of nonlinear structural changesMetin girmek için buraya tıklayın veya dokunun.. Enders and Lee ( 2012 ) show that Fourier is more efficient in correcting the bias in unit root tests under smooth changes and less efficient in abrupt changes (Enders & Lee 2012 ). MS-type regime models are capable of capturing sudden and abrupt shifts in regimes in addition to determining the dating and duration of regimes. Footnote 8

Markov regime-switching ARDL model

The MS-ARDL model is a nonlinear error correction (NEC) model that allows nonlinearity in both long-run and short-run dynamics simultaneously. Therefore, MS-type regime changes (Hamilton 1989 ) are integrated into the ARDL model to achieve the Markov regime-switching autoregressive distributed lag (MS-ARDL) model. Various other forms of NEC are evaluated by Saikkonen ( 2008 ). Among these models, Saikkonen ( 2005 ) allows regime changes governed by an indicator function to achieve a threshold NEC. Saikkonen ( 2008 ) also discusses possible extensions to Markovian regimes to achieve NEC models.

Significant models on modeling NEC have been proposed with various nonlinear techniques. A general tendency for NEC modeling so far has been to keep the short-run parameters linear while allowing error correction parameters to be regime-specific generalizations of Engle-Granger methodology (Kapetanios et al. 2006 ; Saikkonen 2005 , 2008 ). Krolzig develops the MS-VEC model in a VAR setting and the MS-VEC allows MS-type changes in the error correction as a nonlinear generalization to Engle-Granger’s cointegration approach (H. M. Krolzig et al. 2002 ). Footnote 9 Pavlyuk applies MS-ARDL model to traffic forecasting (Pavlyuk 2017 ); however, the model is an MS-ARX model and does not utilize the AR and DL terms in the spirit of NEC modeling and PSS-type ARDL cointegration.

Other NEC models include Kapetanios et al. which allow exponential smooth transition functions to capture regime-dependent error correction (Kapetanios et al. 2006 ). Shin et al. developed a nonlinear ARDL (NARDL) framework with a threshold-type nonlinearity instead of MS (Shin et al. 2013 ). Bildirici and Ersin generalize the ARDL to smooth transition type nonlinearity with the smooth transition ARDL (STARDL) model. The STARDL model generalizes ARDL to nonlinearity and asymmetry both for the long- and short-run relations (M. Bildirici & Ersin 2018b ). With this respect, both STARDL and the NARDL models relax the symmetry assumption for either the long- or the short-run terms.

An MS-ARDL model with two or more regimes is

where \({\alpha }^{{s}_{t}}={\left\{{\alpha }_{0}^{{s}_{t}},{\alpha }_{1,i}^{{s}_{t}},{\alpha }_{2,i}^{{s}_{t}}...,{\alpha }_{k+1}^{{s}_{t}}\right\}}^{\prime}\) is the short and \({\lambda }^{{s}_{t}}={\left\{{\lambda }_{1}^{{s}_{t}},{\lambda }_{2}^{{s}_{t}},...,{\lambda }_{k+1}^{{s}_{t}}\right\}}^{\prime}\) is the long-run parameter vector, both being regime-dependent; regime changes are governed with s t for r number of regimes \({s}_{t}\in \left\{\mathrm{1,2},...,r\right\}\) . Hence, s t  = 1, s t  = 2,…, and s t  =  r is a finite regime sequence. \(N\left( 0,\sum_{}^{}\left( s_{t} \right) \right)\) is distributed with zero conditional mean and regime-dependent \(\sum \left({s}_{t}\right)\) nonnegative conditional variance. As a result, \({\varepsilon }_{t}^{{s}_{t}}\) are allowed to be locally homoskedastic for sub-regression spaces, while being globally heteroskedastic. For a similar approach, see Saikkonen ( 2008 ). The model generalizes the single-regime ARDL in Eq. ( 4 ) to MS-type regime switches in Eq. ( 8 ).

The generalization of ARDL bound testing is necessary in the MS-ARDL modeling stages. Once the existence of MS-ARDL type nonlinearity is accepted against linear ARDL following the Davies linearity test, the null hypothesis of no cointegration relation is

which means neither linear nor nonlinear error correction exists, to be tested against the alternative of MS-type nonlinear cointegration,

defining a regime-dependent cointegration in each distinct regime. The testing requires a conventional F test approach, the calculated F statistic is F MSARDL , and if it passes both the upper and lower bounds, F MSARDL  >  F PSS,upper and F MSARDL  >  F PSS,Lower , the H 0 null hypothesis of no cointegration is rejected against the alternative H 1 , that is, MS-ARDL-type cointegration with r number of regimes. The proposed F MSARDL test statistic follows an F distribution, F ( q , n  −  r ( k  + 1) − 1), with q  =  r ( k  + 1) where r represents the number of regimes and k  + 1 is the number of \({\lambda }^{{s}_{t}}\) tested for cointegration for each regime.

By replacing the long-run part with the regime-specific error correction mechanism, reduced form nonlinear MS-ARDL error correction representation of Eq. ( 8 ) is

\({\omega }^{{s}_{t}}\) is a regime-specific error correction parameter for \({\eta }_{t-1}\) , the error-correction term. If the error correction parameter estimate, \({\widehat{\omega }}^{{s}_{t}}\) , is statistically accepted to lie between \(-1<{\widehat{\omega }}^{ {s}_{t}=r}<0\) , regime-specific error correction duration is calculated as 1/ \({\omega }^{{s}_{t}}\) , which holds for each r distinct regimes as long as \({\widehat{\omega }}^{ {s}_{t}=1}\ne {\widehat{\omega }}^{ {s}_{t}=2}\ne ...\ne {\widehat{\omega }}^{ {s}_{t}=r}\) . Footnote 10

The conditional probability density of time series y t is stated as

where \({\phi }_{r}\) is the vector of parameters in r  = 1,2, … , r number of regimes (H.-M. Krolzig 1997 ). The Markov chain defining the regime-switching process for the model is as follows:

where p ij is the probability of being in regime i at time t conditional on the state (or regime) j at time t −  1 (Hamilton 1989 ). Similar to the MS-AR and MS-VAR models, p ij is subject to

where \(P\left\{\left.{s}_{t}\right|{s}_{t-1};\rho \right\}\) is the probability of state s t at period t conditional on the previous state s t − 1 (M. E. Bildirici 2020 ). The switching variable, s t , is an unobserved discrete-state Markov chain, which governs the endogenous switches in r number of regimes (Krolzig & Toro 2002 ). Footnote 11 In each distinct regime, a locally linear ARDL sub-space exists defining regime-specific relations among modeled time series. Hence, it is an irreducible ergodic Markov process with r number of states for which the transition matrix is (Hamilton 1989 )

Consistent with the MS-VAR and MS-AR models, the Markov chain follows the irreducible and ergodic process and each p ij has an unconditional and stationary distribution given the ergodicity of the Markov process (H.-M. Krolzig 1997 ). The probability of \({s}_{t-1}=i\) at t −  1 is conditional on the information set available and the parameter set, \(\Omega_{t-1};\;\phi_r\) . Hence, in the iteration process, for t  = 1, 2, …, T , the probability for the previous period is used as an input:

The present state \({\xi }_{it}\) includes all information regarding the Markovian process that follows in the future (H.-M. Krolzig 1997 ):

The conditional log-likelihood is stated as \(\log\;f\;(y_1,y_2,...,y_T\vert y_0;\;\phi)=\sum\log\;f\;(y_t\vert\Omega_{t-1};\;\phi)\) .

For a two-regime MS-ARDL model, Eqs. ( 15 ) and ( 16 ) become.

where the unconditional distribution of each p ij is

and the calculation leads to

In the case of two regimes, observations are conveyed into the first sub-regression space if \(Pr({s}_{t}= \, 1\left|{Y}_{T}\right.)\ge 0.5\) or to the second if \(Pr({s}_{t}= \, 1\left|{Y}_{T}\right.)<0.5\) . In the estimation step, the expectation maximization (EM) algorithm is utilized (Hamilton 1989 ).

Markov regime-switching vector autoregressive distributed lag model

The MS-ARDL model assumes both the long-run and short-run dynamics to follow nonlinear regime-switching. A vector autoregressive (VAR) generalization of the MS-ARDL model is necessary to investigate the existence of a single cointegration vector. In addition, the MS-VARDL model could be easily adapted to examine nonlinear Granger causality between the analyzed variables depending on the distinct regimes. Therefore, it is convenient to write the Markov-switching vector ARDL (MS-VARDL) model as a VAR generalization of Eq. ( 8 ). For simplicity, a two-variable, two-regime MS-VARDL model is given as

where \({\alpha }_{1,i}^{{s}_{t}},{\alpha }_{2,i}^{{s}_{t}}\) and \({\theta }_{1,i}^{{s}_{t}},{\theta }_{2,i}^{{s}_{t}}\) are the short-run parameter sets in MS-VARDL vectors 1 and 2 and \({\lambda }_{1}^{{s}_{t}},{\lambda }_{2}^{{s}_{t}}\) and \({\tau }_{1}^{{s}_{t}},{\tau }_{2}^{{s}_{t}}\) are the vector-specific long-run parameters. Given that \({s}_{t}=\mathrm{1,2}\) , cointegration testing necessitates repeating the MS-ARDL cointegration test separately for each vector in Eq. ( 22 ) for \({\lambda }_{1}^{{s}_{t}},{\lambda }_{2}^{{s}_{t}}\) and for \(\tau_{1}^{{s_{t} }} ,\tau_{2}^{{s_{t} }}\) . In vector 1, zero-restricted \(\lambda_{1}^{{s_{t} }} ,\lambda_{2}^{{s_{t} }}\) lead to the null hypothesis of no cointegration (linear or nonlinear) in both regimes:

For the second vector, the null of no-cointegration,

is tested against nonlinear cointegration as

For both tests, F MSARDL test statistic follows an F distribution as \(F_{MSARDL} \sim F(q,n - r\left( {m + n + k + 1) - 2} \right)\) and for the two-variate, two-regime model, q  = 4. As a next step, one could also estimate a restricted error correction form of MS-VARDL for confirmatory purposes:

To test no-cointegration (linear or nonlinear) against nonlinear cointegration, hypotheses are \(H_{0} :\omega_{1}^{{s_{t} }} = 0\) and \({H}_{1}:{\omega }_{1}^{{s}_{t}}\ne {0}{{\text{for}}}{s}_{t}=\mathrm{1,2}\) in vector 1, and \(H_{0} :\omega_{2}^{{s_{t} }} = 0^{{}}\) and \(H_{1} :\omega_{2}^{{s_{t} }} \ne 0\) in vector 2 of the model given in Eq. ( 27 ). Vector-specific F MS-VARDL test statistic follows \(F_{MS - VARDL} \sim (2,n - r\left( {m + n + 1} \right) - 2)\) . The MS-VARDL modeling steps proposed above aim at testing nonlinear ARDL-type error correction occurring in each vector. For specific applications, researchers could also consider testing the existence of regime-specific nonlinear cointegration (M. Bildirici & Ersin 2018b ). In this case, once MS-VARDL given in Eq. ( 22 ) is estimated sub-tests targeting specific regimes for specific vectors are likely. As a typical, assume testing regime 1 of vector 1, a low volatility or economic growth regime. Regime-specific hypotheses are \(H_{0} :\lambda_{1}^{{s_{t} = 1}} = \lambda_{2}^{{s_{t} = 1}} = 0\) , \(H_{1} :\lambda_{1}^{{s_{t} = 1}} \ne \lambda_{2}^{{s_{t} = 1}} \ne 0\) . For regime-specific or global nonlinear cointegration, readers are referred to M. Bildirici and Ersin ( 2018b ).

To achieve the existence of multiple regimes, Davies tests should be applied. Further, the stability of the ergodic switching probabilities should be examined with the diagonal of Eq. ( 15 ) or in a two-regime model, the p 11 and p 22 in Eq. ( 18 ) so that p 11  < 0.5, p 22  < 0.5 to achieve persistence in each regime in addition to confirming their statistical significance.

MS-VARDL in Eq. ( 22 ) reduces to the MS vector error correction (MS-VEC) model (H.-M. Krolzig & Toro 2002 ) under very mild restrictions applied on the long- and short-term parameters. The MS-VEC generalizes the VEC model to MS and cointegration methodology. The MS-VARDL, on the other hand, generalizes MS-ARDL to MS-VARDL. The nonlinear MS-VEC model is given as (Clements & Krolzig 2002 ; H.-M. Krolzig 1997 )

\(\delta^{{s_{t} }}\) is a drift term that is a function that shifts the intercept in the long-run equation. \(\beta^{\prime}\) is the long-run parameter vector and B i is the short-run parameter set. The short-run parameters are not subject to regime-switching. Y t is the variable matrix and the model is distinguished as a shifting mean regime-switching model for s t  = 1,2,…, r number of regimes. By applying zero restrictions to an MS-VARDL, the reduced form MS-VEC representation exists. Footnote 12

Econometric results

The study will focus on the following steps in the empirical section:

Unit root (UR) testing with a battery of tests allowing different forms of data-generating processes. Included tests are ADF, KPSS, KSS, and F-ADF. KPSS is robust to various forms of structural breaks, the KSS test (Kapetanios et al. 2006 ) tests unit root against nonlinear stationary series, and F-ADF is the Fourier ADF test of Engle and Lee (2011) known to be robust to a wide form of nonlinear series in addition to smooth structural breaks.

F bound testing with traditional single-regime ARDL and Johansen cointegration test to investigate the existence of cointegration.

The BDS test (Broock et al. 1996 ) is applied to investigate the nonlinearity of the series.

Nonlinear regime-dependent bound testing is tested with the MS-ARDL test.

Single-regime ARDL and nonlinear MS-ARDL models are estimated.

Determination of regime durations, datings, and regime switching probabilities for MS-ARDL.

Linearity is tested against regime-dependent nonlinearity with F tests.

Model evaluation with diagnostics tests.

The determination of the direction of causality and comparative analysis with single-regime causality (VEC-based) and regime-switching causality (MS-VARDL-based).

Inference and policy recommendations following the direction of causality determination.

The study is among one of the studies that utilize historically long datasets in the literature focusing on environmental sustainability within econometric respects. In terms of the evaluation of the effects of cement on the environment, the study is, to our knowledge, a pioneering study that evaluates a historically long sample for the post-1900 period in terms of focusing on the econometric relations between CO 2 emissions and cement production. Footnote 13 The sample covers the 1900–2021 period for the USA and the dataset is yearly. The period contains several significant events, including the First and Second World Wars, the 1973 Oil Crises, and important economic crises, such as the 1929 Great Depression, the Great Recession in 2008, and recently, COVID-19. The emission data represents CO 2 emissions from cement production in the USA and is in kilotons of CO 2 . Cement production ( CP t ) is in billion metric tons and is available from the Andrew ( 2022 ) database which obtains the yearly CP t data from the U.S. Geological Survey (Andrew 2018 , 2022 ). Variables are subject to natural logarithms as LCO t  = ln( CO 2t ) and LCP t  = ln( CP t ). As reported in the following section, these level series contain unit roots and are integrated of order 1. After first differencing, the respectful series are Δ LCO t and Δ LCP t , which also represent the growth rates. The descriptive statistics are reported in Table  1 .

For the level series, Jarque–Bera test statistics imply that, at 5% level of significance, normality for level variables cannot be rejected. For the first differenced series, series are not normally distributed and are subject to skewness and excess kurtosis. In the next step, series are tested for stationarity and unit roots.

Unit root and stationarity tests

The unit root tests are used to determine the order of integration of the series and whether the variables are I (0) or I (1). A battery of tests are applied which include traditional tests in addition to those robust to various forms of nonlinearity. These tests include the Augmented Dickey-Fuller (ADF), Kwiatkowski-Phillips-Schmidt-Shin (KPSS), Kapetanios-Shin-Snell (KSS), and Enders and Lee (2011) Fourier-ADF. The test results are reported in Table  2 . The traditional linear unit root test, mainly the ADF test, is known to have size distortions under nonlinearity and structural changes and the ADF test tends to over-reject the null hypothesis of unit root (Nelson et al. 2001 ). KPSS tests stationarity under the null and the test is known to be less influenced by nonlinear series. The KSS tests the null of the unit root series against the alternative of stationary time series following a nonlinear STAR process under the alternative. Enders and Lee’s ( 2012 ) Fourier ADF test is utilized to test the unit root null hypothesis and the test is known to be robust against smooth forms of structural changes. The unit root tests utilized in the study evaluate stationarity under forms of structural changes and nonlinearity. In all tests reported in Table  2 , both LCP t and LCO t series are integrated of a common order of 1, and they become stationary once first differenced.

BDS test results

Broock, Deckert, and Sheinkman (BDS) test is a test based on correlation dimension and the test examines the independent and identically distributed i.i.d. series under the null hypothesis against the alternative of nonnormality caused by series following nonlinear or chaotic behavior. Broock et al. ( 1996 ) provide a recent treatment of the test (Broock et al. 1996 ). BDS test provides an investigation of deviations from independence and is known to be efficient in detecting different forms of nonlinearity. BDS test is also suggested as a model architecture determination tool similar to tests such as the Ljung-Box test of autocorrelation; hence, the test could also be used on residuals of models to investigate remaining nonlinearity (Broock et al. 1996 ).

It should be noted that the unit root and stationarity tests determined that series in levels follow I (1) processes and the model in the next section will utilize both the levels and the first differenced series. In Table  3 , BDS test results are given for series in levels and for series in first differences. For robustness, the variables are also tested with the Tsay test of threshold-type nonlinearity (Tsay 1986 ). The results are given in the last column of Table  3 .

Since BDS and z -test statistics are greater than the critical values at conventional significance levels, the null hypothesis of i.i.d. is rejected for both LCO t and LCP t , and results favor nonrejection of the alternative hypothesis suggesting that series in levels follow nonlinear processes. For their first differenced counterparts, BDS results suggest similar results at conventional significance levels for all embedded dimensions the test is conducted for. Tsay test results are given in the last section of Table  3 , where the test is repeated for different lag orders. The Tsay test results confirm the rejection of linearity for both level and first differenced series at conventional significance levels against the alternative of nonlinearity. The overall results suggest that linear models could not be appropriate due to the nonlinear characteristics of the data analyzed.

MS-ARDL dating of contractions in cement production-emission cycle

In the first stage, we examined the estimated MS-ARDL model in terms of its relation with the NBER contraction (including economic recessions and crises) periods. The results are reported in Table  4 .

The first column includes the estimated contraction dates for cement production in the cement industry and the CO 2 emissions due to cement production. The second column reports the contraction periods reported by the NBER (National Bureau of Economic Recessions). Therefore, the table provides NBER and the authors’ calculations obtained from the MS-ARDL which will provide important insights. The overall look shows that for the majority, the dates and durations match with those reported by NBER. It should be noted that an exact match should not be expected all the time. NBER dating focuses on economic recessions in the whole industry and the model results focus on cement-industry and cement-induced CO 2 emissions. Though NBER dates and cement-CO 2 cycles are generally in line, it is also expected to have leading and especially lagging relations in the CO 2 emissions instead of exact coincidences all the time. The general outlook is, though these are not in general, they occur mainly after deep recessions and crises and after such periods, once expansion in the economy starts, cement production and especially CO 2 emissions follow in certain additional years which leads to acceleration in cement-induced CO 2 cycles. Investigation of a set of selected periods will hinder important information regarding the cement production and CO 2 emission cycles and their relations to the economy as a whole.

NBER contraction dates are calculated from NBER’s business cycle reference dates. NBER announces trough dates and the duration of contraction periods in months starting from the date of the trough. Though NBER cycles are for the economy as a whole, the contraction dates and durations reported by the MS-ARDL model estimations are for cement-induced CO 2 emissions and cement production. The findings suggest that, depending on the economic recession, the cycle dates do not match all the time. As a result, the findings suggest that the cement production and CO 2 cycle generally lags the economy’s business cycle in general except for distinct cases such as the 1914–1916 contraction which shows that the industry was already in crisis. Further, one could combine 1914–1916 and 1917–1921 contractions which totals 1914–1921, which coincides with the 1914–1915 and 1919–1921 economic cycles. It should be noted that these economic cycle dates coincide with WWI and the post-WW1 period explaining the length and duration of the contraction in the cement output and cement-induced CO 2 emissions. Another cycle starting in 1991 is in line with the year of the Golf War. The economic contraction was estimated to be for 1991–1992 by NBER. With our calculations benefiting from the estimated MS-ARDL model, the recession is estimated for 1991–1994, 2 more years, compared to the NBER cycle for the cement-production and cement-induced CO 2 emission relation. Findings show that following deep recessions and wars, tightening in cement production and cement-induced CO 2 emissions follow with a lag. However, it quickly shifts back again to the track of increasing emissions which occurs during the expansionary economic periods. For other dates, we note that the cycles reported by NBER and the MS-ARDL estimations are close matches. The WWII period is taken into consideration as a significant case that lasted between 1939 and 1945. The NBER dates are 1938–1939 and 1945–1946 which present the contraction years which match with those for the cement production and cement-based CO 2 emission cycle.

Another important finding is that the model catches the dates especially crisis periods and deep recessions especially lasting more than 1 year. Given that the model utilizes cement production instead of cement consumption, these findings are expected. Cement consumption responds quickly through declines in demand in the recession periods. However, cement production continues production and stocks the excess supply as inventories and this is especially so in the short recessions which take between 6 and 11 months. As typical, the 1949 recession lasted 11 months according to NBER but no contractions were captured for cement production and cement-induced CO 2 emissions. If growth rates are to be noted, cement production declined by 12% in both 1980 and 1982 and for the cement industry, more drastic declines are captured by the MS-ARDL model. For example, cement production declined by 27% in 1918 (WWI) and 39% in 1944 (WWII), but only a 6% decline in 1930, just after the 1929 Great Depression. In 1931, cement production declined by 31% and by 50% in 1932 corresponding to the deep recession period afterward. After the Oil Crisis in 1973–1974, cement production decreased by 18%. Further, the decline amounted to 30% in 2009, the Great Recession, and only 12% weakening in early 2020 during the COVID-19 shutdown. Footnote 14 The findings confirm that cement production responds to deep recessions and crises and during mild recessions, the effects of business-cycle contractions are relatively less on the cement industry and therefore on cement production-induced CO 2 emissions in such periods. This is expected since both variables are production-based and during recessions and expansions, cement consumption responds relatively fast and with moderate response to economic fluctuations. Consequently, cement-production-induced CO 2 emissions react to profound recessions and economic crises by deteriorating CO 2 emissions. Otherwise, the emitting effect continues given the level of cement production under mild recessions.

Linear (single-regime) ARDL results

To provide a baseline analysis, linear, single-regime ARDL modeling steps are conducted. Results are reported in Table  5 . Investigation of cointegration with the ARDL model also provides crucial input for the determination of the direction of cointegration in addition to providing the opportunity to investigate the residuals for remaining nonlinearity which cannot be captured with the linear model.

The ARDL model is determined as ARDL(1,1) with Schwarz information criterion (SC). By taking each variable as the dependent variable one by one, bound tests are repeated. F PSS is calculated as 13.745, favoring cointegration, if LCO t is assumed as the dependent variable, the cement-production-induced CO 2 . For the counter case, by taking LCP t as the dependent, F PSS  = 3.1081, at the 1% significance level, F PSS  <  F lower and F PSS  <  F upper , leading to the decision of no cointegration. (Narayan 2014 ; Pesaran et al. 2001 ). The results favored cointegration relation and long-run association between LCO t and LCP t and the direction of relation is determined as LCO t  =  f ( LCP t ). For confirmatory analysis, the Johansen cointegration test is conducted. Results are reported in the second part of Table  5 . Johansen test confirms a single cointegration equation for the LCO t and LCP t relation.

The selected ARDL model is tested with the BDS test (Broock et al. 1996 ) and results are reported in Table  6 . As suggested by Broock et al. ( 1996 ), the BDS test could be used as a test of remaining nonlinearity if used for the residuals of the model. Accordingly, the linear ARDL model fails to capture the nonlinearity in the residuals and therefore the nonlinear relation between cement production and CO 2 emissions at conventional significance levels. Hence, the linear ARDL model cannot be accepted under remaining nonlinearity and to avoid possible inefficient policy recommendations, nonlinear ARDL methods should be followed (M. Bildirici & Ersin 2018b ). The analysis will continue with nonlinear MS-ARDL-type nonlinearity testing and modeling.

MS-ARDL nonlinear cointegration test results

In this step, the null of linearity in residuals of the linear ARDL model is tested against the alternative of MS-ARDL-type nonlinearity. The Davies-type linearity test statistic is calculated as F  = 18.0521, favoring the rejection of the single-regime model against the regime-switching model with 2 regimes. After that, the MS-ARDL test of no cointegration is conducted. In the test, no cointegration (linear or nonlinear) is tested under the null hypothesis against nonlinear cointegration. The test results are given in Table  7 .

Assuming LCO t as the dependent variable, F MS-ARDL test statistic = 9.882477, statistically larger than the 5% critical lower and upper bounds and the results show that LCO t and LCP t have a nonlinear relation in the long run. By assuming LCP t as the dependent variable, a second F MS-ARDL is calculated and is equal to 3.98456, lower than the upper and lower critical F values, leading to the rejection of cointegration at conventional significance levels. Hence, the results determined nonlinear ARDL cointegration between cement production and cement-production-induced CO 2 emissions in addition to determining a single cointegration vector exists only by taking the emissions as the dependent variable. Footnote 15

Model estimation results

Following the nonlinear cointegration test results, a two-regime MS-ARDL model is estimated. In addition, the linear single-regime ARDL model is estimated for comparative purposes. Following Tables 6 and 7 , cement-induced CO 2 emissions are taken as the dependent variable for both the ARDL and MS-ARDL models, and optimum lag length is selected as three for the ARDL and as two for the MS-ARDL model with the Schwarz information criterion. The MS-ARDL model is determined to have two regimes following the linearity tests. Footnote 16 The ARDL and MS-ARDL model estimation results are given in Table  8 where the long-run dynamics are reported in the first part followed by the short-run dynamics, the regime-switching probabilities, and the diagnostic test results.

The linear ARDL model estimation results are in column 1. Once evaluated, though the parameters of cement production are significant in the long run suggesting the net effect of cement production to be positive on emissions, certain results lead us to be cautious about the ARDL model results. First, the error correction mechanism fails to hold since the parameter of ECT t − 1 is estimated as − 0.015, suggesting an error correction duration approaching 66 years, relatively too long compared to the nonlinear and regime-specific error correction parameter estimates; however, the results are not reliable for the linear model due to the factors listed below. Additionally, the error correction term is statistically insignificant for the linear model, suggesting that the error correction cannot be established for the linear model. The remainder of the linear ARDL model is evaluated with BDS and RESET tests. The BDS test results in Table  6 confirmed the remaining nonlinearity in the residuals of the linear ARDL, possibly leading to biased parameter estimates if nonlinearity in the series is ignored. Among the diagnostics tests, Breusch-Pagan-Godfrey (BPG) and Ramsey’s RESET tests reported in the last section favored homoskedastic residuals, but the model was mis-specified at a 5% significance level. If evaluated together, the estimation results of the linear ARDL model fail to produce satisfactory results, and due to remaining nonlinearity in the residuals, the ARDL parameters for the linear model, if interpreted and if taken for policy purposes, would be seriously misleading.

If the diagnostics tests are evaluated for the MS-ARDL model, the goodness of fit statistics of the MS-ARDL model favors a better fit of the nonlinear model over its single-regime counterpart. Ramsey’s RESET and Breusch-Pagan-Godfrey (BPG) test results for the linear model conclude model-misspecification and heteroskedastic residuals at conventional statistical significance levels. The BPG and RESET test results for the MS-ARDL model favor no homoskedasticity in the residuals and no-model-misspecification. The BPG and RESET tests favor no misspecification and homoscedastic residuals for the MS-ARDL model. Once considered together with the nonlinearity test results, diagnostics tests lead to the existence of remaining nonlinearity in the residuals in the ARDL model and selection for the MS-ARDL results over its linear counterpart. For the MS-ARDL model results reported in columns 2 and 3, the remaining nonlinearity in the residuals is tested with the BDS test, and test results confirm no remaining nonlinearity. As a result, the estimation results of the MS-ARDL model will be taken as central in the study to examine the relations between cement production and emissions.

For the MS-ARDL results given in Table  8 , the first and second regimes correspond to the recession and crisis regime (regime 1) and expansion regime (regime 2), respectively. If an overlook is presented to the estimated nonlinear long-run dynamics, it is striking that the impact of cement production on CO 2 emissions is positive in both regimes with different magnitudes. In the long-run equation of regime 1, the relevant coefficients of LCP t − 1 and LCP t − 2 are estimated as 0.286 and 0.198, and in regime 2, 0.167 and 0.159, respectively. The parameters are statistically significant at the 5% significance level, with one exception: the parameter of LCP t − 2 is significant at the 10% significance level only. Hence, a 1% increase in cement production at periods t −  1 and t −  2 leads to 0.286% and 0.198% increases in the CO 2 emissions in regime 1, compared to 0.167% and 0.159% effects in regime 2, the last one being significant at 10% only, but the positive effect persists. The overall result is that the positive effects of cement production on CO 2 emissions cannot be rejected in the USA for the long run. It should be noted that the positive effect of cement production is positive in both regimes.

The short-run dynamics are presented in the second section of Table  8 . Once the parameters of Δ LCP t are investigated for regimes, they confirm the effects of cement production on CO 2 emissions in the short-run in addition to the long-run dynamics presented. Similarly, the findings confirm regime-dependent and asymmetric effects of cement production in the short run. The parameter estimates for Δ LCP t − 1 and Δ LCP t − 2 are − 0.60 and 0.79 in regime 1 and − 0.0007 and 0.14275 in regime 2. Though a 1% increase in the previous year’s cement production decreased emissions by 0.60 in regime 1, the same parameter in regime 2 is estimated as − 0.0007, and almost no effect exists in regime 2 for the first lag of Δ LCP t − 1 . However, for the second lag, the parameter estimate of Δ LCP t − 2 is estimated as 0.14, significant at 5% significance level, and a 1% increase leading to a 0.79% increase in emissions in regime 1 and 0.14% increase in regime 2. Short-run dynamics also confirm significant and positive impacts of cement production on emissions during both regimes and in terms of the long-run portion of the model, this positive effect increases especially during the cement production expansion regimes.

The significance, sign, and size of the error correction terms play a crucial role in establishing cointegration. The error correction parameters are estimated as − 0.25748 and − 0.48296 for regimes 1 and 2. Hence, 25.7% (48.3%) of the deviations from the long-run equilibrium are corrected within 1 period, and the error correction towards the long-run equilibrium takes 3.9 (2.07) years in regime 1 (regime 2). Overall results show that asymmetry and regime dependence are important factors determining the effects of cement production on emissions. The effects of cement production are positive in both regimes while being larger both in the short and in the long run. Additionally, the error correction mechanisms are asymmetric among regimes; the mechanism takes twice as long in regime 1 compared to regime 2. The estimated probability of staying at the regime at period t conditional on the regime at period t  − 1 determines the persistence of both regimes. The estimated regime probabilities are p ( s t  = 1 | s t  = 1) = 0.885 and p ( s t  = 2 | s t  = 2) = 0.940845 for regimes 1 and 2, signifying a high degree of persistence in both regimes and regime 2 being relatively more persistent and longer lasting.

MS-ARDL causality results

In the next section, MS-ARDL-based causality analysis is reported. The determination of the direction of causality under regime dependency plays a crucial role in policy suggestions. In the context of the methodology presented in the “ Econometric methodology ” section, once the MS-ARDL model is extended to the MS-VARDL model, regime-specific Granger noncausality test results are calculated and reported in Table  9 . For comparative purposes, linear noncausality test results are reported in the last column.

The regime-specific causality results could be easily determined through the utilization of the methods followed in the paper. According to our results, the null hypothesis of Granger noncausality from cement production to CO 2 is rejected and the alternative is accepted in both regimes in addition to the linear model given in the last section. Hence, the results confirm nonlinear unidirectional causality from cement production to emissions in regime 1, the high-emission regime. This finding is in line with the linear causality test results. However, by investigating regime-specific causality results, our model provides bidirectional causality between cement production and CO 2 in regime 2. As a result, the method the paper provides led to additional insights given the feedback effects specific to regime 2. Hence, the feedback effect due to bidirectional causality is a phenomenon occurring specifically in regime 2, the low-emission regime, in contrast to the unidirectional causal effect from cement production to emissions in regime 1.

Given the fact that the nonlinear causality testing utilizes the MS-VARDL results, after the determination of the directions of causality, our method also allows the determination of the sign of the causal effect by evaluating the regime-specific parameter estimates. Footnote 17 The results are given in the last row of Table  9 . At the 5% significance level, for the determined causal effects in both regimes, the signs of parameters are positive confirming the positive effects of cement production on emissions in all specifications in addition to the positive effect of emissions on the acceleration of cement production in regime 2.

The findings of this paper indicate that reducing CO 2 emissions is contingent upon curtailing cement production, as it is the primary source of CO 2 emissions and this result is obtained in both regimes. Consequently, though asymmetry between the effects of cement production on CO 2 emissions exists, this asymmetry is mainly in terms of the magnitude, but not in terms of the sign of the effect. In addition, nonlinear causality results provided important deviations from the traditional causality results obtained with linear Granger causality techniques. If the findings are evaluated as a whole, the MS-ARDL results are led to the same results as the regime-dependent causality results and confirm these results. Traditional causality results are in line with the causality results in regime 1. Conversely, the causality results in changes in regime 2. Thus, the policy suggestions should be determined independently for regimes 1 and 2, the deep recession and crisis regime, and the expansionary regime, respectively.

Following the estimation results above, interesting results are obtained once the linear ARDL and nonlinear MS-ARDL are considered. The general finding suggests that regime dependency and nonlinearity should be taken into consideration for policy suggestions focusing on the negative effects of the cement industry on environmentally hazardous greenhouse gases. Depending on the regime, such negative effects accelerate and policymakers should consider the regime the economy and the industry are in since the CO 2 emitting effect of the industry depends on the regime type. Utilizing the outcomes of MS-VARDL analysis for nonlinear causality assessment, our approach not only establishes the causal directions but also enables the determination of the causal effect’s polarity through an examination of regime-specific parameter estimations. These outcomes are presented in the final row of Table  9 . At a significance level of 5%, the parameter signs for established causal effects in both regimes are positive, affirming the beneficial impact of cement production on emissions across all specifications. Furthermore, a positive influence of emissions on the acceleration of cement production is confirmed in regime 2. When considering the entirety of the findings, the MS-ARDL results align with regime-dependent causality outcomes, thereby reinforcing these conclusions. Traditional causality results are consistent with the causality outcomes observed in regime 1. In contrast, the causality directions diverge in regime 2. Accordingly, causality results affirm nonlinear unidirectional causality from cement production to emissions within regime 1. In terms of unidirectional links, the findings align with the findings of the linear causality tests for regime 1 only. The nonlinear method reveals bidirectional causality between cement production and CO 2 emissions in regime 2, in contrast to the unidirectional causality in regime 1. As a result, the method presented in this paper yields supplementary insights by uncovering feedback effects specific to regime 2, and the regimes that the industry is at gives vital information since the feedback effect could lead to a circular effect resulting in a cycle of more emissions. Hence, the phenomenon of feedback effects stemming from bidirectional causality manifests uniquely within regime 2, in contrast to the unidirectional causal effect from cement production to emissions observed in regime 1, and policies should aim at avoiding the negative implications on the environment by aiming at the alteration of the type of industrial production techniques with newer technologies on cement production.

Greenhouse gases and the global warming

When the literature was analyzed, vast amounts of environmental pollutants, encompassing SO 2 , NO x , CO, and PM, were discharged during cement production (Lei et al. 2011 ). The production of cement involves the high-temperature calcination of carbonate minerals, resulting in clinker formation and the release of CO 2 into the atmosphere (Xi et al. 2016 ). CO 2 emissions from cement production stem from two primary sources. Firstly, a chemical reaction occurs during the production of the central cement component. The cement generates oxides (lime, CaO), and CO 2 is emitted due to heat effects. These “process” emissions contribute to approximately 5% of total anthropogenic CO 2 emissions, excluding land-use changes (Boden et al. 1999 ). Secondly, nonrenewable energy combustion is used to heat raw materials to temperatures exceeding 1000 °C (IEA 2022b ). Around 90% of worldwide CO 2 emissions from industrial processes result from cement-related activities (M. E. Bildirici 2019 , 2020 ) and the cement industry’s combined emissions account for approximately 8% of the global CO 2 output (Andrew 2018 ; Le Quéré et al. 2018 ).

Overall, a variety of gases, called greenhouse gases (GHG), contribute to the greenhouse effect and global warming. The sizable portion of GHG emissions is dominated by CO 2 emissions (US EPA 2023 ). US Environmental Protection Agency reports that the shares of GHGs are CO 2 at 79.4%, methane (CH4) at 11.5%, nitrous oxide at 6.2%, and the remaining GHGs are fluorinated gases with a total contribution of 3% to the GHG effect (US EPA 2023 ). As the main driver of climate change due to the GHG effect, the recent positive trend of CO 2 in the last century is considered a result of human activity due to the burning of fossil fuels (coal, oil, and natural gas), deforestation, and industrial processes, and about more than 65% reduction of current CO 2 releases is needed to achieve environmental goals (Belbute & Pereira 2020 ). Therefore, CO 2 is among the top contributors to global climate change and the reversal necessitates great political commitment.

Historical relations among cement production, cement-induced CO 2 emissions, and business cycles

As confirmed by our empirical findings, cement production and CO 2 emissions are interrelated. Cement production is a derived demand of construction investments accelerating as economic growth accelerates. It is shown that the cement and construction industry and economic growth relation have important linkages (M. E. Bildirici 2019 ). Economic production is known to follow fluctuations known as business cycles, which include expansionary and recessionary periods. Throughout history, other factors that led to abrupt changes in the production cycle include economic crises, the Great Depression in 1929, the 1973 Oil Crisis, and World Wars (WWI and II). Not only do economic cycles have strong ties with cement as an important ingredient for construction, but also cement industry is also expected to follow a similar pattern possessing expansionary and contractionary episodes in line with the business cycle. Therefore, business cycles also create a derived cement demand during expansions, after crises and recessions. Hence, acceleration in cement production following recessions and even during recessions due to policies encouraging economic growth to overcome such downturns. Such cyclical behaviors in economic business cycles are nonlinear and due to their relation with economic activity, cement production also follows cycles and nonlinearity. Further, cement production is an important emitter of CO 2 . The overlook suggests that the production process of cement and the demand for energy that cement production necessitates are among the top channels that lead to environmental degradation. Various factors with relations to cement production include urbanization and the land-use change (Mishra et al. 2022 ; Zhou et al. 2021 ), which contribute to CO 2 emissions.

Yearly cement production (solid line) and CO 2 emissions from cement production (dashed line) are depicted in Fig.  1 for the 1900–2021 period. The figure also included the recession dates (as a grey bar) obtained from the National Bureau of Economic Recessions (NBER). As seen in Fig.  1 , the fluctuations in CO 2 from cement and cement production are closely linked and a positive association exists between the two series. The inclines and declines in both coincide in terms of occurrence and year and terms of the length of duration in the majority of cases. The recessions in economic business cycles lead to declined economic production, coupled with both declines in CO 2 emissions from cement and cement production in the USA. This relation becomes clearer, especially during the 1929 Great Depression and 2008 Great Recession with sharp declines in both series. The declines also coincide with the NBER dating of recessions. Another example is the decline in CO 2 emissions and cement production during the first year of COVID-19 in 2020, which was reversed afterward during the economic expansion that followed in 2021. Hence, the fluctuations in cement production are argued to be in line with economic business cycles, consisting of expansionary and recessionary episodes, leading to similar cycles in cement production and cement-based CO 2 emissions.

Source : U.S. Geological Survey and NBER. Note that cement production (right-hand side) is in billion metric tons. CO 2 emission from cement is for tons of CO 2 from 1 ton of cement produced and is per capita

NBER recessions, cement production, and CO 2 emissions from cement production in the USA, 1900–2021.

However, there are exceptions such as disputes and wars. During these periods, cement production could increase leading to increased CO 2 . Historical experience shows that during and after periods of conflicts, wars, and economic crises, economic construction investments accelerate. Overlook is construction and cement are cyclical and subject to nonlinearity and so are the emissions. Another point is that expansions are relatively longer lasting compared to recessions. Such economic growth periods lead to inclined cement production and CO 2 emissions. It is convenient to accept that economic recessions and crises are periods during which the policymakers aim to encourage the economy with expansionary economic policies. During such policies, construction and cement production is an important sector to achieve economic growth. Last but not least, depending on the stage of the economy, the long-run relationship between cement consumption and CO 2 emissions is also bound by the state of the economy.

History also shows that wars generate demand for construction during the process and the reconstruction period afterward. The statistics for the last century confirm that global cement production has amplified more than 30 times when economic growth accelerated in the 1950s following World War II (WWII) and the demand for cement production fast-tracked due to urban reconstruction of the after-war Europe and participant countries of WWII (Diefendorf 1989 ). Further, during the expansionary period after the 1980s and 1990s, cement consumption achieved a second period of upward trend, an increase of cement production nearly 4 times in the post-1980s and 1990s, and yearly cement production reached 0.5 tons per person in the world in mid-2010 (Andrew 2018 ). The post-1980s period corresponds to trade liberalization and globalization policies in the world. Along with post-conflict periods, cement production accelerates after economic recessions and after economic crises. The prominent crises include the Great Depression in 1929, the oil crises and their aftereffects after 1973, the exchange rate mechanism (ERM) in the late 2000s, the Southeast Asia crisis in the mid-1990s, and the Great Recession in 2008, which followed economic policies aimed at acceleration of economic growth and CO 2 (M. Bildirici & Ersin 2018b ). More recently, following the COVID-19 pandemic, nations also applied economic growth policies economic sudden-stop in early 2020. Following lock-downs the end of 2020 and year 2021 experienced high inclines in greenhouse gases and the recent economic recovery from COVID-19 in 2021 is a “carbon-intensive recovery” with a more than 1200-Mt increase in CO 2 releases in a single year (M. Bildirici & Ersin 2018b ). This incline in emissions has been drastically more than those observed during the recovery periods following the 2009 financial crisis (M. Bildirici & Ersin 2018b ). As a result, the econometric models focusing on cement and CO 2 emissions require the integration of nonlinear dynamics taken expansionary and recessionary regimes in the long-run relations.

As shown in the empirical section, the cement-induced CO 2 emission fluctuations are in close synchronization with economic business cycles. Furthermore, the cement industry’s CO 2 emissions are closely related to the source of energy. Figure  2 depicts the CO 2 emissions resulting from cement and other industries in addition to the CO 2 emissions from different sources of energy, specifically focusing on the fossil-fuel energy types including oil, coal, gas, and flaring in the USA for the 1920–2022 period. The overlook suggests an upward trend of CO 2 emissions from cement historically, similar to the upward trend followed by CO 2 emissions from different energy sources. For the whole period, oil is a major emitter with a nonreversing upward trend. After WWII, both oil- and coal-based CO 2 emissions continued to incline similar to cement-induced CO 2 emissions. While the pace of oil- and coal-based CO 2 slowed down, especially for coal after the 2000s, gas and flaring became the major emitters of CO 2 as the economic development and the necessary energy inclined. The slowing pace of the upward trend of oil- and coal-based CO 2 is coupled with cement-induced CO 2 . Among all CO 2 sources, Fig.  2 also depicts the sharp fluctuations in the 1929 Great Depression, 1973 Oil Crisis, and 2008–2009 Global Crisis.

Source : Global Carbon Budget

CO 2 emissions in the USA resulting from cement and other industries and CO 2 emissions produced by different energy sources, 1920–2022.

Figure  3 aims to provide a focused look at the comparison between total territorial CO 2 emissions and cement-induced CO 2 emissions in the USA. The overlook highlights the close ties between the CO 2 emissions of the economy and the CO 2 emissions from the cement industry. The total territorial CO 2 emissions (in orange, values on the left axis) followed an upward trend towards the 1970s, which is closely followed by the cement-induced CO 2 emissions. The period ended with the transformation of industrial production by reducing the dependence on oil, which occurred following the 1973 Oil Crisis in the USA. The total CO 2 emissions gained back its rally in the early 1980s for almost 3 decades without a significant interruption, except for minor slow-downs during economic recessions, which are quite negligible. The same pattern is followed by the cement-induced CO 2 . The rally of CO 2 emissions ended after the economic boom in 2007 and after the 2009 crisis after which the USA started to adopt policies to control the enormous level of CO 2 emissions. Though the climb is reversed, the amounts should be carefully addressed. The CO 2 emissions in 2022 were 5000 Mt a year, equal to the amount in the early 1970s when the economy was highly oil-dependent in production and energy. For cement-induced CO 2 emissions, the level reached is far worse; it is close to 45 Mt of CO 2 , far above the levels in the 1970s. CO 2 emissions from cement production are given with the black line in Mt CO 2 .

CO 2 emissions (given in orange, values on the left axis) in the territorial USA, and CO 2 emissions specifically from cement production (black, right axis), in Mt CO 2 , 1960–2022.

The data in Fig.  3 focuses specifically on the CO 2 produced by the production processes and avoids the energy consumption in the industry. Combined with it, the total effect depends on the type of energy consumed, and the relative dependence on nonrenewable energy sources is quite high in the USA. The overall result is that the fluctuations in total CO 2 emissions in the USA and the fluctuations in the cement-induced CO 2 emissions from the cement industry are highly synchronized, with a strong positive correlation (rho = 0.76). The fluctuations in cement-induced CO 2 are more pronounced; they closely capture the recessions and crises with significant volatility. Years 1974 and 1981 as well as 2008 denote the most drastic drops in CO 2 from cement production after a significant reduction in economic production, corresponding to deep economic crises and dispute periods; however, it gained its pace back again in emitting. The examination of Figs. 2 and 3 together confirms the empirical findings of the study and the conclusions. The overall impact of cement production on CO 2 emissions is positive and is regime-dependent for its positive effect on worsening environmental pollution.

Cement production as a driver of economic growth

The overall results in this study also confirm the cement industry’s role in driving economic growth indirectly since the economic expansion periods coincide with expansionary periods in cement production. This study does not directly investigate cement production and economic growth relation empirically and for such treatment, readers are referred to M. E. Bildirici ( 2019 , 2020 ). However, our study points to the industry’s CO 2 emissions and their relation to cement production cycles characterized by expansionary and contractionary regimes. The findings have important implications. Given the environmental repercussions of the cement sector, our study determines that it is imperative to devise effective environmental remedies and the cement industry should be in the focus not only in terms of its connection to CO 2 and GHG effect but also in terms of its strong ties with economic growth and business cycles. Hence, the results confirm the necessity to invest in greening cement production technology and to increase energy efficiency in addition to production techniques in cement.

Policy recommendations

We provided a set of policy aspects in the previous section, which include improving energy efficiency, renewable energy investments, and taking feedback effects in control, especially in relatively higher cement production regimes. Therefore, policies should focus on the reduction of emissions in the sector more with new techniques in cement production, especially during such periods. An interconnected approach is needed that concentrates both on economic and green cement production aspects (Poudyal & Adhikari 2021 ). A potential reduction in cement production might correlate with slowed economic growth, which is an undesired option for economic policymakers. Therefore, though noting such effects, the reduction of emissions of cement through technology requires immediate action. For such technologies, a set of recent research suggests various methods to reduce the environmental effects of cement production. These include negative emission technologies and decarbonization of the industry (Ren et al. 2023 ), carbon capture and storage, and nanomaterials and supplementary materials to be used as cement complementary in cement production (Poudyal & Adhikari 2021 ). The effects of renewable energy and urbanization (Danish et al. 2020 ) as well as innovation focusing on reducing sectoral emissions in construction have been shown in the literature (Erdoğan et al. 2020 ). In the context of cement production, policies focusing on sustainable urbanization and clean energy could contribute to the reduction of the indirect emissions led by the cement industry.

For the cement industry, one of the remedies lies in transitioning towards renewable energy sources to replace fossil fuels for cement manufacturing and the high amounts of energy consumption during the production process. To facilitate this transition, policymakers should establish a subsidy program incentivizing companies to adopt renewable energy technologies. Another policy recommendation is to accelerate investments and research and development for energy efficiency in the sector. In a country-wide analysis, energy efficiency surges are shown to have stronger positive effects on the environment compared to renewables. However, it should be kept in mind that there are different forms of renewable energy sources with varying effects on the environment. Further, it is shown that the transition to renewable energy is costly and benefits could be achieved only in the long run (M. Bildirici & Ersin 2023 ). Policies should re-evaluate the thresholds to achieve and the timeline for net zero carbon transition. This requires steps to be taken to reduce the significant amount of CO 2 emissions of the industry faster than it is planned in the USA. Lastly, governments addressing environmental issues can achieve preventative health benefits by averting certain illnesses by reducing GHG emissions; the results indicate the need for focusing on the role of industries, and among these, the cement industry is the top third polluter in addition to its production techniques that require a significant amount of energy. To reverse adverse health effects in addition to considering the ties of economic growth and cement production, the cement-induced CO 2 emissions led to a significant amount of CO 2 emissions, though some steps are taken in the production techniques of cement compared to the pre-1950s; however, more efforts should be made on production technologies. Such focus can contribute to the formulation of strategies for proactive public health measures, which should include the cement industry, its relation to environmental sustainability economic growth, and energy in the context of the environment-health nexus.

The cement production activities directly cause emissions during the cement production activities and the size of emissions of such direct emissions deserve special attention for achieving sustainable environment and economic development. The investigation of the long-run effects of cement production is crucial for global warming and climate change. Given the nonlinear nature of the CO 2 emissions and cement production datasets, the paper aimed at providing a hybrid approach that integrated the Markov-switching models to the ARDL-type cointegration methodology to achieve methods to provide tools to examine the nonlinear and regime-dependent long-run and short-run effects in addition to nonlinear causality modeling. By utilizing a historically long period, covering the 1900–2021 sample, the cement production and its effects on greenhouse gas emissions were examined for the USA. The investigation of the relationship is of crucial importance for sustainability in economic development and the environment since the level of hazardous emissions in the cement production industry is among the top third polluters.

In this study, by employing the Markov-switching to the ARDL analysis, the regression space is divided into two distinct regimes. Regime 1 represents periods of deep recessions and crises, while regime 2 characterizes expansionary periods in the industry. The latter has historically lasted relatively longer in terms of duration. The empirical findings show that cement production has significant positive effects both in the short run and in the long run in both regimes. Further, the regime dating provided by the MS-ARDL model yields insightful findings. The years classified under the crisis regime closely align with severe economic recessions and crises as well as wards such as the 1973 Oil Crisis and World Wars I and II. Regime 2 is notable for encompassing events such as the 1929 Great Depression, the 1973 Oil Crisis, the 2009 Great Recession, and more recently, the economic shutdown resulting from the COVID-19 pandemic. During such periods, especially after the crises, economic policies aim to revive the economic growth back to its track, and the cement industry, being closely tied to economic cycles, demonstrates resilience in short-lived crises. It is observed that cement production persists during shorter recessions, with production continuing and sector inventories being maintained unless the recession is severe and prolonged. Furthermore, before exiting recessionary periods, both cement production and associated CO 2 emissions from cement production revert to an increasing trend at a faster pace than the previous. Consequently, cement-induced CO 2 emissions do not decelerate at all in both regimes and emissions continue to escalate as long as cement production activities persist.

Following the generalization of the ARDL model to the novel MS-ARDL, the model is further extended to the MS-VARDL model that capture nonlinear causality relations and the directions of the causal links among the variables analyzed. The novel MS-VARDL method in this study incorporates regime-dependent causal relationships, testing regime-dependent causal directions, which are vital for the determination of the causality direction between cement production and CO 2 emissions for policy formation. Therefore, the nonlinear causality analyses developed in this study overcome inefficient causal relations in tests that ignore the regime-dependent and nonlinear dynamics in the analyzed series. Further, avoiding such aspects of data would result in incorrectly determined causality directions, leading to inefficient policy recommendations. Hence, the testing of causality with the novel method is of paramount importance, especially for the environment-related emissions dataset and the cement production series subject to nonlinearity in this study.

The regime-specific causality results from the MS-VARDL approach revealed positive and causal effects of cement production in both regimes though the magnitude varies depending on the regime. The findings align with the findings we obtained from the single-regime (i.e., linear) VAR-type causality analysis in terms of capturing the direction of causality in a general sense. However, the regime distinction made in causality testing revealed significant information over the traditional method. The magnitude of the causal effect is notably amplified in the crisis regime, while maintaining causality also in the expansion regime, but with a lower positive effect. Additionally, the methodology employed in the study identified bi-directional causal effects between cement production and CO 2 emissions, particularly evident in the second regime, suggesting feedback effects between emissions and cement production.

The determination of regime-specific causal effects provided important policy suggestions for the policymakers focusing on the cement industry and its environmental impacts. Accordingly, regime dependence on the causal links necessitates regime-specific policy measures and if the policymaker utilizes traditional approaches, the magnitude of the industry is estimated to be relatively lower than it is in reality. It should also be noted that the USA has a strong commitment to net-zero policies specifically designed for the cement industry towards lowering emissions to net-zero in 2050. However, the findings in our study highlight the underestimation of the severity of greenhouse gas emissions with traditional methods. Given this fact, policy recommendations of the paper underline the necessity of stronger measures towards net-zero policies, state-level government subsidies to achieve greater commitment to renewable energies share in total energy, subsidies to direct investments in the carbon-capture industry, subsidies to the cement industry to achieve pace in energy efficiency, and giving incentives to internalize emission externalities by the industry. Further, as listed in the policy recommendation section, a set of technological improvements are needed to revise the ongoing GHG effects of the cement industry to achieve a sustainable environment and sustainable economic development. Increasing the speed of energy transition towards renewable energy, and reduction of energy consumption with more energy-efficient solutions in the sector, coupled with net-zero policies and technological investments for supplementary material use in the cement industry are among the measures to be taken.

The feedback relation that produces cycles of GHG emissions, especially in the second regime deserves attention. The strong positive ties with economic cycles and industrial production of the industry generate significant amounts of CO 2 emissions, especially during such periods. The reduction of economic growth is one option but it is not a desired one. Conversely, to cover environmental costs, investments in various emission reduction technologies such as carbon capture and storage are among the viable options that should be taken into focus by policymakers.

The study has limitations due to the availability of data. The nonlinear method utilized in this study requires a long span of data and a large sample size; hence, the study’s empirical focus is restricted to the USA and the 1900–2021 period. As a result, China and India were not examined. For future studies, we suggest extending the analysis to a larger set of countries. Another suggestion is to examine the nexus with a panel of countries.

Data availability

Data used in this study are publicly available from the quoted databases reported under the data subheading given in the empirical section. Data are also available upon request from the corresponding author.

Bildirici and Ersin ( 2023 ) investigate a set of countries who engage on Industry 4.0 with highest shares of Industry 4.0 innovations in addition to large shares of international trade and the share of fossil-fuel energy use in these countries are interestingly high, 82% in the USA, 87.67% in China, 93.03% in Japan, 78% in Germany, 74% in Canada, and 80% in UK. Lowest practice is achieved by France with 46% of fossil-fuel energy use in total.

China leads global cement production by a significant margin, estimated at 2.1 billion metric tons in 2022, representing over half of the world’s total cement output. India follows as the second-largest cement producer globally, with production totaling 370 million metric tons in 2022. Vietnam ranked third in global cement production for that year, producing 120 million metric tons (source: https://www.statista.com/statistics/1087115/global-cement-production-volume/ ).

Such innovations will provide important insights to reduce the emission effects of cement production. However, due to the focus of the study, this literature is kept out of focus in the study.

The USA, which produces cement throughout 37 states, is the third-largest cement manufacturer in the world as of 2000. In addition to being a major source of industrial process-related emissions in the USA, cement production also contributes significantly to CO 2 emissions not only from processing, but also from combustion. The process necessitates high temperatures which is achieved largely with 12% fuel burning, 72% with coal energy, and the rest with various sources including natural gas, coke, and oil (Hanle et al. 2004 ).

The empirical findings revealed the nonlinear association between cement production, air pollution, economic growth, and mortality rate with Bayesian MS-VAR and MS-Bayesian Granger causality tests for the USA, Turkey, India, Brazil, and China and findings confirmed cement production as Granger causes not only air pollution and economic growth, but also mortality rates in all regimes for all countries analyzed (M. E. Bildirici 2020 ).

Following these critiques regarding the efficiency of the test, McNown et al. ( 2018 ) proposed a bootstrapped ARDL (BARDL) test as an accompanying test to the F PSS test and the bound critical values of Pesaran et al. ( 2001 ). In the proposed BARDL method (McNown et al. 2018 ), the utilization of three tests are suggested as compulsory to determine the presence of no-cointegration, cointegration, and degenerate cases. In the state of degenerate-2, the supplied bound critical values of Pesaran et al. ( 2001 ) are inefficient and under degenerate case-1, the method is effective.

The STARDL method relaxes the form of nonlinearity by allowing nonlinearity both in the short-run and long-run dynamics and derive the relevant tests. For the STARDL modeling and relevant nonlinear cointegration testing, readers are referred to M. Bildirici and Ersin (2018b). The model generalizes STAR models of Granger and Teräsvirta ( 1993 ) and Luukkonen et al. ( 1988 ) to nonlinear ARDL.

Further, STAR type nonlinear models allow capturing different speed of transition between regimes, both smooth and abrupt, However, MS models and threshold models are efficient under abrupt and sudden regime shifts.

The MS-VEC model is applied to environmental degradation, hydropower, and nuclear energies for their asymmetric long- and short-run relations (Bildirici 2020 ).

Note that the representation above is a theoretical representation and Davies type linearity tests to determine the number of regimes generally selects 2 regimes and for selected set of applications, 3 regimes.

These techniques, in contrast to conventional vector autoregressive (VAR) and vector error correction (VEC) models, also made it possible to evaluate the model without using dummy variables and to examine various stages of the economy.

The model presented is a Markov-Switching Intercept and Heteroskedasticity (MSIH-VEC), no regime switching allowed in the long-run and short-run parameters. However, due to the regime switching being introduced to the drift term, the filtered residual is already affected from the changes in drift.

The pioneering papers that provide a historical perspective (Grossman & Krueger 1991 ; Stern et al. 1996 ) are generalized to nonlinearity with a set of studies. Studies suggest a coverage of centuries long data to examine nonlinear relations among and cycles of emission and GDP growth rates historically (M. Bildirici & Ersin 2018b ; Ersin 2016 ).

1954 and 1961 have 10 months lasted recessions; 1958, 1991, and 2001 recessions lasted 8 months; 1970 recession lasted 11 months. After 1970, 1974 recession lasted 16 months, relatively a long-lasting recession which followed the oil crisis. A deep recession occurred between 1975 and mid 1976. In early 1980s, policies included trade liberalization policies in the world which had implications as a short recession that did not affect the cement industry. A difference in the table is 1982–1983 recession which lasted 16 months. However, the model did not capture a contraction in the period. If data investigated, we noted no drastic decline in the cement production for this period, providing a special case.

In addition, following the proposal given in the “Methodology” section, we calculated F MS-ARDL statistics for single regimes only. Given that the model has 2 variables and 2 states, after estimating an MS(2)-VARDL(2) model, 4 hypotheses are tested by applying zero restrictions. At 5% significance level, results suggest that if emission variable is the dependent variable, cointegration cannot be rejected for regime 1 and regime 2 in addition to nonlinear cointegration in both regimes. In contrast, if cement production is taken as the dependent variable, there exists no cointegration in regime 1, inconclusive result in regime 2, and no cointegration in both regimes, i.e., no nonlinear cointegration.

A model with 3 regimes is also estimated; however, the estimation led to inconsistent results. Though the sample covers a long period of 1900–2021, the dataset is yearly and sample size restricts estimation of nonlinear models with more than 2 regimes. Therefore, two regimes are assumed.

To save space, the MS-VARDL estimation results are reported. They are available upon request. In the determination of signs of causality, the statistical significance of the parameters, sign, and size of parameter estimates are crucial.

Alkama R, Cescatti A (2016) Climate change: biophysical climate impacts of recent changes in global forest cover. Science 351(6273):600-604.   https://doi.org/10.1126/SCIENCE.AAC8083

Article   CAS   Google Scholar  

Andrew RM (2018) Global CO 2 emissions from cement production. Earth Syst Sci Data 10:195-217.  https://doi.org/10.5194/essd-10-195-2018

Article   Google Scholar  

Andrew RM (2022) Global CO 2 emissions from cement production. Zenodo. https://zenodo.org/records/7081360#.Y8GagnZBy5d . Accessed 1 May 2024

Anjum Z, Burke P, Gerlagh R, Stern D, Anjum Z, Burke P, Gerlagh R, Stern D (2014) Modeling the emissions-income relationship using long-run growth rates. Centre Clim Energy Policy, Crawford Sch Public Policy, Australian Natl Univ. https://doi.org/10.22004/ag.econ.249422

Banerjee P, Arčabić V, Lee H (2017) Fourier ADL cointegration test to approximate smooth breaks with new evidence from Crude Oil Market. Econ Model 67:114-124.  https://doi.org/10.1016/J.ECONMOD.2016.11.004

Bekun FV, Alola AA, Gyamfi BA, Kwakwa PA, Uzuner G (2022) Econometrics analysis on cement production and environmental quality in European Union countries. Int J Environ Sci Technol.  https://doi.org/10.1007/s13762-022-04302-9

Belbute JM, Pereira AM (2020) Reference forecasts for CO2 emissions from fossil-fuel combustion and cement production in Portugal. Energy Policy 144:111642.  https://doi.org/10.1016/J.ENPOL.2020.111642

Bildirici ME (2019) Cement production, environmental pollution, and economic growth: evidence from China and USA. Clean Technol Environ Policy 21(4):783-793.  https://doi.org/10.1007/s10098-019-01667-3

Bildirici ME (2020) The relationship between cement production, mortality rate, air quality, and economic growth for China, India, Brazil, Turkey, and the USA: MScBVAR and MScBGC analysis. Environ Sci Pollut Res 27:2248-2263.  https://doi.org/10.1007/s11356-019-06586-w

Bildirici M, Ersin Ö (2018a) Markov-switching vector autoregressive neural networks and sensitivity analysis of environment, economic growth and petrol prices. Environ Sci Pollut Res 25(31):31630-31655.  https://doi.org/10.1007/s11356-018-3062-3

Bildirici M, Ersin ÖÖ (2018b) Economic growth and CO2 emissions: an investigation with smooth transition autoregressive distributed lag models for the 1800–2014 period in the USA. Environ Sci Pollut Res 25(1):200-219.  https://doi.org/10.1007/s11356-017-0244-3

Bildirici M, Ersin ÖÖ (2023) Nexus between Industry 4.0 and environmental sustainability: a Fourier panel bootstrap cointegration and causality analysis. J Clean Prod 386(135786):1-18.  https://doi.org/10.1016/J.JCLEPRO.2022.135786

Boden TA, Marland G, Andres RJ (1999) Global, regional, and national fossil-fuel CO2 emissions (1751–2014) (V. 2017). https://doi.org/10.3334/CDIAC/00001_V2017

Broock WA, Scheinkman JA, Dechert WD, LeBaron B (1996) A test for independence based on the correlation dimension. Economet Rev 15(3):197-235.  https://doi.org/10.1080/07474939608800353

Cai B, Wang J, He J Geng Y (2016) Evaluating CO2 emission performance in China’s cement industry: an enterprise perspective. Appl Energy 166:191-200.  https://doi.org/10.1016/j.apenergy.2015.11.006

Cao Y, Guo L, Qu Y (2022) Evaluating the dynamic effects of mitigation instruments on CO2 emissions in China’s nonferrous metal industry: a vector autoregression analysis. Sci Total Environ 853. https://doi.org/10.1016/j.scitotenv.2022.158409

Chen W, Yan S (2022) The decoupling relationship between CO2 emissions and economic growth in the Chinese mining industry under the context of carbon neutrality. J Clean Prod 379. https://doi.org/10.1016/j.jclepro.2022.134692

Clements MP, Krolzig HM (2002) Can oil shocks explain asymmetries in the US Business Cycle? Advances in Markov-Switching Models 41–60. https://doi.org/10.1007/978-3-642-51182-0_3

Costa FN, Ribeiro DV (2020) Reduction in CO2 emissions during production of cement, with partial replacement of traditional raw materials by civil construction waste (CCW). J Clean Prod 276. https://doi.org/10.1016/J.JCLEPRO.2020.123302

Danish, Ulucak R, Khan SUD (2020) Determinants of the ecological footprint: role of renewable energy, natural resources, and urbanization. Sustain Cities Soc 54:101996. https://doi.org/10.1016/J.SCS.2019.101996

DATIS (2020) Worldwide cement production from 2015 to 2019. Datis export group. https://datis-inc.com/blog/worldwide-cement-production-from-2015-to-2019/#United_States . Accessed 1 May 2024

Diefendorf JM (1989) Urban reconstruction in Europe after World War II. Urban Studies 26(1):128-143

Dirik C, Şahin S, Engin P (2019) Environmental efficiency evaluation of Turkish cement industry: an application of data envelopment analysis. Energ Effi 12(8):2079-2098.  https://doi.org/10.1007/S12053-018-9764-Z/TABLES/6

Enders W, Lee J (2012) A unit root test using a Fourier series to approximate smooth breaks. Oxford Bull Econ Stat 74(4):574-599.  https://doi.org/10.1111/J.1468-0084.2011.00662.X

Engle RF, Granger CWJ (1987) Co-integration and error correction: representation, estimation, and testing. Appl Econ 39(3):107-135.  https://doi.org/10.2307/1913236

EPA (2023) Sources of greenhouse gas emissions and removals. United States Environmental Protection Agency. https://www.epa.gov/ghgemissions/overview-greenhouse-gases . Accessed 17 Jan 2024

Erdoğan S, Yıldırım S, Yıldırım DÇ, Gedikli A (2020) The effects of innovation on sectoral carbon emissions: evidence from G20 countries. J Environ Manage 267:110637.  https://doi.org/10.1016/J.JENVMAN.2020.110637

Ersin ÖÖ (2016) The nonlinear relationship of environmental degradation and income for the 1870–2011 period in selected developed countries: the dynamic panel-STAR approach. Procedia Econ Finance 38:318-339.  https://doi.org/10.1016/S2212-5671(16)30205-2

Forzieri G, Alkama R, Miralles DG, Cescatti A (2017) Satellites reveal contrasting responses of regional climate to the widespread greening of Earth. Science 356(6343):1180-1184.  https://doi.org/10.1126/SCIENCE.AAL1727

Gao T, Shen L, Shen M, Liu L, Chen F, Gao L (2017) Evolution and projection of CO2 emissions for China’s cement industry from 1980 to 2020. Renew Sustain Energy Rev 74:522-537.  https://doi.org/10.1016/j.rser.2017.02.006

Granger C, Teräsvirta T (1993) Modelling non-linear economic relationships. Oxford University Press, Oxford

Grossman GM, Krueger AB (1991) Environmental impacts of a North American free trade agreement. Natl Bur Econ Res Work Pap Ser 3914(3914):1-57.  https://doi.org/10.3386/w3914

Hamilton JD (1989) A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57(2):357.  https://doi.org/10.2307/1912559

Hanle LJ, Jayaraman KR, Smith JS (2004) CO 2 emissions profile of the U.S. cement industry (working papers). https://www3.epa.gov/ttnchie1/conference/ei13/ghg/hanle.pdf . Accessed 12 Jan 2024

Hoffman FM, Randerson JT, Arora VK, Bao Q, Cadule P, Ji D, Jones CD, Kawamiya M, Khatiwala S, Lindsay K, Obata A, Shevliakova E, Six KD, Tjiputra JF, Volodin EM, Wu T (2014) Causes and implications of persistent atmospheric carbon dioxide biases in Earth System Models. J Geophys Res Biogeosci 119(2):141-162.  https://doi.org/10.1002/2013JG002381

Huang PJ, Huang SL, Marcotullio PJ (2019) Relationships between CO2 emissions and embodied energy in building construction: a historical analysis of Taipei. Build Environ 155:360-375.  https://doi.org/10.1016/J.BUILDENV.2019.03.059

IEA (2022a) Southeast asia energy outlook 2022: key findings. International energy agency world energy outlook 2022. https://www.iea.org/reports/southeast-asia-energy-outlook-2022 . Accessed 1 May 2024

IEA (2022b) Renewable energy market update 2022: outlook for 2022 and 2023. International energy agency IEA 50. https://www.iea.org/reports/renewable-energy-market-update-may-2022 . Accessed 15 Jan 2024

IEA (2022c) World energy outlook 2022 Extended dataset. International energy agency IEA 50. https://www.iea.org/data-and-statistics/data-product/world-energy-outlook-2022-extended-dataset . Accessed 2 Feb 2024

Kapetanios V, Shin Y, Snell A (2006) Testing for cointegration in nonlinear smooth transition error correction models. Economet Theor 22(2):279-303.  https://doi.org/10.1017/S0266466606060129

Karlsson I, Rootzén J, Johnsson F (2020) Reaching net-zero carbon emissions in construction supply chains — analysis of a Swedish road construction project. Renew Sustain Energy Rev 120:109651.  https://doi.org/10.1016/J.RSER.2019.109651

Karlsson I, Rootzén J, Johnsson F, Erlandsson M (2021) Achieving net-zero carbon emissions in construction supply chains — a multidimensional analysis of residential building systems. Dev Built Environ 8:100059.  https://doi.org/10.1016/J.DIBE.2021.100059

Ke J, Zheng N, Fridley D, Price L, Zhou N (2012) Potential energy savings and CO2 emissions reduction of China’s cement industry. Energy Policy 45:739-751.  https://doi.org/10.1016/J.ENPOL.2012.03.036

Krolzig HM, Marcellino M, Mizon GE (2002) A Markov-switching vector equilibrium correction model of the UK labour market. Empir Econ 27(2):233-254.  https://doi.org/10.1007/s001810100117

Krolzig HM, Toro J (2002) Testing for super-exogeneity in the presence of common deterministic shifts. Annales d’Économie et de Statistique 67/68:41–71. https://doi.org/10.2307/20076342

Krolzig HM (1997) The Markov-switching vector autoregressive model. In Lecture Notes in Economics and Mathematical Systems (45):46–28. https://doi.org/10.1007/978-3-642-51684-9_2

Kumar Mandal S, Madheswaran S (2010) Environmental efficiency of the Indian cement industry: an interstate analysis. Energy Policy 38(2):1108–1118. https://doi.org/10.1016/j.enpol.2009.10.063

Le Quéré C, Andrew RM, Friedlingstein P, Sitch S, Pongratz J, Manning AC, Ivar Korsbakken J, Peters GP, Canadell JG, Jackson RB, Boden TA, Tans PP, Andrews OD, Arora VK, Bakker DCE, Barbero L, Becker M, Betts RA, Bopp L, … Zhu D (2018) Global carbon budget 2017. Earth Syst Sci Data 10(1):405–448. https://doi.org/10.5194/ESSD-10-405-2018

Lei Y, Zhang Q, Nielsen C, He K (2011) An inventory of primary air pollutants and CO2 emissions from cement production in China, 1990–2020. Atmos Environ 45(1):147-154.  https://doi.org/10.1016/J.ATMOSENV.2010.09.034

Li Y, Zuo Z, Cheng Y, Cheng J, Xu D (2023) Towards a decoupling between regional economic growth and CO2 emissions in China’s mining industry: a comprehensive decomposition framework. Resour Policy 80:103271.  https://doi.org/10.1016/J.RESOURPOL.2022.103271

Liang H, Lin S, Wang J (2022) Impact of technological innovation on carbon emissions in China’s logistics industry: based on the rebound effect. J Clean Prod 377. https://doi.org/10.1016/j.jclepro.2022.134371

Lin B, Teng Y (2022) Decoupling of economic and carbon emission linkages: evidence from manufacturing industry chains. J Environ Manag 322. https://doi.org/10.1016/j.jenvman.2022.116081

Lin B, Zhang Z (2016) Carbon emissions in China’s cement industry: a sector and policy analysis. Renew Sustain Energy Rev 58:1387-1394.  https://doi.org/10.1016/J.RSER.2015.12.348

Liu Y, Wang J, Wang X, Wu H, Guo F, Song Y (20220 A study of CO2 emissions in China’s domestic construction industry based on non-competitive input-output. Sustain Prod Consum 32:743-754.  https://doi.org/10.1016/j.spc.2022.05.024

Liu M, Zhang X, Zhang M, Feng Y, Liu Y, Wen J, Liu L (2021) Influencing factors of carbon emissions in transportation industry based on C–D function and LMDI decomposition model: China as an example. Environ Impact Assess Rev 90. https://doi.org/10.1016/j.eiar.2021.106623

Lopez R (1994) The environment as a factor of production: the effects of economic growth and trade liberalization. J Environ Econ Manag 27(2):163-184.  https://doi.org/10.1006/jeem.1994.1032

Luukkonen R, Saikkonen P, Teräsvirta T (1988) Testing linearity against smooth transition autoregressive models. Biometrika 75(3):491-499

Martins MA de B, Crispim A, Ferreira ML, dos Santos IF, Melo M de L NM, Barros RM, Filho GLT (2023) Evaluating the energy consumption and greenhouse gas emissions from managing municipal, construction, and demolition solid waste. Clean Waste Syst 4:100070. https://doi.org/10.1016/J.CLWAS.2022.100070

McNown R, Sam CY, Goh SK (2018) Bootstrapping the autoregressive distributed lag test for cointegration. Appl Econ 50(130):1509-1521.  https://doi.org/10.1080/00036846.2017.1366643

Mishra A, Humpenöder F, Churkina G, Reyer CPO, Beier F, Bodirsky BL, Schellnhuber HJ, Lotze-Campen H, Popp A (2022) Land use change and carbon emissions of a transformation to timber cities. Nat Commun 13(1):1-12.  https://doi.org/10.1038/s41467-022-32244-w

Naqi A, Jang JG (2019) Recent progress in green cement technology utilizing low-carbon emission fuels and raw materials: a review. Sustainability 11(2):537.  https://doi.org/10.3390/SU11020537

Narayan PK (2014) Reformulating critical values for the bounds F-statistics approach to cointegration: an application to the tourism demand model for Fiji. Monash Univ Dept Econ Discuss Pap 02(04):1-39.  https://doi.org/10.4225/03/5938ABDA7B4AB

Nelson CR, Piger J, Zivot E (2001) Markov regime switching and unit-root tests. J Bus Econ Stat 19(4):404-415.  https://doi.org/10.1198/07350010152596655

Oke AE, Aigbavboa CO, Dlamini SA (2017) Carbon emission trading in South African construction industry. Energy Procedia 142:2371-2376.  https://doi.org/10.1016/J.EGYPRO.2017.12.169

Ozturk I, Acaravci A (2013) The long-run and causal analysis of energy, growth, openness and financial development on carbon emissions in Turkey. Energy Econ 36:262-267.  https://doi.org/10.1016/j.eneco.2012.08.025

Pakdel A, Ayatollahi H, Sattary S (2021) Embodied energy and CO2 emissions of life cycle assessment (LCA) in the traditional and contemporary Iranian construction systems. J Build Eng 39:102310.  https://doi.org/10.1016/J.JOBE.2021.102310

Pavlyuk D (2017) On application of regime-switching models for short-term traffic flow forecasting. In: Zamojski W, Mazurkiewicz J, Sugier J, Walkowiak T, Kacprzyk J (eds) Advances in dependability engineering of complex systems. DepCoS-RELCOMEX 2017. Advances in intelligent systems and computing, vol 582. Springer, Cham. pp 340–349. https://doi.org/10.1007/978-3-319-59415-6_33

Pesaran HM, Shin Y, Smith RJ (2001) Bounds testing approaches to the analysis of level relationships. J Appl Economet 16:289-326.  https://doi.org/10.1002/jae.616

Pettersson F, Maddison D, Acar S, Söderholm P (2014) Convergence of carbon dioxide emissions: a review of the literature*. Int Rev Environ Resour Econ 7(2):141-178.

Poudyal L, Adhikari K (2021) Environmental sustainability in cement industry: an integrated approach for green and economical cement production. Res Environ Sustain 4. https://doi.org/10.1016/J.RESENV.2021.100024

Ren M, Ma T, Fang C, Liu X, Guo C, Zhang S, Zhou Z, Zhu Y, Dai H, Huang C (2023) Negative emission technology is key to decarbonizing China’s cement industry. Appl Energy 329. https://doi.org/10.1016/j.apenergy.2022.120254

Richardson AD, Keenan TF, Migliavacca M, Ryu Y, Sonnentag O, Toomey M (2013) Climate change, phenology, and phenological control of vegetation feedbacks to the climate system. Agric for Meteorol 169:156-173.  https://doi.org/10.1016/J.AGRFORMET.2012.09.012

Rodgers L (2018) Climate change: the massive CO 2 emitter you may not know about. BBC News. https://www.bbc.com/news/science-environment-46455844 . Accessed 1 May 2024

Saikkonen P (2005) Stability results for nonlinear error correction models. J Econ 127(1):69-81.  https://doi.org/10.1016/j.jeconom.2004.03.001

Saikkonen P (2008) Stability of regime switching error correction models under linear cointegration. Economet Theor 24(1):294-318.  https://doi.org/10.1017/S0266466608080122

Selden TM, Song D (1994) Environmental quality and development: is there a Kuznets curve for air pollution emissions? J Environ Econ Manag 27(2):147-162.  https://doi.org/10.1006/JEEM.1994.1031

Shin Y, Yu B, Greenwood-Nimmo M (2013) Modelling asymmetric cointegration and dynamic multipliers in a nonlinear ARDL framework. Festschrift Honor Peter Schmidt 44:1-35.  https://doi.org/10.1007/978-1-4899-8008-3

Stern DI (1994) The rise and fall of the environmental Kuznets curve. J Environ Econ Manag 27:147–167.  https://doi.org/10.1016/j.worlddev.2004.03.004

Stern DI, Common SM, Barbier BE (1996) Economic growth and environmental degradation: the environmental Kuznets curve and sustainable development. In World Development 24(7) https://doi.org/10.1016/0305-750X(96)00032-0

Stern DI, Gerlagh R, Burke PJ (2017) Modeling the emissions–income relationship using long-run growth rates. Environ Dev Econ 1–26. https://doi.org/10.1017/S1355770X17000109

Supino S, Malandrino O, Testa M, Sica D (2016) Sustainability in the EU cement industry: the Italian and German experiences. J Clean Prod 112:430-442.  https://doi.org/10.1016/J.JCLEPRO.2015.09.022

Tan C, Yu X, Guan Y (2022) A technology-driven pathway to net-zero carbon emissions for China’s cement industry. Appl Energy 325. https://doi.org/10.1016/j.apenergy.2022.119804

Teller P, Renzoni R, Germain A, Delaisse P, D’Inverno H (2000) Use of LCI for the decision-making of a Belgian cement producer: a common methodology for accounting CO2 emissions related to the cement life cycle. In: 8th LCA Case Studies Symposium SETAC-Europe

The Guardian (2019) Concrete: the most destructive material on Earth. The Guardian. https://www.theguardian.com/cities/2019/feb/25/concrete-the-most-destructive-material-on-earth . Accessed 14 Dec 2023

Tsay RS (1986) Nonlinearity tests for time series. Biometrika 73(2):461.  https://doi.org/10.2307/2336223

UNCC (2022) Climate plans remain insufficient: more ambitious action needed now. United Nations climate change. https://unfccc.int/news/climate-plans-remain-insufficient-more-ambitious-action-needed-now . Accessed 1 May 2024

USGS (2024) National minerals information center: cement statistics and information. U.S. geological survey. https://www.usgs.gov/centers/national-minerals-information-center . Accessed 8 Mar 2024

Vorayos N, Jaitiang T (2020) Energy-environmental performance of Thai’s cement industry. Energy Rep 6:460-466.  https://doi.org/10.1016/J.EGYR.2019.11.103

Wang Y, Zhu Q, Geng Y (2013) Trajectory and driving factors for GHG emissions in the Chinese cement industry. J Clean Prod 53:252-260.  https://doi.org/10.1016/J.JCLEPRO.2013.04.001

WPR (2024) Cement production by country 2024. World population review. https://worldpopulationreview.com/country-rankings/cement-production-by-country . Accessed 2 Jan 2024

Worrell E, Price L, Martin N, Hendriks C, Meida LO (2001) Carbon dioxide emissions from the global cement industry. Annual Rev of Environ and Res 6. https://doi.org/10.1146/annurev.energy.26.1.303

Wu T, Ng ST, Chen J (2022) Deciphering the CO2 emissions and emission intensity of cement sector in China through decomposition analysis. J Clean Prod 352. https://doi.org/10.1016/J.JCLEPRO.2022.131627

Xi F, Davis SJ, Ciais P, Crawford-Brown D, Guan D, Pade C, Shi T, Syddall M, Lv J, Ji L, Bing L, Wang J, Wei W, Yang KH, Lagerblad B, Galan I, Andrade C,Y Zhang Y, Liu Z (2016) Substantial global carbon uptake by cement carbonation. Nat Geosci 9(12):880-883.  https://doi.org/10.1038/ngeo2840

Xia D, Zhang L (2022) Coupling coordination degree between coal production reduction and CO2 emission reduction in coal industry. Energy 258. https://doi.org/10.1016/j.energy.2022.124902

Xin D, Ahmad M, Khattak SI (2022) Impact of innovation in climate change mitigation technologies related to chemical industry on carbon dioxide emissions in the United States. J Clean Prod 379. https://doi.org/10.1016/j.jclepro.2022.134746

Xu JH, Fleiter T, Eichhammer W, Fan Y (2012) Energy consumption and CO2 emissions in China’s cement industry: a perspective from LMDI decomposition analysis. Energy Policy 50:821-832.  https://doi.org/10.1016/J.ENPOL.2012.08.038

Xu B, Xu R (2022) Assessing the role of environmental regulations in improving energy efficiency and reducing CO2 emissions: evidence from the logistics industry. Environ Impact Assess Rev 96. https://doi.org/10.1016/j.eiar.2022.106831

Xu H, Li Y, Zheng Y, Xu X (2022) Analysis of spatial associations in the energy–carbon emission efficiency of the transportation industry and its influencing factors: evidence from China. Environ Impact Assess Rev 97. https://doi.org/10.1016/j.eiar.2022.106905

Yu Y, Yazan DM, Bhochhibhoya S, Volker L (2021) Towards circular economy through industrial symbiosis in the Dutch construction industry: a case of recycled concrete aggregates. J Clean Prod 293. https://doi.org/10.1016/j.jclepro.2021.126083

Zhang Y, Zhang Y, Zhu H, Zhou P, Liu S, Lei X, Li Y, Li B, Ning P (2022) Life cycle assessment of pollutants and emission reduction strategies based on the energy structure of the nonferrous metal industry in China. Energy 261. https://doi.org/10.1016/j.energy.2022.125148

Zhang J, Shen J, Xu L, Zhang Q (2023) The CO2 emission reduction path towards carbon neutrality in the Chinese steel industry: a review. Environ Impact Assess Rev 99. https://doi.org/10.1016/j.eiar.2022.107017

Zhao Q, Gao W, Su Y, Wang T (2022) Carbon emissions trajectory and driving force from the construction industry with a city-scale: a case study of Hangzhou, China. Sustain Cities Soc 104283. https://doi.org/10.1016/j.scs.2022.104283

Zheng J, Chen A, Yao J, Ren Y, Zheng W, Lin F, Shi J, Guan A, Wang W (2022) Combination method of multiple molding technologies for reducing energy and carbon emission in the foundry industry. Sustain Mater Technol e00522. https://doi.org/10.1016/j.susmat.2022.e00522

Zhou Y, Chen M, Tang Z, Mei Z (2021) Urbanization, land use change, and carbon emissions: quantitative assessments for city-level carbon emissions in Beijing-Tianjin-Hebei region. Sustain Cities Soc 66:102701.  https://doi.org/10.1016/J.SCS.2020.102701

Download references

Open access funding provided by the Scientific and Technological Research Council of Türkiye (TÜBİTAK).

Author information

Authors and affiliations.

Department of Economics, Faculty of Economics and Administrative Sciences, Davutpaşa Campus, Yıldız Technical University, 34220, Istanbul, Turkey

Melike E. Bildirici

Dept. of International Trade, Faculty of Business, Sütlüce Campus, İstanbul Ticaret University, 34445, Istanbul, Turkey

Özgür Ömer Ersin

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by Melike Bildirici and Özgür  Ömer Ersin. The first draft of the manuscript was written by Melike Bildirici and Özgür Ömer Ersin. Both authors commented on previous versions of the manuscript and all authors read and approved the final manuscript.

Corresponding author

Correspondence to Özgür Ömer Ersin .

Ethics declarations

Ethical approval and consent to participate.

Not applicable.

Consent for publication

All authors agreed for publication.

Conflict of interest

The authors declare no competing interests.

Additional information

Responsible Editor: Ilhan Ozturk

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Bildirici, M.E., Ersin, Ö.Ö. Cement production and CO 2 emission cycles in the USA: evidence from MS-ARDL and MS-VARDL causality methods with century-long data. Environ Sci Pollut Res (2024). https://doi.org/10.1007/s11356-024-33489-2

Download citation

Received : 01 November 2023

Accepted : 22 April 2024

Published : 10 May 2024

DOI : https://doi.org/10.1007/s11356-024-33489-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Air pollution
• Environmental sustainability
• Markov-switching
• Cointegration

JEL Classification

• Find a journal
• Publish with us
• Track your research