3 stages of literature review

• University of Texas Libraries

Literature Reviews

Steps in the literature review process.

• What is a literature review?
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• You may need to some exploratory searching of the literature to get a sense of scope, to determine whether you need to narrow or broaden your focus
• Identify databases that provide the most relevant sources, and identify relevant terms (controlled vocabularies) to add to your search strategy
• Finalize your research question
• Think about relevant dates, geographies (and languages), methods, and conflicting points of view
• Conduct searches in the published literature via the identified databases
• Check to see if this topic has been covered in other discipline's databases
• Examine the citations of on-point articles for keywords, authors, and previous research (via references) and cited reference searching.
• Save your search results in a citation management tool (such as Zotero, Mendeley or EndNote)
• De-duplicate your search results
• Make sure that you've found the seminal pieces -- they have been cited many times, and their work is considered foundational 
• Check with your professor or a librarian to make sure your search has been comprehensive
• Evaluate the strengths and weaknesses of individual sources and evaluate for bias, methodologies, and thoroughness
• Group your results in to an organizational structure that will support why your research needs to be done, or that provides the answer to your research question  
• Develop your conclusions
• Are there gaps in the literature?
• Where has significant research taken place, and who has done it?
• Is there consensus or debate on this topic?
• Which methodological approaches work best?
• For example: Background, Current Practices, Critics and Proponents, Where/How this study will fit in 
• Organize your citations and focus on your research question and pertinent studies
• Compile your bibliography

Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.

Videos Tutorials about Literature Reviews

This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.

Recommended Reading

• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Literature Reviews

• Getting started

What is a literature review?

Why conduct a literature review, stages of a literature review, lit reviews: an overview (video), check out these books.

• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject.

Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field.

Analysis: Involves critically evaluating and summarizing key findings, methodologies, and debates found in academic literature.

Identifying Gaps: Aims to pinpoint areas where there is a lack of research or unresolved questions, highlighting opportunities for further investigation.

Contextualization: Enables researchers to understand how their work fits into the broader academic conversation and contributes to the existing body of knowledge.

tl;dr  A literature review critically examines and synthesizes existing scholarly research and publications on a specific topic to provide a comprehensive understanding of the current state of knowledge in the field.

What is a literature review NOT?

❌ An annotated bibliography

❌ Original research

❌ A summary

❌ Something to be conducted at the end of your research

❌ An opinion piece

❌ A chronological compilation of studies

The reason for conducting a literature review is to:

Literature Reviews: An Overview for Graduate Students

While this 9-minute video from NCSU is geared toward graduate students, it is useful for anyone conducting a literature review.

Writing the literature review: A practical guide

Available 3rd floor of Perkins

Writing literature reviews: A guide for students of the social and behavioral sciences

Available online!

So, you have to write a literature review: A guided workbook for engineers

Telling a research story: Writing a literature review

The literature review: Six steps to success

Systematic approaches to a successful literature review

Request from Duke Medical Center Library

Doing a systematic review: A student's guide

• Next: Types of reviews >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 22, 2024 12:03 PM
• URL: https://libguides.usc.edu/writingguide

• Franklin University |
• Help & Support |
• Locations & Maps |

• | Research Guides

To access Safari eBooks,

• Select not listed in the Select Your Institution drop down menu.
• Enter your Franklin email address and click Go
• click "Already a user? Click here" link
• Enter your Franklin email and the password you used to create your Safari account.

Continue Close

Literature Review

• Getting Started
• Framing the Literature Review

Literature Review Process

• Mistakes to Avoid & Additional Help

The structure of a literature review should include the following :

• An overview of the subject, issue or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories (e.g. works that support of a particular position, those against, and those offering alternative approaches entirely),
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research

The critical evaluation of each work should consider :

• Provenance  -- what are the author's credentials? Are the author's arguments supported by evidence (e.g. primary historical material, case studies, narratives, statistics, recent scientific findings)?
• Objectivity  -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness  -- which of the author's theses are most/least convincing?
• Value  -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

Development of the Literature Review

Four stages:.

• Introduce the reader to the importance of the topic being studied . The reader is oriented to the significance of the study and the research questions or hypotheses to follow.
• Places the problem into a particular context  that defines the parameters of what is to be investigated.
• Provides the framework for reporting the results  and indicates what is probably necessary to conduct the study and explain how the findings will present this information.
• Literature search -- finding materials relevant to the subject being explored.
• Evaluation of resources  -- determining which literature makes a significant contribution to the understanding of the topic.
• Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review:

Sources and expectations.  if your assignment is not very specific about what form your literature review should take, seek clarification from your professor by asking these questions:.

• Roughly how many sources should I include?
• What types of sources should I review (books, journal articles, websites)?
• Should I summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should I evaluate the sources?
• Should I provide subheadings and other background information, such as definitions and/or a history?

Find Models.   When reviewing the current literature, examine how authors in your discipline or area of interest have organized their literature reviews. Read not only for information, but also to get a sense of the types of themes you might want to look for in your own research review.

Narrow the topic.  the narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources., consider whether your sources are current and applicable.  s ome disciplines require that you use information that is as current as possible. this is very common in the sciences where research conducted only two years ago could be obsolete. however, when writing a review in the social sciences, a survey of the history of the literature may be what is needed because what is important is how perspectives have changed over the years or within a certain time period. try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. you can also use this method to consider what is consider by scholars to be a "hot topic" and what is not., follow the bread crumb trail.  the bibliography or reference section of sources you read are excellent entry points for further exploration. you might find resourced listed in a bibliography that points you in the direction you wish to take your own research., ways to organize your literature review, chronologically:  .

If your review follows the chronological method, you could write about the materials according to when they were published or the time period they cover.

By Publication:  

Order your sources chronologically by publication date, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies.

Conceptual Categories:

The literature review is organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it will still be organized chronologically reflecting technological developments in media. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most.

Methodological:  

A methodological approach focuses on the methods utilized by the researcher.  A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Sections of Your Literature Review:  

Once you've decided on the organizational method for your literature review, the sections you need to include should be easy to figure out because they arise from your organizational strategy.

Here are examples of other sections you may need to include depending on the type of review you write:

• Current Situation : information necessary to understand the topic or focus of the literature review.
• History : the chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : the criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed articles and journals.
• Standards : the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence:

A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be Selective:  

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological.

Use Quotes Sparingly:  

Some short quotes are okay if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Sometimes you may need to quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute your own summary and interpretation of the literature.

Summarize and Synthesize:  

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to their own work.

Keep Your Own Voice:  

While the literature review presents others' ideas, your voice (the writer's) should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording.

Use Caution When Paraphrasing:  

When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

• << Previous: Getting Started
• Next: Mistakes to Avoid & Additional Help >>
• Last Updated: Oct 3, 2023 2:44 PM
• URL: https://guides.franklin.edu/LITREVIEW

Research Process :: Step by Step

• Introduction
• Select Topic
• Identify Keywords
• Background Information
• Develop Research Questions
• Refine Topic
• Search Strategy
• Popular Databases
• Evaluate Sources
• Types of Periodicals
• Reading Scholarly Articles
• Primary & Secondary Sources
• Organize / Take Notes
• Writing & Grammar Resources
• Annotated Bibliography
• Literature Review
• Citation Styles
• Paraphrasing
• Privacy / Confidentiality
• Research Process
• Selecting Your Topic
• Identifying Keywords
• Gathering Background Info
• Evaluating Sources

Organize the literature review into sections that present themes or identify trends, including relevant theory. You are not trying to list all the material published, but to synthesize and evaluate it according to the guiding concept of your thesis or research question.  

What is a literature review?

A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries

A literature review must do these things:

• be organized around and related directly to the thesis or research question you are developing
• synthesize results into a summary of what is and is not known
• identify areas of controversy in the literature
• formulate questions that need further research

Ask yourself questions like these:

• What is the specific thesis, problem, or research question that my literature review helps to define?
• What type of literature review am I conducting? Am I looking at issues of theory? methodology? policy? quantitative research (e.g. on the effectiveness of a new procedure)? qualitative research (e.g., studies of loneliness among migrant workers)?
• What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)? What discipline am I working in (e.g., nursing psychology, sociology, medicine)?
• How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
• Have I critically analyzed the literature I use? Do I follow through a set of concepts and questions, comparing items to each other in the ways they deal with them? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
• Have I cited and discussed studies contrary to my perspective?
• Will the reader find my literature review relevant, appropriate, and useful?

Ask yourself questions like these about each book or article you include:

• Has the author formulated a problem/issue?
• Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
• Could the problem have been approached more effectively from another perspective?
• What is the author's research orientation (e.g., interpretive, critical science, combination)?
• What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
• What is the relationship between the theoretical and research perspectives?
• Has the author evaluated the literature relevant to the problem/issue? Does the author include literature taking positions she or he does not agree with?
• In a research study, how good are the basic components of the study design (e.g., population, intervention, outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
• In material written for a popular readership, does the author use appeals to emotion, one-sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
• How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?
• In what ways does this book or article contribute to our understanding of the problem under study, and in what ways is it useful for practice? What are the strengths and limitations?
• How does this book or article relate to the specific thesis or question I am developing?

Text written by Dena Taylor, Health Sciences Writing Centre, University of Toronto

http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review

• << Previous: Annotated Bibliography
• Next: Step 5: Cite Sources >>
• Last Updated: May 21, 2024 10:11 AM
• URL: https://libguides.uta.edu/researchprocess

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

• Internet Privacy
• Accessibility
• Problems with a guide? Contact Us.

Mini-Module: How to Write a Literature Review

• 1. What is a Literature Review?
• 2. Features of a Literature Review

3. Writing a Literature Review

• 4. Structuring a Literature Review
• 5. What Should I do Next?

Mini-Modules

Back to Skills for Success: Home

Academic Skills Home

Get ready for university, get ready for university, when writing a literature review, you should think about the following stages:  , thinking of ideas.

Brainstorm and source key literature in your area(s) — Books, papers, articles and so on written by key authors in the field — Policy and guidance documents

These are the stages when you find you are reading a lot – and it may not all make sense yet! Don’t worry, this is perfectly natural.

Identify journal and magazine articles appropriate to your area of study and check the indexes for suitable articles. Follow up references and bibliographies in books and articles. Browse the library catalogues, look at the shelves. Refer to specialist reader lists from other parts of your course.

Narrowing it down

After so much reading, you need to refine! There are four key elements to consider:

• Identify the focus of the field – this will be the general topic or subject area within which the problem of issue you are investigating is set.
• Select the appropriate sources of information – from what you have read, what can you use directly? Indirectly?
• Extract information of direct relevance – a Literature Review isn’t the time to show off how much you have read. Keep it relevant!
• Concentrate on those texts which provide information you need – you will need to have clarity in your writing

 Notes

Some top tips:

• Paraphrase 
• Ask questions and make comments – this is your criticality!
• Keep detailed referencing information in your notes – author, date, title, publisher

Sort and prioritise the literature you have already See which authors/ideas compliment each other See which authors/ideas disagree with each other.

Think about the best way to organise your literature Review: – Chronologically? – Thematically? – By ‘different schools of thought’?

Write, and rewrite

This stage can feel quite laborious and repetitive – but remember that high quality work is always the result of a careful drafting and redrafting process.

• << Previous: 2. Features of a Literature Review
• Next: 4. Structuring a Literature Review >>
• Last Updated: May 21, 2024 1:58 PM
• URL: https://library.lsbu.ac.uk/c.php?g=719086

Graduate Research: Guide to the Literature Review

• "Literature review" defined
• Research Communication Graphic
• Literature Review Steps
• Search techniques
• Finding Additional "Items
• Evaluating information
• Citing Styles
• Ethical Use of Information
• Research Databases This link opens in a new window
• Get Full Text
• Reading a Scholarly Article
• Author Rights
• Selecting a publisher

Introduction to Research Process: Literature Review Steps

When seeking information for a literature review or for any purpose, it helps to understand information-seeking as a process that you can follow. 5 Each of the six (6) steps has its own section in this web page with more detail. Do (and re-do) the following six steps:

1. Define your topic. The first step is defining your task -- choosing a topic and noting the questions you have about the topic. This will provide a focus that guides your strategy in step II and will provide potential words to use in searches in step III.

2. Develop a strategy. Strategy involves figuring out where the information might be and identifying the best tools for finding those types of sources. The strategy section identifies specific types of research databases to use for specific purposes.

3. Locate the information . In this step, you implement the strategy developed in II in order to actually locate specific articles, books, technical reports, etc.

4. Use and Evaluate the information. Having located relevant and useful material, in step IV you read and analyze the items to determine whether they have value for your project and credibility as sources.

5. Synthesize. In step V, you will make sense of what you've learned and demonstrate your knowledge. You will thoroughly understand, organize and integrate the information --become knowledgeable-- so that you are able to use your own words to support and explain your research project and its relationship to existing research by others.

6. Evaluate your work. At every step along the way, you should evaluate your work. However, this final step is a last check to make sure your work is complete and of high quality.

Continue below to begin working through the process.

5. Eisenberg, M. B., & Berkowitz, R. E. (1990). Information Problem-Solving: the Big Six Skills Approach to Library & Information Skills Instruction . Norwood, NJ: Ablex Publishing.

1. Define your topic.

I. Define your topic

A.  Many students have difficulty selecting a topic. You want to find a topic you find interesting and will enjoy learning more about.

B.   Students often select a topic that is too broad.  You may have a broad topic in mind initially and will need to narrow it.

1. To help narrow a broad topic :

a. Brainstorm.  

1). Try this technique for brainstorming to narrow your focus.   

a) Step 1.  Write down your broad topic.

b) Step 2. Write down a "specific kind" or "specific aspect" of the topic you identified in step 1.  

c) Step 3. Write down an aspect  --such as an attribute or behavior-- of the "specific kind" you identified in step 2.  

d) Step 4.  Continue to add  levels of specificity as needed to get to a focus that is manageable. However, you may want to begin researching the literature before narrowing further to give yourself the opportunity to explore what others are doing and how that might impact the direction that you take for your own research.                     

2) Three examples of using the narrowing technique. These examples start with very, very broad topics, so the topic at step 3 or 4 in these examples would be used for a preliminary search in the literature in order to identify a more specific focus.  Greater specificity than level 3 or 4 will ultimately be necessary for developing a specific research question. And we may discover in our preliminary research that we need to alter the direction that we originally were taking.

a) Example 1.      

             Step 1. information security

                      Step  2. protocols

                              Step 3.  handshake protocol

            Brainstorming has brought us to focus on the handshake protocol.

b) Example 2.  

            Step 1. information security

                     Step 2. single sign-on authentication

                              Step 3.  analyzing

                                       Step 4. methods

            Brainstorming has brought us to focus on methods for analyzing the security of single sign-on authentication

c) Example 3.  The diagram below is an example using the broad topic of "software" to show two potential ways to begin to narrow the topic. 

C. Once you have completed the brainstorming process and your topic is more focused, you can do preliminary research to help you identify a specific research question . 

1) Examine overview sources such as subject-specific encyclopedias and textbooks that are likely to break down your specific topic into sub-topics and to highlight core issues that could serve as possible research questions. [See section II. below on developing a strategy to learn how to find these encyclopedias]

2). Search the broad topic in a research database that includes scholarly journals and professional magazines (to find technical and scholarly articles) and scan recent article titles for ideas. [See section II. below on developing a strategy to learn how to find trade and scholarly journal articles]

D. Once you have identified a research question or questions, ask yourself what you need to know to answer the questions. For example,

1. What new knowledge do I need to gain?

2. What has already been answered by prior research of other scholars?

E.  Use the answers to the questions in C. to identify what words  to use to describe the topic when you are doing searches.

1. Identify key words

a.  For example , if you are investigating "security audits in banking", key terms to combine in your searches would be: security, audits, banking.

2.  Create a list of alternative ways of referring to a key word or phrase

a.For example , "information assurance" may be referred to in various ways such as: "information assurance," "information security," and "computer security."

b. Use these alternatives when doing searches.

3. As you are searching, pay attention to how others are writing about the topic and add new words or phrases to your searches if appropriate.

2. Develop a strategy.

II. Develop a strategy for finding the information. 

A. Start by considering what types of source might contain the information you need .  Do you need a dictionary for definitions? a directory for an address? the history of a concept or technique that might be in a book or specialized encyclopedia? today's tech news in an online tech magazine or newspaper?  current research in a journal article? background information that might be in a specialized encyclopedia? data or statistics from a specific organization or website?  Note that you will typically have online access to these source types.

B. This section provides a description of some of the common types of information needed for research.  

1. For technical and business analysis , look for articles in technical and trade magazines . These articles are written by information technology professionals to help other IT professionals do their jobs better. Content might include news on new developments in hardware or software, techniques, tools, and practical advice. Technical journals are also likely to have product ads relevant to information technology workers and to have job ads. Examples iof technical magazines include Network Computing and IEEE Spectrum .

2. To read original research studies , look for articles in scholarly journals and conference proceedings . They will provide articles written by  information technology professionals who are reporting original research; that is, research that has been done by the authors and is being reported for the first time. The audience for original research articles is other information technology scholars and professionals. Examples of scholarly journals include Journal of Applied Security Research , Journal of Management Information Systems , IEEE Transactions on Computers , and ACM Transactions on Information and System Security .

3. For original research being reported to funding agencies , look for technical reports on agency websites. Technical reports are researcher reports to funding agencies about progress on or completion of research funded by the agency.

4. For in-depth, comprehensive information on a topic , look for book-length volumes . All chapters in the book might be written by the same author(s) or might be a collection of separate papers written by different authors.

5. To learn about an unfamiliar topic , use textbooks ,  specialized encyclopedias and handbooks to get get overviews of topics, history/background, and key issues explained.

6. For instructions for hardware, software, networking, etc., look for manuals  that provide step-by-step instructions.

7. For technical details about inventions (devices, instruments, machines), look for patent documents .

C.   NOTE -  In order to search for and find original research studies,  it will help if you  understand  how information is produced, packaged  and  communicated  within your profession. This is explained in the tab  "Research Communication: Graphic."

3. Locate the information.

III. Locate the information

A. Use search tools designed to find the sources you want.  Types of sources were described in section II. above. 

Always feel free to Ask a librarian for assistance when you have questions about where and how locate the information you need.

B. Evaluate the search results (no matter where you find the information)

1. Evaluate the items you find using at least these 5 criteria:

a. accuracy -- is the information reliable and error free?

1) Is there an editor or someone who verifies/checks the information?

2) Is there adequate documentation: bibliography, footnotes, credits?

3) Are the conclusions justified by the information presented?

b. authority -- is the source of the information reputable?

1) How did you find the source of information: an index to edited/peer-reviewed material, in a bibliography from a published article, etc.?

2) What type of source is it: sensationalistic, popular, scholarly?

c. objectivity -- does the information show bias?

1) What is the purpose of the information: to inform, persuade, explain, sway opinion, advertise?

2) Does the source show political or cultural biases?

d. currency -- is the information current? does it cover the time period you need?

e. coverage -- does it provide the evidence or information you need?

2. Is the search producing the material you need? -- the right content? the right quality? right time period? right geographical location? etc. If not, are you using

a. the right sources?

b. the right tools to get to the sources?

c. are you using the right words to describe the topic?

3. Have you discovered additional terms that should be searched? If so, search those terms.

4. Have you discovered additional questions you need to answer? If so, return to section A above to begin to answer new questions.

4. Use and evaluate the information.

IV. Use the information.

A. Read, hear or view the source

1. Evaluate: Does the material answer your question(s)? -- right content? If not, return to B.

2. Evaluate: Is the material appropriate? -- right quality? If not, return to B.

B. Extract the information from the source : copy/download information, take notes, record citation, keep track of items using a citation manager.

1. Note taking (these steps will help you when you begin to write your thesis and/or document your project.):

a. Write the keywords you use in your searches to avoid duplicating previous searches if you return to search a research database again. Keeping track of keywords used will also save you time if your search is interrupted or you need return and do the search again for some other reason. It will help you remember which search terms worked successfully in which databases

b. Write the citations or record the information needed to cite each article/document you plan to read and use, or make sure that any saved a copy of the article includes all the information needed to cite it. Some article pdf files may not include all of the information needed to cite, and it's a waste of your valuable time to have to go back to search and find the items again in order to be able to cite them. Using citation management software such as EndNote will help keep track of citations and help create bibliographies for your research papers.

c. Write a summary of each article you read and/or why you want to use it.

5. Synthesize.

V. Synthesize.

A. Organize and integrate information from multiple sources

B. Present the information (create report, speech, etc. that communicates)

C. Cite material using the style required by your professor or by the venue (conference, publication, etc.). For help with citation styles, see  Guide to Citing Sources .  A link to the citing guide is also available in the "Get Help" section on the left side of the Library home page

6. Evaluate your work.

VI. Evaluate the paper, speech, or whatever you are using to communicate your research.

A. Is it effective?

B. Does it meet the requirements?

C. Ask another student or colleague to provide constructive criticism of your paper/project.

• << Previous: Research Communication Graphic
• Next: Search techniques >>
• Last Updated: Apr 15, 2024 3:27 PM
• URL: https://library.dsu.edu/graduate-research

How to write a Literature Review: Literature review process

• Literature review process
• Purpose of a literature review
• Evaluating sources
• Managing sources
• Request a literature search
• Selecting the approach to use
• Quantitative vs qualitative method
• Summary of different research methodologies
• Research design vs research methodology
• Diagram: importance of research
• Attributes of a good research scholar

Step 1: Select a topic

• Select a topic you can manage in the time frame you have to complete your project.
• Establish your research questions and organize your literature into logical categories around the subject/ topic areas of your questions.  Your research questions must be specific enou gh to guide you to the relevant literature.
• Make sure you understand the concept of ‘broader’ and ‘narrower’ terms.  The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the literature.

Step 2: Identify the most relevant sources on your topic

Use a variety of resources - locate books , journals , and documents that contain useful information and ideas on your topic. Internet sites , theses & dissertations , conference papers , ePrints and government or industry reports can also be included. Do not rely solely on electronic full-text material which is more easily available. Reference sources such as dictionaries can assist in defining terminology, and encyclopaedias may provide useful introductions to your topic by experts in the field and will list key references.

Step 3 : Search and refine

• Unisa has a number of databases that provide full text access to articles, that allow you to refine your search to ‘peer reviewed’ journals.  These are scholarly journals which go through a rigorous process of quality assessment by several researchers or subject specialists in the academic community before they are accepted for publication. 
• Use the And, Or, Not operators, Wildcards and Logical Brackets when searching in the databases.  For instance, you can use And to narrow your search while the operator OR expands your search.  Not, on the other hand, helps to exclude irrelevant information from your search results.  Please click here for more information on searching.

Literature review process - an overview

Step 3: search and refine.

• Unisa has a number of  databases  that provide full text access to articles, that allow you to refine your search to ‘peer reviewed’ journals.  These are scholarly journals which go through a rigorous process of quality assessment by several researchers or subject specialists in the academic community before they are accepted for publication. 
• Use the  And, Or, Not  operators,  Wildcards  and  Logical Brackets  when searching in the databases.  For instance, you can use  And  to narrow your search while  the  operator  OR  expands your search.   Not,  on the other hand,   helps to exclude   irrelevant information from your search results.  Please click  here  for more information on searching.

How do I write a literature review

See the chapter below for a helpful overview of the literature review process, especially the sections on how to analyse the literature you have gathered and how to write up your literature review:

Literature Reviews and Bibliographic Searches. 2006. In V. Desai, & R. Potter (Eds.),  Doing Development Research.  (pp. 209-222). London, England: SAGE Publications, Ltd. Available at:  http://0-dx.doi.org.oasis.unisa.ac.za/10.4135/9781849208925.n22     (A student will be prompted at some stage for his/ her student number and myUnisa password. A staff member will be prompted at some stage for his/ her Unisa Network username and login password).

This book is available in the  Sage Research Methods Online  database.

Step 4: Read and analyse

Group the sources into the  themes  and  sub-themes  of your topic.  As you read widely but selectively in your topic area, consider what themes or issues connect your sources together.

• Do they present one or different solutions?
• Is there an aspect of the field that is missing?
• How well do they present the material and do they portray it according to an appropriate theory?
• Do they reveal a trend in the field?
• A raging debate?
• Pick one of these themes to focus the organization of your review.

Step 5: Write the literature review

You can organize the review in many ways; for example, you can center the review  historically  (how the topic has been dealt with over time); or center it on the  theoretical positions  surrounding your topic (those for a position vs. those against, for example); or you can focus on how each of your sources contributes to your understanding of your project.

Your literature review should include:

• an  introduction  which explains how your review is organized.
• a  body  which contains the  headings  and  subheadings  that provide a map to show the various perspectives of your argument. In other words the body contains the evaluation of the materials you want to include on your topic.
• a  summary .

Some of the information on this page is indebted to the sources below:

Caldwell College Library

Monmouth University Library

University of Cape Town Libraries

• << Previous: Home
• Next: Purpose of a literature review >>
• Last Updated: Apr 30, 2024 1:19 PM
• URL: https://libguides.unisa.ac.za/literature_review

Research Methods

Chapter 3 literature review, 3.1 introduction.

The literature review is often the weak part in master theses.

There are various reasons for this.

One is that doing a proper literature review is not an easy task, and time-consuming.

It is not easy because two things may happen. - Either there is so much literature that summarizing it all is a daunting task. - Or there is little or even none to be found.

Too much literature is a luxury problem.

The way to handle this problem is by using a mind-map approach.

If your topic is, say, university performance , then zoom in on those aspects that you think are especially relevant.

If you still want to stick to the broad topic of university performance, then you can limit your search to recent items. For example, in Google Scholar you can customize the time range in a search to the last 5 years, thereby reducing the number of hits.

Likewise, refined or advanced searches can be applied in e-libraries like Ebsco and Science Direct . Still, if your research is on historic thinking on your topic, you need to adopt an alternative strategy to narrow down the number of hits.

Obviously, your objective in searching from whichever databases you are using, is in detecting the most relevant items and filtering out irrelevant ones. It takes experience, trial-and-error and common sense thinking to come to good results.

Given the sheer endless of papers, reports, books, newspaper articles and other items that can be found on any topic, the probability that two researchers – equally brilliant – on the same topic will end up with identical reference lists, is zero. For that reason, it is good practice to explicitly state your search strategies at the beginning of chapter 2.

3.2 Search Strategies: Snowballing

One strategy that saves a lot of time, is snowballing.

The idea is that in a first step, you identify, say, three of four key articles which are recent, high quality (published in A or B journals), and relevant (in the sense of being very close to your topic).

Since the authors of these articles have dealt with the same challenge of finding relevant articles, you can select articles from their lists of references. Then the snowball starts rolling and growing, as the second-wave articles again will contain some interesting references.

In this way, you will quickly acquaint yourself with the relevant theories and empirical studies in your field of interest.

The disadvantage of snowballing is that, by definition, you will travel back in time. So, make sure to start with recent articles, published no earlier than two or three years ago.

3.3 Be Selective!

We cannot emphasize enough that the review has to be critical. Often, students tend to write their literature review in the fallen-bookcase style. That is, as a collection of summaries of things that various authors have said but lacking a clear structure, leaving the reader with a why-am-I-reading-this feeling.

The messages are: - Write with an objective in mind - Structure the review - Use your own words and interpretations - Make clear why and how the items reviewed are important to your study.

3.4 Structure

With regard to the structure of your literature review, there are various way to do it. As long as the structure is clear and deliberate, it will ensure that the literature review becomes a relevant part of your thesis.

Three main structures are:

3.4.1 Chronological

This is especially relevant if you want to describe the trends in thinking on the topic over time.

For example, the views on the role of labor continue to change.

Nowadays, robotization and 3D-printing are efficient substitutes for human labor. But already in 1995, Rifkin wrote about the decline of the labor force. What can we learn from the views on the matter, over time?

Note that the snowballing-technique might miss out this old item!

Rifkin, J. (1995). The End of Work: The Decline of the Global Labor Force and the Dawn of the Post-Market Era. New York: G. P. Putnam’s Sons

3.4.2 By schools of thought

It is possible that even today, there are differing views on a certain topic.

For example, while globalization is seen as a trend of converging cultures, many researchers from empirical evidence find that cultural differences between nations persist – and are even reinforced in the face of globalization. Here, it makes sense to group the arguments of both camps, rather than use a chronological structure.

3.4.3 By (sub)topic.

Especially if your topic is broad, like in university performance , it makes sense to break down your discussion by topics.

For example, you can have sections on:

• Student performance
• Citations of university publications
• Performance of private versus public universities

3.5 Critical Review

One indicator of a critical review, is your ability to summarize the various items that you have reviewed in your own words, and in relation to one another. Reviews in the style of Johnson (2010) said “[literal quote]”, and Peters (2017) claimed “[literal quote]”, leave it very much to the reader to give it an interpretation. The value added of the researcher, is more than just selecting or collecting a number of papers. Without structure, and without paraphrasing and interpretation, the review is simply not a critical review of literature.

3.6 Role of the Literature Review in your Research

The literature review is an input to both your research questions and the design of your study.

As we have said before, in the initial stages you start reading driven by your initial research ideas, and you refine your ideas based on reading. It is a creative, circular process – rather than a linear from-goals-to-decisions process.

But once this part of the process is done, a linear type of project appears on the horizon.

The end-point of the first stage is a research model plus methodology.

The research model, in a nutshell, contains all the variables on which you need data, and the relationships between these variables.

The methodology is the way you are going to collect the data, for example doing a survey with a number of respondents in this region and that period of time.

It helps to see the literature as the bridge between the creative process (what do I want?) and the research project (how to collect and analyze the data?). The research model is, in essence, that bridge. The elements of the model are based on your research questions and the literature. We would not expect to see variables in your research model that are not discussed in the review of literature. In the same vein, why would you discuss issues in the review of literature that are not included in the research model?

There are exceptions to this black-and-white view. In your research model, you may include variables that have not been described in literature, like for example country or organization specific actors that you think are of relevance to the solution of the problem at hand (remember: it’s applied research!). You can also deliberately exclude variables that are discussed in the literature as being relevant, on the grounds of the need to focus on the issues that you think are most prevalent. Adding more variables to your research, and questions to your survey, may have consequences for the time and resources available.

3.7 Beware of Plagiarism

Plagiarism is defined as:

• To use the ideas or words of someone else as your own
• To use someone else's work without crediting him or her
• To present as your new and original idea, a work that is derived from an existing source.

Plagiarism is an act of fraud. It involves both stealing someone else's work and lying about it afterward.

All of following are considered plagiarism:

• Turning in someone else's work as your own
• Copying words or ideas from someone else without giving credit
• Failing to put a quotation in quotation marks
• Giving incorrect information about the source of a quotation
• Changing words but copying the sentence structure of a source without giving credit
• Copying so many words or ideas from a source that it makes up the majority of your work - whether you give credit or not.

Plagiarism can be avoided by properly citing sources.

Acknowledging that material has been borrowed and providing the reader with the information necessary to find that source is enough to prevent.

However, as indicated above, a soft form of plagiarism occurs if a substantial part of your thesis is based on ideas from others, and the added value in terms of discussion, interpretation and putting the reviewed items into the context of your thesis, is limited. Even if this soft form of plagiarism is not punished as such, then still the score on your literature review will be low – as it can hardly be called a critical review of literature.

3.8 Referencing

In referencing, it is mandatory to use the so-called APA-style. APA stands for American Psychological Association. A detailed overview of the APA-style can be found using this link .

Detailed information on referencing, using tools in MS Word and Mendeley is provided in the next chapter.

3.9 How To Write A Literature Review In Three Simple Steps

As a guideline, use the three-step approach:

• Step 1: Find relevant research
• Step 2: Log, catalog, & synthesize
• Step 3: Outlining & Writing up

Details on each of these three steps are in the video below.

Chaos to Clarity: Structuring Your Literature Review Format

Master literature review format! Learn key sections, effective citation & analysis tips to write a strong academic review.

Ever wondered how to dive into a mountain of books and articles and come up with something that not just makes sense but shines new light on a topic? What if there was a way to neatly tie together all that information, spot what’s missing, and maybe even pave the way for discoveries? 

That’s what you are going to learn in this article, literature reviews—a place where chaos meets order, and where your insights could set the stage for the next big thing. Let’s break down the literature review format , your essential guide to properly writing a literature review.

Dissecting Literature Review Format 

There are 6 main sections to make a note of while writing a literature review. Those are:

The Introduction Section

Topic background, conceptual framework.

• Synthesis and Evaluation in Literature Reviews
• Conclusion for Your Literature Review
• Reference List in Your Literature Review

Also Read: Essential Components of a Literature Review

The introduction of your literature review is where you set the stage for the entire document. It’s your first opportunity to engage your readers and provide a clear blueprint of what your review will cover and why it matters. This section does more than merely introduce the topic; it establishes the context, defines the scope, and outlines the purpose and objectives of your literature review.

Things to keep in mind while writing an introduction:

• Craft a compelling opening
• Establish the Context and Justification
• Define the Scope and Objectives
• Lay out the Structure
• Give an overview of the Structure

The “Topic Background” section of a literature review serves as the cornerstone for understanding the evolution and current state of the subject matter. It is divided into two crucial sub-sections: Historical Context and Current State of the Topic . 

Delving into these areas provides you with a comprehensive backdrop against which the literature review is framed, enriching the reader’s understanding of why the topic is of interest and what has influenced its development to the current state.

Historical Context

The Historical Context is fundamental in setting the stage for the entire literature review. This section is not just a chronology of events or developments; it’s a curated narrative that highlights the key milestones and turning points that have significantly impacted the topic. 

By examining the historical evolution, the review establishes a timeline of how understanding and perspectives have shifted over the years.

Summary Of Key Historical Developments

This involves identifying and summarizing the major breakthroughs, shifts in thinking, or seminal works that have shaped the topic. It’s important to focus on developments that have a direct relevance to the current understanding and state of the subject. For example, if the topic is about the evolution of renewable energy technologies, this part would outline the initial discovery and use of renewable sources, significant technological innovations, and pivotal policy decisions that have influenced the field.

Relevance Of Historical Context To The Topic

After outlining the key historical developments, it’s crucial to connect these events to the present topic. This means discussing how past events have laid the groundwork for current theories, practices, or debates within the field. It involves analyzing the impact of historical milestones on the subject matter, and explaining how they have contributed to current knowledge, challenges, and research questions. This section makes it clear why understanding history is essential for anyone researching or studying the topic today.

Current State Of The Topic

Moving from the historical context, the review transitions to the present with the Current State of the Topic. This part assesses the latest research, trends, debates, and technological advancements that define the subject area at the moment.

Current Trends Or Updates

Here, the focus shifts to what is happening in the field right now. This could include recent research findings, emerging theories, new methodologies, or the latest technological innovations. The aim is to provide a snapshot of the current research landscape, identifying what themes, questions, or problems are being actively explored. For instance, in the context of digital marketing, this might involve discussing the rise of artificial intelligence in customer relationship management or the impact of social media trends on marketing strategies.

Impact Of These Trends On The Subject Matter

The final step is to assess the implications of these current trends for the topic. This includes considering how recent developments have advanced the field, the challenges they present, and the opportunities they open up for future research. It’s about connecting the dots between what’s happening now and what it means for the subject area moving forward. This not only helps to frame the research questions that the literature review will address but also sets the stage for identifying gaps in the current knowledge, thereby guiding the direction of future studies.

Also Read: What is a literature review? Get the concept and start using it

When doing a literature review, it’s essential to lay a solid foundation for your exploration through a well-defined conceptual framework. This framework acts as a compass, guiding your review’s direction by establishing the key concepts, theories, and perspectives that underpin your topic. 

Definitions And Descriptions

Before diving into the depths of your literature review, it’s crucial to start with the basics. This means clearly identifying and defining the key concepts related to your topic. Think of this as setting the stage for your readers, ensuring they have a clear understanding of the fundamental terms and ideas you will be exploring.

Key Concepts Related To The Topic

Begin by listing the essential concepts central to your review. These are the building blocks of your topic, the terms that will repeatedly appear throughout your exploration. 

Detailed Definitions And Their Relevance

Once you’ve identified these concepts, provide precise and comprehensive definitions for each. Don’t hesitate to explore different dimensions or interpretations of these terms, as this can enrich your readers’ understanding. More importantly, discuss why these concepts are crucial to your review. How do they shape the scope of your exploration? How do they relate to each other and to the broader topic? This step ensures that your readers are not just familiar with the terms but also understand their significance within your review’s context.

Theoretical Perspectives

With the key concepts clearly defined, it’s time to frame your literature review within relevant theoretical perspectives. This is where you align your exploration with existing theories, models, or frameworks that provide insights into your topic.

Important Theories Related To The Topic

Identify the theories that are foundational to your topic. These could range from well-established theories that have long guided research in your field to more contemporary models that offer new insights. For example, a review of organizational behavior might draw on theories of motivation, leadership styles, and organizational culture.

Evaluation Of These Theories And Their Influence On The Topic

After pinpointing the relevant theories, critically assess their contributions to the topic. Consider questions like: How have these theories shaped understanding of the topic? What insights do they offer, and where do they fall short? Are there controversies or debates surrounding these theories? This evaluation not only deepens your review’s analytical depth but also positions your work within the larger academic conversation.

Synthesis And Evaluation In Literature Reviews

The “Synthesis and Evaluation” section is where your literature review truly comes to life. Here, you’re not just summarizing what others have said; you’re weaving together diverse strands of research to present a cohesive picture of the topic at hand.

Comparison And Contrast Of Sources

Synthesizing the literature involves more than listing findings from various studies; it’s about drawing connections between them, highlighting areas of agreement and dispute, and weaving these into a narrative that adds depth and breadth to your understanding of the topic.

Comparative Analysis

Start by grouping your sources based on similarities in their findings, methodologies, or theoretical approaches. This clustering will help you identify trends and common themes across the literature. For example, if several studies have found similar outcomes under comparable conditions, these findings can be grouped to strengthen a particular argument or observation about the topic.

Contrasts Or Conflicts Among Sources

Equally important is the identification of discrepancies in the literature. Do some studies present findings that directly contradict others? Are there differences in how researchers have interpreted similar data? Highlighting these conflicts is crucial, as it can indicate areas where the topic is still evolving or where further research is needed. It also shows your ability to critically engage with the material, a hallmark of scholarly rigor.

Analysis Of Gaps In Literature

One of your primary tasks in the synthesis and evaluation section is to identify what’s missing in the current body of research. This requires a critical eye and a deep understanding of both your topic and the broader field in which it resides.

Identification Of Research Gaps

As you comb through the literature, ask yourself: What questions remain unanswered? Are there underexplored areas or populations? Perhaps certain methodologies have been overlooked, or theoretical perspectives have not been considered. Pinpointing these gaps is not a mere exercise in academic critique; it’s a vital step in advancing knowledge within the field.

Implications Of These Gaps For Future Research

Highlighting gaps in the literature sets the stage for future studies. It’s where you, as the reviewer, can suggest new research directions that could fill these voids or further explore the topic. Discussing the implications of these gaps not only enriches your review but also contributes to the ongoing scholarly conversation. 

Conclusion For Your Literature Review

The conclusion of your literature review is where you bring together all the strands of your argument, synthesizing the insights gained and highlighting the significance of your findings. It’s not just a summary of what has been discussed; it’s an opportunity to underscore the relevance of the review, reflect on the broader implications of your synthesis and evaluation, and suggest directions for future research. 

Summary Of Key Points

Start your conclusion by succinctly summarizing the main points and findings of your review. This isn’t about rehashing every detail but rather about distilling the essence of your exploration. Highlight the critical trends, themes, and conflicts you’ve uncovered, and remind your readers of the significance of these discoveries.

Relevance And Implications Of The Literature For The Topic

Next, focus on the relevance and implications of your findings. This involves stepping back to consider the bigger picture—how does your literature review contribute to the understanding of your topic? Discuss the impact of the trends and gaps you’ve identified on the field, and elaborate on how your synthesis of the literature advances or enriches existing knowledge.

Reflection On The Research Process

Reflecting on the research process itself can provide valuable insights. Consider discussing the challenges you encountered in navigating the literature, such as dealing with conflicting findings or the scarcity of research on certain aspects of your topic. 

Directions For Future Research

One of the most critical aspects of your conclusion is to suggest directions for future research. Be as precise as possible, whether suggesting new methodologies, theoretical frameworks, or specific topics that warrant deeper investigation.

Final Thoughts

End your conclusion with a strong closing statement that reiterates the value of your literature review. Emphasize the importance of continued research on your topic and the potential it holds for advanced understanding within your field. A compelling conclusion reaffirms the significance of your work, leaving your readers with a clear sense of its contribution and the urgent need for further exploration.

Reference List In Your Literature Review

The Reference List is the backbone of your literature review, providing a comprehensive compilation of all the sources you’ve cited throughout your exploration. It’s not merely a formality but a crucial component that lends credibility and rigor to your work.

Importance Of Accuracy And Consistency

The cornerstone of a reliable Reference List is accuracy and consistency in citation style. Whether you’re adhering to APA , MLA , Chicago , or another academic citation format, it’s vital to apply the rules with precision. This includes correctly formatting author names, publication dates, titles, and publication details. 

Organizing Your References

While different citation styles have their own rules for listing references, organizing them in a way that enhances readability and accessibility is universally beneficial. Alphabetical order by the author’s last name is the most common method, as it allows readers to easily locate sources.

Comprehensive Coverage

Your Reference List should be exhaustive, including every work you’ve cited in your review. This extends beyond journal articles and books to encompass reports, conference papers, online resources, and any other materials that have informed your analysis.

The Value Of Annotations

While not always required, providing brief annotations for key sources can add tremendous value to your Reference List. An annotated bibliography offers a succinct summary of each source’s main arguments, methodologies, and findings, as well as its relevance to your literature review.

Digital Accessibility

In today’s digital age, considering the accessibility of your referenced works can greatly enhance the utility of your Reference List. Whenever possible, include Digital Object Identifiers (DOIs) or stable URLs for online sources, ensuring readers can directly access the materials. 

Also read: What Is A DOI? Exploring The Purpose And Importance

Reflecting On Ethical Scholarship

Finally, your Reference List is a reflection of ethical scholarship. By accurately citing all the sources that have informed your work, you’re honoring the intellectual property of other researchers and upholding the academic community’s standards of integrity and respect. 

Crafting a meticulous Reference List is an essential aspect of your literature review that underscores the credibility, depth, and ethical foundation of your research. By adhering to the principles of accuracy, comprehensiveness, and accessibility, you not only facilitate further inquiry but also pay homage to the collective endeavor of knowledge advancement in your field.

Related Article: Navigating the AMA Citation Format: Best Tips for Referencing

In conclusion, writing a literature review involves meticulous structuring, beginning with an engaging introduction that sets the stage, followed by a detailed exploration of the topic’s background, including its historical context and current state. 

A robust conceptual framework lays the groundwork for analysis, leading to a critical synthesis and evaluation of relevant literature. 

The conclusion ties together the review’s key findings and implications, while the reference list meticulously catalogs all cited works. Mastering each section ensures a comprehensive and insightful review, essential for advancing academic understanding and contributing to scholarly discussions.

Related Article: Preliminary Literature Review: A Guide for Effective Research

Science Figures, Graphical Abstracts, And Infographics For Your Research

Revolutionize your research with infographics from Mind the Graph . From science figures, graphical abstracts to infographics, you can unleash the power of creative visuals with this user-friendly platform and make your research captivating. 

Subscribe to our newsletter

Exclusive high quality content about effective visual communication in science.

Unlock Your Creativity

Create infographics, presentations and other scientifically-accurate designs without hassle — absolutely free for 7 days!

About Sowjanya Pedada

Sowjanya is a passionate writer and an avid reader. She holds MBA in Agribusiness Management and now is working as a content writer. She loves to play with words and hopes to make a difference in the world through her writings. Apart from writing, she is interested in reading fiction novels and doing craftwork. She also loves to travel and explore different cuisines and spend time with her family and friends.

Content tags

Systematic Review

• Library Help
• What is a Systematic Review (SR)?

Steps of a Systematic Review

• Framing a Research Question
• Developing a Search Strategy
• Searching the Literature
• Managing the Process
• Meta-analysis
• Publishing your Systematic Review

Forms and templates

Image: David Parmenter's Shop

• PICO Template
• Inclusion/Exclusion Criteria
• Database Search Log
• Review Matrix
• Cochrane Tool for Assessing Risk of Bias in Included Studies

   • PRISMA Flow Diagram  - Record the numbers of retrieved references and included/excluded studies. You can use the Create Flow Diagram tool to automate the process.

   •  PRISMA Checklist - Checklist of items to include when reporting a systematic review or meta-analysis

PRISMA 2020 and PRISMA-S: Common Questions on Tracking Records and the Flow Diagram

• PROSPERO Template
• Manuscript Template
• Steps of SR (text)
• Steps of SR (visual)
• Steps of SR (PIECES)

Adapted from  A Guide to Conducting Systematic Reviews: Steps in a Systematic Review by Cornell University Library

Source: Cochrane Consumers and Communications  (infographics are free to use and licensed under Creative Commons )

Check the following visual resources titled " What Are Systematic Reviews?"

• Video  with closed captions available
• Animated Storyboard
• << Previous: What is a Systematic Review (SR)?
• Next: Framing a Research Question >>
• Last Updated: May 8, 2024 1:44 PM
• URL: https://lib.guides.umd.edu/SR

• My Library Account
• Articles, Books & More
• Course Reserves
• Site Search
• Advanced Search
• Sac State Library
• Research Guides
• Writing a Literature Review
• 6 Stages to Writing a Literature Review
• What is a Literature Review?
• Literature Review Examples
• Organizing Your Literature Review
• Managing your Citations
• Further Reading on Lit Reviews

Academic Phrasebank

• Last Updated: Mar 11, 2024 1:00 PM
• URL: https://csus.libguides.com/litreview

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Mark Access Health Policy
• v.11(1); 2023
• PMC10392303

Rapid literature review: definition and methodology

Beata smela.

a Assignity, Cracow, Poland

Mondher Toumi

b Public Health Department, Aix-Marseille University, Marseille, France

Karolina Świerk

Clement francois, małgorzata biernikiewicz.

c Studio Slowa, Wroclaw, Poland

Emilie Clay

d Clever-Access, Paris, France

Laurent Boyer

Introduction: A rapid literature review (RLR) is an alternative to systematic literature review (SLR) that can speed up the analysis of newly published data. The objective was to identify and summarize available information regarding different approaches to defining RLR and the methodology applied to the conduct of such reviews.

Methods: The Medline and EMBASE databases, as well as the grey literature, were searched using the set of keywords and their combination related to the targeted and rapid review, as well as design, approach, and methodology. Of the 3,898 records retrieved, 12 articles were included.

Results: Specific definition of RLRs has only been developed in 2021. In terms of methodology, the RLR should be completed within shorter timeframes using simplified procedures in comparison to SLRs, while maintaining a similar level of transparency and minimizing bias. Inherent components of the RLR process should be a clear research question, search protocol, simplified process of study selection, data extraction, and quality assurance.

Conclusions: There is a lack of consensus on the formal definition of the RLR and the best approaches to perform it. The evidence-based supporting methods are evolving, and more work is needed to define the most robust approaches.

Introduction

A systematic literature review (SLR) summarizes the results of all available studies on a specific topic and provides a high level of evidence. Authors of the SLR have to follow an advanced plan that covers defining a priori information regarding the research question, sources they are going to search, inclusion criteria applied to choose studies answering the research question, and information regarding how they are going to summarize findings [ 1 ].

The rigor and transparency of SLRs make them the most reliable form of literature review [ 2 ], providing a comprehensive, objective summary of the evidence for a given topic [ 3 , 4 ]. On the other hand, the SLR process is usually very time-consuming and requires a lot of human resources. Taking into account a high increase of newly published data and a growing need to analyze information in the fastest possible way, rapid literature reviews (RLRs) often replace standard SLRs.

There are several guidelines on the methodology of RLRs [ 5–11 ]; however, only recently, one publication from 2021 attempted to construct a unified definition [ 11 ]. Generally, by RLRs, researchers understand evidence synthesis during which some of the components of the systematic approach are being used to facilitate answering a focused research question; however, scope restrictions and a narrower search strategy help to make the project manageable in a shorter time and to get the key conclusions faster [ 4 ].

The objective of this research was to collect and summarize available information on different approaches to the definition and methodology of RLRs. An RLR has been run to capture publications providing data that fit the project objective.

To find publications reporting information on the methodology of RLRs, searches were run in the Medline and EMBASE databases in November 2022. The following keywords were searched for in titles and abstracts: ‘targeted adj2 review’ OR ‘focused adj2 review’ OR ‘rapid adj2 review’, and ‘methodology’ OR ‘design’ OR ‘scheme’ OR ‘approach’. The grey literature was identified using Google Scholar with keywords including ‘targeted review methodology’ OR ‘focused review methodology’ OR ‘rapid review methodology’. Only publications in English were included, and the date of publication was restricted to year 2016 onward in order to identify the most up-to-date literature. The reference lists of each included article were searched manually to obtain the potentially eligible articles. Titles and abstracts of the retrieved records were first screened to exclude articles that were evidently irrelevant. The full texts of potentially relevant papers were further reviewed to examine their eligibility.

A pre-defined Excel grid was developed to extract the following information related to the methodology of RLR from guidelines:

• Definition,
• Research question and searches,
• Studies selection,
• Data extraction and quality assessment,
• Additional information.

There was no restriction on the study types to be analyzed; any study reporting on the methodology of RLRs could be included: reviews, practice guidelines, commentaries, and expert opinions on RLR relevant to healthcare policymakers or practitioners. The data extraction and evidence summary were conducted by one analyst and further examined by a senior analyst to ensure that relevant information was not omitted. Disagreements were resolved by discussion and consensus.

Studies selection

A total of 3,898 records (3,864 articles from a database search and 34 grey literature from Google Scholar) were retrieved. After removing duplicates, titles and abstracts of 3,813 articles were uploaded and screened. The full texts of 43 articles were analyzed resulting in 12 articles selected for this review, including 7 guidelines [ 5–11 ] on the methodology of RLRs, together with 2 papers summarizing the results of the Delphi consensus on the topic [ 12 , 13 ], and 3 publications analyzing and assessing different approaches to RLRs [ 4 , 14 , 15 ].

Overall, seven guidelines were identified: from the World Health Organization (WHO) [ 5 ], National Collaborating Centre for Methods and Tools (NCCMT) [ 7 ], the UK government [ 8 ], the Oxford Centre for Evidence Based Medicine [ 9 ], the Cochrane group [ 6 , 11 ], and one multi-national review [ 10 ]. Among the papers that did not describe the guidelines, Gordon et al. [ 4 ] proposed 12 tips for conducting a rapid review in the right settings and discussed why these reviews may be more beneficial in some circumstances. The objective of work conducted by Tricco et al. [ 13 ] and Pandor et al. [ 12 ] was to collect and compare perceptions of rapid reviews from stakeholders, including researchers, policymakers, industry, journal editors, and healthcare providers, and to reach a consensus outlining the domains to consider when deciding on approaches for RLRs. Haby et al. [ 14 ] run a rapid review of systematic reviews and primary studies to find out the best way to conduct an RLR in health policy and practice. In Tricco et al. (2022) [ 15 ], JBI position statement for RLRs is presented.

From all the seven identified guidelines information regarding definitions the authors used for RLRs, approach to the PICOS criteria and search strategy development, studies selection, data extractions, quality assessment, and reporting were extracted.

Cochrane Rapid Reviews Methods Group developed methods guidance based on scoping review of the underlying evidence, primary methods studies conducted, as well as surveys sent to Cochrane representative and discussion among those with expertise [ 11 ]. They analyzed over 300 RLRs or RLR method papers and based on the methodology of those studies, constructed a broad definition RLR, one that meets a minimum set of requirements identified in the thematic analysis: ‘ A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting a variety of methods to produce evidence in a resource-efficient manner .’ This interpretation aligns with more than 50% of RLRs identified in this study. The authors additionally provided several other definitions, depending on specific situations or requirements (e.g., when RLR is produced on stakeholder’s request). It was additionally underlined that RLRs should be driven by the need of timely evidence for decision-making purposes [ 11 ].

Rapid reviews vary in their objective, format, and methods used for evidence synthesis. This is a quite new area, and still no agreement on optimal methods can be found [ 5 ]. All of the definitions are highlighting that RLRs are completed within shorter timeframes than SLRs, and also lack of time is one of the main reasons they are conducted. It has been suggested that most rapid reviews are conducted within 12 weeks; however, some of the resources suggest time between a few weeks to no more than 6 months [ 5 , 6 ]. Some of the definitions are highlighting that RLRs follow the SLR process, but certain phases of the process are simplified or omitted to retrieve information in a time-saving way [ 6 , 7 ]. Different mechanisms are used to enhance the timeliness of reviews. They can be used independently or concurrently: increasing the intensity of work by intensifying the efforts of multiple analysts by parallelization of tasks, using review shortcuts whereby one or more systematic review steps may be reduced, automatizing review steps by using new technologies [ 5 ]. The UK government report [ 8 ] referred to two different RLRs: in the form of quick scoping reviews (QSR) or rapid evidence assessments (REA). While being less resource and time-consuming compared to standard SLRs, QSRs and REAs are designed to be similarly transparent and to minimize bias. QSRs can be applied to rather open-ended questions, e.g., ‘what do we know about something’ but both, QSRs and REAs, provide an understanding of the volume and characteristics of evidence on a specific topic, allowing answering questions by maximizing the use of existing data, and providing a clear picture of the adequacy of existing evidence [ 8 ].

Research questions and searches

The guidelines suggest creating a clear research question and search protocol at the beginning of the project. Additionally, to not duplicate RLRs, the Cochrane Rapid Reviews Methods Group encourages all people working on RLRs to consider registering their search protocol with PROSPERO, the international prospective register of reviews; however, so far they are not formally registered in most cases [ 5 , 6 ]. They also recommend involving key stakeholders (review users) to set and refine the review question, criteria, and outcomes, as well as consulting them through the entire process [ 11 ].

Regarding research questions, it is better to structure them in a neutral way rather than focus on a specific direction for the outcome. By doing so, the researcher is in a better position to identify all the relevant evidence [ 7 ]. Authors can add a second, supportive research question when needed [ 8 ]. It is encouraged to limit the number of interventions, comparators and outcomes, to focus on the ones that are most important for decision-making [ 11 ]. Useful could be also reviewing additional materials, e.g., SLRs on the topic, as well as conducting a quick literature search to better understand the topic before starting with RLRs [ 7 ]. In SLRs researchers usually do not need to care a lot about time spent on creating PICOS, they need to make sure that the scope is broad enough, and they cannot use many restrictions. When working on RLRs, a reviewer may spend more or less time defining each of the components of the study question, and the main step is making sure that PICOS addresses the needs of those who requested the rapid review, and at the same time, it is feasible within the required time frame [ 7 ]. Search protocol should contain an outline of how the following review steps are to be carried out, including selected search keywords and a full strategy, a list of data sources, precise inclusion and exclusion criteria, a strategy for data extraction and critical appraisal, and a plan of how the information will be synthesized [ 8 ].

In terms of searches running, in most cases, an exhaustive process will not be feasible. Researchers should make sure that the search is effective and efficient to produce results in a timely manner. Cochrane Rapid Reviews Methods Group recommends involving an information specialist and conducting peer review of at least one search strategy [ 11 ]. According to the rapid review guidebook by McMaster University [ 7 ], it is important that RLRs, especially those that support policy and program decisions, are being fed by the results of a body of literature, rather than single studies, when possible. It would result in more generalizable findings applied at the level of a population and serve more realistic findings for program decisions [ 7 ]. It is important to document the search strategy, together with a record of the date and any date limits of the search, so that it can easily be run again, modified, or updated. Furthermore, the information on the individual databases included in platform services should always be reported, as this depends on organizations’ subscriptions and must be included for transparency and repeatability [ 7 , 8 ]. Good solution for RLRs is narrowing the scope or searching a limited number of databases and other sources [ 7 ]. Often, the authors use the PubMed/MEDLINE, Cochrane Library, and Embase databases. In most reviews, two or more databases are searched, and common limits are language (usually restricted to English), date, study design, and geographical area. Some RLRs include searching of grey literature; however, contact with authors is rather uncommon [ 5 , 8 ]. According to the flexible framework for restricted systematic review published by the University of Oxford, the search should be run in at least one major scientific database such as PubMed, and one other source, e.g., Google Scholar [ 9 ]. Grey literature and unpublished evidence may be particularly needed and important for intervention questions. It is related to the fact that studies that do not report the effects of interventions are less likely to be published [ 8 ]. If there is any type of evidence that will not be considered by the RLRs, e.g., reviews or theoretical and conceptual studies, it should also be stated in the protocol together with justification [ 8 ]. Additionally, authors of a practical guide published by WHO suggest using a staged search to identify existing SLRs at the beginning, and then focusing on studies with other designs [ 5 ]. If a low number of citations have been retrieved, it is acceptable to expand searches, remove some of the limits, and add additional databases and sources [ 7 ].

Searching for RLRs is an iterative process, and revising the approach is usually needed [ 7 ]. Changes should be confirmed with stakeholders and should be tracked and reflected in the final report [ 5 ].

The next step in the rapid review is the selection of studies consisting of two phases: screening of titles and abstracts, and analysis of full texts. Prior to screening initiation, it is recommended to conduct a pilot exercise using the same 30–50 abstracts and 5–10 full-texts for the entire screening team in order to calibrate and test the review form [ 11 ]. In contrast to SLRs, it can be done by one reviewer with or without verification by a second one. If verification is performed, usually the second reviewer checks only a subset of records and compares them. Cochrane Group, in contrast, recommends a stricter approach: at least 20% of references should be double-screened at titles and abstracts stage, and while the rest of the references may be screened by one reviewer, the excluded items need to be re-examined by second reviewer; similar approach is used in full-text screening [ 11 ]. This helps to ensure that bias was reduced and that the PICOS criteria are applied in a relevant way [ 5 , 8 , 9 , 11 ]. During the analysis of titles and abstracts, there is no need to report reasons for exclusion; however, they should be tracked for all excluded full texts [ 7 ].

Data extraction and quality assessment

According to the WHO guide, the most common method for data extraction in RLRs is extraction done by a single reviewer with or without partial verification. The authors point out that a reasonable approach is to use a second reviewer to check a random sample of at least 10% of the extractions for accuracy. Dual performance is more necessary for the extraction of quantitative results than for descriptive study information. In contrast, Cochrane group recommends that second reviewer should check the correctness and completeness of all data [ 11 ]. When possible, extractions should be limited to key characteristics and outcomes of the study. The same approach to data extraction is also suggested for a quality assessment process within rapid reviews [ 5 , 9 , 11 ]. Authors of the guidebook from McMaster University highlight that data extraction should be done ideally by two reviewers independently and consensus on the discrepancies should always be reached [ 7 ]. The final decision on the approach to this important step of review should depend on the available time and should also reflect the complexity of the research question [ 9 ].

For screening, analysis of full texts, extractions, and quality assessments, researchers can use information technologies to support them by making these review steps more efficient [ 5 ].

Before data reporting, a reviewer should prepare a document with key message headings, executive summary, background related to the topic and status of the current knowledge, project question, synthesis of findings, conclusions, and recommendations. According to the McMaster University guidebook, a report should be structured in a 1:2:20 format, that is, one page for key messages, two pages for an executive summary, and a full report of up to 20 pages [ 7 ]. All the limitations of the RLRs should be analyzed, and conclusions should be drawn with caution [ 5 ]. The quality of the accumulated evidence and the strength of recommendations can be assessed using, e.g., the GRADE system [ 5 ]. When working on references quoting, researchers should remember to use a primary source, not secondary references [ 7 ]. It would be worth considering the support of some software tools to automate reporting steps. Additionally, any standardization of the process and the usage of templates can support report development and enhance the transparency of the review [ 5 ].

Ideally, all the review steps should be completed during RLRs; however, often some steps may need skipping or will not be completed as thoroughly as should because of time constraints. It is always crucial to decide which steps may be skipped, and which are the key ones, depending on the project [ 7 ]. Guidelines suggest that it may be helpful to invite researchers with experience in the operations of SLRs to participate in the rapid review development [ 5 , 9 ]. As some of the steps will be completed by one reviewer only, it is important to provide them with relevant training at the beginning of the process, as well as during the review, to minimize the risk of mistakes [ 5 ].

Additional information

Depending on the policy goal and available resources and deadlines, methodology of the RLRs may be modified. Wilson et al. [ 10 ] provided extensive guidelines for performing RLR within days (e.g., to inform urgent internal policy discussions and/or management decisions), weeks (e.g., to inform public debates), or months (e.g., to inform policy development cycles that have a longer timeline, but that cannot wait for a traditional full systematic review). These approaches vary in terms of data synthesis, types of considered evidence and project management considerations.

In shortest timeframes, focused questions and subquestions should be formulated, typically to conduct a policy analysis; the report should consist of tables along with a brief narrative summary. Evidence from SLRs is often considered, as well as key informant interviews may be conducted to identify additional literature and insights about the topic, while primary studies and other types of evidence are not typically feasible due to time restrictions. The review would be best conducted with 1–2 reviewers sharing the work, enabling rapid iterations of the review. As for RLRs with longer timeline (weeks), these may use a mix of policy, systems and political analysis. Structure of the review would be similar to shorter RLRs – tabular with short narrative summary, as the timeline does not allow for comprehensive synthesis of data. Besides SLRs, primary studies and other evidence may be feasible in this timeframe, if obtained using the targeted searches in the most relevant databases. The review team should be larger, and standardized procedures for reviewing of the results and data extraction should be applied. In contrast to previous timeframe, merit review process may be feasible. For both timeframes, brief consultations with small transdisciplinary team should be conducted at the beginning and in the final stage of the review to discuss important matters.

For RLRs spanning several months, more comprehensive methodology may be adapted in terms of data synthesis and types of evidence. However, authors advise that review may be best conducted with a small review team in order to allow for more in-depth interpretation and iteration.

Studies analyzing methodology

There have been two interesting publications summarizing the results of Delphi consensus on the RLR methodology identified and included in this review [ 12 , 13 ].

Tricco et al. [ 13 ] first conducted an international survey and scoping review to collect information on the possible approaches to the running of rapid reviews, based on which, they employed a modified Delphi method that included inputs from 113 stakeholders to explore the most optimized approach. Among the six most frequent rapid review approaches (not all detailed here) being evaluated, the approach that combines inclusion of published literature only, a search of more than one database and limitations by date and language, study selection by one analyst, data extraction, and quality assessment by one analyst and one verifier, was perceived as the most feasible approach (72%, 81/113 responses) with the potentially lowest risk of bias (12%, 12/103). The approach ranked as the first one when considering timelines assumes updating of the search from a previously published review, no additional limits on search, studies selection and data extraction done by one reviewer, and no quality assessment. Finally, based on the publication, the most comprehensive RLRs can be made by moving on with the following rules: searching more than one database and grey literature and using date restriction, and assigning one reviewer working on screening, data extraction, and risk of bias assessment ( Table 1 ). Pandor et al. [ 12 ] introduced a decision tool for SelecTing Approaches for Rapid Reviews (STARR) that were produced through the Delphi consensus of international experts through an iterative and rigorous process. Participants were asked to assess the importance of predefined items in four domains related to the rapid review process: interaction with commissioners, understanding the evidence base, data extraction and synthesis methods, and reporting of rapid review methods. All items assigned to four domains achieved > 70% of consensus, and in that way, the first consensus-driven tool has been created that supports authors of RLRs in planning and deciding on approaches.

Six most frequent approaches to RLRs (adapted from Tricco et al. [ 13 ]).

Haby et al. [ 14 ] run searches of 11 databases and two websites and developed a comprehensive overview of the methodology of RLRs. With five SLRs and one RCT being finally included, they identified the following approaches used in RLRs to make them faster than full SLRs: limiting the number and scope of questions, searching fewer databases, limited searching of grey literature, restrictions on language and date (e.g., English only, most recent publications), updating the existing SLRs, eliminating or limiting hand searches of reference lists, noniterative search strategies, eliminating consultation with experts, limiting dual study selection, data extraction and quality assessment, minimal data synthesis with short concise conclusions or recommendations. All the SLRs included in this review were consistent in stating that no agreed definition of rapid reviews is available, and there is still no final agreement on the best methodological rules to be followed.

Gordon et al. [ 4 ] explained the advantages of performing a focused review and provided 12 tips for its conduction. They define focused reviews as ‘a form of knowledge synthesis in which the components of the systematic process are applied to facilitate the analysis of a focused research question’. The first tip presented by the authors is related to deciding if a focused review is a right solution for the considered project. RLRs will suit emerging topics, approaches, or assessments where early synthesis can support doctors, policymakers, etc., but also can direct future research. The second, third, and fourth tips highlight the importance of running preliminary searches and considering narrowing the results by using reasonable constraints taking into account the local context, problems, efficiency perspectives, and available time. Further tips include creating a team of experienced reviewers working on the RLRs, thinking about the target journal from the beginning of work on the rapid review, registering the search protocol on the PROSPERO registry, and the need for contacting authors of papers when data available in publications are missing or incongruent. The last three tips are related to the choice of evidence synthesis method, using the visual presentation of data, and considering and describing all the limitations of the focused review.

Finally, a new publication by Tricco et al. from 2022, describing JBI position statement [ 15 ] underlined that for the time being, there is no specific tool for critical appraisal of the RLR’s methodological quality. Instead, reviewers may use available tools to assess the risk of bias or quality of SLRs, like ROBIS, the JBI critical appraisal tools, or the assessment of multiple systematic reviews (AMSTAR).

Inconsistency in the definitions and methodologies of RLR

Although RLR was broadly perceived as an approach to quicken the conduct of conventional SLR, there is a lack of consensus on the formal definition of the RLR, so as to the best approaches to perform it. Only in 2021, a study proposing unified definition was published; however, it is important to note that the most accurate definition was only matching slightly over 50% of papers analysed by the authors, which underlines the lack of homogeneity in the field [ 11 ]. The evidence-based supporting methods are evolving, and more evidence is needed to define the most robust approaches [ 5 ].

Diverse terms are used to describe the RLR, including ‘rapid review’, focused systematic review’, ‘quick scoping reviews’, and ‘rapid evidence assessments’. Although the general principles of conducting RLR are to accelerate the whole process, complexity was seen in the methodologies used for RLRs, as reflected in this study. Also, inconsistencies related to the scope of the questions, search strategies, inclusion criteria, study screening, full-text review, quality assessment, and evidence presentation were implied. All these factors may hamper decision-making about optimal methodologies for conducting rapid reviews, and as a result, the efficiency of RLR might be decreased. Additionally, researchers may tend to report the methodology of their reviews without a sufficient level of detail, making it difficult to appraise the quality and robustness of their work.

Advantages and weaknesses of RLR

Although RLR used simplified approaches for evidence synthesis compared with SLR, the methodologies for RLR should be replicable, rigorous, and transparent to the greatest extent [ 16 ]. When time and resources are limited, RLR could be a practical and efficient tool to provide the summary of evidence that is critical for making rapid clinical or policy-related decisions [ 5 ]. Focusing on specific questions that are of controversy or special interest could be powerful in reaffirming whether the existing recommendation statements are still appropriate [ 17 ].

The weakness of RLR should also be borne in mind, and the trade-off of using RLR should be carefully considered regarding the thoroughness of the search, breadth of a research question, and depth of analysis [ 18 ]. If allowed, SLR is preferred over RLR considering that some relevant studies might be omitted with narrowed search strategies and simplified screening process [ 14 ]. Additionally, omitting the quality assessment of included studies could result in an increased risk of bias, making the comprehensiveness of RLR compromised [ 13 ]. Furthermore, in situations that require high accuracy, for example, where a small relative difference in an intervention has great impacts, for the purpose of drafting clinical guidelines, or making licensing decisions, a comprehensive SLR may remain the priority [ 19 ]. Therefore, clear communications with policymakers are recommended to reach an agreement on whether an RLR is justified and whether the methodologies of RLR are acceptable to address the unanswered questions [ 18 ].

Disclosure statement

No potential conflict of interest was reported by the author(s).

Evolution and global research trends of immunity in diabetic nephropathy: a bibliometric and visual analysis from 2004 to 2023

• Nephrology – Original Paper
• Open access
• Published: 17 May 2024

Cite this article

You have full access to this open access article

• Jianlong Zhou 1 ,
• Lv Zhu 2 &
• Rensong Yue 3  

149 Accesses

Explore all metrics

Diabetic nephropathy (DN) is the leading cause of end-stage renal disease, with an increasing prevalence worldwide, but its pathomechanisms remain incompletely understood. Accumulating evidence suggests that immunity plays an important role in the development of DN. Many papers have been published in the field over the last 20 years, but there has been no bibliometric review of the research hotspots and trends in the field. This study aimed to assess the current research status and future trends of the link between immune and DN using bibliometric analysis.

Publications on the association between immunity and DN from 2004 to 2023 were retrieved from the Web of Science Core Collection database and screened according to inclusion criteria. VOSviewer and CiteSpace software were employed to visualize research trends and hotspots in the field. Data including author, institution, country, journal, reference, and keyword were analyzed.

Ultimately 1246 publications meeting the criteria were included in the bibliometric analysis, involving 838 articles (84.96%) and 408 reviews (15.04%). The literature covered 81 countries and regions, 1751 institutions, and 6584 authors. The top 2 countries in terms of the number of publications were China (435) and the United States (318), and they collaborated most frequently. The United States had the highest number of citations for published papers (18,161), far exceeding the other countries. England had 38 publications but had the highest average number of citations (92.32). The University of California system was the most prolific institution (25 papers, 1062 citations, 42.48 citations per paper). Frontiers in Immunology was the most prolific journal in the field (30 papers). The most cited journal was Kidney International (863 citations). The analysis of keywords and references showed that inflammation, ferroptosis, and lipid metabolism may be future research hotspots in this field.

Conclusions

The number of publications related to immunity and DN has increased annually over the past 20 years, with a significant increase in the last 3 years especially. Our results identified research hotspots and trends in the field. These findings provide valuable perspectives for future research, enhancing our understanding of the immune-related mechanisms of DN and exploring potential therapeutic strategies.

Similar content being viewed by others

Global trends and hotspots in IgA nephropathy: a bibliometric analysis and knowledge map visualization from 2012 to 2023

Research trends and frontiers in lupus nephritis: a bibliometric analysis from 2012 to 2022

The evolution and future of diabetic kidney disease research: a bibliometric analysis

Avoid common mistakes on your manuscript.

Introduction

Diabetic nephropathy (DN), a common and serious microvascular complication of diabetes, is the leading cause of end-stage renal disease (ESRD) and is associated with increased risks of cardiovascular disease and death in high-risk individuals [ 1 , 2 ]. The International Diabetes Federation (IDF) estimated that 537 million adults aged 20 to 79 years suffered from diabetes in 2021. The global number of patients with diabetes is projected to increase to 784 million by 2045 [ 3 ]. Up to 40% of people with diabetes will develop chronic kidney disease (CKD) [ 4 ]. According to a recent global burden of disease study, the incidence cases of CKD due to type 2 diabetes worldwide have increased by 74%, from 1.35 million cases in 1990 to 2.35 million cases in 2017 [ 5 ]. DN not only reduces the quality of life of diabetic patients but also imposes a serious economic burden on the family and society [ 6 ]. Hyperglycemia in diabetic patients has long been recognized as the initiating factor for the development of DN [ 7 ]. Renal hemodynamic impairment caused by hyperglycemia and disorders of glucose and lipid metabolism are the two major pathophysiologic bases of DN [ 8 ]. Based on these two points of view, the treatment of DN mainly focuses on the control of blood glucose, blood lipids, blood pressure, and the improvement of renal hemodynamics [ 9 ]. However, even if the control of blood glucose, blood lipids, and blood pressure is achieved, the development of DN cannot be completely prevented [ 10 ]. This suggests that there are other mechanisms involved in the development of DN in addition to the factors mentioned above. Furthermore, although renal replacement therapy (RRT) is the mainstay for patients with ESRD, its accessibility is severely lacking in low- and lower-middle-income countries [ 11 , 12 ]. Unfortunately, the pathogenesis of DN is not fully understood [ 13 ]. Therefore, there is an urgent to improve the understanding of the pathogenesis of DN to diagnose it at an early stage and to discover new therapeutic agents.

The immune system consists of innate and adaptive immunity [ 14 ]. Immune cell infiltration is an important feature of DN [ 15 ]. There is growing evidence that immunity plays an important role in the pathogenesis of DN [ 16 , 17 , 18 , 19 ], which involves key members of the innate and adaptive immune systems [ 20 , 21 , 22 ]. The immune response involved in DN is mainly an intrinsic immune response [ 23 ]. Macrophages are the most common infiltrating cells in DN renal tissues and are associated with decreased renal function. Studies have shown that diabetes mellitus promotes the expression of Toll-like receptor 4 (TLR4) in macrophages and renal tubular cells, which induces inflammatory cytokines, such as interleukin-6 (IL-6), thereby causing amplification of renal tubulointerstitial inflammation and exacerbation of injury [ 24 ]. In renal biopsy tissues from patients with DN, macrophages were found to accumulate predominantly in renal tubules that underwent injury (e.g., around tubular dilatation, atrophy, and apoptotic cells) during the early stages of DN, which was significantly and positively correlated with the patient's serum creatinine and proteinuria levels as well as glomerulosclerosis and interstitial fibrosis [ 25 ]. Adaptive immune system component cells include helper (CD4 +) T cells, cytotoxic (CD8 +) T cells, and B cells. The development of kidney disease in diabetic patients is associated with activation of circulating T cells and increased T cells and C–C motif chemokine 5 (CCL5) in the kidney [ 26 ]. Besides, a study using single-cell RNA sequencing found that immune cell marker genes including EIF4B, RICTOR, and PRKCB were significantly higher expressed in diabetic kidney specimens than in controls, and experimentally validated that they may serve as potential therapeutic targets for DN [ 27 ]. Overall, these findings suggest a major role for immunity in the development of DN.

Emerging in 1969, bibliometrics is a complex analytical method that combines mathematics, statistics, and bibliography [ 28 ]. Bibliometric analysis differs from traditional systematic evaluation in that it allows for quantitative and qualitative assessment of publications. Bibliometric analysis and visualization of literature, authors, institutions, and countries/regions can help to understand the research hotspots and trends in a particular field over a certain period [ 29 ]. Several researchers have used bibliometrics to explore the relationship between intestinal microbiota and DN [ 30 ], mucosal immunity, and IGA nephropathy [ 31 ], as well as the immune system and osteoporosis [ 32 ]. Over the past two decades, more and more studies on the relationship between immunity and DN have begun to appear [ 16 , 17 , 18 , 19 ]. However, to the best of our knowledge, there is currently no published bibliometric analysis focusing on immunization and DN. Therefore, to gain a more comprehensive understanding of the evolution and trends in this field, we visualized various bibliometric indicators by combing the research results on immunization and DN during the past twenty years by using bibliometric analysis tools. We hope to provide valuable insights for future research through the construction of a scientific knowledge graph of the field.

Materials and methods

Data sources and search strategy.

The Web of Science Core Collection (WoSCC) database ( https://www.webofscience.com/wos/woscc/basic-search ) has better accuracy in labeling literature types than any other database and is considered the best choice for literature analysis [ 33 ]. Therefore, we chose to conduct the retrieval in this database. On January 15, 2024, we searched the WOS database for all articles published between 2004 and December 31, 2023, related to the role of immunity in DN. The search strategy was as follows: ((((((TS = (immunity)) OR TS = (immunize)) OR TS = (immune)) OR TS = (immunization)) OR TS = (immunifaction))) OR TS = (immunothera*) AND (((((((((((((((((TS = (Diabetic Nephropathies)) OR TS = (Nephropathies, Diabetic)) OR TS = (Nephropathy, Diabetic)) OR TS = (Diabetic Nephropathy)) OR TS = (Diabetic Kidney Disease)) OR TS = (Diabetic Kidney Diseases)) OR TS = (Kidney Disease, Diabetic)) OR TS = (Kidney Diseases, Diabetic)) OR TS = (Diabetic Glomerulosclerosis)) OR TS = (Glomerulosclerosis, Diabetic)) OR TS = (Intracapillary Glomerulosclerosis)) OR TS = (Nodular Glomerulosclerosis)) OR TS = (Glomerulosclerosis, Nodular)) OR TS = (Kimmelstiel-Wilson Syndrome)) OR TS = (Kimmelstiel Wilson Syndrome)) OR TS = (Syndrome, Kimmelstiel-Wilson)) OR TS = (Kimmelstiel-Wilson Disease)) OR TS = (Kimmelstiel Wilson Disease).

Study selection and data extraction

All relevant publications were independently assessed by two authors and each disagreement was fully discussed with a third author. Literature selection for this study was based on the following inclusion criteria: (1) there were full-text publications related to the role of immunity in DN; (2) the manuscripts of the articles and reviews were written in English; and (3) the literature was published from January 1, 2004, to December 31, 2023. The exclusion criteria were as follows: (1) the topic was not related to the role of immunization in DN; and (2) the type of article was a conference abstract, news, briefing paper, etc. Plain text versions of the publications were exported for analysis. The following data were extracted from the included publications: title, institution, country, journal, year of publication, number of citations, keywords, references, etc.

Bibliometric and visualized analysis

GraphPad Prism v8.0.2 software was used to analyze trends and proportions of annual publications and national publications. In addition, CtieSpace (Premium version 6.2.4R (64-bit)) and VOSviewer (version 1.6.18) were used to analyze the extracted data and visualize the scientific knowledge graph. The VOSviewer v.1.6.17, created by Waltman et al. in 2009, is a free JAVA-based software for analyzing large amounts of literature data and displaying it as a map format [ 34 ]. In this study, VOSviewer software was used to create visual graphs and analyze the most prolific journals, author collaborations, and high frequency keywords. To visualize the results of research in a particular field by mapping the literature co-citation network, Professor Chaomei Chen created the CiteSpace (6.1.6R) software, which envisions the use of an experimental framework for studying new concepts and evaluating existing technologies [ 35 ]. This enables users to better understand areas of knowledge, research frontiers, and trends, and to predict their future research perspectives. This study used CiteSpace software to visualize country/institution collaborations, co-cited journals, co-cited authors, co-cited references, and keyword clustering.

This study was a bibliometric analysis of existing publications and did not require ethical approval.

Literature search and characteristics

According to the search strategy, the WoSCC database contained 1431 publications on the role of immunity in DN. First, we excluded 148 studies that were not between 2004 and 2023. Then, 27 studies including book chapters, errata, and editorial material were excluded. Ultimately, after excluding 10 studies that were not written in English, 1246 studies meeting the inclusion criteria were included in the bibliometric analysis, including 838 (84.96%) articles and 408 reviews (15.04%). The specific literature search process is shown in Fig.  1 . The literature covered 81 countries and regions, 1751 institutions, and 6584 authors.

Flow diagram of the literature search

Since 2004, there has been a slow increase in the number of publications per year (Fig.  2 A). We divided the growth process of publications into three stages (Fig.  2 A). The number of publications grew slowly from 2004 to 2008, with fewer than 30 articles per year, suggesting that the field has not received much attention from researchers. The number of publications gradually increased from 2009 to 2013, indicating that the area has gradually entered the researchers' field of vision. After 2014, the number of publications in this field increased rapidly and peaked in 2023, which shows that the field has received widespread attention since 2014.

A Trend chart of annual publication volume. B Line graph of national annual literature publication volume. C Heatmap of national annual literature publication volume

Countries and institutions

Research on the role of immunity in DN has been conducted in 81 countries and regions. Figure 2 B and C showed the annual publication volume of the top 10 countries during the last two decades, and the top 5 countries in this field were the United States, China, Germany, Italy, and Iran. China accounts for 34.91% of the total number of papers published, far more than any other country (Table  1 ). Among the top ten countries/regions in terms of the number of published papers, papers from the United States were cited 18,161 times (Table  1 ), far exceeding all other countries/regions, and its citation/publication ratio (57.11) ranked 5th among all countries/regions, indicating that the quality of its published papers is generally high. China ranked first in the number of publications (435) while ranking second in the number of citations (10,756), and its citation/publication ratio (24.73) ranked at the back of the list, suggesting that the quality of its published papers is generally low.

The collaboration network among countries (Fig.  3 A) showed that there was close cooperation between China with the highest production and the United States. The United States had close cooperation with Germany, Italy, and the United Kingdom, while China cooperated more closely with India, Japan, and Egypt. China had not only a large number of publications but also a high citation frequency, indicating that it was currently the leading country in the field. In recent years, countries such as the United States and Japan have seen a rapid increase in the number of publications, which may be related to China's cooperation.

A Collaboration network diagram among countries. B Collaboration network diagram among institutions. C Density chart of published literature in journals. D Journal co-cited network map

1751 institutions systematically published papers on the role of immunity in DN. Among the top ten institutions in terms of number of publications, five were from the United States, two were from China, one was from Egypt, one was from Australia, and one was from Spain (Table  2 , Fig.  3 B). The University of California System published the most literature (25 papers, 1062 citations, 42.48 citations per paper). Egyptian Knowledge Bank (EKB) (24 papers, 296 citations, 12.33 citations per paper) ranked second and Harvard University (21 papers, 1411 citations, 67.19 citations per paper) ranked third. After further analysis, we found that domestic and foreign institutions prefer to cooperate with their domestic units, so we call for strengthening cooperation between domestic and foreign institutions and breaking down academic barriers.

Journals analysis

Table 3 and Fig.  3 C showed the top 10 most productive and most cited journals. The Frontiers in Immunology (30, 2.41%) was the journal with the most publications in this field, followed by Frontiers in Endocrinology (29, 2.33%), International Journal of Molecular Sciences (29, 2.33%) and Journal of the American Society of Nephrology (24 articles, 1.93%). Among the top ten most prolific journals, Nature Reviews Nephrology has the highest impact factor (IF) of 41.5. All these journals were classified as Q1 or Q2.

Journal impact is determined by its frequency of being co-cited, which indicates whether the journal has a significant impact on the scientific community [ 36 ]. According to Fig.  3 D and Table  4 , the journal with the highest number of co-citations is Kidney International (863 citations), followed by the Journal of the American Society of Nephrology (827 citations) and Diabetes (707). Among the top 10 most co-cited journals, Kidney International was cited 863 times and had the highest IF of 19.6. Of the co-cited journals, all were distributed in Q1 or Q2.

Co-cited references analysis

With a time slice of one year and a period ranging from 2000 to 2023, the co-cited references network had 960 nodes and 3,478 links (Fig.  4 A). Among the top 10 most co-cited references (Table  5 ), the article titled " Innate immunity in diabetic kidney disease" in Nature Reviews Nephrology (IF = 41.5) ranked first as the most co-cited reference, and Escobar- Morreale, Hector F was the first author of the article [ 37 ]. Increasing evidence suggests that renal inflammation is an important factor in the pathogenesis and progression of DN and anti-inflammatory therapies may be nephroprotective in DN. In this context, immune cells that activate innate immunity and renal resident cells play a crucial role in triggering and maintaining inflammation. In this article [ 37 ], the authors also discussed the mechanisms by which innate immune pathways may contribute to DN and the therapeutic potential of targeting these pathways. For example, Toll-like receptors can induce aseptic tubulointerstitial inflammatory responses through the NF-kB signaling pathway. The NLRP3 inflammasome links the sensing of metabolic stress in diabetic kidneys to the activation of proinflammatory cascades through the induction of IL-1 β and IL-18. Thus, the author suggests that research targeting these innate immune pathways may lead to the development of novel therapies for DN [ 37 ].

A Co-cited references network diagram. B Cluster analysis of co-cited references. C Timeline ridge plot of co-cited references

We performed co-citation reference clustering and temporal clustering analysis (Fig. 4 B and C). We found that hyperglycemia (cluster3), systems biology (cluster9), glycation (cluster10), and chemokines (cluster11) were the early research hotspots. Toll-like receptor (cluster2), immune complexes (cluster5), nuclear hormone receptor (cluster7), human immunodeficiency virus (cluster12), rage (cluster 13), complement system (cluster14), and triptolide (cluster15) were research hotspots in the mid-term. Ferroptosis (cluster0), cytokines (cluster1), b7-1 (cluster4), inflammasome (cluster6), p33-mapk (cluster8), and mesangial cells (cluster16) were the hot topics and trends in this field.

Keywords analysis

By analyzing the keywords, we can quickly understand the overview and development direction of this field. Based on the co-occurrence of keywords in VOSwiever software, the most popular keyword was inflammation (269), followed by expression (207), oxidative stress (143), and activation (137) (Table  6 , Fig.  5 A, B). After removing useless keywords and merging synonyms, We constructed a network containing 172 keywords with at least 12 occurrences, yielding a total of 4 different clusters. Cluster 1 (red) had 54 keywords mainly related to risk factors for DN such as insulin resistance, obesity, diabetes mellitus, genes, metabolic syndrome and so on. Cluster 2 (green) had 53 keywords involving oxidative stress, nitric oxide, and so on.. Cluster 3 (blue) contained 40 keywords, mainly related to immunity and inflammation, such as macrophages, immune complexes, and so on. Cluster 4 (yellow) contained 25 keywords related to pathological changes in DN, such as fibrosis, renal injury, focal segmental glomerulosclerosis, etc. We plotted a cluster diagram through CiteSpace software to visualize the research hotspots over time (Fig. 5 C, D).

A High-frequency keywords network diagram. B Density chart of keywords. C keywords clustering timeline ridge plot. D Cluster analysis of keywords

Burst analysis of co-cited references and keywords

Using CiteSpace software, we derived the 50 most reliable citation bursts in the field of the role of immunity in DN. One of the most cited (12.77) references was titled " Toll-Like Receptor 4 Promotes Tubular Inflammation in Diabetic Nephropathy" published in the Journal of the American Society of Nephrology [ 38 ]. The first author of the article was Lin, Miao, which concluded that inflammation was a cause of tubulointerstitial lesions in DN. Toll-like receptors can regulate immune responses and inflammatory diseases, but their function in DN is unclear. In this study [ 38 ], the authors found that the expression of TLR4 was increased in the renal tubules of patients with DN and the intensity of its expression was directly related to interstitial macrophage infiltration and hemoglobin A1c levels and inversely associated with the estimated glomerular filtration rate. In vitro experiments demonstrated that hyperglycemia induces TLR4 expression through protein kinase C activation in a time- and dose-dependent manner, leading to up-regulated expression of IL-6 and chemokine (C–C motif) ligand 2 (CCL-2) in human proximal tubular epithelial cells through IκB/NF-κB activation. Overall, these studies indicated that TLR4-mediated pathways may promote tubulointerstitial inflammation in DN [ 38 ]. All 50 references were published from 2004 to 2023, suggesting that these papers were frequently cited during the last 20 years. Importantly, 27 of these papers are currently at peak citation (Fig.  6 A), which implies that the study of the role of immunity in DN will continue to be of interest in the future.

A Visualization chart of references with the strongest citation bursts. B Visualization chart of keywords with the strongest citation bursts

Among the 786 strongest bursts of keywords in the field, we focused on the 50 keywords with the strongest bursts (Fig.  6 B), including microalbuminuria, low density lipoprotein, renin angiotensin system, insulin resistance, macrophages, nod mice, toll-like receptors, systemic lupus erythematosus, dendritic cells, inflammatory cytokines, immune cell infiltration, etc. These keywords represent current research hotspots in the field and possible future research trends.

More and more studies show that the occurrence and progression of DN is closely related to immunity [ 39 , 40 ]. So far, our study is the first bibliometric review, summary, and foresight in the field. In this study, we analyzed 1,246 pieces of literature from the WoSCC on the role of immunity in DN, and summarized the trends and potential future research hotspots in the field over the past 20 years, hoping to provide insightful perspectives for future studies.

General information

The number of papers published each year can visually reflect the development status and popularity of a particular field. Our study found that the literature on the role of immunity in DN has tended to increase annually over the past 20 years. It could be roughly categorized into three phases: 2004–2008 was a stagnant period with less than 30 articles per year; 2009–2013 was a slow-growth period with between 30 and 50 articles; and 2014–2023 was a rapid growth period, with a rapid increase in the number of papers per year in the field and reaching a peak in 2023, especially in the last three years when the number of papers per year has exceeded 120. This indicates that the field has attracted sustained academic attention and has become a new hotspot in current research on the DN. Therefore, we venture to speculate that the number of publications in this field will continue to increase.

The number of publications in a research field is an important indicator to assess the level of scientific research of a country, institution or author [ 41 ]. Our study found that 81 countries and regions have conducted research on the role of immunity in DN. The top 10 countries published a total of 1,165 publications, with the top 2 countries being China (435) and the United States (318). Obviously, the outstanding contribution of the United States in this field is due to its long-term scientific research foundation, a large number of excellent research talents, abundant research funding, and relatively free academic atmosphere [ 42 ]. Interestingly, China, as a developing country, has surpassed the United States to rank first in the number of publications in this field, despite its weak research foundation and late start. This may be related to our large population base, the strategy of rejuvenating the country through science and education, and the fact that we have the largest number of diabetics worldwide [ 43 ]. Generally, the total number of citations is a major measure of the quality and attractiveness of an article [ 44 ]. Although China ranked first in terms of the number of publications, the total number of citations in the literature was 10,756, and the average number of citations per publication was 24.73, which ranked 8th among the top ten countries in terms of the number of publications. This showed that the quality of the papers published in China was generally low. While the United States had the highest number of citations for publications at 18,161, far outstripping other countries. Notably, the number of papers published in England was 38, but the average number of citations per publication was as high as 92.32, ranking first. This reflected the higher quality of the papers published in England and the United States. Regarding country collaboration, we found close and extensive cooperation between China and the United States. The United States cooperated closely with countries such as Germany, Italy, and England, while China cooperated even more closely with countries such as India, Japan, and Egypt. From the analysis of institutions, among the top ten institutions in terms of the number of publications, five were from the United States and two were from China. Of which, the University of California System, which belongs to the United States, published the most publications (25 papers, 1,062 citations, 42.48 citations per paper). This also reflected the strong research strength of the United States. Shanghai Jiao Tong University from China ranked sixth (18 papers, 920 citations, 51.11 citations per paper), which may be related to Shanghai's location in China's Yangtze River Delta. This may be due to the fact that it is one of the most active regions for economic development, openness, and innovation in China [ 45 ]. After further analysis, we found that domestic and foreign institutions preferred to collaborate with organizations within their own countries, and we speculated that this phenomenon might be related to factors such as geographical and cultural differences, varying quality of research methods, and official barriers. If these challenges can be actively addressed, this will facilitate broader international cooperation and innovation, ultimately advancing the field and contributing to scientific progress globally.

Journal source distribution and journal co-citation analysis can provide researchers with valuable information that can help them quickly find the most appropriate target journal when searching the literature or submitting a study [ 46 ]. Among the top ten most prolific journals, Frontiers in Immunology (30 papers, 2.41%) and Frontiers in Endocrinology (29 papers, 2.33%) from the Frontiers publishing platform were ranked first and second, respectively. This indicated that the majority of articles in the field might be considered for publication on this publishing platform. In addition, our study found that Nature Reviews Nephrology was the journal with the highest impact factor of 41.5. The most co-cited journal was Kidney International (863 citations), followed by the Journal of the American Society of Nephrology (827 citations) and Diabetes (707 citations). Most of these journals were located in the Q1 region. This showed that studies on the role of immunity in DN were often able to be published in journals of high impact and scholarly value.

Research hotspots and trends

To reveal the research frontiers and trends in the field, we performed cluster analysis, co-occurrence analysis, and burst analysis of co-cited references and keywords. The most co-cited reference was entitled " Innate immunity in diabetic kidney disease". This article explored in detail the mechanisms by which innate immune pathways may contribute to DKD and the therapeutic potential of targeting these pathways. Timeline clustering analysis of the co-cited references showed that the latest hot topics in the field involved ferroptosis, cytokines, b7-1, and inflammasome. According to the co-occurrence of keywords, the hottest keywords were mainly inflammation and oxidative stress. Burst analysis of references showed the most cited reference was entitled " Toll-Like Receptor 4 Promotes Tubular Inflammation in Diabetic Nephropathy". The article emphasized the role of TLR4 in promoting tubular inflammation in DN. The 50 keywords with the strongest citation bursts in this field were mainly related to low density lipoprotein, renin angiotensin system, insulin resistance, macrophages, toll-like receptors, dendritic cells, inflammatory cytokines, immune cell infiltration, and so on. Overall, immune-related mechanisms are still the hotspots of DN research, and future research trends may involve the following three aspects: (1) the mechanism of immune-related inflammation involved in DN; (2) the role of ferroptosis in immunity and DN; and (3) the role of lipid metabolism in immunity and DN.

DN was thought to be the result of the interaction of hemodynamic and metabolic factors. Its pathogenesis involved many factors and pathways, among which immune cells and immune-related chronic inflammatory responses played an important role in the occurrence and development of DN [ 20 ]. Macrophages are key members of the mononuclear phagocyte system and part of innate immunity. High glucose and late glycosylation end products in the DN environment promote macrophage recruitment, migration, and activation, and activated macrophages release proinflammatory factors, leading to renal injury and fibrosis [ 47 ]. Dendritic cells (DCS) are the primary regulators of innate and adaptive immune responses. It can interact with B cells and T cells to manipulate humoral and cellular immune responses [ 48 ]. Studies have shown that high glucose can trigger DC maturation and induce a proinflammatory cytokine profile in human DCs, which subsequently mediates tubulointerstitial injury in DN [ 49 , 50 ]. Besides that, there were many other inflammation-related cytokines aggregated in the renal tissues of patients in the development of DN [ 18 , 19 , 20 ], such as chemokines [ 51 ], cell adhesion factors [ 52 ], growth factors, inflammatory factors, nuclear factors, and so on. However, this mechanism remains controversial and deserves further investigation.

Ferroptosis, a novel mode of cell death induced by iron-dependent oxidative damage, is characterized by lipid peroxidation due to intracellular iron overload and accumulation of reactive oxygen species (ROS) [ 53 ]. Recent studies have shown that ferroptosis has been identified as one of the forms of cell death for several immune cells, affecting the immune response. There may be potential interactions between ferroptosis and the immune response in some cases [ 54 ]. Iron ions were reabsorbed by renal tubules after glomerular filtration, and renal tubular epithelial cells were extremely active sites for iron ions and ROS. Characteristic changes of ferroptosis, such as iron overload, lipid peroxidation, and mitochondrial alterations, were detected in human renal proximal tubular epithelial cells (HK-2) cultured in high glucose [ 55 ]. It was found that ferroptosis was closely associated with renal tubular cell death in diabetic conditions [ 56 ]. These findings have also been used to explain the action mechanisms of drugs which may be potentially valuable in the treatment of DN. For example, Ghanim et al.'s study found that dagliflozin was effective in reducing serum iron-regulatory protein levels and improving iron metabolism disorders in patients with type 2 diabetes mellitus [ 57 ]. Huang et al. observed that liraglutide alleviated glomerular extracellular matrix accumulation and renal injury in DN by enhancing Wnt/β-catenin signaling [ 58 ]. Therefore, an in-depth exploration of the mechanism of ferroptosis in immunity and DN could provide new therapeutic ideas and potential drug targets for DN patients.

Studies have shown that lipid metabolism could regulate the differentiation and function of immune cells [ 59 ]. T helper cells secreting IL-17 play a pathogenic role in a variety of inflammatory and autoimmune diseases, and its development requires endogenous fatty acid synthesis [ 60 ]. Different types of macrophages have different requirements for fatty acid synthesis and catabolism. Factors that promote M1 macrophages induce fatty acid synthesis, whereas anti-inflammatory signals that favor M2 macrophages drive fatty acid oxidation [ 61 ]. Conversely, immune cells are an important part of the lipid microenvironment, influencing local and systemic lipid metabolism [ 62 ]. Renal lipid homeostasis has received increasing attention in recent years [ 63 ]. Altered fatty acid and cholesterol metabolism were recognized as crucial pathways for renal lipid accumulation, inflammation, oxidative stress, and fibrosis [ 64 ]. Oxidized low-density lipoprotein (ox-LDL) may promote the development of DN by damaging renal endothelial, thylakoid, and tubular cells through lipid metabolic pathways, inflammatory damage, and hemodynamic factors [ 65 ]. In addition, abnormal renal lipid metabolism was found to be present in the kidneys of mice with DN and high glucose-induced renal podocytes [ 66 ]. Thus, podocyte lipid accumulation has been considered a potential therapeutic target for DN. Looking forward, understanding more regulatory details of lipid metabolism involved in immunity and DN will provide more valuable clues for the study and treatment of DN.

Limitations

Although, we were able to gain a detailed view of the research evolution and global trends in the relationship between immunity and DN using visual analysis tools such as CiteSpace and VOSviewer. However, this study also has some limitations. First, we only retrieved the WoSCC database in this study, and there were many other noteworthy databases that have not yet been searched, such as the Scopus, PubMed, and Embase databases. However, considering that the WoSCC database is one of the most used and comprehensive global databases [ 67 , 68 , 69 , 70 , 71 ], it is sufficient to reflect the general trend in this research field. Second, we only analyzed studies published in English, which could lead to the fact that some non-English studies could be missed due to language limitations. Finally, considering the presence of near-synonyms, abbreviations, and full names, the keyword bursts may differ from the actual results. Therefore, the results need to be interpreted with caution.

In conclusion, immunization plays an important role in the development of DN. The number of publications related to immunity and DN has increased annually over the past 20 years, with a significant increase especially in the last 3 years, indicating that researchers are increasingly interested in this field. China and the United States are the two countries with the highest number of publications and close collaboration. The University of California System is the most prolific institution. Frontiers in Immunology is the journal with the most publications in the field. Future directions may involve immune-related inflammation, ferroptosis, and lipid metabolism. This study summarized the current situation and global trends in research on the role of immunity in DN and provided valuable suggestions and ideas for future research.

Data availability

All original contributions are included in the manuscript.

Reddy MA, Sumanth P, Lanting L, Yuan H, Wang M, Mar D, Alpers CE, Bomsztyk K, Natarajan R (2014) Losartan reverses permissive epigenetic changes in renal glomeruli of diabetic db/db mice. Kidney Int 85:362–373. https://doi.org/10.1038/ki.2013.387

Article   CAS   PubMed   Google Scholar  

Kim H, Bae YU, Jeon JS, Noh H, Park HK, Byun DW, Han DC, Ryu S, Kwon SH (2019) The circulating exosomal microRNAs related to albuminuria in patients with diabetic nephropathy. J Transl Med 17:236. https://doi.org/10.1186/s12967-019-1983-3

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bae JH, Han KD, Ko SH, Yang YS, Choi JH, Choi KM, Kwon HS, Won KC (2022) Diabetes fact sheet in Korea 2021. Diabetes Metab J 46:417–426. https://doi.org/10.4093/dmj.2022.0106

Article   PubMed   PubMed Central   Google Scholar  

Yamada K, Takata T, Iyama T, Hamada S, Mae Y, Sugihara T, Isomoto H (2022) Fluorescence imaging using enzyme-activatable probes for detecting diabetic kidney disease and glomerular diseases. Int J Mol Sci. https://doi.org/10.3390/ijms23158150

Li H, Lu W, Wang A, Jiang H, Lyu J (2021) Changing epidemiology of chronic kidney disease as a result of type 2 diabetes mellitus from 1990 to 2017: estimates from global burden of disease 2017. J Diabetes Investig 12:346–356. https://doi.org/10.1111/jdi.13355

Article   PubMed   Google Scholar  

Yang SK, Li AM, Han YC, Peng CH, Song N, Yang M, Zhan M, Zeng LF, Song PA, Zhang W et al (2019) Mitochondria-targeted peptide SS31 attenuates renal tubulointerstitial injury via inhibiting mitochondrial fission in diabetic mice. Oxid Med Cell Longev 2019:2346580. https://doi.org/10.1155/2019/2346580

Peng J, Li X, Zhang D, Chen JK, Su Y, Smith SB, Dong Z (2015) Hyperglycemia, p53, and mitochondrial pathway of apoptosis are involved in the susceptibility of diabetic models to ischemic acute kidney injury. Kidney Int 87:137–150. https://doi.org/10.1038/ki.2014.226

Xu T, Xu X, Zhang L, Zhang K, Wei Q, Zhu L, Yu Y, Xiao L, Lin L, Qian W et al (2021) Lipidomics reveals serum specific lipid alterations in diabetic nephropathy. Front Endocrinol (Lausanne) 12:781417. https://doi.org/10.3389/fendo.2021.781417

Jiang ZH, Tang YZ, Song HN, Yang M, Li B, Ni CL (2020) miRNA-342 suppresses renal interstitial fibrosis in diabetic nephropathy by targeting SOX6. Int J Mol Med 45:45–52. https://doi.org/10.3892/ijmm.2019.4388

Hwang YC, Kim SW, Hur KY, Cha BS, Kim IJ, Park TS, Baik SH, Yoon KH, Lee KW, Lee IK et al (2019) Predictive factors for efficacy of AST-120 treatment in diabetic nephropathy: a prospective single-arm, open-label. Multi-Center Study J Korean Med Sci 34:e117. https://doi.org/10.3346/jkms.2019.34.e117

Htay H, Bello AK, Levin A, Lunney M, Osman MA, Ye F, Ashuntantang GE, Bellorin-Font E, Gharbi MB, Davison SN et al (2021) Hemodialysis use and practice patterns: an international survey study. Am J Kidney Dis 77:326-335.e321. https://doi.org/10.1053/j.ajkd.2020.05.030

Tannor EK, Hutton-Mensah K, Opare-Addo P, Agyei MK, Gyan KF, Inusah AJ, Nyann BI, Amo-Antwi K, Luyckx V, Okpechi I (2023) Fifty years of hemodialysis in Ghana-current status, utilization and cost of dialysis services. BMC Health Serv Res 23:1170. https://doi.org/10.1186/s12913-023-10154-x

Ma T, Li X, Zhu Y, Yu S, Liu T, Zhang X, Chen D, Du S, Chen T, Chen S et al (2022) Excessive activation of notch signaling in macrophages promote kidney inflammation, fibrosis, and necroptosis. Front Immunol 13:835879. https://doi.org/10.3389/fimmu.2022.835879

Hao X, Chi H, Tang X, Xing J, Sheng X, Zhan W (2021) The functions of β-Defensin in flounder (Paralichthys olivaceus): antibiosis chemotaxis and modulation of phagocytosis. Biology (Basel). https://doi.org/10.3390/biology10121247

Wilson PC, Wu H, Kirita Y, Uchimura K, Ledru N, Rennke HG, Welling PA, Waikar SS, Humphreys BD (2019) The single-cell transcriptomic landscape of early human diabetic nephropathy. Proc Natl Acad Sci U S A 116:19619–19625. https://doi.org/10.1073/pnas.1908706116

Fu J, Akat KM, Sun ZG, Zhang WJ, Schlondorff D, Liu ZH, Tuschl T, Lee K, He JC (2019) Single-cell RNA profiling of glomerular cells shows dynamic changes in experimental diabetic kidney disease. J Am Soc Nephrol 30:533–545. https://doi.org/10.1681/asn.2018090896

Sabapathy V, Stremska ME, Mohammad S, Corey RL, Sharma PR, Sharma R (2019) Novel immunomodulatory cytokine regulates inflammation, diabetes, and obesity to protect from diabetic nephropathy. Front Pharmacol 10:11. https://doi.org/10.3389/fphar.2019.00572

Article   CAS   Google Scholar  

Navarro-Gonzalez JF, Mora-Fernandez C (2008) The role of inflammatory cytokines in diabetic nephropathy. J Am Soc Nephrol 19:433–442. https://doi.org/10.1681/asn.2007091048

Duran-Salgado MB, Rubio-Guerra AF (2014) Diabetic nephropathy and inflammation. World J Diabetes 5:393–398. https://doi.org/10.4239/wjd.v5.i3.393

Zheng ZH, Zheng F (2016) Immune cells and inflammation in diabetic nephropathy. J Diabetes Res 2016:10. https://doi.org/10.1155/2016/1841690

Tesch GH (2017) Diabetic nephropathy – is this an immune disorder? Clin Sci 131:2183–2199. https://doi.org/10.1042/cs20160636

Kong LY, Andrikopoulos S, MacIsaac RJ, Mackay LK, Nikolic-Paterson DJ, Torkamani N, Zafari N, Marin ECS, Ekinci EI (2022) Role of the adaptive immune system in diabetic kidney disease. J Diabetes Investig 13:213–226. https://doi.org/10.1111/jdi.13725

Wada J, Makino H (2016) Innate immunity in diabetes and diabetic nephropathy. Nat Rev Nephrol 12:13–26. https://doi.org/10.1038/nrneph.2015.175

Kim SM, Lee SH, Kim YG, Kim SY, Seo JW, Choi YW, Kim DJ, Jeong KH, Lee TW, Ihm CG et al (2015) Hyperuricemia-induced NLRP3 activation of macrophages contributes to the progression of diabetic nephropathy. Am J Physiol Renal Physiol 308:F993-f1003. https://doi.org/10.1152/ajprenal.00637.2014

Meshkani R, Vakili S (2016) Tissue resident macrophages: Key players in the pathogenesis of type 2 diabetes and its complications. Clin Chim Acta 462:77–89. https://doi.org/10.1016/j.cca.2016.08.015

Moon JY, Jeong KH, Lee TW, Ihm CG, Lim SJ, Lee SH (2012) Aberrant recruitment and activation of T cells in diabetic nephropathy. Am J Nephrol 35:164–174. https://doi.org/10.1159/000334928

Lu X, Li L, Suo L, Huang P, Wang H, Han S, Cao M (2022) Single-cell rna sequencing profiles identify important pathophysiologic factors in the progression of diabetic nephropathy. Front Cell Dev Biol 10:798316. https://doi.org/10.3389/fcell.2022.798316

Gao Y, Wang YL, Kong D, Qu B, Su XJ, Li H, Pi HY (2015) Nerve autografts and tissue-engineered materials for the repair of peripheral nerve injuries: a 5-year bibliometric analysis. Neural Regen Res 10:1003–1008. https://doi.org/10.4103/1673-5374.158369

Wang Y, Li W, Wu H, Han Y, Wu H, Lin Z, Zhang B (2023) Global status and trends in gout research from 2012 to 2021: a bibliometric and visual analysis. Clin Rheumatol 42:1371–1388. https://doi.org/10.1007/s10067-023-06508-9

Gao LW, Yang XY, Yu YF, Yin S, Tong KK, Hu G, Jian WX, Tian Z (2023) Bibliometric analysis of intestinal microbiota in diabetic nephropathy. Eur Rev Med Pharmacol Sci 27:8812–8828. https://doi.org/10.26355/eurrev_202309_33802

Chen X, Yan Z, Pan Q, Zhang C, Chen Y, Liang X, Li S, Wu G (2024) Bibliometric analysis of mucosal immunity in IgA nephropathy from 1990 to 2022. Immun Inflamm Dis 12:e1156. https://doi.org/10.1002/iid3.1156

Zhao J, Dou Y, Liang G, Huang H, Hong K, Yang W, Zhou G, Sha B, Liu J, Zeng L (2023) Global publication trends and research hotspots of the immune system and osteoporosis: a bibliometric and visualization analysis from 2012 to 2022. Endocr Metab Immune Disord Drug Targets. https://doi.org/10.2174/0118715303257269231011073100

Wang Y, Xie M, Zhang M, Zhao X, Zhu X, Wang Y, Chen Y, Chen J, Sun X (2021) Publication trends of research on polypoidal choroidal vasculopathy during 2001–2020: a 20-year bibliometric study. Front Med (Lausanne) 8:785126. https://doi.org/10.3389/fmed.2021.785126

van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84:523–538. https://doi.org/10.1007/s11192-009-0146-3

Chen C (2004) Searching for intellectual turning points: progressive knowledge domain visualization. Proc Natl Acad Sci U S A 101(Suppl 1):5303–5310. https://doi.org/10.1073/pnas.0307513100

Zhang J, Lin M, Huang Y, Wang Y, Huang T, Wu Z, Li Z, Xu J, Zhao R, Luo X (2022) Harnessing hyaluronic acid for the treatment of osteoarthritis: a bibliometric analysis. Front Bioeng Biotechnol 10:961459. https://doi.org/10.3389/fbioe.2022.961459

Tang SCW, Yiu WH (2020) Innate immunity in diabetic kidney disease. Nat Rev Nephrol 16:206–222. https://doi.org/10.1038/s41581-019-0234-4

Lin M, Yiu WH, Wu HJ, Chan LY, Leung JC, Au WS, Chan KW, Lai KN, Tang SC (2012) Toll-like receptor 4 promotes tubular inflammation in diabetic nephropathy. J Am Soc Nephrol 23:86–102. https://doi.org/10.1681/asn.2010111210

Han YZ, Zheng HJ, Du BX, Zhang Y, Zhu XY, Li J, Wang YX, Liu WJ (2023) Role of gut microbiota, immune imbalance, and allostatic load in the occurrence and development of diabetic kidney disease. J Diabetes Res 2023:8871677. https://doi.org/10.1155/2023/8871677

Wang J, Wang L, Pang Z, Ge Q, Wu Y, Qi X (2023) Integrated analysis of ferroptosis and immunity-related genes associated with diabetic kidney disease. Diabetes Metab Syndr Obes 16:3773–3793. https://doi.org/10.2147/dmso.S434970

Joshi MA (2014) Bibliometric indicators for evaluating the quality of scientifc publications. J Contemp Dent Pract 15:258–262. https://doi.org/10.5005/jp-journals-10024-1525

He W, Deng Y, Luo X (2022) Bibliometric analysis of the global research status and trends of the association between Vitamin D and infections from 2001 to 2021. Front Public Health 10:934106. https://doi.org/10.3389/fpubh.2022.934106

Xu J, Chen F, Liu T, Wang T, Zhang J, Yuan H, Wang M (2019) Brain functional networks in type 2 diabetes mellitus patients: a resting-state functional MRI study. Front Neurosci 13:239. https://doi.org/10.3389/fnins.2019.00239

Liu GY, Jin Y, Zhang Q, Li R (2015) Peripheral nerve repair: a hot spot analysis on treatment methods from 2010 to 2014. Neural Regen Res 10:996–1002. https://doi.org/10.4103/1673-5374.158368

Feng H, Grifoll M, Yang Z, Zheng P, Martin-Mallofre A (2020) Visualization of container throughput evolution of the Yangtze River Delta multi-port system: the ternary diagram method. Transp Res Part E: Logist Transp Rev 142:102039. https://doi.org/10.1016/j.tre.2020.102039

Article   Google Scholar  

Shen Z, Hu J, Wu H, Chen Z, Wu W, Lin J, Xu Z, Kong J, Lin T (2022) Global research trends and foci of artificial intelligence-based tumor pathology: a scientometric study. J Transl Med 20:409. https://doi.org/10.1186/s12967-022-03615-0

Kim SM, Lee SH, Kim YG, Kim SY, Seo JW, Choi YW, Kim DJ, Jeong KH, Lee TW, Ihm CG et al (2015) Hyperuricemia-induced NLRP3 activation of macrophages contributes to the progression of diabetic nephropathy. Am J Physiol-Renal Physiol. 308:F993–F1003. https://doi.org/10.1152/ajprenal.00637.2014

Chen PF, Liu XS, Sun YF, Zhou P, Wang YL, Zhang YG (2016) Dendritic cell targeted vaccines: recent progresses and challenges. Human Vaccines Immunother 12:612–622. https://doi.org/10.1080/21645515.2015.1105415

Lu H, Yao K, Huang D, Sun AJ, Zou YZ, Qian JY, Ge JB (2013) High glucose induces upregulation of scavenger receptors and promotes maturation of dendritic cells. Cardiovasc Diabetol 12:12. https://doi.org/10.1186/1475-2840-12-80

Kim H, Kim M, Lee HY, Park HY, Jhun H, Kim S (2021) Role of dendritic cell in diabetic nephropathy. Int J Mol Sci 22:13. https://doi.org/10.3390/ijms22147554

Elmarakby AA, Abdelsayed R, Yao Liu J, Mozaffari MS (2010) Inflammatory cytokines as predictive markers for early detection and progression of diabetic nephropathy. EPMA J 1:117–129. https://doi.org/10.1007/s13167-010-0004-7

Bui TM, Wiesolek HL, Sumagin R (2020) ICAM-1: a master regulator of cellular responses in inflammation, injury resolution, and tumorigenesis. J Leukoc Biol 108:787–799. https://doi.org/10.1002/jlb.2mr0220-549r

Yuan S, Wei C, Liu G, Zhang L, Li J, Li L, Cai S, Fang L (2022) Sorafenib attenuates liver fibrosis by triggering hepatic stellate cell ferroptosis via HIF-1α/SLC7A11 pathway. Cell Prolif 55:e13158. https://doi.org/10.1111/cpr.13158

Xu S, Min J, Wang F (2021) Ferroptosis: an emerging player in immune cells. Science Bulletin 66:2257–2260. https://doi.org/10.1016/j.scib.2021.02.026

Song J, Sheng J, Lei J, Gan W, Yang Y (2022) Mitochondrial targeted antioxidant SKQ1 ameliorates acute kidney injury by inhibiting ferroptosis. Oxid Med Cell Longev 2022:2223957. https://doi.org/10.1155/2022/2223957

Zhou LT, Zhang ZJ, Cao JY, Chen H, Zhu YS, Wu X, Nawabi AQ, Liu X, Shan W, Zhang Y et al (2021) The unique molecular mechanism of diabetic nephropathy: a bioinformatics analysis of over 250 microarray datasets. Clin Kidney J 14:1626–1638. https://doi.org/10.1093/ckj/sfaa190

Ghanim H, Abuaysheh S, Hejna J, Green K, Batra M, Makdissi A, Chaudhuri A, Dandona P (2020) Dapagliflozin suppresses hepcidin and increases erythropoiesis. J Clin Endocrinol Metab. https://doi.org/10.1210/clinem/dgaa057

Huang L, Lin T, Shi M, Chen X, Wu P (2020) Liraglutide suppresses production of extracellular matrix proteins and ameliorates renal injury of diabetic nephropathy by enhancing Wnt/β-catenin signaling. Am J Physiol Renal Physiol 319:F458-f468. https://doi.org/10.1152/ajprenal.00128.2020

Fu S, He K, Tian C, Sun H, Zhu C, Bai S, Liu J, Wu Q, Xie D, Yue T et al (2020) Impaired lipid biosynthesis hinders anti-tumor efficacy of intratumoral iNKT cells. Nat Commun 11:438. https://doi.org/10.1038/s41467-020-14332-x

Berod L, Friedrich C, Nandan A, Freitag J, Hagemann S, Harmrolfs K, Sandouk A, Hesse C, Castro CN, Bähre H et al (2014) De novo fatty acid synthesis controls the fate between regulatory T and T helper 17 cells. Nat Med 20:1327–1333. https://doi.org/10.1038/nm.3704

Howie D, Ten Bokum A, Cobbold SP, Yu Z, Kessler BM, Waldmann H (1860) A novel role for triglyceride metabolism in Foxp3 expression. Front Immunol 2019:10. https://doi.org/10.3389/fimmu.2019.01860

Rajbhandari P, Arneson D, Hart SK, Ahn IS, Diamante G, Santos LC, Zaghari N, Feng AC, Thomas BJ, Vergnes L et al (2019) Single cell analysis reveals immune cell-adipocyte crosstalk regulating the transcription of thermogenic adipocytes. Elife. https://doi.org/10.7554/eLife.49501

Opazo-Ríos L, Mas S, Marín-Royo G, Mezzano S, Gómez-Guerrero C, Moreno JA, Egido J (2020) Lipotoxicity and diabetic nephropathy: novel mechanistic insights and therapeutic opportunities. Int J Mol Sci. https://doi.org/10.3390/ijms21072632

D’Agati VD, Chagnac A, de Vries AP, Levi M, Porrini E, Herman-Edelstein M, Praga M (2016) Obesity-related glomerulopathy: clinical and pathologic characteristics and pathogenesis. Nat Rev Nephrol 12:453–471. https://doi.org/10.1038/nrneph.2016.75

Roumeliotis S, Georgianos PI, Roumeliotis A, Eleftheriadis T, Stamou A, Manolopoulos VG, Panagoutsos S, Liakopoulos V (2021) Oxidized LDL modifies the association between proteinuria and deterioration of kidney function in proteinuric diabetic kidney disease. Life (Basel). https://doi.org/10.3390/life11060504

Zhang Y, Yao H, Li C, Sun W, Chen X, Cao Y, Liu Y, Liu Y, Chen J, Qi J et al (2022) Gandi capsule improved podocyte lipid metabolism of diabetic nephropathy mice through SIRT1/AMPK/HNF4A pathway. Oxid Med Cell Longev 2022:6275505. https://doi.org/10.1155/2022/6275505

Wang J, Zhao W, Zhang Z, Liu X, Xie T, Wang L, Xue Y, Zhang Y (2024) A journey of challenges and victories: a bibliometric worldview of nanomedicine since the 21st century. Adv Mater 36:e2308915. https://doi.org/10.1002/adma.202308915

Sun Z, Liu X, Pan D, Wang J (2024) Unveiling the panorama and emerging boundaries in the field of aging biomarkers. Asian J Surg 47:2065–2066. https://doi.org/10.1016/j.asjsur.2024.01.014

Wang L, Wang J, Zhang Y, Zhang H (2024) Current perspectives and trends of the research on hypertensive nephropathy: a bibliometric analysis from 2000 to 2023. Ren Fail 46:2310122. https://doi.org/10.1080/0886022x.2024.2310122

Wang J, Liu X, Pan D, Cai X, Xue Y, Huang J (2024) Chronic kidney disease in the shadow of COVID-19: insights from the bibliometric analysis. Int Urol Nephrol 56:683–697. https://doi.org/10.1007/s11255-023-03706-x

Wang J, Wang X, Cai X, Pan D (2023) Global trends and hotspots in IgA nephropathy: a bibliometric analysis and knowledge map visualization from 2012 to 2023. Int Urol Nephrol 55:3197–3207. https://doi.org/10.1007/s11255-023-03598-x

Download references

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Clinical Medicine, People’s Hospital of Deyang City, Deyang, China

Jianlong Zhou

West China Center of Excellence for Pancreatitis, Institute of Integrated Traditional Chinese and Western Medicine, West China Hospital, Sichuan University, Chengdu, China

Department of Clinical Medicine, Hospital of Chengdu University of Traditional Chinese Medicine, Chengdu, China

Rensong Yue

You can also search for this author in PubMed   Google Scholar

Contributions

JZ: conceptualization, methodology, visualization, original draft preparation. LZ: investigation, visualization. RY: supervision, and revision. All the authors contributed to the article and approved the submitted version.

Corresponding author

Correspondence to Jianlong Zhou .

Ethics declarations

Conflict of interests.

The study was conducted without any business or financial relationships that could be interpreted as potential conflicts of interest.

Ethical approval

This study did not involve animal or human experimentation. Therefore, ethical approval from an institutional review board was not required. Author Contributions.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Zhou, J., Zhu, L. & Yue, R. Evolution and global research trends of immunity in diabetic nephropathy: a bibliometric and visual analysis from 2004 to 2023. Int Urol Nephrol (2024). https://doi.org/10.1007/s11255-024-04081-x

Download citation

Received : 06 April 2024

Accepted : 12 May 2024

Published : 17 May 2024

DOI : https://doi.org/10.1007/s11255-024-04081-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Diabetic nephropathy
• Bibliometric
• Visual analysis
• Research trends
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 23.5.2024 in Vol 12 (2024)

Digital Phenotyping for Stress, Anxiety, and Mild Depression: Systematic Literature Review

Authors of this article:

• Adrien Choi, BAdvSci(Hons)   ; 
• Aysel Ooi, BSocSci, MInfoTech   ; 
• Danielle Lottridge, PhD  

School of Computer Science, Faculty of Science, University of Auckland, Auckland, New Zealand

Corresponding Author:

Danielle Lottridge, PhD

School of Computer Science

Faculty of Science

University of Auckland

38 Princes Street

Auckland, 1010

New Zealand

Phone: 64 9 373 7599 ext 82930

Email: [email protected]

Background: Unaddressed early-stage mental health issues, including stress, anxiety, and mild depression, can become a burden for individuals in the long term. Digital phenotyping involves capturing continuous behavioral data via digital smartphone devices to monitor human behavior and can potentially identify milder symptoms before they become serious.

Objective: This systematic literature review aimed to answer the following questions: (1) what is the evidence of the effectiveness of digital phenotyping using smartphones in identifying behavioral patterns related to stress, anxiety, and mild depression? and (2) in particular, which smartphone sensors are found to be effective, and what are the associated challenges?

Methods: We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) process to identify 36 papers (reporting on 40 studies) to assess the key smartphone sensors related to stress, anxiety, and mild depression. We excluded studies conducted with nonadult participants (eg, teenagers and children) and clinical populations, as well as personality measurement and phobia studies. As we focused on the effectiveness of digital phenotyping using smartphones, results related to wearable devices were excluded.

Results: We categorized the studies into 3 major groups based on the recruited participants: studies with students enrolled in universities, studies with adults who were unaffiliated to any particular organization, and studies with employees employed in an organization. The study length varied from 10 days to 3 years. A range of passive sensors were used in the studies, including GPS, Bluetooth, accelerometer, microphone, illuminance, gyroscope, and Wi-Fi. These were used to assess locations visited; mobility; speech patterns; phone use, such as screen checking; time spent in bed; physical activity; sleep; and aspects of social interactions, such as the number of interactions and response time. Of the 40 included studies, 31 (78%) used machine learning models for prediction; most others (n=8, 20%) used descriptive statistics. Students and adults who experienced stress, anxiety, or depression visited fewer locations, were more sedentary, had irregular sleep, and accrued increased phone use. In contrast to students and adults, less mobility was seen as positive for employees because less mobility in workplaces was associated with higher performance. Overall, travel, physical activity, sleep, social interaction, and phone use were related to stress, anxiety, and mild depression.

Conclusions: This study focused on understanding whether smartphone sensors can be effectively used to detect behavioral patterns associated with stress, anxiety, and mild depression in nonclinical participants. The reviewed studies provided evidence that smartphone sensors are effective in identifying behavioral patterns associated with stress, anxiety, and mild depression.

Introduction

Digital phenotyping is “the moment-by-moment quantification of the individual level human phenotype in situ using data from personal digital devices” [ 1 ]. Digital phenotyping applies the concept of phenotypes, in other words, the observable characteristics resulting from the genotype and environment, to conceptualize observable patterns in individuals’ digital data. In the last decade, digital phenotyping studies have been able to compare typical and atypical patterns in daily activities to correlate atypical behavior with negative emotions [ 2 , 3 ]. Behavioral patterns include variations in mobility, frequency of being in various locations, and sleep patterns. In smartphones, user data can be stored, managed, interpreted, and captured in enormous amounts [ 1 , 4 , 5 ]. This can be done actively or passively. Active data collection requires the user to self-report and complete surveys, whereas passive sensing collects data automatically without user input [ 5 ]. Most studies combine active and passive sensing to more accurately detect and predict behavioral abnormalities. Modern smartphone analytics can be used for the discovery of commonalities and abnormalities in user behavior. The ease of using passive sensing makes it an ideal data gathering method for mental health studies [ 6 - 8 ] and an ideal technique for assessing mental health [ 9 ].

Digital phenotyping has been successful in the early detection and prediction of behaviors related to neuropharmacology [ 10 ]; cardiovascular diseases [ 11 ]; diabetes [ 12 ]; and major severe injuries, such as spinal cord injury [ 13 ], motivating further adoption. Digital phenotyping has also proven useful for the detection of severe mental health issues, such as schizophrenia [ 14 , 15 ], bipolar disorder [ 16 ], and suicidal thoughts [ 17 ]. Digital phenotyping has been so successful for specialized, clinical populations that it is increasingly considered for mass market use with nonclinical populations. Digital phenotyping applications and software tools have been used to capture employee information, such as their screen time and clicking patterns [ 18 ]. However, there are not many digital phenotyping studies that have specifically examined the detection or prediction of stress, anxiety, and mild depression.

Individuals with stress, anxiety, and mild depression can develop chronic mental health symptoms that impact their mobility, satisfaction with life, and social interaction [ 19 , 20 ]. When these symptoms are not detected early, they worsen, and the impact is more significant [ 21 - 23 ], increasing the need for medication and hospitalization. This makes mild mental health symptoms a valid target for digital phenotyping, as its goal is to enable early detection and, subsequently, early treatment. Smartphones are increasingly ubiquitous [ 24 ], which makes them an optimal platform for digital phenotyping. We constrained our systematic literature search to the more challenging problem of the detection of mild mental health symptoms using only smartphone sensors and excluded studies that used additional wearable sensors. In general, we believe that additional wearables might increase the effectiveness of digital phenotyping in detecting stress, anxiety, and mild depression. Given the ubiquity of smartphones, we aimed to answer the following question: what is the effectiveness of digital phenotyping using smartphone sensors in detecting stress, anxiety, and mild depression?

The objective of this systematic literature review was to better understand the current uses of digital phenotyping and results of using digital phenotyping for the detection and prediction of mild behavioral patterns related to stress, anxiety, and mild depression. The 2 research questions this review sought to answer were as follows:

• What is the evidence of the effectiveness of digital phenotyping using smartphones in identifying behavioral patterns related to stress, anxiety, and mild depression?
• In particular, which smartphone sensors are found to be effective, and what are the associated challenges?

For these research questions, we considered statistically significant associations between sensor patterns and behavioral patterns as evidence of effectiveness.

Type of Studies

This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [ 25 ] ( Multimedia Appendix 1 ). Figure 1 shows the reviewing process and search results. In the first round of screening studies, 1 author excluded studies that were not relevant to the research questions. Another author reran the queries for confirmation. Studies were included in this review if they were conducted to measure and detect stress, anxiety, or mild depression, even if they included other variables, such as job performance, promotion, or discrimination. We included studies in which data were collected through smartphones with an iOS (Apple Inc) or Android (Google LLC) operating system. Data collected through wearable devices were excluded. We included studies in which the participants were adults aged ≥18 years and were from a nonclinical population. Studies conducted with nonadult participants (eg, teenagers and children) were excluded. Given our research questions, if the studies’ participants had or had had any severe mental health disorder, such as schizophrenia, bipolar disorder, or psychosis, they were not included. We also excluded personality and character measurement and phobia studies. The primary research language was English. The studies included were conducted from September 2010 to September 2023. Peer-reviewed conference articles and journal articles were included. The data we wished to extract were the study aim, data collected, operating system in the smartphone used for data collection, behavioral patterns identified, surveys used for verification, and sample size. A total of 3 authors reviewed the studies independently to extract data and confirm the extracted data. After the first round of data extraction, 1 author re-examined the studies to extract the predictive modeling used. These data are presented in the Results section. We noticed that participants in the included studies fell into 1 of 3 major groups (ie, students, adults, and employees). We refer to the participants of the studies that recruited adults enrolled in universities as “students,” participants of the studies that recruited adults unaffiliated to any particular organization as “adults,” and participants of the studies that recruited adults employed at a particular organization as “employees.”

Search Strategy

A total of 3 databases were queried: Web of Science, ACM, and PubMed. PubMed is a medicine-based database, ACM is a technology-based database, and Web of Science is a cross-domain database. The search query was the same for the 3 platforms: “digital phenotyping” OR “passive sensing” AND (stress OR anxiety OR ((mild OR moderate) AND depression)).

The study length varied from 10 days [ 26 ] to 3 years [ 27 ]. One study [ 28 ] conducted in-depth interviews with students lasting an average of 4.5 hours per person, and another study was a controlled laboratory study [ 29 ]. These 2 studies are not presented in Table 1 . In the studies conducted with students, a semester or spring or winter term was a common duration. The studies with general nonclinical adult populations were typically longer than those with students.

Number of Participants

The number of participants ranged from a minimum of 7 adults [ 26 ] to a maximum of 18,000 adults [ 27 ]. Apart from the 3-year longitudinal study with 18,000 participants [ 27 ], the average number of participants was 129.4 (SD 184.01). We observed a pattern of attrition, where the number of participants who completed the study was lower than the number of the participants recruited. The number of participants reported in this review is the final sample size. For example, one of the studies [ 52 ] recruited 112 participants, of whom 84 (75%) completed the study. In the study by Pratap et al [ 55 ], there was a drastic drop in participants, with only 359 (30.42%) of the 1180 enrolled participants completing the study. Another significant drop was seen in the study by Nepal et al [ 40 ], where 750 participants were interested in the research, whereas only 141 (18.8%) of them completed the study. Some studies were less affected; for example, 86 participants started the study by Rhim et al [ 49 ], and 78 (91%) completed it.

Publication Years of the Studies

Although the query started with the year 2010, the earliest publication was from 2014 [ 26 ], extending to articles published as of April 2023 [ 35 ]. Over the years, the interest in detecting and predicting stress, anxiety, and mild depression in the nonclinical population has increased ( Table 2 ).

Studies With the iOS and Android Operating Systems

The Android operating system was more common than iOS. Among the 40 included studies, only 2 (5%) were compatible with only iOS [ 29 , 51 ]. A total of 27 (68%) studies were available for both iOS and Android [ 26 , 28 , 30 , 34 , 35 , 37 - 42 , 45 - 47 , 50 , 53 - 61 ]. A total of 11 (28%) studies were for only Android users [ 27 , 31 - 33 , 36 , 42 - 44 , 48 , 49 , 52 ]. The reasons identified for the use of the Android operating system were that it has more freedom to capture more modalities, such as keyboard typing and use of apps, and that Android devices enable apps to run more easily in the background [ 49 ].

Studies With Students

Table 3 presents the data extracted from the studies that were conducted with student populations. The average length of the studies with students was 158.6 (SD 176.4) days. The average number of participants was 137.3 (SD 152.1). There were significantly more studies with students than studies with employees or general adults. The sample sizes of the studies with students were similar to those of the studies with adults but smaller than those of the studies with employees. In the studies with students, various passive sensors were used, and some were found to be effective for detection, prediction, or both.

Of the 28 studies with students, 23 (82%) used machine learning models for prediction. A total of 12 studies (43%) [ 30 , 31 , 33 , 37 , 38 , 44 , 46 , 47 , 54 ] used decision tree–based methods, and 9 studies (32%) [ 37 , 39 , 42 , 49 - 51 , 57 , 58 ] used regression-based methods. A total of 3 (11%) studies conducted in recent years [ 43 , 60 , 61 ] used deep neural networks because of their enhanced ability to discern underlying patterns in large unstructured data sets. Tree-based models have the best performance when trained with structured data, and the reported studies mostly used tree-based models and structured data. Among the 28 studies, 2 studies [ 57 , 60 ] conducted in 2023 addressed the generalizability of their proposed detection method and verified its applicability across students from various years, classes, and institutions. Two (7%) studies [ 42 , 43 ] in Table 3 used the StudentLife data set [ 62 ]. Each study contributed substantial original analyses including different behavioral patterns and was considered a “study” in this systematic review. Entries with “N/A” in the predictive modeling column indicate that the study did not involve any attempts to predict future occurrences. However, these studies may still contain statistical analyses as part of their research approach. Overall, students who experienced depression, anxiety, and stress visited fewer locations [ 39 , 44 , 50 , 58 - 60 ] and were more sedentary [ 47 , 50 , 58 - 60 ]. Depression was also associated with shorter or irregular sleep [ 35 , 46 , 47 , 50 , 52 , 59 , 60 ] and accrued phone use [ 46 , 47 , 50 , 51 , 58 - 60 ].

a PHQ: Patient Health Questionnaire.

b N/A: not applicable.

c DASS: Depression Anxiety Stress Scales.

d SIAS: Social Interaction Anxiety Scale.

e GAD-7: Generalized Anxiety Disorder Scale-7.

f PQ: Prodromal Questionnaire.

g PSS: Perceived Stress Scale.

h PSQI: Pittsburgh Sleep Quality Index.

i BASIS: Behavior and Symptom Identification Scale.

j SF: Short Form Health Survey.

k SFS: Social Functioning Schedule Scale.

l CGI: Clinical Global Impressions Scale.

m HDRS: Hamilton Depression Rating Scale.

n CAS: Coronavirus Anxiety Scale.

o HAI: Health Anxiety Inventory.

p UCLA: University of California, Los Angeles.

q XGBoost: extreme gradient boosting.

r LOOCV: leave-one-out cross validation.

s PANAS: Positive and Negative Affect Schedule.

t MPSM: Mobile Photographic Stress Meter.

u LSTM: long short-term memory.

v CNN: convolutional neural network.

w WEMWBS: Warwick-Edinburgh Mental Well-Being Scale.

x BDI: Beck Depression Inventory.

y EMA: ecological momentary assessment.

z PAM: Patient Activation Measure.

aa BFI: Big Five Inventory.

ab LASSO: least absolute shrinkage and selection operator.

ac COMOSWB: Concise Measure of Subjective Well-Being.

ad SAS: Sport Anxiety Scale.

ae PPC: Perceived Personal Control.

af SSS: Social Support Scale.

ag MAAS: Mindful Attention Awareness Scale.

ah ERQ: Emotion Regulation Questionnaire.

ai BRS: Brief Resilience Scale.

aj CES-D: Center for Epidemiological Studies-Depression.

ak STAI: State Trait Anxiety Inventory.

al LOSOCV: leave-one-subject-out cross validation.

am AdaBoost: adaptive boosting.

an LightGBM: light gradient boosting machine.

ao MERF: mixed-effects random forest.

ap TIPI: Ten-Item Personality inventory.

aq ESS: Epworth Sleepiness Scale.

ar MCTQ: Munich Chronotype Questionnaire.

as PROMIS: Patient-Reported Outcomes Measurement Information System.

at BHM: Behavioral Health Measure.

au CD-RISC: Connor-Davidson Resilience Scale.

av 1D-CNN: 1-dimensional convolutional neural network.

aw MCDCNN: multi-channel deep convolutional neural network.

ax ResNet: residual network.

ay TWIESN: time warping invariant echo state network.

az FCNN: fully convolutional neural network.

ba AUROC: area under the receiver operating characteristic curve.

bb DWAI: Digital Working Alliance Inventory.

Studies With Adults

Table 4 presents the data extracted from the studies conducted with the general adult population. The average study duration was 201.6 (SD 367) days. Apart from a 3-year longitudinal study with 18,000 participants, the average number of participants was 123.4 (SD 139.8). Of the 8 studies with adults, 2 (25%) [ 32 , 52 ] were conducted with the same set of participants. A total of 3 (38%) studies used predictive modeling, with regression-based models being the most common [ 34 , 36 , 52 ], and 1 (12%) study identified gender differences in behavioral patterns [ 27 ]. Overall, the research with adults showed that GPS, accelerometer, ambient audio, and illuminance data related to individuals’ emotional state. Adults with depression were less likely to leave home and were less physically active, whereas adults who were socially anxious were more active and left their home more often but avoided going to places where they needed to socially interact.

a N/A: not applicable.

b LSAS: Liebowitz Social Anxiety Scale.

c GAD-7: Generalized Anxiety Disorder Assessment-7.

d PHQ: Patient Health Questionnaire.

e SDS: Sheehan Disability Scale.

f LASSO: least absolute shrinkage and selection operator.

g BIS: Barratt Impulsiveness Scale.

h UPPS: Impulsive Behavior Scale.

i PAM: Patient Activation Measure.

j XGBoost: extreme gradient boosting.

k STAI: State Trait Anxiety Inventory.

l PANAS: Positive and Negative Affect Schedule.

m PSS: Perceived Stress Scale.

n MAAS: Mindful Attention Awareness Scale.

o SPIN: Social Phobia Inventory.

p RBM: Restricted Boltzmann Machine.

Studies With Employees

Table 5 presents the data extracted from the studies that were conducted with employees. Among the 4 studies with employees, 1 (25%) study recruited its own participants [ 56 ], and the other 3 (75%) studies [ 40 - 42 ] used the Tesserae data set [ 63 ]. Compared with students and adults, the employee population was the least studied, with the fewest articles. However, the studies with employees had the largest number of participants, with a mean of 427.3 (SD 280.3). All 4 studies used regression-based predictive modeling, and 2 (50%) of them [ 40 , 56 ] evaluated a variety of models, with logistic regression, support vector machine, and random forest being the most common methods. Detecting and predicting employees’ stress in workplaces were examined in tandem with employees’ work performance. The research goal for these studies was to understand the underlining reasons for lowered work-related productivity. In contrast to the other 2 populations (ie, students and adults), less mobility was seen as positive for employees because less mobility in workplaces was associated with more positivity and higher performance.

a XGBoost: extreme gradient boosting.

b ITP: Psychological Type Indicator.

c IRB: in-role behavior.

d OCB: organizational citizenship behavior.

e CWB: counterproductive work behavior.

f ROCKET: random convolutional kernel transform.

g EMA: ecological momentary assessment.

h PAM: Patient Activation Measure.

i PANAS: Positive and Negative Affect Schedule.

Passive Sensors

Table 6 provides an overview of the range of sensors used to detect patterns related to mild mental health symptoms and summarizes the evidence of the effectiveness of the various sensors. The first column lists the sensor, and the second column presents how the data from that sensor are interpreted; in other words, it presents the behavior-related information that the sensor data are intended to represent. The third column indicates which articles found significant associations between the specific sensor and stress, anxiety, or mild depression. The fourth column indicates which articles found no significant associations between the specific sensor and mental health outcomes (ie, explicitly stated so in the articles). In the subsequent sections, we discuss the types of activities detected by the sensors.

Social Interaction: Call and Text Logs, Audio, Microphone, and Bluetooth

The social interaction of an individual is reflective of their current mood and mental state [ 44 , 64 , 65 , 66 ]. Individuals with depression and stress may be expected to decrease their social interactions. This is measured through the frequency of receiving texts and calls, how fast individuals respond, and the frequency of being around others. Among the 40 included studies, 18 (45%) [ 27 , 28 , 30 , 31 , 33 - 35 , 37 , 44 - 48 , 51 , 53 , 55 , 60 ] examined call logs to understand social interaction patterns, mainly through the number of incoming and outgoing calls, the number of missed calls, and the duration of calls. Individuals who experience depression and stress may engage in longer outgoing calls [ 51 ]. Evening communications were predictive of depression [ 47 ], anxiety, and loneliness [ 54 ]. Students who experienced discriminations [ 53 ] and anxious participants had more evening communications [ 54 ]. Metadata on SMS text messages were examined in 10 (25%) [ 27 , 28 , 30 , 31 , 33 , 37 , 44 , 45 , 48 , 55 ] of the 40 studies, including the frequency of receiving SMS text messages and the average time of responses. People who are socially anxious were found to take different amounts of time to respond to SMS text messages and calls [ 33 ]. Increases in the number of calls were associated with increased social anxiety [ 48 ]. Those who experienced social anxiety were less likely to call or text in public [ 44 ]. For students, fewer conversations were associated with more stress [ 39 ] and more mood instability [ 42 ]. One of the studies found that more emotionally unstable individuals tended to text more than emotionally stable individuals [ 27 ].

Location: GPS, Bluetooth, and Wi-Fi

Location data can provide insights into individuals’ mental health state in terms of the normal or abnormal variety and frequency of locations visited [ 67 ]. As presented in Table 6 , GPS has been one of the most commonly used passive sensors for stress, anxiety, and mild depression research. The findings regarding location consistently demonstrate that students and adults who experienced depression, anxiety, or stress tended to visit fewer places [ 39 , 44 , 50 , 58 - 60 ]. One of the studies [ 48 ] found that location data are highly inversely correlated with mild depression severity. The main way in which this is measured is through the frequency of exiting the house, the variety of locations visited, and mobility. The frequency of exiting the house is less for individuals who are depressed, and there is less variety in the visited locations for individuals who are socially anxious. Individuals who are feeling depressed often experience being less energetic [ 68 , 69 ]. Overall, negative emotions were associated with time spent at specific locations, but this is also affected by personal routines and preferences [ 30 ]. For students, stress and lower subjective well-being were associated with more time spent on campus [ 39 , 49 ] and less time spent at campus food locations [ 39 ]. Students who experienced depression spent more time at home [ 60 ], whereas individuals at higher risk of psychosis spent less time at home [ 51 ]. Time spent at exercise locations was positively correlated with changes in depressive symptoms [ 48 ]. Another study [ 38 ] distinguished between students experiencing severe stress and those with normal stress levels, revealing that students with severe stress spent significantly less time on campus and were less involved in work-related activities compared with their counterparts with normal stress levels. As for employees, higher performers were found to visit fewer locations on weekday evenings but more locations during weekends [ 56 ].

Voice Recognition: Audio

The microphone is used to measure audio data of speech and ambient noises. One of the studies [ 26 ] examined how people with stress speak by analyzing their voice, including the speed of speech, how energetic their vocality is, and the pitch. One caveat is that the study by Adams et al [ 26 ] used audio captured within laboratory environments and found that stress could be recognized from the absence of speech. In variable environments, it will be harder to recognize the changing voice patterns. One study found that generalized anxiety and depression related to reward-related words in ambient speech, and social anxiety related to vision-related words [ 32 ]. Another study [ 52 ] identified that people with depression tend to speak less and use more death-related words.

Sleep: Accelerometer, Audio, and Illuminance

Sleep is highly correlated with individuals’ mental state [ 26 , 35 , 36 , 42 , 45 - 47 , 59 , 60 ]. Among the 40 included studies, 5 (13%) [ 35 , 46 , 52 , 60 ] found that more disturbed sleep correlated with more depressive symptoms. However, occasional sleep disturbance is not necessarily predictive. For example, for those with social anxiety, sleep disturbance might be positive because it suggests night-time activity and social interactions. Metadata on the time spent in darkness can be indicative of sleep patterns. The study by Fukazawa et al [ 36 ] stated that anxiety levels increase when the time spent in darkness increases. The study by Di Matteo et al [ 52 ] found that individuals with symptoms related to social anxiety and depression spent less time in darker environments. Another study [ 39 ] stated that stress changed students’ sleep patterns, where they became less likely to move around between 6 PM and midnight. Of the 40 studies, 6 (15%) found that shorter sleep duration was correlated with more mood instability [ 42 ], more depressive symptoms [ 59 , 60 ], and more stress [ 36 , 44 ]. One of the studies [ 45 ] also found that the student population, in general, tended to sleep less during examination periods and slept more during breaks, and they felt more stressed during both breaks and examination periods.

Phone Use: On and Off Screen, Lock and Unlock, and App Use

Today, smartphones are used for self-regulated “distractions,” such as the use of social media [ 38 ]. This type of self-regulated distraction can temporarily reduce stress. The study by Chikersal et al [ 47 ] showed that depression can impact concentration levels, so if distraction by phone can be measured, this could be a potential predictive marker. Several studies found that increasing phone use was correlated with more depressive symptoms [ 46 , 47 , 50 , 52 , 58 - 60 ], anxiety [ 52 , 59 ], impulsivity [ 34 ] and lower subjective-wellbeing [ 49 ]. The study by Morshed et al [ 42 ] outlined that for postsemester depression, phone use at night is not predictive, whereas another study [ 47 ] summarized that phone use during the day is predictive of depression. More frequent phone locks or unlocks correlated with higher levels of depressive symptoms [ 60 ] and impulsivity [ 34 ]. Higher performing employees tended to unlock their phones less frequently in the evenings [ 56 ]. Additionally, individuals who were promoted spent more time on their phones during early mornings and late evenings, with more unlocks occurring during nighttime compared with their nonpromoted counterparts [ 40 ].

Physical Activity and Mobility: Accelerometer

According to Table 6 , along with GPS, accelerometer is one of the most widely used passive sensors in digital phenotyping research to monitor participant’s mobility, activity, and sedentary periods. Increased sedentary time was correlated with increased depressive symptoms [ 47 , 48 , 50 , 58 - 60 ], increased mood instability [ 27 , 42 ], increased stress [ 36 ] and decreased subjective well-being [ 49 ]. Exercise duration was positively correlated with changes in anxiety [ 36 ] and depressive symptoms [ 48 ]. The study by Mirjafari et al [ 56 ] found that the amount of movement and physical activity was related to employee’s stress level and highlighted that if the activity is regular, it should reduce stress. Different occupations require different levels of physical activity, social interactions, and mobility. For instance, developers spend most of their time at their desks, and their tasks might require less social interaction and mobility at work, but this does not mean they are more stressed. Project managers have more mobility during the day, and this may be because they need to move around to meet with the stakeholders [ 56 ]. Several studies have observed variations in mobility and gait consistency. The study by Boukhechba et al [ 44 ] reported that individuals with high social anxiety exhibited a narrower range of activities, whereas the study by Xu et al [ 60 ] revealed that students experiencing depression demonstrated more consistent mobility patterns. Additionally, accelerometer data indicated that individuals with low social anxiety maintained a steady walking pace, whereas those with high social anxiety tended to walk more rapidly and with greater irregularity [ 33 ].

Muscle Activity: Keyboard

Stress can cause muscle tension [ 70 , 71 ]. One of the studies [ 29 ] collected the data of users with stress via a keyboard in a laboratory environment and found that typing pressure significantly increased under stressful conditions.

Digital phenotyping for mild mental health symptoms in nonclinical participants can present ethical challenges, limitations to the research, and technical challenges. We review the challenges that were stated in the literature.

Ethical Challenges

Among the 40 included studies, 7 (18%) specifically mentioned privacy-related ethical concerns [ 28 , 31 , 35 , 36 , 40 , 41 , 43 ]. A major concern for participants across several studies was whether authorities, such as employers or teachers, will have access to their data. One of the studies [ 28 ] conducted in-depth interviews with 15 students to understand their perspectives on digital phenotyping through app prototypes. They found that the students’ core concerns were whether the acquainted university staff had access to the data. They also found that students’ acceptability of such apps depends on the perceived relevancy of the data collected and the effects on students’ devices. The study by Nepal et al [ 40 ] with employees reported a similar privacy concern of whether the employees’ data would be leaked to their boss; if the boss is aware of a potential mental health issue, it may impact their work performance ratings.

The methods of collecting and storing passive sensing data also present privacy concerns [ 28 , 70 , 72 ], particularly when the tracked data involve sensitive topics, such as mental health [ 72 ]. Sensors that infer individuals’ social interactions provide insights into their mental health status [ 26 , 36 , 53 ]. However, these types of data were less likely to be shared by participants because of privacy concerns. In the study by Rooksby et al [ 28 ], students identified camera, microphone, call log, and keyboard data as highly unacceptable types of data to capture.

Location data were associated with privacy and security concerns. In the study by Wen et al [ 34 ], participants felt uncomfortable with location tracking because it might breach their privacy and were hesitant to log their location when they moved from one place to another. Some studies excluded specific sensors to protect the participant’s privacy. Location data were not recorded owing to security concerns, even though they could provide valuable insights into the mental state [ 36 , 38 ]. In the study by Adams et al [ 26 ], the microphone was disabled to capture calls and conversations while individuals were talking to their family members. Another ethical concern was regarding the misuse of data. The main focus in studies of digital phenotyping using smartphones was on tracking participants’ usual behavioral patterns and identifying whether they behaved unusually. There were concerns regarding secondary uses. For example, participants’ leaked data can be used for advertising purposes or to create content [ 34 , 41 ].

Limitations to the Research

Coping mechanisms related to stress and anxiety vary among individuals [ 22 ]. Individual differences can make it challenging to label individuals as stressed, anxious, or depressed, particularly nonclinical participants. Certain behavioral patterns can be generally expected; however, not all individuals will follow the same pattern. To make generalizable and powerful analyses and understand behavioral patterns associated with mild mental health concerns, it is recommended to study diverse groups for longer than a 2-week period. Of the 40 included studies, 2 (5%) [ 33 , 39 ] focused on a particular demographic subset, namely, undergraduate students. Therefore, the generalizability of the studies is limited. In the studies by Rooksby et al [ 28 ], Exposito et al [ 29 ], and Wang et al [ 50 ], limited variation in representation was seen as a major limitation. The studies by Rhim et al [ 49 ], DaSilva et al [ 39 ], and Fukazawa et al [ 36 ] stressed the importance of selecting a wider age group, as younger people use their smartphones proactively, whereas older people’s behavioral patterns might show differences when they are experiencing mild mental health symptoms. The study by Nepal et al [ 40 ] suggested that diverse population testing is required for more reliable results, considering interindividual differences. Furthermore, the accuracy and effectiveness of machine learning models are highly affected by data set quality. We noticed that over the last 4 years [ 38 , 46 , 57 , 60 ], there has been increased focus on the generalizability of machine learning models, with the goal of assessing generalizability across students from various years, classes, and institutions.

Technical Challenges

Digital phenotyping studies on mild symptoms related to mental health with nonclinical participants presented technical challenges. A main concern was the accuracy of the sensor data collected from smartphones. The study by Fukazawa et al [ 36 ] sought to understand the time spent in darkness and its effects on the relationship between stress and anxiety patterns and sleep. However, when individuals carried their smartphone in their pockets or bags, the smartphone could not detect the darkness of the environment. This presented a challenge because illuminance data were captured even when the phone was not used actively. Similar concerns were raised in the study by Di Matteo et al [ 52 ]. The time spent in darkness feature did not distinguish whether the device was in a dark room or a dark location (ie, in the pocket). The study by Melcher et al [ 35 ] stated that the captured accelerometer data may not accurately represent daily activity, as not all participants constantly carried their phones throughout the day. In the study by Di Matteo et al [ 32 ], environmental audio did not produce clear transcripts in louder environments. This study mentioned that transcripts were produced based on dictionaries, so language analysis of complex speech, such as metaphors and sarcasm, was ignored. Therefore, the entire content of the conversation might not be correctly interpreted. In the study by Di Matteo et al [ 52 ], similar challenges were identified, as the speech data produced from smartphones were not clear. The recorded voices of the participants were masked by those of the people around them or even sound from other sources such as television or radio. Moreover, it was not possible to identify whether the death-related words came from the participants or from the people they interacted with.

Another technical challenge identified was battery life [ 47 ]. As expected, moment-by-moment data collection requires high power use, which might shorten the battery life. Participants had to charge their phones more often, which was inconvenient, and altered their usual behavior because they could not carry their phones as usual when the phones were charging. The study by Chikersal et al [ 47 ] mentioned another technical limitation: the transfer rate was affected if the app stopped working randomly. During these times, data were not transferred or collected. With the increase in the use of 5G technology, Wi-Fi data for indoor locations may cease to be relevant. In the study by Zakaria et al [ 38 ], some users were on their 5G indoors rather than their Wi-Fi, and this may point to a future trend of the use of 5G. We now turn to the discussion.

Principal Findings

This literature review examined digital phenotyping studies that detected and predicted stress, anxiety, and depression in their mild states in nonclinical populations using data collected from smartphones. The primary objective of digital phenotyping in the context of mild mental health was similar among the 3 participant cohorts: students, adults, and employees. However, notable distinctions emerged among these groups. Among university students, the geographical proximity and relevance of the university campus were discerned as influential factors. Moreover, academic pursuits, particularly coursework and study-related activities, assumed significance within this demographic. Conversely, among employees, work aspects held salience, accompanied by the workplace environment. The remaining studies encompassed a general population cohort, delineated by undisclosed characteristics. Overall, we found that identifying behavioral abnormalities related to stress and anxiety was possible but raised certain challenges. Generalized stress and anxiety symptoms vary largely among individuals, whereas serious diagnoses, such as bipolar disorder or schizophrenia, have well-documented behavioral changes. Sleep was a strong predictor variable, yet some individuals tended to sleep more while they were stressed, whereas others lacked sleep under stress. This may be one of the reasons why there are fewer studies and reviews completed on stress and anxiety compared with studies on serious conditions such as bipolar disorder, severe depression, and schizophrenia. Another reason is that clinical psychologists and psychiatrists who are familiar with clinical populations are leading the digital phenotyping research.

Studies tended to use self-report to categorize nonclinical populations as stressed, depressed, or anxious. It was not always clear whether the identified patterns of the passive sensor data would effectively discriminate among groups. Most studies used prestudy and poststudy surveys to identify participants’ mental state. There were concerns raised regarding the accuracy of the categorization of self-report surveys. For instance, the study by Sefidgar et al [ 53 ] stated that students with stress may not report themselves as very stressed. Melcher et al [ 70 ] conducted a review and found that students were concerned regarding their professors learning about their data [ 71 ]. Thus, the accuracy of self-report remains an issue for passive sensing studies that use self-report labels, especially when there are privacy concerns. This may be related to the high dropout rates in the studies.

Many types of data sensors were used in the reviewed studies. Few articles related sensor patterns to specific symptoms validated by relevant psychological evidence. One of the studies [ 46 ] extracted interpretable rules (such as intermittent sleep episodes or number of bouts of being asleep or number of outgoing calls during weekends) through association rule mining to distinguish the behavioral patterns between students who were depressed and students who were not. However, although the behavioral patterns were identified, they were not validated to be exclusive to the addressed mental health issue; for example, high mobility and physical activity do not necessarily mean that the person is not stressed. In the study by Tseng et al [ 45 ], students were more mobile during the examination week, despite being under high pressure and stress. In the same study, some students were less mobile when studying for their examinations, which we cannot necessarily be interpreted as being under stress. Of the 40 included studies, 4 (10%) [ 35 , 58 , 59 , 61 ] explored the effects of the COVID-19 pandemic on behavioral and mental health. Additional recent investigations, which independently gathered their own data sets during the COVID-19 pandemic, have shown that quarantine measures have influenced individual behavioral patterns. For the purpose of making precise predictions in digital phenotyping, it is imperative to consider contextual and environmental factors.

Privacy and secondary data uses were the main concerns identified for digital phenotyping. Individuals using digital phenotyping systems have the right to provide informed consent. This means that they should be made aware of how all their data will be used, who will have access to their data, where their data will be stored, and for how long their data will be stored, and they have the right to decline to participate. We urge researchers and medical practitioners to carefully consider the system design and requirements because data transferred to the cloud and other services may fall under various service agreements. To empower end users and improve the quality of digital phenotyping systems, we recommend that transparent algorithms and explainable artificial intelligence be combined with user-accessible and understandable displays so that adults can engage in the process of identifying and categorizing patterns related to mild mental health symptoms.

The digital phenotyping research focused on in this review may enable the design of tailored intervention programs for nonclinical participants who are showing symptoms of stress, anxiety, and mild depression. Most of the studies included in this review were conducted within a restricted timeline and limited scope of detection and prediction. Only 4 (10%) of the 40 studies mentioned potential intervention programs upon predicting stress, anxiety, and mild depression [ 31 , 38 , 47 , 53 ].

Our review has some limitations. We excluded studies conducted with teenagers, children, and adults who were clinically diagnosed. Thus, we missed studies that focused on the detection and prediction of stress, anxiety, and mild depression in these populations. These populations are likely to show different patterns than those in adults who are not clinically diagnosed. Further, we excluded studies conducted using technologies other than smartphones. We chose this more limited subset of technologies to scope findings related to widely available technologies. The availability of technologies is changing rapidly, and wearables such as smartwatches are becoming more common. As wearable technologies become ubiquitous, we recommend including them in future systematic reviews.

This literature review is unique in that it examines studies focused on the behavioral patterns of nonclinical populations, namely students, employees, and adults who are stressed, anxious, or mildly depressed. We examined each type of sensor and indicated when it was significantly associated with mild mental health symptoms. We identified commonalities in the studies in terms of ethical challenges, limitations to the research, and technical challenges.

Conclusions

This systematic literature review found that digital phenotyping can be an effective way of identifying certain behavioral patterns related to stress, anxiety, and mild depression. A range of passive sensors was used in the studies, such as GPS, Bluetooth, ambient audio, light sensors, accelerometers, microphones, illuminance, and Wi-Fi. We found that location, physical activity, and social interaction data were highly related to participants’ mental health and well-being. The surveyed literature discussed the ethical and technical challenges that limit the accuracy and generalizability of results. One of the greatest challenges was privacy concerns, and these were primarily related to camera, location, SMS text message, and call log data. Another challenge was the significant variation among individuals and their unique behaviors related to mental health. Finally, technical limitations have not been fully resolved, with issues such as the sensor for illuminance still capturing data while not in use reducing the accuracy of the collected data. It is hoped that this overview of digital phenotyping and mental health studies conducted in the last decade, including the common privacy and technical concerns, can help move this area of research forward, ultimately improving the quality of passive sensing, and provide benefits in terms of the early detection of relevant mild mental health phenomena.

Acknowledgments

The authors would like to thank the School of Computer Science at the University of Auckland for their financial support.

Conflicts of Interest

None declared.

The PRISMA 2020 checklist.

• Onnela JP, Rauch SL. Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology. Jun 2016;41(7):1691-1696. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Houts CR, Patrick-Lake B, Clay I, Wirth RJ. The path forward for digital measures: suppressing the desire to compare apples and pineapples. Digit Biomark. Nov 26, 2020;4(Suppl 1):3-12. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Venkatesh S, Christensen H. Toward clinical digital phenotyping: a timely opportunity to consider purpose, quality, and safety. NPJ Digit Med. Sep 06, 2019;2:88. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Berry JD, Paganoni S, Carlson K, Burke K, Weber H, Staples P, et al. Design and results of a smartphone-based digital phenotyping study to quantify ALS progression. Ann Clin Transl Neurol. Apr 3, 2019;6(5):873-881. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Trifan A, Oliveira M, Oliveira JL. Passive sensing of health outcomes through smartphones: systematic review of current solutions and possible limitations. JMIR Mhealth Uhealth. Aug 23, 2019;7(8):e12649. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Torous J, Gershon A, Hays R, Onnela JP, Baker JT. Digital phenotyping for the busy psychiatrist: clinical implications and relevance. Psychiatr Ann. 2019;49(5):196-201. [ FREE Full text ] [ CrossRef ]
• Maharjan SM, Poudyal A, van Heerden A, Byanjankar P, Thapa A, Islam C, et al. Passive sensing on mobile devices to improve mental health services with adolescent and young mothers in low-resource settings: the role of families in feasibility and acceptability. BMC Med Inform Decis Mak. Apr 07, 2021;21(1):117. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Adler DA, Ben-Zeev D, Tseng VW, Kane JM, Brian RE, Campbell AT, et al. Predicting early warning signs of psychotic relapse from passive sensing data: an approach using encoder-decoder neural networks. JMIR Mhealth Uhealth. Aug 31, 2020;8(8):e19962. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Potier R. The digital phenotyping project: a psychoanalytical and network theory perspective. Front Psychol. Jul 15, 2020;11:1218. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kokel D, Rennekamp AJ, Shah AH, Liebel U, Peterson RT. Behavioral barcoding in the cloud: embracing data-intensive digital phenotyping in neuropharmacology. Trends Biotechnol. Aug 2012;30(8):421-425. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Teo JX, Davila S, Yang C, Hii AA, Pua CJ, Yap J, et al. Digital phenotyping by consumer wearables identifies sleep-associated markers of cardiovascular disease risk and biological aging. Commun Biol. Oct 4, 2019;2:361. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fagherazzi G. Deep digital phenotyping and digital twins for precision health: time to dig deeper. J Med Internet Res. Mar 03, 2020;22(3):e16770. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mercier HW, Hamner JW, Torous J, Onnela JP, Taylor JA. Digital phenotyping to quantify psychosocial well-being trajectories after spinal cord injury. Am J Phys Med Rehabil. Dec 2020;99(12):1138-1144. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rodriguez-Villa E, Rauseo-Ricupero N, Camacho E, Wisniewski H, Keshavan M, Torous J. The digital clinic: implementing technology and augmenting care for mental health. Gen Hosp Psychiatry. 2020;66:59-66. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barnett I, Torous J, Staples P, Sandoval L, Keshavan M, Onnela JP. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology. Jul 2018;43(8):1660-1666. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Orsolini L, Fiorani M, Volpe U. Digital phenotyping in bipolar disorder: which integration with clinical endophenotypes and biomarkers? Int J Mol Sci. Oct 16, 2020;21(20):7684. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kleiman EM, Turner BJ, Fedor S, Beale EE, Picard RW, Huffman JC, et al. Digital phenotyping of suicidal thoughts. Depress Anxiety. Jul 2018;35(7):601-608. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Onnela JP. Opportunities and challenges in the collection and analysis of digital phenotyping data. Neuropsychopharmacology. Jan 2021;46(1):45-54. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lardier DT, Lee CY, Rodas JM, Garcia-Reid P, Reid RJ. The effect of perceived college-related stress on depression, life satisfaction, and school satisfaction: the coping strategies of hispanic college students from a hispanic serving institution. Educ Urban Soc. Jan 02, 2020;52(8):1204-1222. [ CrossRef ]
• Rezaei A, Mousanezhad Jeddi E. Relationship between wisdom, perceived control of internal states, perceived stress, social intelligence, information processing styles and life satisfaction among college students. Curr Psychol. Feb 17, 2018;39:927-933. [ FREE Full text ] [ CrossRef ]
• Futo J. Dealing with mental health issues on campus starts with early recognition and intervention. Campus Law Enforc J. Jun 2011;41(3):22. [ FREE Full text ]
• Jonason PK, Talbot D, Cunningham ML, Chonody J. Higher-order coping strategies: who uses them and what outcomes are linked to them. Pers Individ Differ. Mar 1, 2020;155:109755. [ FREE Full text ] [ CrossRef ]
• Jorm AF. Mental health literacy: empowering the community to take action for better mental health. Am Psychol. Apr 2012;67(3):231-243. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Number of smartphone mobile network subscriptions worldwide from 2016 to 2022, with forecasts from 2023 to 2028. Statista. URL: https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/ [accessed 2023-11-04]
• PRISMA flow diagram. PRISMA. URL: http://www.prisma-statement.org/PRISMAStatement/FlowDiagram [accessed 2023-11-04]
• Adams P, Rabbi M, Rahman T, Matthews M, Voida A, Gay G, et al. Towards personal stress informatics: comparing minimally invasive techniques for measuring daily stress in the wild. In: Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare. 2014. Presented at: 8th International Conference on Pervasive Computing Technologies for Healthcare; May 20-23, 2014; Oldenburg, Germany. [ CrossRef ]
• Servia-Rodríguez S, Rachuri KK, Mascolo C, Rentfrow PJ, Lathia N, Sandstrom GM. Mobile sensing at the service of mental well-being: a large-scale longitudinal study. In: Proceedings of the 26th International Conference on World Wide Web. 2017. Presented at: WWW '17: 26th International World Wide Web Conference; April 3-7, 2017; Perth, Australia. [ CrossRef ]
• Rooksby J, Morrison A, Murray-Rust D. Student perspectives on digital phenotyping: the acceptability of using smartphone data to assess mental health. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019. Presented at: CHI '19: CHI Conference on Human Factors in Computing Systems; May 4-9, 2019; Glasgow, UK. [ CrossRef ]
• Exposito M, Hernandez J, Picard RW. Affective keys: towards unobtrusive stress sensing of smartphone users. In: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. 2018. Presented at: MobileHCI '18: 20th International Conference on Human-Computer Interaction with Mobile Devices and Services; September 3-6, 2018; Barcelona, Spain. [ CrossRef ]
• Cai L, Boukhechba M, Wu C, Chow PI, Teachman BA, Barnes LE, et al. State affect recognition using smartphone sensing data. In: Proceedings of the 2018 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies. 2018. Presented at: CHASE '18: ACM/IEEE International Conference on Connected Health: Applications, Systems and Engineering Technologies; September 26-28, 2018; Washington, DC. [ CrossRef ]
• Boukhechba M, Cai L, Chow PI, Fua K, Gerber MS, Teachman BA, et al. Contextual analysis to understand compliance with smartphone-based ecological momentary assessment. In: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare. 2018. Presented at: PervasiveHealth '18: 12th EAI International Conference on Pervasive Computing Technologies for Healthcare; May 21-24, 2018; New York, NY. [ CrossRef ]
• Di Matteo D, Wang W, Fotinos K, Lokuge S, Yu J, Sternat T, et al. Smartphone-detected ambient speech and self-reported measures of anxiety and depression: exploratory observational study. JMIR Form Res. Jan 29, 2021;5(1):e22723. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jacobson NC, Summers B, Wilhelm S. Digital biomarkers of social anxiety severity: digital phenotyping using passive smartphone sensors. J Med Internet Res. May 29, 2020;22(5):e16875. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wen H, Sobolev M, Vitale R, Kizer J, Pollak JP, Muench F, et al. mPulse mobile sensing model for passive detection of impulsive behavior: exploratory prediction study. JMIR Ment Health. Jan 27, 2021;8(1):e25019. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Melcher J, Lavoie J, Hays R, D'Mello R, Rauseo-Ricupero N, Camacho E, et al. Digital phenotyping of student mental health during COVID-19: an observational study of 100 college students. J Am Coll Health. Apr 2023;71(3):736-748. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fukazawa Y, Ito T, Okimura T, Yamashita Y, Maeda T, Ota J. Predicting anxiety state using smartphone-based passive sensing. J Biomed Inform. May 2019;93:103151. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rashid H, Mendu S, Daniel KE, Beltzer ML, Teachman BA, Boukhechba M, et al. Predicting subjective measures of social anxiety from sparsely collected mobile sensor data. Proc ACM Interact Mob Wearable Ubiquitous Technol. Sep 04, 2020;4(3):1-24. [ FREE Full text ] [ CrossRef ]
• Zakaria C, Balan R, Lee Y. StressMon: scalable detection of perceived stress and depression using passive sensing of changes in work routines and group interactions. Proc ACM Hum Comput Interact. Nov 07, 2019;3(CSCW):1-29. [ FREE Full text ] [ CrossRef ]
• DaSilva AW, Huckins JF, Wang R, Wang W, Wagner DD, Campbell AT. Correlates of stress in the college environment uncovered by the application of penalized generalized estimating equations to mobile sensing data. JMIR Mhealth Uhealth. Mar 19, 2019;7(3):e12084. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nepal S, Mirjafari S, Martinez GJ, Audia P, Striegel A, Campbell AT. Detecting job promotion in information workers using mobile sensing. Proc ACM Interact Mob Wearable Ubiquitous Technol. Sep 04, 2020;4(3):1-28. [ FREE Full text ] [ CrossRef ]
• Saha K, Reddy MD, Mattingly S, Moskal E, Sirigiri A, De Choudhury M. LibRA: on LinkedIn based role ambiguity and its relationship with wellbeing and job performance. Proc ACM Hum Comput Interact. Nov 07, 2019;3(CSCW):1-30. [ FREE Full text ] [ CrossRef ]
• Morshed MB, Saha K, Li R, D'Mello SK, De Choudhury M, Abowd GD, et al. Prediction of mood instability with passive sensing. Proc ACM Interact Mob Wearable Ubiquitous Technol. Sep 09, 2019;3(3):1-21. [ FREE Full text ] [ CrossRef ]
• Acikmese Y, Alptekin SE. Prediction of stress levels with LSTM and passive mobile sensors. Procedia Comput Sci. 2019;159:658-667. [ FREE Full text ] [ CrossRef ]
• Boukhechba M, Huang Y, Chow P, Fua K, Teachman BA, Barnes LE. Monitoring social anxiety from mobility and communication patterns. In: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. 2017. Presented at: UbiComp '17: The 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing; September 11-15, 2017; Maui, Hawaii. [ CrossRef ]
• Tseng VW, Merrill M, Wittleder F, Abdullah S, Aung MH, Choudhury T. Assessing mental health issues on college campuses: preliminary findings from a pilot study. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 2016. Presented at: UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing; September 12-16, 2016; Heidelberg, Germany. [ CrossRef ]
• Xu X, Chikersal P, Doryab A, Villalba DK, Dutcher JM, Tumminia MJ, et al. Leveraging routine behavior and contextually-filtered features for depression detection among college students. Proc ACM Interact Mob Wearable Ubiquitous Technol. Sep 09, 2019;3(3):1-33. [ FREE Full text ] [ CrossRef ]
• Chikersal P, Doryab A, Tumminia M, Villalba DK, Dutcher JM, Liu X, et al. Detecting depression and predicting its onset using longitudinal symptoms captured by passive sensing: a machine learning approach with robust feature selection. ACM Trans Comput Hum Interact. Jan 20, 2021;28(1):1-41. [ FREE Full text ] [ CrossRef ]
• Meyerhoff J, Liu T, Kording KP, Ungar LH, Kaiser SM, Karr CJ, et al. Evaluation of changes in depression, anxiety, and social anxiety using smartphone sensor features: longitudinal cohort study. J Med Internet Res. Sep 03, 2021;23(9):e22844. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rhim S, Lee U, Han K. Tracking and modeling subjective well-being using smartphone-based digital phenotype. In: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 2020. Presented at: UMAP '20; July 14-17, 2020; Genoa, Italy. [ CrossRef ]
• Wang R, Wang W, daSilva A, Huckins JF, Kelley WM, Heatherton TF, et al. Tracking depression dynamics in college students using mobile phone and wearable sensing. Proc ACM Interact Mob Wearable Ubiquitous Technol. Mar 26, 2018;2(1):1-26. [ FREE Full text ] [ CrossRef ]
• Currey D, Torous J. Digital phenotyping correlations in larger mental health samples: analysis and replication. BJPsych Open. Jun 03, 2022;8(4):e106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Di Matteo D, Fotinos K, Lokuge S, Mason G, Sternat T, Katzman MA, et al. Automated screening for social anxiety, generalized anxiety, and depression from objective smartphone-collected data: cross-sectional study. J Med Internet Res. Aug 13, 2021;23(8):e28918. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sefidgar YS, Seo W, Kuehn KS, Althoff T, Browning A, Riskin E, et al. Passively-sensed behavioral correlates of discrimination events in college students. Proc ACM Hum Comput Interact. Nov 2019;3(CSCW):1-29. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mendu S, Baglione A, Baee S, Wu C, Ng B, Shaked A, et al. A framework for understanding the relationship between social media discourse and mental health. Proc ACM Hum Comput Interact. Oct 15, 2020;4(CSCW2):1-23. [ FREE Full text ] [ CrossRef ]
• Pratap A, Anguera JA, Renn BN, Neto EC, Volponi J, Mooney SD, et al. The feasibility of using smartphones to assess and remediate depression in Hispanic/Latino individuals nationally. In: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. 2017. Presented at: UbiComp '17: The 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing; September 11-15, 2017; Maui, Hawaii. [ CrossRef ]
• Mirjafari S, Masaba K, Grover T, Wang W, Audia P, Campbell AT, et al. Differentiating higher and lower job performers in the workplace using mobile sensing. Proc ACM Interact Mob Wearable Ubiquitous Technol. Jun 21, 2019;3(2):1-24. [ FREE Full text ] [ CrossRef ]
• Currey D, Hays R, Torous J. Digital phenotyping models of symptom improvement in college mental health: generalizability across two cohorts. J Technol Behav Sci. Mar 02, 2023:1-14. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckins JF, daSilva AW, Wang W, Hedlund E, Rogers C, Nepal SK, et al. Mental health and behavior of college students during the early phases of the COVID-19 pandemic: longitudinal smartphone and ecological momentary assessment study. J Med Internet Res. Jun 17, 2020;22(6):e20185. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mack DL, DaSilva AW, Rogers C, Hedlund E, Murphy EI, Vojdanovski V, et al. Mental health and behavior of college students during the COVID-19 pandemic: longitudinal mobile smartphone and ecological momentary assessment study, part II. J Med Internet Res. Jun 04, 2021;23(6):e28892. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xu X, Liu X, Zhang H, Wang W, Nepal S, Sefidgar Y, et al. GLOBEM: cross-dataset generalization of longitudinal human behavior modeling. Proc ACM Interact Mob Wearable Ubiquitous Technol. Jan 11, 2023;6(4):1-34. [ FREE Full text ] [ CrossRef ]
• Nepal S, Wang W, Vojdanovski V, Huckins JF, daSilva A, Meyer M, et al. COVID student study: a year in the life of college students during the COVID-19 pandemic through the lens of mobile phone sensing. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 2022. Presented at: CHI '22: CHI Conference on Human Factors in Computing Systems; April 29-May 5, 2022; New Orleans, LA. [ CrossRef ]
• Wang R, Chen F, Chen Z, Li T, Tignor S, Zhou X, et al. StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2014. Presented at: UbiComp '14: The 2014 ACM Conference on Ubiquitous Computing; September 13-17, 2014; Seattle, WA. [ CrossRef ]
• Mattingly SM, Gregg JM, Audia P, Bayraktaroglu AE, Campbell AT, Chawla NV, et al. The tesserae project: large-scale, longitudinal, in situ, multimodal sensing of information workers. In: Proceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 2019. Presented at: CHI '19: CHI Conference on Human Factors in Computing Systems; May 4-9, 2019; Glasgow, UK. [ CrossRef ]
• Teles A, Barros F, Rodrigues I, Barbosa A, Silva F, Coutinho L, et al. Internet of things applied to mental health: concepts, applications, and perspectives. In: Gupta N, Paiva S, editors. IoT and ICT for Healthcare Applications. Cham, Switzerland. Springer; Aug 13, 2020.
• Jin J, Gao B, Yang S, Zhao B, Luo L, Woo WL. Attention-block deep learning based features fusion in wearable social sensor for mental wellbeing evaluations. IEEE Access. May 13, 2020;8:89258-89268. [ FREE Full text ] [ CrossRef ]
• Nyqvist F, Forsman AK, Giuntoli G, Cattan M. Social capital as a resource for mental well-being in older people: a systematic review. Aging Ment Health. 2013;17(4):394-410. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Diagnostic And Statistical Manual Of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR). Washington, DC. American Psychiatric Association; 2013.
• Park SC, Kim D. The centrality of depression and anxiety symptoms in major depressive disorder determined using a network analysis. J Affect Disord. Jun 15, 2020;271:19-26. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peltz JS, Bodenlos JS, Kingery JN, Rogge RD. The role of financial strain in college students' work hours, sleep, and mental health. J Am Coll Health. 2021;69(6):577-584. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Melcher J, Hays R, Torous J. Digital phenotyping for mental health of college students: a clinical review. Evid Based Ment Health. Nov 2020;23(4):161-166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cosgrove L, Karter JM, McGinley M, Morrill Z. Digital phenotyping and digital psychotropic drugs: mental health surveillance tools that threaten human rights. Health Hum Rights. Dec 2020;22(2):33-39. [ FREE Full text ] [ Medline ]
• Zacher H, Rudolph CW. Individual differences and changes in subjective wellbeing during the early stages of the COVID-19 pandemic. Am Psychol. Jan 2021;76(1):50-62. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by L Buis; submitted 01.07.22; peer-reviewed by J Rooksby, KS Sahu, A Joseph; comments to author 09.08.22; revised version received 03.10.22; accepted 27.09.23; published 23.05.24.

©Adrien Choi, Aysel Ooi, Danielle Lottridge. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 23.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.