peer reviewed research on

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.25(3); 2014 Oct

Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide

Jacalyn kelly.

1 Clinical Biochemistry, Department of Pediatric Laboratory Medicine, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada

Tara Sadeghieh

Khosrow adeli.

2 Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada

3 Chair, Communications and Publications Division (CPD), International Federation for Sick Clinical Chemistry (IFCC), Milan, Italy

The authors declare no conflicts of interest regarding publication of this article.

Peer review has been defined as a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field. It functions to encourage authors to meet the accepted high standards of their discipline and to control the dissemination of research data to ensure that unwarranted claims, unacceptable interpretations or personal views are not published without prior expert review. Despite its wide-spread use by most journals, the peer review process has also been widely criticised due to the slowness of the process to publish new findings and due to perceived bias by the editors and/or reviewers. Within the scientific community, peer review has become an essential component of the academic writing process. It helps ensure that papers published in scientific journals answer meaningful research questions and draw accurate conclusions based on professionally executed experimentation. Submission of low quality manuscripts has become increasingly prevalent, and peer review acts as a filter to prevent this work from reaching the scientific community. The major advantage of a peer review process is that peer-reviewed articles provide a trusted form of scientific communication. Since scientific knowledge is cumulative and builds on itself, this trust is particularly important. Despite the positive impacts of peer review, critics argue that the peer review process stifles innovation in experimentation, and acts as a poor screen against plagiarism. Despite its downfalls, there has not yet been a foolproof system developed to take the place of peer review, however, researchers have been looking into electronic means of improving the peer review process. Unfortunately, the recent explosion in online only/electronic journals has led to mass publication of a large number of scientific articles with little or no peer review. This poses significant risk to advances in scientific knowledge and its future potential. The current article summarizes the peer review process, highlights the pros and cons associated with different types of peer review, and describes new methods for improving peer review.

WHAT IS PEER REVIEW AND WHAT IS ITS PURPOSE?

Peer Review is defined as “a process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same field” ( 1 ). Peer review is intended to serve two primary purposes. Firstly, it acts as a filter to ensure that only high quality research is published, especially in reputable journals, by determining the validity, significance and originality of the study. Secondly, peer review is intended to improve the quality of manuscripts that are deemed suitable for publication. Peer reviewers provide suggestions to authors on how to improve the quality of their manuscripts, and also identify any errors that need correcting before publication.

HISTORY OF PEER REVIEW

The concept of peer review was developed long before the scholarly journal. In fact, the peer review process is thought to have been used as a method of evaluating written work since ancient Greece ( 2 ). The peer review process was first described by a physician named Ishaq bin Ali al-Rahwi of Syria, who lived from 854-931 CE, in his book Ethics of the Physician ( 2 ). There, he stated that physicians must take notes describing the state of their patients’ medical conditions upon each visit. Following treatment, the notes were scrutinized by a local medical council to determine whether the physician had met the required standards of medical care. If the medical council deemed that the appropriate standards were not met, the physician in question could receive a lawsuit from the maltreated patient ( 2 ).

The invention of the printing press in 1453 allowed written documents to be distributed to the general public ( 3 ). At this time, it became more important to regulate the quality of the written material that became publicly available, and editing by peers increased in prevalence. In 1620, Francis Bacon wrote the work Novum Organum, where he described what eventually became known as the first universal method for generating and assessing new science ( 3 ). His work was instrumental in shaping the Scientific Method ( 3 ). In 1665, the French Journal des sçavans and the English Philosophical Transactions of the Royal Society were the first scientific journals to systematically publish research results ( 4 ). Philosophical Transactions of the Royal Society is thought to be the first journal to formalize the peer review process in 1665 ( 5 ), however, it is important to note that peer review was initially introduced to help editors decide which manuscripts to publish in their journals, and at that time it did not serve to ensure the validity of the research ( 6 ). It did not take long for the peer review process to evolve, and shortly thereafter papers were distributed to reviewers with the intent of authenticating the integrity of the research study before publication. The Royal Society of Edinburgh adhered to the following peer review process, published in their Medical Essays and Observations in 1731: “Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author.” ( 7 ). The Royal Society of London adopted this review procedure in 1752 and developed the “Committee on Papers” to review manuscripts before they were published in Philosophical Transactions ( 6 ).

Peer review in the systematized and institutionalized form has developed immensely since the Second World War, at least partly due to the large increase in scientific research during this period ( 7 ). It is now used not only to ensure that a scientific manuscript is experimentally and ethically sound, but also to determine which papers sufficiently meet the journal’s standards of quality and originality before publication. Peer review is now standard practice by most credible scientific journals, and is an essential part of determining the credibility and quality of work submitted.

IMPACT OF THE PEER REVIEW PROCESS

Peer review has become the foundation of the scholarly publication system because it effectively subjects an author’s work to the scrutiny of other experts in the field. Thus, it encourages authors to strive to produce high quality research that will advance the field. Peer review also supports and maintains integrity and authenticity in the advancement of science. A scientific hypothesis or statement is generally not accepted by the academic community unless it has been published in a peer-reviewed journal ( 8 ). The Institute for Scientific Information ( ISI ) only considers journals that are peer-reviewed as candidates to receive Impact Factors. Peer review is a well-established process which has been a formal part of scientific communication for over 300 years.

OVERVIEW OF THE PEER REVIEW PROCESS

The peer review process begins when a scientist completes a research study and writes a manuscript that describes the purpose, experimental design, results, and conclusions of the study. The scientist then submits this paper to a suitable journal that specializes in a relevant research field, a step referred to as pre-submission. The editors of the journal will review the paper to ensure that the subject matter is in line with that of the journal, and that it fits with the editorial platform. Very few papers pass this initial evaluation. If the journal editors feel the paper sufficiently meets these requirements and is written by a credible source, they will send the paper to accomplished researchers in the field for a formal peer review. Peer reviewers are also known as referees (this process is summarized in Figure 1 ). The role of the editor is to select the most appropriate manuscripts for the journal, and to implement and monitor the peer review process. Editors must ensure that peer reviews are conducted fairly, and in an effective and timely manner. They must also ensure that there are no conflicts of interest involved in the peer review process.

Overview of the review process

When a reviewer is provided with a paper, he or she reads it carefully and scrutinizes it to evaluate the validity of the science, the quality of the experimental design, and the appropriateness of the methods used. The reviewer also assesses the significance of the research, and judges whether the work will contribute to advancement in the field by evaluating the importance of the findings, and determining the originality of the research. Additionally, reviewers identify any scientific errors and references that are missing or incorrect. Peer reviewers give recommendations to the editor regarding whether the paper should be accepted, rejected, or improved before publication in the journal. The editor will mediate author-referee discussion in order to clarify the priority of certain referee requests, suggest areas that can be strengthened, and overrule reviewer recommendations that are beyond the study’s scope ( 9 ). If the paper is accepted, as per suggestion by the peer reviewer, the paper goes into the production stage, where it is tweaked and formatted by the editors, and finally published in the scientific journal. An overview of the review process is presented in Figure 1 .

WHO CONDUCTS REVIEWS?

Peer reviews are conducted by scientific experts with specialized knowledge on the content of the manuscript, as well as by scientists with a more general knowledge base. Peer reviewers can be anyone who has competence and expertise in the subject areas that the journal covers. Reviewers can range from young and up-and-coming researchers to old masters in the field. Often, the young reviewers are the most responsive and deliver the best quality reviews, though this is not always the case. On average, a reviewer will conduct approximately eight reviews per year, according to a study on peer review by the Publishing Research Consortium (PRC) ( 7 ). Journals will often have a pool of reviewers with diverse backgrounds to allow for many different perspectives. They will also keep a rather large reviewer bank, so that reviewers do not get burnt out, overwhelmed or time constrained from reviewing multiple articles simultaneously.

WHY DO REVIEWERS REVIEW?

Referees are typically not paid to conduct peer reviews and the process takes considerable effort, so the question is raised as to what incentive referees have to review at all. Some feel an academic duty to perform reviews, and are of the mentality that if their peers are expected to review their papers, then they should review the work of their peers as well. Reviewers may also have personal contacts with editors, and may want to assist as much as possible. Others review to keep up-to-date with the latest developments in their field, and reading new scientific papers is an effective way to do so. Some scientists use peer review as an opportunity to advance their own research as it stimulates new ideas and allows them to read about new experimental techniques. Other reviewers are keen on building associations with prestigious journals and editors and becoming part of their community, as sometimes reviewers who show dedication to the journal are later hired as editors. Some scientists see peer review as a chance to become aware of the latest research before their peers, and thus be first to develop new insights from the material. Finally, in terms of career development, peer reviewing can be desirable as it is often noted on one’s resume or CV. Many institutions consider a researcher’s involvement in peer review when assessing their performance for promotions ( 11 ). Peer reviewing can also be an effective way for a scientist to show their superiors that they are committed to their scientific field ( 5 ).

ARE REVIEWERS KEEN TO REVIEW?

A 2009 international survey of 4000 peer reviewers conducted by the charity Sense About Science at the British Science Festival at the University of Surrey, found that 90% of reviewers were keen to peer review ( 12 ). One third of respondents to the survey said they were happy to review up to five papers per year, and an additional one third of respondents were happy to review up to ten.

HOW LONG DOES IT TAKE TO REVIEW ONE PAPER?

On average, it takes approximately six hours to review one paper ( 12 ), however, this number may vary greatly depending on the content of the paper and the nature of the peer reviewer. One in every 100 participants in the “Sense About Science” survey claims to have taken more than 100 hours to review their last paper ( 12 ).

HOW TO DETERMINE IF A JOURNAL IS PEER REVIEWED

Ulrichsweb is a directory that provides information on over 300,000 periodicals, including information regarding which journals are peer reviewed ( 13 ). After logging into the system using an institutional login (eg. from the University of Toronto), search terms, journal titles or ISSN numbers can be entered into the search bar. The database provides the title, publisher, and country of origin of the journal, and indicates whether the journal is still actively publishing. The black book symbol (labelled ‘refereed’) reveals that the journal is peer reviewed.

THE EVALUATION CRITERIA FOR PEER REVIEW OF SCIENTIFIC PAPERS

As previously mentioned, when a reviewer receives a scientific manuscript, he/she will first determine if the subject matter is well suited for the content of the journal. The reviewer will then consider whether the research question is important and original, a process which may be aided by a literature scan of review articles.

Scientific papers submitted for peer review usually follow a specific structure that begins with the title, followed by the abstract, introduction, methodology, results, discussion, conclusions, and references. The title must be descriptive and include the concept and organism investigated, and potentially the variable manipulated and the systems used in the study. The peer reviewer evaluates if the title is descriptive enough, and ensures that it is clear and concise. A study by the National Association of Realtors (NAR) published by the Oxford University Press in 2006 indicated that the title of a manuscript plays a significant role in determining reader interest, as 72% of respondents said they could usually judge whether an article will be of interest to them based on the title and the author, while 13% of respondents claimed to always be able to do so ( 14 ).

The abstract is a summary of the paper, which briefly mentions the background or purpose, methods, key results, and major conclusions of the study. The peer reviewer assesses whether the abstract is sufficiently informative and if the content of the abstract is consistent with the rest of the paper. The NAR study indicated that 40% of respondents could determine whether an article would be of interest to them based on the abstract alone 60-80% of the time, while 32% could judge an article based on the abstract 80-100% of the time ( 14 ). This demonstrates that the abstract alone is often used to assess the value of an article.

The introduction of a scientific paper presents the research question in the context of what is already known about the topic, in order to identify why the question being studied is of interest to the scientific community, and what gap in knowledge the study aims to fill ( 15 ). The introduction identifies the study’s purpose and scope, briefly describes the general methods of investigation, and outlines the hypothesis and predictions ( 15 ). The peer reviewer determines whether the introduction provides sufficient background information on the research topic, and ensures that the research question and hypothesis are clearly identifiable.

The methods section describes the experimental procedures, and explains why each experiment was conducted. The methods section also includes the equipment and reagents used in the investigation. The methods section should be detailed enough that it can be used it to repeat the experiment ( 15 ). Methods are written in the past tense and in the active voice. The peer reviewer assesses whether the appropriate methods were used to answer the research question, and if they were written with sufficient detail. If information is missing from the methods section, it is the peer reviewer’s job to identify what details need to be added.

The results section is where the outcomes of the experiment and trends in the data are explained without judgement, bias or interpretation ( 15 ). This section can include statistical tests performed on the data, as well as figures and tables in addition to the text. The peer reviewer ensures that the results are described with sufficient detail, and determines their credibility. Reviewers also confirm that the text is consistent with the information presented in tables and figures, and that all figures and tables included are important and relevant ( 15 ). The peer reviewer will also make sure that table and figure captions are appropriate both contextually and in length, and that tables and figures present the data accurately.

The discussion section is where the data is analyzed. Here, the results are interpreted and related to past studies ( 15 ). The discussion describes the meaning and significance of the results in terms of the research question and hypothesis, and states whether the hypothesis was supported or rejected. This section may also provide possible explanations for unusual results and suggestions for future research ( 15 ). The discussion should end with a conclusions section that summarizes the major findings of the investigation. The peer reviewer determines whether the discussion is clear and focused, and whether the conclusions are an appropriate interpretation of the results. Reviewers also ensure that the discussion addresses the limitations of the study, any anomalies in the results, the relationship of the study to previous research, and the theoretical implications and practical applications of the study.

The references are found at the end of the paper, and list all of the information sources cited in the text to describe the background, methods, and/or interpret results. Depending on the citation method used, the references are listed in alphabetical order according to author last name, or numbered according to the order in which they appear in the paper. The peer reviewer ensures that references are used appropriately, cited accurately, formatted correctly, and that none are missing.

Finally, the peer reviewer determines whether the paper is clearly written and if the content seems logical. After thoroughly reading through the entire manuscript, they determine whether it meets the journal’s standards for publication,

and whether it falls within the top 25% of papers in its field ( 16 ) to determine priority for publication. An overview of what a peer reviewer looks for when evaluating a manuscript, in order of importance, is presented in Figure 2 .

How a peer review evaluates a manuscript

To increase the chance of success in the peer review process, the author must ensure that the paper fully complies with the journal guidelines before submission. The author must also be open to criticism and suggested revisions, and learn from mistakes made in previous submissions.

ADVANTAGES AND DISADVANTAGES OF THE DIFFERENT TYPES OF PEER REVIEW

The peer review process is generally conducted in one of three ways: open review, single-blind review, or double-blind review. In an open review, both the author of the paper and the peer reviewer know one another’s identity. Alternatively, in single-blind review, the reviewer’s identity is kept private, but the author’s identity is revealed to the reviewer. In double-blind review, the identities of both the reviewer and author are kept anonymous. Open peer review is advantageous in that it prevents the reviewer from leaving malicious comments, being careless, or procrastinating completion of the review ( 2 ). It encourages reviewers to be open and honest without being disrespectful. Open reviewing also discourages plagiarism amongst authors ( 2 ). On the other hand, open peer review can also prevent reviewers from being honest for fear of developing bad rapport with the author. The reviewer may withhold or tone down their criticisms in order to be polite ( 2 ). This is especially true when younger reviewers are given a more esteemed author’s work, in which case the reviewer may be hesitant to provide criticism for fear that it will damper their relationship with a superior ( 2 ). According to the Sense About Science survey, editors find that completely open reviewing decreases the number of people willing to participate, and leads to reviews of little value ( 12 ). In the aforementioned study by the PRC, only 23% of authors surveyed had experience with open peer review ( 7 ).

Single-blind peer review is by far the most common. In the PRC study, 85% of authors surveyed had experience with single-blind peer review ( 7 ). This method is advantageous as the reviewer is more likely to provide honest feedback when their identity is concealed ( 2 ). This allows the reviewer to make independent decisions without the influence of the author ( 2 ). The main disadvantage of reviewer anonymity, however, is that reviewers who receive manuscripts on subjects similar to their own research may be tempted to delay completing the review in order to publish their own data first ( 2 ).

Double-blind peer review is advantageous as it prevents the reviewer from being biased against the author based on their country of origin or previous work ( 2 ). This allows the paper to be judged based on the quality of the content, rather than the reputation of the author. The Sense About Science survey indicates that 76% of researchers think double-blind peer review is a good idea ( 12 ), and the PRC survey indicates that 45% of authors have had experience with double-blind peer review ( 7 ). The disadvantage of double-blind peer review is that, especially in niche areas of research, it can sometimes be easy for the reviewer to determine the identity of the author based on writing style, subject matter or self-citation, and thus, impart bias ( 2 ).

Masking the author’s identity from peer reviewers, as is the case in double-blind review, is generally thought to minimize bias and maintain review quality. A study by Justice et al. in 1998 investigated whether masking author identity affected the quality of the review ( 17 ). One hundred and eighteen manuscripts were randomized; 26 were peer reviewed as normal, and 92 were moved into the ‘intervention’ arm, where editor quality assessments were completed for 77 manuscripts and author quality assessments were completed for 40 manuscripts ( 17 ). There was no perceived difference in quality between the masked and unmasked reviews. Additionally, the masking itself was often unsuccessful, especially with well-known authors ( 17 ). However, a previous study conducted by McNutt et al. had different results ( 18 ). In this case, blinding was successful 73% of the time, and they found that when author identity was masked, the quality of review was slightly higher ( 18 ). Although Justice et al. argued that this difference was too small to be consequential, their study targeted only biomedical journals, and the results cannot be generalized to journals of a different subject matter ( 17 ). Additionally, there were problems masking the identities of well-known authors, introducing a flaw in the methods. Regardless, Justice et al. concluded that masking author identity from reviewers may not improve review quality ( 17 ).

In addition to open, single-blind and double-blind peer review, there are two experimental forms of peer review. In some cases, following publication, papers may be subjected to post-publication peer review. As many papers are now published online, the scientific community has the opportunity to comment on these papers, engage in online discussions and post a formal review. For example, online publishers PLOS and BioMed Central have enabled scientists to post comments on published papers if they are registered users of the site ( 10 ). Philica is another journal launched with this experimental form of peer review. Only 8% of authors surveyed in the PRC study had experience with post-publication review ( 7 ). Another experimental form of peer review called Dynamic Peer Review has also emerged. Dynamic peer review is conducted on websites such as Naboj, which allow scientists to conduct peer reviews on articles in the preprint media ( 19 ). The peer review is conducted on repositories and is a continuous process, which allows the public to see both the article and the reviews as the article is being developed ( 19 ). Dynamic peer review helps prevent plagiarism as the scientific community will already be familiar with the work before the peer reviewed version appears in print ( 19 ). Dynamic review also reduces the time lag between manuscript submission and publishing. An example of a preprint server is the ‘arXiv’ developed by Paul Ginsparg in 1991, which is used primarily by physicists ( 19 ). These alternative forms of peer review are still un-established and experimental. Traditional peer review is time-tested and still highly utilized. All methods of peer review have their advantages and deficiencies, and all are prone to error.

PEER REVIEW OF OPEN ACCESS JOURNALS

Open access (OA) journals are becoming increasingly popular as they allow the potential for widespread distribution of publications in a timely manner ( 20 ). Nevertheless, there can be issues regarding the peer review process of open access journals. In a study published in Science in 2013, John Bohannon submitted 304 slightly different versions of a fictional scientific paper (written by a fake author, working out of a non-existent institution) to a selected group of OA journals. This study was performed in order to determine whether papers submitted to OA journals are properly reviewed before publication in comparison to subscription-based journals. The journals in this study were selected from the Directory of Open Access Journals (DOAJ) and Biall’s List, a list of journals which are potentially predatory, and all required a fee for publishing ( 21 ). Of the 304 journals, 157 accepted a fake paper, suggesting that acceptance was based on financial interest rather than the quality of article itself, while 98 journals promptly rejected the fakes ( 21 ). Although this study highlights useful information on the problems associated with lower quality publishers that do not have an effective peer review system in place, the article also generalizes the study results to all OA journals, which can be detrimental to the general perception of OA journals. There were two limitations of the study that made it impossible to accurately determine the relationship between peer review and OA journals: 1) there was no control group (subscription-based journals), and 2) the fake papers were sent to a non-randomized selection of journals, resulting in bias.

JOURNAL ACCEPTANCE RATES

Based on a recent survey, the average acceptance rate for papers submitted to scientific journals is about 50% ( 7 ). Twenty percent of the submitted manuscripts that are not accepted are rejected prior to review, and 30% are rejected following review ( 7 ). Of the 50% accepted, 41% are accepted with the condition of revision, while only 9% are accepted without the request for revision ( 7 ).

SATISFACTION WITH THE PEER REVIEW SYSTEM

Based on a recent survey by the PRC, 64% of academics are satisfied with the current system of peer review, and only 12% claimed to be ‘dissatisfied’ ( 7 ). The large majority, 85%, agreed with the statement that ‘scientific communication is greatly helped by peer review’ ( 7 ). There was a similarly high level of support (83%) for the idea that peer review ‘provides control in scientific communication’ ( 7 ).

HOW TO PEER REVIEW EFFECTIVELY

The following are ten tips on how to be an effective peer reviewer as indicated by Brian Lucey, an expert on the subject ( 22 ):

1) Be professional

Peer review is a mutual responsibility among fellow scientists, and scientists are expected, as part of the academic community, to take part in peer review. If one is to expect others to review their work, they should commit to reviewing the work of others as well, and put effort into it.

2) Be pleasant

If the paper is of low quality, suggest that it be rejected, but do not leave ad hominem comments. There is no benefit to being ruthless.

3) Read the invite

When emailing a scientist to ask them to conduct a peer review, the majority of journals will provide a link to either accept or reject. Do not respond to the email, respond to the link.

4) Be helpful

Suggest how the authors can overcome the shortcomings in their paper. A review should guide the author on what is good and what needs work from the reviewer’s perspective.

5) Be scientific

The peer reviewer plays the role of a scientific peer, not an editor for proofreading or decision-making. Don’t fill a review with comments on editorial and typographic issues. Instead, focus on adding value with scientific knowledge and commenting on the credibility of the research conducted and conclusions drawn. If the paper has a lot of typographical errors, suggest that it be professionally proof edited as part of the review.

6) Be timely

Stick to the timeline given when conducting a peer review. Editors track who is reviewing what and when and will know if someone is late on completing a review. It is important to be timely both out of respect for the journal and the author, as well as to not develop a reputation of being late for review deadlines.

7) Be realistic

The peer reviewer must be realistic about the work presented, the changes they suggest and their role. Peer reviewers may set the bar too high for the paper they are editing by proposing changes that are too ambitious and editors must override them.

8) Be empathetic

Ensure that the review is scientific, helpful and courteous. Be sensitive and respectful with word choice and tone in a review.

Remember that both specialists and generalists can provide valuable insight when peer reviewing. Editors will try to get both specialised and general reviewers for any particular paper to allow for different perspectives. If someone is asked to review, the editor has determined they have a valid and useful role to play, even if the paper is not in their area of expertise.

10) Be organised

A review requires structure and logical flow. A reviewer should proofread their review before submitting it for structural, grammatical and spelling errors as well as for clarity. Most publishers provide short guides on structuring a peer review on their website. Begin with an overview of the proposed improvements; then provide feedback on the paper structure, the quality of data sources and methods of investigation used, the logical flow of argument, and the validity of conclusions drawn. Then provide feedback on style, voice and lexical concerns, with suggestions on how to improve.

In addition, the American Physiology Society (APS) recommends in its Peer Review 101 Handout that peer reviewers should put themselves in both the editor’s and author’s shoes to ensure that they provide what both the editor and the author need and expect ( 11 ). To please the editor, the reviewer should ensure that the peer review is completed on time, and that it provides clear explanations to back up recommendations. To be helpful to the author, the reviewer must ensure that their feedback is constructive. It is suggested that the reviewer take time to think about the paper; they should read it once, wait at least a day, and then re-read it before writing the review ( 11 ). The APS also suggests that Graduate students and researchers pay attention to how peer reviewers edit their work, as well as to what edits they find helpful, in order to learn how to peer review effectively ( 11 ). Additionally, it is suggested that Graduate students practice reviewing by editing their peers’ papers and asking a faculty member for feedback on their efforts. It is recommended that young scientists offer to peer review as often as possible in order to become skilled at the process ( 11 ). The majority of students, fellows and trainees do not get formal training in peer review, but rather learn by observing their mentors. According to the APS, one acquires experience through networking and referrals, and should therefore try to strengthen relationships with journal editors by offering to review manuscripts ( 11 ). The APS also suggests that experienced reviewers provide constructive feedback to students and junior colleagues on their peer review efforts, and encourages them to peer review to demonstrate the importance of this process in improving science ( 11 ).

The peer reviewer should only comment on areas of the manuscript that they are knowledgeable about ( 23 ). If there is any section of the manuscript they feel they are not qualified to review, they should mention this in their comments and not provide further feedback on that section. The peer reviewer is not permitted to share any part of the manuscript with a colleague (even if they may be more knowledgeable in the subject matter) without first obtaining permission from the editor ( 23 ). If a peer reviewer comes across something they are unsure of in the paper, they can consult the literature to try and gain insight. It is important for scientists to remember that if a paper can be improved by the expertise of one of their colleagues, the journal must be informed of the colleague’s help, and approval must be obtained for their colleague to read the protected document. Additionally, the colleague must be identified in the confidential comments to the editor, in order to ensure that he/she is appropriately credited for any contributions ( 23 ). It is the job of the reviewer to make sure that the colleague assisting is aware of the confidentiality of the peer review process ( 23 ). Once the review is complete, the manuscript must be destroyed and cannot be saved electronically by the reviewers ( 23 ).

COMMON ERRORS IN SCIENTIFIC PAPERS

When performing a peer review, there are some common scientific errors to look out for. Most of these errors are violations of logic and common sense: these may include contradicting statements, unwarranted conclusions, suggestion of causation when there is only support for correlation, inappropriate extrapolation, circular reasoning, or pursuit of a trivial question ( 24 ). It is also common for authors to suggest that two variables are different because the effects of one variable are statistically significant while the effects of the other variable are not, rather than directly comparing the two variables ( 24 ). Authors sometimes oversee a confounding variable and do not control for it, or forget to include important details on how their experiments were controlled or the physical state of the organisms studied ( 24 ). Another common fault is the author’s failure to define terms or use words with precision, as these practices can mislead readers ( 24 ). Jargon and/or misused terms can be a serious problem in papers. Inaccurate statements about specific citations are also a common occurrence ( 24 ). Additionally, many studies produce knowledge that can be applied to areas of science outside the scope of the original study, therefore it is better for reviewers to look at the novelty of the idea, conclusions, data, and methodology, rather than scrutinize whether or not the paper answered the specific question at hand ( 24 ). Although it is important to recognize these points, when performing a review it is generally better practice for the peer reviewer to not focus on a checklist of things that could be wrong, but rather carefully identify the problems specific to each paper and continuously ask themselves if anything is missing ( 24 ). An extremely detailed description of how to conduct peer review effectively is presented in the paper How I Review an Original Scientific Article written by Frederic G. Hoppin, Jr. It can be accessed through the American Physiological Society website under the Peer Review Resources section.

CRITICISM OF PEER REVIEW

A major criticism of peer review is that there is little evidence that the process actually works, that it is actually an effective screen for good quality scientific work, and that it actually improves the quality of scientific literature. As a 2002 study published in the Journal of the American Medical Association concluded, ‘Editorial peer review, although widely used, is largely untested and its effects are uncertain’ ( 25 ). Critics also argue that peer review is not effective at detecting errors. Highlighting this point, an experiment by Godlee et al. published in the British Medical Journal (BMJ) inserted eight deliberate errors into a paper that was nearly ready for publication, and then sent the paper to 420 potential reviewers ( 7 ). Of the 420 reviewers that received the paper, 221 (53%) responded, the average number of errors spotted by reviewers was two, no reviewer spotted more than five errors, and 35 reviewers (16%) did not spot any.

Another criticism of peer review is that the process is not conducted thoroughly by scientific conferences with the goal of obtaining large numbers of submitted papers. Such conferences often accept any paper sent in, regardless of its credibility or the prevalence of errors, because the more papers they accept, the more money they can make from author registration fees ( 26 ). This misconduct was exposed in 2014 by three MIT graduate students by the names of Jeremy Stribling, Dan Aguayo and Maxwell Krohn, who developed a simple computer program called SCIgen that generates nonsense papers and presents them as scientific papers ( 26 ). Subsequently, a nonsense SCIgen paper submitted to a conference was promptly accepted. Nature recently reported that French researcher Cyril Labbé discovered that sixteen SCIgen nonsense papers had been used by the German academic publisher Springer ( 26 ). Over 100 nonsense papers generated by SCIgen were published by the US Institute of Electrical and Electronic Engineers (IEEE) ( 26 ). Both organisations have been working to remove the papers. Labbé developed a program to detect SCIgen papers and has made it freely available to ensure publishers and conference organizers do not accept nonsense work in the future. It is available at this link: http://scigendetect.on.imag.fr/main.php ( 26 ).

Additionally, peer review is often criticized for being unable to accurately detect plagiarism. However, many believe that detecting plagiarism cannot practically be included as a component of peer review. As explained by Alice Tuff, development manager at Sense About Science, ‘The vast majority of authors and reviewers think peer review should detect plagiarism (81%) but only a minority (38%) think it is capable. The academic time involved in detecting plagiarism through peer review would cause the system to grind to a halt’ ( 27 ). Publishing house Elsevier began developing electronic plagiarism tools with the help of journal editors in 2009 to help improve this issue ( 27 ).

It has also been argued that peer review has lowered research quality by limiting creativity amongst researchers. Proponents of this view claim that peer review has repressed scientists from pursuing innovative research ideas and bold research questions that have the potential to make major advances and paradigm shifts in the field, as they believe that this work will likely be rejected by their peers upon review ( 28 ). Indeed, in some cases peer review may result in rejection of innovative research, as some studies may not seem particularly strong initially, yet may be capable of yielding very interesting and useful developments when examined under different circumstances, or in the light of new information ( 28 ). Scientists that do not believe in peer review argue that the process stifles the development of ingenious ideas, and thus the release of fresh knowledge and new developments into the scientific community.

Another issue that peer review is criticized for, is that there are a limited number of people that are competent to conduct peer review compared to the vast number of papers that need reviewing. An enormous number of papers published (1.3 million papers in 23,750 journals in 2006), but the number of competent peer reviewers available could not have reviewed them all ( 29 ). Thus, people who lack the required expertise to analyze the quality of a research paper are conducting reviews, and weak papers are being accepted as a result. It is now possible to publish any paper in an obscure journal that claims to be peer-reviewed, though the paper or journal itself could be substandard ( 29 ). On a similar note, the US National Library of Medicine indexes 39 journals that specialize in alternative medicine, and though they all identify themselves as “peer-reviewed”, they rarely publish any high quality research ( 29 ). This highlights the fact that peer review of more controversial or specialized work is typically performed by people who are interested and hold similar views or opinions as the author, which can cause bias in their review. For instance, a paper on homeopathy is likely to be reviewed by fellow practicing homeopaths, and thus is likely to be accepted as credible, though other scientists may find the paper to be nonsense ( 29 ). In some cases, papers are initially published, but their credibility is challenged at a later date and they are subsequently retracted. Retraction Watch is a website dedicated to revealing papers that have been retracted after publishing, potentially due to improper peer review ( 30 ).

Additionally, despite its many positive outcomes, peer review is also criticized for being a delay to the dissemination of new knowledge into the scientific community, and as an unpaid-activity that takes scientists’ time away from activities that they would otherwise prioritize, such as research and teaching, for which they are paid ( 31 ). As described by Eva Amsen, Outreach Director for F1000Research, peer review was originally developed as a means of helping editors choose which papers to publish when journals had to limit the number of papers they could print in one issue ( 32 ). However, nowadays most journals are available online, either exclusively or in addition to print, and many journals have very limited printing runs ( 32 ). Since there are no longer page limits to journals, any good work can and should be published. Consequently, being selective for the purpose of saving space in a journal is no longer a valid excuse that peer reviewers can use to reject a paper ( 32 ). However, some reviewers have used this excuse when they have personal ulterior motives, such as getting their own research published first.

RECENT INITIATIVES TOWARDS IMPROVING PEER REVIEW

F1000Research was launched in January 2013 by Faculty of 1000 as an open access journal that immediately publishes papers (after an initial check to ensure that the paper is in fact produced by a scientist and has not been plagiarised), and then conducts transparent post-publication peer review ( 32 ). F1000Research aims to prevent delays in new science reaching the academic community that are caused by prolonged publication times ( 32 ). It also aims to make peer reviewing more fair by eliminating any anonymity, which prevents reviewers from delaying the completion of a review so they can publish their own similar work first ( 32 ). F1000Research offers completely open peer review, where everything is published, including the name of the reviewers, their review reports, and the editorial decision letters ( 32 ).

PeerJ was founded by Jason Hoyt and Peter Binfield in June 2012 as an open access, peer reviewed scholarly journal for the Biological and Medical Sciences ( 33 ). PeerJ selects articles to publish based only on scientific and methodological soundness, not on subjective determinants of ‘impact ’, ‘novelty’ or ‘interest’ ( 34 ). It works on a “lifetime publishing plan” model which charges scientists for publishing plans that give them lifetime rights to publish with PeerJ, rather than charging them per publication ( 34 ). PeerJ also encourages open peer review, and authors are given the option to post the full peer review history of their submission with their published article ( 34 ). PeerJ also offers a pre-print review service called PeerJ Pre-prints, in which paper drafts are reviewed before being sent to PeerJ to publish ( 34 ).

Rubriq is an independent peer review service designed by Shashi Mudunuri and Keith Collier to improve the peer review system ( 35 ). Rubriq is intended to decrease redundancy in the peer review process so that the time lost in redundant reviewing can be put back into research ( 35 ). According to Keith Collier, over 15 million hours are lost each year to redundant peer review, as papers get rejected from one journal and are subsequently submitted to a less prestigious journal where they are reviewed again ( 35 ). Authors often have to submit their manuscript to multiple journals, and are often rejected multiple times before they find the right match. This process could take months or even years ( 35 ). Rubriq makes peer review portable in order to help authors choose the journal that is best suited for their manuscript from the beginning, thus reducing the time before their paper is published ( 35 ). Rubriq operates under an author-pay model, in which the author pays a fee and their manuscript undergoes double-blind peer review by three expert academic reviewers using a standardized scorecard ( 35 ). The majority of the author’s fee goes towards a reviewer honorarium ( 35 ). The papers are also screened for plagiarism using iThenticate ( 35 ). Once the manuscript has been reviewed by the three experts, the most appropriate journal for submission is determined based on the topic and quality of the paper ( 35 ). The paper is returned to the author in 1-2 weeks with the Rubriq Report ( 35 ). The author can then submit their paper to the suggested journal with the Rubriq Report attached. The Rubriq Report will give the journal editors a much stronger incentive to consider the paper as it shows that three experts have recommended the paper to them ( 35 ). Rubriq also has its benefits for reviewers; the Rubriq scorecard gives structure to the peer review process, and thus makes it consistent and efficient, which decreases time and stress for the reviewer. Reviewers also receive feedback on their reviews and most significantly, they are compensated for their time ( 35 ). Journals also benefit, as they receive pre-screened papers, reducing the number of papers sent to their own reviewers, which often end up rejected ( 35 ). This can reduce reviewer fatigue, and allow only higher-quality articles to be sent to their peer reviewers ( 35 ).

According to Eva Amsen, peer review and scientific publishing are moving in a new direction, in which all papers will be posted online, and a post-publication peer review will take place that is independent of specific journal criteria and solely focused on improving paper quality ( 32 ). Journals will then choose papers that they find relevant based on the peer reviews and publish those papers as a collection ( 32 ). In this process, peer review and individual journals are uncoupled ( 32 ). In Keith Collier’s opinion, post-publication peer review is likely to become more prevalent as a complement to pre-publication peer review, but not as a replacement ( 35 ). Post-publication peer review will not serve to identify errors and fraud but will provide an additional measurement of impact ( 35 ). Collier also believes that as journals and publishers consolidate into larger systems, there will be stronger potential for “cascading” and shared peer review ( 35 ).

CONCLUDING REMARKS

Peer review has become fundamental in assisting editors in selecting credible, high quality, novel and interesting research papers to publish in scientific journals and to ensure the correction of any errors or issues present in submitted papers. Though the peer review process still has some flaws and deficiencies, a more suitable screening method for scientific papers has not yet been proposed or developed. Researchers have begun and must continue to look for means of addressing the current issues with peer review to ensure that it is a full-proof system that ensures only quality research papers are released into the scientific community.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on December 17, 2021 by Tegan George . Revised on June 22, 2023.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, other interesting articles, frequently asked questions about peer reviews.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Prevent plagiarism. Run a free check.

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymized) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymized comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymized) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymized) review —where the identities of the author, reviewers, and editors are all anonymized—does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimizes potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymize everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimize back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarize the argument in your own words

Summarizing the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organized. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

Tip: Try not to focus too much on the minor issues. If the manuscript has a lot of typos, consider making a note that the author should address spelling and grammar issues, rather than going through and fixing each one.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticized, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the “compliment sandwich,” where you “sandwich” your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarized or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published. There is also high risk of publication bias , where journals are more likely to publish studies with positive findings than studies with negative findings.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Thematic analysis
• Discourse analysis
• Cohort study
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Peer review is a process of evaluating submissions to an academic journal. Utilizing rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication. For this reason, academic journals are often considered among the most credible sources you can use in a research project– provided that the journal itself is trustworthy and well-regarded.

In general, the peer review process follows the following steps: 

• Reject the manuscript and send it back to author, or 
• Send it onward to the selected peer reviewer(s) 
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made. 
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field. It acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure. 

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is Peer Review? | Types & Examples. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/peer-review/

Is this article helpful?

Tegan George

Other students also liked, what are credible sources & how to spot them | examples, ethical considerations in research | types & examples, applying the craap test & evaluating sources, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Navigation group

Home banner.

Where scientists empower society

Creating solutions for healthy lives on a healthy planet.

most-cited publisher

largest publisher

2.5 billion

article views and downloads

Main Content

• Editors and reviewers
• Collaborators

Find a journal

We have a home for your research. Our community led journals cover more than 1,500 academic disciplines and are some of the largest and most cited in their fields.

Submit your research

Start your submission and get more impact for your research by publishing with us.

Author guidelines

Ready to publish? Check our author guidelines for everything you need to know about submitting, from choosing a journal and section to preparing your manuscript.

Peer review

Our efficient collaborative peer review means you’ll get a decision on your manuscript in an average of 61 days.

Article publishing charges (APCs) apply to articles that are accepted for publication by our external and independent editorial boards

Press office

Visit our press office for key media contact information, as well as Frontiers’ media kit, including our embargo policy, logos, key facts, leadership bios, and imagery.

Institutional partnerships

Join more than 555 institutions around the world already benefiting from an institutional membership with Frontiers, including CERN, Max Planck Society, and the University of Oxford.

Publishing partnerships

Partner with Frontiers and make your society’s transition to open access a reality with our custom-built platform and publishing expertise.

Policy Labs

Connecting experts from business, science, and policy to strengthen the dialogue between scientific research and informed policymaking.

How we publish

All Frontiers journals are community-run and fully open access, so every research article we publish is immediately and permanently free to read.

Editor guidelines

Reviewing a manuscript? See our guidelines for everything you need to know about our peer review process.

Become an editor

Apply to join an editorial board and collaborate with an international team of carefully selected independent researchers.

My assignments

It’s easy to find and track your editorial assignments with our platform, 'My Frontiers' – saving you time to spend on your own research.

Scientists call for urgent action to prevent immune-mediated illnesses caused by climate change and biodiversity loss

Climate change, pollution, and collapsing biodiversity are damaging our immune systems, but improving the environment offers effective and fast-acting protection.

Safeguarding peer review to ensure quality at scale

Making scientific research open has never been more important. But for research to be trusted, it must be of the highest quality. Facing an industry-wide rise in fraudulent science, Frontiers has increased its focus on safeguarding quality.

Chronic stress and inflammation linked to societal and environmental impacts in new study 

Scientists hypothesize that as-yet unrecognized inflammatory stress is spreading among people at unprecedented rates, affecting our cognitive ability to address climate change, war, and other critical issues.

Tiny crustaceans discovered preying on live jellyfish during harsh Arctic night

Scientists used DNA metabarcoding to show for the first time that jellyfish are an important food for amphipods during the Arctic polar night in waters off Svalbard, at a time of year when other food resources are scarce.

Why studying astronauts’ microbiomes is crucial to ensure deep space mission success

In a new Frontiers’ guest editorial, Prof Dr Lembit Sihver, director of CRREAT at the Nuclear Physics Institute of the Czech Academy of Sciences and his co-authors explore the impact the microbiome has on human health in space.

Cake and cookies may increase Alzheimer’s risk: Here are five Frontiers articles you won’t want to miss

At Frontiers, we bring some of the world’s best research to a global audience. But with tens of thousands of articles published each year, it’s impossible to cover all of them. Here are just five amazing papers you may have missed.

2024's top 10 tech-driven Research Topics

Frontiers has compiled a list of 10 Research Topics that embrace the potential of technology to advance scientific breakthroughs and change the world for the better.

Get the latest research updates, subscribe to our newsletter

• Library databases
• Library website

Evaluating Resources: Peer Review

What is peer review.

The term peer review can be confusing, since in some of your courses you may be asked to review the work of your peers. When we talk about peer-reviewed journal articles, this has nothing to do with your peers!

Peer-reviewed journals, also called refereed journals, are journals that use a specific scholarly review process to try to ensure the accuracy and reliability of published articles. When an article is submitted to a peer-reviewed journal for publication, the journal sends the article to other scholars/experts in that field and has them review the article for accuracy and reliability.

Find out more about peer review with our Peer Review Guide:

• Peer Review Guide

Types of peer review

Single blind.

In this process, the names of the reviewers are not known to the author(s). The reviewers do know the name of the author(s).

Double blind

Here, neither reviewers or authors know each other's names.

In the open review process, both reviewers and authors know each other's names.

What about editorial review?

Journals also use an editorial review process. This is not the same as peer review. In an editorial review process an article is evaluated for style guidelines and for clarity. Reviewers here do not look at technical accuracy or errors in data or methodology, but instead look at grammar, style, and whether an article is well written.

What is the difference between scholarly and peer review?

Not all scholarly journals are peer reviewed, but all peer-reviewed journals are scholarly.

• Things that are written for a scholarly or academic audience are considered scholarly writing.
• Peer-reviewed journals are a part of the larger category of scholarly writing.
• Scholarly writing includes many resources that are not peer reviewed, such as books, textbooks, and dissertations.

Scholarly writing does not come with a label that says scholarly . You will need to evaluate the resource to see if it is

• aimed at a scholarly audience
• reporting research, theories or other types of information important to scholars
• documenting and citing sources used to help authenticate the research done

The standard peer review process only applies to journals. While scholarly writing has certainly been edited and reviewed, peer review is a specific process only used by peer-reviewed journals. Books and dissertations may be scholarly, but are not considered peer reviewed.

Check out Select the Right Source for help with what kinds of resources are appropriate for discussion posts, assignments, projects, and more:

• Select the Right Source

How do I locate or verify peer-reviewed articles?

The peer review process is initiated by the journal publisher before an article is even published. Nowhere in the article will it tell you whether or not the article has gone through a peer review process.

You can locate peer-reviewed articles in the Library databases, typically by checking a limiter box.

• Quick Answer: How do I find scholarly, peer reviewed journal articles?

You can verify whether a journal uses a peer review process by using Ulrich's Periodicals Directory.

• Quick Answer: How do I verify that my article is peer reviewed?

What about resources that are not peer-reviewed?

Limiting your search to peer review is a way that you can ensure that you're looking at scholarly journal articles, and not popular or trade publications. Because peer-reviewed articles have been vetted by experts in the field, they are viewed as being held to a higher standard, and therefore are considered to be a high quality source. Professors often prefer peer-reviewed articles because they are considered to be of higher quality.

There are times, though, when the information you need may not be available in a peer-reviewed article.

• You may need to find original work on a theory that was first published in a book.
• You may need to find very current statistical data that comes from a government website.
• You may need background information that comes from a scholarly encyclopedia.

You will want to evaluate these resources to make sure that they are the best source for the information you need.

Note: If you are required for an assignment to find information from a peer-reviewed journal, then you will not be able to use non-peer-reviewed sources such as books, dissertations, or government websites. It's always best to clarify any questions over assignments with your professor.

• Previous Page: Evaluation Methods
• Next Page: Primary & Secondary Sources
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections

How to Write a Peer Review

When you write a peer review for a manuscript, what should you include in your comments? What should you leave out? And how should the review be formatted?

This guide provides quick tips for writing and organizing your reviewer report.

Review Outline

Use an outline for your reviewer report so it’s easy for the editors and author to follow. This will also help you keep your comments organized.

Think about structuring your review like an inverted pyramid. Put the most important information at the top, followed by details and examples in the center, and any additional points at the very bottom.

Here’s how your outline might look:

1. Summary of the research and your overall impression

In your own words, summarize what the manuscript claims to report. This shows the editor how you interpreted the manuscript and will highlight any major differences in perspective between you and the other reviewers. Give an overview of the manuscript’s strengths and weaknesses. Think about this as your “take-home” message for the editors. End this section with your recommended course of action.

2. Discussion of specific areas for improvement

It’s helpful to divide this section into two parts: one for major issues and one for minor issues. Within each section, you can talk about the biggest issues first or go systematically figure-by-figure or claim-by-claim. Number each item so that your points are easy to follow (this will also make it easier for the authors to respond to each point). Refer to specific lines, pages, sections, or figure and table numbers so the authors (and editors) know exactly what you’re talking about.

Major vs. minor issues

What’s the difference between a major and minor issue? Major issues should consist of the essential points the authors need to address before the manuscript can proceed. Make sure you focus on what is  fundamental for the current study . In other words, it’s not helpful to recommend additional work that would be considered the “next step” in the study. Minor issues are still important but typically will not affect the overall conclusions of the manuscript. Here are some examples of what would might go in the “minor” category:

• Missing references (but depending on what is missing, this could also be a major issue)
• Technical clarifications (e.g., the authors should clarify how a reagent works)
• Data presentation (e.g., the authors should present p-values differently)
• Typos, spelling, grammar, and phrasing issues

3. Any other points

Confidential comments for the editors.

Some journals have a space for reviewers to enter confidential comments about the manuscript. Use this space to mention concerns about the submission that you’d want the editors to consider before sharing your feedback with the authors, such as concerns about ethical guidelines or language quality. Any serious issues should be raised directly and immediately with the journal as well.

This section is also where you will disclose any potentially competing interests, and mention whether you’re willing to look at a revised version of the manuscript.

Do not use this space to critique the manuscript, since comments entered here will not be passed along to the authors.  If you’re not sure what should go in the confidential comments, read the reviewer instructions or check with the journal first before submitting your review. If you are reviewing for a journal that does not offer a space for confidential comments, consider writing to the editorial office directly with your concerns.

Get this outline in a template

Giving Feedback

Giving feedback is hard. Giving effective feedback can be even more challenging. Remember that your ultimate goal is to discuss what the authors would need to do in order to qualify for publication. The point is not to nitpick every piece of the manuscript. Your focus should be on providing constructive and critical feedback that the authors can use to improve their study.

If you’ve ever had your own work reviewed, you already know that it’s not always easy to receive feedback. Follow the golden rule: Write the type of review you’d want to receive if you were the author. Even if you decide not to identify yourself in the review, you should write comments that you would be comfortable signing your name to.

In your comments, use phrases like “ the authors’ discussion of X” instead of “ your discussion of X .” This will depersonalize the feedback and keep the focus on the manuscript instead of the authors.

General guidelines for effective feedback

• Justify your recommendation with concrete evidence and specific examples.
• Be specific so the authors know what they need to do to improve.
• Be thorough. This might be the only time you read the manuscript.
• Be professional and respectful. The authors will be reading these comments too.
• Remember to say what you liked about the manuscript!

Don’t

• Recommend additional experiments or  unnecessary elements that are out of scope for the study or for the journal criteria.
• Tell the authors exactly how to revise their manuscript—you don’t need to do their work for them.
• Use the review to promote your own research or hypotheses.
• Focus on typos and grammar. If the manuscript needs significant editing for language and writing quality, just mention this in your comments.
• Submit your review without proofreading it and checking everything one more time.

Before and After: Sample Reviewer Comments

Keeping in mind the guidelines above, how do you put your thoughts into words? Here are some sample “before” and “after” reviewer comments

✗ Before

“The authors appear to have no idea what they are talking about. I don’t think they have read any of the literature on this topic.”

✓ After

“The study fails to address how the findings relate to previous research in this area. The authors should rewrite their Introduction and Discussion to reference the related literature, especially recently published work such as Darwin et al.”

“The writing is so bad, it is practically unreadable. I could barely bring myself to finish it.”

“While the study appears to be sound, the language is unclear, making it difficult to follow. I advise the authors work with a writing coach or copyeditor to improve the flow and readability of the text.”

“It’s obvious that this type of experiment should have been included. I have no idea why the authors didn’t use it. This is a big mistake.”

“The authors are off to a good start, however, this study requires additional experiments, particularly [type of experiment]. Alternatively, the authors should include more information that clarifies and justifies their choice of methods.”

Suggested Language for Tricky Situations

You might find yourself in a situation where you’re not sure how to explain the problem or provide feedback in a constructive and respectful way. Here is some suggested language for common issues you might experience.

What you think : The manuscript is fatally flawed. What you could say: “The study does not appear to be sound” or “the authors have missed something crucial”.

What you think : You don’t completely understand the manuscript. What you could say : “The authors should clarify the following sections to avoid confusion…”

What you think : The technical details don’t make sense. What you could say : “The technical details should be expanded and clarified to ensure that readers understand exactly what the researchers studied.”

What you think: The writing is terrible. What you could say : “The authors should revise the language to improve readability.”

What you think : The authors have over-interpreted the findings. What you could say : “The authors aim to demonstrate [XYZ], however, the data does not fully support this conclusion. Specifically…”

What does a good review look like?

Check out the peer review examples at F1000 Research to see how other reviewers write up their reports and give constructive feedback to authors.

Time to Submit the Review!

Be sure you turn in your report on time. Need an extension? Tell the journal so that they know what to expect. If you need a lot of extra time, the journal might need to contact other reviewers or notify the author about the delay.

Tip: Building a relationship with an editor

You’ll be more likely to be asked to review again if you provide high-quality feedback and if you turn in the review on time. Especially if it’s your first review for a journal, it’s important to show that you are reliable. Prove yourself once and you’ll get asked to review again!

• Getting started as a reviewer
• Responding to an invitation
• Reading a manuscript
• Writing a peer review

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

Peer Reviewed Literature

What is peer review, terminology, peer review what does that mean, what types of articles are peer-reviewed, what information is not peer-reviewed, what about google scholar.

• How do I find peer-reviewed articles?
• Scholarly vs. Popular Sources

Research Librarian

For more help on this topic, please contact our Research Help Desk: [email protected] or 781-768-7303. Stay up-to-date on our current hours . Note: all hours are EST.

This Guide was created by Carolyn Swidrak (retired).

Research findings are communicated in many ways.  One of the most important ways is through publication in scholarly, peer-reviewed journals.

Research published in scholarly journals is held to a high standard.  It must make a credible and significant contribution to the discipline.  To ensure a very high level of quality, articles that are submitted to scholarly journals undergo a process called peer-review.

Once an article has been submitted for publication, it is reviewed by other independent, academic experts (at least two) in the same field as the authors.  These are the peers.  The peers evaluate the research and decide if it is good enough and important enough to publish.  Usually there is a back-and-forth exchange between the reviewers and the authors, including requests for revisions, before an article is published. 

Peer review is a rigorous process but the intensity varies by journal.  Some journals are very prestigious and receive many submissions for publication.  They publish only the very best, most highly regarded research. 

The terms scholarly, academic, peer-reviewed and refereed are sometimes used interchangeably, although there are slight differences.

Scholarly and academic may refer to peer-reviewed articles, but not all scholarly and academic journals are peer-reviewed (although most are.)  For example, the Harvard Business Review is an academic journal but it is editorially reviewed, not peer-reviewed.

Peer-reviewed and refereed are identical terms.

From  Peer Review in 3 Minutes  [Video], by the North Carolina State University Library, 2014, YouTube (https://youtu.be/rOCQZ7QnoN0).

Peer reviewed articles can include:

• Original research (empirical studies)
• Review articles
• Systematic reviews
• Meta-analyses

There is much excellent, credible information in existence that is NOT peer-reviewed.  Peer-review is simply ONE MEASURE of quality. 

Much of this information is referred to as "gray literature."

Government Agencies

Government websites such as the Centers for Disease Control (CDC) publish high level, trustworthy information.  However, most of it is not peer-reviewed.  (Some of their publications are peer-reviewed, however. The journal Emerging Infectious Diseases, published by the CDC is one example.)

Conference Proceedings

Papers from conference proceedings are not usually peer-reviewed.  They may go on to become published articles in a peer-reviewed journal. 

Dissertations

Dissertations are written by doctoral candidates, and while they are academic they are not peer-reviewed.

Many students like Google Scholar because it is easy to use.  While the results from Google Scholar are generally academic they are not necessarily peer-reviewed.  Typically, you will find:

• Peer reviewed journal articles (although they are not identified as peer-reviewed)
• Unpublished scholarly articles (not peer-reviewed)
• Masters theses, doctoral dissertations and other degree publications (not peer-reviewed)
• Book citations and links to some books (not necessarily peer-reviewed)
• Next: How do I find peer-reviewed articles? >>
• Last Updated: Feb 12, 2024 9:39 AM
• URL: https://libguides.regiscollege.edu/peer_review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on 6 May 2022 by Tegan George . Revised on 2 September 2022.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, frequently asked questions about peer review.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Prevent plagiarism, run a free check.

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymised) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymised comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymised) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymised) review – where the identities of the author, reviewers, and editors are all anonymised – does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimises potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymise everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimise back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarise the argument in your own words

Summarising the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organised. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticised, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the ‘compliment sandwich’, where you ‘sandwich’ your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarised or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defence, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

Peer review is a process of evaluating submissions to an academic journal. Utilising rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication.

For this reason, academic journals are often considered among the most credible sources you can use in a research project – provided that the journal itself is trustworthy and well regarded.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field.

It acts as a first defence, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

In general, the peer review process follows the following steps:

• Reject the manuscript and send it back to author, or
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2022, September 02). What Is Peer Review? | Types & Examples. Scribbr. Retrieved 9 April 2024, from https://www.scribbr.co.uk/research-methods/peer-reviews/

Is this article helpful?

Tegan George

Other students also liked, what is a double-blind study | introduction & examples, a quick guide to experimental design | 5 steps & examples, data cleaning | a guide with examples & steps.

Disclaimer » Advertising

• HealthyChildren.org

• Previous Article
• Next Article

What is the Purpose of Peer Review?

What makes a good peer reviewer, how do you decide whether to review a paper, how do you complete a peer review, limitations of peer review, conclusions, research methods: how to perform an effective peer review.

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• CME Quiz Close Quiz
• Open the PDF for in another window
• Get Permissions
• Cite Icon Cite
• Search Site

Elise Peterson Lu , Brett G. Fischer , Melissa A. Plesac , Andrew P.J. Olson; Research Methods: How to Perform an Effective Peer Review. Hosp Pediatr November 2022; 12 (11): e409–e413. https://doi.org/10.1542/hpeds.2022-006764

Download citation file:

• Ris (Zotero)
• Reference Manager

Scientific peer review has existed for centuries and is a cornerstone of the scientific publication process. Because the number of scientific publications has rapidly increased over the past decades, so has the number of peer reviews and peer reviewers. In this paper, drawing on the relevant medical literature and our collective experience as peer reviewers, we provide a user guide to the peer review process, including discussion of the purpose and limitations of peer review, the qualities of a good peer reviewer, and a step-by-step process of how to conduct an effective peer review.

Peer review has been a part of scientific publications since 1665, when the Philosophical Transactions of the Royal Society became the first publication to formalize a system of expert review. 1 , 2   It became an institutionalized part of science in the latter half of the 20 th century and is now the standard in scientific research publications. 3   In 2012, there were more than 28 000 scholarly peer-reviewed journals and more than 3 million peer reviewed articles are now published annually. 3 , 4   However, even with this volume, most peer reviewers learn to review “on the (unpaid) job” and no standard training system exists to ensure quality and consistency. 5   Expectations and format vary between journals and most, but not all, provide basic instructions for reviewers. In this paper, we provide a general introduction to the peer review process and identify common strategies for success as well as pitfalls to avoid.

Modern peer review serves 2 primary purposes: (1) as “a screen before the diffusion of new knowledge” 6   and (2) as a method to improve the quality of published work. 1 , 5  

As screeners, peer reviewers evaluate the quality, validity, relevance, and significance of research before publication to maintain the credibility of the publications they serve and their fields of study. 1 , 2 , 7   Although peer reviewers are not the final decision makers on publication (that role belongs to the editor), their recommendations affect editorial decisions and thoughtful comments influence an article’s fate. 6 , 8  

As advisors and evaluators of manuscripts, reviewers have an opportunity and responsibility to give authors an outside expert’s perspective on their work. 9   They provide feedback that can improve methodology, enhance rigor, improve clarity, and redefine the scope of articles. 5 , 8 , 10   This often happens even if a paper is not ultimately accepted at the reviewer’s journal because peer reviewers’ comments are incorporated into revised drafts that are submitted to another journal. In a 2019 survey of authors, reviewers, and editors, 83% said that peer review helps science communication and 90% of authors reported that peer review improved their last paper. 11  

Expertise: Peer reviewers should be up to date with current literature, practice guidelines, and methodology within their subject area. However, academic rank and seniority do not define expertise and are not actually correlated with performance in peer review. 13  

Professionalism: Reviewers should be reliable and objective, aware of their own biases, and respectful of the confidentiality of the peer review process.

Critical skill : Reviewers should be organized, thorough, and detailed in their critique with the goal of improving the manuscript under their review, regardless of disposition. They should provide constructive comments that are specific and addressable, referencing literature when possible. A peer reviewer should leave a paper better than he or she found it.

Is the manuscript within your area of expertise? Generally, if you are asked to review a paper, it is because an editor felt that you were a qualified expert. In a 2019 survey, 74% of requested reviews were within the reviewer’s area of expertise. 11   This, of course, does not mean that you must be widely published in the area, only that you have enough expertise and comfort with the topic to critique and add to the paper.

Do you have any biases that may affect your review? Are there elements of the methodology, content area, or theory with which you disagree? Some disagreements between authors and reviewers are common, expected, and even helpful. However, if a reviewer fundamentally disagrees with an author’s premise such that he or she cannot be constructive, the review invitation should be declined.

Do you have the time? The average review for a clinical journal takes 5 to 6 hours, though many take longer depending on the complexity of the research and the experience of the reviewer. 1 , 14   Journals vary on the requested timeline for return of reviews, though it is usually 1 to 4 weeks. Peer review is often the longest part of the publication process and delays contribute to slower dissemination of important work and decreased author satisfaction. 15   Be mindful of your schedule and only accept a review invitation if you can reasonably return the review in the requested time.

Once you have determined that you are the right person and decided to take on the review, reply to the inviting e-mail or click the associated link to accept (or decline) the invitation. Journal editors invite a limited number of reviewers at a time and wait for responses before inviting others. A common complaint among journal editors surveyed was that reviewers would often take days to weeks to respond to requests, or not respond at all, making it difficult to find appropriate reviewers and prolonging an already long process. 5  

Now that you have decided to take on the review, it is best of have a systematic way of both evaluating the manuscript and writing the review. Various suggestions exist in the literature, but we will describe our standard procedure for review, incorporating specific do’s and don’ts summarized in Table 1 .

Dos and Don’ts of Peer Review

First, read the manuscript once without making notes or forming opinions to get a sense of the paper as whole. Assess the overall tone and flow and define what the authors identify as the main point of their work. Does the work overall make sense? Do the authors tell the story effectively?

Next, read the manuscript again with an eye toward review, taking notes and formulating thoughts on strengths and weaknesses. Consider the methodology and identify the specific type of research described. Refer to the corresponding reporting guideline if applicable (CONSORT for randomized control trials, STROBE for observational studies, PRISMA for systematic reviews). Reporting guidelines often include a checklist, flow diagram, or structured text giving a minimum list of information needed in a manuscript based on the type of research done. 16   This allows the reviewer to formulate a more nuanced and specific assessment of the manuscript.

Next, review the main findings, the significance of the work, and what contribution it makes to the field. Examine the presentation and flow of the manuscript but do not copy edit the text. At this point, you should start to write your review. Some journals provide a format for their reviews, but often it is up to the reviewer. In surveys of journal editors and reviewers, a review organized by manuscript section was the most favored, 5 , 6   so that is what we will describe here.

As you write your review, consider starting with a brief summary of the work that identifies the main topic, explains the basic approach, and describes the findings and conclusions. 12 , 17   Though not universally included in all reviews, we have found this step to be helpful in ensuring that the work is conveyed clearly enough for the reviewer to summarize it. Include brief notes on the significance of the work and what it adds to current knowledge. Critique the presentation of the work: is it clearly written? Is its length appropriate? List any major concerns with the work overall, such as major methodological flaws or inaccurate conclusions that should disqualify it from publication, though do not comment directly on disposition. Then perform your review by section:

Abstract : Is it consistent with the rest of the paper? Does it adequately describe the major points?

Introduction : This section should provide adequate background to explain the need for the study. Generally, classic or highly relevant studies should be cited, but citations do not have to be exhaustive. The research question and hypothesis should be clearly stated.

Methods: Evaluate both the methods themselves and the way in which they are explained. Does the methodology used meet the needs of the questions proposed? Is there sufficient detail to explain what the authors did and, if not, what needs to be added? For clinical research, examine the inclusion/exclusion criteria, control populations, and possible sources of bias. Reporting guidelines can be particularly helpful in determining the appropriateness of the methods and how they are reported.

Some journals will expect an evaluation of the statistics used, whereas others will have a separate statistician evaluate, and the reviewers are generally not expected to have an exhaustive knowledge of statistical methods. Clarify expectations if needed and, if you do not feel qualified to evaluate the statistics, make this clear in your review.

Results: Evaluate the presentation of the results. Is information given in sufficient detail to assess credibility? Are the results consistent with the methodology reported? Are the figures and tables consistent with the text, easy to interpret, and relevant to the work? Make note of data that could be better detailed in figures or tables, rather than included in the text. Make note of inappropriate interpretation in the results section (this should be in discussion) or rehashing of methods.

Discussion: Evaluate the authors’ interpretation of their results, how they address limitations, and the implications of their work. How does the work contribute to the field, and do the authors adequately describe those contributions? Make note of overinterpretation or conclusions not supported by the data.

The length of your review often correlates with your opinion of the quality of the work. If an article has major flaws that you think preclude publication, write a brief review that focuses on the big picture. Articles that may not be accepted but still represent quality work merit longer reviews aimed at helping the author improve the work for resubmission elsewhere.

Generally, do not include your recommendation on disposition in the body of the review itself. Acceptance or rejection is ultimately determined by the editor and including your recommendation in your comments to the authors can be confusing. A journal editor’s decision on acceptance or rejection may depend on more factors than just the quality of the work, including the subject area, journal priorities, other contemporaneous submissions, and page constraints.

Many submission sites include a separate question asking whether to accept, accept with major revision, or reject. If this specific format is not included, then add your recommendation in the “confidential notes to the editor.” Your recommendation should be consistent with the content of your review: don’t give a glowing review but recommend rejection or harshly criticize a manuscript but recommend publication. Last, regardless of your ultimate recommendation on disposition, it is imperative to use respectful and professional language and tone in your written review.

Although peer review is often described as the “gatekeeper” of science and characterized as a quality control measure, peer review is not ideally designed to detect fundamental errors, plagiarism, or fraud. In multiple studies, peer reviewers detected only 20% to 33% of intentionally inserted errors in scientific manuscripts. 18 , 19   Plagiarism similarly is not detected in peer review, largely because of the huge volume of literature available to plagiarize. Most journals now use computer software to identify plagiarism before a manuscript goes to peer review. Finally, outright fraud often goes undetected in peer review. Reviewers start from a position of respect for the authors and trust the data they are given barring obvious inconsistencies. Ultimately, reviewers are “gatekeepers, not detectives.” 7  

Peer review is also limited by bias. Even with the best of intentions, reviewers bring biases including but not limited to prestige bias, affiliation bias, nationality bias, language bias, gender bias, content bias, confirmation bias, bias against interdisciplinary research, publication bias, conservatism, and bias of conflict of interest. 3 , 4 , 6   For example, peer reviewers score methodology higher and are more likely to recommend publication when prestigious author names or institutions are visible. 20   Although bias can be mitigated both by the reviewer and by the journal, it cannot be eliminated. Reviewers should be mindful of their own biases while performing reviews and work to actively mitigate them. For example, if English language editing is necessary, state this with specific examples rather than suggesting the authors seek editing by a “native English speaker.”

Peer review is an essential, though imperfect, part of the forward movement of science. Peer review can function as both a gatekeeper to protect the published record of science and a mechanism to improve research at the level of individual manuscripts. Here, we have described our strategy, summarized in Table 2 , for performing a thorough peer review, with a focus on organization, objectivity, and constructiveness. By using a systematized strategy to evaluate manuscripts and an organized format for writing reviews, you can provide a relatively objective perspective in editorial decision-making. By providing specific and constructive feedback to authors, you contribute to the quality of the published literature.

Take-home Points

FUNDING: No external funding.

CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest to disclose.

Dr Lu performed the literature review and wrote the manuscript. Dr Fischer assisted in the literature review and reviewed and edited the manuscript. Dr Plesac provided background information on the process of peer review, reviewed and edited the manuscript, and completed revisions. Dr Olson provided background information and practical advice, critically reviewed and revised the manuscript, and approved the final manuscript.

Advertising Disclaimer »

Citing articles via

Email alerts.

Affiliations

• Editorial Board
• Editorial Policies
• Pediatrics On Call
• Online ISSN 2154-1671
• Print ISSN 2154-1663
• Pediatrics Open Science
• Hospital Pediatrics
• Pediatrics in Review
• AAP Grand Rounds
• Latest News
• Pediatric Care Online
• Red Book Online
• Pediatric Patient Education
• AAP Toolkits
• AAP Pediatric Coding Newsletter

First 1,000 Days Knowledge Center

Institutions/librarians, group practices, licensing/permissions, integrations, advertising.

• Privacy Statement | Accessibility Statement | Terms of Use | Support Center | Contact Us
• © Copyright American Academy of Pediatrics

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Explore millions of high-quality primary sources and images from around the world, including artworks, maps, photographs, and more.

Explore migration issues through a variety of media types

• Part of The Streets are Talking: Public Forms of Creative Expression from Around the World
• Part of The Journal of Economic Perspectives, Vol. 34, No. 1 (Winter 2020)
• Part of Cato Institute (Aug. 3, 2021)
• Part of University of California Press
• Part of Open: Smithsonian National Museum of African American History & Culture
• Part of Indiana Journal of Global Legal Studies, Vol. 19, No. 1 (Winter 2012)
• Part of R Street Institute (Nov. 1, 2020)
• Part of Leuven University Press
• Part of UN Secretary-General Papers: Ban Ki-moon (2007-2016)
• Part of Perspectives on Terrorism, Vol. 12, No. 4 (August 2018)
• Part of Leveraging Lives: Serbia and Illegal Tunisian Migration to Europe, Carnegie Endowment for International Peace (Mar. 1, 2023)
• Part of UCL Press

Harness the power of visual materials—explore more than 3 million images now on JSTOR.

Enhance your scholarly research with underground newspapers, magazines, and journals.

Explore collections in the arts, sciences, and literature from the world’s leading museums, archives, and scholars.

• Follow us on Facebook
• Follow us on Twitter
• Criminal Justice
• Environment
• Politics & Government
• Race & Gender

Expert Commentary

What’s peer review? 5 things you should know before covering research

Is peer-reviewed research really superior? Why should journalists note in their stories whether studies have been peer reviewed? We explain.

Republish this article

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License .

by Denise-Marie Ordway, The Journalist's Resource May 8, 2021

This <a target="_blank" href="https://journalistsresource.org/media/peer-review-research-journalists/">article</a> first appeared on <a target="_blank" href="https://journalistsresource.org">The Journalist's Resource</a> and is republished here under a Creative Commons license.<img src="https://journalistsresource.org/wp-content/uploads/2020/11/cropped-jr-favicon-150x150.png" style="width:1em;height:1em;margin-left:10px;">

As scholars and other experts rush to release new research aimed at better understanding the coronavirus pandemic, newsrooms must be more careful than ever in vetting the biomedical studies they choose to cover. One of the first steps journalists should take to gauge the quality of all types of research is answering this important question: Has the paper undergone peer review?

Peer review is a formal process through which researchers evaluate and provide feedback on one another’s work, ideally filtering out flawed and low-quality studies while strengthening others. Academic journals generally do not publish papers that have not survived the process. Researchers often share studies that have not been peer reviewed — usually referred to as working papers or preprints — by posting them to online servers and repositories.

It’s worth noting the world’s largest preprint servers for life sciences — bioRxiv — and health sciences — medRxiv — screen papers for plagiarism and content that is offensive, non-scientific or might pose a health or biosecurity risk. But there are preprint servers in other fields that do not apply the same level of scrutiny.

While peer review is intended for quality control, it is imperfect. For example, reviewers, who often are college faculty with expertise in the same field as the work they are examining, sometimes fail to detect fraud, data discrepancies and other problems. Even some of the most prestigious journals with the most rigorous peer-review processes have had to retract articles. Retractions are rare, however.

 “Only about four of every 10,000 papers are now retracted. And although the rate roughly doubled from 2003 to 2009, it has remained level since 2012,” Science magazine reported in 2018 .

As of early May 2021, a total of 108 papers about COVID-19 , the bulk of which appeared in journals, had been withdrawn, according to Retraction Watch , which maintains an online database of research retractions going back decades.

Despite its flaws, researchers, overall, seem confident in peer review. During a 2019 survey of more than 3,000 researchers across disciplines in multiple countries, 85% agreed or strongly agreed that without peer review, there is no control in scientific communication. The survey — conducted by Elsevier , one of the world’s largest journal publishers, and Sense about Science , a London-based nonprofit promoting public interest in science and evidence — also finds 90% of participating researchers agreed or strongly agreed that peer review improves the quality of research.

Several published studies present similar findings. A 2017 paper in Learned Publishing indicates early career researchers are “generally supportive of peer review” but complain the process is time-consuming and that reviewers, who typically work on a volunteer basis, should be rewarded with some sort of professional acknowledgement or payment.

Regardless of the type of research journalists cover, they should have at least a basic understanding of the peer-review process and its benefits and shortcomings.

Below, we explain some of the most important aspects with help from several experts, including Diane Sullenberger , executive editor of Proceedings of the National Academy of Sciences ; Miriam Lewis Sabin , a senior editor at The Lancet ; and John Inglis , executive director of Cold Spring Harbor Laboratory Press and co-founder of bioRxiv and medRxiv.

1. Peer reviewers are not fraud detectors. They also do not verify the accuracy of a research study.

The peer-review process is meant to validate research, not verify it. Reviewers typically do not authenticate the study’s data or make sure its authors actually followed the procedures they say they followed to reach their conclusions. Reviewers, sometimes called referees, also do not determine whether findings are correct, given the data and other evidence used to reach them.

Reviewers do examine academic papers to answer a range of relevant questions. They look at whether the research questions are clear, for example, and whether the study’s design, sampling methods and analysis are appropriate for answering those questions. They also assess whether the paper answers such questions as:

• Is the study explained clearly enough and in enough detail that another researcher could replicate it?
• How does the study challenge or add to the body of knowledge on this topic?
• Does it fit the standards and scope of the journal to which it was submitted?
• If the study involves humans or animals, did the authors acquire the required approvals and meet ethical standards?
• Does it give proper attribution to earlier research?

When German theologian Henry Oldenburg created the first journal dedicated to science in 1665, he considered the key functions of a research journal to be registration, certification, dissemination and archiving, writes Robert Campbell , a senior publisher at Wiley-Blackwell Publishing, in the book Academic and Professional Publishing .

Peer review is considered the gold standard for assessing research content, Sullenberger explained in an email interview. But journalists must understand it is not infallible, she added.

“Science is self-correcting through replication and reproducibility, and research fraud can be difficult to detect in peer review,” she wrote.

2. Journalists can help the public recognize the value of peer review by noting whether the studies they cover have been peer reviewed.

Scholars, research organizations and others regularly criticize news outlets for failing to explain whether new research they report on or the older studies they incorporate into their stories have undergone peer review. It’s important that journalists differentiate between peer-reviewed research and preprint papers, which often present preliminary findings.

Sullenberger told JR : “Greater clarity when journalists cover unreviewed preprints is needed; they should not be reported as having the same validity and authority as peer-reviewed research papers. “

A recent study in the journal Health Communication finds that many of the news articles written about COVID-related preprints during the first four months of 2020 did not indicate the scientific uncertainty of that research. About 43% of the stories analyzed did not mention the research was a preprint, unreviewed, preliminary or in need of verification.

At the time of that study, however, many of the journalists drawn into reporting the frenzy of stories about the pandemic were unfamiliar with preprints, Inglis says. Today, he adds, journalists covering the coronavirus are much more likely to include phrases such as “not yet peer reviewed” to describe preprints.

Sense About Science urges the public to pay attention to whether a study being discussed in a government meeting or in the media has been peer reviewed. “The more we ask, ‘is it peer reviewed?’ the more obliged reporters will be to include this information,” the organization asserts in a leaflet it created to help the public scrutinize the scientific information featured in news stories.

Knowing whether research has been peer reviewed helps a person judge how much weight to give the claims being made by its authors, Tracey Brown , the managing director of Sense About Science, explained during an interview with The Scholarly Kitchen blog.

“We have to establish an understanding that the status of research findings is as important as the findings themselves,” Brown says in a prepared statement . “This understanding has the capacity to improve the decisions we make across all of society.”

3. Peer reviewers help decide a study’s fate.

Journal editors typically assign two or more reviewers to each research paper. Some also employ a statistical specialist.

While the selection process differs, journals choose reviewers based on factors such as expertise, reputation and the journal’s prior experience with the reviewer. While it can be difficult to recruit scientists willing to examine manuscripts because of the time required for proper scrutiny, many do it because of “a sense of duty to help advance their disciplines, as well as the need for reciprocity, knowing other researchers volunteer to peer review their manuscript submissions,” Science magazine reported earlier this year .

Reviewers can make recommendations about whether a journal should accept, reject or send a paper back for minor or major revisions. Reviewers usually submit reports offering their overall impressions of a paper and suggestions for improvements. Most often, though, the final decision lies with one or more of the journal’s editors or its editorial board.

Inglis, a former assistant editor of The Lancet who is now a publisher of five peer-reviewed journals, says a common criticism of the peer-review process is its lengthy timeline, which can span from weeks to a year or more. Another complaint: Sometimes, journals send a study back and notify the authors that they would be willing to accept or reconsider the paper for publication if the authors do more research.

“Sometimes, the demands made are completely unrealistic,” Inglis adds. “The criticism from the authors is that editors don’t know that when they say ‘Do this additional experiment,’ that’s another year [added to the timeline]. Meanwhile, the work is perfectly valid.”

Inglis says bioRxiv (pronounced “bio-archive”) and medRxiv (pronounced “med-archive”) were created so researchers could disseminate preliminary versions of their papers, allowing the scientific community to immediately use and start building on those findings and data.

4. The peer-review process varies significantly among academic journals.

There are several kinds of peer review, and journals often state on their websites which one they use. The most common are single-blinded peer review, which allows reviewers to know the authors’ identities while reviewers’ identities remain anonymous, and double-blinded peer review, in which authors and reviewers are unaware of each other’s identities.

Both have advantages. Advocates argue anonymity protects reviewers from retribution. It also helps shield authors from biases based on factors such as gender, nationality, language and affiliations with less prestigious institutions, Tony Ross-Hellauer , a postdoctoral researcher at the Know-Center in Austria, writes in “ What is Open Peer Review? A Systematic Review ,” published on the European open access platform F1000Research in 2017.

Keeping identities secret can create problems, however.

“At the editorial level, lack of transparency means that editors can unilaterally reject submissions or shape review outcomes by selecting reviewers based on their known preference for or aversion to certain theories and methods,” Ross-Hellauer writes. He adds that reviewers, “shielded by anonymity, may act unethically in their own interests by concealing conflicts of interest.”

A newer type of peer review, called open peer review , is not as prevalent. But the scientific community has ongoing discussions about whether its greater transparency might help improve research quality.

While there is no universally accepted definition of open peer review, also known as open identity peer review, the identities of both authors and reviewers typically are made known to each other. Ross-Hellauer notes that disclosing reviewers’ names may force them “to think more carefully about the scientific issues and to write more thoughtful reviews.”

A growing number of journals are posting not just the papers they accept but also the feedback peer reviewers gave the papers’ authors.

5. Peer review continues to evolve.

Some journals have started initiating peer review after a paper is published instead of beforehand, although this still is not common. MedEdPublish , an online scholarly journal, is one of those that employ post-publication peer review . Its papers undergo peer review on the website by members of the medical education community, which could include the journal’s editor, members of its editorial board or a panel of reviewers.

Under the MedEdPublish model, a paper has undergone formal peer review after at least two members of the journal’s review panel evaluate it. The paper can be critiqued and improved over time as a living document on the journal website.

“Post-publication peer review follows an open and transparent process, which aims to avoid editorial bias while increasing the speed of publication,” according to the website. “We use an ‘open identities’ principle, whereby all reviewers submit their feedback publicly, under their own name, and everyone visiting an article page can see all peer review reports, referee names, and comments, and can join the discussion if they wish.”

Another noteworthy shift: Some journals are working to diversify their pools of reviewers by ensuring women, racial and ethnic minorities, and scientists from other countries help appraise and select studies for publication.

Research indicates the overwhelming majority of experts chosen as reviewers are men. A study published earlier this year in Science Advances examines internal data for 145 scholarly journals across fields and finds that women comprised 21% of their reviewers between 2010 and 2016. At journals dedicated to biomedical and health research, 24.6% of reviewers were women.

The Lancet medical journal has set targets for increasing the number of women and scientists from low- and middle-income countries, Sabin, one of its senior editors, wrote in an email interview with JR . In 2019, The Lancet family of journals announced its Diversity Pledge .

“We track, monitor, and report representation of authors, reviewers, and editorial advisors by gender and across geography,” Sabin told JR in an e-mail.

She added that the journal formed a task force late last year to, among other things, examine its policies and processes to find ways to increase the representation of experts who are racial and ethnic minorities.

The Coalition for Diversity and Inclusion in Scholarly Communications has focused on the issue globally. More than 90 organizations have adopted the coalition’s Joint Statement of Principles , which aims to “promote involvement, innovation, and expanded access to leadership opportunities that maximize engagement across identity groups and professional levels.”

Identity groups include racial and sexual minorities, military veterans, pregnant women, parents and people from lower social classes and socioeconomic backgrounds.

The Journalist’s Resource would like to thank Rick Weiss, the director of SciLine, and Meredith Drosback, SciLine’s associate director of science, for their help in creating this tip sheet.

About The Author

Denise-Marie Ordway

MIT Global Change

Search form, peer-reviewed research.

Journal articles and book chapters.

Inspired by our mission?

You can help to sustain our research by donating through Giving to MIT .

Essential Agriculture Scholarly Databases

Other useful scholarly databases.

• Find Extension Program Resources
• Find Newspapers, Trade Publications, Datasets, Government Reports, and More
• Find Images for your Presentations
• Citation Management Applications
• Citing Images in Your Presentations
• CSE Citation Help

Ask a Question

You can find scholarly research by searching a large multidisciplinary databases or by a searching smaller, agricultural database.

There are advantages to each database. If I'm really trying to be thorough, I will utilize several databases. 

Note: There may be significant, but not complete, overlap in coverage within these databases.

Best Databases for Finding Agriculture and Food Science Research

The Washington State University Library Catalog (Search It)   locates pretty much everything you'd hope to find on a subject. This includes journal articles, books, newspapers, multimedia, extension resources, archival materials, etc. 

Web of Science   is a very powerful, mostly STEM, database. It is great for finding scholarly research. There is also a lot of useful advanced functionality for those who want to dive deeper.

AGRICOLA  is an agricultural sciences scholarly database maintained by the U.S. Department of Agriculture's National Agricultural Library (NAL). NAL has also released a new open-source database called SEARCH which searches across NAL resources. 

• PubAg This link opens in a new window PubAg is a portal to USDA-authored and other highly relevant agricultural research. Launched in January 2015, it delivers over 320,499 full-text journal articles on the agricultural sciences, in addition to over 3,496,900 citations
• PubMed This link to PubMed is for those affiliated with WSU. PubMed comprises more than 30 million citations for biomedical literature from MEDLINE, life science journals and online books. Citations may include links to full-text content from PubMed Central and publisher web sites.
• SciFindern Scholar This service works best with Internet Explorer. First time users MUST register. Full text access to: Chemical Abstracts since 1907, Chemical Abstracts Registry CASREACT: since 1974, Medline since 1985, Chemcats and ChemList since 1979. To learn how to get the most out of CAS SciFinderN, see the SciFinder FAQ and collection of on-demand training videos, webinars, and how-to guides .
• << Previous: Welcome
• Next: Find Extension Program Resources >>
• Last Updated: Apr 11, 2024 2:13 PM
• URL: https://libguides.libraries.wsu.edu/agri587

• Open access
• Published: 10 April 2024

Understanding social needs screening and demographic data collection in primary care practices serving Maryland Medicare patients

• Claire M. Starling 1 ,
• Marjanna Smith 1 ,
• Sadaf Kazi 2 , 3 ,
• Arianna Milicia 3 ,
• Rachel Grisham 4 ,
• Emily Gruber 4 ,
• Joseph Blumenthal 5 &
• Hannah Arem 1 , 6  

BMC Health Services Research volume  24 , Article number:  448 ( 2024 ) Cite this article

Metrics details

Health outcomes are strongly impacted by social determinants of health, including social risk factors and patient demographics, due to structural inequities and discrimination. Primary care is viewed as a potential medical setting to assess and address individual health-related social needs and to collect detailed patient demographics to assess and advance health equity, but limited literature evaluates such processes.

We conducted an analysis of cross-sectional survey data collected from n  = 507 Maryland Primary Care Program (MDPCP) practices through Care Transformation Requirements (CTR) reporting in 2022. Descriptive statistics were used to summarize practice responses on social needs screening and demographic data collection. A stepwise regression analysis was conducted to determine factors predicting screening of all vs. a targeted subset of beneficiaries for unmet social needs.

Almost all practices (99%) reported conducting some form of social needs screening and demographic data collection. Practices reported variation in what screening tools or demographic questions were employed, frequency of screening, and how information was used. More than 75% of practices reported prioritizing transportation, food insecurity, housing instability, financial resource strain, and social isolation.

Conclusions

Within the MDPCP program there was widespread implementation of social needs screenings and demographic data collection. However, there was room for additional supports in addressing some challenging social needs and increasing detailed demographics. Further research is needed to understand any adjustments to clinical care in response to identified social needs or application of data for uses such as assessing progress towards health equity and the subsequent impact on clinical care and health outcomes .

Peer Review reports

There is increasing attention on the impact of factors such as economic stability, education, neighborhood, and built environment on healthcare outcomes and, in particular, how primary care settings can assess and address individual level health-related social needs (HRSN) [ 1 , 2 ]. In turn, the American Academy of Pediatrics (AAP) and the American Academy of Family Physicians (AAFP) both recommend that primary care providers screen and address social needs as part of routine primary care visits [ 3 ]. Patients with unmet social needs are at a higher risk of missing appointments, frequent emergency room visits, and hospitalization and rehospitalization [ 4 , 5 ]. Identifying social needs and collecting detailed patient demographics in primary care can be used to tailor care, allocate resources effectively, and advocate for equitable policies, making these workflows a critical step towards advancing health equity [ 1 , 2 , 3 ].

Despite acknowledgement of the importance of integrating social care in clinical settings including a recent mandate by the Centers for Medicare and Medicaid services for screening in inpatient settings, the implementation of social needs screening and demographic data collection is complex and resource intensive [ 6 , 7 ]. Furthermore, patients who screen positive for social needs may decline assistance to address those needs. These occurrences may prove frustrating to those conducting screening if they lack sufficient training on delivering screening or assisting individuals with addressing social needs [ 8 ]. Additionally, while many practices already collect basic demographic data such as age, ethnicity, and race, demographic information is not always collected in a culturally sensitive or inclusive manner. Demographic data collection processes are not standardized, and many demographic fields (e.g., education level, sexual orientation, and disability status) are sometimes not asked at all. As part of a contract to provide technical assistance to Maryland Primary Care Program (MDPCP) practices to support social needs screening and demographic data collection, we explored collected survey data to understand current practices around social needs screening and demographic data collection as well as potential areas for growth in screening delivery.

Study population

MDPCP is a voluntary program for eligible primary care practices that provides funding and support for the delivery of advanced primary care for Medicare beneficiaries throughout Maryland. MDPCP supports the overall health care transformation process and allows primary care providers to play an increased role in disease prevention, management of chronic disease, and prevention of unnecessary hospital utilization [ 9 ]. The primary goal of MDPCP is the sustainable transformation of primary care across Maryland to include all the elements of advanced primary care to support the health needs of state residents [ 9 ]. MDPCP is co-administered by teams at the Maryland Department of Health and the Center for Medicare and Medicaid Innovation (CMMI). At the time of the survey, the MDPCP network included n  = 507 participating primary care practices representative of every county in Maryland.

MDPCP offers a combination of financial incentives and other supports tailored to primary care practices. These incentives encompass non-visit-based payments specifically designed for care coordination initiatives, as well as performance-based incentives, rewarding practices for achieving clinical quality, patient experience, and utilization benchmarks. In addition to financial incentives, MDPCP provides a variety of additional supports for care transformation MDPCP practices are paired with a Practice Transformation Coaches, who provide guidance, answer questions, and work directly with practices to improve processes that improve quality of care and decrease costs. In addition to Coaches, practices have access to the MDPCP Learning System encompassing a myriad of learning opportunities including User Groups, All-Practice Calls, and other collaborative forums for practices to learn from subject matter experts and fellow participants. Practices also have access to a handful of Guides including the Advancing Primary Care Guide, which provides information on MDPCP requirements, tactics for advancing the functions of primary care, and achieving care transformation. Additionally, practices have the option to partner with a Care Transformation Organization (CTO), who can assist with care management or other related patient services.

Data collection

Care transformation requirement (CTR) reporting questions ask MDPCP participants about progress on specific MDPCP requirements that span the five comprehensive primary care functions (Appendix 1 ). The five key functions of advanced primary care are care management, access and continuity, comprehensiveness, and coordination across the continuum of care, beneficiary and caregiver experience, and planned care for health outcomes. The questionnaire is developed by CMMI, and MDPCP participants respond in the online Centers for Medicare and Medicaid Services (CMS) program portal twice annually, as a requirement of program participation (Appendix 2 ). The survey used in this analysis was collected in the third quarter of 2022. This analysis was deemed exempt by the Georgetown/MedStar Institutional Review Board (Study 4698).

Statistical analysis

We used descriptive statistics to review social needs screening and demographic data collection responses from MDPCP practices . We conducted additional analysis to investigate responses by practice characteristics including practice size (small 1–2, medium 3–7, large 8 + providers) and hospital affiliation (yes or no). Further, a stepwise regression analysis was used to determine factors predicting the routine screening of beneficiaries for unmet social needs, comparing all beneficiaries to a specific targeted subsection. Variables used in the model were practice size, and hospital affiliation. 487 of the 507 records were used for regression analyses. We excluded practices if they did not report screening beneficiaries ( n  = 4), practice size ( n  = 1), or hospital affiliation status ( n  = 15). SAS 9.4 (Cary, NC) was used in all analyses.

Practice responses on social needs screening and referral processes are presented in Table  1 . Among the MDPCP practices, nearly all reported some form of social needs screening for all (63%) or at least some (36%) beneficiaries. Many practices reported utilizing a social needs screening tool developed by the practice or affiliated health system (32%). Other practices reported screening using standardized screening tools, including, an unspecified standardized tool (21%); EHR-specific tool (19%); Accountable Health Communities (14%); and PRAPARE (5%). There was substantial variation in EHR vendors, with 23% of practices using EPIC, 17% using eClinicalWorks, 14% using Cerner, and 11% using Athenahealth. Approximately half (49.5%) of the practices reported conducting social needs screening annually, while 18% of practices reported conducting screenings at every visit and 15% when indicated based on reason for visit. Just over a quarter (27%) of practices reported linking responses to discrete ICD-10 or Social Determinants of Health (SDOH) Z codes.

Survey responses revealed variability regarding which patients receive social needs screening, screening frequency, EHR integration and use of Z-codes based on practice characteristics (Appendix 3 ). In an exploratory multivariate logistic regression we found that practices with a hospital affiliation were more likely to screen a targeted population than all patients (OR = 1.54, 95% CI = 1.05–2.27) and practices that were small- (1–2 providers) or medium-sized (3–7 providers) were more likely to screen all patients. (OR = 0.46, 95% CI = 0.26–0.80; OR = 0.46, 95% CI = 0.27–0.78, respectively; data shown in text only). Practices had the opportunity to describe which beneficiaries were targeted. Responses included individuals at high risk ( n  = 67) or experiencing recent mental or clinical health events ( n  = 18), participants in care management or care coordination programs ( n  = 82), Health Equity Advancement Resource and Transformation (HEART) patients ( n  = 25), and attendees of annual wellness visits ( n  = 40).

When practices were asked to select social needs that they prioritize, common responses were transportation (93%), food insecurity (89%), housing instability (86%), financial resource strain (85%), and social isolation (84%) (Table  2 ). The least common needs prioritized were internet access (42%), phone access (46%), employment (48%), and language access (51%). Practices also reported which social needs were most challenging to support. The greatest challenges came with addressing housing instability (31%), internet access (31%), financial resource strain (30%), and medication affordability (30%).

Nearly all practices reported collecting patient demographics in some capacity (99%), with most practices reporting that demographic data are collected by a staff member (70%), collected at every visit (51%), annually (23%), or only at the patient’s initial visit (20%). Race and primary language were collected by nearly all practices (96%), gender identity was collected by 92%, relationship status by 87%, ethnicity by 87%, and employment status by 84% of practices. Other demographic factors were less commonly asked: only 49% of practices reported asking about sexual orientation, 48% asked about disability status, and 38% asked about highest level of education.

In this study we found that primary care practices participating in the MDPCP program overall had a high rate of social risk factor screening, with many using screeners that had been developed to meet individual practice needs. Commonly prioritized domains included transportation, food insecurity, housing instability, financial strain, and social isolation, the last being a commonly cited problem among older adults. Describing patterns of screening and demographics in this sample of practices across the state increase understanding of successes and challenges in real-world practice settings and informs potential future interventions.

Determining which patients should be screened and by whom in a busy primary care setting, as well as who can respond to identified needs, can be challenging. In our study there were differences both in which patients were screened and how often by practice [ 10 , 11 ]. Open ended responses suggested that among some MDPCP practices, screening was performed only for individuals who qualify for extra social assistance through the MDPCP program (i.e., those who qualify due to medical complexity and area deprivation index). Although we did not find other published literature focused specifically on Medicare patients at the state level, we found literature on programs focused on social needs screening among Medicaid populations in several states. Like Maryland practices, standardized measures and consistent approaches to measuring social needs have not been adopted or required in many states [ 12 , 13 , 14 ]. Further, a high percentage of the Maryland practices reported using home grown and standardized screening tools with additional questions to meet the practices’ needs. While the ability to aggregate social needs data across care settings can be challenging with different screeners, there is national movement to harmonize domains across various social risk factor screeners through the Gravity Project and the Office of the National Coordinator [ 12 , 15 ]. Notably, CMS has mandated social needs reporting in the inpatient setting beginning January 2024 for five specific domains, but has not specified a single tool or set of tools given that while there are some validated subsets of questions (e.g., Hunger Vital Signs), there is currently no gold standard tool [ 16 ]. Potential hurdles in requiring specific tools may include limitations on EHR technology, referral processes, and provider or staff level comfort and training in asking specific questions. Furthermore, implementing screening without supports for training the staff on trauma-informed approaches and how to respond to identified needs has the potential to cause more harm than benefit to patients. Thus, toolkits established by various professional societies and public health societies may be useful to determine which tools are most appropriate for a given practice and how to integrate them into care where practices have not yet started screening or encounter challenges [ 17 , 18 , 19 ].

Regarding practices with a hospital affiliation being more likely to screen a targeted population, one possibility is that practices affiliated with hospitals may have access to additional resources and supports that facilitate targeted screening efforts. Hospitals often have established practices including social risk factor screening for targeted subpopulations to address costly hospital readmissions, which may encourage affiliated practices to deliver more targeted screening practices. While it is unclear why small or medium-sized practices were more likely to screen all patients than a sub-population, it may have to do with more autonomy in workflow process, less customization of the EHR to target sub-populations, or differences in staffing and provider to patient ratios. While we cannot explain these differences from the survey alone, findings suggest that the size and affiliation of practices play a role in their screening practices, highlighting the importance of considering practice characteristics when designing specific supportive interventions or policies aimed at increasing screening rates.

It is important to highlight that MDPCP practices have achieved impressive levels of social needs screening and demographic data collection implementation. This success could be attributed largely to the program’s requirements and incentives to screen beneficiaries for social needs and collect demographic information. Additionally, the program provides technical support and resources to meet these requirements and to stand up social needs screening workflows if not already in place. By joining MDPCP, participating practices have demonstrated a commitment to advanced primary care, further indicating MDPCP participation may be associated with higher uptake of these workflows, as opposed to primary care practices who do not participate in similar value-based programs. Other states considering such programs may look to some of these supports when rolling out new requirements or incentives.

While the findings highlight the high level of social needs screening and demographic data collection, challenges in addressing identified needs may also be due to various factors including complexity of workflows and staffing, patients with social needs declining assistance, or limited local resource availability [ 20 ]. Previous research suggests patients may decline social needs assistance in healthcare settings if they do not feel like they need help, are confused about what is offered, are not confident that the assistance would be helpful, have experienced previous negative experiences, or feel fear and mistrust related to disclosing personal information [ 8 ]. In areas that posed the greatest referral challenges, policy efforts may be needed to deliver services and bridge the gaps to access. For example, the challenge of addressing housing needs is not newly identified; previous literature has shown increasing costs and declining supply have contributed to national housing availability and affordability challenges [ 21 , 22 ]. Medication cost continues to be a major problem cited in the literature, especially for older populations with a higher incidence of chronic diseases [ 23 , 24 ]. Financial strain among individuals often poses a challenge as financial needs fluctuate frequently, and changes can be dramatic; further, these changing needs over time are often not resolved by a one-time intervention and require long-term involvements [ 11 ]. Though research on the effects of internet access and health outcomes is still emerging, literature suggests investment in digital infrastructure by federal, state, and local governments is needed for further development of the internet as a means of addressing long-standing inequality in health [ 25 , 26 ]. While food insecurity and transportation were top needs prioritized within MDPCP practices, they did not present the same level of challenge to practices, perhaps due to wider availability of resources, partnerships, and supports such as transportation vouchers.

Although addressing connection to resources continues to be a challenge for practices, there are opportunities to leverage information from social needs screenings and demographic data collection in several other ways to improve care. Aggregate screening and demographic data can be used for quality improvement initiatives within primary care practices by analyzing trends and patterns in social needs data to help practices identify areas of unmet need, track outcomes, and update protocols for screening and referral processes. Additionally, data can be used to advocate for policy changes to address systemic issues affecting patients’ health outcomes. However, challenges in utilizing information from social needs screening and demographic data collection may still exist due to limited resources and capacity and lack of provider awareness and training availability.

Increased collection of detailed demographic data, particularly regarding sexual orientation, education level, and disability status presents an opportunity for improvement in primary care. Furthermore, collecting detailed demographic information can better allow practices to understand the need for targeted educational materials, track quality indicators, and address challenges faced by historically marginalized populations [ 26 , 27 ]. Still, even with good data collection approaches, some practices do not have the infrastructure or resources to analyze data to assess disparities in care or outcomes.

This study's strengths lie in its comprehensive analysis of a diverse range of primary care practices across Maryland. The inclusion of 507 practices with variations in size, location, and demographics enhances the representativeness of the findings and improves the generalizability of the results to a broader population. Consequently, the findings derived from studying a large population can contribute to a stronger evidence base for decision-making in healthcare and support the development of effective interventions and policies. A limitation of the study is the reliance on self-report, which may depend on the participants’ perspectives. Additionally, MDPCP practices meet eligibility criteria and voluntarily select to join the program, so these practices may be better equipped to join a value-based program that includes requirements or incentives to screen for social needs. Despite the limitations, our findings are novel in that few published studies highlight current practices at scale on social risk factor screening and referral in outpatient primary care settings for adults. Future research is warranted to show what strategies effectively increase uptake and drive meaningful change in social-needs responsive healthcare delivery.

MDPCP practices have demonstrated widespread adoption of social risk factor screenings and needs prioritization. While practices have implemented strategies to link patients to resources to address needs, challenges remain with providing social needs resources to beneficiaries from the primary care setting. Additionally, there is room for improvement in collecting certain demographic data fields within primary care practices. As the present analysis was based on cross-sectional data, future studies are needed to understand how to effect change in implementing or scaling social risk factor screening and detailed demographic data collection at the practice level. Additionally, future work is needed to understand how care is adjusted in response to identified social needs and how that impacts outcomes at the patient level.

Availability of data and materials

To access the datasets examined in this study, interested parties must follow the procedure outlined by the CMS. Requests should be submitted through the CMS website (cms.gov), and any queries can be directed to [email protected].

Abbreviations

Maryland Primary Care Program

Care Transformation Requirements

Center for Medicare and Medicaid Innovation

Centers for Medicare and Medicaid Services

Electronic Health Record

Social Determinants of Health

Social Determinants of Health. World Health Organization, 2023, at https://www.who.int/health-topics/social-determinants-of-health#tab=tab_1 . Accessed 9 Apr 2023.

DeVoe JE, Bazemore AW, Cottrell EK, et al. Perspectives in Primary Care: A Conceptual Framework and Path for Integrating Social Determinants of Health into Primary Care Practice. Ann Fam Med. 2016;14(2):104–8. https://doi.org/10.1370/afm.1917 .

Article   PubMed   PubMed Central   Google Scholar  

Lax Y, Bathory E, Braganza S. Pediatric primary care and subspecialist providers’ comfort, attitudes and practices screening and referring for social determinants of health. BMC Health Serv Res. 2021;21(1):956. https://doi.org/10.1186/s12913-021-06975-3 . PMID:34511119;PMCID:PMC8436516.

Andermann A; CLEAR Collaboration. Taking action on the social determinants of health in clinical practice: a framework for health professionals. CMAJ. 2016;188(17–18):E474–83. https://doi.org/10.1503/cmaj.160177 .

Article   Google Scholar  

Berkowitz SA, Seligman HK, Meigs JB, Basu S. Food insecurity, healthcare utilization, and high cost: a longitudinal cohort study. Am J Manag Care. 2018;24(9):399–404. https://pubmed.ncbi.nlm.nih.gov/33989068/ . PMID: 30222918; PMCID: PMC6426124.

PubMed   PubMed Central   Google Scholar  

McQueen A, Li L, Herrick CJ, Verdecias N, Brown DS, Broussard DJ, Smith RE, Kreuter M. Social Needs, Chronic Conditions, and Health Care Utilization among Medicaid Beneficiaries. Popul Health Manag. 2021;24(6):681–90. https://doi.org/10.1089/pop.2021.0065 . PMID: 33989068; PMCID: PMC8713253.

National Academies of Sciences, Engineering, and Medicine. Integrating Social Care into the Delivery of Health Care: Moving Upstream to Improve the Nation’s Health. Washington: The National Academies Press; 2019. https://doi.org/10.17226/25467 .

Book   Google Scholar  

Pfeiffer EJ, De Paula CL, Flores WO, Lavallee AJ. Barriers to Patients’ Acceptance of Social Care Interventions in Clinic Settings. Am J Prev Med. 2022;63(3 Suppl 2):S116–21. https://doi.org/10.1016/j.amepre.2022.03.035 . Epub 2022 Aug 17 PMID: 35987523.

Article   PubMed   Google Scholar  

Centers for Medicare and Medicaid Services, Centers for Medicare and Medicaid Innovation Sate Innovations Group. https://innovation.cms.gov/files/x/mdtcocm-rfa.pdf . Accessed 9 Apr 2023.

Sandhu S, Xu J, Eisenson H, Prvu Bettger J. Workforce Models to Screen for and Address Patients’ Unmet Social Needs in the Clinic Setting: A Scoping Review. J Prim Care Community Health. 2021;12:21501327211021020. https://doi.org/10.1177/21501327211021021 . PMID: 34053370; PMCID: PMC8772357.

Kreuter MW, Thompson T, McQueen A, Garg R. Addressing Social Needs in Health Care Settings: Evidence, Challenges, and Opportunities for Public Health. Annu Rev Public Health. 2021;42:329–44. https://doi.org/10.1146/annurev-publhealth-090419-102204 . Epub 2021 Dec 16. PMID: 33326298; PMCID: PMC8240195.

Measuring Social Determinants of Health among Medicaid Beneficiaries: Early State Lessons. Center for Health Care Strategies, Inc. 2016. https://www.chcs.org/media/CHCS-SDOH-Measures-Brief_120716_FINAL.pdf . Accessed 21 June 2023.

States Reporting Social Determinants of Health Related Policies Required in Medicaid Managed Care. Kaiser Family Foundation. 2022. https://www.kff.org/other/state-indicator/states-reporting-social-determinant-of-health-related-policies-required-in-medicaid-managed-care-contracts/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D . Accessed 21 June 2023.

Social Determinants of Health Measurement Work Group: Final Report. Oregon Health Authority. 2021. https://www.oregon.gov/oha/HPA/ANALYTICS/SDOH%20Page%20Documents/SDOH%20final%20report%202_10_21.pdf . Accessed 21 June 2023.

The Gravity Project: Accelerating National Social Determinants of Health (SDOH) Data Standards. Gravity Project. 2022. https://confluence.hl7.org/download/attachments/46892727/Gravity%20Overview%20One%20Pager%2020220209.pdf?version=1&modificationDate=1644529823670&api=v2 . Accessed 21 June 2023.

Quality ID #487: Screening for Social Drivers of Health. Centers for Medicare & Medicaid Services. 2022. https://qpp.cms.gov/docs/QPP_quality_measure_specifications/CQM-Measures/2023_Measure_487_MIPSCQM.pdf . Accessed 21 June 2023.

Identifying and Addressing Social Needs in Primary Care Settings. Agency for Healthcare Research and Quality. 2021. https://www.ahrq.gov/sites/default/files/wysiwyg/evidencenow/tools-and-materials/social-needs-tool.pdf . Accessed 21 June 2023.

The Health Leads Social Health Data Toolkit. Health Leads. 2021. https://healthleadsusa.org/wp-content/uploads/2021/02/Health-Leads-Social-Health-Data-Toolkit.pdf . Accessed 21 June 2023.

A Guide to Using the Accountable Health Communities Health-Related Social Needs Screening Took: Promising Practices and Key Insights. Centers for Medicare & Medicaid Services. 2023. https://www.cms.gov/priorities/innovation/media/document/ahcm-screeningtool-companion . Accessed 21 June 2023.

Beidler LB, Razon N, Lang H, Fraze TK. “More than just giving them a piece of paper”: Interviews with Primary Care on Social Needs Referrals to Community-Based Organizations. J Gen Intern Med. 2022;37(16):4160–7. https://doi.org/10.1007/s11606-022-07531-3 . Epub 2022 Apr 14. PMID: 35426010; PMCID: PMC9708990.

Child Care and Housing: Big Expenses with Too Little Help Available. Center on Budget and Policy Priortities. 2019. https://www.cbpp.org/research/housing/child-care-and-housing-big-expenses-with-too-little-help-available . Accessed 13 July 2023.

Key facts about housing affordability in the U.S. Pew Research Center. 2022. https://www.pewresearch.org/short-reads/2022/03/23/key-facts-about-housing-affordability-in-the-u-s/ . Accessed 13 July 2023.

Soumerai SB, Pierre-Jacques M, Zhang F, et al. Cost-related medication nonadherence among elderly and disabled medicare beneficiaries: a national survey 1 year before the medicare drug benefit. Arch Intern Med. 2006;166(17):1829–35. https://doi.org/10.1001/archinte.166.17.1829 . PMID: 17000938.

Naci H, Soumerai SB, Ross-Degnan D, Zhang F, Briesacher BA, Gurwitz JH, Madden JM. Medication affordability gains following Medicare Part D are eroding among elderly with multiple chronic conditions. Health Aff (Millwood). 2014;33(8):1435–43. https://doi.org/10.1377/hlthaff.2013.1067 . PMID:25092846;PMCID:PMC4340076.

Studies and Data Analytics on Broadband and Health. Federal Communications Commision. 2022. https://www.fcc.gov/health/sdoh/studies-and-data-analytics . Accessed 13 July 2023.

Yu J, Meng S. Impacts of the Internet on Health Inequality and Healthcare Access: A Cross-Country Study. Front Public Health. 2022;9(10):935608. https://doi.org/10.3389/fpubh.2022.935608 . PMID:35757602;PMCID:PMC9218541.

Grasso C, McDowell MJ, Goldhammer H, Keuroghlian AS. Planning and implementing sexual orientation and gender identity data collection in electronic health records. J Am Med Inform Assoc. 2019;26(1):66–70. https://doi.org/10.1093/jamia/ocy137 . PMID:30445621;PMCID:PMC6657380.

Download references

Acknowledgements

Not applicable

This study was supported by the Centers for Disease Control and Prevention’s National Initiative to Address COVID-19 Health Disparities Among Populations at High-Risk and Underserved, Including Racial and Ethnic Minority Populations and Rural Communities grant number OT21-2103 through the Maryland Department of Health.

Author information

Authors and affiliations.

Implementation Science, Healthcare Delivery Research Program, MedStar Health Research Institute, 6525 Belcrest Road, Suite 700, Hyattsville, MD, 20782, USA

Claire M. Starling, Marjanna Smith & Hannah Arem

Department of Emergency Medicine, Georgetown University School of Medicine, 3900 Reservoir Road, Washington, NWDC, 20007, USA

National Center for Human Factors in Healthcare, MedStar Health Research Institute, 3007 Tilden St.Suite 6N, Washington, NWDC, 20008, USA

Sadaf Kazi & Arianna Milicia

Maryland Primary Care Program, Maryland Department of Health, 201 W. Preston Street, Baltimore, MD, 21201, USA

Rachel Grisham & Emily Gruber

MedStar Center for Biostatistics, Informatics and Data Science, MedStar Health Research Institute, 3007 Tilden St.Suite 6N, Washington, NWDC, 20008, USA

Joseph Blumenthal

Department of Oncology, Georgetown University School of Medicine, 3900 Reservoir Road, Washington, NWDC, 20007, USA

Hannah Arem

You can also search for this author in PubMed   Google Scholar

Contributions

CMS and HA analyzed and interpreted the data and drafted the manuscript. MS, SK, AM, RG, EG, and JB contributed to revising the manuscript. All authors approved the version to be published and agreed to be accountable for the accuracy and integrity of the data.

Corresponding author

Correspondence to Claire M. Starling .

Ethics declarations

Ethics approval and consent to participate.

No consent was obtained to collect this data originally as it is mandated as part of CMS reporting. Using this data in the aggregate for publication was reviewed by the Georgetown University/Medstar Health IRB and deemed exempt (study 4698, modification approval date: December 15, 2022). CMS and the Maryland Department of Health approved the Georgetown/MedStar IRB decision.

Consent for publication

Not Applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Starling, C.M., Smith, M., Kazi, S. et al. Understanding social needs screening and demographic data collection in primary care practices serving Maryland Medicare patients. BMC Health Serv Res 24 , 448 (2024). https://doi.org/10.1186/s12913-024-10948-7

Download citation

Received : 22 January 2024

Accepted : 03 April 2024

Published : 10 April 2024

DOI : https://doi.org/10.1186/s12913-024-10948-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social needs screening
• Demographic data collection
• Primary care
• Community resources

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

CPS Online Library Research Guide (UNH Manchester Library): Find Peer-Reviewed Articles

• Home & Table of Contents
• Different Types of Information
• The Savvy Information Consumer
• Primary & Secondary Sources
• Periodical Literature
• Peer Review
• Research Glossary
• Business Research Databases
• Understanding the Assignment
• Preliminary Considerations
• 7 Steps to Completing a Research Assignment
• Define the Topic
• Find & Evaluate Your Sources
• Research Integrity & Citing Your Sources
• Searching for Information
• Evaluating Information
• How to Read an Academic Journal Article This link opens in a new window
• Evaluating Social Media Sources
• Writing Your Research Paper
• Create a Literature Review
• Summarize an Article
• How to Write an Abstract
• How to Write a Book Review
• How to Do an Annotated Bibliography
• Finding Professional Organizations
• Find Key Journals in Your Field of Study
• Find Peer-Reviewed Articles
• Write a Research Paper Proposal
• Research a Company This link opens in a new window
• Citing Your Sources
• Avoiding Plagiarism
• NHCUC Libraries
• Other Types of Research
• Academic Libraries in NH
• Government Documents
• Using Google for Academic Research
• Information Literacy
• Developing Effective Library Research Assignments
• Using Permalinks
• Primary Source Websites

Find Peer Reviewed Articles

You have been asked to find peer-reviewed (sometimes called scholarly) articles on a topic. Here is what you do:

• Go to the library homepage
• Begin a search in the search box using your keywords or subject terms
• Limit your search results to  Available Online and   Peer-reviewed Journals . Click Apply Filters .

• << Previous: Find Key Journals in Your Field of Study
• Next: Write a Research Paper Proposal >>
• Last Updated: Apr 9, 2024 2:30 PM
• URL: https://libraryguides.unh.edu/CPSonlineLibraryResearch

NIH Simplified Peer Review Framework for Research Project Grants (Webinar) - April 17

Posted April 10, 2024

Discipline(s): All

The National Institutes of Health (NIH) is simplifying the framework for the peer review of most Research Project Grant (RPG) applications effective for due dates on or aft er January 25, 2025. These changes are designed to address the complexity of the peer review process and mitigate potential bias. Make plans to hear the latest updates, timelines, and how these changes will impact existing and new funding opportunities.

A Q&A with NIH experts will follow the presentation to address additional questions.

Reasonable Accommodations:

This webinar will be closed-captioned and will include an American Sign Language (ASL) interpreter. Requests for reasonable accommodations should be submitted at least five days before the event to [email protected] .

For more information and to register, click here

Southern Illinois University Edwardsville is authorized to operate as a postsecondary educational institution by the  Illinois Board of Higher Education .

• Current Students
• Faculty & Staff
• Community & Business Partners