Scholarly peer review

Scholarly peer review or academic peer review is the process of having a draft version of a researcher's methods and findings reviewed by experts in the same field. Peer review is widely used for helping the academic publisher decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph, or in the proceedings of an academic conference. If the identities of authors are not revealed to each other, the procedure is called dual-anonymous peer review.
Academic peer review requires a community of experts in a given academic field, who are qualified and able to perform reasonably impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish, and the significance of an idea may never be widely appreciated among its contemporaries. Peer review is generally considered necessary to academic quality and is used in most major scholarly journals. However, peer review does not prevent publication of invalid research, and as experimentally controlled studies of this process are difficult to arrange, direct evidence that peer review improves the quality of published papers is scarce. One recent analysis of randomized controlled trial abstracts found that editorial and peer review processes led to substantive improvements between submission and publication.

History

The first record of an editorial pre-publication peer-review is from 1665 by Henry Oldenburg, the founding editor of Philosophical Transactions of the Royal Society at the Royal Society of London.
The first peer-reviewed publication might have been the Medical Essays and Observations published by the Royal Society of Edinburgh in 1731. The present-day peer-review system evolved from this 18th-century process, began to involve external reviewers in the mid-19th-century, and did not become commonplace until the mid-20th-century.
Peer review became a touchstone of the scientific method, but until the end of the 19th century was often performed directly by an editor-in-chief or editorial committee. Editors of scientific journals at that time made publication decisions without seeking outside input, i.e. an external panel of reviewers, giving established authors latitude in their journalistic discretion. For example, Albert Einstein's four revolutionary Annus Mirabilis papers in the 1905 issue of Annalen der Physik were evaluated by the journal's editor-in-chief, Max Planck, and its co-editor, Wilhelm Wien, both future Nobel prize winners and together experts on the topics of these papers. On a much later occasion, Einstein was severely critical of the external review process, saying that he had not authorized John Torrence Tate Sr., the editor in chief of Physical Review, to show his manuscript "to specialists before it is printed", and informing him that he would "publish the paper elsewhere"which he did, with substantial modifications.
While some medical journals started to systematically appoint external reviewers, it is only since the middle of the 20th century that this practice has spread widely and that external reviewers have been given some visibility within academic journals, including being thanked by authors and editors. A 2003 editorial in Nature stated that, in the early 20th century, "the burden of proof was generally on the opponents rather than the proponents of new ideas." Nature itself instituted formal peer review only in 1967. Journals such as Science and the American Journal of Medicine increasingly relied on external reviewers in the 1950s and 1960s, in part to reduce the editorial workload. In the 20th century, peer review also became common for science funding allocations. This process appears to have developed independently from that of editorial peer review.
Pragmatically, peer review refers to the work done during the screening of submitted manuscripts. This process encourages authors to meet the accepted standards of their discipline and reduces the dissemination of irrelevant findings, unwarranted claims, unacceptable interpretations, and personal views. Publications that have not undergone peer review are likely to be regarded with suspicion by academic scholars and professionals. Non-peer-reviewed work does not contribute, or contributes less, to the academic credit of a scholar, although this heavily depends on the field.

Justification

It is difficult and time-consuming for authors and researchers, whether individually or in a team, to spot and provide feedback on every mistake or flaw in a complicated piece of work. This is not necessarily a reflection on those concerned, but because with a new and perhaps eclectic subject, an opportunity for improvement may be more obvious to someone with special expertise or who simply looks at it with a fresh eye. Therefore, showing work to others increases the probability that weaknesses will be identified and improved. For both grant-funding and publication in a scholarly journal, it is also normally a requirement that the subject is both novel and substantial.
The decision whether or not to publish a scholarly article, or what should be modified before publication, ultimately lies with the publisher to which the manuscript has been submitted. Similarly, the decision whether or not to fund a proposed project rests with an official of the funding agency. These individuals usually refer to the opinion of one or more reviewers in making their decision. This is primarily for three reasons:

Workload. A small group of editors/assessors cannot devote sufficient time to each of the many articles submitted to many journals.
Miscellany of ideas. Were the editor/assessor to judge all submitted material themselves, approved material would solely reflect their opinion.
Limited expertise. An editor/assessor cannot be expected to be sufficiently expert in all areas covered by a single journal or funding agency to adequately judge all submitted material.

Reviewers are often [|anonymous] and independent. However, some reviewers may choose to waive their anonymity, and in other limited circumstances, such as the examination of a formal complaint against the referee, or a court order, the reviewer's identity may have to be disclosed. Anonymity may be unilateral or reciprocal.
Since reviewers are normally selected from experts in the fields discussed in the article, the process of peer review helps to keep some invalid or unsubstantiated claims out of the body of published research and knowledge. Scholars will read published articles outside their limited area of detailed expertise, and then rely, to some degree, on the peer-review process to have provided reliable and credible research that they can build upon for subsequent or related research. Significant scandal ensues when an author is found to have falsified the research included in an article, as other scholars, and the field of study itself, may have relied upon the invalid research.
For US universities, peer reviewing of books before publication is a requirement for full membership of the Association of American University Presses.

Procedure

In the case of proposed publications, the publisher sends advance copies of an author's work or ideas to researchers or scholars who are experts in the field. Communication is nowadays normally by e-mail or through a web-based manuscript processing system such as ScholarOne, Scholastica, or Open Journal Systems. Depending on the field of study and on the specific journal, there are usually one to three referees for a given article. For example, Springer states that there are two or three reviewers per article.
The peer-review process involves three steps:

Step 1: Desk evaluation

An editor evaluates the manuscript to judge whether the paper will be passed on to journal referees. At this phase many articles receive a "desk reject", that is, the editor chooses not to pass along the article. The authors may or may not receive a letter of explanation.
Desk rejection is intended to be a streamlined process so that editors may move past nonviable manuscripts quickly and provide authors with the opportunity to pursue a more suitable journal. For example, the European Accounting Review editors subject each manuscript to three questions to decide whether a manuscript moves forward to referees: 1) Is the article a fit for the journal's aims and scope, 2) is the paper content does it follow format and technical specifications? If "no" to any of these, the manuscript receives a desk rejection.
Desk rejection rates vary by journal. For example, in 2017 researchers at the World Bank compiled rejection rates of several global economics journals; the desk rejection rate ranged from 21% to 66%. The American Psychological Association publishes rejection rates for several major publications in the field, and although they do not specify whether the rejection is pre- or post- desk evaluation, their figures in 2016 ranged from a low of 49% to a high of 90%.

Step 2: External review

If the paper is not desk rejected, the editors send the manuscript to the referees, who are chosen for their expertise and distance from the authors. At this point, referees may reject, accept without changes or instruct the authors to revise and resubmit.
Reasons vary for acceptance of an article by editors, but Elsevier published an article where three editors weigh in on factors that drive article acceptance. These factors include whether the manuscript: delivers "new insight into an important issue", will be useful to practitioners, advances or proposes a new theory, raises new questions, has appropriate methods and conclusion, presents a good argument based on the literature, and tells a good story. One editor notes that he likes papers that he "wished he'd done" himself.
These referees each return an evaluation of the work to the editor, noting weaknesses or problems along with suggestions for improvement. Typically, most of the referees' comments are eventually seen by the author, though a referee can also send 'for your eyes only' comments to the publisher; scientific journals observe this convention almost universally. The editor then evaluates the referees' comments, her or his own opinion of the manuscript before passing a decision back to the author, usually with the referees' comments.
Referees' evaluations usually include an explicit recommendation of what to do with the manuscript or proposal, often chosen from options provided by the journal or funding agency. For example, Nature recommends four courses of action:

to unconditionally accept the manuscript or the proposal,
to accept it in the event that its authors improve it in certain ways
to reject it, but encourage revision and invite re-submission
to reject it outright.

During this process, the role of the referees is advisory. The editor is typically under no obligation to accept the opinions of the referees. However, the editor will most often follow the advice of the referees. Furthermore, the referees in scientific publication do not act as a group, do not communicate with each other, and typically are not aware of each other's identities or evaluations. Proponents argue that if the reviewers of a paper are unknown to each other, the editor can more easily verify the objectivity of the reviews. There is usually no requirement that the referees achieve consensus, with the decision instead often made by the editor based on her best judgement of the arguments.
In situations where multiple referees disagree substantially about the quality of a work, there are a number of strategies for reaching a decision. The paper may be rejected outright, or the editor may choose which reviewer's point the authors should address. When a publisher receives very positive and very negative reviews for the same manuscript, the editor will often solicit one or more additional reviews as a tie-breaker. As another strategy in the case of ties, the publisher may invite authors to reply to a referee's criticisms and permit a compelling rebuttal to break the tie. If a publisher does not feel confident to weigh the persuasiveness of a rebuttal, the publisher may solicit a response from the referee who made the original criticism. An editor may convey communications back and forth between authors and a referee, in effect allowing them to debate a point.
Even in these cases, however, publishers do not allow multiple referees to confer with each other, though each reviewer may often see earlier comments submitted by other reviewers. The goal of the process is explicitly not to reach consensus or to persuade anyone to change their opinions, but instead to provide material for an informed editorial decision. One early study regarding referee disagreement found that agreement was greater than chance, if not much greater than chance, on six of seven article attributes, but this study was small and it was conducted on only one journal. At least one study has found that reviewer disagreement is not common, but this study is also small and on only one journal.
Traditionally, reviewers would often remain anonymous to the authors, but this standard varies both with time and with academic field. In some academic fields, most journals offer the reviewer the option of remaining anonymous or not, or a referee may opt to sign a review, thereby relinquishing anonymity. Published papers sometimes contain, in the acknowledgments section, thanks to anonymous or named referees who helped improve the paper. For example, Nature journals provide this option.
Sometimes authors may exclude certain reviewers: one study conducted on the Journal of Investigative Dermatology found that excluding reviewers doubled the chances of article acceptance. Some scholars are uncomfortable with this idea, arguing that it distorts the scientific process. Others argue that it protects against referees who are biased in some manner. In some cases, authors can choose referees for their manuscripts. mSphere, an open-access journal in microbial science, has moved to this model. Editor-in-Chief Mike Imperiale says this process is designed to reduce the time it takes to review papers and permit the authors to choose the most appropriate reviewers. But a scandal in 2015 shows how this choosing reviewers can encourage fraudulent reviews. Fake reviews were submitted to the Journal of the Renin-Angiotensin-Aldosterone System in the names of author-recommended reviewers, causing the journal to eliminate this option.