We recently stated here that peer review is a source of great strength for APS journals, but did not discuss why. What are the elements of a helpful report? Who benefits from such a report, and how? This is discussed in this Editorial, directed to referees. Please do not read on if you are an author (or imagine that you are not an author as you do).
We editors hope for reports that provide sufficient information to allow us to reach a decision about publication. To accomplish this a review should provide a clear indication that a referee has studied the manuscript and grasped its main message, and also provide substantive support for any conclusion, favorable or not. Physical Review Letters maintains high standards, so a lack of indication not to publish is not a basis on which we will accept a Letter. If the recommendation is to publish, in PRL or in another journal, additional information is useful to improve the manuscript. A more detailed analysis of the manuscript can show where a paper is unclear, incomplete, or otherwise flawed.
For a recommendation not to publish, we seek a report that provides enough information to make an unfavorable decision understandable to the authors. Editors have a large stake in bringing the process to a close, to avoid excessive effort in multiple rounds of review. Our goal is to publish good papers, and turn away poor ones, efficiently.
We are of course extremely grateful for the efforts of our referees, and are always gratified to receive the many careful evaluations that you send us. Many thanks! The need for efficiency, however, may lead us to apply a relative scale to our expectations for a report, one tailored to our view of the manuscript. For an unpromising manuscript we might find a not very detailed unfavorable report to be sufficient, although we always want a substantive support for the conclusion. Similarly, for a paper that looks good to us an abbreviated favorable report might serve. Favorable reviews are often the least detailed, however, and we do need concrete reasons to publish, as well as to reject.
Authors have different priorities, which lead to a different view of a helpful report. They first hope for a favorable review. If the report suggests publication, then authors usually are glad to receive additional comments that improve the text. The review process benefits many Letters, and authors often prefer the revised version, when it is published.
If the conclusion of the report is not to publish, the situation is trickier. Compared to editors, authors interpret reviews as more favorable. For example, a review saying “this work might be important, if it were correct and readable” would be read by an editor as quite unfavorable, while an author might focus on the idea that “the referee said our work is important.” Reports that are more negative can reduce this, but authors may conclude that a strongly worded report is indicative of bias or other unprofessional behavior by the reviewer, and object. The challenge for a helpful report is to find the middle ground.
Further, for unfavorable reviews authors may expect proof that the paper is not publishable. Proof of a negative of this sort in general is impossible, in particular when the party seeking the proof has a strong interest in the outcome. Related to this, in contrast to editors, authors tend to assess unfavorable reviews on an absolute scale, and conclude that a short negative report is insufficient, by definition. (This is not the case for favorable reviews, which authors find to be more than enough, no matter how thin they might be. We have yet to see an author write, “Ignore Referee B, who suggests that the manuscript be published, but for reasons that are all wrong.”)
In conclusion, with any rejection authors want proof that their paper is not publishable. Editors, on the other hand, seek clear signals to publish, and are not inclined to accept a Letter only because there are no reasons to reject. These issues make it difficult for authors to understand and accept both a negative referee report and negative decision by an editor. Often a solution that is satisfactory to all is elusive, and the best that referees and editors can hope to do is to minimize the authors’ unhappiness.
