Pains and Gains of Peer-Reviewing in Software Engineering

Abstract

Standard Reviewing Procedures The final acceptance decision for a paper can be taken by following different alternative peer-reviewing processes (these review processes are implemented for both conferences and journals). The standard setup can be described as follows: while authors are usually not aware of the reviewers' identity (so-called blind review setting), reviewers may be aware of the authors' identity or not, resulting in single-blind or double-blind review processes, respectively. In the rare cases when the authors are aware of the reviewers' identity, the reviewing process is considered zero-blind. Other feasible alternatives are characterized by the amount of stages implemented before the acceptance decision is made (single-stage vs. multi-stage review processes) and the public visibility of review comments (open/public vs. closed review processes). The actual process of finding agreement regarding the papers' acceptance or rejection has also a certain bandwidth, ranging from delegating the responsibility of the decision-making to few people over staged committee and board setups to organizing physical/virtual meetings involving many people.