Double-blind is good but open would be better

Abstract

Peer review in software engineering is considered, same as for other disciplines, to be a key element of the research process, yet it is often perceived as not to work fully well. To understand the pains and gains in the peer review system, we ran a survey with open and closed questions with the authors and PC members of ICSE 2014/2015/2016. We received 241 responses (29% response rate). 67% of the respondents identified themselves as professors. We analyzed the responses quantitatively and qualitatively (with open coding). All questions were optional. Agreement scales had 10 points, so mild levels of agreement could be expressed but there was no undecided middle point. The resulting article appeared in Information and Software Technology in 2018 [1] and we also disclosed the anonymized data set [2].