How can we improve peer review?

Toby Handfield (Monash University)

Peer review is very important and yet a highly imperfect part of the academic publication process. Institutions rely on peer review as a guarantee of quality, but evidence for the consistency of peer review is patchy at best. Given that peer review is important and could be better, why don't journals do more to make it more reliable? What factors influence a journal's decision to put more or less effort into ensuring high quality peer review? And what will the implications of particular journal policies be, given that authors may respond strategically by changing where they submit their work?

In this talk I will present work in progress that addresses these questions, using both a formal model and empirical data. We develop a model of journal and author behavior, in which journals decide how selective they will be, and how much effort to put into peer review. Authors decide whether to submit to the journal or not, based on the quality of their paper, and on the journal's choices. I also present some survey data of journal editors, to validate the assumptions of our model. The emerging picture is that there are deep structural barriers to improving peer review, and that the effect of these barriers will be exacerbated if journals care not just about quality, but also about being perceived to be highly selective.