New Directions in Science Funding Policy

Normal 0 false false false DE X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin-top:0cm; mso-para-margin-right:0cm; mso-para-margin-bottom:8.0pt; mso-para-margin-left:0cm; line-height:107%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi; mso-ansi-language:DE; mso-fareast-language:EN-US;}

The time-honored system of evaluating grant proposal through assessments of scientific peers has become increasingly scrutinized. Troubling studies suggest that peer review may be unreliable at identifying merit, unintentionally discriminate against already marginalized scientists, disincentivize innovation and creativity, and be unduly expensive. As a result of these criticisms, the proper place of peer review in science funding policy is no longer clear. The purpose of this workshop is to identify what implications existing research on peer review in grant evaluations has for its future. Questions to be considered include:

(1)   Are the troubles with peer review systematic or can peer review be augmented or supplemented to sufficiently address the criticisms? If so, what augmentations or supplementations seem most promising? Lotteries? Diversifying review panels? Pre-screening programs? De-biasing resources for peer reviewers? Crowd-sourcing?

(2)   How might changes to peer review impact public perceptions of science funding decisions? Is peer review a “noble lie” that must be told to uphold public trust?

(3)   Are some of the criticisms more pressing than others? For instance, should promoting innovation in science take priority over the potential fairness of peer review?

(4)   How should the current criticisms of peer review be interpreted? For example, is inter-rater reliability a better metric than predictive validity for evaluating peer review?

(5)   Are there specific criteria (e.g., significance, novelty) that are more worrisome than others?

(6)   What future directions should empirical work on peer review take? What questions are in the direst need of answers?

(7)   What prospects are there for expanding the notion of “peer” to include stakeholders in grant evaluation?

(8)   Should AI be incorporated into grant peer review? If so, how and what are potential strengths or drawbacks in doing so?

(9)   Should peer review allow science to self-regulate, without significant political interference?

In answering questions such as these, we hope to provide a state-of-the-field assessment while reflecting upon next steps for research and practice. This will be done by an international and interdisciplinary group of scholars and will aim to incorporate representatives from funding agencies as part of the discussion.