Building a consensus in grant review

Experience of a funding scheme helps to reduce disagreement among grant reviewers, says Marco Seeber

One of the most concerning issues in the evaluation of research grant proposals is the high level of disagreement between reviewers. Asking different experts to assess the same proposal often results in wildly differing scores. 

Studies of funders’ decisions at agencies in Austria, Australia and the United States have found that the consistency of reviewers’ judgments is only a little above random. Reviewers’ biases—such as a systematic tendency to give higher or lower scores—seem to have a greater influence on the final decisions than differences between proposals.