Lowering the bar for entry will mean more applications, and all the savings will go to lower-quality proposals. The Commission will need to be flexible, says Peter Fisch.
Low success rates have perhaps been the biggest bugbear for researchers applying to Horizon 2020. Figures published in November 2016 showed that only 10.7 per cent of submitted proposals were funded in 2015, which is an all-time low.
Demand for EU research funding in Europe and beyond obviously exceeds supply. The legal, economic and political environment provides little scope for quick improvement, but the European Commission has introduced some measures to ease the pain.
The most notable of these is greater use of a two-stage evaluation process, in the hope of reducing the overall burden of work put into unsuccessful applications. This allows research teams to submit a brief initial application, with a full proposal required only if the first stage is successful.
However, changing the application process will affect the behaviour of applicants and evaluators. And contrary to widespread belief, the overall effect of such changes is not guaranteed to be positive.
Standard economic theory, for example, predicts that lowering the price of entry by requiring a short abstract instead of a fully developed proposal will cause an increase in the number of applications. Even if the burden on each first-round applicant is lower, a massive increase in the number of applicants could substantially reduce the initial benefits, or even increase the total time spent on applications.
To make things more complicated, different fields are likely to respond differently. In some areas such as aeronautics, a large proportion of the research community is already involved in Horizon 2020. Here, there is little scope for additional applications.
But in the social sciences and humanities, only a tiny fraction of researchers are actively involved in Horizon 2020. Here, a massive increase in applications can be expected, bringing already low success rates down to unacceptable levels.
Judging the worth of short, abstract-type proposals is a demanding and difficult task. Evaluators might be inclined to be ‘nice’ and give a substantial number of submissions the benefit of the doubt.
More second-round proposals, though, would also increase the overall burden on applicants, so the unpleasant conclusion is that to be efficient, evaluators should act in a very stringent and disciplined manner that may go against their natural instincts.
There is also an inherent risk at the individual level. Faced with a crowded and highly competitive first round, applicants will be tempted to improve their chances by beefing up their proposal, making it longer and more comprehensive—and more costly—than initially intended. Those that make it to the second round, anxious to seize a substantially higher chance of success, will face a similar temptation to invest more in the proposal so as not to miss a unique opportunity.
Faced with these complications, it is important that the implementation of two-stage calls is monitored carefully from the outset and if necessary adjusted.
The Commission should go for flexible implementation, taking into account the specificities in each programme area, to try and avoid the risk of a massive increase in applications. It should also discourage evaluators from being too nice.
Meanwhile, researchers should reflect on the appropriate level of effort for Horizon 2020 applications. They should resist the current received wisdom, fuelled by external consultants and advisers, that more is always better. Instead, it might help to develop application standards for both stages of the process, with the aim of introducing some discipline into the preparation of proposals and setting out the dos and don’ts.
Finally, it is amazing that two glaring issues related to two-stage calls have been absent from the public debate.
The first is that two rounds of applications and evaluations will take several months longer than one. This is not necessarily a bad thing, but the general tacit acceptance of greater delays contrasts starkly with the atmosphere of three or four years ago, when reducing time to grant seemed to be the number-one political objective.
The other issue relates to the simple question of who will benefit from a two-stage process. Strong applicants will not be better off and might even lose out, as they will need two rounds of evaluation and substantially more time to get their ideas funded. All the savings in effort will go to the applicants eliminated in the first round.
This is not a good outcome for a programme that aims to strengthen scientific excellence. Surprisingly, none of the otherwise vocal lobby groups that normally defend such excellence at every occasion have raised this issue so far.
Something to add? Email comment@ResearchResearch.com
Peter Fisch, former head of unit in the European Commission’s directorate-general for research and innovation, blogs on research policy at www.peter-fisch.eu
This article also appeared in Research Europe