Image: Hash Milhan [CC BY 2.0] via Wikimedia Commons
Survey of 2021 exercise will help shape future assessments, say Catriona Manville and James Wilsdon
Last month’s call by Nancy Rothwell, vice-chancellor of the University of Manchester and incoming chair of the Russell Group of universities, for more sustainable and agile research funding, was a shot across the bow, signalling a renewed push by research-intensive universities for a funding and assessment system with a lighter touch.
With the government and UK Research and Innovation also signed up to a fresh blitz on bureaucracy, as set out in a recent policy paper from the business and education departments, it is only a matter of time before attention turns again to the design and effectiveness of the Research Excellence Framework as a multi-purpose tool for allocating quality-related funding.
The deadline for submissions to REF2021 is 31 March next year—only six months away—having been extended to ease pressure on universities through the pandemic. Wisely, ministers have chosen to leave the REF alone for now. But once the 2021 cycle is complete, it seems reasonable to expect another root-and-branch review of its purposes, methods, burdens and benefits.
Work in progress
A RAND Europe study last year for Research England, which conducts the REF in partnership with its equivalents in the devolved nations, is a helpful guide to the options likely to be on the table, as assessment processes evolve to reflect wider changes in the research landscape.
In particular, the study highlighted academics’ growing desire to produce a diversity of outputs from their research. This is balanced by an appreciation that journal articles will remain prominent, owing in part to their importance in the reward system for both individuals and institutions.
The study also focused on how technology might support future assessment—in, for example, identifying panel members, supporting eligibility checks and moderating reviews. However, the study concluded, that in the first instance at least, there may be a trade-off between reducing the burden of assessment and increasing the value of the information it produces. That creates a need for the sector to debate its current priorities and aims.
The REF evokes strong reactions, from both its critics and defenders. Formal consultations are often dominated by institutional voices, or by those who shout loudest. The last review of the REF, led by Nicholas Stern in 2016, prompted several changes designed to streamline the process.
Before embarking on a further round of changes, it is important to listen and learn from researchers and managers about what has and has not worked well. We need to understand, with more rigour and detail, how researchers across different disciplines, career stages, institutions and geographies have experienced the 2021 assessment cycle. Where and how has it been implemented well, and what more could be improved?
Real-time review
In 2018, Research England initiated a pilot study across four universities aimed at tapping into this more diverse set of perspectives. This published its findings last year, and recommended that efforts at bottom-up, distributed evaluation—what’s known as the real-time REF—should continue as the 2021 cycle draws to a close.
For this reason, a new and expanded phase of this work is being launched this week. Led by RAND Europe, with the support of Research England and academic partners at Cardiff and Sheffield universities, the Real-Time REF review will evaluate UK researchers’ perceptions of the assessment process while they are preparing and submitting their work to REF2021.
To ensure that the study reflects what’s happening on the ground, we hope to get input from across all disciplines and career stages. Any researcher who is eligible to submit to the REF can participate in the online survey launched today. So please log on and fire away—we are keen to hear what you think.
A cohort of 25 universities, from Aberdeen to Exeter, has also agreed to participate in a more structured way, ensuring that all regions of the UK are represented. The survey will be followed by a series of online focus groups to dig more deeply into prominent perceptions and emerging themes.
Many organisations and individuals will have insights and views as to how the REF should be designed to meet the needs of the UK research system. This study aims to ensure that these voices are heard and can inform a framework that is fit for the future.
Catriona Manville is a research leader at RAND Europe. James Wilsdon is professor of research policy at the University of Sheffield and director of the Research on Research Institute
A version of this article also appeared in Research Fortnight